In this work, we train multilingual autoregressive language models on a balanced corpus covering a diverse set of languages, 
and study their few- and zero-shot learning capabilities in a wide range of tasks.