Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
translate Arabic to English
tokenizer.src_lang = "ar_AR"
encoded_ar = tokenizer(article_ar, return_tensors="pt")
generated_tokens = model.generate(**encoded_ar, forced_bos_token_id=tokenizer.lang_code_to_id["en_XX"])
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
=> "The Secretary-General of the United Nations says there is no military solution in Syria."