Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
Flatten these two lists so you can tokenize them, and then unflatten them afterward so each example has a corresponding input_ids, attention_mask, and labels field.