Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
Remove any columns you don't need:
tokenized_squad = squad.map(preprocess_function, batched=True, remove_columns=squad["train"].column_names)
Now create a batch of examples using [DefaultDataCollator].