| Training | |
| [LongformerForMaskedLM] is trained the exact same way [RobertaForMaskedLM] is | |
| trained and should be used as follows: | |
| thon | |
| input_ids = tokenizer.encode("This is a sentence from [MASK] training data", return_tensors="pt") | |
| mlm_labels = tokenizer.encode("This is a sentence from the training data", return_tensors="pt") | |
| loss = model(input_ids, labels=input_ids, masked_lm_labels=mlm_labels)[0] | |
| Resources | |
| Text classification task guide | |
| Token classification task guide | |
| Question answering task guide | |
| Masked language modeling task guide | |
| Multiple choice task guide | |
| LongformerConfig | |
| [[autodoc]] LongformerConfig | |
| LongformerTokenizer | |
| [[autodoc]] LongformerTokenizer | |
| LongformerTokenizerFast | |
| [[autodoc]] LongformerTokenizerFast | |
| Longformer specific outputs | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerBaseModelOutput | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerBaseModelOutputWithPooling | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerMaskedLMOutput | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerQuestionAnsweringModelOutput | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerSequenceClassifierOutput | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerMultipleChoiceModelOutput | |
| [[autodoc]] models.longformer.modeling_longformer.LongformerTokenClassifierOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerBaseModelOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerBaseModelOutputWithPooling | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerMaskedLMOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerQuestionAnsweringModelOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerSequenceClassifierOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerMultipleChoiceModelOutput | |
| [[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerTokenClassifierOutput | |
| LongformerModel | |
| [[autodoc]] LongformerModel | |
| - forward | |
| LongformerForMaskedLM | |
| [[autodoc]] LongformerForMaskedLM | |
| - forward | |
| LongformerForSequenceClassification | |
| [[autodoc]] LongformerForSequenceClassification | |
| - forward | |
| LongformerForMultipleChoice | |
| [[autodoc]] LongformerForMultipleChoice | |
| - forward | |
| LongformerForTokenClassification | |
| [[autodoc]] LongformerForTokenClassification | |
| - forward | |
| LongformerForQuestionAnswering | |
| [[autodoc]] LongformerForQuestionAnswering | |
| - forward | |
| TFLongformerModel | |
| [[autodoc]] TFLongformerModel | |
| - call | |
| TFLongformerForMaskedLM | |
| [[autodoc]] TFLongformerForMaskedLM | |
| - call | |
| TFLongformerForQuestionAnswering | |
| [[autodoc]] TFLongformerForQuestionAnswering | |
| - call | |
| TFLongformerForSequenceClassification | |
| [[autodoc]] TFLongformerForSequenceClassification | |
| - call | |
| TFLongformerForTokenClassification | |
| [[autodoc]] TFLongformerForTokenClassification | |
| - call | |
| TFLongformerForMultipleChoice | |
| [[autodoc]] TFLongformerForMultipleChoice | |
| - call |