File size: 2,932 Bytes
5fa1a76 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
Training
[LongformerForMaskedLM] is trained the exact same way [RobertaForMaskedLM] is
trained and should be used as follows:
thon
input_ids = tokenizer.encode("This is a sentence from [MASK] training data", return_tensors="pt")
mlm_labels = tokenizer.encode("This is a sentence from the training data", return_tensors="pt")
loss = model(input_ids, labels=input_ids, masked_lm_labels=mlm_labels)[0]
Resources
Text classification task guide
Token classification task guide
Question answering task guide
Masked language modeling task guide
Multiple choice task guide
LongformerConfig
[[autodoc]] LongformerConfig
LongformerTokenizer
[[autodoc]] LongformerTokenizer
LongformerTokenizerFast
[[autodoc]] LongformerTokenizerFast
Longformer specific outputs
[[autodoc]] models.longformer.modeling_longformer.LongformerBaseModelOutput
[[autodoc]] models.longformer.modeling_longformer.LongformerBaseModelOutputWithPooling
[[autodoc]] models.longformer.modeling_longformer.LongformerMaskedLMOutput
[[autodoc]] models.longformer.modeling_longformer.LongformerQuestionAnsweringModelOutput
[[autodoc]] models.longformer.modeling_longformer.LongformerSequenceClassifierOutput
[[autodoc]] models.longformer.modeling_longformer.LongformerMultipleChoiceModelOutput
[[autodoc]] models.longformer.modeling_longformer.LongformerTokenClassifierOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerBaseModelOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerBaseModelOutputWithPooling
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerMaskedLMOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerQuestionAnsweringModelOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerSequenceClassifierOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerMultipleChoiceModelOutput
[[autodoc]] models.longformer.modeling_tf_longformer.TFLongformerTokenClassifierOutput
LongformerModel
[[autodoc]] LongformerModel
- forward
LongformerForMaskedLM
[[autodoc]] LongformerForMaskedLM
- forward
LongformerForSequenceClassification
[[autodoc]] LongformerForSequenceClassification
- forward
LongformerForMultipleChoice
[[autodoc]] LongformerForMultipleChoice
- forward
LongformerForTokenClassification
[[autodoc]] LongformerForTokenClassification
- forward
LongformerForQuestionAnswering
[[autodoc]] LongformerForQuestionAnswering
- forward
TFLongformerModel
[[autodoc]] TFLongformerModel
- call
TFLongformerForMaskedLM
[[autodoc]] TFLongformerForMaskedLM
- call
TFLongformerForQuestionAnswering
[[autodoc]] TFLongformerForQuestionAnswering
- call
TFLongformerForSequenceClassification
[[autodoc]] TFLongformerForSequenceClassification
- call
TFLongformerForTokenClassification
[[autodoc]] TFLongformerForTokenClassification
- call
TFLongformerForMultipleChoice
[[autodoc]] TFLongformerForMultipleChoice
- call |