chunked/nltk_chunking/model_doc_roberta/chunk_2.txt

It builds on BERT and modifies key hyperparameters, removing the next-sentence pretraining objective and training with
much larger mini-batches and learning rates.