chunked/nltk_chunking/model_doc_realm/chunk_5.txt

For the
first time, we show how to pre-train such a knowledge retriever in an unsupervised manner, using masked language
modeling as the learning signal and backpropagating through a retrieval step that considers millions of documents.