chunked/nltk_chunking/model_doc_mega/chunk_1.txt

MEGA proposes a new approach to self-attention with each encoder layer having a multi-headed exponential moving average in addition to a single head of standard dot-product attention, giving the attention mechanism 
stronger positional biases.