chunked/nltk_chunking/model_doc_convbert/chunk_5.txt

The novel convolution heads, together with the
rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context
learning.