chunked/nltk_chunking/model_doc_funnel/chunk_6.txt

In addition, to perform token-level predictions as required by common pretraining
objectives, Funnel-Transformer is able to recover a deep representation for each token from the reduced hidden sequence
via a decoder.