Utilize chunking with MegaConfig.use_chunking and control chunk size with MegaConfig.chunk_size 

Implementation Notes

The original implementation of MEGA had an inconsistent expectation of attention masks for padding and causal self-attention between the softmax attention and Laplace/squared ReLU method.