That repository contains a multi-GPU and multi-node implementation of the Megatron Language models.