Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
PipelineParallel (PP)
Parallelism technique in which the model is split up vertically (layer-level) across multiple GPUs, so that only one or
several layers of the model are placed on a single GPU.