File size: 198 Bytes
5fa1a76
 
 
1
2
3
PipelineParallel (PP)
Parallelism technique in which the model is split up vertically (layer-level) across multiple GPUs, so that only one or 
several layers of the model are placed on a single GPU.