fairscale

FairScale: A PyTorch library for large-scale and high-performance training.