Cite
Notes
Only stored in your browser.
Attribution
TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training
arXiv 2024
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
arXiv 2023
Ranger21: a synergistic deep learning optimizer
arXiv 2021
from 3 papers
Andrew Gu
Chien-chin Huang
Ajit Mathews
Alban Desmaison
Bernard Nguyen
Can Balioglu
Geeta Chauhan
Gokul Nadathur
Hamid Shojanazeri
Howard Huang