TrainingActive
TWIST
High-Efficiency LLM Training System via Strand Interleaving on NVIDIA Hopper GPUs.
llmtrainingdistributed-systemshopper
MLSys Lab
Advancing Scalable and Efficient Systems
Distributed LLM training systems, tensor core optimization, and technical notes.
Featured
Latest from the lab.
Blog
TWIST
Technical note with overview, layer breakdown, and MFU comparison figures.
A technical note for DHeLlam, the ICCD 2025 Best Paper, summarizing the currently available overview, layer breakdown, and H200 MFU comparison figures.
TWISTtechnical notesystemsDHeLlam
Mantle: Efficient Hierarchical Metadata Management for Cloud Object Storage Services
Mantle proposes a two-layer metadata architecture for cloud object storage, enabling scalable hierarchical namespace management.
cloud storagemetadata managementdistributed file systemsobject storage
Research Areas
What we work on.
parallel computing / distributed systems / AI systems
Parallel and Distributed Intelligent Computing
Automatic micro-batch co-execution, tensor core scheduling, and efficient distributed LLM training.
storage / data systems / infrastructure
Storage Systems
Cloud-edge query processing, data-centric system design.
data-centric / systems / large models
Data-Centric System Design
Data-centric perspective across large model system design problems.
Blog
Technical writeups.
Blog
TWIST
Technical note with overview, layer breakdown, and MFU comparison figures.
A technical note for DHeLlam, the ICCD 2025 Best Paper, summarizing the currently available overview, layer breakdown, and H200 MFU comparison figures.
TWISTtechnical notesystemsDHeLlam
Projects
Research projects.
TrainingActive
TWIST
High-Efficiency LLM Training System via Strand Interleaving on NVIDIA Hopper GPUs.
llmtrainingdistributed-systemshopper
SystemsActive
Adacluster
Adaptive clustering system for large-scale ML workloads with dynamic resource scheduling.
ml-systemsschedulingresource-management
Publications
Research publications and award-winning publications from the lab.
Peer-reviewed work on distributed training, tensor core optimization, and efficient AI systems published at top-tier venues.
View all publicationsMantle: Efficient Hierarchical Metadata Management for Cloud Object Storage Services
Mantle proposes a two-layer metadata architecture for cloud object storage, enabling scalable hierarchical namespace management.
cloud storagemetadata managementdistributed file systemsobject storage
AutoCCL: Automated Collective Communication Tuning for Accelerating Distributed and Parallel DNN Training
AutoCCL automates collective communication tuning to accelerate distributed and parallel DNN training.
collective communicationDNN trainingdistributed systemsauto-tuning
GLPilot: Efficient Distributed GNN Training With Learnable Embeddings
GLPilot introduces a staleness-bounded embedding buffering mechanism to reduce remote fetches in distributed GNN training with learnable vertex embeddings.
GNNdistributed traininggraph learningembeddings