Principles and Practice of Parallel Programming

• PerFlow: A Domain Specific Framework for Automatic Performance Analysis of Parallel Applications
• FasterMoE: Modeling and Optimizing Training of Large-Scale Dynamic Pre-Trained Models
• Scaling Graph Traversal to 281 Trillion Edges with 40 Million Cores
• Vapro: Performance Variance Detection and Diagnosis for Production-Run Parallel Applications
• BAGUALU: Targeting Brain Scale Pretrained Models with over 37 Million Cores