LM-SimBench: A Unified Framework, Dataset, and Benchmark for Large Language Model Training Simulation
LM-SimBench is a unified framework, profiling dataset, and benchmark for evaluating LLM training simulators. It decomposes simulation into functional stages from model compilation to schedule-level aggregation and connects execution representations and cost estimators to measurable evaluation targets. The dataset profiles 23 model settings from 10 dense and Mixture-of-Experts LLM families on an Ascend NPU cluster, yielding 348 feasible configurations across five parallelism dimensions and activation recomputation. Evaluations show the benchmark can support simulator accuracy comparisons, configuration-selection assessment, and prediction-error diagnosis, with matched GPU-NPU experiments validating cross-platform applicability.
It gives researchers and infrastructure engineers a structured way to compare LLM training simulators and parallelism configurations without expensive distributed training runs.
Comments & Questions (0)
No comments yet
Be the first to comment!