DataHelix: CPU-GPU Co-Scheduling for Multimodal Model Training
DataHelix is a metadata-first CPU-GPU co-scheduling data plane designed to reduce pipeline bubbles in multimodal model training. It coordinates preprocessing across data-parallel ranks by treating GPU compute windows as readiness budgets and deciding sample placement while data remains lightweight descriptors. The system combines descriptor-level token-latent-aware batching with cost-gated collective rebalancing to correct residual skew only when transfer cost is justified. Across 11 video and multimodal workloads on 8 Ascend 910B accelerators, it reports a 67.8% lower mean deadline-miss rate, 87.0% lower mean p95 input-wait latency, and 2.72x geometric-mean throughput over rank-local DataLoaders.
It offers ML systems engineers a concrete scheduling approach for reducing data-pipeline stalls in heterogeneous multimodal training.
Comments & Questions (0)
No comments yet
Be the first to comment!