Research
BubbleTea Reuses MoE Expert-Parallel Idle Time for Fine-Tuning
The paper introduces BubbleTea, a colocation system that fills idle GPU bubbles during Mixture-of-Experts LLM inference with short chunks of fine-tuning work. It predicts each GPU's per-layer bubble from the router's token assignment and uses compiler passes to derive fine-tuning chunks from the served model without porting architectures to a separate training engine. Evaluated on three MoE models across four A100 GPUs, BubbleTea reports 1.9–6.2× faster fine-tuning throughput than LLMStation while keeping p90 inference latency within 10% of serving alone on 50 of 63 workloads; on bursty traces it trains 3.3× faster than LLMStation and 14× faster than a stock trainer colocated under CUDA MPS.
Mixture-of-ExpertsGPU colocationLLM fine-tuning