Live intelligence

Latest AI updates

1 updates
Research

Inference-Environment Coordination Cuts LLM Agent Batch Makespan by Up to 65%

The paper targets batch makespan for LLM agent tasks that alternate between inference and environment interaction, a pattern common in benchmark evaluation, synthetic-data generation and RL rollouts. It introduces a workload-based resource model for GPU memory allocation between KV cache and recurrent state in hybrid LLMs, plus trajectory-aware routing and scheduling for cache reuse and load balancing. Under constrained environment capacity, it decouples sample admission from environment execution and adds Chain-Advancing Action Scheduling, which prioritizes long chains to shorten the batch tail. Live evaluations with Ling-3.0-Flash and Ling-3.1-Flash on BFCL, SWE-bench Multilingual, WideSearch and MCP-Atlas report makespan reductions of 10.4% to 65.0% with aligned evaluation scores.

LLM agent workloadsinference systemsscheduling