Live intelligence

Latest AI updates

2 updates
Research

Inference-Environment Coordination Cuts LLM Agent Batch Makespan by Up to 65%

The paper targets batch makespan for LLM agent tasks that alternate between inference and environment interaction, a pattern common in benchmark evaluation, synthetic-data generation and RL rollouts. It introduces a workload-based resource model for GPU memory allocation between KV cache and recurrent state in hybrid LLMs, plus trajectory-aware routing and scheduling for cache reuse and load balancing. Under constrained environment capacity, it decouples sample admission from environment execution and adds Chain-Advancing Action Scheduling, which prioritizes long chains to shorten the batch tail. Live evaluations with Ling-3.0-Flash and Ling-3.1-Flash on BFCL, SWE-bench Multilingual, WideSearch and MCP-Atlas report makespan reductions of 10.4% to 65.0% with aligned evaluation scores.

LLM agent workloadsinference systemsscheduling
Research

Berkeley AI Research Survey Maps Adaptive Parallel Reasoning for Efficient LLM Inference

A Berkeley AI Research blog post surveys adaptive parallel reasoning (APR), a paradigm where reasoning models decide when to decompose tasks, spawn concurrent threads, and coordinate them. It contrasts APR with fixed methods like self-consistency and tree search, and details inference system designs including Multiverse's KV-cache stitching and ThreadWeaver's engine-agnostic client orchestration. The post covers training approaches and highlights trade-offs in memory management and throughput.

adaptive parallel reasoningLLM inference efficiencyinference systems