Research
ServerBench: Benchmarking Coding Agents for Designing Specialized LLM Serving Systems
ServerBench is a new benchmark that evaluates coding agents' ability to design complete, specialized LLM serving systems. Agents are scored on a dollars-per-dollar metric, balancing latency-qualified token value against GPU cost. In tests across 11 coding-agent models, Claude Sonnet 5.5 achieved 2.97 dollars per dollar on Gemma E4B, outperforming the vLLM reference by 4.87 times. The benchmark includes stateful replay and independent verification to ensure correctness.
LLM serving systemsagentic software designbenchmarking