Cost-Aware Agentic System Optimizes Features Across Fleet-Scale Ranking Models
Researchers describe an end-to-end agentic system that optimizes feature selection for large fleets of ranking models drawing from a shared pool of tens of thousands of features, under both fleet-wide and per-model resource constraints such as pipeline cost and serving QPS. LLM agents propose hypotheses while deterministic tools execute and verify experiments; for each model an LLM generates and shortlists cross features, importance methods rank candidates, and feature blending merges rankings via Pareto fronts for multi-task models. An LLM Researcher agent then tunes selection across 10-25 asynchronous trials within resource caps, followed by a trimming step to hold pipeline cost to budget. The authors report an average 0.31% gain in normalized entropy, a 2.3% pipeline cost reduction from trimming with no regression beyond noise, and a 0.20% improvement in a primary online metric in A/B tests on launched models.
It offers a concrete, production-validated blueprint for combining LLM agents with deterministic tooling to optimize ML features under cost and throughput constraints.
Comments & Questions (0)
No comments yet
Be the first to comment!