Translate
Live intelligence

Latest AI updates

2 updates
AllModelsProductsIndustryResearchTutorialOpinion
Research

Task-Efficient Agent Evaluation Using Cached Traces

This paper introduces a method to estimate an agentic system's benchmark performance from a small number of task executions by leveraging rich execution traces. The approach extracts structured summaries with an LLM and embeds them to create representations that preserve task-related signal while suppressing identity-related noise. On SWE-bench Verified and Terminal-Bench 2.0, combining this geometry-based prediction with an item-response model reduces mean absolute error significantly compared to baselines.

arxiv13h agoScore 75
AI evaluationagentic systemsefficient benchmarking
Research Editor selected

DiabAgent: Agentic Physiological Control for Type 1 Diabetes

DiabAgent introduces a research suite for agentic physiological control, including the APC framework, DiabetSim simulator, and SweetBench benchmark. It evaluates AI agents in managing Type 1 diabetes glucose control, achieving a COGI of 73.5 with Claude Opus 5.5 versus 71.2 for OpenAPS. The work demonstrates that individual adaptation and contextual information improve control, and hybrid rule-based/agent approaches balance performance and cost.

Why it matters: Provides a concrete, novel framework and benchmark for AI-driven physiological control, with quantitative results and practical implications for medical AI.

arxiv2d agoScore 82
AI in healthcareType 1 diabetesAgentic systems