Live intelligence

Latest AI updates

3 updates
Research

Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

The paper introduces Evolution Under Fire, a staged framework that jointly evolves executable safety harnesses and adaptive injection stressors for tool-using language-model agents. A utility-constrained, Pareto-scheduled tree search evolves complete harnesses from execution evidence while the task model, benchmark, tools, and evaluator remain fixed; adaptive stressor evolution then discovers utility-preserving attacks that drive a later stage of harness evolution. On AgentDyn with GPT-4o-mini, the evolved harness cuts attack success from 50.4% to 3.0% while improving clean and attacked utility versus the initial harness. On Agent-SafetyBench with Qwen2.5-7B-Instruct, attack success falls from 82.6% to 62.0% with helpfulness and runtime validity retained, and the harness transfers without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9%.

AI agent securityprompt injection defenseevolutionary algorithms
Research

WebArena-Pro: A Reproducible Multimodal Benchmark for Web Agents

WebArena-Pro is a new benchmark for web agents comprising 300 tasks across 20 self-hosted web applications in six domain categories. It extends evaluation beyond text and vision by treating audio and video as core observations, with dedicated actions for extracting information from each. The abstract reports top performance of 62.3% from Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5, while the strongest GenericAgent configurations reached 30.3%. The benchmark runs tasks in isolation to support reproducible parallel evaluation and aims to combine live-benchmark coverage with human-validated reproducibility.

WebArena-Proweb agentsmultimodal evaluation
Models Editor selected

Anthropic ships Claude Fable 5.2 and Mythos 5.2, retaking top LLM spot

Anthropic released Claude Fable 5.2 and Mythos 5.2, claiming gains in coding, reasoning and vision, with Fable 5.2 topping the Artificial Analysis Intelligence Index at 65 and setting records on HLE-Diamond and FrontierCode 2.0. Per-token pricing falls 20% to $8/$40 per million input/output tokens, but heavier test generation and subagent use push cost per task higher on some benchmarks; medium is the default effort and max effort underperforms high on several evals. Subscription access is restricted via tenure-scaled resets and a 50% weekly-limit cap for Fable, while data retention is required by default so safety classifiers can run. Launch-day integrations include Cursor, GitHub Copilot, Perplexity and Factory, and Anthropic says its most capable model remains internal.

Why it matters: It is a major frontier-model release with independent benchmark results, pricing changes, cost-per-task tradeoffs and practical deployment caveats that directly affect model selection.

Anthropicfrontier modelsLLM benchmarks