Live intelligence

Latest AI updates

15 updates
Research

KnowledgeClaw: Towards Agentic Knowledge Engineering via Omnimodal Hypergraphs

This arXiv paper formalizes Omnimodal Knowledge Engineering (OmniKE), arguing that a live knowledge base needs omnimodality, a closed-loop lifecycle and multi-granular application simultaneously. It releases OmniKE-Bench, comprising six tasks with 1,829 model-free items over 3,960 real files in five modalities. The authors propose KnowledgeClaw, an agentic system that builds an omnimodal typed hypergraph, maintains it with a closed-loop lifecycle agent and routes queries to the needed granularity. KnowledgeClaw ranks first on all six tasks with an overall OKE of 55.9, reported as +14.6 over the strongest baseline, with software and data publicly available.

Knowledge engineeringLLM agentsBenchmarks
Research

NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding

NexSight presents a method for building verifiable tasks to evaluate and train agentic long-video understanding. Its SightForge pipeline accepts a task only if a verifier blind to the draft answer can reconstruct it from evidence densely reacquired from the raw video and counterexample checks fail to refute it. This produces SightLine, 310 evidence-traceable tasks from 108 long videos, evaluated through SightLoop, a shared multiscale harness supporting temporal exploration, local inspection, evidence integration, and revisiting. The strongest model reaches only 54.52% on SightLine but 79.35% when given annotated evidence, while Sight-35B nearly doubles its base model's interactive accuracy from 21.29% to 42.26% with under 6K examples.

long-video understandingagentic AIbenchmarks
Research

Accelerate Bench: Evaluating Agents for Application-Level Program Optimization

Researchers introduce Accelerate Bench, a benchmark of 32 CPU optimization tasks adapted from scientific applications and supercomputing competitions. Each task supplies a working program plus workload, correctness requirements, timing boundaries and permitted edits, with public testing for iteration and a separate verifier scoring the final submission. On version 1.1, 16 coding agents (model plus harness pairings) score between 25.5% and 67.4% Score@1, with most programs correct and speedup separating the leaders. The two leading agents reach at least 2x speedup on 22 or more of the 32 tasks, while the other fourteen hit 2x on only 8 to 17 tasks, leaving 125 of their 412 correct outcomes below 1.25x. Controlled replays show build and runtime settings alone can swing identical source code between 0.29x and 2.12x, highlighting the need for coordination across a full application.

coding agentsprogram optimizationbenchmarks
Research

AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents

Researchers introduce AstroMemBench, a benchmark for evaluating long-term memory systems in embodied spacecraft agents operating under communication delays, resource limits and safety constraints. The benchmark uses a Scenario-Challenge-Capability design method and covers 16 memory subchallenges through 140 templates and 2240 instances across nine tasks in planetary surface operations, on-orbit servicing and deep-space exploration. A deterministic environment scores actions for task completion, state changes, safety and resource use, and ablations under nine conditions test sensitivity to memory recovery and design differences. Experiments comparing six configurations find that Mem0, Chain-of-Memory, LightMem and RAG improve same-session mean scores over a native Pi Agent, with no clear advantage of the three memory systems over RAG, while Mem0 outperforms RAG by 12.25 points after a session restart. The benchmark and evaluation code are publicly released.

memory systemsembodied AIspacecraft autonomy
Research

LLM-Generated World Models Can Be Mechanically Valid but Structurally Wrong

A new arXiv study evaluates whether five LLMs can generate formal world models for neuro-symbolic reasoning as partially observable stochastic games. It introduces a 44-scenario benchmark and analyzes 410 first-pass generations across mechanical validity, reference-relative structural agreement, and semantic criteria. In the game track, 134 of 135 models passed a strict mechanical audit, but mean structural F1 was only 0.366 and just 28 of 134 exactly matched the authored reference. Naming a country in the cultural track changed structural representations for most LLMs while producing only small semantic-score changes.

LLM world modelsneuro-symbolic reasoningbenchmarks
Research

GeoMind benchmark targets spatial reasoning and memory in robotic manipulation

Researchers introduced GeoMind, a benchmark for structured spatial understanding in tabletop robotic manipulation. It contains 54 evaluated task variants focused on spatial reasoning and spatial memory, along with a scalable pipeline for generating and annotating task instances with verified targets and executable interactions. Evaluations across embodied agents found consistent weaknesses, including difficulty tracking identities after spatial changes and translating relational or geometric constraints into actions. The benchmark is released publicly on GitHub.

robotic manipulationembodied agentsspatial understanding
Research

FondueQA Benchmark Probes How LLMs Fill Gaps in Incomplete Retrieval Evidence

Researchers introduce FondueQA, a multi-hop QA benchmark in which every required fact is uniquely localized to a designated evidence document, enabling controlled document-level evidence ablations. The design addresses benchmark contamination concerns that make it hard to tell whether correct answers under missing evidence reflect general parametric knowledge or memorization of benchmark instances. Experiments show parametric compensation depends on entity popularity and the structural role of the missing evidence, with bridging evidence sometimes more useful than the answer-bearing document. Models can recall a missing fact in isolation yet fail to use it under partial evidence, especially when plausible competing entities appear in context, reducing Exact Match by up to 15 percentage points.

LLM evaluationretrieval-augmented generationbenchmarks
Research

MemoryDojo: Benchmarking Poisoning Through LLM Agent Memory

MemoryDojo is a benchmark for tracing how poisoned information propagates through LLM agent memory systems across sessions and measuring its influence on later agent behavior. It comprises 155 multi-session long-horizon trajectories spanning four domains and supports five different memory systems. The authors report that all evaluated memory systems are susceptible to memory poisoning, with substantial variation in downstream effects across architectures. They also propose a memory-aware poison optimization process that increases poisoning efficacy, highlighting differences between memory architectures.

LLM agent securityagent memory systemsadversarial robustness
Models

webAI Releases TwIL-8B, an 8.8B Reasoning Model Scoring 98.2 on MATH-500

webAI has released TwIL-8B, an 8.8B dense reasoning model post-trained from IBM's Granite 4.2 8B, optimized for math, code, and instruction following. It achieves 98.2 on MATH-500 and 81.6 on LiveCodeBench, while using 38% fewer reasoning tokens than Qwen3.5-9B on AIME 2026. The model is available on Hugging Face under a non-commercial license, with GGUF builds for local CPU or modest-VRAM GPU inference.

webAIIBM Granitereasoning models
Research

ANGLER and REEF: Adaptive Red-Teaming Benchmark for Web Agents

Researchers introduce REEF, a benchmark of red-teaming executable environments with state-based verifiers for measuring web agent security, covering 9 websites across travel, commerce, productivity, media and communication. Each benign task is paired with a malicious objective delivered through 5 attack types at the user and environment level. ANGLER builds on REEF's execution feedback to evolve attacks via LLM-based mutation and near-miss seed selection rather than a fixed attack set. Across 8 VLM-based web agents, ANGLER reached a 20.36% end-to-end attack success rate versus 15.24% for the non-adaptive setting, and cut benign task success from 61.9% to 44.4%.

web agentsAI securityred-teaming
Research

GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

The paper identifies Correct-but-Visually-Misgrounded Reasoning (CVMR) in vision-language models, where models can reach correct final answers despite misperceiving visual information during intermediate reasoning. It proposes GRIT, a training framework that incorporates visual grounding into reinforcement learning with verifiable rewards (RLVR) to mitigate CVMR. The authors also introduce GRIT-Bench for systematic evaluation across text length and reasoning depth. Experiments report improved reasoning accuracy and grounding fidelity over existing RLVR methods.

vision-language modelsreinforcement learningvisual grounding
Research Editor selected

BURGER benchmark tests VLA generalization through robotic drawing

Researchers introduced BURGER, a language-conditioned robotic drawing benchmark with a prompt-driven pipeline for generating continuous drawing trajectories in simulation and the real world. Using 28,200 simulated demonstrations across 282 drawing tasks, they evaluated five representative vision-language-action models on data scaling, language grounding, and multimodal behavior. The benchmark revealed distinct failure patterns and indicated that some VLA models benefit from greater task diversity and demonstration scale and can generalize to unseen compositions. A trained policy was also deployed on a Franka Panda arm to draw reactively under human intervention.

Why it matters: It provides a structured benchmark and empirical analysis for measuring how well vision-language-action models generalize across language-conditioned robotic tasks.

vision-language-action modelsrobotic generalizationbenchmarks
Research

Task Advancement Memory improves long-horizon GUI agents

A new arXiv paper introduces Task Advancement Memory (TAM), which treats verified task progress as both execution state and supervision for GUI agents. Task Advancement State tracks how verified progress changes remaining goals so agents can build on earlier results across applications, while post-task supervision abstracts interactions that produced verified local outcomes and keeps useful operations from both successful and failed tasks. On MobileWorld GUI-only tasks, Advancement State alone raises success from 12.0% to 28.2%. Full TAM reports a 17.6% relative gain in task success over the strongest evaluated baseline, a 38.7% reduction in lifecycle tokens per solved task, and state-of-the-art success on SPA-Bench single-app English Level 3.

GUI agentsTask Advancement Memoryagent memory
Models

Moonshot AI Releases Kimi K3.1 Pro: 2.8T-Parameter Open-Weight MoE Model with 1M-Token Context

Moonshot AI has released Kimi K3.1 Pro, a post-trained successor to Kimi K3 with 2.8T total and 104B active parameters in a mixture-of-experts architecture and a 1M-token context window. It is available now through an OpenAI-compatible API ($2/$10 per million input/output tokens, cheaper than K3), with a 2-3x faster Highspeed variant. Moonshot reports near-frontier coding results, including 51.7 on SWE-Marathon and 65.7% on Terminal-Bench 4.0, but a large gap on science terminal tasks (38.1 on T

Moonshot AIKimi K3.1 ProLLM releases
Research

Taste-Bench exposes weak long-term strategic choices in AI agents

A new benchmark called Taste-Bench evaluates how well AI agents make strategic choices during long-running tasks. According to the report, the best-performing model, GPT-5.6 Sol, selects the correct path on just under 60 percent of simple binary questions. The benchmark also finds that longer reasoning does not improve these decisions.

Taste-BenchAI agentsbenchmarks