Live intelligence

Latest AI updates

47 updates

Research Editor selected

WebHarbor: Docking Real Websites for Evolving GUI Agent Environments

WebHarbor is a suite of 153 executable, dockerized website environments that reconstruct real-world web services with deterministic control, covering 3,518 tasks including multi-step and post-authentication workflows. Built via human-agent collaboration, it provides stable evaluation and high-fidelity synthetic environments for RL training. A Qwen 4B-level GUI agent trained with RL in WebHarbor generalizes to unseen tasks and competes with leading open-source agents across four benchmarks.

Why it matters: Offers a concrete, novel infrastructure for GUI agent training and evaluation, with demonstrated generalization results, valuable for researchers and developers.

arxiv2d agoScore 85

GUI web agentsenvironment simulationreinforcement learning

Research Editor selected

Self-Improvement with Auxiliary Rewards: Learning from Verified Traces and Evaluating Transfer

This paper investigates using auxiliary trajectory preferences, specifically confidence-based (SC) and entropy-based (UID-Conj.) metrics, to enhance small language model self-improvement and transfer under limited supervision. Online experiments with Qwen3-1.7B on 30 AIME problems show UID-Conj. improves macro-average by 2.08 percentage points over unmodified TTRL, while offline weighted SFT yields a 1.93 point gain with a 95% CI of [0.44, 3.44]. The benefits are task-dependent, with SC rewarding instruction following and UID-Conj. producing stronger training outcomes in several settings.

Why it matters: Provides a concrete method for improving small LM self-improvement using trajectory preferences, with empirical gains on reasoning tasks and transfer benchmarks.

arxiv3d agoScore 85

reinforcement learninglanguage model trainingreward modeling

Research Editor selected

Subliminal Reinforcement Learning: traits travel through a judge's scores

A new study demonstrates that a student model can acquire a trait (e.g., preference for an animal) solely from a judge's scores during group relative policy optimization, even when the training data contains no semantic trace of that trait. The effect was confirmed across 17 out of 48 experimental cells, including those using only numbers. The authors identify a four-stage chain required for transfer and show that removing the reward difference between judges eliminates the effect. The findings have direct implications for AI safety, as undesirable dispositions could similarly propagate through judge scores.

Why it matters: Reveals a subtle but significant risk in reinforcement learning where traits can be transferred without explicit data, informing AI alignment and safety measures.

arxiv3d agoScore 88

reinforcement learningAI alignmentmodel behavior

Research

ART of PIV: Agentic Real-Time Particle Image Velocimetry for Active Flow Control

Researchers introduce ART of PIV, a learned agent that selects a particle image velocimetry pipeline configuration for each image pair under a specified speed budget. Across cross-correlation, classical optical-flow, and learning-based estimators, it reduced average end-point error by 12.2–22.4% relative to the best single estimator at matched speed, with gains extending to 1,000–4,000 image pairs per second. In a simulated velocity-regulation task, it lowered mean normalized closed-loop cost by 13.24–13.59 percentage points versus a tuned fixed estimator. The work also demonstrates tuning on a tabletop water channel and training a DreamerV3 reinforcement-learning controller in the physical world to track a prescribed flow field in a fluidic pinball wake.

arxiv3d agoScore 68

Particle image velocimetryActive flow controlReinforcement learning

Research

Pinching Antennas for ISAC: Waveguide Architectures, Solutions, and Future Directions

This arXiv paper reviews pinching antenna systems (PASS) for integrated sensing and communication (ISAC) in 6G wireless networks. It compares single-waveguide, multi-waveguide, and segmented-waveguide architectures, and presents a case study using segment-hysteresis reinforcement learning for joint optimization. The work highlights tradeoffs between waveguide loss and aperture gain, and outlines future research directions including digital twins and trustworthy control.

arxiv3d agoScore 72

integrated sensing and communication6G wireless systemsantenna array optimization

Research

From Representation to Readout: Mechanistic Analysis of Membership Signals in Language Models

This arXiv paper presents a mechanistic study of membership inference signals in large language models, introducing Decodable Information Score, Used Information Score, and Readout Accessibility Gap (DUG) to separate membership information in internal representations from information exposed through readout. Across supervised fine-tuning and reinforcement-learning-based membership inference methods, it reports membership-specific effects in both decodable information and readout under SFT, but primarily in readout under RL methods, with activation patching localizing effects to intermediate layers. The paper also examines reinforcement learning with verifiable rewards contamination and finds behavioral changes can emerge without a corresponding increase in likelihood-based membership inference signals.

arxiv3d agoScore 72

membership inferenceLLM interpretabilityreinforcement learning

Research Editor selected

KEEPER: Learning Control-Sufficient Textual Memory for Long-Horizon LLM Agents

The paper introduces KEEPER, a framework that analyzes auxiliary objectives for recursive textual memory in LLM agents, identifying self-consistency bias where incorrect claims persist in memory. It compares four supervision targets and finds reward-related prediction achieves the highest task performance before and after offline reinforcement learning. A claim-level analysis reveals error persistence despite corrective observations, motivating evaluation of error persistence and valid-information retention alongside task outcomes.

Why it matters: Provides a concrete, recent research contribution addressing a critical failure mode in long-horizon LLM agent memory, with practical implications for agent reliability.

arxiv3d agoScore 85

LLM agentsmemory managementreinforcement learning

Research

KARU: Reinforcement Unlearning of LLM Knowledge Associations

Researchers propose Knowledge Association (KA) as the target of LLM unlearning, defined as the conditional predictive distribution linking a query context to its target knowledge. They introduce Knowledge Association Reinforcement Unlearning (KARU), a reinforcement learning framework that combines token-level reward construction, distribution-level regularization, and proximity-aware retain sampling. Experiments across multiple LLMs and benchmarks report improved forgetting effectiveness while maintaining comparable retention and general utility.

arxiv3d agoScore 72

LLM unlearningKnowledge associationsReinforcement learning

Research

GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

The paper identifies Correct-but-Visually-Misgrounded Reasoning (CVMR) in vision-language models, where models can reach correct final answers despite misperceiving visual information during intermediate reasoning. It proposes GRIT, a training framework that incorporates visual grounding into reinforcement learning with verifiable rewards (RLVR) to mitigate CVMR. The authors also introduce GRIT-Bench for systematic evaluation across text length and reasoning depth. Experiments report improved reasoning accuracy and grounding fidelity over existing RLVR methods.

arxiv3d agoScore 76

vision-language modelsreinforcement learningvisual grounding

Research

TSBPO: Tree-Structured Sampling and Update for Training Interactive Language Agents

The paper introduces TSBPO, a tree-structured sampling-to-update training paradigm for interactive language agents trained from terminal outcomes. It combines a likelihood-fitted Root/Branch allocator that targets contrasting-outcome continuations, sibling-based trajectory advantages with observation-group step advantages, and complete-response token-loss aggregation. With a Qwen2.5-1.5B actor, the authors report 97.79% seen and 96.64% unseen stochastic success on ALFWorld, plus 81.07% per-episode success on WebShop over six stochastic episodes per test goal. An ablation and aggregation comparison are also reported.

arxiv3d agoScore 78

interactive language agentsreinforcement learningTSBPO

Research

LIT: Co-Evolving Video Director Agents Improve Production and Interaction

The paper introduces LIT, a framework for video director agents that co-evolves directing and user interaction capabilities. It uses a simulated user to probe weaknesses and a meta engineer to revise strategies, forming an evolutionary loop. On Test and OOD Test splits, LIT improves production and interaction scores by 19.1% and 32.0% relative to the initial pipeline.

arxiv3d agoScore 72

AI agent evolutionvideo generationhuman-AI interaction

Research

Geo-FSRL: Geometry-Optimized Few-Step Flow Reinforcement Learning

Geo-FSRL introduces a selection principle for stochastic policies in few-step flow models, minimizing conditional KL divergence for velocity-output updates. It derives a closed-form optimizer within a coefficient-preserving family of isotropic Gaussian transitions, improving training stability. The method achieves higher final reward scores than Flow-GRPO and Flow-CPS on image and video generation under 4-step settings.

arxiv3d agoScore 78

few-step flow modelsreinforcement learninggenerative AI

Research Editor selected

CRoS: A New Causality Reward for More Faithful Chain-of-Thought Reasoning

Researchers propose CRoS, a causality metric that perturbs the order of reasoning steps without altering content, avoiding confounding effects in existing perturbation-based metrics. On Reasoning Gym with Gemma-3-4B, CRoS improves out-of-domain accuracy, CoT quality, and faithfulness compared to existing rewards. It achieves 46.8% accuracy on unseen datasets, outperforming the strongest baseline by 3 percentage points.

Why it matters: This introduces a novel method to improve reasoning faithfulness in RL-trained models, with concrete benchmark gains, valuable for AI researchers and practitioners.

arxiv3d agoScore 82

Chain-of-Thought ReasoningReinforcement LearningCausality Metrics

Research Editor selected

MASCAL Trains AI Agents to Share Calibrated Confidence During Collaboration

The paper introduces MASCAL, a reinforcement-learning approach that trains collaborating AI agents to attach calibrated correctness probabilities to their answers. Confidence-aware weighting helps downstream agents learn to use these signals when deciding whether to trust or verify upstream results. Across three collaboration architectures covering Web retrieval and reasoning, the authors report improved accuracy and calibration, with final confidence scores supporting selective answering and targeted rechecking.

Why it matters: The research addresses unreliable handoffs between AI agents with an explicit confidence signal that can inform verification decisions.

arxiv6d agoScore 84

MASCALMulti-agent collaborationConfidence calibration

Research

Value of Information Links Memory Retrieval and Credit Assignment in Reinforcement Learning

A new paper proposes using value of information, estimated from archived rollouts with a learned world model, to address both memory retrieval and credit assignment in reinforcement learning. The authors derive a lower bound on the value of an action that acquires an observation and use the same quantity to characterize optimal retrieval sets. A pilot retrieval system solves a memory task with reduced context, though the abstract does not report quantified latency or energy savings.

arxiv6d agoScore 74

Reinforcement learningMemory retrievalValue of information

Research

SCARL: Sequence Confidence-Aligned Reinforcement Learning for RLVR

The paper identifies a sequence-confidence misalignment in reinforcement learning with verifiable rewards: because the advantage is blind to a policy's confidence, a response the policy barely found and one it produces with near certainty receive the same update, and a confident error is corrected no harder than an exploratory one. SCARL reshapes each advantage, without changing its sign, by the alignment gap between the response's group-relative confidence and the outcome that its advantage certifies, normalized into per-class shares of the advantage budget; it provably reduces to the base estimator when a class's confidences agree and adds no extra passes and only one hyperparameter. On Qwen3-4B-Base it lifts average Pass@32 by 2.4 points over DAPO and is the strongest of nine compute-matched methods on coverage and per-draw accuracy. Applied unchanged across four estimators and four backbones, it improves 14 of 16 settings while preserving policy entropy and widening the confidence-correctness separation; code is released.

arxiv6d agoScore 78

Reinforcement learningLLM reasoningConfidence calibration

Research Editor selected

Simulation-based reinforcement learning improves AI psychiatric intake in role-played encounters

An arXiv preprint reports that multi-turn reinforcement learning in simulation improved a language model’s structured psychiatric intake performance compared with its identically prompted base model. In 120 encounters with psychiatrists role-playing standardized patients, independent psychiatrist evaluators preferred the trained agent in 76.7% of matched scenario pairs, with improvements in communication and differential diagnosis. In adversarial simulated presentations, under-triage of acute safety risks fell from 30.5% to 12.9%; the findings do not establish effectiveness with real patients, and residual gaps require safeguards and clinician oversight.

Why it matters: The study provides concrete evidence that simulation-based training can improve psychiatric conversational agents while highlighting safety limitations and the gap between role-played evaluation and real-world care.

arxiv6d agoScore 87

Reinforcement learningPsychiatric intakeConversational AI

Research Editor selected

TeamGym benchmarks LLM agents on team decisions with distributed information

Researchers introduced TeamGym, an environment for evaluating and training language-model agents on team decision-making tasks built from legal, finance, medical, and software engineering corpora. TeamGym uses a method to identify "pivotal information"—a minimal set of facts that is individually necessary and collectively sufficient for a correct decision—so it can control how information is distributed and measure whether agents integrate it. Across 8 language models and 12 source corpora, distributing pivotal information lowered success by 22.9 percentage points versus giving it to a single model, and requiring agents to extract that information themselves lowered success by a further 19.9 points. Reinforcement learning on final-decision loss improved decision quality and produced emergent coordination in participation, speaker selection, and communication.

Why it matters: It provides a concrete benchmark and training method for a key multi-agent weakness: sharing and integrating distributed information.

arxivOct 29, 2026Score 82

TeamGymmulti-agent systemsLLM evaluation