Translate
Live intelligence

Latest AI updates

1,712 updates
AllModelsProductsIndustryResearchTutorialOpinion
Research

Token Identity Brier Flow Improves Continued Training of Continuous Language Flows

Researchers propose Token Identity Brier Flow (TIBF), which applies squared supervision directly to token identities when continuing pretrained language flows, combining network scores with token-matching scores under a Gaussian observation model and using the Brier loss for token probabilities. They show that squared error in predicted token probabilities bounds the squared errors of continuous denoising predictions and flow velocities, and provide a risk decomposition separating observation uncertainty from posterior fitting error. Under matched continuation budgets with frozen embeddings, TIBF achieves Gen.PPL values of 33.96 on OWT and 81.17 on LM1B, outperforming continuous-MSE and cross-entropy continuation controls and attaining the lowest Gen.PPL at every tested sampling budget on both datasets.

arxiv14h agoScore 70
continuous language modelingdiffusion modelslanguage modeling
Research

World-Model-Guided Navigation Distills Foresight into Fast Policy

Researchers propose a training-time approach for human-like navigation in multi-agent scenes, using world models as preference oracles to distill foresight into a fast policy. Two task-aligned world models score candidate actions by predicted consequences, and the policy is refined via group-relative preference distillation. At deployment, the world models are removed, enabling single forward-pass action selection. Evaluations across five scenarios show safer, more socially coherent, and more human-like navigation than baselines.

arxiv14h agoScore 74
world modelsrobot navigationreinforcement learning
Research

O²PD: Overlap-Gated On-Policy Distillation Improves Token-Level Alignment

This paper introduces O²PD, a new method for on-policy distillation in language models. It addresses two key questions: which tokens to align and how to align them. The method applies distillation only to the first 20% of student-generated tokens, where teacher supervision is more reliable, and uses an overlap-gated α-divergence to adaptively switch between mode-covering and mode-seeking behavior. Theoretical analysis identifies problems in existing top-k OPD and proposes support strategies to improve alignment.

arxiv14h agoScore 70
knowledge distillationlanguage model trainingalignment
Research

LLM Token Probabilities as Confidence Estimates in Human Behavior Simulation: A Luxembourgish Next Like Prediction Case Study

This paper investigates whether LLM token probabilities can serve as confidence estimates in human behavior simulation, using a Next Like Prediction task on Luxembourgish news platform data. The authors find that open-weight models (Ministral 3B/8B) are overconfident with in-context learning alone, and a simple statistical baseline outperforms them on both accuracy and calibration. Scaling model size improves accuracy but degrades calibration, while task-specific fine-tuning significantly improves both, reducing Expected Calibration Error from 0.2 to 0.006 and boosting accuracy from 0.536 to 0.758.

arxiv14h agoScore 70
LLM calibrationhuman behavior simulationtoken probabilities
Research

RELOAD: Looping Layers in Typed Decision Models Improves Reasoning and Calibration

RELOAD is a method that retrofits loops into pretrained decision models, allowing them to spend variable test-time compute in latent space without adding parameters. When trained with the model, these loops improve accuracy and calibration, especially on problems requiring longer reasoning chains than seen in training. Across six models from 0.4B to 4B parameters, encoders gain up to 23 points, and a lightweight controller preserves most gains at about three-quarters of the compute.

arxiv14h agoScore 74
adaptive computationmodel efficiencyreasoning
Research

Scaling Laws for SAE Training Data

This paper investigates how the number of unique activations affects sparse autoencoder (SAE) training when the total training budget is fixed. It finds that quality improves rapidly with buffer size but then plateaus, where replaying the buffer is nearly as effective as using fresh activations. The authors propose a scaling law for limited data and validate it across various models, layers, and SAE variants. They provide a practical rule: choose the smallest buffer that reaches the quality plateau and replay it, reducing activation storage by 8-64x with negligible quality loss.

arxiv14h agoScore 77
sparse autoencodersscaling lawsmodel interpretability
Research

RIVER: Evidence-guided multi-agent retrieval for enzyme–reaction discovery

RIVER is a multi-agent framework that coordinates LLMs, biochemical tools, and knowledge resources to perform bidirectional enzyme–reaction retrieval. It assembles multi-level evidence and uses validation modules to rerank candidates, achieving top success rates on public benchmarks and outperforming general-purpose and biology-specialized multi-agent systems in E. coli retrieval tasks.

arxiv14h agoScore 72
enzyme function annotationmulti-agent systemsretrieval
Research

New Study Reveals How Irrelevant Context Degrades LLM Accuracy and How to Fix It

A new arXiv paper identifies a mechanism called the Intrusion Pathway that explains why language models lose accuracy when prompts contain irrelevant, off-topic context. The authors localize the problem to a sparse set of 'intrusion heads' and show that knocking out about 1% of attention heads recovers 57% of the performance drop at bounded cost. This offers a practical intervention for improving robustness in RAG, tool-using, and long-context systems.

arxiv14h agoScore 74
language model robustnessattention mechanismsretrieval-augmented generation
Research

Axis-LoRA: Probing and Adapting Embedding Models for Personal-Memory Retrieval

Researchers find that frozen dense embedding models poorly capture time, place, person, and event axes in personal-memory retrieval, with axis changes barely shifting embeddings. They propose Axis-LoRA, a per-axis adapter with four LoRA branches and a per-query gate, which improves retrieval accuracy on benchmarks like LongMemEval, ATM-Bench, and LoCoMo while training only a small fraction of parameters.

arxiv14h agoScore 70
retrieval-augmented generationparameter-efficient fine-tuningpersonal AI
Research

STAG-Net: Lightweight Head Pose Estimation with Structure-Aware Attention

STAG-Net is a new lightweight head pose estimation method that uses structure-aware attention over landmark groups, achieving state-of-the-art accuracy among landmark-based approaches on standard benchmarks. It requires only 0.012 to 0.043 million parameters and 24.1 to 83.6 thousand FLOPs, about three orders of magnitude fewer than FSA-Net. The FaceMesh pipeline runs at 322.9 FPS on a Tesla T4 GPU, nearly twice as fast as the fastest image-based baseline, and training completes within 28 minutes.

arxiv14h agoScore 70
head pose estimationefficient neural networksfacial landmark processing
Research

Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding

This paper presents a theoretical framework linking masked diffusion models for text generation to statistical physics of driven many-particle systems. It shows that the error in parallel decoding corresponds to entropy production, and for Markov data, the generation process maps to random sequential adsorption with labelled gaps, solved exactly. The authors derive a thermodynamic metric from mutual information, showing optimal schedules are geodesics, and that longer text correlations concentrate error at the start of generation.

arxiv14h agoScore 75
masked diffusion modelsparallel text generationstatistical physics of machine learning
Research

LLMs can audit clinical prediction networks via computation alone, not weight access

Researchers tested whether frontier LLMs (GPT-6 Astra and others) can audit small clinical prediction networks by inspecting their weights or using a calculation tool. Without tools, Astra succeeded on 353-weight networks but failed at 3,457 weights; with a tool that exposed predictions (black-box) or weights (white-box), it reliably recovered planted relationships and detected biased subgroups in networks up to 38,401 weights. The study concludes that for auditing these networks, computation and elicitation matter, not access to the weights.

arxiv14h agoScore 75
large language modelsclinical AImodel auditing
Research

CORE: A Training-Free Harness for Efficient Deep Research Search

CORE is a new training-free harness for deep research agents that balances long-passage retrieval with short evidence snippets. It uses a dense retriever for coarse search and a late-interaction reranker to extract only the most relevant snippets, while context management allows expanding context around retrieved sources and suppresses duplicates. In controlled comparisons, CORE achieves state-of-the-art accuracy and evidence recall across multiple agent models and benchmarks.

arxiv14h agoScore 74
COREdeep research agentsretrieval-augmented generation
Research

NEDNet: Decomposing Network Effects for Causal Estimation under Interference

Researchers propose NEDNet, a neural network that decomposes outcome responses into baseline, treatment, and exposure pathways for causal effect estimation under network interference. It uses path-specific neighborhood contextualization and a reference-normalized decomposition to improve accuracy. Experiments on BlogCatalog and Flickr benchmarks show NEDNet achieves the lowest PEHE for individual, peer, and total effects.

arxiv14h agoScore 77
causal effect estimationnetwork datatreatment effects
Research

InnerLogic-Surg: An Open Surgical Foundation Model via Continual Pretraining

Researchers present InnerLogic-Surg, an open family of surgical vision encoders (21M–300M parameters) obtained by continually pretraining a transformer on millions of surgical images from nearly 3,000 hours of video spanning over fifteen procedure types. The method inherits DINOv3's pretraining stage and adds a learned feature-upsampling module for efficient high-resolution dense prediction. Across 20 surgical tasks including workflow, instrument and anatomy recognition, segmentation, detection, and depth estimation, the largest model achieves the best linear-probe performance under the same architecture. The work demonstrates that surgical continual pretraining yields transferable representations that generalize across procedures and task types, offering a scalable alternative to procedure-specific approaches.

arxiv14h agoScore 75
surgical computer visionfoundation modelsmedical AI
Research Editor selected

Active In-Context Learning for Tabular Foundation Models: When, Where, How

This paper studies active in-context learning for tabular foundation models (TFMs), where labels are acquired iteratively to build the context. The authors propose and evaluate target-aware and information-gain-based acquisition rules against classical methods across twenty benchmarks. Key findings show that diversity-based acquisition works best early, target-aware nearest-neighbor selection is crucial, and the EPIC rule retains 80-87% of full-data performance with only 5-10% of the pool labeled.

Why it matters: Provides concrete guidance on label-efficient strategies for tabular foundation models, a practical concern for real-world deployment.

arxiv14h agoScore 83
active learningtabular datafoundation models
Research Editor selected

When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale

A new arXiv paper decomposes the effects of quantization and pruning on language models, separating direct damage from network compensation. The authors find that for comparable damage, smaller models restore most of the lost performance, while larger models retain more loss because compensation offsets less. This implies that compression that is nearly harmless in small models can become costly in large ones, challenging the assumption that scaling alone mitigates compression side effects.

Why it matters: Provides a novel, empirically grounded decomposition of compression damage versus compensation across model scales, with direct implications for efficient deployment of large language models.

arxiv14h agoScore 88
model compressionscaling lawsattention mechanisms
Research Editor selected

DISCERN: Distributional Synthetic Replay for Incremental Machine Unlearning

DISCERN is a new method for incremental class unlearning that removes classes from a model sequentially without access to the retain set. It fits per-class parametric distributions to synthetic embeddings, selecting the best family via statistical tests and cross-validated log-likelihood. On transformer backbones, DISCERN remains stable after 20 sequential deletions, while a strong simultaneous baseline collapses. The method reduces resurfacing of forgotten classes, acting as a stabilizer of accumulated forgetting.

Why it matters: This paper introduces a novel approach to a practical problem (incremental unlearning) with strong empirical results on transformers, offering clear value for researchers and practitioners.

arxiv14h agoScore 85
machine unlearningincremental class unlearningsynthetic data replay