Live intelligence

Latest AI updates

7 updates

Research

InfMasking: A Unified Variational Framework for Synergistic Multimodal Learning

InfMasking is a multimodal learning framework that randomly masks each modality's features before fusion, producing diverse partial combinations of cross-modal evidence. The fused representations from masked views must remain consistent with a reference target: an unmasked fused view in unsupervised training or the task-label class proxy when labels are available. The method models masked fused representations as Gaussian and optimizes closed-form variational bounds. On evaluated benchmarks, it improves predictive performance and yields higher mean PID synergy estimates and InterSHAP interaction attribution, with code released.

arxiv13h agoScore 68

multimodal learningvariational inferencefeature masking

Research

Text-Only Training for Zero-Shot Video Temporal Localization

This paper explores training a text-only LLM on synthetic textual video descriptions to perform temporal grounding, receiving video only at inference. The authors generate a 134K-example corpus, adapt Qwen3-8B with LoRA on text embeddings, and propose Nearest Entity Replacement (NER) to bridge the modality gap. NER improves zero-shot performance on Charades-STA and QV-Highlights, but a significant gap remains.

arxiv1d agoScore 78

Video Temporal LocalizationMultimodal LearningZero-Shot Transfer

Research Editor selected

Therapy-FIS: First Public Dataset for Automated Assessment of Therapist Interpersonal Skills

Therapy-FIS is a new dataset for automated assessment of Facilitative Interpersonal Skills (FIS) in psychotherapy, containing facial, acoustic, and textual signals from 890 therapist recordings and 8 patient videos. It includes eight FIS subscale ratings and an overall score, and supports two benchmark tasks: Therapist Performance Rating and Joint Therapist–mock-Patient Performance Rating. The authors also introduce a baseline model, DyadicFIS, which achieves best macro CCC scores of 0.2807 and 0.3299 on the two tasks.

Why it matters: This is the first publicly available dataset for automated FIS assessment, addressing a critical data scarcity and enabling reproducible research in mental health AI.

arxiv2d agoScore 85

Facilitative Interpersonal SkillsAutomated assessmentMental health AI

Research Editor selected

BREM-NET: Multimodal Neural Model Separates Behaviorally Relevant Dynamics

Researchers introduce BREM-NET, a nonlinear multimodal dynamical model that integrates behavior with multiple neural modalities like spiking activity and local field potentials. It separates behaviorally relevant from residual neural dynamics during inference, improving behavior decoding, neural prediction, and latent alignment with behavior. The model remains robust to asynchronous modalities and dropouts, addressing real-world recording challenges.

Why it matters: This presents a concrete new AI model with demonstrated improvements on a public dataset, offering practical value for neuroscience and brain-computer interface research.

arxiv2d agoScore 82

neural data fusionbrain-behavior modelingmultimodal learning

Products Editor selected

openJiuwen Agent-Memory: First Audio-Visual Dual-Stream Memory for Agents

openJiuwen introduces Agent-Memory, a system that gives AI agents a dual-stream memory for both audio and visual content, enabling them to understand long videos and recall key events accurately. The system reportedly boosts long-video memory precision and cuts question-answering latency by up to 97%. This marks a significant step in agent memory capabilities for multimodal understanding.

Why it matters: Provides a concrete, recent AI development in agent memory for long-video understanding, with measurable performance gains.

qbitai2d agoScore 82

AI agentsmultimodal learningvideo understanding

Research

ERIM: Explicit Rule Induction for Multimodal In-Context Learning

Researchers propose ERIM, a framework that improves multimodal in-context learning by explicitly inducing latent rules from demonstrations rather than relying on direct input-output mapping. It uses a two-stage training strategy combining supervised fine-tuning and reinforcement learning to teach MLLMs to perform rule induction. Experiments show consistent performance gains and improved stability across diverse MICL settings.

arxiv4d agoScore 78

multimodal learningin-context learninglarge language models

Products

Sensorium: Open-Source Library for Multimodal Surgical Data Collection

Sensorium is an open-source Python library for recording, processing, annotating, and curating multimodal surgical data from robot and sensor streams using ROS/ROS2. It was evaluated across 1516 recordings from phantom, ex vivo, and in vivo porcine experiments. The library aims to reduce cost and improve repeatability in embodied surgical AI data collection.

arxiv4d agoScore 72

surgical roboticsdata curationmultimodal learning