Live intelligence

Latest AI updates

2 updates
Research

AURA Uses a Frozen General-Audio Codec for Spatial Audio-Language Tasks

AURA investigates whether a general-audio codec trained without spatial supervision can support both sound identification and localization in multi-source scenes. The authors find that spatial cues, including inter-channel differences, remain in the frozen codec but are not readily accessible to a language model due to audio-text information asymmetry. Their controlled experiments show that scene-description completion and inter-channel difference exposure each improve left/right accuracy, while binding directions to the queried source requires both. AURA keeps the codec and language model frozen and exceeds the strongest dedicated system on SpatialSoundQA by 10.31 points in dual-source direction accuracy using about 40% of its trainable parameters.

spatial audioaudio-language modelsrepresentation learning
Research

AudioSphere: Self-Supervised Spatial Audio Representations for Frame-Level SELD

AudioSphere is a multi-channel masked autoencoder trained on Ambisonics recordings that reconstructs spectral content and directional cues using a joint spatiotemporal masking strategy over time-frequency patches and intensity vectors. The model supports frame-level predictions and produces clip-level representations through aggregation. The authors also release RealSELD, a standardized benchmark for evaluating frozen backbones on real-world multi-channel sound event detection and localization. They report that AudioSphere enables SELD with a frozen self-supervised backbone and matches or exceeds prior self-supervised audio embedding models on clip-level classification and frame-level sound event detection.

spatial audioself-supervised learningaudio representation