Live intelligence

Latest AI updates

3 updates
Research

Neuron-Level Specialization in Mixture-of-Experts Language Models

This arXiv paper introduces Routing-Aware Magnitude Profiling (RAMP) to study domain-specialized computation at the neuron level in Mixture-of-Experts models. It reports domain-specialized neuron sets with 58-74% overlap across same-domain samples versus 15-24% across different domains. The analysis traces neuron importance through pre-training and post-training across three open MoE families, finding continued reorganization between pre-training checkpoints but largely preserved rankings after post-training. It also presents RAMP-based Pruning (RAMPr), which preserves 91-98% of full performance at 75% neuron retention across six models and, at 50% retention, reduces resident GPU memory by 36-43% and improves batched prefill throughput by 17-39%.

Mixture-of-ExpertsModel PruningLanguage Models
Research

Representational Atoms in Neural Networks

The paper formally defines fundamental representational units (FRUs) and introduces fidelity and stability as evaluation criteria. It proves that threshold-activated sparse autoencoders can identify FRUs with high fidelity and stability under appropriate conditions, and reports large-scale experiments supporting the result. Further analyses indicate that higher-stability FRUs are more monosemantic and can be composed across layers to recover semantic organization corresponding to human conceptual structure. The work offers a unified framework for defining, evaluating, and identifying representational units in neural networks.

neural network interpretabilitysparse autoencodersrepresentational units
Research Editor selected

Weight-space programming audits and repairs flawed clinical decision networks

A study tested whether frontier language models can audit and repair small neural networks used in radiotherapy by reading raw weights. Claude Opus 5.5 identified broken rules in 16 of 18 flawed networks and repaired them by changing a median of 3 parameters, outperforming fine-tuning which altered nearly every parameter. The model could not certify healthy networks or reliably find hidden triggers, indicating limits in verification.

Why it matters: Demonstrates a novel AI capability with practical implications for auditing clinical AI models, backed by concrete experimental results.

AI safetyclinical decision supportneural network interpretability