Live intelligence

Latest AI updates

9 updates
Research

WeightBench: Benchmarking LLMs as Weight-Space Programmers

The paper introduces WeightBench, a benchmark for evaluating how well large language models can read, modify, and create neural-network weights without gradient descent. It covers 42 tasks across five tracks, including weight reading, fixing, writing, recognizing impossible edits, and data-free compression, on models ranging from 2K-parameter policies to 7B language models. Results show that frontier agents can handle some small-network tasks but still struggle to patch language models, compress them without data, or reliably identify impossible edits.

Large language modelsWeight-Space ProgrammingAI benchmarks
Research

Swanada: Split-Encoder Capability-Routed Conformer CTC and Sub-30 MB On-Device Speech Recognition Across Twelve Scripts

Swanada is an open-source 16 kHz streaming ASR system covering Classical Sanskrit, Indian English, and ten Indian languages across browser WebAssembly, Node.js, and Python. It uses capability-routed shared INT8 Conformer encoders: a 17-layer 177.38 MiB Studio tier at d=512 and a distilled 14-layer 25.35 MiB Mobile tier at d=256, paired with twelve hot-swappable 515.0 KiB script-specific CTC heads. The design reports 11.6x and 67.8x reductions in 12-language storage, with runtime techniques including dynamic positional synthesis, MatMul pointwise convolutions, and QInt8 head calibration. On held-out and open public test clips, it reports 5.43% and 8.87% CER on Studio, 18.56% and 19.23% on Mobile, at 0.0097x-0.0264x RTF with 30.6 ms median TTFT.

Speech RecognitionOn-device AIMultilingual ASR
Research Editor selected

When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale

A new arXiv paper decomposes the effects of quantization and pruning on language models, separating direct damage from network compensation. The authors find that for comparable damage, smaller models restore most of the lost performance, while larger models retain more loss because compensation offsets less. This implies that compression that is nearly harmless in small models can become costly in large ones, challenging the assumption that scaling alone mitigates compression side effects.

Why it matters: Provides a novel, empirically grounded decomposition of compression damage versus compensation across model scales, with direct implications for efficient deployment of large language models.

model compressionscaling lawsattention mechanisms
Research

DictLLM: Dictionary-Based Post-Training Compression for LLMs

DictLLM is a new post-training compression method that decomposes weight matrices into a shared dictionary and layer-specific sparse coefficients, using SVD and Hessian-based sparsification and quantization. It achieves a 5.1x compression of LLaMA-7B with only a 2.9-point perplexity increase on WikiText-2, outperforming existing low-rank methods. The approach requires no further training and is designed for efficient deployment on mobile devices.

model compressionlarge language modelsefficient inference
Research

AutoCompress: Hardware-Guided Model Compression for LLMs

AutoCompress is a new framework that integrates hardware and workload constraints directly into differentiable architecture search for LLM pruning. It jointly optimizes global and per-layer structure using serving-engine cost models, improving accuracy over Minitron across multiple GPUs and workloads. The method also compresses a 120B model to 74B parameters while retaining 95% of benchmark performance.

model compressionhardware-aware optimizationLLM serving
Research

Context Compression Weakens Refusal Alignment in LLMs

A new study shows that encoder-based context compression systematically weakens refusal behavior in instruction-tuned language models, even when reconstruction fidelity is high. The effect persists across model families and compression architectures, and is traced to attenuation of the decoder's refusal direction. The authors propose Memory Steering, a lightweight inference-time intervention that restores refusal rates without retraining.

AI safetymodel compressionalignment
Research Editor selected

Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention

Zyphra released ZHA, a sparse attention method that uses a multi-level tree indexer to select relevant KV cache entries without scanning the full context. The method adds only 81,920 trainable parameters to frozen pretrained models like Qwen3.6-35B-A3B and achieves competitive or better results on reasoning, coding, and retrieval benchmarks compared to dense attention. On LongBench v2, the deep-tree variant scores 63.4% (vs. 58.0% dense) while reading only about 1.1% of keys. Projected speedups at billion-token contexts are 373–417× over dense attention, though no optimized kernels have been released yet.

Why it matters: Offers a concrete, parameter-efficient approach to subquadratic attention that retrofits existing models, with strong benchmark results and clear architectural novelty.

attention mechanismsefficient inferencelong-context models
Research Editor selected

ETAPS: Gradual Pruning with Sparse Adaptation Achieves High Quality at Extreme Sparsity

Researchers propose ETAPS, a pruning method that gradually tightens a global sparsity constraint while training a sparse delta with a small fixed budget. This approach avoids model-sized optimizer states, enabling efficient pruning of LLMs at extreme sparsity levels. Across models from 125M to 13B parameters, ETAPS achieves lower perplexity than the prior state-of-the-art ELSA at sparsities from 50% to 90%, with up to 42% perplexity reduction. It also reduces cost significantly, pruning LLaMA2-7B on a single H200 with 5x smaller persistent state and 49% fewer GPU-hours compared to ELSA.

Why it matters: This paper presents a concrete, recent advancement in LLM pruning that improves both quality and efficiency, offering practical value for deploying large models.

model compressionlarge language modelsefficient inference
Research

A Unified Hessian-Guided Framework for Post-Training Quantization

The paper proposes a unified post-training quantization framework for large language models that jointly optimizes quantization ranges, linear transformations, and weight rounding. It derives a Hessian-guided approximation of output changes from joint weight and activation quantization, using a tractable Fisher approximation to form a common output-sensitivity objective. The authors claim the method remains practical for modern LLMs and outperforms state-of-the-art transformation and rounding methods in channel-wise and block-wise settings.

Post-training quantizationLarge language modelsModel compression