Live intelligence

Latest AI updates

6 updates

Research

DictLLM: Dictionary-Based Post-Training Compression for LLMs

DictLLM is a new post-training compression method that decomposes weight matrices into a shared dictionary and layer-specific sparse coefficients, using SVD and Hessian-based sparsification and quantization. It achieves a 5.1x compression of LLaMA-7B with only a 2.9-point perplexity increase on WikiText-2, outperforming existing low-rank methods. The approach requires no further training and is designed for efficient deployment on mobile devices.

arxiv14h agoScore 72

model compressionlarge language modelsefficient inference

Research

StepBack: Step-Level Error-Localized Resampling for Efficient Test-Time Reasoning

StepBack is a new decoding strategy that improves test-time reasoning efficiency by localizing likely first errors in draft reasoning trajectories and resampling only the error step and its suffix, followed by majority voting. It matches the accuracy of standard self-consistency across seven math and multi-hop QA benchmarks while reducing token usage by up to 41.26%. The method can also be combined with adaptive inference methods like Adaptive Consistency for further efficiency gains.

arxiv14h agoScore 70

efficient inferenceLLM reasoningself-consistency

Research Editor selected

D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding

D3D is a standalone diffusion drafter that removes inter-round dependency in speculative decoding, allowing drafting of successor blocks during verification in a packed pass. It combines blockwise denoising with auxiliary autoregressive supervision and prunes layers to reduce cost. Across seven benchmarks, D3D achieves 5.00x and 5.71x speedups with Qwen3-8B and Qwen3-32B targets, outperforming DFlash.

Why it matters: This paper presents a concrete, recent AI development with measurable speedup improvements and open-source code, offering practical value for efficient LLM inference.

arxiv1d agoScore 88

Speculative DecodingDiffusion ModelsEfficient Inference

Research Editor selected

LAND: Learning an Adaptive Number of Draws for Bayesian Best-of-N Selection

LAND dynamically allocates samples in best-of-N decoding based on the expected marginal value of additional draws, learned from calibration data. It stops sampling when the expected improvement falls below a compute price, improving efficiency. In machine translation experiments, LAND achieves better quality at matched compute or requires 20-40% fewer generations at matched quality.

Why it matters: This paper presents a concrete, novel method to improve compute efficiency of best-of-N decoding, with clear experimental results, valuable for practitioners.

arxiv3d agoScore 82

decoding strategiesefficient inferencemachine translation

Research Editor selected

Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention

Zyphra released ZHA, a sparse attention method that uses a multi-level tree indexer to select relevant KV cache entries without scanning the full context. The method adds only 81,920 trainable parameters to frozen pretrained models like Qwen3.6-35B-A3B and achieves competitive or better results on reasoning, coding, and retrieval benchmarks compared to dense attention. On LongBench v2, the deep-tree variant scores 63.4% (vs. 58.0% dense) while reading only about 1.1% of keys. Projected speedups at billion-token contexts are 373–417× over dense attention, though no optimized kernels have been released yet.

Why it matters: Offers a concrete, parameter-efficient approach to subquadratic attention that retrofits existing models, with strong benchmark results and clear architectural novelty.

MarkTechPost3d agoScore 88

attention mechanismsefficient inferencelong-context models

Research Editor selected

ETAPS: Gradual Pruning with Sparse Adaptation Achieves High Quality at Extreme Sparsity

Researchers propose ETAPS, a pruning method that gradually tightens a global sparsity constraint while training a sparse delta with a small fixed budget. This approach avoids model-sized optimizer states, enabling efficient pruning of LLMs at extreme sparsity levels. Across models from 125M to 13B parameters, ETAPS achieves lower perplexity than the prior state-of-the-art ELSA at sparsities from 50% to 90%, with up to 42% perplexity reduction. It also reduces cost significantly, pruning LLaMA2-7B on a single H200 with 5x smaller persistent state and 49% fewer GPU-hours compared to ELSA.

Why it matters: This paper presents a concrete, recent advancement in LLM pruning that improves both quality and efficiency, offering practical value for deploying large models.

arxiv4d agoScore 85

model compressionlarge language modelsefficient inference