Zero-Gradient Circuit Grafting Adds Exact Arithmetic to Pretrained LLMs
The paper introduces closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained language-model weights without training examples or runtime hooks. Its compiler combines Mellin-GLU quadrature, cross-token scale cancellation, and width extension to synthesize signed multi-digit arithmetic, Boolean, Majority, and Parity circuits while preserving natural-language competence. On 6-digit arithmetic, IBM Granite 4.2 3B and Alibaba Qwen 3.5 4B reportedly reach 99.17% and 100.00% ALU-prefix accuracy, with 98%-100% mid-sentence resumption and better results than a 12.8K-sample LoRA baseline. The work also reports a block-sparse sidecar holding A100 memory at 7.89-10.65 GiB and passing all 14 pre-registered long-context Tango splits.
It presents a training-free method for making pretrained LLMs perform exact arithmetic reliably, with concrete accuracy and memory figures relevant to model editing and inference deployment.
Comments & Questions (0)
No comments yet
Be the first to comment!