Live intelligence

Latest AI updates

1 updates
Research Editor selected

Lost in Rounding: bf16 Recurrent State Makes Hybrid Reasoning Models Think Longer

A new arXiv study finds that storing cached recurrent state in bf16 with round-to-nearest makes hybrid reasoning models more verbose and increases 32K-token cap hits. Across six open hybrid models and 590 competition-math problems, five of six reasoned 1.07x to 1.27x longer; Qwen3.5-9B accuracy fell from 85.6% to 70.8% and cap hits rose from 118 to 223. On LiveCodeBench, the two tested models lost 12.6 and 13.4 pass@1 points, and a 128K budget only halved the accuracy gap. Stochastic rounding in bf16 largely removes the effect, with an integer-hash generator matching FP32 accuracy and cap hits at bf16 memory and decoding speed.

Why it matters: It gives inference and evaluation teams a concrete numerical-precision failure mode in hybrid reasoning models plus a practical mitigation.

hybrid reasoning modelsnumerical precisionstochastic rounding