Sirius-Tiamat: When Compression Flips a 421M Decision Encoder's Answers
The Sirius-Tiamat study compresses Laya, a 421M-parameter one-pass decision encoder that returns option probabilities in a single forward pass, and measures how often compressed variants disagree with the full-precision fp32 model while tracking expected calibration error. On 1,221 in-family synthetic items, bf16 changes 0.2% of decisions but 2.2% of 800 out-of-family ARC, MMLU and HellaSwag questions; 6-bit grouped rounding changes 1.3% and 5.9%, and deleting six layers changes 0.9% and 9.1% while ECE rises from 5.9% to 39.2%. A single fitted temperature requiring no labels restores ECE to 3.6%, and at equal stored size rounding flips fewer decisions than layer deletion. CPU dynamic int8 changes 30% of decisions almost entirely through one layer, and distilling a 16-layer student cuts in-family flips from 44% to 9%, though the authors caution the data are synthetic, LLM-labelled and cover one architecture.
Gives deployment teams concrete per-method flip and calibration numbers showing that quantization and pruning degrade out-of-family decisions far more than in-family ones.
Comments & Questions (0)
No comments yet
Be the first to comment!