Live intelligence

Latest AI updates

3 updates
Research

Protein Language Models Fail Amino-Acid Relabeling Invariance Test

A new arXiv paper introduces a diagnostic framework that globally relabels the amino-acid alphabet while preserving alignment-level information, column entropies, and pairwise edit distances. Classical pseudolikelihood Potts models and inverse-covariance estimators remain invariant under this transformation, but protein language models do not. Across ESM-2, MSA Transformer, and AMPLIFY, and scales from 8M to 3B parameters, pLM contact maps collapse under arbitrary relabeling. The authors attribute this to reliance on canonical residue identities, or "alphabet anchoring," rather than pure coevolutionary statistics.

protein language modelscontact map predictionmodel robustness
Research

Study Stress-Tests Time-Series Foundation Models for Sampling Frequency Invariance

A new arXiv paper evaluates four prominent time-series foundation models—MOMENT, Chronos-2, Moirai-MoE, and TimesFM—under changes in sensor sampling frequency. It introduces three metrics, FSI, FDI, and CDI, to quantify embedding shifts, forecasting degradation, and downstream performance degradation. The study examines whether model behavior tracks lost information bandwidth rather than the sampling frequency value itself. It characterizes how invariance depends on the margin between native sampling frequency and task-required bandwidth, and how much TSFM choice and information loss explain performance changes.

Time-Series Foundation ModelsSampling Frequency InvarianceSensor Data
Research

Maya a1: Do the Options Read the Model, or Does Their Position? Order Sensitivity in a Non-Autoregressive Decision Model

Researchers re-engineered a 421M-parameter non-autoregressive multiple-choice checkpoint and tested whether its answer depends on where an option is listed. Across 951 Qwen-verified questions under cyclic rotations, the released checkpoint changed answers for 25.8% of questions, or 19.7% on the 605 English-language items, with a 10.1-point accuracy gap between gold-first and gold-last placement. Fine-tuning with option shuffling reduced the flip rate to 7.2%, versus 11.4% for identical rows without shuffling, while a confidence-weighted vote over orderings reached 69.7% versus 69.1% for a single pass at 4.1x compute. Fine-tuning with the authors' parallel layout on the same rows produced a 0.5% flip rate.

model robustnessmultiple-choice question answeringnon-autoregressive models