Protein Language Models Fail Amino-Acid Relabeling Invariance Test
A new arXiv paper introduces a diagnostic framework that globally relabels the amino-acid alphabet while preserving alignment-level information, column entropies, and pairwise edit distances. Classical pseudolikelihood Potts models and inverse-covariance estimators remain invariant under this transformation, but protein language models do not. Across ESM-2, MSA Transformer, and AMPLIFY, and scales from 8M to 3B parameters, pLM contact maps collapse under arbitrary relabeling. The authors attribute this to reliance on canonical residue identities, or "alphabet anchoring," rather than pure coevolutionary statistics.
It exposes a concrete representational bias in protein language models and cautions against assuming they learn the same invariant coevolutionary signals as classical methods.
Comments & Questions (0)
No comments yet
Be the first to comment!