GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning
The paper identifies Correct-but-Visually-Misgrounded Reasoning (CVMR) in vision-language models, where models can reach correct final answers despite misperceiving visual information during intermediate reasoning. It pr...