Two-channel constrained GRPO calibrates VLM safety budgets
Researchers introduce two-channel constrained GRPO to enforce safety budgets on harmful assistance and over-refusal in vision-language models. The method addresses two failure modes: scorer mean bias in the Lagrange multiplier and within-group reward standardization that cancels the multiplier before it reaches the gradient. On a Qwen3-VL-2B policy, held-out harm on 1,037 adversarial prompts falls from 8.9% under standard GRPO to 4.8%, versus 7.6% for an uncalibrated budget, while over-refusal stays at most 2% on 400 benign queries and MMBench is unchanged. Gains are larger when intent is in the image and on the unseen HADES benchmark, with results holding across seeds, alternative judges and a human audit.
Why it matters: It offers a concrete constrained-RL fix for calibrating VLM safety behavior, with measured reductions in harmful assistance and limited over-refusal.