PEARL: A Durable Method for Suppressing Misaligned Behaviors in Language Models
This paper introduces PEARL, a GRPO-based reinforcement learning variant that suppresses undesired model behaviors more durably than standard methods. It tests on two deliberately misaligned model organisms with narrow and broad triggers, showing strong immediate suppression, preserved task accuracy, and durability under further fine-tuning. The method is most beneficial for broad misalignments that resist targeted SFT.
Why it matters: Provides a concrete, novel method with empirical results on durability, a key challenge in AI safety.