Latest AI updates

Logprob Reduction Fails to Verify Causal PII Suppression in LLMs

This paper shows that SAE-based feature interventions for suppressing PII leakage in LLMs, when evaluated only by log-probability reduction, often produce false positives. Across 36 cases, 47% appeared successful under logprob-only evaluation, but only 2.8% passed a multimetric framework with random direction controls. The authors identify two failure modes and propose causally grounded PII suppression via circuit-level localization and causal mediation analysis.

Why it matters: Provides concrete evidence that a common evaluation metric is unreliable)Skip and proposes a more rigorous methodology, directly impacting AI safety research.