Back to AI News
arxiv

Transparency Disclosures for Chain-of-Thought Monitorability

A new arXiv paper argues that frontier AI developers should report specific indicators to track threats to chain-of-thought (CoT) monitoring and support coordination. It proposes five indicators spanning third-party CoT access and base architecture specifications, CoT quality (naturalness and faithfulness), and monitoring effectiveness (detection and uplift). An audit of disclosures from Anthropic, OpenAI, and Google DeepMind found no developer covers the full set, though each indicator has been reported by at least one. The authors recommend a cross-developer audit with deeper access, via embedded evaluators or an Independent Verification Organization.

AI Brief

It gives AI safety and policy readers a concrete disclosure framework and a comparative audit of current frontier-lab practices for CoT monitorability.

arXiv:2611.13873v1 Announce Type: cross Abstract: Chain-of-thought (CoT) monitoring is one of the most valuable tools for monitoring advanced AI systems, but it is under threat. We argue that frontier AI developers should report specific indicators to make these threats trackable and to better facilitate coordination to preserve CoT monitorability. We identify five indicators that are both highly relevant and reasonably feasible to disclose, one for each link in a five-step chain on which CoT monitoring depends. The first, third-party CoT access and base architecture specifications, also underpins external verification of the rest. The others span CoT quality (naturalness and faithfulness) and monitoring effectiveness (detection and uplift). An audit of current disclosures by Anthropic, OpenAI, and Google DeepMind shows that no developer covers the full set, but that every indicator has been reported by at least one frontier developer. Because current disclosures rely largely on developers' own evaluations, we also recommend that a cross-developer audit be repeated with deeper access than public disclosures allow, either by embedded evaluators working within developers or by an Independent Verification Organization.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!