Industry
Google and Microsoft join OpenAI's new website crawling standard for opting out of AI training
OpenAI has proposed a new website crawling standard that allows sites to opt out of AI training via a single robots.txt line. Google, Microsoft, and Anthropic are among the companies signing up to the plan. The standard only prevents training, not crawling, so bots may still visit sites for other purposes.
AI data governanceweb standardscontent opt-out