AI
Four AI labs back one robots.txt switch for training opt-outs
Google, OpenAI, Microsoft and Anthropic backed a shared robots.txt policy token, __usage_ai-training, that lets a site state one AI-training preference for every participating provider instead of naming each crawler. OpenAI aims to implement it in the coming months, Google plans to read an opt-out as a Google-Extended disallow, and Microsoft will honour it from 1 February 2027. The draft, written by an Anthropic engineer, adds no new grammar and does not limit crawling.