Web Security

Cloudflare ties AI crawler choices to robots.txt Cloudflare has announced Bot Preference Sync, a feature that updates robots.txt based on a site owner’s AI bot settings for Search, Agent and Training traffic. The company says the tool is intended to keep published crawler preferences aligned...

Cloudflare refines certificate alerts for domain owners Cloudflare has made Certificate Transparency Monitoring generally available after changing how the service decides which certificate events deserve an email alert. The main update is a filter that suppresses certificates Cloudflare issued...

Cloudflare reframes bot defense for agentic traffic Cloudflare says web security teams need to judge automated traffic by behavior over time, not by a single request or challenge. The company is positioning its bot tools around continuous Trust evaluation as human sessions increasingly mix with...