Skip to content

Blog

Notes from a fact-driven publisher.

Perspectives on fact-driven publishing, search, and how data-driven content earns citations from AI answer engines.

Five small crawler tokens stopped at a concrete wall marked with orange X marks, while one orange lane passes through a gap to a row of documents

Publishing in the Age of AI Josh McWilliam

Should Publishers Block AI Crawlers? What the robots.txt Data Says About the Trade

79% of top news sites block AI training bots — but 71% also block the retrieval bots that decide who gets cited, and a growing share of AI requests ignore robots.txt anyway. We separate training crawlers from answer crawlers, tally who blocks what, and lay out the actual trade: a licensing negotiation you may never get, versus a citation you can measure.

Read more →

A stone flywheel ringed by four stacks of cards and orange arrows, turning beside a rising orange bar chart

Audience Development Josh McWilliam

Rankings Are a Flywheel: How Share-Worthy Data Replaces Link Building

Two-thirds of web pages have no backlinks, and referring domains remain the strongest ranking correlate there is. Rankings solve both problems at once: the ranked have a reason to share, the local press has a reason to write, and each release compounds the last. The mechanics, the methodology rules that make rankings defensible, and what one release looks like at scale.

Read more →

Put the engine behind your brand.

See how we partner with brands and media companies.