Beyond block or allow: How pay-per-crawl is reshaping public data monetization

For most of the web's history, content platforms operated on a simple binary: open or blocked. Then generative AI changed everything.