minus-squareskvlp@lemmy.wtftoTechnology@lemmy.ml•Top 10 website in the world ( July 2025 )linkfedilinkarrow-up3·27 days agoIt seems there are ways to stop crawlers. Do a web search for “stop ai crawlers” or similar to learn more. I hope it doesn’t escalate into an arms race, but I realise I might be disappointed. linkfedilink
It seems there are ways to stop crawlers. Do a web search for “stop ai crawlers” or similar to learn more. I hope it doesn’t escalate into an arms race, but I realise I might be disappointed.