LLM scrapers are taking down FOSS projects' infrastructure, and it's getting worse.

LLM scrapers are taking down FOSS projects’ infrastructure, and it’s getting worse.

  • refalo@programming.dev
    2·
    2 days ago

    What you’re doing is filtering out bots that can’t be bothered to execute JavaScript. You don’t need to do a computational heavy PoW task to do that.

    Most bots and scrapers from what I’ve seen already are using (headless) full browsers, and hence are executing javascript, so I think anything that slows them down or increases their cost can reduce the traffic they bring.

    Canvas fingerprinting filters out bots better than PoW

    Source? I strongly disagree, and it’s not hard to change your browser characteristics to get a new canvas fingerprint every time, some browsers like firefox even have built-in options for it.

    • YetiSkotch@ieji.de
      1·
      2 days ago

      @refalo @sudo If Proof of Work gets widely adopted I foresee a future where bot running data-centers can out-compute humans to visit sites, while old devices of users in poorer countries struggle to compute the required task for hours … Or is that fear misguided?