Mod Archive Forums Mod Archive Forums
Advanced search  

News:

Please note: Your main modarchive.org account will not work here, you must create a forum account to post on the forums.

Pages: [1]   Go Down

Author Topic: About the major showdowns... (AI bots)  (Read 1228 times)

0 Members and 2 Guests are viewing this topic.

looper231

  • Administrator
  • Rising Star
  • *
  • Offline Offline
  • Posts: 206
  • Casual guy
    • View Profile
    • My site
About the major showdowns... (AI bots)
« on: September 15, 2025, 10:22:58 »

:breakpage:
:breakhead:

Recently, Modarchive has been experiencing major slowdowns. The frequency of this happening was not that big in the beginning, but now it reached to am almost daily thing now. At first we thought it was just a DDoS, but now it's apparent, that TMA is being targeted by AI scrapers.

So far, We've tried a number of solutions which proved to be ineffective. We're trying our best to fight this problem. I am very sorry for the inconvenience this brought over the past months.

// LooPeR231
:breakhead:
:breakpage:
Logged
"Beyond the reach of human range. A bit of evil. A touch of strange" - Paradox

GEARvision

  • New User
  • Offline Offline
  • Posts: 8
    • View Profile
Re: About the major showdowns... (AI bots)
« Reply #1 on: Yesterday at 16:54:08 »

Hi - Use goaway or Anubis
These scripts stop bots - I wish you luck

--- Details ---

This is the official message from this script

Quote
Why am I seeing this?
You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.

Anubis is a compromise. Anubis uses a Proof-of-Work scheme in the vein of Hashcash, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.

Ultimately, this is a hack whose real purpose is to give a "good enough" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.

Please note that Anubis requires the use of modern JavaScript features that plugins like JShelter will disable. Please disable JShelter or other such plugins for this domain.




Protected by Anubis from Techaro. Made with ❤️ in 🇨🇦.
Link to Anubis
Proof-Of-Work

You can see, this script does as little effort to stop a whole flood of bots.
Here are several pages - under the following lines - that employ this script.
FFMPEG's ultra powerful multimedia utility that I personally use - I made my animated Avatar with this!
Another example, using an external YouTube video player - It uses goaway - Useful to save on bandwidth, but bots attacked it often too. Not anymore - It uses a variant that does not use modern JavaScript. Cool! This is the only variant I can run in Internet Explorer. Not that you should follow old browsers.
Logged
My all time games & software are UT +mods & Smuggler's Run 2 by Rockstar; on the other side FFmpeg (CLI multimedia editor) & Zstandard (add-on for 7zip)
Aka RICOTTA_19216

looper231

  • Administrator
  • Rising Star
  • *
  • Offline Offline
  • Posts: 206
  • Casual guy
    • View Profile
    • My site
Re: About the major showdowns... (AI bots)
« Reply #2 on: Yesterday at 17:07:33 »

To briefly periphrase an explanation of Saga Musix: This is simply not possible to implement, because of the way TMA operates on the server machine.
Logged
"Beyond the reach of human range. A bit of evil. A touch of strange" - Paradox

GEARvision

  • New User
  • Offline Offline
  • Posts: 8
    • View Profile
Re: About the major showdowns... (AI bots)
« Reply #3 on: Yesterday at 17:30:02 »

Glad to see a response as quick as you made it!

Looks like limitations exist on the implementation... :'(

I saw that sometimes either one of the two work - It is always good to try ^^
Logged
My all time games & software are UT +mods & Smuggler's Run 2 by Rockstar; on the other side FFmpeg (CLI multimedia editor) & Zstandard (add-on for 7zip)
Aka RICOTTA_19216
Pages: [1]   Go Up