* * * * *
Bad robot! Bad! Bad!
The load on tower shot up, making it quite unresponsive. A few minutes later
it settled down enough for me to check the system out.
I would think that programmers who know enough to check for robots.txt [1]
would be smart enough to limit the number of concurrent connections to any
one website. But not for something called “QuepasaCreep” which slammed this
site pulling down the entire archive in about a five minute span (typically,
about two or three hits per second).
It seems to have passed; I can only hope they realized just how poorly their
robot was behaving.
[1]
http://www.robotstxt.org/wc/norobots.html
Email author at
[email protected]
