A little postscript to my last post,
2025-06-07My_Session_with_the_Bots.txt, written before I forget the
details completely. Before I ended with an Apache MaxRequestWorkers
setting of 350, since 450 caused the server's 1GB of RAM to run out
before queuing connections to the PHP script which was getting
DDoS'ed for some reason. Of course having written that I soon
discovered my server bogged down again, stuck on that Apache
process limit. Gradually raising it, I found that the RAM
requirements just to serve the 503 error response to the
now-blocked requests containing PHPSESSID were less than I thought
before. I was able to up the process limit to 950 and actual
simultaneous Apache processes topped out around 800 while logs
showed hundreds of requests from random Brazilian/SE-Asian IPs
being denied per second.
This is obviously a limitation of Apache's configuration options,
because before blocking the requests based on that pointless
PHPSESSID query string the Apache processes used much more RAM to
serve the dynamic PHP page rather than just a few headers with an
error response (the bots didn't retrieve the error page). What I
really need is a way to automatically scale the process limit based
on available RAM and average RAM usage per Apache process, but
strangely that doesn't seem to be available.
It seems to me that it should be possible to write a script to do
this by editing the MaxRequestWorkers setting and reload Apache.
But testing it would be time consuming and at this point my tally
of jobs to do Vs time to do them is already hopeless. Plus any new
job that doesn't immediately serve a profit motive ought to be
excluded given my current situation, and so long as my website
works now, improvements ought to first and foremost be to
making/finding products to sell there that people want to buy.
I haven't got Apache's caching module enabled, something I'm
considering now, but presumably the different PHPSESSID values
would have defeated that anyway. Anyway, the attack slowly died
away in request volume and then went back again to the old noise of
one to three AmazonBot hits per second.
By the way isn't it odd that in order to provide a website that few
real people actually want to look at, one now has to cater for
demand equivalent to it being viewed my millions of people a day.
How is the internet even still working when this sort of load is
being placed on it? Yes other people like to block large IP ranges
rather than trying to absorb the bots, or use one of the
bot-blocker services that are suddenly blocking me everywhere now,
but that's really just breaking the internet even more directly. As
one of the comparatively few real humans online, I demand not to
have every other website accuse me of being a bot unless I run
Firefox with lots of random scripts allowed through NoScript.
Scripts from third-party services who doubtless have
side-businesses of collecting and selling data on this real living
human that they caught out there in the sea of bots.
At least it is kind-of cool that the cheapest VPS I found could
serve millions of dynamic webpage requests per day from real
humans, if they weren't all accompanied by a larger swarm of hungry
bots. The lightweight code and small size of content on my site
makes it super fast even though Apache (chosen because I didn't
expect to be dealing with such loads) isn't the best choice for
performance. Imagine how low-spec a system just to serve human
requests could be? A Rapberry Pi Zero would be huge overkill. But
then it's that sort of cheap modern processing power that allows
people to run these DDoS/scraper bots, so it's all a vicious cycle,
and I only won this time because this attacker/scraper was so
idiotic as to just hit one PHP script with a pointless query string.
But all this broad thinking is obviously where I'm going wrong in
life. Stuff the whys and wheres, I need to make money somehow. Hmm,
what if I made a huge bot farm running half-arsed code to scrape
all the websites on the internet to death and feed it onto some AI
model I can sell to people making their own half-arsed AI junk?
Yeah, great idea, that'd put food on the table.
