* * * * *
The Better-Half of my Day Yesterday
Gah.
At least today is turning out to be a much better day than yesterday.
Yesterday, I had the distinct pleasure of learning just how braindead a
Riverstone Networks RS 3000 switch/router can be (and that is a post unto
itself [1]).
I also had the distinct pleasure of renumbering (assigning new IP (Internet
Protocol) addresses) a thousand sites (literally—I ended up renumbering 1,200
websites), a process that didn't go quite as smoothly as I had wished. This
mess started about two weeks ago when R (I manage a few servers for him)
informed me that his largest customer (who has the aformentioned 1,200
websites) wanted distinct IP addresses for each one. That meant I first had
to secure enough IP addresses and get them routed to the server, which took
most of the time.
Meanwhile, I was sent a list of sites from the customer, which was smaller
(by about oh … 300 or so) than the actual list of sites on the webserver. So
now I had to reconsile both my list and the customer list. I ended up with
five lists:
1. sites that were in both lists
2. sites that I need to add
3. sites that I had, but the customer didn't list
4. sites that had expired (domain registration expired)
5. sites that were no longer hosted with us (found via DNS (Domain Name
Service) queries)
Then there was the back-and-forth exchange with the customer that boiled down
to: keep all sites (even expired and those that no longer point to the
server) and give as many sites their own IP address as possible (I ended up
giving the first two lists their own IP, with the last three lists sharing a
few addresses).
Then last night was the Great Renumbering.
One thousand (plus) sites. Five lists. Making sure I keep a record of which
site gets which address. And yes, it was rough. I started at 2:00 am
(technically very early this morning) and ended at around 3:30 am, spending
the hour and a half running custom Perl scripts, generating DNS zone files
and an Apache configuration file [2]. Messed up a few times—mixing up two
lists, forgetting to update the serial number in a zone file, simple mistakes
like that.
Then at 7:30 am I get a call from the customer. A couple of domains seem to
be down. Stumble over to the computer, log into the server, and find out that
I forgot to run one of the lists (no wonder the new configuration file seemed
a bit small). Oh, and DNS for a few domains was borked (about six zones out
of 1,200 had some custom records). I was surprised at how few sites got
broken during the process (and given that the customer was confused about
what sites where active, it's even more amazing).
But man, am I tirezzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
[1]
gopher://gopher.conman.org/0Phlog:2005/10/11.2
[2]
http://httpd.apache.org/docs/1.3/mod/directives.html
Email author at
[email protected]