* * * * *
The magic switch
> I called another hacker over to look at it. He had never seen the switch
> before either. Closer examination revealed that the switch had only one
> wire running to it! The other end of the wire did disappear into the maze
> of wires inside the computer, but it's a basic fact of electricity that a
> switch can't do anything unless there are two wires connected to it. This
> switch had a wire connected on one side and no wire on its other side.
>
> It was clear that this switch was someone's idea of a silly joke. Convinced
> by our reasoning that the switch was inoperative, we flipped it. The
> computer instantly crashed.
>
“A Story About ‘Magic’ [1]”
The machine was incommunicado anyway.
We couldn't ping it, and it didn't respond when we plugged a keyboard into
the machine. Dan, the network engineer had been wanting to move the machine
anyway, since a colocated server shouldn't really be in the core router room
(it was there because at the time, that was the only space for it). I figured
why not? We'll be rebooting it anyway. Dan noted the phone number on the
front of the computer (not a bad idea actually) and said I should notify the
customer about the situation.
“Hello, this is Sean from—”
“I don't speak English,” said the woman's voice on the other end of the phone
number. Perfect English, a slight hint of an accent I couldn't quite place.
Instinctively I raised my voice, because, you know, an increase in volume
always brings about an increase in language comprehension. “Is there anyone
there that speaks English?”
“I don't speak English.” Click.
Um …
Right.
Smirk's on vacation for the week, and only he knows his accounting system
enough to plumb customer phone numbers from it. All I had to work with was
the one scrawled on the front of the computer, and hey, I gave it a shot.
This machine needs to be moved, then restarted. The move itself went quickly,
even though the machine was an ungainly large tower based system. Plugged in
the network cable, power, monitor and keyboard from the crash cart, and hit
the power switch. Power supply fan started, then stopped—nothing. Hit the
switch again. Power supply fan started, then stopped nothing. Went around to
the front of the machine to see what might be going on. Hit the switch. Power
light went green, the CD (Compact Disc)-ROM (Read Only Media) light flashed
briefly, as did the floppy drive. Then all went dark.
XXXX!
Checking the machine, I found a switch on the back of the machine. So for
several minutes I tried various combinations of switches. Then I unplugged
the network card. Then the keyboard. The machine finally powered up.
Whew!
Um … maybe.
The machine came up with this overly large Hewlet-Packard [2] logo and just
sat there. I wanted to nervously tap the Caps Lock key, but then remembered,
the keyboard wasn't plugged in. So I just stood there nervously tapping the
keyboard anyway. After three or so minutes I saw the familiar “Lilo” prompt,
followly quickly by Linux booting.
I then stood there watching disk partition after disk partition being
scanned. Two disks, each appeared to have seven or eight partitions. After
about ten minutes the screen blanker kicked in.
Okay, I unplugged the monitor and put the crash cart away—there wasn't much I
could do at that point and I figured it was another few minutes it would be
up and chugging away serving whatever it was it served.
A bit later the customer called, saying his machine was down. I should have
come up by now. Hooked the crash cart back up to it, keyboard still wouldn't
respond and the screen was still blanked. Nothing much else to do but reboot
it.
Okay, wait for three minutes looking at the Hewlet-Packard logo, then Linux
is booting and Oh! fsck needed to be run manually on one of the partitions
and it's asking for a root password.
I don't have a root password.
Okay, let's try booting into single user mode.
Nope, wait three minutes for the Hewlet-Packard logo, boot into single user
mode and still need a password.
Okay, let's try using a rescue CD and manually check the partition.
Yes, waited three minutes and the CD worked.
Okay, take out said CD, and reboot the machine. Wait three minutes, see Linux
boot, and the network interface failed to initialize.
XXXXXXXXXXXX!
Put the CD back in, then shut the machine down and try initializing the
network interface using the rescue CD to see if it's a hardware or software
issue.
Note, this time, instead of just resetting the machine, I power cycled.
Well, power downed. It wouldn't power back up.
And by now the customer was on the phone asking what had happened.
I told him, and asked him if there was any special procedure I needed to
follow to get the machine up and running.
He wasn't aware of anything special, other than making sure the switch in the
back was in the “On” position, and toggling the front button.
This was not good.
Customer suggested I unplug the CD-ROM. Which meant I had to open the
computer. Which meant I had to remove the top to remove the side to remove
the power from the CD-ROM. The side of the computer case had to slide all the
way out. Not just back a bit then fall to the side. Nope, it had to
sliiiiiiiiiiiiide all the way off. Not an easy thing to do while it was still
in the rack.
Still wouldn't power up.
So, remove power to the floppy drive.
Still wouldn't power up.
By this time, I had both P and Dan crowded around the machine, and Dan
noticed that the power supply for this huge system with dual harddrives and a
CPU (Central Processing Unit) the size of Rhode Island only had a 250W power
supply.
Okay. Swap in a larger power supply.
Now, mind you, I was on the phone with the customer while all this was going
on—I had a mobile phone with a headset.
Take the machine back to the rack, power up.
Hallelujah that worked!
Machine powered up, sat there at the Hewlet-Packard logo for three minutes,
booted into Linux and still refused to initialize the network interface. And
now, I realized that the CD I wanted to boot was stuck in the still unpowered
CD-ROM (the new power supply didn't have enough connectors on it to supply
power to the CD-ROM).
Next scramble, new network card, in a different slot than the old card.
Now the wait was five minutes, and it still didn't work.
Next scramble, found a network card that was the same model as the one that
was originally in the machine.
Wait five minutes, boot and hallelujah it initalized the network.
Only the machine was in the middle of the floor, with its cover off.
So the customer logged in, shut it down, so we could put the covers back on
and slap it into the rack.
Wait five minutes—
—and the network card didn't work.
At the customer suggestion, I removed the covers, took out the replacement
card, held it for a few moments, then put it back in. Powered on, waited five
minutes, saw the system boot and—
—the network card was working.
“I'm not putting the covers back on. I don't want to touch the machine,” I
said.
“I don't blame you,” said the customer.
I'm not touching that machine again.
[1]
http://catb.org/~esr/jargon/html/magic-story.html
[2]
http://www.hp.com/
Email author at
[email protected]