ISP troubles

The website was offline for a couple of days. Here's how things went down:

My ISP was switching upstream providers, and as a result, required IP address changes on all its machines. The changeover happened at 10pm EST Sunday night, and I altered my name server entries around that time, in anticipation of the change.

However, the new provider had some routing issues, and the machines were not actually online or accessible until Monday at noon. I hope someone got in trouble for that. One of our machines came back online right away at that point, but my webserver failed to come back up. I could ping it, but no SSH or webserver access. I guessed there were problems with the services or maybe a firewall issue. We had a person on-site to reboot the machine.

Except he rebooted the wrong machine. Now, I know some people might not know the difference between a 2-U and a 4-U machine, so I mentioned that the machine I wanted rebooting was about a handspan tall, when viewing from the front of the rack. That wasn't an apt description, apparently, as the working server went offline for a few minutes.

By 3pm Tuesday, a sysadmin was available to fix the machine; the same guy who changed the IPs in the first place. It was indeed a firewall issue, and he wasn't aware I was using a firewall (which begs the question: what was he using?).

So the machine's back up, and I'm just hammering out some of the bugs in DNS.