August 18, 2004
Server Admin Guy
Here in this inaugural blog on the newly-created category “Server Admin Guy”–the category that charts the ups and downs of administering a Debian Linux server (this one at St. Odilo School)–I write a happy message to the St. Odilo Technology Committee (Techcom). I give a final update to a week-long process in bringing back network services, after a major SNAFU by SBC-Ameritech (they obliterated the longstanding account’s static IP numbers).
To: Techcom (Saint Odilo Technology Committee)
SUBJECT: Final (??) Crisis Update
Less than a week after we lost our IP numbers (but not
much less), our system is back and just about fully operational.
There were many hurdles:
- Figuring out the problem was not our equipment: MAJOR HASSLE, with these highlights: swapping our our server, router, cables, re-configuring bunches of things. Thanks, Bill Donegan, for carrying the backup server from the rectory to the school.
- Keeping our old IP Numbers: IMPOSSIBLE.
- Getting new IP numbers: NOT TOO DIFFICULT: Thanks, Frank, of SBC for staying on at work two hours after normal quitting time Friday night to re-configure our account.
- Getting new IP numbers to work: ALMOST IMPOSSIBLE: Thanks Al, for your weekend intervention to resolve some very mystifying account tie-ups and finding us the person (Guy) who knew exactly what to do to get our new numbers functioning. Al’s help cut at least two days off our network downtime.
- Getting our new IP numbers propagated throughout the worldwide DNS database: VERY FRUSTRATING and MYSTIFYING, as all indications were that the new info was being distributed, but apparently (after nearly three days of no change) was not. Thanks Ed, for directing me to the place we needed to go to get things fixed: Our domain name account with Network Solutions. Unfortunately, there is some kind of problem with our DNS server, such that it doesn’t effectively send updates to all the world’s DNS servers. In fact, my current theory (as validated by some googling) is that the notification our server is sending is correct, but the servers of the world aren’t understanding it for some reason. (“I’m not wrong, everyone else is!”) But whatever–Ed, I found at our Network Solutions account a way to use their DNS server to propagate new records. So I sent that through, and the needed changes pointing people to our new numbers started appearing within a few hours. It’s not a solution that makes me perfectly happy (we really shouldn’t have to rely on Network Solution’s DNS server to propagate our change), but it’s perfectly effective. (Other googling, btw, inclines me to think that maybe our change would have gone through once our old records “expired” on the servers of the world–something that would have happened after seven days.)
Thanks Paul, for dropping by the house yesterday, as you were girding yourself for phone battle with SBC to ask them for all kinds of restitution. In the old days, it was called Blood Money. I think we’d settle for some credit or free upgrades. Did you get any? (If not, we’ll settle for the blood, right?) Thanks for your moral support over the weekend during your vacation.
Whew. Bye… –Angelo
P.S.: Pardon this Anti-Olympic Moment, but may I avoid the fate of Phidippides who gave birth to the Olympic Movement by dropping dead after delivering his joyous message, delayed by a 26-mile run from Marathon to Athens: “We win!” [plop!]