Being a sysadmin

My sleep is seldom affected by being one of a few people at my company who spends part of his time doing system administration, but this week has been a sure exception. We moved our whole public server infrastructure to a new section of our data center (complete with new IP addresses and routing), implemented load balancing of two separate clusters of web front-end machines, migrated two database servers to new hardware, and set up database replication for our web-facing databases. And we did it in sort of a last-minute, pre-product-launch scramble with what shoestring planning we could cobble together, while working on other high-priority projects and with very limited down time and, as far as I can tell, very little in the way of experience among our staff with implementing any of these things in a production environment. I’m not sure it could have gone more smoothly had we planned it for three months. It’s inexplicable, really. Of course, helping to make all this happen necessitated my putting in long hours over the weekend and waking up at times like 1:00 a.m. or 4:30 a.m. before or after an otherwise full workday to minimize the impact of down time. We coordinated this with sysadmins in Germany, California, and Tennessee and a data center in Texas. With my dad coming into town this weekend and a pumpkin-carving planned for tonight, I aim to take off around lunchtime (having started work at 4:30 this morning after staying up late to watch the Red Sox take the second game of the World Series) unless somebody threatens to fire me for doing so.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s