On a chilly spring afternoon, IBM’s western regional manager invited us to bid on a most challenging project: “Due to new earthquake guidelines we need to upgrade our most critical data center floor, panels and struts… a shutdown will cost millions.”
I was nonplussed, figuring that a dozen or so racks was no real challenge. We have often engineered complex logistical solutions, with secure mirrors and rapid deployments. But, when Scott told me that the center was comprised of over 400 fully stocked IBM, EMC, Dell, and HP enclosures he had my full attention.
Failure Not an Option
He explained that these enclosures managed both the active electrical grid and customer service for a few million customers in multiple western states. A failure would most likely result in rolling blackouts, lawsuits, and very angry customers.
He was putting the project out for bid, and he had come to us on the recommendation of a large ISP whose operations we had relocated a few months ago. We won the bid after some major internecine battles, which culminated with a 76 person conference call between 75 sharp stakeholders and yours truly.
We considered many options. Traditional relocation and virtual accommodation methods were impractical due to both the politics and key operational concerns. Jointly we decided on an innovative vertical solution.
Logistical Challenges
Vertical methods, wherein the equipment would be lifted off the raised floor panels by a minimum of 2 feet, and held for 2 hours, was needless to say controversial. However, we quickly discovered that the lifting of the equipment was only a small portion of our challenge.
I recalled the sage advice that “all tasks are fundamentally simple… it’s the quality of solutions to the inevitable sharp problems that occur, that is…