in case of fire...break glass

Probably one of the most stressful scenarios a technology professional will face will be a total outage of systems and the need to fail over to a disaster recovery or backup location. The pressure to get your systems back online can be the difference between making plan for the year or even hundreds of thousands of revenue dollars.

I recently noticed wall switches in our building that say, "in case of fire break glass" or "pull switch". What it boils down to is in case of an emergency event follow these simple instructions and help will be on the way immediately.

As technology professionals we sometimes feel an overly heightened sense of confidence that we will know what to do and do not need to document a process to recovery. I can attest I think i know what to do to bring 90% of my systems back online in my head, but that's thinking from a calm scenario not a stressed one.

I have been tried in the event of a disaster outage several times in my IT career and I can validate the need for a written and comprehensive plan for recovery of systems.

The best way to get started is to have a log of all your systems you would need in the event of an outage and then have a written process for the recovery of these systems individually, with instructions that can be followed by someone other than yourself.

Then prioritize the order in which these systems should be restored to best serve the business needs.

After this exercise is complete actually test your plan, the best laid plans can sometimes fail from missing the most common overlooked process or procedure.

As my scout son says, be prepared. You will be happier in the long run having a plan at the crucial time when needed and having the confidence in knowing it works.

Have a great week!