Fact: Something is broken – you need to get to work.

1.  Stay calm.

Remaining calm is the best strategy to surviving a system outage and keeping your focus on the current task.  Also, remaining calm will produce less emotional and techinical errors in a workplace.  

2.  Notify all users.

Proactively preparing for a system outage would include creating a Notification Plan based on different scenarios.  Using company e-mail, instant messaging, have a Co-Worker announce the outage or put a sign up saying “System is down.  Working on it now!”.

3.  Recruit a “buffer” to answer any questions.

After the initial notification, users will have many questions.  It would be ideal to recruit someone who is available to answers those questions, especially ETA questions.

4.  Be polite and understanding.

Remember to be polite and understanding when it comes to upset users.  In Step 1, keeping your focus on the current task to avoid making errors, also try to avoid “play-by-play” status updates.

5.  Documentation.

Always document each step of the recovery process.  Start with the initial problem or outage, followed by what you find, what commands you used, what errors you received and all attempts to fix the outage.  Lastly, document the steps that resolved the outage.

6.  Present the facts.

During the resolution process and the aftermath, always remain honest and present the facts.  If you caused the problem or outage, do not try to cover it up because it will complicate the process of resolving the outage.

7. Take your time.

Although, time is of the essense when system is down, take your time while troubleshooting.  If you are not sure what will resolve the problem, taking your time and implementing different solutions at a steady pace will allow you to narrow down the exact problem and resolution.

8.  Call Support.

Having product support is an essential tool for IT departments.  Calling support and submitting a ticket may consist of being on hold, waiting for a call back and sometimes an onsite visit can be a smart business decision.

9.  Stay positive.

Staying positive and continue building a knowledge of system events will prepare you for problem situations.  Having strong work relationships with co-workers will be useful for emergency situations.

10.  Be Prepared.

To be prepared for every system problem is almost impossible.  Continue building a knowledge of your environment is the strongest way to prepare for problem situations, previously mentioned in Step 9.