4-Step Recovery Checklist for Uninterruptible Power Supplies and other IT Equipment after an Outage

Power outages have become more frequent and severe in recent years.  Communities in Texas and throughout the southern U.S. are recovering after the largest forced blackout and power grid failure in history.  As an IT professional, you may be well-versed in the ways to prepare and protect IT equipment — uninterruptible power supplies, generators, etc. — in the event of a power outage. (If you need a refresher, this blog post is a good place to start.)

But what about after the power comes back on? What are the questions you need to ask to evaluate your IT infrastructure during the recovery stage?
In this post, I outline four questions you need to ask as you assess your IT equipment after a power outage.

IT equipment recovery checklist   
      
1. Did I plan accordingly?

Even the most advanced facility cannot guarantee 100 percent availability of its IT infrastructure — there’s always a risk of an outage. One of the best ways to ensure a fast and effective response when an outage does happen is to have an emergency plan in place.

No plan?  If you didn’t have a plan in place, it’s time to create one. This paper: “How to Prepare and Respond to Data Center Emergencies, ” is a helpful resource. It includes detailed information on the essential elements to include in your plan, including emergency response procedures, emergency drills, and incident management information. Keep in mind, this document is specific to data centers, but you can apply it to any facility with critical IT infrastructure.

Have a plan?  Great.  Take a look back at your emergency plan and determine if it met your needs. The biggest question to answer is: Did everything work as intended? If everything worked as planned, meaning there was no asset / facility damage or data loss, the right people mobilized quickly, and you wouldn’t do anything differently, then kudos to you.

But due to the nature of power outages, emergency responses aren’t always seamless. If you noticed some areas for improvement, drill down further into the problem by moving onto the next question.

2. Did the system in place work as expected?

Your emergency plan likely includes many elements to keep your IT infrastructure protected during an outage. For example, it may detail what to do during a utility or transformer failure or how to troubleshoot standby generators and uninterruptible power supplies.
Because you’ve determined that there were areas for improvement with your emergency plan, get down to the root of the issues. Common problems that impact IT infrastructure during and after an outage involve circuits, backup power, cooling, and software — so start there.

• Were the right circuits and devices protected?
• Was the uninterruptible power supply runtime adequate?
• Did my backup generators turn on?
• Did my software safely shut down critical systems before the uninterruptible power supply ran out of battery?
• Were all preventative maintenance procedures for the equipment (uninterruptible power supply, generators, etc.) up to date?

A “no” answer to any of these questions warrants a deeper dive into the problem. For example, if the uninterruptible power supply did not run long enough, you may need to work with Brisk Worldwide to replace older batteries or swap your battery cabinet with a larger one to increase runtime.

3. Was there damage? 

Power outages often result from natural disasters. During these events, high winds, water, and fire, along with seemingly minor environmental changes like a temperature fluctuation, can damage IT equipment and cause downtime.

At this stage in the recovery process, identify the location and scope of the damage. And as you or a service provider correct breaks or issues, also remember to address the root cause. Let’s say a server was damaged from a voltage spike.  In that case, you may want to look at enhanced surge protection solutions.

4. What would I do differently next time?

Wrap up your checklist by gauging your overall feeling about the power outage response. If you felt comfortable with your plan and execution, then note that in your plan, detail any lessons learned, and rest assured that you’re ready for the next outage.

But if you note any glaring areas for improvement, it’s time to revisit your power outage strategy. As you put together a revamped plan, consider both clear misses and near misses. And after the new version is complete, perform emergency drills to evaluate and fine-tune the process.

Additional support for IT infrastructure

Power outages are a given. But a swift recovery isn’t. Following the four-step checklist outlined in this blog will help you assess your equipment and plan, and give you direction on how to best prepare for the next outage.

Do you need additional support?  We can help.  Click here to explore more power outage resources.
This NEMA report outlines standards for evaluating, replacing, and / or reconditioning water-damaged electrical equipment.