How DCs should plan in a pandemic
Published 12 March 2021
With the Covid-19 pandemic, disaster preparedness has become the ‘New Normal’.
Lockdown efforts, which have been seen as essential in many countries, to safeguard citizen’s lives, seems to be the way forward. In line with this, the first and an essential step for management in a data centre, is to develop a specific pandemic preparedness and response plan.
“Share the pandemic plan with all employees, stakeholders, vendors, suppliers and key customers. Establish a system whereby elements of the plan are tested, updated and changes disseminated on a regular basis,” pointed out Hideaki Fujimaki, CEO, Spiralgroup.biz Ltd.
Efforts need to be ongoing
The pandemic has also forced organisations to address gaps in the preparedness and recovery plan on an ongoing basis, rather than a one-time effort. “The pandemic plan should incorporate a tiered response, clearly identifying the actions to be taken at each level and the circumstances that would trigger implementation of the next level,” stated Fujimaki.
Most organizations have a three to five-level contingency plan, ranging from pre-pandemic) operations, to taking reasonable precautions, through lights-out operation and, in worst cases, a complete site shutdown with transfer of critical applications and operations to backup sites.
“The plan should be practised or role-played if possible. At every level, the plan should clearly identify the trigger(s) to implement that level, the decision-makers authorized to direct escalation to that level and the appropriate actions for operations,” said Fujimaki.
This should include policies for facility access, on-site activities, staffing and sanitization.
Also, in such a scenario, with IT assets being critical, maximum acceptable downtime, reduction in redundancy and/or recovery time for all equipment, disruption or failure response procedures, minimum acceptable staffing levels, should be factored in. Additionally, staff protection by using temperature checks, contact tracing, reporting of symptoms, site access, minimum acceptable levels of critical on-site activities, such as equipment maintenance have to be in place, noted Fujimaki.
A tiered-response plan should include plans to meet the challenges of operating with reduced staff, including situations in which staff may be unable to access the site or may need to leave the site on short notice. It should include a staffing threat matrix for various scenarios of employee absenteeism, according to Fujimaki.
There is also a need to look at potential alternatives. Where practical, organisations should include provisions for the use of third-party staff, as a contingency measure.
“Recognise that any change from normal processes can increase the risk of human error or extend response times in case of emergency,” said Fujimaki. The plan should make provisions for a multi-peak/wave pandemic, taking into account a second wave, possibly only weeks after the first and possibly worse, when supplies and finances are depleted, staff is fatigued, and maintenance has been deferred. “Multiple waves/seasonal re-occurrences may also be likely. Long-term contingencies, which could include vaccine unavailability, critical supplier going out of business, should be considered and planned for as well.
Protecting the Business
Management should confer with insurance companies and legal advisors on relevant items, such as cleaning requirements, service level agreements (SLAs), notifications, etc. For data centers in areas where there is no clear regulatory mandate, management should decide — in consultation with insurance companies, legal advisors, Human Resources (HR) departments and other business unit(s) — at which response level to institute certain response measures, noted Fujimaki.
Some data centres are officially considered to be part of the critical national infrastructure. While this may confer some advantages, such as priority access to fuel, it may also mean that plans need to be shared with and agreed to by overseeing authorities. As part of the strategic plan development, clarify the status of key data center workers (whether they are classed as essential workers) and of the data center (whether it is deemed part of the nation’s critical infrastructure). Determine precisely what these terms means, as it could differ in every country and what documentation is needed for situations involving staff travel, shortage of fuel, amongst others, said Fujimaki.
Even with the best planning and communication, a pandemic is likely to have an impact on a data centre’s operations’ budget. The Executive management will need to assess the situation and prepare accordingly.
Government support is available in many countries. Beyond that, as with the case with other abnormal events (e.g., equipment failure or severe weather event), management typically takes the reasonable approach of instructing operations team to spend what is necessary to protect staff and the data center infrastructure, keeping track of the costs. Justifications of expenditures should be examined as a part of an ongoing review process, points out Fujimaki.
A pandemic presents challenges for data center construction, major upgrades or extensions of capacity. Construction speed has a big impact on cost and delays in one area is bound to impact other areas and a range of suppliers. The impact of all potential disruptions, from the availability of staff to a shortage of construction material, must be assessed and weighed.
Managing supply chains
Confer with suppliers to understand the risks — current and potential — for disruptions, including possible long-term disruptions, beginning with critical spares and consumables. Understand the geographic regions where key components are sourced or manufactured, and what the available alternatives are if supply chains are disrupted, opined Fujimaki.
Also, one of the key factors which nobody is paying attention to has to do with regard to certain countries with aging population. In some geographies means that despite best efforts, the data centre industry may be more vulnerable than other industries to a pandemic. “This presents a challenge, given the existing and well-documented staffing shortages the industry faces. A more age-diverse workforce may prove more resilient,” stated Fujimaki.
What the pandemic has taught organisations is that disaster preparedness should be no longer thought of as a contingency plan but a proactive one.