Power outage at data centre postpones hospital surgeries in Oregon

What happened

On 7 February 2019, OneNeck's The Vault data centre in Bend, Oregon had a power failure of both its main and backup power systems, a company spokesperson told the Bend Bulletin, as reported by Data Center Dynamics. The report describes a 45-minute fault. Services for most customers were restored within 45 minutes to one hour, and local provider BendBroadband was back within 20 minutes. St Charles Health System saw a network outage at 7:30am, used downtime procedures and postponed elective surgeries, which resumed when its systems came back up at 2pm.

OneNeck said it would look into why the redundant system failed and gave no further cause. The site is described as Tier III and LEED certified, with a flywheel UPS, two 1.2MW generators and a 750kW critical load.

Contributing factors

The operator reported that both main and backup power failed. Why the redundant system failed has not been reported. A flywheel UPS and on-site generators are described for the site, but the source does not link them to the failure.

Lessons learned

1. Main and backup power at a data centre can fail together, so dependent operators still need their own downtime procedures.
2. Recovery can differ by customer: some services returned within minutes while a hospital system stayed down for several hours.
3. After a dual supply failure, the operator should investigate common-mode causes and tell customers what is known and what is not.

Source

datacenterdynamics.com