Power outage at an Equinix data centre disrupts AWS Direct Connect in Ashburn

What happened

A north-easter knocked out utility power in the region, and Equinix said weather affected power at a property it leases in Ashburn, Virginia. Contingency plans were activated, but some customers still lost service. Amazon reported that AWS Direct Connect lost connectivity to Equinix facilities DC1 to DC6 and DC10 to DC12, which sit on one Ashburn campus, from about 6:20 am Eastern on 2 March for close to four hours. It is not reported whether power was lost across the whole campus or at one building. CoreSite said VA1 and VA2 in Reston did not lose power, although Direct Connect links to those sites also dropped.

Contributing factors

Widespread utility outages from the north-easter, including about 15,000 customers without power in Loudoun County, as reported by Dominion Energy. Equinix said its contingency actions did not prevent an interruption for some customers. Why links to the CoreSite sites dropped was not explained.

Lessons learned

1. Keep redundant private cloud links, and use fast failure detection so traffic can move when a link drops.
2. Test contingency plans for utility loss in severe weather, including whether one site holds links used by a whole campus.
3. Give critical cloud connections more than one region, because a regional weather event can affect several facilities at once.

Source

datacenterknowledge.com