Circuit breaker trip causes power cut at Telehouse 2 data centre, Paris

What happened

On Wednesday 11 April 2012, a circuit breaker on the first-floor west side of the Telehouse 2 (TH2) data centre on boulevard Voltaire, Paris, tripped at 10:17. Power was restored at 10:53. Telehouse said that customers supplied at 48V or with dual feeds were not affected. Some hosted websites were offline for around 30 minutes, according to the report. Interconnection equipment housed at the facility was also affected, causing slowdowns and partial outages on some network routes.

OVH reported a partial impact on its links with Free, SFR, Numericable and Google. The exchange point FranceIX reported a temporary drop in traffic on its infrastructure, part of which is hosted at Telehouse 2. Free reported a generalised network problem on its Paris DSLAMs at the same time, but a link to the incident had not been established. Telehouse confirmed the incident around 12:30. The source gives no cause beyond the breaker trip.

Contributing factors

The cause of the breaker trip was not given. Interconnection equipment in the facility was affected, which spread the impact to operators beyond the hosted customers. Customers supplied at 48V or with dual feeds were not affected.

Lessons learned

1. Dual-feed and 48V supply arrangements kept customers running through a single breaker trip, so critical loads should be fed from independent distribution paths.
2. Interconnection and exchange equipment in the same facility can lose service when one power branch trips, so operators should map which network routes depend on a single electrical section.
3. Publish a clear cause as well as timings once it is known, because the initial statement gave times but no cause.

More lessons on electrical protection and isolation, emergency response and public communication, from every incident on the register.

Source

clubic.com