OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
CHI<>PAL
Incident Report for Network & Infrastructure
Resolved
Nous avons 10x10G down entre chicago et palo alto. Nous contactons le fournisseur. Ça sature un peu à LAX nous allons couper du peering EQUINIX sur SNJ et PAL.

Update(s):

Date: 2017-03-22 07:35:36 UTC
La coupure de fibre a été réparée. Le trafic a été rétabli un peu après 4h du matin GMT+1 (23h EST).

Rapport du founisseur:

Outage Start: March 21, 2017 17:08 GMT

Outage Stop: March 22, 2017 02:00 GMT

Root Cause: A fiber cut between Kansas City, MO and Lancaster, KS impacted unprotected traffic.

Fix Action: Field Services spliced the fiber.

Reason for Outage (RFO) Summary: The Level 3 Transport Network Operations Center (NOC) responded to alarms indicating a possible fiber cut between Kansas City, MO and Lancaster, KS that impacted unprotected traffic. Level 3 Field Services was engaged and conducted fiber testing. After fiber testing was completed, a fiber break was isolated to a location in Kansas. The fiber cabling was damaged due to a boring operation in the area. A repair team was dispatched to the site and excavated the damaged fiber cabling. Once the site was excavated, replacement fiber cable was placed and both ends of the existing fiber cable was prepared. Field Services spliced the fiber and all alarms cleared. The Technical Service Center confirmed service restoral with clients.

Date: 2017-03-22 00:49:33 UTC
Confirmation du fournisseur:
03/22/2017 00:23:06 GMT - Field Services have completed splicing of the orange tube and have started splicing of the green tube.

03/21/2017 23:34:23 GMT - Field Services completed splicing of the blue tube, and are splicing the orange tube at this time. The Transport NOC is observing alarms clear at this time, the estimated restoral time of 03:30 GMT remains.

03/21/2017 23:27:26 GMT - Field Services have commenced splicing of the blue tube at this time, an estimated restoral time of 03:30 GMT has been provided.

03/21/2017 22:36:42 GMT - The Level 3 Transport NOC and Level 3 Field Services informs that both ends of the fiber cable are being prepared for splicing at this time. As soon as the fiber has been prepared, splicing will begin. An estimated time of restoral (ETR) of March 22, 2017 at 03:30 GMT has been provided. Updates will continue to be provided hourly or as information becomes available.

03/21/2017 22:01:19 GMT - The Level 3 Transport NOC informs that the replacement fiber cable is onsite. The South end of the existing fiber cable has been prepared for splicing and the North end is being prepared at this time. Updates will continue to be provided hourly or as information becomes available.

03/21/2017 21:43:57 GMT - The Level 3 Transport NOC informs that excavations to expose the damaged fiber cable and fiber preparation work continue at this time.

03/21/2017 20:45:54 GMT - The Level 3 Transport NOC informs that excavations to expose the damaged fiber cable are in progress. Updates will continue to be provided hourly or as information becomes available.

03/21/2017 19:45:19 GMT - The Level 3 Transport NOC informs that the repair team is onsite assessing the damage. Updates will continue to be provided hourly or as information becomes available.

03/21/2017 18:47:37 GMT - The Level 3 Transport NOC informs that a fiber break has been isolated to a location in Kansas. The fiber cabling was damaged due to a boring operation in the area. A Level 3 repair team has been dispatched to the location with an estimated time of arrival (ETA) of 19:30 GMT.

03/21/2017 18:30:06 GMT - Level 3 Field Services is engaged and is conducting fiber testing at this time. Updates will continue to be provided hourly or as information becomes available.

03/21/2017 17:29:47 GMT - The Level 3 Transport Network Operations Center (NOC) responded to alarms indicating a possible fiber cut between Kansas City, MO and Lancaster, KS that is impacting unprotected traffic. The Transport NOC is investigating at this time. Please be advised that updates for this event will be relayed at a minimum of hourly unless otherwise noted. The information conveyed hereafter is associated to live troubleshooting effort and as the discovery process evolves through to service resolution, ticket closure, or post incident review, details may evolve.



Date: 2017-03-21 23:00:21 UTC
Plus de saturation. Nous contactons le fournisseur.
Posted Mar 21, 2017 - 22:41 UTC