OVHcloud Public Cloud Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
LBaaS - GRA
Incident Report for Public Cloud
Resolved
• Incident summary / Résumé de l'incident : incident
• Start time / Heure de début : Estimated : 2020-05-20 12:20:00 UTC
• Impact / Périmètre affecté : Load Balancers
• Impact type / Type d'impact : delay delivery
• Estimated time to recovery / Temps de résolution estimé : 1 hour
• Actions undertaken / Actions entreprises : Ports creation process is stuck. We're unlocking the creation process.


Each Load balancer is assigned a port from our Cloud scheduler. This process is currently in a locked state so that few Load balancers delivered since yesteray 8PM are seeing delivery delays issues.
The team is currently working on it.

Update(s):

Date: 2020-05-21 04:28:18 UTC
All load balancers are now fully functionnal.
We will publish the post mortem as soon as we can.

Date: 2020-05-21 03:59:55 UTC
Scheduled load balancers are doing fine. We continue the deployment.
We expect the situation to get back to normal in about 30minutes from now.

Date: 2020-05-21 00:14:02 UTC
Tests are OK, we start the reschedule of a first batch of deployments.

Date: 2020-05-20 22:46:12 UTC
We're processing functional tests over all deployments

Date: 2020-05-20 21:39:36 UTC
We've identified the root cause which is linked to the libVirt component. We are not able to quick fix this root cause and we will plan a maintenance later to address it. In the mean time, we've asked our internal customer (K8S) to rollback its Load Balancing strategy to avoid any impact for new services.

We're still catching up the situation for Customers that have spawned a LBaaS, and we are going to update this task until we have remaining impacted customers.

Date: 2020-05-20 18:37:08 UTC
root cause has been identified, we are currently rolling the deployment of the fix

Date: 2020-05-20 13:47:27 UTC
We experience anormal long duration calls.


Date: 2020-05-20 13:19:23 UTC
Lock status has been resolved. We're recovering stucked load balancers
Posted May 20, 2020 - 09:40 UTC