rssLink RSS for all categories
 
icon_red
icon_green
icon_blue
icon_red
icon_orange
icon_green
icon_green
icon_red
icon_green
icon_red
icon_green
icon_green
icon_green
icon_red
icon_green
icon_orange
icon_green
icon_blue
icon_red
icon_red
icon_green
icon_green
icon_green
icon_blue
icon_orange
icon_green
icon_green
icon_red
icon_green
icon_red
icon_red
 

FS#28150 — Front-end servers

Attached to Project— Web Hosting / CloudDB
Incident
P19 / All clusters
CLOSED
100%
We encounter performance issues on our front-end load-balancer since 2017-11-04 05h30 UTC.

Our teams are investigating the root cause.
Date:  Tuesday, 14 November 2017, 00:47AM
Reason for closing:  Done
Comment by OVH - Sunday, 05 November 2017, 07:39AM

The infrastructure has stabilized overnight.
Cluster005 is no longer impacted by the incident.

In the meantime, we have identified several possibilities to improve the service for the weekend.
A maintenance operation is scheduled on Sunday around 14h (UTC+1) to apply a temporary fix.

All teams are involved in resolving the situation as soon as possible.


Comment by OVH - Sunday, 05 November 2017, 14:32PM

Operations in progress, we currently have no ETA.


Comment by OVH - Sunday, 05 November 2017, 16:18PM

Our operations are still in progress.
We are forced to temporarily disable the SSL certificate deployment process on our front-end servers.

This WILL NOT affect already deployed certificates NOR affect SSL security in any way.
Only the new or renewed certificates will have their deployment delayed.


Comment by OVH - Sunday, 05 November 2017, 16:40PM

We have to restart part of our VAC infrastructure in RBX.
All active connections will be interrupted and future traffic is going to be rerouted to the other rack.


Comment by OVH - Sunday, 05 November 2017, 17:00PM

We have identified the root cause.
Our web hosting in P19 should now be fully operational.

For now, we'll keep this ticket open and continue our investigations to fully understand what happened.


Comment by OVH - Sunday, 05 November 2017, 18:17PM

We can now confirm all services are up and running in P19.
The SSL certificates deployment process has been enabled too.

We keep this ticket open for now.