rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_green
icon_red
icon_green
icon_green
icon_orange
icon_red
icon_red
icon_green
icon_green
icon_green
icon_orange
icon_green
icon_red
icon_green
icon_blue
icon_red
icon_red
icon_green
icon_red
icon_green
icon_green
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_green
icon_orange
icon_red
icon_green
icon_green
icon_green
icon_green
 

FS#50151 — Nodes are not available on SBG

Attached to Project— Kubernetes
Incident
Backend / Core
CLOSED
100%
Due to incident on Keystone, a lot of nodes deliver during the incident period are not working.
Travaux on Keystone incident: https://travaux.ovh.net/?do=details&id=50148

We are reinstall all nodes impacted by this incident.

Only new nodes delivery during incident are impacted, not all olds nodes on SBG region only.

**UPDATE: All nodes in SBG are impacted, communication with control plane is impacted too
Date:  Tuesday, 27 April 2021, 00:03AM
Reason for closing:  Done
Comment by OVH - Monday, 26 April 2021, 21:46PM

We are still investigating the root cause


Comment by OVH - Monday, 26 April 2021, 22:18PM

The incident from Keystone is not the root causes of this incident.

We have another issue on our IP Load Balancing who impact all customers on SBG.

*Update the name of this travaux


Comment by OVH - Monday, 26 April 2021, 23:26PM

The issue on IP Load Balancing is fixed.

We are going to recovery all nodes in errors.

The communication between nodes and Api Server is recovery
The api servers are accessible from Public Internet too


Comment by OVH - Tuesday, 27 April 2021, 00:03AM

All nodes are recovery

If you see a missing nodes with state "Ready,SchedulingDisabled" you can apply a "kubectl uncordon $node_name"