rssLink RSS for all categories
 
icon_red
icon_green
icon_blue
icon_blue
icon_red
icon_green
icon_green
icon_orange
icon_green
icon_red
icon_red
icon_green
icon_green
icon_red
icon_orange
icon_red
icon_green
icon_green
icon_red
icon_red
icon_green
icon_green
icon_green
icon_blue
icon_orange
icon_green
icon_green
icon_green
icon_blue
icon_orange
icon_red
 

FS#29276 — meltdown/spectre gra

Attached to Project— Docker
Maintenance
Backend / Core
Planned
90%
Following the different CVE publicized :
- CVE-2017-5715 (branch target injection – Spectre)
- CVE-2017-5753 (bounds check bypass – Spectre)
- CVE-2017-5754 (rogue data cache load – Meltdown)

Patches are dependent on OS environment : OVH is maintaining a list of vendors patches => http://travaux.ovh.net/?do=details&id=29257


Before update, we are vulnerable to :
- CVE-2017-5715 (branch target injection – Spectre)
- CVE-2017-5753 (bounds check bypass – Spectre)
- CVE-2017-5754 (rogue data cache load – Meltdown)

After update, we will be vulnerable to :
- CVE-2017-5715 (branch target injection – Spectre)
- CVE-2017-5753 (bounds check bypass – Spectre)

Spectre flaws mitigation is not available for the moment.
Our teams are working on the deployment of an Intel microcode (during the system boot and/or EFI). This microcode would require kernel counter-measures (understand patch/update) to fully mitigate against Variant 2 / CVE-2017-5715 (https://security-center.intel.com/advisory.aspx?intelid=INTEL-SA-00088&languageid=en-fr).




The kernel is being tested at the moment.

The tests are conclusive, so we will proceed to an update of the kernel on the whole public cluster.

All hosts of the public clusters will be reboot and updated in a rolling manner during the 2018-01-08 daytime. We expect less than 1 minute of downtime per host.

During the reboot you may be unable to deploy containers if you have only one host. You may experience containers restart too.



More informations about the vulnerabilities, the impact and action plan can be found here : https://www.ovh.com/fr/blog/vulnerabilites-meltdown-spectre-cpu-x86-64-ovh-pleinement-mobilise/
Comment by OVH - Tuesday, 09 January 2018, 10:05AM

The first updated kernel we get during 2018-01-08 present a bug.
We have identify it and patch the kernel to fix it.

We are updating all hosts against this new fixed kernel if initial tests are conclusive.


Comment by OVH - Thursday, 11 January 2018, 10:00AM

A new kernel has been released and tested with success.
It has been deployed on every slave in every cluster (GRA, BHS, P19).
Only few internal servers remain.


Comment by OVH - Friday, 12 January 2018, 11:59AM

Kernels on LB have been downgraded due to performance issues (up to 500% slower). We're investigating on it.
All slaves exposed to customers apps are patched. Only masters and LB (both not exposed) are waiting for the upgrade as soon as performance issue is spotted and fixed.