rssLink RSS for all categories
 
icon_red
icon_green
icon_blue
icon_orange
icon_blue
icon_green
icon_green
icon_orange
icon_green
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_green
icon_red
icon_green
icon_green
icon_red
icon_red
icon_green
icon_green
icon_green
icon_red
icon_orange
icon_green
icon_green
icon_blue
icon_green
icon_green
icon_green
 

FS#26382 — Instance storage GRA1

Attached to Project— Cloud
Maintenance
cloud
CLOSED
100%
Starting on the 2nd of August evening we'll modify the configuration of a storage cluster in GRA1.
Data rebalance during this operation will lower performance of the cluster.

Date:  Thursday, 10 August 2017, 15:38PM
Reason for closing:  Done
Comment by OVH - Thursday, 03 August 2017, 00:42AM

A high latency was perceptible during 45 minutes, between 22:30 and 23:15 (CET).

We advise to not reboot VM if slow IO is occuring.


Comment by OVH - Friday, 04 August 2017, 12:03PM

This rebalance is strongly necessary,it will provide better preformance and better threading for the cluster.

Rebalance is still in progress at 40% ETA to finish in 3 days

During this rebalance some slow I/O can be seen, but should disappear progressively


Comment by OVH - Friday, 04 August 2017, 15:38PM

To lower impact of rebalancing and improve service performance we decided to enable it only during lower usage-periods. In next few days we will start rebalance at 01:00 (GMT+2) and disable it at 07:00 (GMT+2).

It'll make whole process longer but performance impact will be lower.


Comment by OVH - Saturday, 05 August 2017, 03:58AM

Operation resumed.


Comment by OVH - Saturday, 05 August 2017, 07:12AM

Operation suspended (data rebalanced: 44%)


Comment by OVH - Sunday, 06 August 2017, 01:03AM

Operation resumed.


Comment by OVH - Sunday, 06 August 2017, 07:01AM

Operation suspended (data rebalanced: 49%)


Comment by OVH - Sunday, 06 August 2017, 09:23AM

Operation resumed. Due to hardware failure, there's a loss of 3rd copy of some part of data, although all cluster is operational, rebalance must be resumed to ensure all 3 copies are safe.


Comment by OVH - Monday, 07 August 2017, 01:01AM

Operation resumed.


Comment by OVH - Monday, 07 August 2017, 07:02AM

Operation suspended (data rebalanced: 59%)


Comment by OVH - Tuesday, 08 August 2017, 00:57AM

Operation resumed.


Comment by OVH - Tuesday, 08 August 2017, 07:10AM

Operation suspended (data rebalanced: 72%)


Comment by OVH - Wednesday, 09 August 2017, 01:01AM

Operation resumed.


Comment by OVH - Wednesday, 09 August 2017, 07:06AM

Operation suspended (data rebalanced: 81%)


Comment by OVH - Thursday, 10 August 2017, 01:01AM

Operation resumed.