Starting on the 2nd of August evening we'll modify the configuration of a storage cluster in GRA1.
Data rebalance during this operation will lower performance of the cluster.
Update(s):
Date: 2017-08-09 23:01:52 UTC Operation resumed.
Date: 2017-08-09 05:06:32 UTC Operation suspended (data rebalanced: 81%)
Date: 2017-08-08 23:01:23 UTC Operation resumed.
Date: 2017-08-08 05:10:48 UTC Operation suspended (data rebalanced: 72%)
Date: 2017-08-07 22:57:21 UTC Operation resumed.
Date: 2017-08-07 05:02:23 UTC Operation suspended (data rebalanced: 59%)
Date: 2017-08-06 23:01:53 UTC Operation resumed.
Date: 2017-08-06 07:23:51 UTC Operation resumed. Due to hardware failure, there's a loss of 3rd copy of some part of data, although all cluster is operational, rebalance must be resumed to ensure all 3 copies are safe.
Date: 2017-08-06 05:01:46 UTC Operation suspended (data rebalanced: 49%)
Date: 2017-08-05 23:03:11 UTC Operation resumed.
Date: 2017-08-05 05:12:02 UTC Operation suspended (data rebalanced: 44%)
Date: 2017-08-05 01:58:40 UTC Operation resumed.
Date: 2017-08-04 13:38:12 UTC To lower impact of rebalancing and improve service performance we decided to enable it only during lower usage-periods. In next few days we will start rebalance at 01:00 (GMT+2) and disable it at 07:00 (GMT+2).
It'll make whole process longer but performance impact will be lower.
Date: 2017-08-04 10:03:48 UTC This rebalance is strongly necessary,it will provide better preformance and better threading for the cluster.
Rebalance is still in progress at 40% ETA to finish in 3 days
During this rebalance some slow I/O can be seen, but should disappear progressively
Date: 2017-08-02 22:42:29 UTC A high latency was perceptible during 45 minutes, between 22:30 and 23:15 (CET).
We advise to not reboot VM if slow IO is occuring.