HSX: failed disk was not correctly replaced

Okay, we had to create a fresh xfs fs on the new disk and preserve the old UUID. Something went wrong during the disk replacement.

HSX is really a nice solution, _but_ we need better monitoring capabilities and a better/more documentation/KB-articles what to do if something goes wrong. Currently we have open a case for everything that does not work as expected. Maybe I miss something, but I did not find anything about the problem with disk for example.

Yes, but I was too optimistic yesterday. Disk was mounted but not used by HSX, no files were created. I restarted cluster and rebooted node (due to other issues) and after that a directory “handle” was created, but still no files or data. 

So I had to create another case which is currenty worked on. It’s a pitty that nobody really checks if everything is correct at the end and I just don’t have the knowledge to see what is missing. HSX is really a black box and the GUI simply does not reflect reality always. I’ve good Linux knowlege but the simple task to see if a replaced disk is really part of hedwig, I don’t know how to check this.

221110-283  case to reopen 221109-316

221109-316 yesterdays case, now active again

220906-341 original case where something went wrong, but in GUI everything looked ok, no indication that anything has gone wrong

I’ve never used a storage solution where a simple task of replacing a failed disk went that wrong. I know that the steps documented seem to be easy, but as we see there seem to be no checks during the process and if something goes wrong, it’s complicated to get everything running again.