[tech] Storage issues

David Adam zanchey at ucc.gu.uwa.edu.au
Thu Aug 29 00:08:15 WST 2013


Possibly due to the power outage yesterday (in which we lost of a bunch of 
non-critical circuits), the SAN lost its write cache, which took the 
VM storage area attached to Medico offline.

In trying to bring it back online, Medico kernel paniced. This is probably 
because some data had been written to the cache, which was lost when the 
cache dropped, and then the filesystem and the kernel disagreed about 
something.

As a result, Medico had to be powercycled, and all the VMs on Medico 
(including Mussel) restarted.

The consistency of the sanspace-vmstore filesystem does not appear to have 
been affected, though it didn't get fscked at boot.

I've added netconsole to the Medico configuration, and enabled bootlogd on 
both Medico and Mantis.

David Adam
zanchey at ucc.gu.uwa.edu.au


More information about the tech mailing list