[ucc] How to mount an unclean ext3 filesystem read-only?

[Alex Dawson]

I'm trying to do something which would seem very silly, but which I
have good reason for. One of our frequent requests here in the land of
virtualization is to recover an individual file from a VM (1050 as of
today!). We have ways of doing it right now, but I'm working on a
streamlined way to do it for individual files. I've worked out how to
do it for Windows..

# Map VMDK to loop device (read only)
losetup -r /dev/loop0 /net/svVSS_datastorename/vmname/vmnane-flat.vmdk
# Scan loop device for partitions
kpartx -va /dev/loop0
# Mount filesystem
mount -o ro /dev/mapper/loop0p2 /mnt/recover/
# Do whatever we need to do
ls -l /mnt/recover/
# Unmount filesystem
umount /mnt/recover/
# Remove partitions
kpartx -d /dev/loop0
# Remove loop
losetup -d /dev/loop0

However, to do it for Linux, it's a little trickier (think.. Inception..).

We mount an ext3 filesystem from within an LVM VG, which is on/in a
VMDK, which is read-only on an NFS datastore, mounted inside another
VM...

# Map VMDK to loop device (read only)
losetup /dev/loop0 /net/svVSS_datastorename/vmname/vmname-flat.vmdk
# Scan loop device for partitions
kpartx -va /dev/loop0
# Scan partitions for LVM PVs
lvscan
# Activate VGroot
vgchange -ay VGroot
# Mount ext3 filesystem from within VGroot
mount -o ro,noatime,nodiratime -t ext3 /dev/mapper/VGroot-LVroot /mnt/recover

.. which gives this warning:

mount: wrong fs type, bad option, bad superblock on /dev/mapper/VGroot-LVroot,
      missing codepage or other error
      In some cases useful info is found in syslog - try
      dmesg | tail  or so

.. which gives this:

EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
printk: 42 messages suppressed.
Buffer I/O error on device dm-4, logical block 0
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 1
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 577
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360450
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360521
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360522
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360523
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360524
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360525
lost page write due to I/O error on dm-4
Buffer I/O error on device dm-4, logical block 360526
lost page write due to I/O error on dm-4
JBD: recovery failed
EXT3-fs: error loading journal.

This doesn't work, because even when trying to mount the ext3
filesystem read-only, the kernel tries to do a recovery of an
"unclean" filesystem, and it fails, because the LVM is read only,
because the VMDK is read-only. If the VG has been cleanly offlined, I
can mount even though the vmdk holding the LVM holding the ext3 FS is
read-only. Trying to mount as ext2 doesn't work either - "EXT2-fs:
dm-4: couldn't mount because of unsupported optional features (4)."

We have another method, using guestfish, but it's a bit slower, and
won't support recovering files from a VMDK that doesn't contain a
bootable OS (ie, additional disks), so for that situation, we can
clone VMDKs and attach them in other VMs for recovery, but that's also
slow and klunky.

So does anyone know a way to mount an unclean ext3 volume from a
read-only device?

Regards,
Alex