[tech] Molmol the slightly more fileserver

Andrew Adamson bob at ucc.gu.uwa.edu.au
Mon Oct 27 23:45:08 AWST 2014


And the file server migration is now largely complete. All files from the 
Netapp and SAN have been moved to /space on molmol and both the Netapp and 
SAN have been switched off. Storage is much simpler now:

Molmol:
/away
/vmstore
/mp3s
/services

Motsugo
/home

The majority of the move (all but /vmstore) only took about an hour to do 
due to pre-syncing a few days before, and only copying the differences on 
the day. rsync -havHAX is both effective and memorable for migrating data 
without screwing with it.

The move took a little longer than I had expected though; even though I 
had pre-synced most of the data a few days before, I mistakenly thought I 
couldn't pre-sync any of /vmstore because small changes to an image would 
make the whole image have to re-sync anyway. Next time I would try the 
--inplace option to only transfer the blocks that have changed. The VM 
files also had a lot of zero space that was taking a long time to copy. 
Adding -S to the options makes rsync deal with sparse files better and 
saved a bit of time once we found the option.

Things left to do are:
-work out what is causing the mega-slowdown with l2arc
-make molmol the primary domain controller for samba and turn off Mylah
-check the rsize and wsize nfs mount options that we've had for the last 
10 years are really still appropriate
-fully decommission and de-rack the SAN
-decide what to do with the netapp

Also thanks to [NTU] and [DAA] for their help with the migration, and 
everyone else who worked on the build over the 2 YEARS that we did it!

Andrew Adamson
bob at ucc.asn.au

|"If you can't beat them, join them, and then beat them."                |
| ---Peter's Laws                                                        |

On Thu, 23 Oct 2014, David Adam wrote:

> On Fri, 15 Aug 2014, David Adam wrote:
> > It would be worth seeing whether this happens under FreeBSD as well, but I 
> > don't think it does. That would definitely need to be sorted out before we 
> > used Debian+ZFS on Molmol.
> 
> So, neither the terrible NFS performance nor the weird namespacing thing 
> got fixed, and Bob wanted to start migrating data, so we have gone with 
> FreeBSD thanks to the two sweetest words in the English language.
> 
> I put some more details in the wiki: http://wiki.ucc.asn.au/Molmol
> 
> I've set up a GEOM mirror (RAID1) for the root filesystem. The reason for 
> not using ZFS as a root filesystem is that there are reports of race 
> conditions if you have multiple pools (e.g. root and space) sharing the 
> same devices, which sounds reasonably nasty.
> 
> David Adam
> UCC Wheel Member
> zanchey@
> _______________________________________________
> List Archives: http://lists.ucc.gu.uwa.edu.au/pipermail/tech
> 
> Unsubscribe here: http://lists.ucc.gu.uwa.edu.au/mailman/options/tech/bob%40ucc.gu.uwa.edu.au
> 


More information about the tech mailing list