[tech] mylah and musundo repurposing, virtualisation and storage

[James Andrewartha]

[BOB] mentioned that he'd like to install Windows 7 on the desktops, which 
need a recent version of Samba to join the domain. We're currently running 
3.3.6 which isn't recent enough, and Blastwave's 3.4.3 version is broken: 
https://www.blastwave.org/forum/viewtopic.php?p=2095#p2095 OpenCSW only 
has 3.0.23, and SXCE b130 (which I upgraded musundo to yesterday) 3.0.37.

He then suggested moving Samba to a non-Solaris machine. A good candidate 
for this would be mylah, which could also take over amanda backups with 
the LTO2 tape library.

I've been wanting to have some sort of network storage for hosting VMs, 
which would ideally be running 64bit OpenSolaris to take advantage of ZFS 
deduplication for VM images and COMSTAR. Unfortunately mylah doesn't have 
AMD64 support, but if Samba, backups, /services and other bulky components 
of musundo:/space were moved to mylah then musundo and its fibre channel 
disk array could be used as the network storage server.

The disks of mylah and musundo are quite similar, 8x146GB SCSI and 18x73GB 
connected by one U160 SCSI bus and two 1Gb FC loops and currently 
configured as RAID6 and RAID10 (well, ZFS mirrors) respectively. Here's 
some benchmarks of the current setups:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
mylah           25G   142  99 75308  66 32936  19   330  98 104472  30 317.8  15
Latency             66344us     224ms     109ms   35798us   53357us     318ms

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
musundo-b130    16G    43  98 49190  38 44650  34   100  98 180856  53  1153  52
Latency               603ms    3469ms    3454ms     271ms     208ms     222ms

As [AHC] has ranted about before, filesystems need a certain amount of 
empty space, and ZFS is particularly sensitive to this - here's a 
benchmark from when musundo:/space had only 26GB free:

Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
musundo-b130     8G    40  97 11209  10  9965   8    97  96 71121  21 226.9  13
Latency               548ms    3661ms    2542ms     402ms     906ms     977ms

There's still some open questions:
 o If we make mylah a Xen server, what OS should be used for the dom0? 
   Debian, which may have problems with multiple CPU domUs, or CentOS 5.4 
   which will be harder to maintain? Or OpenSolaris, which has its own 
   well-integrated Xen support and fancy network management (Crossbow)?
 o What virtualisation software do we use for the Sun servers from Arts, 
   when they arrive? Do we want to virtualise Windows Server 2008, which
   will require using VMWare, or dedicate an entire machine to it? Or run 
   VMWare on one, and Xen on the others? Again, if Xen, what dom0 OS?
 o Do we set up a separate network segment for storage traffic, which
   would require another gigabit switch, or is our main network 
   underutilised enough that it'd be overkill?

To summarise: Move samba, /services and backups to mylah, turn musundo 
into a reliable network storage server for VMs. It's a good short term 
solution that's also a decent long term solution if nothing more gets 
done. If no-one has any objections, I think we should do this before 
O'Day, with the downtime occurring at the cleanup.

-- 
# TRS-80              trs80(a)ucc.gu.uwa.edu.au #/ "Otherwise Bub here will do \
# UCC Wheel Member     http://trs80.ucc.asn.au/ #|  what squirrels do best     |
[ "There's nobody getting rich writing          ]|  -- Collect and hide your   |
[  software that I know of" -- Bill Gates, 1980 ]\  nuts." -- Acid Reflux #231 /