[tech] More space on Molmol

[Frames]

Thanks Zanchey
I got pretty close.

Spinning up a netapp for the webcam archieve is certainly a nonaweful idea.
And that could hold us by until someone gets up the guts to try a 
distributed storage solution.
Or 2.5inch drives show up.



Gozz was suggesting we could be storing the webcams as a compressed 
video format,
which should work well, as a lot of the frames are very similar.
That (and other things) could be experimented with, when the webcams are 
on other storage.

[*OX]


On 9/08/2016 9:13 PM, David Adam wrote:
> On Fri, 5 Aug 2016, Frames wrote:
>> Ok, here is the deal with Molmol. We're running out space.  The situation is
>> just below and the potential solutions are under that.
>>
>>   * It has its book disk on an internal USB.
>>   * It has its system disk on a 2.5inch SSD internally.
>>       o Using 1 of its 4, motherboard SATA 3 (Black) ports
>>   * It has 2 2.5 inch SDDs in raid for disk cache in the first 5.2inch bay,
>>       o they are connected to its Motherboard SAS (White) ports (running
>>         SATA3). There ports are not multiplexed -- that are endpoint
>>         SAS, ie SATA ports
> FYI, this is not quite right.
>
> The pair of SSDs are partitioned into:
>   * the bootloader (ada0p1 / ada1p1),
>   * the root filesystem (ada0p2 / ada1p2 as a GEOM mirror)
>   * the main drive array's SLOG devices (ada0p3 / ada1p3 as a ZFS mirror)
>   * swap (ada0p4)
>
> The other SSD is the L2ARC (ZFS cache) device. This came out of another
> machine and if it dies we can shrug and replace it.
>
> The internal USB is someone's USB stick. I used it to update the firmware
> at some stage; it is not used in the boot process.
>
>>   * It has piles of SATA 2 ports on the motherboard. Like 8 or so. These
>>     are Blue.
>>   * It has nothing in the bottom 5.2 inch bay, and it's fully accessible.
>>       o *We could stick some hard drives of any description here *(may
>>         be hard to get 2 x 3.5inches)
>>       o And use any of the free connectors (prob SATA3 -- of which we
>>         have 3 )
>>   * It s 8 taken 2.5 inch bays (hotswappable), which match to 2 4x SAS
>>     multipliers, which we have connected to the two ports on its SAS
>>     card and we run SATA 3 through that.
>>   * It has 8 free 2.5 inch bays (hotswappable), which match to 2 4x SAS
>>     multipliers
>>       o due to the physical nature of the SAS multiplexer (all 4,
>>         multiplexer are on once card), we can probably not directly
>>         connect those to SATA cable*** (more or less, see below)
>>
>> That more or less summarizes what we're working with. Here's the potential
>> angles of approach.
>> /NOTE: No matter what we do, we need to add drives in pairs, for RAID 1+0, and
>> we really would like at very least 2TB more space. Which requires at least 4TB
>> of drives../
> <snip lots of options>
>
> The other possibility is that we move or remove the webcam archives. Don't
> we have lots of big fileserving hardware that isn't being used? Can we get
> some of that back in service?
>
> Alternatively, I vaguely thought about running the whole archive through
> that flash new lossless JPEG algorithm[1] and then teaching the webserver
> how to decode it.
>
> [1]: https://blogs.dropbox.com/tech/2016/07/lepton-image-compression-saving-22-losslessly-from-images-at-15mbs/
>
> David Adam
> zanchey@