[tech] [committee] Temperature Monitoring in Server Room [repost]

[Andrew Williams]

On 2019-03-19 3:36 PM, Melissa Star wrote:

> As for what UCC does, are you saying that the club should choose externally written code over the code written by a club member?
> 
> I've had a look at Icigna2, and it seems really good, to be fair. But it would not give me the experience of interacting more directly with the hardware and writing some interesting code.
> 
> And it would not encourage others to make the type of home-grown solutions that encourage learning.

It depends what people want to learn - having experience with a 
widely-used server monitoring tool would be good experience for anyone 
wanting to get a sysadmin job in a large organisation. Working with 
hardware and rolling your own code is good experience for anyone wanting 
to be a developer.

> One thought - running such a thing on a VM is a bad idea given it should run on every single machine to report its own internal state, not all machines can run VMs, and that will waste a hell of a lot of CPU and resources.

You _can_ run an instance of icinga on every machine, and set them up as 
a cluster, but you wouldn't run each instance in a VM - that way it 
could only report the state of its own VM, not the host machine.

In practice, I found it easier to run one instance of the icinga process 
and web server (which we have a VM for, but you don't need one). That 
process then uses SSH to run the plugins on each machine - essentially, 
to monitor disk space, for example, it's doing the equivalent of 'ssh 
foo at bar "df"' once every hour, or day, or whatever interval you specify. 
You need to set up a monitoring account with the icinga public SSH key 
on each monitored machine, to accept the SSH connections without a 
password.

Andrew