[tech] [committee] Temperature Monitoring in Server Room [repost]

[Melissa Star]

Hi,

Admittedly being a developer it’s often easier for me to code my way out of a problem than to work out if a third party tool is reliable and secure.

Of course, like every developer I “assume” my code is always reliable and secure, until the horrible real world annoys my by proving otherwise.

I don’t just “want to be” a developer or a system admin, I am both and have been for years. But I do want to be a better developer and sysadmin, and having tools warn me that my servers are overheating or an SSD is starting to reallocate bad blocks at increasing speed is a good idea.

In my case I have an application development framework that manages servers already but doesn’t handle this functionality. It would be a useful addition to the framework.

So how difficult is it for icinga to call external code in response to events?

Sent from my iPhone

> On 19 Mar 2019, at 3:55 pm, Andrew Williams <andrew at ucc.gu.uwa.edu.au> wrote:
> 
>> On 2019-03-19 3:36 PM, Melissa Star wrote:
>> 
>> As for what UCC does, are you saying that the club should choose externally written code over the code written by a club member?
>> I've had a look at Icigna2, and it seems really good, to be fair. But it would not give me the experience of interacting more directly with the hardware and writing some interesting code.
>> And it would not encourage others to make the type of home-grown solutions that encourage learning.
> 
> It depends what people want to learn - having experience with a widely-used server monitoring tool would be good experience for anyone wanting to get a sysadmin job in a large organisation. Working with hardware and rolling your own code is good experience for anyone wanting to be a developer.
> 
>> One thought - running such a thing on a VM is a bad idea given it should run on every single machine to report its own internal state, not all machines can run VMs, and that will waste a hell of a lot of CPU and resources.
> 
> You _can_ run an instance of icinga on every machine, and set them up as a cluster, but you wouldn't run each instance in a VM - that way it could only report the state of its own VM, not the host machine.
> 
> In practice, I found it easier to run one instance of the icinga process and web server (which we have a VM for, but you don't need one). That process then uses SSH to run the plugins on each machine - essentially, to monitor disk space, for example, it's doing the equivalent of 'ssh foo at bar "df"' once every hour, or day, or whatever interval you specify. You need to set up a monitoring account with the icinga public SSH key on each monitored machine, to accept the SSH connections without a password.
> 
> Andrew