[tech] Responding to Server Room Heat, Ashera Shutdown and Remote Control

[Melissa Star]

Hi Everyone,

After discussion with Felix last night, I am proposing the following:


Remote Power-Off for Ashera (and optionally other servers)

There was some discussion last night regarding the need to be able to remotely power down servers.

For Ashera, I have created a shutdown at ashera.ucc.asn.au account. Logging in to this account via ssh will cause the machine to immediately power down.

Note that there is no "confirmation prompt" and once the machine is powered down it must be physically restarted in the server room.

This was achieved by:

1. Creating the shutdown user and assigning group "operator".

2. Change Shell (chsh) to /usr/local/bin/bash 

3. Placing the command "shutdown -P now" into the .bashrc file. This can also be done using the init file of any other shell of your choice. 

I haven't set up SSH keys for the account yet, but could either use the standard SSH authorized_keys file for wheel, or preferably I could implement a command accessible to wheel that would cause ashera (and if desired, other servers) to shut down as needed.


As Ashera is my own sever co-located at UCC, I've already implemented this feature.



Making Remote Power-Off possible for UCC servers on command (apocalypse command)

If you are interested, I could also write a bash script that would respond to "apocalypse <machine name> <time delay>" by shutting down the appropriate machines. Access would be wheel only.

E.g.

	apocalypse ashera now 
		would shut down ashera inmmediately.

	apocalypse <group> now
		would shut down any defined group of servers. (E.g. apocalypse VMs now, or  apocalypse left-rack now). 

	apocalypse * now
		would power down every server that is part of the system.

I could then write a script that could read a heat sensor in the server room and automatically begin to shut down machines in order of priority as heat increases past a critical point.


Let me know what you think.


Regards,

Melissa






-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20190305/dadc9898/attachment.htm