[tech] Long Running Computations on Motsugo

[Lyndon White]

Hi all,
I am currently running some computations on motsugo,
for my final year project.
Inline with the Clubs Objects (sic):
"To be an organised association of students attending The University of 
Western Australia, and supporters, for the advancement of computer 
science and technologies, both at the University and in the broader 
community.*"
*

Motsugo has 8 Physical Cores, and with hyper threading that makes 16 
logical cores.
I am running 6 processes.  You can see them in htop names "python 
motsugo_*.py"
[SZM] is also running a process for his final year project, you can see 
it in htop as "/tests/calculatepi.test"

That means that there is one free physcical core,
and 10 free logical cores.
However hyperthreading doesn't work well with long running computational 
tasks like mine,
so any processes sharing the physcial cores I am using are going to be 
in for a unfun time.

My current estimation (which is more or less a total guess) is that my 
processes will run for the next 2 weeks.
While it is set up to be mostly resumable if it is terminated (I would 
loose maybe several hours of work and have to do some minor reconfig). I 
would really prefer if noone killed them, or rebooted motsugo.
When this workload finishes I have 3 more similar loads.

I would also recommend anyone considering other similar long running 
computational workloads,
find another server.

I am currently setting up to be running similar experiments on SIP lab 
computers (Signal and Information Processing).
I have serveral quad cores waiting for me there (running ubuntu 10 :-( ).
However I basically need all the computational power I can get, so will 
be continuing to use motsugo.

If you are interested in the actual computer science I am advancing:
Have my project proposal abstract:
"After a large amount of time and computational resources have been 
invested in training a very large neural net, it is desirable to 
leverage that investment to create new neural nets for related tasks. 
This is known as domain adaptation. This project seeks to demonstrate a 
method which can be used for this. To achieve this goal, it is necessary 
to isolate which par ts of the neural net contain useful reusable 
information. Such infor mation would be an abstract description of the 
relationship of features within the input data-space."
If anyone is actually interested i can talk about this stuff all day and 
have a bunch of documents and stuffs.

Regards
[*OX]