[phenixbb] Recovering Autobuild from tty disconnect

Francis E Reyes Francis.Reyes at Colorado.EDU
Fri Aug 26 15:42:30 PDT 2011


Our job submission node disconnects 'idle' after 4 hours of inactivity. As most grid / autobuild  jobs take about that long I end being disconnected while the grid jobs complete. 

[1] Screen is not an option since access to the job submission node is via a load balancer and none of the job submission nodes allow access unless you're coming from the load balancer . i.e. there's no way to get to original machine where your screen has been started

[2] all processes are sent SIGKILL during this automatic logout preventing the use of nohup.. will background=True work?

[3] In the meantime, is it possible to resurrect the current autobuild? I've got FINISHED in all the autobuild's submitted to the grid, but everytime I try to combine_only=True run=my_run_number, I start from the beginning (with submitting the autobuild jobs to the cluster) even when I've moved the best of each build into MULTIPLE_MODELS.


Thanks!

F


---------------------------------------------
Francis E. Reyes M.Sc.
215 UCB
University of Colorado at Boulder







More information about the phenixbb mailing list