[phenixbb] queueing system support in GUI

[Francois Berenger]

Hello,

A little bit related, if you have many small jobs that need to be
done and you want to take advantage of all you CPUs and even
some neighbor ones (if you have a user account on other workstations),
I developped this recently:

http://savannah.nongnu.org/projects/par/

However, this is a simple software. Sun Grid Engine and other
related tools are far more complete.

Regards,
Francois.

Nathaniel Echols wrote:
> The most recent nightly build (dev-457) adds the ability to run jobs on 
> Sun Grid Engine.  To activate this feature, go to Preferences->Processes 
> and click "Activate queueing system support"; this will add a queue job 
> option to the menu that appears when you click the Run button.  I'm 
> hoping that the default command-line arguments I picked are suitable for 
> most environments.  Obviously the filesystem that you run jobs on must 
> be visible both to the host machine and to the machines in the queueing 
> system.  The same PHENIX installation must also be accessible from both. 
>  You need to be able to submit jobs from the machine that you're running 
> the GUI on.  (FYI, you could be running everything on a single 
> multi-core computer - there's no reason why this has to be done on a 
> cluster, but that's what I'm testing it on.)
> 
> You can view the current queue status from the main GUI via "Show queue 
> status" in the utilities menu.  (You can't actually delete jobs from 
> there yet, despite the misleading button that I forgot to remove.)
> 
> Some caveats:
> - This should, in theory, run on heterogeneous clusters, but I haven't 
> tried this because all of our systems are 64-bit Linux.
> - I haven't tried it on Mac either.
> - There will probably be some weirdness associated with closing the GUI 
> and resuming jobs while still running, especially if you try to abort them.
> - The built-in parallelization of AutoBuild and the ligand 
> identification program will not play nice with the queueing system when 
> using this mechanism (this is really a defect in queueing systems, not 
> AutoBuild).  I think it will be relatively easy to fix this, but for 
> now, you're better off using a single multiprocessor machine for 
> parallel AutoBuild runs in the GUI.
> - If the queueing system or remote host crashes, the GUI will still 
> think the job is running.
> 
> Those of you who are interested in using this, please let me know how it 
> works and what features are missing.  As always, I fully expect that 
> this will break on some setups.
> 
> thanks,
> Nat