Re: [phenixbb] Autobuild submitted jobs
Tom -- It turns out that each of the omit region's jobs submits another job, but that latter job is only the "test run_command" job. Why test the run_command if no subsequent jobs are submitted? Our solution, thanks to an extremely savvy cluster manager, was to install the qsub (Torque?) package on each of the computing nodes (117). Also we ran the top-level process on the head (scheduler) node, so as not to tie up a computing node while it waits for its sibling jobs (the omit regions) to finish the calculation. Overall wall-time with a separate node allocated for each omit region was about 50 minutes. Interestingly, although each node has 8 processors, the OMIT region jobs seems to only use one processor! Interestingly when phenix.refine jobs are run on my Macbook Pro, the operating system is savvy enough to spread it over 2 processors. -Mark ______________________________ Mark A. Saper, Ph.D. Associate Professor of Biological Chemistry University of Michigan Biophysics, 3040 Chemistry Building 930 N University Ave Ann Arbor MI 48109-1055 U.S.A. [email protected] phone (734) 764-3353 fax (734) 764-3323 http://www.biochem.med.umich.edu/?q=saper http://www.strucbio.umich.edu/
On Wed, May 26, 2010 at 7:16 PM, Mark A Saper
Interestingly, although each node has 8 processors, the OMIT region jobs seems to only use one processor!
The individual jobs spawned by AutoBuild are single-processor; the queueing system may have spread them out over as many nodes as possible, but they could just as well run on all 8 processors of one node, as long as enough memory is available. Interestingly when phenix.refine jobs are run on my Macbook Pro, the
operating system is savvy enough to spread it over 2 processors.
phenix.refine itself isn't actually using both processors at once, though - I'm not sure how Python and the OS work internally, but only one Python or C++ thread is running at a time, and it may switch between processors. (I've noticed the same behavior on my Macs.) If you use the GUI, that will be running in a separate process, but it doesn't do very much other than process messages from phenix.refine, plus whatever is required to display the model and maps (if you watch them in Coot, etc.). -Nat
Hi Mark, I'm glad you got this job running on your big cluster! In answer to your question, Autobuild tries the test run_command even if it doesn't need it, that's all. All the best, Tom T On May 26, 2010, at 8:16 PM, Mark A Saper wrote:
Tom --
It turns out that each of the omit region's jobs submits another job, but that latter job is only the "test run_command" job. Why test the run_command if no subsequent jobs are submitted?
Our solution, thanks to an extremely savvy cluster manager, was to install the qsub (Torque?) package on each of the computing nodes (117). Also we ran the top-level process on the head (scheduler) node, so as not to tie up a computing node while it waits for its sibling jobs (the omit regions) to finish the calculation. Overall wall-time with a separate node allocated for each omit region was about 50 minutes. Interestingly, although each node has 8 processors, the OMIT region jobs seems to only use one processor! Interestingly when phenix.refine jobs are run on my Macbook Pro, the operating system is savvy enough to spread it over 2 processors.
-Mark ______________________________ Mark A. Saper, Ph.D. Associate Professor of Biological Chemistry University of Michigan
Biophysics, 3040 Chemistry Building 930 N University Ave Ann Arbor MI 48109-1055 U.S.A.
[email protected] phone (734) 764-3353 fax (734) 764-3323 http://www.biochem.med.umich.edu/?q=saper http://www.strucbio.umich.edu/
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
Thomas C. Terwilliger Mail Stop M888 Los Alamos National Laboratory Los Alamos, NM 87545 Tel: 505-667-0072 email: [email protected] Fax: 505-665-3024 SOLVE web site: http://solve.lanl.gov PHENIX web site: http:www.phenix-online.org ISFI Integrated Center for Structure and Function Innovation web site: http://techcenter.mbi.ucla.edu TB Structural Genomics Consortium web site: http://www.doe-mbi.ucla.edu/TB CBSS Center for Bio-Security Science web site: http://www.lanl.gov/cbss
participants (3)
-
Mark A Saper -
Nathaniel Echols -
Tom Terwilliger