[phenixbb] Autobuild on parallel
Terwilliger, Thomas Charles
terwilliger at lanl.gov
Mon Apr 27 08:27:17 PDT 2015
Hi Kaushik,
I'm sorry for the problem with parallel autobuilding!
I think that parallel_autobuild is unfortunately not going to do what you want. It is for running many parallel jobs with the same starting model and combining all the results. It doesn't work with multiple starting models as I think you want it to.
On the error you got, I'm not quite sure what happened but it seems that perhaps the command "qsub" was not found:
RuntimeError: child process stderr output:
command: 'qsub background=False nproc=20 "~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1"'
/bin/sh: qsub: command not found
It is a little surprising that this would happen in parallel_autobuild and not in autobuild. However perhaps in autobuild you specified that the multiple jobs are to be run with "sh" (run_command=sh) and in parallel_autobuild it was with qsub (run_command=qsub).
Let me know if that does not help,
All the best,
Tom T
________________________________
From: phenixbb-bounces at phenix-online.org [phenixbb-bounces at phenix-online.org] on behalf of Kaushik Hatti [hskaushik at gmail.com]
Sent: Monday, April 27, 2015 12:05 AM
To: phenixbb at phenix-online.org
Subject: [phenixbb] Autobuild on parallel
Hi,
What is the best way to run Autobuild if I need to run it on hundreds of datasets (through command-line)? I have a single machine with 24 cores and access to a cluster with 128 cores (on parallel; accessible only through PBS script).
Each Autobuild job with nproc=4 on a single core machine takes ~12 hours to run (nproc > 4 wouldn't make any difference in the speed of computation?!). However, I read that I could set nproc=10 (or more) and run on parallel (https://www.phenix-online.org/documentation/reference/autobuild.html#parallel-jobs-nproc-nbatch-number-of-parallel-models-and-how-autobuild-works-in-parallel). When I tried the same through PBS script, I got the following error,
XXXXXXXXXXXXXX
Running up to 1 jobs in parallel... with total of 3 jobs
Splitting work into 3 jobs and running with 1 processors using qsub background=False nproc=20
background=False in ~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0
Final job will be run with sh with background=True
Starting job 1...Log will be: ~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1.log
Traceback (most recent call last):
File "~opt/phenix-1.9-1692/phenix/phenix/autosol/AutoBaseExtend.py", line 989, in DoNextMethod
self.CarryOutBest() # to be obtained after it is finished
File "~opt/phenix-1.9-1692/phenix/phenix/autosol/AutoBaseExtend.py", line 2147, in CarryOutBest
getattr(self,str(self.application_method))() # call this fn
File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 2364, in AutoBuild_build_cycle
self.AutoBuild_rebuild_cycle_run()
File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 4647, in AutoBuild_rebuild_cycle_run
always_reuse_model=always_reuse_model)
File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 7452, in AutoBuild_build_refine
mtz_file=mtz_file,mtz_ref_file=mtz_ref_file)
File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 7696, in run_standard_build_in_parallel
r.run(out=sys.stdout)
File "~opt/phenix-1.9-1692/phenix/phenix/autosol/run_group_of_wizards.py", line 294, in run
self.start_run(run_file,last=is_last)
File "~opt/phenix-1.9-1692/phenix/phenix/autosol/run_group_of_wizards.py", line 604, in start_run
cmd+" "+self.add_double_quote(run_file,escape_space=False)).raise_if_errors()
File "~opt/phenix-1.9-1692/cctbx_project/libtbx/easy_run.py", line 37, in raise_if_errors
raise Error(msg)
RuntimeError: child process stderr output:
command: 'qsub background=False nproc=20 "~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1"'
/bin/sh: qsub: command not found
XXXXXXXXXXXXXX
Please let me know if I can share eff and log files of the run.
Please advice,
Kaushik
--
People living deeply have no fear of death - Anais Nin
Caution: I am still the dumbest person I have ever known :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/phenixbb/attachments/20150427/65ec248a/attachment-0001.htm>
More information about the phenixbb
mailing list