Dear Tim and Tom, Thanks for the response. After attempting multiple Parallel runs through PBS array options, I have decided to go with Tim's suggestion of running each job with 1 core. The time difference between nproc=4 and nproc=1 is negligible for my dataset and its lot easier to run without PBS and queues. Thanks so much, Kaushik On Mon, Apr 27, 2015 at 8:57 PM, Terwilliger, Thomas Charles < [email protected]> wrote:
Hi Kaushik,
I'm sorry for the problem with parallel autobuilding!
I think that parallel_autobuild is unfortunately not going to do what you want. It is for running many parallel jobs with the same starting model and combining all the results. It doesn't work with multiple starting models as I think you want it to.
On the error you got, I'm not quite sure what happened but it seems that perhaps the command "qsub" was not found:
RuntimeError: child process stderr output: command: 'qsub background=False nproc=20 "~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1"' /bin/sh: qsub: command not found
It is a little surprising that this would happen in parallel_autobuild and not in autobuild. However perhaps in autobuild you specified that the multiple jobs are to be run with "sh" (run_command=sh) and in parallel_autobuild it was with qsub (run_command=qsub).
Let me know if that does not help,
All the best, Tom T
------------------------------ *From:* [email protected] [ [email protected]] on behalf of Kaushik Hatti [ [email protected]] *Sent:* Monday, April 27, 2015 12:05 AM *To:* [email protected] *Subject:* [phenixbb] Autobuild on parallel
Hi,
What is the best way to run Autobuild if I need to run it on hundreds of datasets (through command-line)? I have a single machine with 24 cores and access to a cluster with 128 cores (on parallel; accessible only through PBS script).
Each Autobuild job with nproc=4 on a single core machine takes ~12 hours to run (nproc > 4 wouldn't make any difference in the speed of computation?!). However, I read that I could set nproc=10 (or more) and run on parallel ( https://www.phenix-online.org/documentation/reference/autobuild.html#paralle...). When I tried the same through PBS script, I got the following error,
XXXXXXXXXXXXXX Running up to 1 jobs in parallel... with total of 3 jobs
Splitting work into 3 jobs and running with 1 processors using qsub background=False nproc=20 background=False in ~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0 Final job will be run with sh with background=True Starting job 1...Log will be: ~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1.log Traceback (most recent call last): File "~opt/phenix-1.9-1692/phenix/phenix/autosol/AutoBaseExtend.py", line 989, in DoNextMethod self.CarryOutBest() # to be obtained after it is finished File "~opt/phenix-1.9-1692/phenix/phenix/autosol/AutoBaseExtend.py", line 2147, in CarryOutBest getattr(self,str(self.application_method))() # call this fn File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 2364, in AutoBuild_build_cycle self.AutoBuild_rebuild_cycle_run() File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 4647, in AutoBuild_rebuild_cycle_run always_reuse_model=always_reuse_model) File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 7452, in AutoBuild_build_refine mtz_file=mtz_file,mtz_ref_file=mtz_ref_file) File "~opt/phenix-1.9-1692/phenix/phenix/wizards/AutoBuild.py", line 7696, in run_standard_build_in_parallel r.run(out=sys.stdout) File "~opt/phenix-1.9-1692/phenix/phenix/autosol/run_group_of_wizards.py", line 294, in run self.start_run(run_file,last=is_last) File "~opt/phenix-1.9-1692/phenix/phenix/autosol/run_group_of_wizards.py", line 604, in start_run cmd+" "+self.add_double_quote(run_file,escape_space=False)).raise_if_errors() File "~opt/phenix-1.9-1692/cctbx_project/libtbx/easy_run.py", line 37, in raise_if_errors raise Error(msg) RuntimeError: child process stderr output: command: 'qsub background=False nproc=20 "~phenixWorkingDir/1yya/part1/8tim/AutoBuild_run_5_/TEMP0/RUN_FILE_1"' /bin/sh: qsub: command not found
XXXXXXXXXXXXXX
Please let me know if I can share eff and log files of the run.
Please advice, Kaushik
-- People living deeply have no fear of death - Anais Nin Caution: I am still the dumbest person I have ever known :-)
-- People living deeply have no fear of death - Anais Nin Caution: I am still the dumbest person I have ever known :-)