[phenixbb] Phenix-Rosetta integration tests fail with Rosetta MPI on RHEL 6.3

[Sina Masoud-Ansari]

Hi Tom,

Thanks for your reply. In that case, that's good news because I have a 
working non-mpi version of Rosetta 3.5 that I would like to make 
available to researchers on our SLURM cluster while I work on getting 
more recent versions of Rosetta working.

Are any of the cases in 
'phenix_regression.wizards.test_command_line_rosetta_quick' suitable for 
submitting against a cluster so that I can check that the basic workflow 
is correct?

My impression from the documentation for mr_rosetta was that I would 
need to use additional options like:

nproc=2 \
group_run_command="sbatch " \
queue_commands="#SBATCH --time=01:00:00" \
queue_commands="#SBATCH --mem-per-cpu=2048"
queue_commands="module load Rosetta/3.5-gcc-4.4.6" \
queue_commands="module load phenix/1.9-16926" \
queue_commands="export PHENIX_ROSETTA_PATH=\$MODULE_ROSETTA_ROOT" \
queue_commands="source phenix_env.sh" \
queue_commands="srun \\"

I have a few questions about this I was hoping this community could help 
with:

Does nproc determine the number of jobs submitted by Phenix?

Also, SLURM is a bit different to PBS in that we need to prefix commands 
with 'srun', for example 'srun echo hello'. Is this handled 
automatically by Phenix (phenix.mr_rosetta)?

Finally I've noticed that rosetta_refine and mr_rosetta have slightly 
different options with regards to queue submission in the documentation.

http://www.phenix-online.org/documentation/reference/rosetta_refine.html
http://www.phenix-online.org/documentation/reference/mr_rosetta.html

For example, rosetta_refine allows the option 'technology = slurm' but 
this is not listed as an option for mr_rosetta.
Are they indeed different?

Thanks again, I appreciate your help

On 12/09/14 1:48 am, Terwilliger, Thomas Charles wrote:
> i Sina,
>
> Thanks for the question!   We have not tried Phenix mr_rosetta with Rosetta MPI so I am not too surprised that it did not work.  We have focused our effort on making it work with clusters using qsub and similar commands. Normally an individual Rosetta job does not take a really long time; rather it is the many Rosetta jobs that collectively take a long time.  Consequently using MPI to parallelize individual jobs seems to me to be likely to take about the same clock time as using many individual processors for separate jobs.
>
> I'll check on the Rosetta error that you mention separately.
>
> All the best,
> Tom T
>    
>
>    
> ________________________________________
> From: phenixbb-bounces at phenix-online.org [phenixbb-bounces at phenix-online.org] on behalf of Sina Masoud-Ansari [s.ansari at auckland.ac.nz]
> Sent: Wednesday, September 10, 2014 10:29 PM
> To: phenixbb at phenix-online.org
> Subject: [phenixbb] Phenix-Rosetta integration tests fail with Rosetta MPI      on RHEL 6.3
>
> Hi there,
>
> I'm wondering anyone has managed to successfully run Phenix with Rosetta
> MPI on Red Hat Enterprise Linux 6.3 or similar?
>
> I currently have a non-MPI build of Rosetta that passes all of the tests
> in phenix_regression.wizards.test_command_line_rosetta_quick
>
> However when running these tests with the MPI build of Rosetta the
> following tests fail:
>
> Running test_relax
> ------------------------------------------------------------
> Numbers do not match: 6 1
> Relaxed solutions  6
> ---
> Relaxed solutions  1
>
> ------------------------------------------------------------
>
> FAILED
> See log files test_relax/test_relax.log test_relax/test_relax_current.log
> =================================================================
> Running test_relax_double
> ------------------------------------------------------------
> Numbers do not match: 6 1
> Relaxed solutions  6
> ---
> Relaxed solutions  1
>
> ------------------------------------------------------------
>
> ------------------------------------------------------------
> Numbers do not match: 6 1
> Relaxed solutions  6
> ---
> Relaxed solutions  1
>
> ------------------------------------------------------------
>
> FAILED
> See log files test_relax_double/test_relax_double.log
> test_relax_double/test_relax_double_current.log
> =================================================================
> Running test_relax_no_sol
> ------------------------------------------------------------
> Numbers do not match: 6 1
> Relaxed solutions  6
> ---
> Relaxed solutions  1
>
> ------------------------------------------------------------
>
> FAILED
> See log files test_relax_no_sol/test_relax_no_sol.log
> test_relax_no_sol/test_relax_no_sol_current.log
>
> I have attempted to attach an archive of the logs.
>
> I'm working with Rosetta 3.5, more recent versions of Rosetta fail other
> tests in phenix_regression.wizards.test_command_line_rosetta_quick ,
> please see https://www.rosettacommons.org/node/3721 for details.
>
> Rosetta 3.5 was compiled with GCC 4.4.6 using OpenMPI 1.6.4
>
> Any help or advice is appreciated.
>
> --
> Sina Masoud-Ansari
> Centre for eResearch
> 409.G21 24 Symonds Street
> University of Auckland
>
> e: s.ansari at auckland.ac.nz
> p: +64 9 3737599 ext 89369
> w: http://www.eresearch.auckland.ac.nz
>

-- 
Sina Masoud-Ansari
Centre for eResearch
409.G21 24 Symonds Street
University of Auckland

e: s.ansari at auckland.ac.nz
p: +64 9 3737599 ext 89369
w: http://www.eresearch.auckland.ac.nz