Hi Tom, Thanks for your reply. In that case, that's good news because I have a working non-mpi version of Rosetta 3.5 that I would like to make available to researchers on our SLURM cluster while I work on getting more recent versions of Rosetta working. Are any of the cases in 'phenix_regression.wizards.test_command_line_rosetta_quick' suitable for submitting against a cluster so that I can check that the basic workflow is correct? My impression from the documentation for mr_rosetta was that I would need to use additional options like: nproc=2 \ group_run_command="sbatch " \ queue_commands="#SBATCH --time=01:00:00" \ queue_commands="#SBATCH --mem-per-cpu=2048" queue_commands="module load Rosetta/3.5-gcc-4.4.6" \ queue_commands="module load phenix/1.9-16926" \ queue_commands="export PHENIX_ROSETTA_PATH=\$MODULE_ROSETTA_ROOT" \ queue_commands="source phenix_env.sh" \ queue_commands="srun \\" I have a few questions about this I was hoping this community could help with: Does nproc determine the number of jobs submitted by Phenix? Also, SLURM is a bit different to PBS in that we need to prefix commands with 'srun', for example 'srun echo hello'. Is this handled automatically by Phenix (phenix.mr_rosetta)? Finally I've noticed that rosetta_refine and mr_rosetta have slightly different options with regards to queue submission in the documentation. http://www.phenix-online.org/documentation/reference/rosetta_refine.html http://www.phenix-online.org/documentation/reference/mr_rosetta.html For example, rosetta_refine allows the option 'technology = slurm' but this is not listed as an option for mr_rosetta. Are they indeed different? Thanks again, I appreciate your help On 12/09/14 1:48 am, Terwilliger, Thomas Charles wrote:

i Sina,

Thanks for the question! We have not tried Phenix mr_rosetta with Rosetta MPI so I am not too surprised that it did not work. We have focused our effort on making it work with clusters using qsub and similar commands. Normally an individual Rosetta job does not take a really long time; rather it is the many Rosetta jobs that collectively take a long time. Consequently using MPI to parallelize individual jobs seems to me to be likely to take about the same clock time as using many individual processors for separate jobs.

I'll check on the Rosetta error that you mention separately.

All the best, Tom T

________________________________________ From: [email protected] [[email protected]] on behalf of Sina Masoud-Ansari [[email protected]] Sent: Wednesday, September 10, 2014 10:29 PM To: [email protected] Subject: [phenixbb] Phenix-Rosetta integration tests fail with Rosetta MPI on RHEL 6.3

Hi there,

I'm wondering anyone has managed to successfully run Phenix with Rosetta MPI on Red Hat Enterprise Linux 6.3 or similar?

I currently have a non-MPI build of Rosetta that passes all of the tests in phenix_regression.wizards.test_command_line_rosetta_quick

However when running these tests with the MPI build of Rosetta the following tests fail:

Running test_relax ------------------------------------------------------------ Numbers do not match: 6 1 Relaxed solutions 6 --- Relaxed solutions 1

------------------------------------------------------------

FAILED See log files test_relax/test_relax.log test_relax/test_relax_current.log ================================================================= Running test_relax_double ------------------------------------------------------------ Numbers do not match: 6 1 Relaxed solutions 6 --- Relaxed solutions 1

------------------------------------------------------------

------------------------------------------------------------ Numbers do not match: 6 1 Relaxed solutions 6 --- Relaxed solutions 1

------------------------------------------------------------

FAILED See log files test_relax_double/test_relax_double.log test_relax_double/test_relax_double_current.log ================================================================= Running test_relax_no_sol ------------------------------------------------------------ Numbers do not match: 6 1 Relaxed solutions 6 --- Relaxed solutions 1

------------------------------------------------------------

FAILED See log files test_relax_no_sol/test_relax_no_sol.log test_relax_no_sol/test_relax_no_sol_current.log

I have attempted to attach an archive of the logs.

I'm working with Rosetta 3.5, more recent versions of Rosetta fail other tests in phenix_regression.wizards.test_command_line_rosetta_quick , please see https://www.rosettacommons.org/node/3721 for details.

Rosetta 3.5 was compiled with GCC 4.4.6 using OpenMPI 1.6.4

Any help or advice is appreciated.

-- Sina Masoud-Ansari Centre for eResearch 409.G21 24 Symonds Street University of Auckland

e: [email protected] p: +64 9 3737599 ext 89369 w: http://www.eresearch.auckland.ac.nz

-- Sina Masoud-Ansari Centre for eResearch 409.G21 24 Symonds Street University of Auckland e: [email protected] p: +64 9 3737599 ext 89369 w: http://www.eresearch.auckland.ac.nz

On Thu, Sep 11, 2014 at 9:43 PM, Sina Masoud-Ansari

wrote:

Finally I've noticed that rosetta_refine and mr_rosetta have slightly different options with regards to queue submission in the documentation.

http://www.phenix-online.org/documentation/reference/rosetta_refine.html http://www.phenix-online.org/documentation/reference/mr_rosetta.html

For example, rosetta_refine allows the option 'technology = slurm' but this is not listed as an option for mr_rosetta. Are they indeed different?

The design and implementation of these programs is about as different as possible, because the refinement requires tighter integration between Phenix and Rosetta. phenix.rosetta_refine is using a generic API for distributing similar jobs across a number of supported clusters, which makes it much easier for us to add support for queuing systems in the individual programs in Phenix, and easier for users to invoke this support, but also requires specific code to handle the different software. So it is slightly less flexible than what MR-Rosetta (or AutoBuild, or LigandFit) is doing internally. -Nat