Parallel Version of Phenix?
Hello Phenix Users and Developers: I am new to Phenix and am trying to help a colleague move his project using Phenix from a single workstation to a Linux cluster. My background is chemistry and high performance computing, but I am not intimately knowledgeable about the workings of Phenix. Are there parts of Phenix that can run on a Linux cluster in parallel. I understand that a good part of Phenix is written in python, but there are certainly numerically intensive aspects of structure refinement that could be amenable to parallelization. It would really help if you could provide an overview of those components that are now running in parallel. A search on the net did not reveal information about a parallel version of Phenix. Thank you for your attention and any possible help. Kind regards, A. R. Rossi
Are there parts of Phenix that can run on a Linux cluster in parallel.
Some wizards can take advantage of multiple CPUs: http://phenix-online.org/documentation/running-wizards.htm Search for "parallel". This doesn't speed up individual tasks, but rather tries the same algorithms with different random seeds in parallel.
I understand that a good part of Phenix is written in python, but there are certainly numerically intensive aspects of structure refinement that could be amenable to parallelization.
If you install from sources and you have gcc 4.3 or higher, the core FFT in refinement will use multiple CPUs. However, there are many other serial calculations. The speedup due to parallel FFTs is hardly noticeable. refinement is notoriously difficult to parallelize. Ralf
What about simulated annealing and phasing? I thought some of the phasing modules are already parallelized in phenix. -- Jianghai On Jun 2, 2009, at 12:36 PM, Ralf W. Grosse-Kunstleve wrote:
Are there parts of Phenix that can run on a Linux cluster in parallel.
Some wizards can take advantage of multiple CPUs:
http://phenix-online.org/documentation/running-wizards.htm
Search for "parallel". This doesn't speed up individual tasks, but rather tries the same algorithms with different random seeds in parallel.
I understand that a good part of Phenix is written in python, but there are certainly numerically intensive aspects of structure refinement that could be amenable to parallelization.
If you install from sources and you have gcc 4.3 or higher, the core FFT in refinement will use multiple CPUs. However, there are many other serial calculations. The speedup due to parallel FFTs is hardly noticeable. refinement is notoriously difficult to parallelize.
Ralf _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
What about simulated annealing
Just the FFT, if you compile from sources with gcc 4.3 or higher.
and phasing? I thought some of the phasing modules are already parallelized in phenix.
That's true. Phaser makes use of OpenMP for some things (I'm uncertain about the details). To use these features, you'll also need to compile from sources with gcc 4.3 or higher. (The standalone version of phaser may actually work with gcc 4.2 or higher.) Ralf
In Phaser, the most CPU-intensive parts of the molecular replacement calculations have been parallelized with OpenMP. Basically, the rule of thumb is that, if we thought users needed a progress bar to see whether anything was happening, it was worth making that bit parallel! This makes some things in Phaser considerably faster if you have a multi-core machine, but it also increases the memory requirements. So we've been working to reduce the memory footprint before encouraging too many people to compile with OpenMP support. At the moment, we feel that SAD phasing in Phaser is fast enough that in a pipeline with substructure determination and automatic model-building, it's nowhere near the limiting step. So there isn't a huge incentive to make that part parallel. Randy On Jun 2 2009, Ralf W. Grosse-Kunstleve wrote:
What about simulated annealing
Just the FFT, if you compile from sources with gcc 4.3 or higher.
and phasing? I thought some of the phasing modules are already parallelized in phenix.
That's true. Phaser makes use of OpenMP for some things (I'm uncertain about the details). To use these features, you'll also need to compile from sources with gcc 4.3 or higher. (The standalone version of phaser may actually work with gcc 4.2 or higher.)
Ralf _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
How much memories will phaser need in parallel mode? For the new linux workstations, they are equipped at least 8 GB memories and memories are cheap. Even my MacBook Pro has 4 GB memories. Would phaser need more than that? -- Jianghai On Jun 2, 2009, at 3:35 PM, Randy J. Read wrote:
In Phaser, the most CPU-intensive parts of the molecular replacement calculations have been parallelized with OpenMP. Basically, the rule of thumb is that, if we thought users needed a progress bar to see whether anything was happening, it was worth making that bit parallel!
This makes some things in Phaser considerably faster if you have a multi-core machine, but it also increases the memory requirements. So we've been working to reduce the memory footprint before encouraging too many people to compile with OpenMP support.
At the moment, we feel that SAD phasing in Phaser is fast enough that in a pipeline with substructure determination and automatic model- building, it's nowhere near the limiting step. So there isn't a huge incentive to make that part parallel.
Randy
On Jun 2 2009, Ralf W. Grosse-Kunstleve wrote:
What about simulated annealing
Just the FFT, if you compile from sources with gcc 4.3 or higher.
and phasing? I thought some of the phasing modules are already parallelized in phenix.
That's true. Phaser makes use of OpenMP for some things (I'm uncertain about the details). To use these features, you'll also need to compile from sources with gcc 4.3 or higher. (The standalone version of phaser may actually work with gcc 4.2 or higher.)
Ralf _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Most problems run on machines with, say, 2Gb. Big problems take more memory, but very few would fail to run on a machine with 8Gb. It's very case dependent and depends on the amount of branching in the search. However, this is continually improving as we find places where we can restrict the growth of memory as the search branches in more directions. We're just trying to avoid unpleasant surprises for people who turn on multiple threads for large problems. Regards, Randy On Jun 2 2009, Jianghai Zhu wrote:
How much memories will phaser need in parallel mode? For the new linux workstations, they are equipped at least 8 GB memories and memories are cheap. Even my MacBook Pro has 4 GB memories. Would phaser need more than that?
-- Jianghai
On Jun 2, 2009, at 3:35 PM, Randy J. Read wrote:
In Phaser, the most CPU-intensive parts of the molecular replacement calculations have been parallelized with OpenMP. Basically, the rule of thumb is that, if we thought users needed a progress bar to see whether anything was happening, it was worth making that bit parallel!
This makes some things in Phaser considerably faster if you have a multi-core machine, but it also increases the memory requirements. So we've been working to reduce the memory footprint before encouraging too many people to compile with OpenMP support.
At the moment, we feel that SAD phasing in Phaser is fast enough that in a pipeline with substructure determination and automatic model- building, it's nowhere near the limiting step. So there isn't a huge incentive to make that part parallel.
Randy
On Jun 2 2009, Ralf W. Grosse-Kunstleve wrote:
What about simulated annealing
Just the FFT, if you compile from sources with gcc 4.3 or higher.
and phasing? I thought some of the phasing modules are already parallelized in phenix.
That's true. Phaser makes use of OpenMP for some things (I'm uncertain about the details). To use these features, you'll also need to compile from sources with gcc 4.3 or higher. (The standalone version of phaser may actually work with gcc 4.2 or higher.)
Ralf _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
participants (4)
-
Angelo Rossi -
Jianghai Zhu -
Ralf W. Grosse-Kunstleve -
Randy J. Read