command line molprobity tools

Engin Özkan

1 May 2013 1 May '13

7:15 p.m.

Hi everybody, I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? BTW, the log file produced by the GUI is not comprehensive at all. I could not find a related mmtbx or cctbx function either. Another mentioned tool, phenix.validate, does not even exist in my installation (dev 1361). Thanks, Engin -- Engin Özkan Post-doctoral Scholar Laboratory of K. Christopher Garcia Howard Hughes Medical Institute Dept of Molecular and Cellular Physiology 279 Campus Drive, Beckman Center B173 Stanford School of Medicine Stanford, CA 94305 w ph: (650)-498-7111 cell: (650)-862-8563

Show replies by date

Pavel Afonine

1 May 1 May

7:19 p.m.

Hi Engin, try: phenix.pdbtools model_stat=true model.pdb If this doesn't do what you want then please let me know. Pavel On 5/1/13 12:15 PM, Engin Özkan wrote:

...

Hi everybody,

I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? BTW, the log file produced by the GUI is not comprehensive at all.

I could not find a related mmtbx or cctbx function either. Another mentioned tool, phenix.validate, does not even exist in my installation (dev 1361).

Thanks, Engin

Engin Özkan

7:26 p.m.

Thanks Pavel. This does half the job by reporting the overall values, but does not list detail. For example, it tells you the Ramachandran outlier percentage, but does not list the outlier residues. Or the bond and angle outliers. I would like to see those output as well. These are all done beautifully in the GUI, but apparently not output as human-readable text files. I looked at full parameter list, but could not see a longer output switch. Engin On 5/1/13 12:19 PM, Pavel Afonine wrote:

...

Hi Engin,

try:

phenix.pdbtools model_stat=true model.pdb

If this doesn't do what you want then please let me know.

Pavel

On 5/1/13 12:15 PM, Engin Özkan wrote:

...
Hi everybody,

I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? BTW, the log file produced by the GUI is not comprehensive at all.

I could not find a related mmtbx or cctbx function either. Another mentioned tool, phenix.validate, does not even exist in my installation (dev 1361).

Thanks, Engin

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

-- Engin Özkan Post-doctoral Scholar Laboratory of K. Christopher Garcia Howard Hughes Medical Institute Dept of Molecular and Cellular Physiology 279 Campus Drive, Beckman Center B173 Stanford School of Medicine Stanford, CA 94305 w ph: (650)-498-7111 cell: (650)-862-8563

Pavel Afonine

7:28 p.m.

Hi Engin, that's right, it outputs only global figures... Tools that Nat listed output detailed lists. Pavel On 5/1/13 12:26 PM, Engin Özkan wrote:

...

Thanks Pavel. This does half the job by reporting the overall values, but does not list detail. For example, it tells you the Ramachandran outlier percentage, but does not list the outlier residues. Or the bond and angle outliers. I would like to see those output as well. These are all done beautifully in the GUI, but apparently not output as human-readable text files.

I looked at full parameter list, but could not see a longer output switch.

Engin

On 5/1/13 12:19 PM, Pavel Afonine wrote:

...
Hi Engin,

try:

phenix.pdbtools model_stat=true model.pdb

If this doesn't do what you want then please let me know.

Pavel

On 5/1/13 12:15 PM, Engin Özkan wrote:

...
Hi everybody,

I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? BTW, the log file produced by the GUI is not comprehensive at all.

I could not find a related mmtbx or cctbx function either. Another mentioned tool, phenix.validate, does not even exist in my installation (dev 1361).

Thanks, Engin

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Nathaniel Echols

7:21 p.m.

On Wed, May 1, 2013 at 12:15 PM, Engin Özkan wrote:

...

I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"?

You can run these tools: phenix.ramalyze model.pdb phenix.rotalyze model.pdb phenix.cbetadev model.pdb phenix.clashscore model.pdb phenix.pdb_interpretation model.pdb restraints.cif write_geo_file=True Or for the really impatient and easily distracted, like me: mmtbx.validation_summary model.pdb

...

BTW, the log file produced by the GUI is not comprehensive at all.

Define "comprehensive"? -Nat

Engin Özkan

7:39 p.m.

Hi Nat, I saw mmtbx.validation_summary, but it says it extracts header info. That is not running molprobity for a file that came from a different refinement program. I like the individual tools, ramalyze (awesome name construction!) has already replaced my using of procheck commandline. Now I can use the others too. Comprehensive, but not the whole kitchen sink, would be what you see in the GUI reported, but in simple text form. The percentages/numbers of outliers, followed by the identity of those outliers. It would be great if when I hit the button for "View Log File" (or more accurately, a "View Result File"), what would appear is what I see in the GUI, which is concise enough but also detailed at the perfect level. Thanks, Engin On 5/1/13 12:21 PM, Nathaniel Echols wrote:

...

On Wed, May 1, 2013 at 12:15 PM, Engin Özkan wrote:

...
I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? You can run these tools:

phenix.ramalyze model.pdb phenix.rotalyze model.pdb phenix.cbetadev model.pdb phenix.clashscore model.pdb phenix.pdb_interpretation model.pdb restraints.cif write_geo_file=True

Or for the really impatient and easily distracted, like me:

mmtbx.validation_summary model.pdb

...
BTW, the log file produced by the GUI is not comprehensive at all. Define "comprehensive"?

-Nat _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Nathaniel Echols

7:51 p.m.

On Wed, May 1, 2013 at 12:39 PM, Engin Özkan wrote:

...

I saw mmtbx.validation_summary, but it says it extracts header info.

Only R-factors and bond and angle RMSDs (and the latter only for files from phenix.refine). The actual MolProbity stuff is calculated fresh. It's not a very sophisticated tool, I just use it to benchmark massive sets of test refinements (where I always run Phenix at the end with a null strategy, to control for differences in bulk solvent and hydrogen handling).

...

That is not running molprobity for a file that came from a different refinement program.

Why would you ever want to run one of those? :)

...

Comprehensive, but not the whole kitchen sink, would be what you see in the GUI reported, but in simple text form. The percentages/numbers of outliers, followed by the identity of those outliers. It would be great if when I hit the button for "View Log File" (or more accurately, a "View Result File"), what would appear is what I see in the GUI, which is concise enough but also detailed at the perfect level.

We have discussed making a unified program ("phenix.molprobity") that would do something like this. There aren't any technical obstacles; the only reason it hasn't happened is that this really needs to happen as a part of a more comprehensive refactoring of both the validation tools in the Phenix GUI, and of the MolProbity server itself. Otherwise we will inevitably end up with another jumbled mess with lots of code duplication. However, in the absence of that, I will add a --verbose flag to mmtbx.validation_summary which will dump the output of the individual programs sequentially. It won't be especially pretty, but it's only a handful of lines of code. -Nat

Engin Özkan

7:56 p.m.

Thanks, Nat. That's very useful! Engin On 5/1/13 12:51 PM, Nathaniel Echols wrote:

...

On Wed, May 1, 2013 at 12:39 PM, Engin Özkan wrote:

...
I saw mmtbx.validation_summary, but it says it extracts header info. Only R-factors and bond and angle RMSDs (and the latter only for files from phenix.refine). The actual MolProbity stuff is calculated fresh. It's not a very sophisticated tool, I just use it to benchmark massive sets of test refinements (where I always run Phenix at the end with a null strategy, to control for differences in bulk solvent and hydrogen handling).

...
That is not running molprobity for a file that came from a different refinement program. Why would you ever want to run one of those? :)

...
Comprehensive, but not the whole kitchen sink, would be what you see in the GUI reported, but in simple text form. The percentages/numbers of outliers, followed by the identity of those outliers. It would be great if when I hit the button for "View Log File" (or more accurately, a "View Result File"), what would appear is what I see in the GUI, which is concise enough but also detailed at the perfect level. We have discussed making a unified program ("phenix.molprobity") that would do something like this. There aren't any technical obstacles; the only reason it hasn't happened is that this really needs to happen as a part of a more comprehensive refactoring of both the validation tools in the Phenix GUI, and of the MolProbity server itself. Otherwise we will inevitably end up with another jumbled mess with lots of code duplication.

However, in the absence of that, I will add a --verbose flag to mmtbx.validation_summary which will dump the output of the individual programs sequentially. It won't be especially pretty, but it's only a handful of lines of code.

-Nat _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Pavel Afonine

8:31 p.m.

On 5/1/13 12:51 PM, Nathaniel Echols wrote:

...

We have discussed making a unified program ("phenix.molprobity") that would do something like this. There aren't any technical obstacles; the only reason it hasn't happened is that this really needs to happen as a part of a more comprehensive refactoring of both the validation tools in the Phenix GUI, and of the MolProbity server itself. Otherwise we will inevitably end up with another jumbled mess with lots of code duplication.

What an excellent idea for someones todo list -:) I would elaborate on this some more... Since validation in this context is not only checking model quality, but also checking data and model-to-data fit qualities, existing tools should probably be logically grouped into three: - data validation (phenix.xtriage, etc..) - model validation (all these numerous jiffies mentioned in this conversation) - model-to-data fit validation (phenix.model_vs_data, phenix.real_space_correlation, phenix.polygon, etc). Since all these tools were developed incrementally, there is duplication in what they output, duplication in relevant code, and there could be even inconsistencies... So that's right - to be healthy every programs needs a rewrite every so often! Pavel

Georg Mlynek

27 May 27 May

4:54 p.m.

New subject: paired refinement in phenix gui

Dear phenix developers, 1) I would like to do paired refinement using the phenix refine gui. It would be nice if you could confirm if my settings are ok, or if there are still some settings in the background which I overlooked, because we see small changes in R and Rfree. a, I added the model refined at higher resolution (e.g. 1.5A) and the mtz file cut at CC (e.g.1.5A) in the "Input data" tab, and set High resolution cutoff in "X-ray data and experimental phases" box to (e.g. 1.8A) In the refinement setting strategy tab I uncheck everything and set Number of cycles = 1. In "other options" I uncheck everything too. b, Alternatively adding a file to the input data with } xray_data { file_name = "/XYZ.mtz" labels = "FP,SIGFP" high_resolution = 1.8 low_resolution = None outliers_rejection = True french_wilson_scale = True french_wilson { max_bins = 60 min_bin_size = 40 } refine { strategy = none sites { individual = None torsion_angles = None rigid_body = None } main { bulk_solvent_and_scale = True apply_overall_isotropic_scale_to_adp = True fix_rotamers = False flip_peptides = False nqh_flips = False use_molprobity = True simulated_annealing = False simulated_annealing_torsion = False ordered_solvent = False } should also do the same or have I overlooking something. 2. For whatever reason I would like to do that (molprobity) - can I (and how can I do this) add the command line commands also in the gui? 3. In december 2012 there was a discussion about weak data and refinement. I would like to know if there are new data available showing that "Whether the benefit they describe is considered cosmetic or non-trivial,....." 4. Additionally could one short comment on what phenix is doing with strong (weak) reflections marked as aliens in XDS. 5. According to the supplementary of P. Andrew Karplus, Kay Diederichs "Linking Crystallographic Model and Data Quality" I would like to do anisotropic refinement, with additional options "ordered_solvent.new_solvent=anisotropic adp.individual.anisotropic="not element H" fix_rotamers=True". Putting this in a file is enough? main { bulk_solvent_and_scale = True apply_overall_isotropic_scale_to_adp = True fix_rotamers = True flip_peptides = False nqh_flips = False use_molprobity = True simulated_annealing = False simulated_annealing_torsion = False ordered_solvent = True } refinement { refine { adp { individual { isotropic = none anisotropic = not element H } } } } The "problem" mentioned in phenix documentation is not a problem with the gui? it is a good idea to constantly monitor the existing solvent molecules and check for new ones by using ordered_solvent=true keyword. If it's decided to refine waters with anisotropic ADP then make sure that the newly added ones are also anisotropic; useordered_solvent.new_solvent=anisotropic (default is isotropic). Thanks in advance and best regards, Georg.

Nathaniel Echols

10:08 p.m.

New subject: paired refinement in phenix gui

On Mon, May 27, 2013 at 9:54 AM, Georg Mlynek wrote:

...

It would be nice if you could confirm if my settings are ok, or if there are still some settings in the background which I overlooked, because we see small changes in R and Rfree.

Changes relative to what? It is not expected that feeding the output PDB file back to phenix.refine (or another program) will exactly reproduce the R-factors from the previous round of refinement, because the precision of the atomic parameters is limited by PDB format. (Using mmCIF instead would be more reliable, but we don't think a change in R-factor of 0.0004 is worth worrying about anyway.)

...

I added the model refined at higher resolution (e.g. 1.5A) and the mtz file cut at CC (e.g.1.5A) in the “Input data” tab, and set High resolution cutoff in “X-ray data and experimental phases” box to (e.g. 1.8A) In the refinement setting strategy tab I uncheck everything and set Number of cycles = 1. In “other options” I uncheck everything too.

This sounds correct, but it's much easier to simply run the validation GUI, because the underlying code (specifically phenix.model_vs_data) is designed to calculate identical R-factors to what phenix.refine shows you.

...

Alternatively adding a file to the input data with

I don't recommend this - using parameter files as input is fine for large repetitive parameter blocks such as custom geometry restraints, selections from the TLSMD server, or nucleic acid base pairs, but anything that attempts to override the controls in the main GUI may not work.

...

2. For whatever reason I would like to do that (molprobity) - can I (and how can I do this) add the command line commands also in the gui?

Could you please clarify?

...

3. In december 2012 there was a discussion about weak data and refinement. I would like to know if there are new data available showing that “Whether the benefit they describe is considered cosmetic or non-trivial,.....”

Maybe Kay could comment on this...

...

4. Additionally could one short comment on what phenix is doing with strong (weak) reflections marked as aliens in XDS.

Assuming that XDS doesn't actually remove these, phenix.refine may or may not discard them at the outlier filtering stage (which can be disabled). I don't know how the different methods for flagging suspect resolutions compare; Kay and/or Pavel would know more about this.

...

According to the supplementary of P. Andrew Karplus, Kay Diederichs “Linking Crystallographic Model and Data Quality” I would like to do anisotropic refinement, with additional options “ordered_solvent.new_solvent=anisotropic adp.individual.anisotropic="not element H" fix_rotamers=True”.

If your resolution really is 1.5Å, I wouldn't recommend making waters anisotropic. There is some difference of opinion whether anisotropic refinement of protein atoms is acceptable; it's probably good to try both ways. (It would be interesting to know whether including the weak high-resolution data allows you to parameterize the model more aggressively.) -Nat

Engin Özkan

1 May 1 May

7:48 p.m.

I tried to run phenix.ramalyze and rotalyze with the outliers_only=True flags. I get the following errors Usage: phenix.ramalyze mypdb.pdb Usage: phenix.rotalyze mypdb.pdb I thought ramalyze used to work with --outliers_only=True as arguments, but that seems to fail, too. Is this a bug, or am I misreading the usage text? Engin On 5/1/13 12:21 PM, Nathaniel Echols wrote:

...

On Wed, May 1, 2013 at 12:15 PM, Engin Özkan wrote:

...
I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? You can run these tools:

phenix.ramalyze model.pdb phenix.rotalyze model.pdb phenix.cbetadev model.pdb phenix.clashscore model.pdb phenix.pdb_interpretation model.pdb restraints.cif write_geo_file=True

Or for the really impatient and easily distracted, like me:

mmtbx.validation_summary model.pdb

...
BTW, the log file produced by the GUI is not comprehensive at all. Define "comprehensive"?

-Nat _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Nathaniel Echols

7:52 p.m.

On Wed, May 1, 2013 at 12:48 PM, Engin Özkan wrote:

...

I tried to run phenix.ramalyze and rotalyze with the outliers_only=True flags. I get the following errors Usage: phenix.ramalyze mypdb.pdb Usage: phenix.rotalyze mypdb.pdb

I thought ramalyze used to work with --outliers_only=True as arguments, but that seems to fail, too. Is this a bug, or am I misreading the usage text?

bug. Will fix this now. -Nat

Engin Özkan

7:54 p.m.

Just figured it out. The problem is with the usage text. It says: Example: phenix.rotalyze pdb=1ubq.pdb outliers_only=True It should say Example: phenix.rotalyze 1ubq.pdb outliers_only=True Engin On 5/1/13 12:48 PM, Engin Özkan wrote:

...

I tried to run phenix.ramalyze and rotalyze with the outliers_only=True flags. I get the following errors Usage: phenix.ramalyze mypdb.pdb Usage: phenix.rotalyze mypdb.pdb

I thought ramalyze used to work with --outliers_only=True as arguments, but that seems to fail, too. Is this a bug, or am I misreading the usage text?

Engin On 5/1/13 12:21 PM, Nathaniel Echols wrote:

...
On Wed, May 1, 2013 at 12:15 PM, Engin Özkan wrote:

...
I want to create a comprehensive, human-readable log file by running validation using molprobity tools within phenix command-line. I do not have structure factors/maps. Apparently, the GUI runs validation through phenix.model_vs_data (according to the .eff file), but in command-line, I cannot get phenix.model_vs_data to run without an mtz/hkl. I guess I could provide it a fake data file, and then set the comprehensive flag to "True"? You can run these tools:

phenix.ramalyze model.pdb phenix.rotalyze model.pdb phenix.cbetadev model.pdb phenix.clashscore model.pdb phenix.pdb_interpretation model.pdb restraints.cif write_geo_file=True

Or for the really impatient and easily distracted, like me:

mmtbx.validation_summary model.pdb

...
BTW, the log file produced by the GUI is not comprehensive at all. Define "comprehensive"?

-Nat _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

4068

Age (days ago)

4094

Last active (days ago)

List overview

Download

13 comments

4 participants

participants (4)

Engin Özkan
Georg Mlynek
Nathaniel Echols
Pavel Afonine