Hi Ralf, Indeed some bond length or angle restraint outliers are from the ligand when refining with phenix. However I don't think this is the problem, I give to refmac and phenix the same ligand dictionary (cif file from ProDRG with link definition added with JLigand). Refmac / phenix give me the following outliers >4 sigma : bond length 0 / 6, bond angle 0 / 23, dihedral 23 / 13, chiral 0 / 0, plan 3 /13. Clearly phenix is not doing a good job (R decrease, but not Rfree). For refmac I'm using an x-ray weighting term (matrix) of 0.8, which is more or less expected for 1.3A resolution. I don't really know how to fix the weights in phenix as I never used it before and did not find any hint in the manual or bb. Lionel 2010/8/12 Ralf W. Grosse-Kunstleve

Hi Lionel,

...

wxc_scale (from default 0.5 downto 0.015), wxu_scale (from default 1 down to 0.03), wc (from default 1 up to 8), with and without optimising wxc wxu. Whatever the parameters, phenix never reached final rmsd better than 0.022 and 2.2.

The first thing I'd check is the phenix.refine log with the list of worst restraints. Look for "Sorted by residual". Without having seen your structure, my first suspect would be problems with the ligand restraints or the covalent link to the protein.

Ralf _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

On Thu, Aug 12, 2010 at 8:11 AM, Lionel Costenaro wrote:

Indeed some bond length or angle restraint outliers are from the ligand when refining with phenix. However I don't think this is the problem, I give to refmac and phenix the same ligand dictionary (cif file from ProDRG with link definition added with JLigand). Refmac / phenix give me the following outliers >4 sigma : bond length 0 / 6, bond angle 0 / 23, dihedral 23 / 13, chiral 0 / 0, plan 3 /13. Clearly phenix is not doing a good job (R decrease, but not Rfree). For refmac I'm using an x-ray weighting term (matrix) of 0.8, which is more or less expected for 1.3A resolution. I don't really know how to fix the weights in phenix as I never used it before and did not find any hint in the manual or bb.

ProDRG's CIF files can be problematic, because they often have ESDs that are inappropriately low and this confuses the minimizer. I'd recommend running phenix.elbow or phenix.ready_set to generate a new CIF file. Other comments: - Are you adding hydrogen atoms? This can help with the geometry (and at this resolution, it will certainly help with the R-factors). - As pavel mentioned, you should turn on the weight optimization, but I would also recommend setting the max allowable RMS(bonds) and RMS(angles) to something more reasonable than the defaults - say, 0.02 and 2.0, if not less. (I forget the parameter names but if you are using the GUI they are all in the same dialog window.) - You should also update to version 1.6.4. -Nat

0.022A rmsd on bonds seems to be a reasonable value at 1.3A. On a different but related note, is there some way to print out the list of bond length deviations from ideal values for all the restrained bonds in the structure? If yes, could you do me a favor and forward me such list for you model? Off-list of course, but I promise to post the findings (assuming that they are interesting). Ed. On Thu, 2010-08-12 at 17:11 +0200, Lionel Costenaro wrote:

Hi Ralf,

Indeed some bond length or angle restraint outliers are from the ligand when refining with phenix. However I don't think this is the problem, I give to refmac and phenix the same ligand dictionary (cif file from ProDRG with link definition added with JLigand). Refmac / phenix give me the following outliers >4 sigma : bond length 0 / 6, bond angle 0 / 23, dihedral 23 / 13, chiral 0 / 0, plan 3 /13. Clearly phenix is not doing a good job (R decrease, but not Rfree). For refmac I'm using an x-ray weighting term (matrix) of 0.8, which is more or less expected for 1.3A resolution. I don't really know how to fix the weights in phenix as I never used it before and did not find any hint in the manual or bb.

Lionel

2010/8/12 Ralf W. Grosse-Kunstleve Hi Lionel,

> wxc_scale (from default 0.5 downto 0.015), wxu_scale (from default 1 down > to 0.03), wc (from default 1 up to 8), with and without optimising wxc wxu. > Whatever the parameters, phenix never reached final rmsd better than 0.022 > and 2.2.

The first thing I'd check is the phenix.refine log with the list of worst restraints. Look for "Sorted by residual". Without having seen your structure, my first suspect would be problems with the ligand restraints or the covalent link to the protein.

Ralf _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

-- "I'd jump in myself, if I weren't so good at whistling." Julian, King of Lemurs

Lionel Also, you can load any CIF file into REEL phenix.reel ligand.cif to get an idea of which restraints are problematic. The restraints with too small ESDs are highlighted. I'm happy to answer any questions about eLBOW or ReadySet! Nigel On Thu, Aug 12, 2010 at 10:03 AM, Ralf W. Grosse-Kunstleve wrote:

...

On a different but related note, is there some way to print out the list of bond length deviations from ideal values for all the restrained bonds in the structure?

Each time phenix.refine starts up it writes these to the .geo file (unless you set write_geo_file=False), "Sorted by residual". You can also set write_final_geo_file=True to get the sorted restraints after refinement.

Ralf _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

-- Nigel W. Moriarty Building 64R0246B, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709     Email : [email protected] Fax   : 510-486-5909       Web  : CCI.LBL.gov

Hi Ed,

Cool, thanks. The .geo file also lists "weights" - I gather these are simply 1/(expected_esd**2)?

Yes, exactly. There is a chapter on this in Hendrickson, W.A. (1985). Meth. Enzym. 115, 252-270. But is is all very elementary: contribution to target function = weight * delta_of_some_sort**2 with weight = 1/(expected_esd**2) and delta depending on the restraint type, e.g. delta = distance_ideal - distance_model or delta = angle_idea - angle_model etc. Ralf

On Fri, Aug 13, 2010 at 10:20 AM, Lionel Costenaro wrote:

So hydrogens were not added (find_and_add_hydrogens = False) and none was in the input pdb, but refined as riding (hydrogens.refine=riding), solvent was not updated (ordered_solvent = False) -there is already 510 waters almost all well defined. To my knowledge, parameters were fine (including refining riding hydrogens) except maybe for the anisotropy of waters for which Pavel recommend isotropy at this resolution.

This is a common point of confusion, especially for people used to REFMAC. The "riding" model only specifies how existing hydrogen atoms should be treated; it does not actually model hydrogens in those positions if they are not present in the input file. So if there weren't any hydrogens in the input file, it was being refined hydrogen-free. To actually place the hydrogens, you need to click the box labeled "Automatically add hydrogens to model", which will run phenix.ready_set to place the new atoms. The find_and_add_hydrogens option is more specialized anyway and should be left alone unless you are working at ultra-high resolution or doing neutron crystallography. - bond outlier CAD - OAC model 1.26 - ideal: ProDRG single 1.36,

readyset deloc 1.30, eLBOW deloc 1.429, smiles deloc 1.269 - bond outlier CBC - OBD model 1.30 - ideal: ProDRG deloc 1.23, readyset double 1.23, eLBOW aromatic 1.768, smiles double 1.234 - angle outlier CAG CAF CAD model 99 - ideal: ProDRG 111, readyset 117, eLBOW 109, smiles 117 - dihedral outlier CBC NBB CAY CAZ model 2.2 - ideal: ProDRG 180, readyset 131.66, eLBOW 62.5, smiles 68

It seems to me that chemistry clearly depends on the software used, amazing.

This shouldn't be a huge surprise - the programs use different methods to specify (or guess) molecule parameters, some of which are more effective than others, and different approximations when optimizing the geometry, most of which are optimized for speed rather than theoretical rigor (unless you are running quantum chemical calculations like the AM1 optimization in eLBOW). As a general rule, it is very difficult to accurately guess the chemistry of a molecule based on a PDB file alone; I suspect that CIFs may have similar problems, but Nigel can clarify. Which method to obtain a "correct" ligand and link definition from scratch

should I use?

I think the correct answer is: use eLBOW with a SMILES string for the ligand, then run ready_set to generate the link, but be sure to supply the CIF file for your ligand to ready_set (instead of letting ready_set create one from scratch based on coordinates). I need to double-check what the GUI returns in a situation like this, because I have not tested this particular case. However, if ready_set tries to make a new CIF with just ligand restraints (*not* the link information), you probably want to ignore that, and keep the rest of the output files. Feel free to email me if you have difficulty running it. (And as mentioned before, you should definitely update to the newest version, because the behavior of some programs has changed, and the eLBOW GUI is relatively new anyway.) -Nat

Hi All, I came back from synchrotron with some good SAD/MAD data. Running some quick autosolve jobs at the beamline already gave interesting Se-mets peaks. At the beamline they use an old version of Phenix (1.3.... I believe). Strangely, at home my latest version of Phenix finds absolutely nothing and quickly end with the following error message: ******************************************************************************** Failed to carry out AutoSol_scale_and_analyze_mad: None of the solve versions worked ******************************************************************************** To test is the problem is in our data, I quickly re-run 3 old mad/sad datasets of structures previously solved in the lab using the old (good) Solve and/or older Phenix versions. Sadly, all SAD/MAD searches quickly end with the same error message. I then tested the 'p9_se_w2.sca' that comes with Phenix.. and once again the same premature failure. Is there something wrong with my Phenix? I'm using version 1.6.4-486 on an XPS 420 Quad running Ubuntu 10.04. Thanks, Gino PS everything else in Phenix seems to work (e.g. refinement, validation, elbow, utilities, etc.) ****************************************************************************** Gino Cingolani, Ph.D. Associate Professor Thomas Jefferson University Dept. of Biochemistry & Molecular Biology 233 South 10th Street - Room 826 Philadelphia PA 19107 Office (215) 503 4573 Lab (215) 503 4595 Fax (215) 923 2117 E-mail: [email protected] ****************************************************************************** "Nati non foste per viver come bruti, ma per seguir virtute e canoscenza" ("You were not born to live like brutes, but to follow virtue and knowledge") Dante, The Divine Comedy (Inferno, XXVI, vv. 119-120) ---- Original message ----

Date: Fri, 13 Aug 2010 10:44:40 -0700 From: Nathaniel Echols Subject: Re: [phenixbb] geometry weight and rmsd bonds - angles To: PHENIX user mailing list

On Fri, Aug 13, 2010 at 10:20 AM, Lionel Costenaro wrote:

So hydrogens were not added (find_and_add_hydrogens = False) and none was in the input pdb, but refined as riding (hydrogens.refine=riding), solvent was not updated (ordered_solvent = False) -there is already 510 waters almost all well defined. To my knowledge, parameters were fine (including refining riding hydrogens) except maybe for the anisotropy of waters for which Pavel recommend isotropy at this resolution.

This is a common point of confusion, especially for people used to REFMAC.  The "riding" model only specifies how existing hydrogen atoms should be treated; it does not actually model hydrogens in those positions if they are not present in the input file.  So if there weren't any hydrogens in the input file, it was being refined hydrogen-free. To actually place the hydrogens, you need to click the box labeled "Automatically add hydrogens to model", which will run phenix.ready_set to place the new atoms.  The find_and_add_hydrogens option is more specialized anyway and should be left alone unless you are working at ultra-high resolution or doing neutron crystallography.

- bond outlier  CAD - OAC   model 1.26 - ideal: ProDRG single 1.36,   readyset deloc 1.30,   eLBOW deloc 1.429,   smiles deloc 1.269 - bond outlier  CBC - OBD   model 1.30 - ideal: ProDRG deloc  1.23,   readyset double 1.23,   eLBOW aromatic 1.768,   smiles double 1.234 - angle outlier CAG CAF CAD   model 99 - ideal: ProDRG 111,   readyset 117,   eLBOW 109,   smiles 117 - dihedral outlier CBC NBB CAY CAZ   model 2.2 - ideal: ProDRG 180,   readyset 131.66,   eLBOW 62.5,   smiles 68

It seems to me that chemistry clearly depends on the software used, amazing.

This shouldn't be a huge surprise - the programs use different methods to specify (or guess) molecule parameters, some of which are more effective than others, and different approximations when optimizing the geometry, most of which are optimized for speed rather than theoretical rigor (unless you are running quantum chemical calculations like the AM1 optimization in eLBOW).  As a general rule, it is very difficult to accurately guess the chemistry of a molecule based on a PDB file alone; I suspect that CIFs may have similar problems, but Nigel can clarify.

Which method to obtain a "correct" ligand and link definition from scratch should I use?

I think the correct answer is: use eLBOW with a SMILES string for the ligand, then run ready_set to generate the link, but be sure to supply the CIF file for your ligand to ready_set (instead of letting ready_set create one from scratch based on coordinates).  I need to double-check what the GUI returns in a situation like this, because I have not tested this particular case.  However, if ready_set tries to make a new CIF with just ligand restraints (*not* the link information), you probably want to ignore that, and keep the rest of the output files.  Feel free to email me if you have difficulty running it.  (And as mentioned before, you should definitely update to the newest version, because the behavior of some programs has changed, and the eLBOW GUI is relatively new anyway.) -Nat ________________ _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Dear Nat, thanks for the prompt reply.

Are you sure you have C-shell installed? Yes it is installed.

The second thing to check is SELinux support, although I thought this problem might have been fixed in 1.6.4. I'm not sure I know how to do this.

Is 'p9_se_w2.sca' reporting the same error message? Yes, I gave the simple command:

phenix.autosol seq_file=seq.dat sites=4 atom_type=Se data=p9_se_w2.sca space_group="I4" unit_cell="113.949 113.949 32.474 90.000 90.000 90.00" resolution=2.4 thoroughness=quick and quickly got: ******************************************************************************** Failed to carry out AutoSol_scale_and_analyze_mad: None of the solve versions worked ******************************************************************************** Thanks! Gino ****************************************************************************** Gino Cingolani, Ph.D. Associate Professor Thomas Jefferson University Dept. of Biochemistry & Molecular Biology 233 South 10th Street - Room 826 Philadelphia PA 19107 Office (215) 503 4573 Lab (215) 503 4595 Fax (215) 923 2117 E-mail: [email protected] ****************************************************************************** "Nati non foste per viver come bruti, ma per seguir virtute e canoscenza" ("You were not born to live like brutes, but to follow virtue and knowledge") Dante, The Divine Comedy (Inferno, XXVI, vv. 119-120) ---- Original message ----

Date: Tue, 17 Aug 2010 10:14:25 -0700 From: Nathaniel Echols Subject: Re: [phenixbb] Failed to carry out AutoSol_scale_and_analyze_mad: None of the solve versions worked To: PHENIX user mailing list

On Tue, Aug 17, 2010 at 9:54 AM, Gino Cingolani wrote:

Strangely, at home my latest version of Phenix finds absolutely nothing and quickly end with the following error message:

******************************************************************************** Failed to carry out AutoSol_scale_and_analyze_mad:

None of the solve versions worked ********************************************************************************

To test is the problem is in our data, I quickly re-run 3 old mad/sad datasets of structures previously solved in the lab using the old (good) Solve and/or older Phenix versions. Sadly, all SAD/MAD searches quickly end with the same error message.

Are you sure you have C-shell installed?  I don't think Ubuntu installs it by default, but the wizards still require it.  If it's missing, this would explain why the older versions fail too. The second thing to check is SELinux support, although I thought this problem might have been fixed in 1.6.4.  

I then tested the 'p9_se_w2.sca' that comes with Phenix.. and once again the same premature failure.

I thought this didn't even use SOLVE (it is SAD so it would use Phaser) - is it reporting the same error message? I will try to test it on an Ubuntu machine later this week. -Nat ________________ _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Hi Lionel,

1-"At 1.32A I would expect phenix.refine resulting in something like 0.022 and 2.2" (Pavel): 0.022 and 2.2 might indeed be correct for my structure at 1.32A. However, if I set bonds_rmsd_max = 0.012 and angles_rmsd_max = 1.5, providing I set the wc weight correctly, I would expect refine to -at least try to- reach this rmsd values.

not necessarily. These values are not the precise targets that are expected to achieve in refinement but rather the soft boundaries for optimal weight search. However this difference is a bit surprising to me - I will check it. Anyway, actual values achieved in refinement 0.022 and 2.2 look more realistic for 1.32A resolution than ad hoc ones, 0.012 and 1.5.

2- I did use optimize_wxc=true and optimize_wxu=true without much better results.

What is "better"?

- I used mostly the default parameters of the gui and just modified strategy (individual_sites all, individual_adp all, occupancy), 10 macro_cycles, and target_weights already mentioned. So hydrogens were not added (find_and_add_hydrogens = False) and none was in the input pdb, but refined as riding (hydrogens.refine=riding), solvent was not updated (ordered_solvent = False) -there is already 510 waters almost all well defined. To my knowledge, parameters were fine (including refining riding hydrogens) except maybe for the anisotropy of waters for which Pavel recommend isotropy at this resolution.

H-atoms should be added to your model before the refinement, so the input PDB contains hydrogens. You can do it using ReadySet! tool: phenix.ready_set model.pdb The command "find_and_add_hydrogens = False" is used for refinement against ultra-high resolution data to add H atoms to water oxygens based on residual map. It's not you case, so you shouldn't use this option.

Do you recommend to automatically update waters, even during late stages of refinement?

Yes. I guess I wrote it to bb yesterday or so. Refinement changes your structure, and changing the structure means some map improvement. Since solvent is added using the maps (mFo-DFc and 2mFo-DFc) it is a good idea to re-check it (solvent) once the maps are updated. The result of such re-check may be some more good water added, or removing some wrong waters originally added to noise density peaks. So, the solvent update is irrelevant to the stage of refinement. Good luck! Pavel.