R, Rfree, and wxu_scale
Hi, Just finished a round of refinement. Rfree went up about 1% and R went down a bit. I am wondering what happened here. Also Rfree always went down after TLS refinement but always went up a lot more after the individual ADP refinement. Is there anything I can do here? Should I use weaker B restraints? I tried to assign a wxu_scale value. But phenix.refine automatically adjust it to 1.81 every time. ------------------------------------------------------------------------ R-factors, x-ray target values and norm of gradient of x-ray target stage r-work r-free xray_target_w xray_target_t 0 : 0.3982 0.5701 6.156859e+00 6.238509e+00 1_bss: 0.2008 0.2494 5.949117e+00 6.088855e+00 1_xyz: 0.2030 0.2497 5.953829e+00 6.091696e+00 1_adp: 0.1998 0.2559 5.952833e+00 6.099340e+00 2_bss: 0.1996 0.2557 5.952517e+00 6.099196e+00 2_xyz: 0.1984 0.2576 5.953246e+00 6.104085e+00 2_adp: 0.1974 0.2578 5.950434e+00 6.102336e+00 3_bss: 0.1972 0.2574 5.950333e+00 6.102222e+00 3_xyz: 0.1997 0.2576 5.956524e+00 6.103647e+00 3_adp: 0.1987 0.2590 5.955554e+00 6.104434e+00 3_bss: 0.1986 0.2588 5.955445e+00 6.104227e+00 ------------------------------------------------------------------------ Weights for target T = Exray * wxc * wxc_scale + Echem * wc and angles between gradient vectors, eg. (d_Exray/d_sites, d_Echem/d_sites) stage wxc wxu wxc_sc wxu_sc /_gxc,gc /_gxu,gu 0 : 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_bss: 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_xyz: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 1_adp: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_bss: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_xyz: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 2_adp: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_bss: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_xyz: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_adp: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_bss: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 ------------------------------------------------------------------------ Jianghai +++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++
Forgive me for saying this but the object of refinement is to build the most accurate model, not to obtain the lowest Rvalue, which will vary with weights applied to diffraction data versus constraints. There have been multiple posts about this on crystallographic BBs in the last few months ... -----Original Message----- From: Jianghai Zhu [mailto:[email protected]] Sent: Sat 8/4/2007 9:19 PM To: PHENIX user mailing list Subject: [phenixbb] R, Rfree, and wxu_scale Hi, Just finished a round of refinement. Rfree went up about 1% and R went down a bit. I am wondering what happened here. Also Rfree always went down after TLS refinement but always went up a lot more after the individual ADP refinement. Is there anything I can do here? Should I use weaker B restraints? I tried to assign a wxu_scale value. But phenix.refine automatically adjust it to 1.81 every time. ------------------------------------------------------------------------ R-factors, x-ray target values and norm of gradient of x-ray target stage r-work r-free xray_target_w xray_target_t 0 : 0.3982 0.5701 6.156859e+00 6.238509e+00 1_bss: 0.2008 0.2494 5.949117e+00 6.088855e+00 1_xyz: 0.2030 0.2497 5.953829e+00 6.091696e+00 1_adp: 0.1998 0.2559 5.952833e+00 6.099340e+00 2_bss: 0.1996 0.2557 5.952517e+00 6.099196e+00 2_xyz: 0.1984 0.2576 5.953246e+00 6.104085e+00 2_adp: 0.1974 0.2578 5.950434e+00 6.102336e+00 3_bss: 0.1972 0.2574 5.950333e+00 6.102222e+00 3_xyz: 0.1997 0.2576 5.956524e+00 6.103647e+00 3_adp: 0.1987 0.2590 5.955554e+00 6.104434e+00 3_bss: 0.1986 0.2588 5.955445e+00 6.104227e+00 ------------------------------------------------------------------------ Weights for target T = Exray * wxc * wxc_scale + Echem * wc and angles between gradient vectors, eg. (d_Exray/d_sites, d_Echem/d_sites) stage wxc wxu wxc_sc wxu_sc /_gxc,gc /_gxu,gu 0 : 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_bss: 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_xyz: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 1_adp: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_bss: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_xyz: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 2_adp: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_bss: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_xyz: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_adp: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_bss: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 ------------------------------------------------------------------------ Jianghai +++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++
Forgive me for saying this but the object of refinement is to build the most accurate model, not to obtain the lowest Rvalue, which will vary with weights applied to diffraction data versus constraints.
This is all true. Although if one observe increase in Rfree, it is cheap to try a few simple and obvious things to see if one can fix it.
There have been multiple posts about this on crystallographic BBs in the last few months ...
Yes, some of these issues were brougt up in the past in phenixbb (not 100% sure, but feels like this). Pavel.
Yes, yes. But before one admits the model with the higher Rfree is the more accurate model, one must make sure the refinement protocol is correct. Jianghai +++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++ On Aug 4, 2007, at 10:06 PM, Mayer, Mark (NIH/NICHD) [E] wrote:
Forgive me for saying this but the object of refinement is to build the most accurate model, not to obtain the lowest Rvalue, which will vary with weights applied to diffraction data versus constraints. There have been multiple posts about this on crystallographic BBs in the last few months ...
-----Original Message----- From: Jianghai Zhu [mailto:[email protected]] Sent: Sat 8/4/2007 9:19 PM To: PHENIX user mailing list Subject: [phenixbb] R, Rfree, and wxu_scale
Hi,
Just finished a round of refinement. Rfree went up about 1% and R went down a bit. I am wondering what happened here. Also Rfree always went down after TLS refinement but always went up a lot more after the individual ADP refinement. Is there anything I can do here? Should I use weaker B restraints? I tried to assign a wxu_scale value. But phenix.refine automatically adjust it to 1.81 every time.
---------------------------------------------------------------------- -- R-factors, x-ray target values and norm of gradient of x-ray target stage r-work r-free xray_target_w xray_target_t 0 : 0.3982 0.5701 6.156859e+00 6.238509e+00 1_bss: 0.2008 0.2494 5.949117e+00 6.088855e+00 1_xyz: 0.2030 0.2497 5.953829e+00 6.091696e+00 1_adp: 0.1998 0.2559 5.952833e+00 6.099340e+00 2_bss: 0.1996 0.2557 5.952517e+00 6.099196e+00 2_xyz: 0.1984 0.2576 5.953246e+00 6.104085e+00 2_adp: 0.1974 0.2578 5.950434e+00 6.102336e+00 3_bss: 0.1972 0.2574 5.950333e+00 6.102222e+00 3_xyz: 0.1997 0.2576 5.956524e+00 6.103647e+00 3_adp: 0.1987 0.2590 5.955554e+00 6.104434e+00 3_bss: 0.1986 0.2588 5.955445e+00 6.104227e+00 ---------------------------------------------------------------------- -- Weights for target T = Exray * wxc * wxc_scale + Echem * wc and angles between gradient vectors, eg. (d_Exray/d_sites, d_Echem/ d_sites) stage wxc wxu wxc_sc wxu_sc /_gxc,gc /_gxu,gu 0 : 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_bss: 5.5611e-01 7.2249e-01 0.050 1.810 96.334 101.736 1_xyz: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 1_adp: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_bss: 4.5001e-01 6.1023e-01 0.050 1.810 96.591 102.016 2_xyz: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 2_adp: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_bss: 5.1326e-01 6.5356e-01 0.050 1.810 112.802 141.393 3_xyz: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_adp: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 3_bss: 3.3568e-01 7.0695e-01 0.050 1.810 111.975 135.760 ---------------------------------------------------------------------- --
Jianghai
+++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
We make sure the default protocols are correct. The correctness of user customized protocols is user's responsibility to verify; however we are free to provide some guidance here. Pavel. Jianghai Zhu wrote:
Yes, yes. But before one admits the model with the higher Rfree is the more accurate model, one must make sure the refinement protocol is correct.
Jianghai
In the manual, I have also seen examples to manually set the target weight (target_weights.fix_wxc and target_weights.fix_wxu). But there is no explanations for their uses. What kind of effects would they cause? Jianghai +++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++ On Aug 4, 2007, at 11:31 PM, Pavel Afonine wrote:
We make sure the default protocols are correct. The correctness of user customized protocols is user's responsibility to verify; however we are free to provide some guidance here.
Pavel.
Jianghai Zhu wrote:
Yes, yes. But before one admits the model with the higher Rfree is the more accurate model, one must make sure the refinement protocol is correct.
Jianghai
phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Hi Jianghai,
In the manual, I have also seen examples to manually set the target weight (target_weights.fix_wxc and target_weights.fix_wxu). But there is no explanations for their uses. What kind of effects would they cause?
These parameters are used to fix the weights to user specified values. In this case the weights are not determined automatically anymore, but the program uses whatever is given by target_weights.fix_wxc and target_weights.fix_wxu (one exception could be for TLS+ADP, but I do not remember exactly). If target_weights.fix_wxc and target_weights.fix_wxu are used, then the total targets are: refinement of coordinates: Etotal = fix_wxc * Exray + wc * Egeom refinement of ADP: Etotal = fix_wxu * Exray + wu * Eadp Note, by default the targets are: refinement of coordinates: Etotal = wxc * wxc_scale * Exray + wc * Egeom refinement of ADP: Etotal = wxu * wxu_scale * Exray + wu * Eadp where wxc and wxu are determined automatically (similar to CNS); wxc_scale and wxu_scale are the scales to provide a room for manual adjustment of this values (if really necessary). Once again (for your information), all these will be explained in phenix.refine documentation in the next upcoming release. Pavel.
Hi Jianghai,
Just finished a round of refinement. Rfree went up about 1% and R went down a bit. I am wondering what happened here.
You may want to play with wxc_scale and wxu_scale, that define relative contribution of X-ray term in coordinate and ADP refinement, correspondingly. The corresponding weights, wxc and wxu, are determined automatically and are good for most of the cases, but as always, there are some deviations (rarely significant). Try different values for wxc_scale: just an array of values differing from default by say 0.25, 0.5, 1.5, 2 times and choose the one that gives you the best Rfree, gap Rfree-Rwork and geometry statistics. Same for wxu_scale. This always almost helps to resolve this kinds of issues.
Also Rfree always went down after TLS refinement but always went up a lot more after the individual ADP refinement. Is there anything I can do here?
Here you must be careful. There are two potential traps to be aware of (these are not bugs but features that people have to be aware of): 1) If you do TLS first, the atoms in output file have ANISOU records. If you then take that output file and refine individual ADP (without using TLS), by default they will be refined as anisotropic (since input file has ANISOU records and those atoms are considered as anisotropic). This is most likely not what you want and also in this case Rfree may go up drastically. 2) If you force them to be refined isotropically, then the anisotropic atoms got converted to isotropic and hence both R and Rfree will go up. This is also you may not want to do. So, the best, if you used TLS refinement once, keep using it in next refinements, otherwise you will fall into one of two traps above.
Should I use weaker B restraints? I tried to assign a wxu_scale value. But phenix.refine automatically adjust it to 1.81 every time.
This is intentional and always happens for TLS+ADP individual refinement. We found this the best by re-refining 350 models from PDB. All these questions are explained in depth in phenix.refine documentation in the next upcoming PHENIX release. Pavel.
I have tested different values for wxc_scale, seems like 0.05 gave me the best Rfree and smallest Rfree-R gap, which of course gave an excellent geometry statistics. My starting model has ANISO in it and I always using individual sites+individual ADP+tls. But then I can't test wxu_scale anymore since phenix.refine will just reset the wxu_scale. Since the best way is to use tls every time, how do test wxu_scale? Jianghai +++++++++++++++++++++++++++++++ Jianghai Zhu, Ph.D CBR Institute for Biomedical Research Department of Pathology Harvard Medical School 200 Longwood Ave., Boston, MA 02115 Ph: 617-278-3211 Fx: 618-278-3232 +++++++++++++++++++++++++++++++ On Aug 4, 2007, at 10:33 PM, Pavel Afonine wrote:
Hi Jianghai,
Just finished a round of refinement. Rfree went up about 1% and R went down a bit. I am wondering what happened here.
You may want to play with wxc_scale and wxu_scale, that define relative contribution of X-ray term in coordinate and ADP refinement, correspondingly. The corresponding weights, wxc and wxu, are determined automatically and are good for most of the cases, but as always, there are some deviations (rarely significant).
Try different values for wxc_scale: just an array of values differing from default by say 0.25, 0.5, 1.5, 2 times and choose the one that gives you the best Rfree, gap Rfree-Rwork and geometry statistics. Same for wxu_scale. This always almost helps to resolve this kinds of issues.
Also Rfree always went down after TLS refinement but always went up a lot more after the individual ADP refinement. Is there anything I can do here?
Here you must be careful. There are two potential traps to be aware of (these are not bugs but features that people have to be aware of):
1) If you do TLS first, the atoms in output file have ANISOU records. If you then take that output file and refine individual ADP (without using TLS), by default they will be refined as anisotropic (since input file has ANISOU records and those atoms are considered as anisotropic). This is most likely not what you want and also in this case Rfree may go up drastically. 2) If you force them to be refined isotropically, then the anisotropic atoms got converted to isotropic and hence both R and Rfree will go up. This is also you may not want to do.
So, the best, if you used TLS refinement once, keep using it in next refinements, otherwise you will fall into one of two traps above.
Should I use weaker B restraints? I tried to assign a wxu_scale value. But phenix.refine automatically adjust it to 1.81 every time.
This is intentional and always happens for TLS+ADP individual refinement. We found this the best by re-refining 350 models from PDB.
All these questions are explained in depth in phenix.refine documentation in the next upcoming PHENIX release.
Pavel. _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
My starting model has ANISO in it and I always using individual sites+individual ADP+tls. But then I can't test wxu_scale anymore since phenix.refine will just reset the wxu_scale. Since the best way is to use tls every time, how do test wxu_scale?
In this particular case (refinement of TLS + ADP individual) wxu_scale is adjusted to the "best" that we found. In this case you are not provided an option to play with this value. This is the best we have right now. Although this is the most optimized way of doing this among existing programs, this will be improved in future. Pavel.
participants (3)
-
Jianghai Zhu
-
Mayer, Mark (NIH/NICHD) [E]
-
Pavel Afonine