Weighting in phenix.real_space_refine
Hi all, How does phenix.real_space_refine deal with restraint weighting? Looking at the output of the program, I get the impression that it splits the model up into different chunks, and alters weights locally based on some model/map correlation metric - is this the case (and are there more specific details available), or how does it work otherwise? Cheers, Oliver.
Hi Oliver, unlike reciprocal space refinement where optimization of the weight requires systematically trying full refinement (which is very slow unless you use many cpus), in real space this can be done very quickly and the speed does not depend on model or map size. The weight calculation procedure implemented in phenix.real_space_refine includes splitting the model into ten randomly picked continues ten-residue long segments, and finding the best weight for each segment. The best weight is considered to be the one that results in a model possessing pre-defined bond and angle rms deviations and that has best model-to-map fit among all trial weights. The obtained array of weights is filtered for outliers and the average weight is calculated and used as the best weight. Parameters defining bond/angle rmsds as targets for weight optimization: target_bonds_rmsd = 0.01 target_angles_rmsd = 1.0 Pavel On 9/12/16 14:14, Oliver Clarke wrote:
Hi all,
How does phenix.real_space_refine deal with restraint weighting? Looking at the output of the program, I get the impression that it splits the model up into different chunks, and alters weights locally based on some model/map correlation metric - is this the case (and are there more specific details available), or how does it work otherwise?
Cheers, Oliver.
Thanks Pavel! So just to be sure - the average weight obtained with the randomly selected fragments is used for the entire model? Or different weights are used for different regions? I guess the latter would be in some ways preferable for large structures in cryoEM where there is a substantial variation in local resolution? Cheers, Oli
On Sep 13, 2016, at 10:54 AM, Pavel Afonine
wrote: Hi Oliver,
unlike reciprocal space refinement where optimization of the weight requires systematically trying full refinement (which is very slow unless you use many cpus), in real space this can be done very quickly and the speed does not depend on model or map size. The weight calculation procedure implemented in phenix.real_space_refine includes splitting the model into ten randomly picked continues ten-residue long segments, and finding the best weight for each segment. The best weight is considered to be the one that results in a model possessing pre-defined bond and angle rms deviations and that has best model-to-map fit among all trial weights. The obtained array of weights is filtered for outliers and the average weight is calculated and used as the best weight.
Parameters defining bond/angle rmsds as targets for weight optimization:
target_bonds_rmsd = 0.01 target_angles_rmsd = 1.0
Pavel
On 9/12/16 14:14, Oliver Clarke wrote:
Hi all,
How does phenix.real_space_refine deal with restraint weighting? Looking at the output of the program, I get the impression that it splits the model up into different chunks, and alters weights locally based on some model/map correlation metric - is this the case (and are there more specific details available), or how does it work otherwise?
Cheers, Oliver.
Hi Pavel, So how do I interpret the weights in the output? For example, with a structure I am refining at the moment, I have the following: "Weight determination summary: number of chunks: 40 random chunks: chunk 33 optimal weight: 0.9990 chunk 30 optimal weight: 10.0000 chunk 16 optimal weight: 9.9990 chunk 10 optimal weight: 9.9990 chunk 20 optimal weight: 9.9990 chunk 16 optimal weight: 9.9990 chunk 31 optimal weight: 0.0040 chunk 12 optimal weight: 5.9990 chunk 19 optimal weight: 10.0000 chunk 23 optimal weight: 0.8980 overall best weight: 9.4279” Is low tight and high loose, as for phenix.refine? And are there any prospects for the future of varying weights across the structure? Often in EM we will have large regions at good resolution, say <4Å, and other regions which while well defined are at lower resolution, say 6Å - applying the same weights to both regions doesn’t seem ideal. Would your suggested strategy at the moment be to split the structure domain-by-domain, refine domains separately and then recombine? Cheers, Oli (Also, there seems to be a very minor bug in the output model stats - “Dist moved from start” changes, but “Dist moved from previous” is always 0.000. Model statistics: Map CC (whole unit cell): 0.662 Map CC (around atoms): 0.773 rmsd (bonds): 0.01 rmsd (angles): 1.04 Dist. moved from start: 0.060 Dist. moved from previous: 0.000 All-atom clashscore 4.47 Ramachandran plot: outliers: 0.27 % allowed: 12.24 % favored: 87.49 % Rotamer outliers: 0.54 % C-beta deviations: 4)
On Sep 13, 2016, at 11:01 AM, Pavel Afonine
wrote: Hi Oliver,
So just to be sure - the average weight obtained with the randomly selected fragments is used for the entire model?
correct.
Pavel
Hi Oliver, the refinement target is T = weight * Tdata + Trestraints . So bigger weight makes map contribution more important and smaller weight emphasizes restraints. In your example, most segments yield weight=10 with a few outliers being less than 1. The average weight used in refinement is about 10 (computed after filtering outliers). Technically it is possible to implement finding optimal weight for each segment but as you can imagine this is going to be terribly slow. I have not experimented with "optimal weight vs local map resolution". But *speculating* I would say this is not important because refinement target used in phenix.real_space_refine does not depend on map shape but only map peaks. Of course you can try splitting domains and refining them separately and then assembling the full model back. But the differences you may see may not necessarily be due to the weight (they may be because domains don't see neighbors, for example). It would be very nice to identity a very specific and clear example (and small too so refinements run fast!) where using local weights vs global actually does make a difference.. May be you or someone on the list has something like this? That would be a starting point for me to do some research in this direction. Thanks for pointing “Dist moved from previous” issue - I will take care of it.. Pavel On 9/13/16 08:12, Oliver Clarke wrote:
Hi Pavel, So how do I interpret the weights in the output? For example, with a structure I am refining at the moment, I have the following:
"Weight determination summary: number of chunks: 40 random chunks: chunk 33 optimal weight: 0.9990 chunk 30 optimal weight: 10.0000 chunk 16 optimal weight: 9.9990 chunk 10 optimal weight: 9.9990 chunk 20 optimal weight: 9.9990 chunk 16 optimal weight: 9.9990 chunk 31 optimal weight: 0.0040 chunk 12 optimal weight: 5.9990 chunk 19 optimal weight: 10.0000 chunk 23 optimal weight: 0.8980 overall best weight: 9.4279”
Is low tight and high loose, as for phenix.refine? And are there any prospects for the future of varying weights across the structure? Often in EM we will have large regions at good resolution, say <4Å, and other regions which while well defined are at lower resolution, say 6Å - applying the same weights to both regions doesn’t seem ideal.
Would your suggested strategy at the moment be to split the structure domain-by-domain, refine domains separately and then recombine?
Cheers, Oli
(Also, there seems to be a very minor bug in the output model stats - “Dist moved from start” changes, but “Dist moved from previous” is always 0.000.
Model statistics: Map CC (whole unit cell): 0.662 Map CC (around atoms): 0.773 rmsd (bonds): 0.01 rmsd (angles): 1.04 Dist. moved from start: 0.060 Dist. moved from previous: 0.000 All-atom clashscore 4.47 Ramachandran plot: outliers: 0.27 % allowed: 12.24 % favored: 87.49 % Rotamer outliers: 0.54 % C-beta deviations: 4)
On Sep 13, 2016, at 11:01 AM, Pavel Afonine
wrote: Hi Oliver,
So just to be sure - the average weight obtained with the randomly selected fragments is used for the entire model? correct.
Pavel
Hi Oliver,
indeed this can be beneficial. We have done this e.g. for a cryo-EM map of
Pol III where for the apo models we see substantial resolution variation
(doi:10.1038/nature16143; doi:10.1111/febs.13732).
Best,
Arjen
On 13 September 2016 at 16:57, Oliver Clarke
Thanks Pavel!
So just to be sure - the average weight obtained with the randomly selected fragments is used for the entire model? Or different weights are used for different regions? I guess the latter would be in some ways preferable for large structures in cryoEM where there is a substantial variation in local resolution?
On Sep 13, 2016, at 10:54 AM, Pavel Afonine
wrote: Hi Oliver,
unlike reciprocal space refinement where optimization of the weight requires systematically trying full refinement (which is very slow unless you use many cpus), in real space this can be done very quickly and the speed does not depend on model or map size. The weight calculation procedure implemented in phenix.real_space_refine includes splitting the model into ten randomly picked continues ten-residue long segments, and finding the best weight for each segment. The best weight is considered to be the one that results in a model possessing
Cheers, Oli pre-defined bond and angle rms deviations and that has best model-to-map fit among all trial weights. The obtained array of weights is filtered for outliers and the average weight is calculated and used as the best weight.
Parameters defining bond/angle rmsds as targets for weight optimization:
target_bonds_rmsd = 0.01 target_angles_rmsd = 1.0
Pavel
On 9/12/16 14:14, Oliver Clarke wrote:
Hi all,
How does phenix.real_space_refine deal with restraint weighting?
Looking at the output of the program, I get the impression that it splits the model up into different chunks, and alters weights locally based on some model/map correlation metric - is this the case (and are there more specific details available), or how does it work otherwise?
Cheers, Oliver.
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Hi Arjen, thanks for pointing this out! I will have a look! Pavel On 9/13/16 08:06, arjen jakobi wrote:
Hi Oliver,
indeed this can be beneficial. We have done this e.g. for a cryo-EM map of Pol III where for the apo models we see substantial resolution variation (doi:10.1038/nature16143; doi:10.1111/febs.13732).
Best, Arjen
On 13 September 2016 at 16:57, Oliver Clarke
mailto:[email protected]> wrote: Thanks Pavel!
So just to be sure - the average weight obtained with the randomly selected fragments is used for the entire model? Or different weights are used for different regions? I guess the latter would be in some ways preferable for large structures in cryoEM where there is a substantial variation in local resolution?
Cheers, Oli > On Sep 13, 2016, at 10:54 AM, Pavel Afonine
mailto:[email protected]> wrote: > > Hi Oliver, > > unlike reciprocal space refinement where optimization of the weight requires systematically trying full refinement (which is very slow unless you use many cpus), in real space this can be done very quickly and the speed does not depend on model or map size. > The weight calculation procedure implemented in phenix.real_space_refine includes splitting the model into ten randomly picked continues ten-residue long segments, and finding the best weight for each segment. The best weight is considered to be the one that results in a model possessing pre-defined bond and angle rms deviations and that has best model-to-map fit among all trial weights. The obtained array of weights is filtered for outliers and the average weight is calculated and used as the best weight. > > Parameters defining bond/angle rmsds as targets for weight optimization: > > target_bonds_rmsd = 0.01 > target_angles_rmsd = 1.0 > > Pavel > > On 9/12/16 14:14, Oliver Clarke wrote: >> Hi all, >> >> How does phenix.real_space_refine deal with restraint weighting? Looking at the output of the program, I get the impression that it splits the model up into different chunks, and alters weights locally based on some model/map correlation metric - is this the case (and are there more specific details available), or how does it work otherwise? >> >> Cheers, >> Oliver. > _______________________________________________ phenixbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected] mailto:[email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
participants (3)
-
arjen jakobi
-
Oliver Clarke
-
Pavel Afonine