phenix real space refine & free R reflections
Dear Phenix bb, I have a rather conceptual question regarding the treatment of free R flags in phenix.refine and phenix.real_space_refine. I am refining an xray structure using phenix.refine (but not using the real space refinement implemented there) and at some point decided to play a bit with the standalone program phenix.real_space_refine. In particular, I was interested in testing real space refinement against different maps (experimental map vs. model-phased). Being used to the excellent handling of free R flags in Phenix, I assumed giving phenix.real_space_refine the output MTZ of a previous phenix.refine run as a map would be safe to do. But I then saw the phenix.refine FAQ states the following: "Are R-free flagged reflections included in the maps used for real-space refinement? No, this is almost guaranteed to bias R-free; these reflections are removed internally prior to map calculation. However, the output maps will include these reflections unless you explicitly request otherwise.” While it makes sense to me that including the free reflections in the maps used for real space refinement will bias R-free, I wonder why the default behavior of phenix.refine is to output maps which include these reflections. I assume most users will use these output maps (for example via the OPEN IN COOT button in the GUI) to iteratively improve their model by local real space refinement in Coot. Conceptually, how is real space refinement performed by phenix.refine (where the free R reflections are not used to calculate the map to refine against) different from manual local real space refinement in Coot? Will the latter not also bias R-free when the default output maps from phenix.refine are used? Best, Hauke
On 05/05/2017 05:01 AM, Hillen, Hauke wrote:
Dear Phenix bb,
I have a rather conceptual question regarding the treatment of free R flags in phenix.refine and phenix.real_space_refine. I am refining an xray structure using phenix.refine (but not using the real space refinement implemented there) and at some point decided to play a bit with the standalone program phenix.real_space_refine. In particular, I was interested in testing real space refinement against different maps (experimental map vs. model-phased). Being used to the excellent handling of free R flags in Phenix, I assumed giving phenix.real_space_refine the output MTZ of a previous phenix.refine run as a map would be safe to do. But I then saw the phenix.refine FAQ states the following:
"Are R-free flagged reflections included in the maps used for real-space refinement? No, this is almost guaranteed to bias R-free; these reflections are removed internally prior to map calculation. However, the output maps will include these reflections unless you explicitly request otherwise.”
While it makes sense to me that including the free reflections in the maps used for real space refinement will bias R-free, I wonder why the default behavior of phenix.refine is to output maps which include these reflections. I assume most users will use these output maps (for example via the OPEN IN COOT button in the GUI) to iteratively improve their model by local real space refinement in Coot. Conceptually, how is real space refinement performed by phenix.refine (where the free R reflections are not used to calculate the map to refine against) different from manual local real space refinement in Coot? Will the latter not also bias R-free when the default output maps from phenix.refine are used?
I would argue just the opposite- free reflections should be included in the maps used for RS refinement. Excluding free reflections is the same as setting them to zero amplitude, so you are still biasing these reflections, just biasing them toward a drastically wrong value (zero) instead of biasing them toward fobs. If the evidence for bias is that R-free gets "better" when you include the free reflections, isn't it more because you avoid biasing them toward zero than because you do bias them toward the Fobs? In the case of density modification, which has the same problem, Ivan Rayment* showed the importance of using the "fill-in" method for missing reflections in making maps. That would seem to be the correct way of implementing R-free for density modification or real-space refinement: you don't give away your Fobs, but at the same time you don't keep forcing the free reflections toward zero. *Acta Crystallogr B. 1983 Aug;39(4):505-516. Acta Cryst. (1983). A39, 102-116 eab
This is essentially what I do by default when making real-space maps to work with. The relevant portion(s) of my maps.params file for phenix.maps would typically look like: map { map_type = 2mFo-DFc format = xplor *ccp4 file_name = None fill_missing_f_obs = True grid_resolution_factor = 1/4. region = *selection cell atom_selection = None atom_selection_buffer = 10 sharpening = False sharpening_b_factor = None exclude_free_r_reflections = True isotropize = True } Cheers, Tristan On 2017-05-05 15:54, Edward A. Berry wrote:
On 05/05/2017 05:01 AM, Hillen, Hauke wrote:
Dear Phenix bb,
I have a rather conceptual question regarding the treatment of free R flags in phenix.refine and phenix.real_space_refine. I am refining an xray structure using phenix.refine (but not using the real space refinement implemented there) and at some point decided to play a bit with the standalone program phenix.real_space_refine. In particular, I was interested in testing real space refinement against different maps (experimental map vs. model-phased). Being used to the excellent handling of free R flags in Phenix, I assumed giving phenix.real_space_refine the output MTZ of a previous phenix.refine run as a map would be safe to do. But I then saw the phenix.refine FAQ states the following:
"Are R-free flagged reflections included in the maps used for real-space refinement? No, this is almost guaranteed to bias R-free; these reflections are removed internally prior to map calculation. However, the output maps will include these reflections unless you explicitly request otherwise.”
While it makes sense to me that including the free reflections in the maps used for real space refinement will bias R-free, I wonder why the default behavior of phenix.refine is to output maps which include these reflections. I assume most users will use these output maps (for example via the OPEN IN COOT button in the GUI) to iteratively improve their model by local real space refinement in Coot. Conceptually, how is real space refinement performed by phenix.refine (where the free R reflections are not used to calculate the map to refine against) different from manual local real space refinement in Coot? Will the latter not also bias R-free when the default output maps from phenix.refine are used?
I would argue just the opposite- free reflections should be included in the maps used for RS refinement. Excluding free reflections is the same as setting them to zero amplitude, so you are still biasing these reflections, just biasing them toward a drastically wrong value (zero) instead of biasing them toward fobs.
If the evidence for bias is that R-free gets "better" when you include the free reflections, isn't it more because you avoid biasing them toward zero than because you do bias them toward the Fobs? In the case of density modification, which has the same problem, Ivan Rayment* showed the importance of using the "fill-in" method for missing reflections in making maps. That would seem to be the correct way of implementing R-free for density modification or real-space refinement: you don't give away your Fobs, but at the same time you don't keep forcing the free reflections toward zero. *Acta Crystallogr B. 1983 Aug;39(4):505-516. Acta Cryst. (1983). A39, 102-116
eab _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
And perhaps that is what phenix is doing for RSR. In terms of phenix keywords, free reflections are excluded, but in fact they are included using values from Fc instead of Fobs. But it would be good to clarify that "fill_missing_f_obs = True" also fills those reflections that are not missing but are excluded due to being flagged free. On 05/05/2017 11:26 AM, Tristan Croll wrote:
This is essentially what I do by default when making real-space maps to work with. The relevant portion(s) of my maps.params file for phenix.maps would typically look like:
map { map_type = 2mFo-DFc format = xplor *ccp4 file_name = None fill_missing_f_obs = True grid_resolution_factor = 1/4. region = *selection cell atom_selection = None atom_selection_buffer = 10 sharpening = False sharpening_b_factor = None exclude_free_r_reflections = True isotropize = True }
Cheers,
Tristan
On 2017-05-05 15:54, Edward A. Berry wrote:
On 05/05/2017 05:01 AM, Hillen, Hauke wrote:
Dear Phenix bb,
I have a rather conceptual question regarding the treatment of free R flags in phenix.refine and phenix.real_space_refine. I am refining an xray structure using phenix.refine (but not using the real space refinement implemented there) and at some point decided to play a bit with the standalone program phenix.real_space_refine. In particular, I was interested in testing real space refinement against different maps (experimental map vs. model-phased). Being used to the excellent handling of free R flags in Phenix, I assumed giving phenix.real_space_refine the output MTZ of a previous phenix.refine run as a map would be safe to do. But I then saw the phenix.refine FAQ states the following:
"Are R-free flagged reflections included in the maps used for real-space refinement? No, this is almost guaranteed to bias R-free; these reflections are removed internally prior to map calculation. However, the output maps will include these reflections unless you explicitly request otherwise.”
While it makes sense to me that including the free reflections in the maps used for real space refinement will bias R-free, I wonder why the default behavior of phenix.refine is to output maps which include these reflections. I assume most users will use these output maps (for example via the OPEN IN COOT button in the GUI) to iteratively improve their model by local real space refinement in Coot. Conceptually, how is real space refinement performed by phenix.refine (where the free R reflections are not used to calculate the map to refine against) different from manual local real space refinement in Coot? Will the latter not also bias R-free when the default output maps from phenix.refine are used?
I would argue just the opposite- free reflections should be included in the maps used for RS refinement. Excluding free reflections is the same as setting them to zero amplitude, so you are still biasing these reflections, just biasing them toward a drastically wrong value (zero) instead of biasing them toward fobs.
If the evidence for bias is that R-free gets "better" when you include the free reflections, isn't it more because you avoid biasing them toward zero than because you do bias them toward the Fobs? In the case of density modification, which has the same problem, Ivan Rayment* showed the importance of using the "fill-in" method for missing reflections in making maps. That would seem to be the correct way of implementing R-free for density modification or real-space refinement: you don't give away your Fobs, but at the same time you don't keep forcing the free reflections toward zero. *Acta Crystallogr B. 1983 Aug;39(4):505-516. Acta Cryst. (1983). A39, 102-116
eab _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
On 05/05/2017 11:42 AM, Edward A. Berry wrote:
But it would be good to clarify that "fill_missing_f_obs = True" also fills those reflections that are not missing but are excluded due to being flagged free. Easy enough to check- it does.
exclude free: 0 11 30 1720.10 340.80 1.00 41.02 4.15 39.46 61.42 32.29 54.27 ? ? 0.38 -118.58 normal: 0 11 30 1720.10 340.80 1.00 41.02 4.15 40.12 61.79 28.53 61.79 28.53 61.79 1.60 -118.21 Both with "fill_missing_f_obs = True" 0 11 30 is present reflection flagged free. The two question marks are the no-fill coefficient. The two values before them, which are present in both files but with different values, are the "filled". (I guess everyone knew that except me)
participants (3)
-
Edward A. Berry -
Hillen, Hauke -
Tristan Croll