[phenixbb] phenix.map_to_model input mtz file failure --caution on using map_to_model with X-ray data
Pavel Afonine
pafonine at lbl.gov
Tue Jun 13 11:15:51 PDT 2017
Hi Ed,
Including free-r reflections into map calculation and then using such
map in real-space refinement of entire model will affect Rfree. Here is
a simple example that illustrates my statement, step-by-step:
1) Get data and model from PDB:
phenix.fetch_pdb 1f8t --mtz
2) Compute two 2mFo-DFc maps: one includes all reflections the other one
has no free-r terms:
phenix.python run.py 1f8t.{pdb,mtz}
This will create an MTZ file (map_coeffs.mtz) that contains Fourier map
coefficients for both maps.
3) Shake model a bit:
phenix.dynamics 1f8t.pdb number_of_steps=500
4) Run real-space refinement using two maps:
phenix.real_space_refine map_coeffs.mtz 1f8t_shaken.pdb
label="work,PHIwork" ncs_constraints=false output.file_name_prefix=work
phenix.real_space_refine map_coeffs.mtz 1f8t_shaken.pdb
label="all,PHIall" ncs_constraints=false output.file_name_prefix=all
5) Compute R-factors using data and real-space refined models:
phenix.model_vs_data 1f8t.mtz all_real_space_refined.pdb
r_work(re-computed) : 0.2419
r_free(re-computed) : 0.2441
phenix.model_vs_data 1f8t.mtz work_real_space_refined.pdb
r_work(re-computed) : 0.2444
r_free(re-computed) : 0.2756
The result is self-explicable and is inline with Tom's reply to Wei.
All files necessary to reproduce calculations above are here:
http://cci.lbl.gov/~afonine/tmp/
All the best,
Pavel
On 6/8/17 10:05, Tim Gruene wrote:
> Hi Ed,
>
> including the 'free' reflections in the map for modelling does not taint the
> value of Rfree. That is a misconception that i s very persistent (as prejudice
> usually are). I believe it was Ian Tickle who formulated that when you simply
> refine long enough towards convergence, all reflections excluded from refinement
> will become independent, i.e. you can assign a new set for Rfree every time
> you refine, if you wish so.
>
> This concept is the reason why Rcomplete (the "better" equivalent to Rfree for
> small data sets with < 10,000 unique reflections), introduced by Axel Brunger,
> works, as we could demonstrate in doi: 10.1073/pnas.1502136112
>
> So nothing to worry about when including all reflections in map calculations.
>
> Cheers,
> Tim
>
> On Thursday, June 8, 2017 12:42:53 PM CEST Edward A. Berry wrote:
>> Hi, Tom,
>> Please forgive what may be a silly question from an outsider who hasn't
>> really kept up with the crystallography literature or even all the Phenix
>> newsletters- What is the evidence that including the free set in real space
>> refinement biases R-free of the resulting model? Is this Rfree also biased
>> when map coefficients use "fill-in" for the excluded free reflections (and
>> is that what phenix.remove_free_from_map does?).
>>
>> My point is that literally excluding the free reflections, as opposed to
>> substituting their values with Fc, will bias the free set toward grossly
>> incorrect values (namely zero) and therefore greatly worsen R-free. Thus if
>> the evidence for bias is that you get worse R-free when you exclude the
>> free set, you have to think about how much of that difference results from
>> bias towards the observed values (when the reflections are included) and
>> how much is from bias towards zero (when the free set is excluded).
>> (Again, I realize this may be all very well understood by the
>> crystallography community and properly taken care of in phenix; I'm just
>> asking for my own information) eab
>>
>> On 06/08/2017 07:28 AM, Terwilliger, Thomas Charles wrote:
>>> Hi Wei,
>>>
>>>
>>> I want to give a word of caution about how to use phenix.map_to_model on
>>> crystallographic data...The bottom line is you should remove the test set
>>> from your map coefficients before running phenix.map_to model on X-ray
>>> data. Here is why:
>>>
>>>
>>> phenix.map_to_model uses real-space refinement, which is refinement
>>> against the map. If you supply map coefficients that include your test
>>> reflections, then you will be refining against data that is in your test
>>> set. This will make your Rfree invalid when you go back and refine your
>>> model against the original crystallographic data.
>>>
>>>
>>> To remove the test set from your map coefficients you can use:
>>>
>>>
>>> phenix.remove_free_from_map map_coeffs=my_map_coeffs.mtz
>>> free_in=my_data_file_with_freeR_flags.mtz
>>> mtz_out=my_map_coeffs_no_free.mtz
>>>
>>>
>>> Also note that phenix.map_to_model uses a fixed map (it does not do
>>> density modification). Consequently for most crystallographic data at
>>> moderate resolution or higher phenix.autobuild is going to do much better
>>> than phenix.map_to_model.
>>>
>>>
>>> All the best,
>>>
>>> Tom T
>>>
>>>
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> --------------------------------------------------------------------------
>>> ---------------------------- *From:* dingding830106 at 163.com
>>> <dingding830106 at 163.com> on behalf of dancingdream at 163.com
>>> <dancingdream at 163.com> *Sent:* Tuesday, June 6, 2017 9:16 PM
>>> *To:* Terwilliger, Thomas Charles
>>> *Cc:* phenixbb at phenix-online.org
>>> *Subject:* Re:Re: [phenixbb] phenix.map_to_model input mtz file failure
>>> Dear Thomas,
>>> I use CAD to convert the labels from FDM->FWT, PHIDM->PHFWT, then submit
>>> this job again (without map_coeffs_labels=... ), and everything seems ok.
>>> Thank you very much for you help.
>>> Best!
>>>
>>>
>>> --
>>> Wei Ding
>>> P.O.Box 603
>>> The Institute of Physics,Chinese Academy of Sciences
>>> Beijing,China
>>> 100190
>>> Tel: +86-10-82649083
>>>
>>> E-mail: dingwei at iphy.ac.cn <mailto:wangli at moon.ibp.ac.cn>
>>>
>>> At 2017-06-07 10:32:14, "Terwilliger, Thomas Charles"
> <terwilliger at lanl.gov> wrote:
>>> Hi Wei,
>>>
>>>
>>> I'm sorry for the trouble!
>>>
>>>
>>> If you supply an MTZ file that has FWT,PHFWT or similar labels, then
>>> you can skip the "labels=...." statement and it should run.
>>>
>>>
>>> Let me know if that does not work!
>>> All the best,
>>>
>>> Tom T
>>>
>>>
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ---------- *From:* phenixbb-bounces at phenix-online.org
>>> <mailto:phenixbb-bounces at phenix-online.org>
>>> <phenixbb-bounces at phenix-online.org
>>> <mailto:phenixbb-bounces at phenix-online.org>> on behalf of
>>> dancingdream at 163.com <mailto:dancingdream at 163.com>
>>> <dancingdream at 163.com <mailto:dancingdream at 163.com>> *Sent:* Tuesday,
>>> June 6, 2017 8:19 PM
>>> *To:* phenixbb at phenix-online.org <mailto:phenixbb at phenix-online.org>
>>> *Subject:* [phenixbb] phenix.map_to_model input mtz file failure
>>> Dear Phenix bb,
>>> I intend to build a initial model by phenix.map_to_model. And the
>>> command line is as follows: phenix.map_to_model_1.12rc0-2787
>>> map_coeffs_file=../rep_dm.mtz map_coeffs_labels="'FP,SIGFP' 'PHIDM'
>>> 'FOMDM'" seq_file=../resolve.seq is_crystal=True
>>> use_sg_symmetry=True density_select=False truncate_at_d_min=True
>>> and the feedback like this:
>>> Sorry: No initial assignment made for map_coeffs. Labels used:
>>> FP,SIGFP PHIDM FOMDM. Available labels: ['PHIB', 'FOM',
>>> 'HLA,HLB,HLC,HLD', 'FP,SIGFP', 'PHIDM', 'FOMDM', 'FDM',
>>> 'HLADM,HLBDM,HLCDM,HLDDM'] NOTE: grouped labels like 'FP,SIGFP' must
>>> stay together,
>>> have commas, and have no spaces. If they come from an MTZ file,
>>> they must be in adjacent columns as well.
>>> Suggested labels to use: PHIDM FOMDM
>>> I try many other input format of map_coeffs_labels, such as
>>> map_coeffs_labels="FP,SIGFP PHIDM FOMDM"
>>> map_coeffs_labels=["FP,SIGFP PHIDM FOMDM"]
>>> ... ...
>>> but the result is the same. Dose anyone can tell me how to fix this
>>> problem? Thank a lot.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Wei Ding
>>> P.O.Box 603
>>> The Institute of Physics,Chinese Academy of Sciences
>>> Beijing,China
>>> 100190
>>> Tel: +86-10-82649083
>>> E-mail: dingwei at iphy.ac.cn <mailto:wangli at moon.ibp.ac.cn>
>>>
>>> _______________________________________________
>>> phenixbb mailing list
>>> phenixbb at phenix-online.org
>>> http://phenix-online.org/mailman/listinfo/phenixbb
>>> Unsubscribe: phenixbb-leave at phenix-online.org
>> _______________________________________________
>> phenixbb mailing list
>> phenixbb at phenix-online.org
>> http://phenix-online.org/mailman/listinfo/phenixbb
>> Unsubscribe: phenixbb-leave at phenix-online.org
>
>
> _______________________________________________
> phenixbb mailing list
> phenixbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/phenixbb
> Unsubscribe: phenixbb-leave at phenix-online.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/phenixbb/attachments/20170613/edccec00/attachment.htm>
More information about the phenixbb
mailing list