<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi Ed,<br>
<br>
Including free-r reflections into map calculation and then using
such map in real-space refinement of entire model will affect Rfree.
Here is a simple example that illustrates my statement,
step-by-step:<br>
<br>
1) Get data and model from PDB:<br>
<br>
<tt>phenix.fetch_pdb 1f8t --mtz</tt><br>
<br>
2) Compute two 2mFo-DFc maps: one includes all reflections the other
one has no free-r terms:<br>
<br>
<tt>phenix.python run.py 1f8t.{pdb,mtz}</tt><tt> </tt><br>
<br>
This will create an MTZ file (<tt>map_coeffs.mtz</tt>) that contains
Fourier map coefficients for both maps.<br>
<br>
3) Shake model a bit:<br>
<br>
<tt>phenix.dynamics 1f8t.pdb number_of_steps=500</tt><br>
<br>
4) Run real-space refinement using two maps:<br>
<br>
<tt>phenix.real_space_refine map_coeffs.mtz 1f8t_shaken.pdb
label="work,PHIwork" ncs_constraints=false
output.file_name_prefix=work</tt><tt><br>
</tt><tt><br>
</tt><tt>phenix.real_space_refine map_coeffs.mtz 1f8t_shaken.pdb
label="all,PHIall" ncs_constraints=false
output.file_name_prefix=all</tt><tt><br>
</tt><br>
5) Compute R-factors using data and real-space refined models:<br>
<br>
<tt>phenix.model_vs_data 1f8t.mtz all_real_space_refined.pdb</tt><tt><br>
</tt><tt> r_work(re-computed) : 0.2419 </tt><tt><br>
</tt><tt> r_free(re-computed) : 0.2441 </tt><tt><br>
</tt><br>
<tt>phenix.model_vs_data 1f8t.mtz work_real_space_refined.pdb</tt><tt><br>
</tt><tt> r_work(re-computed) : 0.2444 </tt><tt><br>
</tt><tt> r_free(re-computed) : 0.2756 </tt><tt><br>
</tt><br>
The result is self-explicable and is inline with Tom's reply to Wei.<br>
<br>
All files necessary to reproduce calculations above are here:<br>
<a class="moz-txt-link-freetext" href="http://cci.lbl.gov/~afonine/tmp/">http://cci.lbl.gov/~afonine/tmp/</a><br>
<br>
All the best,<br>
Pavel<br>
<br>
<br>
<div class="moz-cite-prefix">On 6/8/17 10:05, Tim Gruene wrote:<br>
</div>
<blockquote type="cite" cite="mid:2448094.6ZgM9WoAQT@vonneumann">
<pre wrap="">Hi Ed,
including the 'free' reflections in the map for modelling does not taint the
value of Rfree. That is a misconception that i s very persistent (as prejudice
usually are). I believe it was Ian Tickle who formulated that when you simply
refine long enough towards convergence, all reflections excluded from refinement
will become independent, i.e. you can assign a new set for Rfree every time
you refine, if you wish so.
This concept is the reason why Rcomplete (the "better" equivalent to Rfree for
small data sets with < 10,000 unique reflections), introduced by Axel Brunger,
works, as we could demonstrate in doi: 10.1073/pnas.1502136112
So nothing to worry about when including all reflections in map calculations.
Cheers,
Tim
On Thursday, June 8, 2017 12:42:53 PM CEST Edward A. Berry wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi, Tom,
Please forgive what may be a silly question from an outsider who hasn't
really kept up with the crystallography literature or even all the Phenix
newsletters- What is the evidence that including the free set in real space
refinement biases R-free of the resulting model? Is this Rfree also biased
when map coefficients use "fill-in" for the excluded free reflections (and
is that what phenix.remove_free_from_map does?).
My point is that literally excluding the free reflections, as opposed to
substituting their values with Fc, will bias the free set toward grossly
incorrect values (namely zero) and therefore greatly worsen R-free. Thus if
the evidence for bias is that you get worse R-free when you exclude the
free set, you have to think about how much of that difference results from
bias towards the observed values (when the reflections are included) and
how much is from bias towards zero (when the free set is excluded).
(Again, I realize this may be all very well understood by the
crystallography community and properly taken care of in phenix; I'm just
asking for my own information) eab
On 06/08/2017 07:28 AM, Terwilliger, Thomas Charles wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Wei,
I want to give a word of caution about how to use phenix.map_to_model on
crystallographic data...The bottom line is you should remove the test set
from your map coefficients before running phenix.map_to model on X-ray
data. Here is why:
phenix.map_to_model uses real-space refinement, which is refinement
against the map. If you supply map coefficients that include your test
reflections, then you will be refining against data that is in your test
set. This will make your Rfree invalid when you go back and refine your
model against the original crystallographic data.
To remove the test set from your map coefficients you can use:
phenix.remove_free_from_map map_coeffs=my_map_coeffs.mtz
free_in=my_data_file_with_freeR_flags.mtz
mtz_out=my_map_coeffs_no_free.mtz
Also note that phenix.map_to_model uses a fixed map (it does not do
density modification). Consequently for most crystallographic data at
moderate resolution or higher phenix.autobuild is going to do much better
than phenix.map_to_model.
All the best,
Tom T
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
--------------------------------------------------------------------------
---------------------------- *From:* <a class="moz-txt-link-abbreviated" href="mailto:dingding830106@163.com">dingding830106@163.com</a>
<a class="moz-txt-link-rfc2396E" href="mailto:dingding830106@163.com"><dingding830106@163.com></a> on behalf of <a class="moz-txt-link-abbreviated" href="mailto:dancingdream@163.com">dancingdream@163.com</a>
<a class="moz-txt-link-rfc2396E" href="mailto:dancingdream@163.com"><dancingdream@163.com></a> *Sent:* Tuesday, June 6, 2017 9:16 PM
*To:* Terwilliger, Thomas Charles
*Cc:* <a class="moz-txt-link-abbreviated" href="mailto:phenixbb@phenix-online.org">phenixbb@phenix-online.org</a>
*Subject:* Re:Re: [phenixbb] phenix.map_to_model input mtz file failure
Dear Thomas,
I use CAD to convert the labels from FDM->FWT, PHIDM->PHFWT, then submit
this job again (without map_coeffs_labels=... ), and everything seems ok.
Thank you very much for you help.
Best!
--
Wei Ding
P.O.Box 603
The Institute of Physics,Chinese Academy of Sciences
Beijing,China
100190
Tel: +86-10-82649083
E-mail: <a class="moz-txt-link-abbreviated" href="mailto:dingwei@iphy.ac.cn">dingwei@iphy.ac.cn</a> <a class="moz-txt-link-rfc2396E" href="mailto:wangli@moon.ibp.ac.cn"><mailto:wangli@moon.ibp.ac.cn></a>
At 2017-06-07 10:32:14, "Terwilliger, Thomas Charles"
</pre>
</blockquote>
</blockquote>
<pre wrap=""><a class="moz-txt-link-rfc2396E" href="mailto:terwilliger@lanl.gov"><terwilliger@lanl.gov></a> wrote:
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap=""> Hi Wei,
I'm sorry for the trouble!
If you supply an MTZ file that has FWT,PHFWT or similar labels, then
you can skip the "labels=...." statement and it should run.
Let me know if that does not work!
All the best,
Tom T
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
----------------------------------------------------------------------
---------- *From:* <a class="moz-txt-link-abbreviated" href="mailto:phenixbb-bounces@phenix-online.org">phenixbb-bounces@phenix-online.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:phenixbb-bounces@phenix-online.org"><mailto:phenixbb-bounces@phenix-online.org></a>
<<a class="moz-txt-link-abbreviated" href="mailto:phenixbb-bounces@phenix-online.org">phenixbb-bounces@phenix-online.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:phenixbb-bounces@phenix-online.org"><mailto:phenixbb-bounces@phenix-online.org></a>> on behalf of
<a class="moz-txt-link-abbreviated" href="mailto:dancingdream@163.com">dancingdream@163.com</a> <a class="moz-txt-link-rfc2396E" href="mailto:dancingdream@163.com"><mailto:dancingdream@163.com></a>
<<a class="moz-txt-link-abbreviated" href="mailto:dancingdream@163.com">dancingdream@163.com</a> <a class="moz-txt-link-rfc2396E" href="mailto:dancingdream@163.com"><mailto:dancingdream@163.com></a>> *Sent:* Tuesday,
June 6, 2017 8:19 PM
*To:* <a class="moz-txt-link-abbreviated" href="mailto:phenixbb@phenix-online.org">phenixbb@phenix-online.org</a> <a class="moz-txt-link-rfc2396E" href="mailto:phenixbb@phenix-online.org"><mailto:phenixbb@phenix-online.org></a>
*Subject:* [phenixbb] phenix.map_to_model input mtz file failure
Dear Phenix bb,
I intend to build a initial model by phenix.map_to_model. And the
command line is as follows: phenix.map_to_model_1.12rc0-2787
map_coeffs_file=../rep_dm.mtz map_coeffs_labels="'FP,SIGFP' 'PHIDM'
'FOMDM'" seq_file=../resolve.seq is_crystal=True
use_sg_symmetry=True density_select=False truncate_at_d_min=True
and the feedback like this:
Sorry: No initial assignment made for map_coeffs. Labels used:
FP,SIGFP PHIDM FOMDM. Available labels: ['PHIB', 'FOM',
'HLA,HLB,HLC,HLD', 'FP,SIGFP', 'PHIDM', 'FOMDM', 'FDM',
'HLADM,HLBDM,HLCDM,HLDDM'] NOTE: grouped labels like 'FP,SIGFP' must
stay together,
have commas, and have no spaces. If they come from an MTZ file,
they must be in adjacent columns as well.
Suggested labels to use: PHIDM FOMDM
I try many other input format of map_coeffs_labels, such as
map_coeffs_labels="FP,SIGFP PHIDM FOMDM"
map_coeffs_labels=["FP,SIGFP PHIDM FOMDM"]
... ...
but the result is the same. Dose anyone can tell me how to fix this
problem? Thank a lot.
--
Wei Ding
P.O.Box 603
The Institute of Physics,Chinese Academy of Sciences
Beijing,China
100190
Tel: +86-10-82649083
E-mail: <a class="moz-txt-link-abbreviated" href="mailto:dingwei@iphy.ac.cn">dingwei@iphy.ac.cn</a> <a class="moz-txt-link-rfc2396E" href="mailto:wangli@moon.ibp.ac.cn"><mailto:wangli@moon.ibp.ac.cn></a>
_______________________________________________
phenixbb mailing list
<a class="moz-txt-link-abbreviated" href="mailto:phenixbb@phenix-online.org">phenixbb@phenix-online.org</a>
<a class="moz-txt-link-freetext" href="http://phenix-online.org/mailman/listinfo/phenixbb">http://phenix-online.org/mailman/listinfo/phenixbb</a>
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:phenixbb-leave@phenix-online.org">phenixbb-leave@phenix-online.org</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
phenixbb mailing list
<a class="moz-txt-link-abbreviated" href="mailto:phenixbb@phenix-online.org">phenixbb@phenix-online.org</a>
<a class="moz-txt-link-freetext" href="http://phenix-online.org/mailman/listinfo/phenixbb">http://phenix-online.org/mailman/listinfo/phenixbb</a>
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:phenixbb-leave@phenix-online.org">phenixbb-leave@phenix-online.org</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
phenixbb mailing list
<a class="moz-txt-link-abbreviated" href="mailto:phenixbb@phenix-online.org">phenixbb@phenix-online.org</a>
<a class="moz-txt-link-freetext" href="http://phenix-online.org/mailman/listinfo/phenixbb">http://phenix-online.org/mailman/listinfo/phenixbb</a>
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:phenixbb-leave@phenix-online.org">phenixbb-leave@phenix-online.org</a></pre>
</blockquote>
<br>
</body>
</html>