Re: [phenixbb] phenix.map_to_model input mtz file failure --caution on using map_to_model with X-ray data

Hi Ed, including the 'free' reflections in the map for modelling does not taint the value of Rfree. That is a misconception that i s very persistent (as prejudice usually are). I believe it was Ian Tickle who formulated that when you simply refine long enough towards convergence, all reflections excluded from refinement will become independent, i.e. you can assign a new set for Rfree every time you refine, if you wish so. This concept is the reason why Rcomplete (the "better" equivalent to Rfree for small data sets with < 10,000 unique reflections), introduced by Axel Brunger, works, as we could demonstrate in doi: 10.1073/pnas.1502136112 So nothing to worry about when including all reflections in map calculations. Cheers, Tim On Thursday, June 8, 2017 12:42:53 PM CEST Edward A. Berry wrote:

Hi, Tom, Please forgive what may be a silly question from an outsider who hasn't really kept up with the crystallography literature or even all the Phenix newsletters- What is the evidence that including the free set in real space refinement biases R-free of the resulting model? Is this Rfree also biased when map coefficients use "fill-in" for the excluded free reflections (and is that what phenix.remove_free_from_map does?).

My point is that literally excluding the free reflections, as opposed to substituting their values with Fc, will bias the free set toward grossly incorrect values (namely zero) and therefore greatly worsen R-free. Thus if the evidence for bias is that you get worse R-free when you exclude the free set, you have to think about how much of that difference results from bias towards the observed values (when the reflections are included) and how much is from bias towards zero (when the free set is excluded). (Again, I realize this may be all very well understood by the crystallography community and properly taken care of in phenix; I'm just asking for my own information) eab

On 06/08/2017 07:28 AM, Terwilliger, Thomas Charles wrote:

...

​Hi Wei,

I want to give a word of caution about how to use phenix.map_to_model on crystallographic data...The bottom line is you should remove the test set from your map coefficients before running phenix.map_to model on X-ray data. Here is why:

phenix.map_to_model uses real-space refinement, which is refinement against the map. If you supply map coefficients that include your test reflections, then you will be refining against data that is in your test set. This will make your Rfree invalid when you go back and refine your model against the original crystallographic data.

To remove the test set from your map coefficients you can use:

phenix.remove_free_from_map map_coeffs=my_map_coeffs.mtz free_in=my_data_file_with_freeR_flags.mtz mtz_out=my_map_coeffs_no_free.mtz

Also note that phenix.map_to_model uses a fixed map (it does not do density modification). Consequently for most crystallographic data at moderate resolution or higher phenix.autobuild is going to do much better than phenix.map_to_model.

All the best,

Tom T

-------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- -------------------------------------------------------------------------- ---------------------------- *From:* [email protected] on behalf of [email protected] *Sent:* Tuesday, June 6, 2017 9:16 PM *To:* Terwilliger, Thomas Charles *Cc:* [email protected] *Subject:* Re:Re: [phenixbb] phenix.map_to_model input mtz file failure Dear Thomas, I use CAD to convert the labels from FDM->FWT, PHIDM->PHFWT, then submit this job again (without map_coeffs_labels=... ), and everything seems ok. Thank you very much for you help. Best!

-- Wei Ding P.O.Box 603 The Institute of Physics,Chinese Academy of Sciences Beijing,China 100190 Tel: +86-10-82649083

E-mail: [email protected] mailto:[email protected]

At 2017-06-07 10:32:14, "Terwilliger, Thomas Charles" wrote: Hi Wei,

I'm sorry for the trouble!

If you supply an MTZ file that has FWT,PHFWT or similar labels, then you can skip the "labels=...." statement and it should run.

Let me know if that does not work! All the best,

Tom T

---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------------------------------------------------------------------- ---------- *From:* [email protected] mailto:[email protected] mailto:[email protected]> on behalf of [email protected] mailto:[email protected] mailto:[email protected]> *Sent:* Tuesday, June 6, 2017 8:19 PM *To:* [email protected] mailto:[email protected] *Subject:* [phenixbb] phenix.map_to_model input mtz file failure Dear Phenix bb, I intend to build a initial model by phenix.map_to_model. And the command line is as follows: phenix.map_to_model_1.12rc0-2787 map_coeffs_file=../rep_dm.mtz map_coeffs_labels="'FP,SIGFP' 'PHIDM' 'FOMDM'" seq_file=../resolve.seq is_crystal=True use_sg_symmetry=True density_select=False truncate_at_d_min=True and the feedback like this: Sorry: No initial assignment made for map_coeffs. Labels used: FP,SIGFP PHIDM FOMDM. Available labels: ['PHIB', 'FOM', 'HLA,HLB,HLC,HLD', 'FP,SIGFP', 'PHIDM', 'FOMDM', 'FDM', 'HLADM,HLBDM,HLCDM,HLDDM'] NOTE: grouped labels like 'FP,SIGFP' must stay together, have commas, and have no spaces. If they come from an MTZ file, they must be in adjacent columns as well. Suggested labels to use: PHIDM FOMDM I try many other input format of map_coeffs_labels, such as map_coeffs_labels="FP,SIGFP PHIDM FOMDM" map_coeffs_labels=["FP,SIGFP PHIDM FOMDM"] ... ... but the result is the same. Dose anyone can tell me how to fix this problem? Thank a lot.

-- Wei Ding P.O.Box 603 The Institute of Physics,Chinese Academy of Sciences Beijing,China 100190 Tel: +86-10-82649083 E-mail: [email protected] mailto:[email protected]

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]

-- -- Paul Scherrer Institut Tim Gruene - persoenlich - OFLC/104 CH-5232 Villigen PSI phone: +41 (0)56 310 5297 GPG Key ID = A46BEE1A