What I have been suggesting to the fine EM people for a while, is to put in the correct scattering cross sections, and refine against phased structure factors in reciprocal space with R-free. One could fairly easily weight phases more than amplitudes with one more tunable parameter. Use R-free and this will get you most of the way there. Seems this may be catching on a bit. George On Feb 17, 2014, at 9:33 PM, Pete Meyer wrote:

Pavel Afonine wrote:

...

One is convert your EM map into structure factors Fobs and HL coefficients using phenix.map_to_structure_factors command and then use phenix.refine to do reciprocal space refinement. This is somewhat indirect way. The good thing about it is that phenix.refine has a broad range of tools to handle low resolution refinement.

Doing reciprocal space refinement for EM strikes me as something that might be problematic, at least from the theoretical sense. The scattering factors going to Fcalc are assuming x-ray scattering, which might not be appropriate for an inverse-transformed EM map. The weighting factors (Fobs:Fcalc agreement - not x-ray:geometry) assume that there's greater uncertainty in the phases than the amplitudes (at least as I understand it); for inverted EM the uncertainty for both components should be comparable. Bulk solvent correction might also need to be tweaked (or disabled), but that's a less-fundamental issue.

That said, I have seen a couple papers where the authors refined against an inverted EM map (and included R/Rfree and FOM statistics) that the reviewers must not have had any problems with. So it's very possible that my impression is incorrect.

Pete

...

Another option is to use phenix.real_space_refine to refine model against the map directly. This is a preferred way but phenix.real_space_refine is new, less mature program and it generally is still under development (though it's functional if you try it now). So if it does not work as you expected or you have any feature requests or any questions or any other feedback - please do not hesitate to let me know, as this may help to improve this tool. All the best, Pavel On 2/17/14, 6:14 AM, Agbandje-Mckenna,Mavis wrote:

...

Dear Pavel,

Thank you so much for getting back to me - this is much appreciated. We received a script from Hong Zhou's group who have been using Phenix to refine the model for their 3.3 Angs cryo-EM Aquareovirus structure that we were going to try but this seems a lot more simpler!

Thanks again. I will let you know how it turns out.

And thank you for the attached pdf.

Best wishes

Mavis

Mavis Agbandje-McKenna, Ph.D. Director, Center for Structural Biology Professor, Dept. of Biochemistry and Molecular Biology Center for Structural Biology The McKnight Brain Institute P.O. Box 100245 Tel.: 1-352-392-5694 University of Florida Fax.: 1-352-392-3422 Gainesville, FL 32610-0245 e-mail: [email protected] Website: http://msg.mbi.ufl.edu ________________________________________ From: Pavel Afonine [[email protected]] Sent: Monday, February 17, 2014 2:28 AM To: Agbandje-Mckenna,Mavis; [email protected] Cc: Lins, Bridget; Drouin,Lauren M Subject: Re: Real space refinement

Hi Mavis,

you can do it in Phenix. Command line tool for this is: phenix.real_space_refine. Just give it map and model and see what happens, like this:

phenix.real_space_refine model.pdb ccp4_formatted_map.map

I wrote this program specifically to do various kinds of model refinement against the map, with the main focus on EM maps. Have a look at the article that briefly describes it here: http://www.phenix-online.org/newsletter/CCN_2013_07.pdf "New tool: phenix.real_space_refine". The article is from last year but I haven't done much since then so it pretty much documents the current state of the art.

Should you have any questions or problems or need any help with this please let me know off list.

All the best, Pavel

On 2/14/14, 8:45 AM, Agbandje-Mckenna,Mavis wrote:

...

Dear Pavel,

I don't believe we have met, but you have received several e-mails from people in my group (including Sujata (Tina) Halder) and also helped them with their use of Phenix - thank you.

I have a question for you regarding real space refinement in Phenix. We have a 3.7 Angs cryo-EM reconstructed structure for a virus capsid and we have been able to build a model for the capsid VP using COOT. I would like to refine this model further against the map using real space refinement. I am trying to find a suitable program that would allow me to do this. I believe Phenix can do it, but I am not sure. Could you give me some direction?

Thanks in advance.

Best wishes

Mavis

Mavis Agbandje-McKenna, Ph.D. Director, Center for Structural Biology Professor, Dept. of Biochemistry and Molecular Biology Center for Structural Biology The McKnight Brain Institute P.O. Box 100245 Tel.: 1-352-392-5694 University of Florida Fax.: 1-352-392-3422 Gainesville, FL 32610-0245 e-mail: [email protected] Website: http://msg.mbi.ufl.edu

---

phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

George N. Phillips, Jr. Ralph and Dorothy Looney Professor Department of Biochemistry and Cell Biology Rice University, MS140 6100 Main Street Houston, Texas 77005-1892 Office: +1 713.348.6951 http://www.bioc.rice.edu/

It doesn't matter that much which space one does the refinement in does it? But cross validation is critical while the methods are being developed. Are there good real space cross validation methods? George by phone

On Feb 17, 2014, at 10:19 PM, Pavel Afonine wrote:

Hi Pete,

these are all good points. Refining against EM data in reciprocal space is just a dirty work-around because refinement programs are all designed to work with X-ray or neutron data, which are Fobs or Iobs, and because of lack of similarly powerful real-space refinement programs.

If your data is the map ideally you should refine your model against the map without having that questionable detour of converting the map into structure factors. Also, many refinement strategies and algorithms can be done differently: for example, one case take a great advantage of local refinement techniques (such as fitting rotamers, morphing, etc..etc..)

All in all that's exactly why I started phenix.real_space_refine tool to efficiently put together all real-space refinement tools we have in CCTBX and Phenix. So once phenix.real_space_refine is fully functional (hopefully soon!) I would suggest we abandon the option of refining against EM map in reciprocal space.

All the best, Pavel

...

On 2/17/14, 7:33 PM, Pete Meyer wrote: Pavel Afonine wrote:

...

One is convert your EM map into structure factors Fobs and HL coefficients using phenix.map_to_structure_factors command and then use phenix.refine to do reciprocal space refinement. This is somewhat indirect way. The good thing about it is that phenix.refine has a broad range of tools to handle low resolution refinement.

Doing reciprocal space refinement for EM strikes me as something that might be problematic, at least from the theoretical sense. The scattering factors going to Fcalc are assuming x-ray scattering, which might not be appropriate for an inverse-transformed EM map. The weighting factors (Fobs:Fcalc agreement - not x-ray:geometry) assume that there's greater uncertainty in the phases than the amplitudes (at least as I understand it); for inverted EM the uncertainty for both components should be comparable. Bulk solvent correction might also need to be tweaked (or disabled), but that's a less-fundamental issue.

That said, I have seen a couple papers where the authors refined against an inverted EM map (and included R/Rfree and FOM statistics) that the reviewers must not have had any problems with. So it's very possible that my impression is incorrect.

Pete

...

Another option is to use phenix.real_space_refine to refine model against the map directly. This is a preferred way but phenix.real_space_refine is new, less mature program and it generally is still under development (though it's functional if you try it now). So if it does not work as you expected or you have any feature requests or any questions or any other feedback - please do not hesitate to let me know, as this may help to improve this tool.

All the best, Pavel

...

On 2/17/14, 6:14 AM, Agbandje-Mckenna,Mavis wrote: Dear Pavel,

Thank you so much for getting back to me - this is much appreciated. We received a script from Hong Zhou's group who have been using Phenix to refine the model for their 3.3 Angs cryo-EM Aquareovirus structure that we were going to try but this seems a lot more simpler!

Thanks again. I will let you know how it turns out.

And thank you for the attached pdf.

Best wishes

Mavis

Mavis Agbandje-McKenna, Ph.D. Director, Center for Structural Biology Professor, Dept. of Biochemistry and Molecular Biology Center for Structural Biology The McKnight Brain Institute P.O. Box 100245 Tel.: 1-352-392-5694 University of Florida Fax.: 1-352-392-3422 Gainesville, FL 32610-0245 e-mail: [email protected] Website: http://msg.mbi.ufl.edu ________________________________________ From: Pavel Afonine [[email protected]] Sent: Monday, February 17, 2014 2:28 AM To: Agbandje-Mckenna,Mavis; [email protected] Cc: Lins, Bridget; Drouin,Lauren M Subject: Re: Real space refinement

Hi Mavis,

you can do it in Phenix. Command line tool for this is: phenix.real_space_refine. Just give it map and model and see what happens, like this:

phenix.real_space_refine model.pdb ccp4_formatted_map.map

I wrote this program specifically to do various kinds of model refinement against the map, with the main focus on EM maps. Have a look at the article that briefly describes it here: http://www.phenix-online.org/newsletter/CCN_2013_07.pdf "New tool: phenix.real_space_refine". The article is from last year but I haven't done much since then so it pretty much documents the current state of the art.

Should you have any questions or problems or need any help with this please let me know off list.

All the best, Pavel

...

On 2/14/14, 8:45 AM, Agbandje-Mckenna,Mavis wrote: Dear Pavel,

I don't believe we have met, but you have received several e-mails from people in my group (including Sujata (Tina) Halder) and also helped them with their use of Phenix - thank you.

I have a question for you regarding real space refinement in Phenix. We have a 3.7 Angs cryo-EM reconstructed structure for a virus capsid and we have been able to build a model for the capsid VP using COOT. I would like to refine this model further against the map using real space refinement. I am trying to find a suitable program that would allow me to do this. I believe Phenix can do it, but I am not sure. Could you give me some direction?

Thanks in advance.

Best wishes

Mavis

Mavis Agbandje-McKenna, Ph.D. Director, Center for Structural Biology Professor, Dept. of Biochemistry and Molecular Biology Center for Structural Biology The McKnight Brain Institute P.O. Box 100245 Tel.: 1-352-392-5694 University of Florida Fax.: 1-352-392-3422 Gainesville, FL 32610-0245 e-mail: [email protected] Website: http://msg.mbi.ufl.edu

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Hi George,

It doesn't matter that much which space one does the refinement in does it?

indeed, it does not matter if we are talking about real-space refinement as minimization of SUM(rho_obs-rho_calc)**2 instead of its reciprocal space equivalent SUM(Fobs-Fcalc)**2 . But I think it does matter a lot if we think of real-space refinement as an accumulation of various methods. In real-space you can afford doing things a) locally and b) if needed. Therefore you can use optimization tools that have way greater convergence radius. For example, instead of gradient-driven minimization you can use such tools as sampling in torsion angle space to fit rotamers or use morphing. You can identify regions that need refinement and refine only these regions. It could be good runtime saving if we are talking about a handful of residues out of whole ribosome. You can utilize various local repacking techniques. And so on, and so on... This is exactly what is being implemented in phenix.real_space_refine. Pavel