[phenixbb] refinement of an ensemble of structures -> cryoEM variability

Pavel Afonine pafonine at lbl.gov
Thu Jan 13 09:42:52 PST 2022 

> How is that different from running parallel jobs of n models against n 
> maps?

For this you need to know how to write programs/scripts (I know you do 
know!). Then make sure programs down the road can handle multi-model 
files (in Phenix we can read/write them but that's pretty much it).

If you have one map and one model (with altlocs) then you can just use 
phenix.real_space_refine as is.

Otherwise yes, you can refine n models against n maps, in parallel or 
sequentially!

Pavel


>
> On 1/13/2022 8:17 AM, Oliver Clarke wrote:
>> Hi,
>>
>> Just to add my two cents, I agree this would be really useful for a 
>> lot of folks. Analysis of continuously distributed variability is 
>> very common these days in cryoEM, and having a way to jointly refine 
>> an ensemble of models against a series of maps would be very handy. 
>> Cryodrgn, 3D-VA in cryosparc, ManifoldEM, multibody refinement in 
>> relion - there are many tools now for generating a series of density 
>> maps potentially corresponding to conformational modes, so having the 
>> capacity in phenix to refine models against each map would be very 
>> helpful.
>>
>> Cheers
>> Oli
>>
>>>
>>> Message: 1
>>> Date: Wed, 12 Jan 2022 10:36:27 +0100
>>> From: vincent Chaptal <vincent.chaptal at ibcp.fr>
>>> To: Guillaume Gaullier <guillaume.gaullier at icm.uu.se>,    Pavel Afonine
>>>     <pafonine at lbl.gov>
>>> Cc: PHENIX user mailing list <phenixbb at phenix-online.org>
>>> Subject: Re: [phenixbb] refinement of an ensemble of structures ->
>>>     cryoEM variability
>>> Message-ID: <ef1a3a20-a0dc-57b0-fcc3-48a9426ce2c3 at ibcp.fr>
>>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>>>
>>> Hi Guillaume,
>>>
>>> thanks for the backup.
>>> It's exactly my feeling also.
>>>
>>> Best
>>> Vincent
>>>
>>> Le 12/01/2022 ? 10:09, Guillaume Gaullier a ?crit?:
>>>> Hi,
>>>>
>>>> I am guessing what we are talking about here are the maps generated by
>>>> cryoSPARC 3D variability analysis. See: Punjani A & Fleet DJ (2021) 3D
>>>> Variability Analysis: Resolving continuous flexibility and discrete
>>>> heterogeneity from single particle cryo-EM.?Journal of Structural
>>>> Biology: 107702 https://doi.org/10.1016/j.jsb.2021.107702
>>>>
>>>> But this is not the only program that generates series of maps to
>>>> describe continuous heterogeneity from single-particle cryoEM images,
>>>> see also cryoDRGN:?Zhong ED, Bepler T, Berger B & Davis JH (2021)
>>>> CryoDRGN: reconstruction of heterogeneous cryo-EM structures using
>>>> neural networks.?Nature Methods: 1?10
>>>> https://doi.org/10.1038/s41592-020-01049-4
>>>>
>>>> Except some ideally rigid particles like apoferritin, pretty much
>>>> everything shows some degree of flexibility that generates continuous
>>>> heterogeneity in cryoEM images. So, my feeling is that a user-friendly
>>>> program to fit a series of models (ideally, auto-generated from a
>>>> single starting model) to a map series is probably going to be a
>>>> standard requirement pretty soon for typical single-particle cryoEM
>>>> projects.
>>>>
>>>> Cheers,
>>>>
>>>> Guillaume
>>>>
>>>>
>>>>> On 11 Jan 2022, at 17:16, Pavel Afonine <pafonine at lbl.gov> wrote:
>>>>>
>>>>> Hi Vincent,
>>>>>
>>>>> this looks like a very specialized task that I've never heard of
>>>>> before! We can add a tool to do that if this becomes something that
>>>>> more than one person does more than once. Meanwhile, a simple script
>>>>> in a language of your?preference (python, linux shell, etc) should do
>>>>> the job. I can help with a script if needed, let me know!
>>>>>
>>>>> Also.. just curious -- what is "3D variability of this map"? Is this
>>>>> one map that is a composition of several map or an ensemble of maps?
>>>>>
>>>>> Pavel
>>>>>
>>>>> On 1/11/22 01:48, vincent Chaptal wrote:
>>>>>> Hi Phenix-ers,
>>>>>>
>>>>>> I thought to ask for something that I believe you have already
>>>>>> implemented, but I'm not sure of the best tool to use.
>>>>>>
>>>>>> I have a cryoEM map where I refine my "high resolution" structure. I
>>>>>> also have the 3D variability of this map that shows?several maps
>>>>>> varying around the consensus high-res map. I want to refine an
>>>>>> ensemble (20) of structures, one for every 20?maps around the
>>>>>> consensus map.
>>>>>> Is there a tool in phenix to do this?
>>>>>>
>>>>>> I could refine individually the high-res structure into each map
>>>>>> incrementally; since every map differs a little from the?original
>>>>>> one, Real-space-refinement could move the structure a little at a
>>>>>> time. Then I could combine all the PDBs in an?ensemble?
>>>>>> A tool to refine variability would be very useful. Input could be a
>>>>>> PDB and an ensemble of maps, and output would be all the?PDBs 
>>>>>> combined?
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> All the best
>>>>>> Vincent
>>>>>>
>>>>>>
>>>>>> -- 
>>>>>> Vincent Chaptal, PhD
>>>>>> Director of GdR APPICOM
>>>>>> Drug Resistance and Membrane Proteins Lab
>>>>>>
>>>>>> MMSB -UMR5086
>>>>>> 7 passage du Vercors
>>>>>> 69007 LYON
>>>>>> FRANCE
>>>>>> +33 4 37 65 29 01
>>>>>> http://www.appicom.cnrs.fr
>>>>>> http://mmsb.cnrs.fr/en/
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> phenixbb mailing list
>>>>>>
>>>>>> phenixbb at phenix-online.org
>>>>>> http://phenix-online.org/mailman/listinfo/phenixbb
>>>>>>
>>>>>> Unsubscribe:
>>>>>> phenixbb-leave at phenix-online.org
>>>>> _______________________________________________
>>>>> phenixbb mailing list
>>>>> phenixbb at phenix-online.org
>>>>> http://phenix-online.org/mailman/listinfo/phenixbb
>>>>> Unsubscribe: phenixbb-leave at phenix-online.org
>>>> Page Title
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> N?r du har kontakt med oss p? Uppsala universitet med e-post s?
>>>> inneb?r det att vi behandlar dina personuppgifter. F?r att l?sa mer om
>>>> hur vi g?r det kan du l?sa h?r:
>>>> http://www.uu.se/om-uu/dataskydd-personuppgifter/
>>>>
>>>> E-mailing Uppsala University means that we will process your personal
>>>> data. For more information on how this is performed, please read here:
>>>> http://www.uu.se/en/about-uu/data-protection-policy
>>> -- 
>>>
>>> Vincent Chaptal, PhD
>>>
>>> Director of GdR APPICOM
>>>
>>> Drug Resistance and Membrane Proteins Lab
>>>
>>>
>>> MMSB -UMR5086
>>>
>>> 7 passage du Vercors
>>>
>>> 69007 LYON
>>>
>>> FRANCE
>>>
>>> +33 4 37 65 29 01
>>>
>>> http://www.appicom.cnrs.fr
>>>
>>> http://mmsb.cnrs.fr/en/
>>>
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL: 
>>> <http://phenix-online.org/pipermail/phenixbb/attachments/20220112/e891a7ad/attachment-0001.htm>
>>>
>>> ------------------------------
>>>
>>> Message: 2
>>> Date: Wed, 12 Jan 2022 18:11:02 +0530
>>> From: Viney Singh <vineysingh22 at gmail.com>
>>> To: phenixbb at phenix-online.org
>>> Subject: [phenixbb] regarding number of unique reflections
>>> Message-ID:
>>>     <CALO_hTx1dtmm2nfkd4+9iPUb+jmAq4kM2ObB=jwrQ5VvCuUTmQ at mail.gmail.com> 
>>>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Dear all,
>>>
>>> First of all sorry for the novice question.
>>>
>>> I *processed* one of my datasets using xds. Since I was not 
>>> expecting an
>>> anomalous signal, I kept Friedel's Law = True.
>>> The number of unique reflections I got: *17,000*
>>> I used xds_ASCII.HKL after correct.LP and converted it to mtz using 
>>> Phenix
>>> with 10% reflections for Rfree calculations.
>>>
>>> Now, when I am trying to* refine *the structure in Phenix, the 
>>> structure is
>>> being refined against *32,300* reflections with *3230* reflections for
>>> Rfree calculation. Looks like refinement is treating Friedel's pair 
>>> as two
>>> different reflections.
>>>
>>> When I am trying to upload PDB on rcsb, on the refinement tab, the 
>>> number
>>> of reflections used for refinement and Rfree calculations are shown as
>>> 32,300 and 3220 respectively, while in the validation report, no. of
>>> reflections used for Rfree calculation are shown as *1700*.
>>>
>>> I would really appreciate if someone can guide me to resolve this
>>> discrepancy.
>>>
>>> Thanks in advance.
>>>
>>> Viney
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL: 
>>> <http://phenix-online.org/pipermail/phenixbb/attachments/20220112/f1d16873/attachment-0001.htm>
>>>
>>> ------------------------------
>>>
>>> Message: 3
>>> Date: Wed, 12 Jan 2022 15:03:49 +0000
>>> From: "Tanner, John J." <TannerJJ at missouri.edu>
>>> To: Viney Singh <vineysingh22 at gmail.com>, "phenixbb at phenix-online.org"
>>>     <phenixbb at phenix-online.org>
>>> Subject: Re: [phenixbb] regarding number of unique reflections
>>> Message-ID:
>>>     <CH0PR01MB7154E88CABD14AE39C4FD543A7529 at CH0PR01MB7154.prod.exchangelabs.com> 
>>>
>>>
>>> Content-Type: text/plain; charset="us-ascii"
>>>
>>> This is a common problem. I use ccp4i CAD to remove all the columns 
>>> from the .mtz file except for F, SIGF, and FreeR_flag, and then use 
>>> this new .mtz file in refinement before depositing. For the next 
>>> structure, remember to do this in the early stages.
>>>
>>> -- 
>>> John J. Tanner
>>> Professor of Biochemistry and Chemistry
>>> Associate Chair of Biochemistry
>>> Department of Biochemistry
>>> University of Missouri
>>> 117 Schweitzer Hall
>>> 503 S College Avenue
>>> Columbia, MO 65211
>>> Phone: 573-884-1280
>>> Email: tannerjj at missouri.edu<mailto:tannerjj at missouri.edu>
>>> https://cafnrfaculty.missouri.edu/tannerlab/
>>> Lab: Schlundt Annex rooms 3,6,9, 203B, 203C
>>> Office: Schlundt Annex 203A
>>>
>>>
>>>
>>>
>>> From: phenixbb-bounces at phenix-online.org 
>>> <phenixbb-bounces at phenix-online.org> on behalf of Viney Singh 
>>> <vineysingh22 at gmail.com>
>>> Date: Wednesday, January 12, 2022 at 6:52 AM
>>> To: phenixbb at phenix-online.org <phenixbb at phenix-online.org>
>>> Subject: [phenixbb] regarding number of unique reflections
>>> WARNING: This message has originated from an External Source. This 
>>> may be a phishing expedition that can result in unauthorized access 
>>> to our IT System. Please use proper judgment and caution when 
>>> opening attachments, clicking links, or responding to this email.
>>> Dear all,
>>>
>>> First of all sorry for the novice question.
>>>
>>> I processed one of my datasets using xds. Since I was not expecting 
>>> an anomalous signal, I kept Friedel's Law = True.
>>> The number of unique reflections I got: 17,000
>>> I used xds_ASCII.HKL after correct.LP and converted it to mtz using 
>>> Phenix with 10% reflections for Rfree calculations.
>>>
>>> Now, when I am trying to refine the structure in Phenix, the 
>>> structure is being refined against 32,300 reflections with 3230 
>>> reflections for Rfree calculation. Looks like refinement is treating 
>>> Friedel's pair as two different reflections.
>>>
>>> When I am trying to upload PDB on rcsb, on the refinement tab, the 
>>> number of reflections used for refinement and Rfree calculations are 
>>> shown as 32,300 and 3220 respectively, while in the validation 
>>> report, no. of reflections used for Rfree calculation are shown as 
>>> 1700.
>>>
>>> I would really appreciate if someone can guide me to resolve this 
>>> discrepancy.
>>>
>>> Thanks in advance.
>>>
>>> Viney
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL: 
>>> <http://phenix-online.org/pipermail/phenixbb/attachments/20220112/70d938d1/attachment-0001.htm>
>>>
>>> ------------------------------
>>>
>>> Message: 4
>>> Date: Wed, 12 Jan 2022 21:28:37 +0530
>>> From: Viney Singh <vineysingh22 at gmail.com>
>>> To: "Tanner, John J." <TannerJJ at missouri.edu>
>>> Cc: "phenixbb at phenix-online.org" <phenixbb at phenix-online.org>
>>> Subject: Re: [phenixbb] regarding number of unique reflections
>>> Message-ID:
>>>     <CALO_hTyJj3bSg_rev1hZVEvr=K91+C9x1AtbdrpiOajsSKk-Hg at mail.gmail.com> 
>>>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Thanks a lot, Dr .Tanner. I will try the suggestion. Currently, I tried
>>> running xds_ASCII.HKL on pointless followed by truncate. Seems like 
>>> mtz is
>>> now all good. But I am wondering if this is the right way to go 
>>> about it?
>>>
>>>
>>>
>>> On Wed, Jan 12, 2022 at 8:33 PM Tanner, John J. <TannerJJ at missouri.edu>
>>> wrote:
>>>
>>>> This is a common problem. I use ccp4i CAD to remove all the columns 
>>>> from
>>>> the .mtz file except for F, SIGF, and FreeR_flag, and then use this 
>>>> new
>>>> .mtz file in refinement before depositing. For the next structure, 
>>>> remember
>>>> to do this in the early stages.
>>>>
>>>>
>>>>
>>>> -- 
>>>>
>>>> John J. Tanner
>>>>
>>>> Professor of Biochemistry and Chemistry
>>>>
>>>> Associate Chair of Biochemistry
>>>>
>>>> Department of Biochemistry
>>>>
>>>> University of Missouri
>>>> 117 Schweitzer Hall
>>>>
>>>> 503 S College Avenue
>>>> Columbia, MO 65211
>>>> Phone: 573-884-1280
>>>>
>>>> Email: tannerjj at missouri.edu <tannerjj at missouri.edu>
>>>> https://cafnrfaculty.missouri.edu/tannerlab/
>>>>
>>>> Lab: Schlundt Annex rooms 3,6,9, 203B, 203C
>>>>
>>>> Office: Schlundt Annex 203A
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From: *phenixbb-bounces at phenix-online.org <
>>>> phenixbb-bounces at phenix-online.org> on behalf of Viney Singh <
>>>> vineysingh22 at gmail.com>
>>>> *Date: *Wednesday, January 12, 2022 at 6:52 AM
>>>> *To: *phenixbb at phenix-online.org <phenixbb at phenix-online.org>
>>>> *Subject: *[phenixbb] regarding number of unique reflections
>>>>
>>>> *WARNING:* This message has originated from an External Source. 
>>>> This may
>>>> be a phishing expedition that can result in unauthorized access to 
>>>> our IT
>>>> System. Please use proper judgment and caution when opening 
>>>> attachments,
>>>> clicking links, or responding to this email.
>>>>
>>>> Dear all,
>>>>
>>>>
>>>>
>>>> First of all sorry for the novice question.
>>>>
>>>>
>>>>
>>>> I *processed* one of my datasets using xds. Since I was not 
>>>> expecting an
>>>> anomalous signal, I kept Friedel's Law = True.
>>>>
>>>> The number of unique reflections I got: *17,000*
>>>>
>>>> I used xds_ASCII.HKL after correct.LP and converted it to mtz using 
>>>> Phenix
>>>> with 10% reflections for Rfree calculations.
>>>>
>>>>
>>>>
>>>> Now, when I am trying to* refine *the structure in Phenix, the 
>>>> structure
>>>> is being refined against *32,300* reflections with *3230* 
>>>> reflections for
>>>> Rfree calculation. Looks like refinement is treating Friedel's pair 
>>>> as two
>>>> different reflections.
>>>>
>>>>
>>>>
>>>> When I am trying to upload PDB on rcsb, on the refinement tab, the 
>>>> number
>>>> of reflections used for refinement and Rfree calculations are shown as
>>>> 32,300 and 3220 respectively, while in the validation report, no. of
>>>> reflections used for Rfree calculation are shown as *1700*.
>>>>
>>>>
>>>>
>>>> I would really appreciate if someone can guide me to resolve this
>>>> discrepancy.
>>>>
>>>>
>>>>
>>>> Thanks in advance.
>>>>
>>>>
>>>>
>>>> Viney
>>>>
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL: 
>>> <http://phenix-online.org/pipermail/phenixbb/attachments/20220112/094a7f44/attachment.htm>
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> phenixbb mailing list
>>> phenixbb at phenix-online.org
>>> http://phenix-online.org/mailman/listinfo/phenixbb
>>>
>>>
>>> End of phenixbb Digest, Vol 194, Issue 2
>>> ****************************************
>>
>> _______________________________________________
>> phenixbb mailing list
>> phenixbb at phenix-online.org
>> http://phenix-online.org/mailman/listinfo/phenixbb
>> Unsubscribe: phenixbb-leave at phenix-online.org
>
> _______________________________________________
> phenixbb mailing list
> phenixbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/phenixbb
> Unsubscribe: phenixbb-leave at phenix-online.org

More information about the phenixbb mailing list