Constrained occ refinement -- sum not equal 1
Hi, I am trying to sort out some issues with the constrained occupancy refinement in phenix 1.4.3. The structure I am working on is fairly high resolution ~1.4A and it looks as if a couple of the alternative conformations would benefit from occupancy refinement, no hydrogens are present yet. I defined the constrained groups like this: occupancies { individual = None remove_selection = None constrained_group { selection = chain A and resid 66 and altid A selection = chain A and resid 66 and altid B } .... I expected that the sum of occupancies would be restrained to 1, which they should according to the manual, but the refinement gives me occupancies of 0.7 for altid A and 0.34 for altid B, starting values were both 0.5. Similar discrepancies were also present in other groups, like 0.5 A, 0.37 B etc. The latter is the largest deviation from 1 I have seen in thus far. The alternate conformations were generated in coot, by splitting at the CA position. Is this a rounding error I just have to contend with or is there some other parameter I need to tweak? Cheers, Carsten
Hi Carsten, they should definitely add up to 1. If the PDB file contains altloc identifiers, then those atoms should be picked up automatically and you don't need to specify any selections. For example, make sure you have something like this in your PDB file (the starting occupancies can be any): ATOM 49 N MET A 10 3.832 -11.001 0.570 1.00 11.55 N ATOM 50 CA AMET A 10 2.523 -10.377 0.709 0.70 12.44 C ATOM 51 CA BMET A 10 2.574 -10.279 0.718 0.30 12.42 C ATOM 54 CB AMET A 10 2.580 -9.129 1.601 0.70 14.67 C ATOM 55 CB BMET A 10 2.822 -8.921 1.394 0.30 13.99 C ATOM 60 CG AMET A 10 1.245 -8.462 1.951 0.70 18.00 C ATOM 61 CG BMET A 10 1.545 -8.166 1.727 0.30 17.90 C ATOM 66 SD AMET A 10 0.659 -7.204 0.780 0.70 19.47 S ATOM 67 SD BMET A 10 1.792 -6.615 2.600 0.30 26.45 S ATOM 68 CE AMET A 10 1.520 -5.730 1.337 0.70 20.21 C ATOM 69 CE BMET A 10 2.371 -5.566 1.271 0.30 26.12 C ATOM 76 C MET A 10 2.021 -10.028 -0.671 1.00 10.57 C ATOM 77 O MET A 10 2.780 -9.724 -1.578 1.00 11.30 O and then everything should be OK. Unfortunately, I can't tell much about the selections you use without looking at the PDB file, so in your case I would just try the above suggestion. Pavel. On 2/4/2009 12:15 PM, Schubert, Carsten [PRDUS] wrote:
Hi,
I am trying to sort out some issues with the constrained occupancy refinement in phenix 1.4.3. The structure I am working on is fairly high resolution ~1.4A and it looks as if a couple of the alternative conformations would benefit from occupancy refinement, no hydrogens are present yet.
I defined the constrained groups like this:
occupancies { individual = None remove_selection = None constrained_group { selection = chain A and resid 66 and altid A selection = chain A and resid 66 and altid B } ....
I expected that the sum of occupancies would be restrained to 1, which they should according to the manual, but the refinement gives me occupancies of 0.7 for altid A and 0.34 for altid B, starting values were both 0.5. Similar discrepancies were also present in other groups, like 0.5 A, 0.37 B etc. The latter is the largest deviation from 1 I have seen in thus far. The alternate conformations were generated in coot, by splitting at the CA position.
Is this a rounding error I just have to contend with or is there some other parameter I need to tweak?
Cheers,
Carsten
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Hi Pavel, that worked. I had tried that initially, but I probably did not have occupancy refinement enabled in the strategy, which explains why the occupancies never deviated from 0.5 and my subsequent attempts at overkill. Thanks Carsten -----Original Message----- From: Pavel Afonine [mailto:[email protected]] Sent: Wednesday, February 04, 2009 3:28 PM To: PHENIX user mailing list Cc: Schubert, Carsten [PRDUS] Subject: Re: [phenixbb] Constrained occ refinement -- sum not equal 1 Hi Carsten, they should definitely add up to 1. If the PDB file contains altloc identifiers, then those atoms should be picked up automatically and you don't need to specify any selections. For example, make sure you have something like this in your PDB file (the starting occupancies can be any): ATOM 49 N MET A 10 3.832 -11.001 0.570 1.00 11.55 N ATOM 50 CA AMET A 10 2.523 -10.377 0.709 0.70 12.44 C ATOM 51 CA BMET A 10 2.574 -10.279 0.718 0.30 12.42 C ATOM 54 CB AMET A 10 2.580 -9.129 1.601 0.70 14.67 C ATOM 55 CB BMET A 10 2.822 -8.921 1.394 0.30 13.99 C ATOM 60 CG AMET A 10 1.245 -8.462 1.951 0.70 18.00 C ATOM 61 CG BMET A 10 1.545 -8.166 1.727 0.30 17.90 C ATOM 66 SD AMET A 10 0.659 -7.204 0.780 0.70 19.47 S ATOM 67 SD BMET A 10 1.792 -6.615 2.600 0.30 26.45 S ATOM 68 CE AMET A 10 1.520 -5.730 1.337 0.70 20.21 C ATOM 69 CE BMET A 10 2.371 -5.566 1.271 0.30 26.12 C ATOM 76 C MET A 10 2.021 -10.028 -0.671 1.00 10.57 C ATOM 77 O MET A 10 2.780 -9.724 -1.578 1.00 11.30 O and then everything should be OK. Unfortunately, I can't tell much about the selections you use without looking at the PDB file, so in your case I would just try the above suggestion. Pavel. On 2/4/2009 12:15 PM, Schubert, Carsten [PRDUS] wrote: Hi, I am trying to sort out some issues with the constrained occupancy refinement in phenix 1.4.3. The structure I am working on is fairly high resolution ~1.4A and it looks as if a couple of the alternative conformations would benefit from occupancy refinement, no hydrogens are present yet. I defined the constrained groups like this: occupancies { individual = None remove_selection = None constrained_group { selection = chain A and resid 66 and altid A selection = chain A and resid 66 and altid B } .... I expected that the sum of occupancies would be restrained to 1, which they should according to the manual, but the refinement gives me occupancies of 0.7 for altid A and 0.34 for altid B, starting values were both 0.5. Similar discrepancies were also present in other groups, like 0.5 A, 0.37 B etc. The latter is the largest deviation from 1 I have seen in thus far. The alternate conformations were generated in coot, by splitting at the CA position. Is this a rounding error I just have to contend with or is there some other parameter I need to tweak? Cheers, Carsten _______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
I am doing joint refinement of a perdeuterated protein, and I am having a
problem with alternate conformations - the deuterium occupancies don't match
the overall occupancies for their conformation:
ATOM 335 N APRO A 19 13.154 0.897 15.243 0.38 8.64
N
ATOM 336 CA APRO A 19 12.556 0.209 16.391 0.38 7.61
C
ATOM 337 CB APRO A 19 13.743 0.043 17.350 0.38 12.47
C
ATOM 338 CG APRO A 19 14.713 1.102 16.926 0.38 9.70
C
ATOM 339 CD APRO A 19 14.594 1.126 15.443 0.38 13.59
C
ATOM 340 C APRO A 19 11.414 0.974 17.057 0.38 13.42
C
ATOM 341 O APRO A 19 10.559 0.334 17.671 0.38 15.65
O
ATOM 342 DA APRO A 19 12.236 -0.672 16.112 0.48 5.78
D
ATOM 343 DB2APRO A 19 13.454 0.173 18.268 0.43 15.49
D
ATOM 344 DB3APRO A 19 14.128 -0.841 17.229 0.52 10.36
D
ATOM 345 DG2APRO A 19 14.460 1.959 17.300 0.60 13.82
D
ATOM 346 DG3APRO A 19 15.610 0.852 17.197 0.30 14.04
D
ATOM 347 DD2APRO A 19 14.850 1.996 15.097 0.48 12.84
D
ATOM 348 DD3APRO A 19 15.119 0.413 15.046 0.40 8.63
D
ATOM 349 N BPRO A 19 13.163 0.886 15.240 0.62 8.58
N
ATOM 350 CA BPRO A 19 12.606 0.252 16.438 0.62 7.70
C
ATOM 351 CB BPRO A 19 13.777 0.307 17.423 0.62 11.89
C
ATOM 352 CG BPRO A 19 14.973 0.224 16.548 0.62 11.13
C
ATOM 353 CD BPRO A 19 14.616 1.094 15.375 0.62 13.76
C
ATOM 354 C BPRO A 19 11.399 0.972 17.012 0.62 13.34
C
ATOM 355 O BPRO A 19 10.491 0.304 17.514 0.62 15.31
O
ATOM 356 DA BPRO A 19 12.371 -0.680 16.253 0.23 7.26
D
ATOM 357 DB2BPRO A 19 13.760 1.148 17.905 0.72 12.42
D
ATOM 358 DB3BPRO A 19 13.734 -0.446 18.034 0.19 14.30
D
ATOM 359 DG2BPRO A 19 15.759 0.552 17.012 0.87 14.63
D
ATOM 360 DG3BPRO A 19 15.094 -0.695 16.264 0.40 9.68
D
ATOM 361 DD2BPRO A 19 14.807 2.022 15.580 0.43 11.93
D
ATOM 362 DD3BPRO A 19 15.083 0.798 14.578 0.31 8.63
D
I am using these refinement options:
refine {
strategy = *individual_sites *rigid_body *individual_adp group_adp tls \
*occupancies group_anomalous
sites {
individual = None
rigid_body = None
}
occupancies {
individual = None
remove_selection = None
constrained_group {
selection = None
}
}
}
and
hydrogens {
refine = *individual riding
refine_adp = one_b_per_residue one_b_per_molecule *individual
refine_occupancies = one_q_per_residue *one_q_per_molecule individual
contribute_to_f_calc = True
}
What refinement options should I be using to keep the D occupancies the same
as the rest of their conformation? Hope you can help!
Regards,
Anna
2009/2/4 Pavel Afonine
Hi Carsten,
they should definitely add up to 1. If the PDB file contains altloc identifiers, then those atoms should be picked up automatically and you don't need to specify any selections. For example, make sure you have something like this in your PDB file (the starting occupancies can be any):
ATOM 49 N MET A 10 3.832 -11.001 0.570 1.00 11.55 N ATOM 50 CA AMET A 10 2.523 -10.377 0.709 0.70 12.44 C ATOM 51 CA BMET A 10 2.574 -10.279 0.718 0.30 12.42 C ATOM 54 CB AMET A 10 2.580 -9.129 1.601 0.70 14.67 C ATOM 55 CB BMET A 10 2.822 -8.921 1.394 0.30 13.99 C ATOM 60 CG AMET A 10 1.245 -8.462 1.951 0.70 18.00 C ATOM 61 CG BMET A 10 1.545 -8.166 1.727 0.30 17.90 C ATOM 66 SD AMET A 10 0.659 -7.204 0.780 0.70 19.47 S ATOM 67 SD BMET A 10 1.792 -6.615 2.600 0.30 26.45 S ATOM 68 CE AMET A 10 1.520 -5.730 1.337 0.70 20.21 C ATOM 69 CE BMET A 10 2.371 -5.566 1.271 0.30 26.12 C ATOM 76 C MET A 10 2.021 -10.028 -0.671 1.00 10.57 C ATOM 77 O MET A 10 2.780 -9.724 -1.578 1.00 11.30 O
and then everything should be OK. Unfortunately, I can't tell much about the selections you use without looking at the PDB file, so in your case I would just try the above suggestion.
Pavel.
On 2/4/2009 12:15 PM, Schubert, Carsten [PRDUS] wrote:
Hi,
I am trying to sort out some issues with the constrained occupancy refinement in phenix 1.4.3. The structure I am working on is fairly high resolution ~1.4A and it looks as if a couple of the alternative conformations would benefit from occupancy refinement, no hydrogens are present yet.
I defined the constrained groups like this:
occupancies { individual = None remove_selection = None constrained_group { selection = chain A and resid 66 and altid A selection = chain A and resid 66 and altid B } ....
I expected that the sum of occupancies would be restrained to 1, which they should according to the manual, but the refinement gives me occupancies of 0.7 for altid A and 0.34 for altid B, starting values were both 0.5. Similar discrepancies were also present in other groups, like 0.5 A, 0.37 B etc. The latter is the largest deviation from 1 I have seen in thus far. The alternate conformations were generated in coot, by splitting at the CA position.
Is this a rounding error I just have to contend with or is there some other parameter I need to tweak?
Cheers,
Carsten
_______________________________________________ phenixbb mailing [email protected]http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Hi Anna, to find out what's going on I just did the following: 1) I copied the lines below into "answer.pdb" file and I re-set the occupancies so they all make sense (add up to 1); 2) Using "answer.pdb" file I simulated two datasets: X-ray and neutron: "data.mtz". What I did is just I computed X-ray and neutron Fcalc, and called their absolute values as F-obs-xray and F-obs-neutron, and stored them in "data.mtz". 3) Then I scrambled the model in "answer.pdb" by randomizing the occupancies of all atoms. This file containing the scrambled models is called "start.pdb". 4) Finally, I took "data.mtz" and "start.pdb" files and ran joint XN refinement in phenix.refine. I did only occupancy refinement. As the refinement result I was expecting to get the PDB file with refined structure where all the occupancies are exactly the same as in "answer.pdb". The command I used is: phenix.refine data.mtz start.pdb params --overwrite 5) Now, open two files "answer.pdb" and "refinement_result_001.pdb", and voila! All the occupancies are as expected. All the input and output files discussed above are located here: http://cci.lbl.gov/~afonine/for_AnneG/ All the above data and files manipulations were done using "phenix.pdbtools" and "phenix.refine". Could you reproduce the refinement I just did ("step 4") using your version of PHENIX ? If not, then please update to the latest PHENIX (http://www.phenix-online.org/download/) and try again. If not successful even then, I will send you the latest development version. Please let me know if you have any questions or problems! Pavel. On 2/10/2009 11:04 AM, Anna Gardberg wrote:
I am doing joint refinement of a perdeuterated protein, and I am having a problem with alternate conformations - the deuterium occupancies don't match the overall occupancies for their conformation:
ATOM 335 N APRO A 19 13.154 0.897 15.243 0.38 8.64 N ATOM 336 CA APRO A 19 12.556 0.209 16.391 0.38 7.61 C ATOM 337 CB APRO A 19 13.743 0.043 17.350 0.38 12.47 C ATOM 338 CG APRO A 19 14.713 1.102 16.926 0.38 9.70 C ATOM 339 CD APRO A 19 14.594 1.126 15.443 0.38 13.59 C ATOM 340 C APRO A 19 11.414 0.974 17.057 0.38 13.42 C ATOM 341 O APRO A 19 10.559 0.334 17.671 0.38 15.65 O ATOM 342 DA APRO A 19 12.236 -0.672 16.112 0.48 5.78 D ATOM 343 DB2APRO A 19 13.454 0.173 18.268 0.43 15.49 D ATOM 344 DB3APRO A 19 14.128 -0.841 17.229 0.52 10.36 D ATOM 345 DG2APRO A 19 14.460 1.959 17.300 0.60 13.82 D ATOM 346 DG3APRO A 19 15.610 0.852 17.197 0.30 14.04 D ATOM 347 DD2APRO A 19 14.850 1.996 15.097 0.48 12.84 D ATOM 348 DD3APRO A 19 15.119 0.413 15.046 0.40 8.63 D ATOM 349 N BPRO A 19 13.163 0.886 15.240 0.62 8.58 N ATOM 350 CA BPRO A 19 12.606 0.252 16.438 0.62 7.70 C ATOM 351 CB BPRO A 19 13.777 0.307 17.423 0.62 11.89 C ATOM 352 CG BPRO A 19 14.973 0.224 16.548 0.62 11.13 C ATOM 353 CD BPRO A 19 14.616 1.094 15.375 0.62 13.76 C ATOM 354 C BPRO A 19 11.399 0.972 17.012 0.62 13.34 C ATOM 355 O BPRO A 19 10.491 0.304 17.514 0.62 15.31 O ATOM 356 DA BPRO A 19 12.371 -0.680 16.253 0.23 7.26 D ATOM 357 DB2BPRO A 19 13.760 1.148 17.905 0.72 12.42 D ATOM 358 DB3BPRO A 19 13.734 -0.446 18.034 0.19 14.30 D ATOM 359 DG2BPRO A 19 15.759 0.552 17.012 0.87 14.63 D ATOM 360 DG3BPRO A 19 15.094 -0.695 16.264 0.40 9.68 D ATOM 361 DD2BPRO A 19 14.807 2.022 15.580 0.43 11.93 D ATOM 362 DD3BPRO A 19 15.083 0.798 14.578 0.31 8.63 D
I am using these refinement options: refine { strategy = *individual_sites *rigid_body *individual_adp group_adp tls \ *occupancies group_anomalous sites { individual = None rigid_body = None } occupancies { individual = None remove_selection = None constrained_group { selection = None } } }
and hydrogens { refine = *individual riding refine_adp = one_b_per_residue one_b_per_molecule *individual refine_occupancies = one_q_per_residue *one_q_per_molecule individual contribute_to_f_calc = True } What refinement options should I be using to keep the D occupancies the same as the rest of their conformation? Hope you can help!
Regards, Anna
Hi, Pavel.
Thanks for the quick reply. I downloaded the files, and I got the result you
predicted - all the occupancies looked neat and tidy as expected. Since you
used those nice occupancies to generate perfect data, which was then refined
against, that's what one would expect, right? So I am still left with the
issue of the occupancies going nuts in my real refinement. Or am I missing
something?
Regards,
Anna
On Tue, Feb 10, 2009 at 4:43 PM, Pavel Afonine
Hi Anna,
to find out what's going on I just did the following:
1) I copied the lines below into "answer.pdb" file and I re-set the occupancies so they all make sense (add up to 1);
2) Using "answer.pdb" file I simulated two datasets: X-ray and neutron: "data.mtz". What I did is just I computed X-ray and neutron Fcalc, and called their absolute values as F-obs-xray and F-obs-neutron, and stored them in "data.mtz".
3) Then I scrambled the model in "answer.pdb" by randomizing the occupancies of all atoms. This file containing the scrambled models is called "start.pdb".
4) Finally, I took "data.mtz" and "start.pdb" files and ran joint XN refinement in phenix.refine. I did only occupancy refinement. As the refinement result I was expecting to get the PDB file with refined structure where all the occupancies are exactly the same as in "answer.pdb". The command I used is:
phenix.refine data.mtz start.pdb params --overwrite
5) Now, open two files "answer.pdb" and "refinement_result_001.pdb", and voila! All the occupancies are as expected.
All the input and output files discussed above are located here: http://cci.lbl.gov/~afonine/for_AnneG/http://cci.lbl.gov/%7Eafonine/for_AnneG/
All the above data and files manipulations were done using "phenix.pdbtools" and "phenix.refine".
Could you reproduce the refinement I just did ("step 4") using your version of PHENIX ? If not, then please update to the latest PHENIX (http://www.phenix-online.org/download/) and try again. If not successful even then, I will send you the latest development version.
Please let me know if you have any questions or problems!
Pavel.
On 2/10/2009 11:04 AM, Anna Gardberg wrote:
I am doing joint refinement of a perdeuterated protein, and I am having a problem with alternate conformations - the deuterium occupancies don't match the overall occupancies for their conformation:
ATOM 335 N APRO A 19 13.154 0.897 15.243 0.38 8.64 N ATOM 336 CA APRO A 19 12.556 0.209 16.391 0.38 7.61 C ATOM 337 CB APRO A 19 13.743 0.043 17.350 0.38 12.47 C ATOM 338 CG APRO A 19 14.713 1.102 16.926 0.38 9.70 C ATOM 339 CD APRO A 19 14.594 1.126 15.443 0.38 13.59 C ATOM 340 C APRO A 19 11.414 0.974 17.057 0.38 13.42 C ATOM 341 O APRO A 19 10.559 0.334 17.671 0.38 15.65 O ATOM 342 DA APRO A 19 12.236 -0.672 16.112 0.48 5.78 D ATOM 343 DB2APRO A 19 13.454 0.173 18.268 0.43 15.49 D ATOM 344 DB3APRO A 19 14.128 -0.841 17.229 0.52 10.36 D ATOM 345 DG2APRO A 19 14.460 1.959 17.300 0.60 13.82 D ATOM 346 DG3APRO A 19 15.610 0.852 17.197 0.30 14.04 D ATOM 347 DD2APRO A 19 14.850 1.996 15.097 0.48 12.84 D ATOM 348 DD3APRO A 19 15.119 0.413 15.046 0.40 8.63 D ATOM 349 N BPRO A 19 13.163 0.886 15.240 0.62 8.58 N ATOM 350 CA BPRO A 19 12.606 0.252 16.438 0.62 7.70 C ATOM 351 CB BPRO A 19 13.777 0.307 17.423 0.62 11.89 C ATOM 352 CG BPRO A 19 14.973 0.224 16.548 0.62 11.13 C ATOM 353 CD BPRO A 19 14.616 1.094 15.375 0.62 13.76 C ATOM 354 C BPRO A 19 11.399 0.972 17.012 0.62 13.34 C ATOM 355 O BPRO A 19 10.491 0.304 17.514 0.62 15.31 O ATOM 356 DA BPRO A 19 12.371 -0.680 16.253 0.23 7.26 D ATOM 357 DB2BPRO A 19 13.760 1.148 17.905 0.72 12.42 D ATOM 358 DB3BPRO A 19 13.734 -0.446 18.034 0.19 14.30 D ATOM 359 DG2BPRO A 19 15.759 0.552 17.012 0.87 14.63 D ATOM 360 DG3BPRO A 19 15.094 -0.695 16.264 0.40 9.68 D ATOM 361 DD2BPRO A 19 14.807 2.022 15.580 0.43 11.93 D ATOM 362 DD3BPRO A 19 15.083 0.798 14.578 0.31 8.63 D
I am using these refinement options: refine { strategy = *individual_sites *rigid_body *individual_adp group_adp tls \ *occupancies group_anomalous sites { individual = None rigid_body = None } occupancies { individual = None remove_selection = None constrained_group { selection = None } } }
and hydrogens { refine = *individual riding refine_adp = one_b_per_residue one_b_per_molecule *individual refine_occupancies = one_q_per_residue *one_q_per_molecule individual contribute_to_f_calc = True } What refinement options should I be using to keep the D occupancies the same as the rest of their conformation? Hope you can help!
Regards, Anna
phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Hi Anne,
Thanks for the quick reply. I downloaded the files, and I got the result you predicted - all the occupancies looked neat and tidy as expected. Since you used those nice occupancies to generate perfect data, which was then refined against, that's what one would expect, right?
The occupancies in starting model, "start.pdb", are all garbage. So, if phenix.refine does something wrong with occupancy refinement, they would never refine to correct values.
So I am still left with the issue of the occupancies going nuts in my real refinement.
The test I did completely replicate the real situation: obviously, phenix.refine "does not know" that data.mtz contained fake Fobs -:) If you are still getting the wrong values then there must be something else going not right. The only way to find out is if you send me the data and model and the exact command you used. In this case I will be able to reproduce the problem myself, investigate some more and hopefully provide a fix. If you decide to send me the data, please make sure you send it to my address only and not to the whole bb. Pavel.
Hi Anna, I found the problem so I don't need the data. I will fix it asap and I will send you the development version once it is available. Thanks! Pavel.
That'll be great, thank you!
-Anna
On Tue, Feb 10, 2009 at 5:29 PM, Pavel Afonine
Hi Anna,
I found the problem so I don't need the data. I will fix it asap and I will send you the development version once it is available.
Thanks! Pavel.
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
participants (3)
-
Anna Gardberg -
Pavel Afonine -
Schubert, Carsten [PRDUS]