Re: [phenixbb] Reflection file woes phenix/ccp4/PDB
Robbie, Yes, the thought that the anomalous R free flags could have something to do with it occurred to me as well, so I checked. As far as I can see, the test reflections are evenly distributed between the + and - sets; they were not selected from the + or - set only. For the vast majority of the test reflections, both Bijvoet mates were measured and both were flagged properly. For some, however, only one Bijvoet mate was measured and flagged. It probably wouldn't do any harm to consolidate the test flags, discarding the few with only one measured Bijvoet mate, but I really don't see why it should be necessary to do so. For now, I have converted the reflection file into the CNS format and sent to the PDB, hoping that they will be able to deal with that one properly. Bit surprised though that no one has run into that problem before. Thanks for your suggestion. MM On Dec 16, 2011, at 3:54 AM, Robbie P. Joosten wrote:
Hi Mischa,
Your mtz file seems to contain 'anomalous' free R-flags (that is what I gather from Pavel's post). This is kind of weird. Do you really want I+ in the work set and I- in the test set? That would bias your R-free. If the two flags match (which I really hope), then you can leave one column out before you (re)deposit your data. This will hopefully solves the problem.
Cheers, Robbie Joosten
Netherlands Cancer Institute
On 12/15/2011 05:40 PM, Machius, Mischa Christian wrote:
Y'all,
While depositing a structure, the PDB 'complained' about my structure-factor file, saying they don't contain R-free flags or are otherwise incomplete. The comment was prompted by their Refmac R-factor test failing.
The structure-factor file I submitted came from phenix.refine (xxx_data.mtz). I am using version 1.7.3-928.
Indeed, using this file in CCP4 fails with the message: "CCP4MTZfile: Mtz column type mismatch: I-obs(+) K-J CCP4MTZfile: Mtz column type mismatch: SIGI-obs(+) M-Q"
The file contains anomalous data. I have no problems with files that do not contain anomalous data.
Needless to say that Phenix uses the structure-factor file just fine when reading it back into, say, its validation utility.
When I use iotbx.reflection_file_reader, I cannot detect any anomalies.
I am at a loss as to how to resolve this issue. I would like to deposit the data as prepared by Phenix so that the PDB tools can use them.
Anybody ran into this issue before? Any ideas for how to resolve it?
Many thanks in advance!
MM _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
On Fri, Dec 16, 2011 at 6:50 AM, Machius, Mischa Christian
Yes, the thought that the anomalous R free flags could have something to do with it occurred to me as well, so I checked. As far as I can see, the test reflections are evenly distributed between the + and - sets; they were not selected from the + or - set only. For the vast majority of the test reflections, both Bijvoet mates were measured and both were flagged properly. For some, however, only one Bijvoet mate was measured and flagged. It probably wouldn't do any harm to consolidate the test flags, discarding the few with only one measured Bijvoet mate, but I really don't see why it should be necessary to do so.
It shouldn't - many/most of those missing Bijvoet mates may simply be centric reflections. The fact that the completeness shows up as significantly less than 100% is (in my opinion) a bug somewhere either in the MTZ library or our wrapper for it, because I've seen this even in files with synthetic, 100% complete anomalous data. You should be able to consolidate the test flags, but you don't need to discard any. -Nat
Misha, you will be surprised how many times people were in situations similar to yours with PDB or PDB-EBI. Myself was not able to submit data after PHENIX refinement, as I ALWAYS separate between anomalous + and - reflections during refinement. PHENIX provides I(+) and I(-) for submission (in my case as I never you F's for refinement, only I's) but not I average, PDB-EBI refuses to accept such data and PHENIX refuses to provide redundant information. The end is that one need to re-refine using diffraction data presentation formalism which PDB will accept in order not to provide data that was not used for refinement but was chosen for submission convenience. Dr Felix Frolow Professor of Structural Biology and Biotechnology Department of Molecular Microbiology and Biotechnology Tel Aviv University 69978, Israel Acta Crystallographica F, co-editor e-mail: [email protected] Tel: ++972-3640-8723 Fax: ++972-3640-9407 Cellular: 0547 459 608 On Dec 16, 2011, at 16:50 , Machius, Mischa Christian wrote:
Robbie,
Yes, the thought that the anomalous R free flags could have something to do with it occurred to me as well, so I checked. As far as I can see, the test reflections are evenly distributed between the + and - sets; they were not selected from the + or - set only. For the vast majority of the test reflections, both Bijvoet mates were measured and both were flagged properly. For some, however, only one Bijvoet mate was measured and flagged. It probably wouldn't do any harm to consolidate the test flags, discarding the few with only one measured Bijvoet mate, but I really don't see why it should be necessary to do so.
For now, I have converted the reflection file into the CNS format and sent to the PDB, hoping that they will be able to deal with that one properly.
Bit surprised though that no one has run into that problem before.
Thanks for your suggestion.
MM
On Dec 16, 2011, at 3:54 AM, Robbie P. Joosten wrote:
Hi Mischa,
Your mtz file seems to contain 'anomalous' free R-flags (that is what I gather from Pavel's post). This is kind of weird. Do you really want I+ in the work set and I- in the test set? That would bias your R-free. If the two flags match (which I really hope), then you can leave one column out before you (re)deposit your data. This will hopefully solves the problem.
Cheers, Robbie Joosten
Netherlands Cancer Institute
On 12/15/2011 05:40 PM, Machius, Mischa Christian wrote:
Y'all,
While depositing a structure, the PDB 'complained' about my structure-factor file, saying they don't contain R-free flags or are otherwise incomplete. The comment was prompted by their Refmac R-factor test failing.
The structure-factor file I submitted came from phenix.refine (xxx_data.mtz). I am using version 1.7.3-928.
Indeed, using this file in CCP4 fails with the message: "CCP4MTZfile: Mtz column type mismatch: I-obs(+) K-J CCP4MTZfile: Mtz column type mismatch: SIGI-obs(+) M-Q"
The file contains anomalous data. I have no problems with files that do not contain anomalous data.
Needless to say that Phenix uses the structure-factor file just fine when reading it back into, say, its validation utility.
When I use iotbx.reflection_file_reader, I cannot detect any anomalies.
I am at a loss as to how to resolve this issue. I would like to deposit the data as prepared by Phenix so that the PDB tools can use them.
Anybody ran into this issue before? Any ideas for how to resolve it?
Many thanks in advance!
MM _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
Y'all, Thanks for your comments, suggestions and empathy. It turns out that there was a bug in the structure factor conversion program that the PDB uses. The phenix mtz file is just fine. Everything appears to be in working order, and nobody should run into the same issue from now on. Cheers! On Dec 16, 2011, at 10:03 AM, Felix Frolow wrote: Misha, you will be surprised how many times people were in situations similar to yours with PDB or PDB-EBI. Myself was not able to submit data after PHENIX refinement, as I ALWAYS separate between anomalous + and - reflections during refinement. PHENIX provides I(+) and I(-) for submission (in my case as I never you F's for refinement, only I's) but not I average, PDB-EBI refuses to accept such data and PHENIX refuses to provide redundant information. The end is that one need to re-refine using diffraction data presentation formalism which PDB will accept in order not to provide data that was not used for refinement but was chosen for submission convenience. Dr Felix Frolow Professor of Structural Biology and Biotechnology Department of Molecular Microbiology and Biotechnology Tel Aviv University 69978, Israel Acta Crystallographica F, co-editor e-mail: [email protected]mailto:[email protected] Tel: ++972-3640-8723 Fax: ++972-3640-9407 Cellular: 0547 459 608 On Dec 16, 2011, at 16:50 , Machius, Mischa Christian wrote: Robbie, Yes, the thought that the anomalous R free flags could have something to do with it occurred to me as well, so I checked. As far as I can see, the test reflections are evenly distributed between the + and - sets; they were not selected from the + or - set only. For the vast majority of the test reflections, both Bijvoet mates were measured and both were flagged properly. For some, however, only one Bijvoet mate was measured and flagged. It probably wouldn't do any harm to consolidate the test flags, discarding the few with only one measured Bijvoet mate, but I really don't see why it should be necessary to do so. For now, I have converted the reflection file into the CNS format and sent to the PDB, hoping that they will be able to deal with that one properly. Bit surprised though that no one has run into that problem before. Thanks for your suggestion. MM On Dec 16, 2011, at 3:54 AM, Robbie P. Joosten wrote: Hi Mischa, Your mtz file seems to contain 'anomalous' free R-flags (that is what I gather from Pavel's post). This is kind of weird. Do you really want I+ in the work set and I- in the test set? That would bias your R-free. If the two flags match (which I really hope), then you can leave one column out before you (re)deposit your data. This will hopefully solves the problem. Cheers, Robbie Joosten Netherlands Cancer Institute On 12/15/2011 05:40 PM, Machius, Mischa Christian wrote: Y'all, While depositing a structure, the PDB 'complained' about my structure-factor file, saying they don't contain R-free flags or are otherwise incomplete. The comment was prompted by their Refmac R-factor test failing. The structure-factor file I submitted came from phenix.refine (xxx_data.mtz). I am using version 1.7.3-928. Indeed, using this file in CCP4 fails with the message: "CCP4MTZfile: Mtz column type mismatch: I-obs(+) K-J CCP4MTZfile: Mtz column type mismatch: SIGI-obs(+) M-Q" The file contains anomalous data. I have no problems with files that do not contain anomalous data. Needless to say that Phenix uses the structure-factor file just fine when reading it back into, say, its validation utility. When I use iotbx.reflection_file_reader, I cannot detect any anomalies. I am at a loss as to how to resolve this issue. I would like to deposit the data as prepared by Phenix so that the PDB tools can use them. Anybody ran into this issue before? Any ideas for how to resolve it? Many thanks in advance! MM _______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb _______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb _______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb
Hi, many thanks for this go to Ezra Peisach who was extremely helpful in addressing this problem at the RCSB end, and Mischa of course too, who reported the problem and fearlessly shared the data! All the best, Pavel On 12/20/11 12:26 PM, Machius, Mischa Christian wrote:
Y'all,
Thanks for your comments, suggestions and empathy.
It turns out that there was a bug in the structure factor conversion program that the PDB uses. The phenix mtz file is just fine. Everything appears to be in working order, and nobody should run into the same issue from now on.
Cheers!
On Dec 16, 2011, at 10:03 AM, Felix Frolow wrote:
Misha, you will be surprised how many times people were in situations similar to yours with PDB or PDB-EBI. Myself was not able to submit data after PHENIX refinement, as I ALWAYS separate between anomalous + and - reflections during refinement. PHENIX provides I(+) and I(-) for submission (in my case as I never you F's for refinement, only I's) but not I average, PDB-EBI refuses to accept such data and PHENIX refuses to provide redundant information. The end is that one need to re-refine using diffraction data presentation formalism which PDB will accept in order not to provide data that was not used for refinement but was chosen for submission convenience. Dr Felix Frolow Professor of Structural Biology and Biotechnology Department of Molecular Microbiology and Biotechnology Tel Aviv University 69978, Israel
Acta Crystallographica F, co-editor
e-mail: [email protected] mailto:[email protected] Tel: ++972-3640-8723 Fax: ++972-3640-9407 Cellular: 0547 459 608
On Dec 16, 2011, at 16:50 , Machius, Mischa Christian wrote:
Robbie,
Yes, the thought that the anomalous R free flags could have something to do with it occurred to me as well, so I checked. As far as I can see, the test reflections are evenly distributed between the + and - sets; they were not selected from the + or - set only. For the vast majority of the test reflections, both Bijvoet mates were measured and both were flagged properly. For some, however, only one Bijvoet mate was measured and flagged. It probably wouldn't do any harm to consolidate the test flags, discarding the few with only one measured Bijvoet mate, but I really don't see why it should be necessary to do so.
For now, I have converted the reflection file into the CNS format and sent to the PDB, hoping that they will be able to deal with that one properly.
Bit surprised though that no one has run into that problem before.
Thanks for your suggestion.
MM
On Dec 16, 2011, at 3:54 AM, Robbie P. Joosten wrote:
Hi Mischa,
Your mtz file seems to contain 'anomalous' free R-flags (that is what I gather from Pavel's post). This is kind of weird. Do you really want I+ in the work set and I- in the test set? That would bias your R-free. If the two flags match (which I really hope), then you can leave one column out before you (re)deposit your data. This will hopefully solves the problem.
Cheers, Robbie Joosten
Netherlands Cancer Institute
On 12/15/2011 05:40 PM, Machius, Mischa Christian wrote:
Y'all,
While depositing a structure, the PDB 'complained' about my structure-factor file, saying they don't contain R-free flags or are otherwise incomplete. The comment was prompted by their Refmac R-factor test failing.
The structure-factor file I submitted came from phenix.refine (xxx_data.mtz). I am using version 1.7.3-928.
Indeed, using this file in CCP4 fails with the message: "CCP4MTZfile: Mtz column type mismatch: I-obs(+) K-J CCP4MTZfile: Mtz column type mismatch: SIGI-obs(+) M-Q"
The file contains anomalous data. I have no problems with files that do not contain anomalous data.
Needless to say that Phenix uses the structure-factor file just fine when reading it back into, say, its validation utility.
When I use iotbx.reflection_file_reader, I cannot detect any anomalies.
I am at a loss as to how to resolve this issue. I would like to deposit the data as prepared by Phenix so that the PDB tools can use them.
Anybody ran into this issue before? Any ideas for how to resolve it?
Many thanks in advance!
MM _______________________________________________ phenixbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
participants (4)
-
Felix Frolow -
Machius, Mischa Christian -
Nathaniel Echols -
Pavel Afonine