Extra map coefficients in 1.21.1-5286
In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use. 1.21rc1_5058 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered 8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model 11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT 12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT 13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT 16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT 1.21.1-5286 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered 8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model 11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1 12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1 13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2 16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2 17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4 18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4 19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill 20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill 21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5 22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5 Best wishes Kevin -- Kevin Jude, PhD Structural Biology Research Specialist, Garcia Lab Howard Hughes Medical Institute Stanford University School of Medicine Beckman B177, 279 Campus Drive, Stanford CA 94305 Phone: (650) 723-6431tel:%28650%29%20723-6431
Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards,
Elke
From: "Dr. Kevin M Jude"
Dear Kevin and Elke,
Can you provide your input files and settings so that we can reproduce the
issue? The settings are available from the "Results" tab with the "Config
file" button. Alternatively, the settings are available by selecting the
job in your job history and clicking the "Show details" button. A dialog
will pop up and show the location of the job and the config file. I am not
able to get the numbered map coefficients with a tutorial dataset with
default settings. Thanks!
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards, Elke
------------------------------ *From: *"Dr. Kevin M Jude"
*To: *"phenixbb" *Sent: *Thursday, May 2, 2024 5:52:56 PM *Subject: *[phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered
8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model
11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT
12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT
13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT
16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered
8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model
11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1
12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1
13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2
16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2
17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4
18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4
19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill
20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill
21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5
22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes
Kevin
--
Kevin Jude, PhD
Structural Biology Research Specialist, Garcia Lab
Howard Hughes Medical Institute
Stanford University School of Medicine
Beckman B177, 279 Campus Drive, Stanford CA 94305
Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Hi Billy,
Here’s a dropbox linkhttps://www.dropbox.com/scl/fo/16qpqwy79pcm7zgl6j608/ADqFMcQcyUw-Jq6g-bt6LBE?rlkey=abq4uyk7oeavb0ckri9tgioek&dl=0 to input and output from one of my jobs
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Best wishes
Kevin
From: Billy Poon
Hi Billy, To add to Kevin's point, we observed this recently also with Staraniso-processed data. So, one more data point supporting that. Should have reported it earlier. Thank you for the fantastic software, Engin On 5/20/24 10:53 AM, Dr. Kevin M Jude wrote:
Hi Billy,
Here’s a dropbox link https://www.dropbox.com/scl/fo/16qpqwy79pcm7zgl6j608/ADqFMcQcyUw-Jq6g-bt6LBE?rlkey=abq4uyk7oeavb0ckri9tgioek&dl=0 to input and output from one of my jobs
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Best wishes
Kevin
*From: *Billy Poon
*Date: *Saturday, May 18, 2024 at 1:23 PM *To: *Elke De Zitter *Cc: *Dr. Kevin M Jude , phenixbb *Subject: *Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke,
Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
wrote: Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards,
Elke
------------------------------------------------------------------------
*From: *"Dr. Kevin M Jude"
*To: *"phenixbb" *Sent: *Thursday, May 2, 2024 5:52:56 PM *Subject: *[phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered
8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model
11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT
12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT
13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT
16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered
8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model
11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1
12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1
13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2
16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2
17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4
18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4
19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill
20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill
21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5
22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes
Kevin
--
Kevin Jude, PhD
Structural Biology Research Specialist, Garcia Lab
Howard Hughes Medical Institute
Stanford University School of Medicine
Beckman B177, 279 Campus Drive, Stanford CA 94305
Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe:[email protected]
-- Engin Özkan, Ph.D. Associate Professor Dept of Biochemistry and Molecular Biology University of Chicago Phone: (773) 834-5498 http://ozkan.uchicago.edu
Hi all,
I am able to reproduce the error now with Kevin's data and settings. We'll
work on a fix to prevent this from happening.
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org
On Mon, May 20, 2024 at 9:05 AM Engin Özkan
Hi Billy,
To add to Kevin's point, we observed this recently also with Staraniso-processed data. So, one more data point supporting that. Should have reported it earlier.
Thank you for the fantastic software,
Engin On 5/20/24 10:53 AM, Dr. Kevin M Jude wrote:
Hi Billy,
Here’s a dropbox link https://www.dropbox.com/scl/fo/16qpqwy79pcm7zgl6j608/ADqFMcQcyUw-Jq6g-bt6LBE?rlkey=abq4uyk7oeavb0ckri9tgioek&dl=0 to input and output from one of my jobs
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Best wishes
Kevin
*From: *Billy Poon
*Date: *Saturday, May 18, 2024 at 1:23 PM *To: *Elke De Zitter *Cc: *Dr. Kevin M Jude , phenixbb *Subject: *Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke,
Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
wrote: Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards,
Elke
------------------------------
*From: *"Dr. Kevin M Jude"
*To: *"phenixbb" *Sent: *Thursday, May 2, 2024 5:52:56 PM *Subject: *[phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered
8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model
11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT
12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT
13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT
16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered
8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model
11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1
12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1
13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2
16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2
17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4
18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4
19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill
20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill
21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5
22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes
Kevin
--
Kevin Jude, PhD
Structural Biology Research Specialist, Garcia Lab
Howard Hughes Medical Institute
Stanford University School of Medicine
Beckman B177, 279 Campus Drive, Stanford CA 94305
Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing [email protected]http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
-- Engin Özkan, Ph.D. Associate Professor Dept of Biochemistry and Molecular Biology University of Chicago Phone: (773) 834-5498http://ozkan.uchicago.edu
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Dear Kevin and phenixbb readers, Clemens Vonrhein has just asked to join the BB but is not at present able to reply to past messages, so he has asked me to post this message on his behalf. Best wishes, Gerard --------------------------------------------------------------- Dear Kevin,
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Your observations should not be related directly to the use of data coming out of STARANISO, i.e. subjected to cut-off by an anisotropic surface (note this it is not strictly correct to describe this cut-off as "ellipsoidal truncation") instead of a spherical/isotropic one - unless there is some special handling within phenix.refine when it encounters an input MTZ file containing an SA_flag column (that describes the signal status [1]). Or perhaps it looks at the history in the MTZ header (that might contain some information about the use of STARANISO and scaling)? There are two alternative explanations for why you now see additional map coefficients: * The MTZ file from STARANISO (if run via autoPROC [2]) might contain additional sets of amplitudes, namely F_early and F_late, that can be used not only to compute F(early)-F(late) radiation-damage detection maps [3], but also to compute normal electron-density and difference-density maps using those observed amplitudes in addition to the default ones used during refinement. * Looking at the completeness pattern, it might just be a case of the latest version now distinguishing the set of amplitudes before any outlier rejections (within refinement) and after such rejections: 1.21rc1_5058 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered 8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model 11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT 12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT 13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT 16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT ==> 98.14% complete amplitudes, but "only" 93.23% complete map-coefficients, i.e. maps computed /after/ outlier rejection 1.21.1-5286 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered 8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model 11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1 12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1 13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2 16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2 17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4 18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4 19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill 20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill 21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5 22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5 ==> 98.14% complete amplitudes, resulting in using all amplitudes before rejection (default = fill): 98.14% complete map coefficients FOFCWT_5 100.00% complete map coefficients 2FOFCWT_4 98.14% complete map coefficients 2FOFCWT_no_fill using amplitudes after rejection (default = nofill) 93.23% complete map coefficients FOFCWT_2 93.23% complete map coefficients 2FOFCWT_1 100.00% complete map coefficients 2FOFCWT_filled Does that make sense? It looks more like some rather useful provision of different ways of handling missing data (DFc completion, outlier rejection, unmeasured due to module gaps, cusp etc). It might be that anisotropic data as described by STARANISO, plus an assumption of isotropic diffraction (up to the highest diffraction limit), triggers the creation of these map coefficient variants. Does this ring a bell with developers? [1] https://staraniso.globalphasing.org/test_set_flags_about.html [2] https://www.globalphasing.com/autoproc/ [3] https://doi.org/10.1107/S2059798324001487 -- On Mon, May 20, 2024 at 03:53:01PM +0000, Dr. Kevin M Jude wrote:
Hi Billy, Here’s a dropbox linkhttps://www.dropbox.com/scl/fo/16qpqwy79pcm7zgl6j608/ADqFMcQcyUw-Jq6g-bt6LBE?rlkey=abq4uyk7oeavb0ckri9tgioek&dl=0 to input and output from one of my jobs
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Best wishes Kevin
From: Billy Poon
Date: Saturday, May 18, 2024 at 1:23 PM To: Elke De Zitter Cc: Dr. Kevin M Jude , phenixbb Subject: Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke, Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
mailto:[email protected]> wrote: Dear Kevin, I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards, Elke
________________________________ From: "Dr. Kevin M Jude"
mailto:[email protected]> To: "phenixbb" mailto:[email protected]> Sent: Thursday, May 2, 2024 5:52:56 PM Subject: [phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered 8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model 11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT 12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT 13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT 16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered 8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model 11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1 12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1 13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2 16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2 17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4 18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4 19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill 20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill 21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5 22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes Kevin
-- Kevin Jude, PhD Structural Biology Research Specialist, Garcia Lab Howard Hughes Medical Institute Stanford University School of Medicine Beckman B177, 279 Campus Drive, Stanford CA 94305 Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]mailto:[email protected]
_______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]mailto:[email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Thanks Gerard and Clemens (and my apologies for oversimplifying the data cutoff description). My data don’t have F_early or F_late, so the second suggestion seems likely relevant to my issue.
Best wishes
Kevin
From: Gerard Bricogne
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Your observations should not be related directly to the use of data coming out of STARANISO, i.e. subjected to cut-off by an anisotropic surface (note this it is not strictly correct to describe this cut-off as "ellipsoidal truncation") instead of a spherical/isotropic one - unless there is some special handling within phenix.refine when it encounters an input MTZ file containing an SA_flag column (that describes the signal status [1]). Or perhaps it looks at the history in the MTZ header (that might contain some information about the use of STARANISO and scaling)? There are two alternative explanations for why you now see additional map coefficients: * The MTZ file from STARANISO (if run via autoPROC [2]) might contain additional sets of amplitudes, namely F_early and F_late, that can be used not only to compute F(early)-F(late) radiation-damage detection maps [3], but also to compute normal electron-density and difference-density maps using those observed amplitudes in addition to the default ones used during refinement. * Looking at the completeness pattern, it might just be a case of the latest version now distinguishing the set of amplitudes before any outlier rejections (within refinement) and after such rejections: 1.21rc1_5058 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered 8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model 11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT 12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT 13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT 16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT ==> 98.14% complete amplitudes, but "only" 93.23% complete map-coefficients, i.e. maps computed /after/ outlier rejection 1.21.1-5286 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label 1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered 8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model 11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1 12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1 13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2 16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2 17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4 18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4 19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill 20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill 21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5 22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5 ==> 98.14% complete amplitudes, resulting in using all amplitudes before rejection (default = fill): 98.14% complete map coefficients FOFCWT_5 100.00% complete map coefficients 2FOFCWT_4 98.14% complete map coefficients 2FOFCWT_no_fill using amplitudes after rejection (default = nofill) 93.23% complete map coefficients FOFCWT_2 93.23% complete map coefficients 2FOFCWT_1 100.00% complete map coefficients 2FOFCWT_filled Does that make sense? It looks more like some rather useful provision of different ways of handling missing data (DFc completion, outlier rejection, unmeasured due to module gaps, cusp etc). It might be that anisotropic data as described by STARANISO, plus an assumption of isotropic diffraction (up to the highest diffraction limit), triggers the creation of these map coefficient variants. Does this ring a bell with developers? [1] https://staraniso.globalphasing.org/test_set_flags_about.html [2] https://www.globalphasing.com/autoproc/ [3] https://doi.org/10.1107/S2059798324001487 -- On Mon, May 20, 2024 at 03:53:01PM +0000, Dr. Kevin M Jude wrote:
Hi Billy, Here’s a dropbox linkhttps://www.dropbox.com/scl/fo/16qpqwy79pcm7zgl6j608/ADqFMcQcyUw-Jq6g-bt6LBE?rlkey=abq4uyk7oeavb0ckri9tgioek&dl=0 to input and output from one of my jobs
I note that I’ve encountered this issue with two anisotropic datasets that were ellipsoidally truncated using staraniso. This particular dataset uses a spherical cutoff, but it belongs to a student and I ran a cycle of refinement within one of my own projects that used staraniso data. May or may not be relevant, I just mention it in case an output setting is being created for the project that I don’t detect.
Best wishes Kevin
From: Billy Poon
Date: Saturday, May 18, 2024 at 1:23 PM To: Elke De Zitter Cc: Dr. Kevin M Jude , phenixbb Subject: Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke, Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
mailto:[email protected]> wrote: Dear Kevin, I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards, Elke
________________________________ From: "Dr. Kevin M Jude"
mailto:[email protected]> To: "phenixbb" mailto:[email protected]> Sent: Thursday, May 2, 2024 5:52:56 PM Subject: [phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered 8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model 11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT 12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT 13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT 16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286 Col Sort Min Max Num % Mean Mean Resolution Type Column num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H 2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K 3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L 4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs 5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs 6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags 7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered 8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered 9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model 10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model 11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1 12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1 13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled 14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled 15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2 16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2 17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4 18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4 19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill 20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill 21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5 22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes Kevin
-- Kevin Jude, PhD Structural Biology Research Specialist, Garcia Lab Howard Hughes Medical Institute Stanford University School of Medicine Beckman B177, 279 Campus Drive, Stanford CA 94305 Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]mailto:[email protected]
_______________________________________________ phenixbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]mailto:[email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Dear Billy,
I don't have this behavior when running with the default settings. I encounter this issue only when I when to output non-default map coefficients (e.g. B-sharpening). When running refinement via the command-line with the "write_maps=true" keyword, I add the following map.params file:
refinement{
electron_density_maps {
apply_default_maps = None
map_coefficients {
map_type = "2mFo-DFc"
format = *mtz phs
mtz_label_amplitudes = "2FOFCWT"
mtz_label_phases = "PH2FOFCWT"
fill_missing_f_obs = True
sharpening = True
sharpening_b_factor = None
}
map_coefficients {
map_type = "2mFo-DFc"
format = *mtz phs
mtz_label_amplitudes = "2FOFCWT_no_fill"
mtz_label_phases = "PH2FOFCWT_no_fill"
fill_missing_f_obs = False
sharpening = True
sharpening_b_factor = None
}
map_coefficients {
map_type = "mFo-DFc"
format = *mtz phs
mtz_label_amplitudes = "FOFCWT"
mtz_label_phases = "PHFOFCWT"
fill_missing_f_obs = False
sharpening = False
sharpening_b_factor = None
}
map {
map_type = "2mFo-DFc"
format = xplor *ccp4
file_name = None
fill_missing_f_obs = True
sharpening = True
sharpening_b_factor = None
}
map {
map_type = "2mFo-DFc"
format = xplor *ccp4
file_name = None
fill_missing_f_obs = False
sharpening = True
sharpening_b_factor = None
}
map {
map_type = "mFo-DFc"
format = xplor *ccp4
file_name = None
fill_missing_f_obs = False
sharpening = False
sharpening_b_factor = None
}
}
}
The mtz files I use have different types of column labels, hence I don't think it has to do with the input file. I can share files off-list if you wish.
All the best
Elke
From: "Billy Poon"
Hi all,
Yes, the issue is with how phenix.refine is parsing non-default output map
parameters. Essentially, the defaults get reapplied and maps with the
default parameters get generated. Because they have the same name, the
numbers are automatically added to the name. The current workaround is to
not change the map settings. We'll let you know when the fix is ready in a
nightly build. We will also make a new point release with the fix.
Let me know if you can create the issue without modifying the output map
parameters. Thank you.
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909
Web: https://phenix-online.org
On Wed, May 22, 2024 at 12:03 AM Elke De Zitter
Dear Billy,
I don't have this behavior when running with the default settings. I encounter this issue only when I when to output non-default map coefficients (e.g. B-sharpening). When running refinement via the command-line with the "write_maps=true" keyword, I add the following map.params file:
refinement{ electron_density_maps { apply_default_maps = None map_coefficients { map_type = "2mFo-DFc" format = *mtz phs mtz_label_amplitudes = "2FOFCWT" mtz_label_phases = "PH2FOFCWT" fill_missing_f_obs = True sharpening = True sharpening_b_factor = None } map_coefficients { map_type = "2mFo-DFc" format = *mtz phs mtz_label_amplitudes = "2FOFCWT_no_fill" mtz_label_phases = "PH2FOFCWT_no_fill" fill_missing_f_obs = False sharpening = True sharpening_b_factor = None } map_coefficients { map_type = "mFo-DFc" format = *mtz phs mtz_label_amplitudes = "FOFCWT" mtz_label_phases = "PHFOFCWT" fill_missing_f_obs = False sharpening = False sharpening_b_factor = None } map { map_type = "2mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = True sharpening = True sharpening_b_factor = None } map { map_type = "2mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = False sharpening = True sharpening_b_factor = None } map { map_type = "mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = False sharpening = False sharpening_b_factor = None } } }
The mtz files I use have different types of column labels, hence I don't think it has to do with the input file. I can share files off-list if you wish.
All the best Elke
------------------------------ *From: *"Billy Poon"
*To: *"elke de-zitter" *Cc: *"Dr. Kevin M Jude" , "phenixbb" < [email protected]> *Sent: *Saturday, May 18, 2024 10:22:59 PM *Subject: *Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke, Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
wrote: Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards, Elke
------------------------------ *From: *"Dr. Kevin M Jude"
*To: *"phenixbb" *Sent: *Thursday, May 2, 2024 5:52:56 PM *Subject: *[phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered
8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model
11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT
12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT
13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT
16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered
8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model
11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1
12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1
13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2
16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2
17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4
18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4
19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill
20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill
21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5
22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes
Kevin
--
Kevin Jude, PhD
Structural Biology Research Specialist, Garcia Lab
Howard Hughes Medical Institute
Stanford University School of Medicine
Beckman B177, 279 Campus Drive, Stanford CA 94305
Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
Dear Billy,
Thanks for having looked into this and for working on a fix.
I cannot reproduce this issue without modifying the output map parameters.
Kind regards,
Elke
From: "Billy Poon"
Hi, I have had this happen now with a completely new dataset that has nothing but I/SIGI, FP/SIGF and CCP4-type free flags, with both latest stable and nightly versions (on an M1 Mac), where no non-default map options were selected. I am confused about this, since I tried to Edit Parameters and Run, and saw no additional 2mFoFc coefficients mentioned in the .eff file. But when I let it proceed, log files shows that additional coefficients are added (see below). I assume that's why multiple 2FoFc's are created and loading automatically to coot fails. I then re-ran a random job from 2019 and that also now gives mtz map files with 2fofc_1 and 2fofc_3, so I am puzzled. I have not changed any output map parameters (definitely when re-running an old job), so the workaround Billy mentioned did not help. Happy to share files, Best, Engin ---- Log file, shows repeated 2mFoFc's defined electron_density_maps { map_coefficients { map_type = "2mFo-DFc" mtz_label_amplitudes = "2FOFCWT" mtz_label_phases = "PH2FOFCWT" } map_coefficients { map_type = "2mFo-DFc" mtz_label_amplitudes = "2FOFCWT_filled" mtz_label_phases = "PH2FOFCWT_filled" fill_missing_f_obs = True } map_coefficients { map_type = "anomalous" mtz_label_amplitudes = "ANOM" mtz_label_phases = "PHANOM" } map_coefficients { map_type = 2mFo-DFc mtz_label_amplitudes = 2FOFCWT mtz_label_phases = PH2FOFCWT fill_missing_f_obs = True } map_coefficients { map_type = 2mFo-DFc mtz_label_amplitudes = 2FOFCWT_no_fill mtz_label_phases = PH2FOFCWT_no_fill } map_coefficients { map_type = mFo-DFc mtz_label_amplitudes = FOFCWT mtz_label_phases = PHFOFCWT } map_coefficients { map_type = anomalous mtz_label_amplitudes = ANOM mtz_label_phases = PANOM } map { map_type = 2mFo-DFc fill_missing_f_obs = True } map { map_type = 2mFo-DFc } map { map_type = mFo-DFc } } On 5/23/24 6:05 PM, Billy Poon wrote:
Hi all,
Yes, the issue is with how phenix.refine is parsing non-default output map parameters. Essentially, the defaults get reapplied and maps with the default parameters get generated. Because they have the same name, the numbers are automatically added to the name. The current workaround is to not change the map settings. We'll let you know when the fix is ready in a nightly build. We will also make a new point release with the fix.
Let me know if you can create the issue without modifying the output map parameters. Thank you.
-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org
On Wed, May 22, 2024 at 12:03 AM Elke De Zitter
wrote: Dear Billy,
I don't have this behavior when running with the default settings. I encounter this issue only when I when to output non-default map coefficients (e.g. B-sharpening). When running refinement via the command-line with the "write_maps=true" keyword, I add the following map.params file:
refinement{ electron_density_maps { apply_default_maps = None map_coefficients { map_type = "2mFo-DFc" format = *mtz phs mtz_label_amplitudes = "2FOFCWT" mtz_label_phases = "PH2FOFCWT" fill_missing_f_obs = True sharpening = True sharpening_b_factor = None } map_coefficients { map_type = "2mFo-DFc" format = *mtz phs mtz_label_amplitudes = "2FOFCWT_no_fill" mtz_label_phases = "PH2FOFCWT_no_fill" fill_missing_f_obs = False sharpening = True sharpening_b_factor = None } map_coefficients { map_type = "mFo-DFc" format = *mtz phs mtz_label_amplitudes = "FOFCWT" mtz_label_phases = "PHFOFCWT" fill_missing_f_obs = False sharpening = False sharpening_b_factor = None } map { map_type = "2mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = True sharpening = True sharpening_b_factor = None } map { map_type = "2mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = False sharpening = True sharpening_b_factor = None } map { map_type = "mFo-DFc" format = xplor *ccp4 file_name = None fill_missing_f_obs = False sharpening = False sharpening_b_factor = None } } }
The mtz files I use have different types of column labels, hence I don't think it has to do with the input file. I can share files off-list if you wish.
All the best Elke
------------------------------------------------------------------------ *From: *"Billy Poon"
*To: *"elke de-zitter" *Cc: *"Dr. Kevin M Jude" , "phenixbb" *Sent: *Saturday, May 18, 2024 10:22:59 PM *Subject: *Re: [phenixbb] Extra map coefficients in 1.21.1-5286 Dear Kevin and Elke, Can you provide your input files and settings so that we can reproduce the issue? The settings are available from the "Results" tab with the "Config file" button. Alternatively, the settings are available by selecting the job in your job history and clicking the "Show details" button. A dialog will pop up and show the location of the job and the config file. I am not able to get the numbered map coefficients with a tutorial dataset with default settings. Thanks!
-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org
On Fri, May 17, 2024 at 6:18 AM Elke De Zitter
wrote: Dear Kevin,
I am facing the same problem upon running phenix.refine 1.21 from the command line with a map.params input file or via the GUI when the default map behavior is altered (e.g. B-sharpening). In phenix.refine 1.20, the new map coefficient parameters would replace the default ones. I have the impression that in phenix.refine 1.21 the new map coefficient parameters add up to the defaults, leading to many additional columns. I find it confusing as well since I have to figure out myself which coefficients are stored in *_1 etc. And, as you pointed out these mtz column labels are not automatically recognized.
It would be nice to have a workaround and get the same output as when using phenix 1.20. I couldn't find one (except for using version 1.20). Suggestions are thus welcome :-)
In case it is relevant, I am running phenix on a Mac. I tried version 1.21.1-5286 and dev-5330.
Kind regards, Elke
------------------------------------------------------------------------ *From: *"Dr. Kevin M Jude"
*To: *"phenixbb" *Sent: *Thursday, May 2, 2024 5:52:56 PM *Subject: *[phenixbb] Extra map coefficients in 1.21.1-5286 In my latest installed version of phenix, I find that additional, redundant map coefficients are being added to the output mtz file. I compare an identical refinement originally run in 1.21rc1_5058 and then repeated in 1.21.1-5286. *_filled and *_4 appear to be redundant, and the final sets of coefficients (*_no_fill and *_5), appear to include the test set reflections, even though I’ve selected Exclude R-free set from map coefficients in the gui. This is a little bit confusing, but primarily an annoyance because auto-open mtz in coot can’t figure out which coefficients to use.
1.21rc1_5058
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.2 2509.6 321 98.14 129.94 129.94 36.90 1.96 F F-obs-filtered
8 NONE 1.3 35.8 321 98.14 10.11 10.11 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.1 2250.2 321 98.14 125.18 125.18 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 4.35 89.81 36.90 1.96 P PHIF-model
11 NONE 0.0 2971.8 1165 93.23 117.81 117.81 36.90 1.96 F 2FOFCWT
12 NONE -180.0 180.0 1165 93.23 4.65 89.89 36.90 1.96 P PH2FOFCWT
13 NONE 0.0 2971.8 0 100.00 117.93 117.93 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.85 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 738.9 1165 93.23 46.86 46.86 36.90 1.96 F FOFCWT
16 NONE -180.0 180.0 1165 93.23 4.75 90.35 36.90 1.96 P PHFOFCWT
1.21.1-5286
Col Sort Min Max Num % Mean Mean Resolution Type Column
num order Missing complete abs. Low High label
1 NONE -42 41 0 100.00 -3.3 16.0 52.38 1.96 H H
2 NONE 0 28 0 100.00 10.3 10.3 52.38 1.96 H K
3 NONE 0 27 0 100.00 10.3 10.3 52.38 1.96 H L
4 NONE -94.4 13012.5 269 98.44 85.01 85.49 45.63 1.96 J I-obs
5 NONE 0.2 337.4 269 98.44 4.44 4.44 45.63 1.96 Q SIGI-obs
6 NONE 0.0 1.0 269 98.44 0.05 0.05 45.63 1.96 I R-free-flags
7 NONE 10.3 2535.0 321 98.14 131.25 131.25 36.90 1.96 F F-obs-filtered
8 NONE 1.3 36.1 321 98.14 10.21 10.21 36.90 1.96 Q SIGF-obs-filtered
9 NONE 0.0 2261.4 321 98.14 126.66 126.66 36.90 1.96 F F-model
10 NONE -180.0 180.0 321 98.14 3.96 89.82 36.90 1.96 P PHIF-model
11 NONE 0.0 3119.6 1165 93.23 119.71 119.71 36.90 1.96 F 2FOFCWT_1
12 NONE -180.0 180.0 1165 93.23 4.33 89.89 36.90 1.96 P PH2FOFCWT_1
13 NONE 0.0 3119.6 0 100.00 119.85 119.85 52.38 1.96 F 2FOFCWT_filled
14 NONE -180.0 180.0 0 100.00 4.44 90.05 52.38 1.96 P PH2FOFCWT_filled
15 NONE 0.0 780.4 1165 93.23 45.28 45.28 36.90 1.96 F FOFCWT_2
16 NONE -180.0 180.0 1165 93.23 4.21 90.36 36.90 1.96 P PHFOFCWT_2
17 NONE 0.0 3119.6 0 100.00 119.74 119.74 52.38 1.96 F 2FOFCWT_4
18 NONE -180.0 180.0 0 100.00 4.60 90.01 52.38 1.96 P PH2FOFCWT_4
19 NONE 0.0 3119.6 321 98.14 119.56 119.56 36.90 1.96 F 2FOFCWT_no_fill
20 NONE -180.0 180.0 321 98.14 4.40 89.94 36.90 1.96 P PH2FOFCWT_no_fill
21 NONE 0.0 780.4 321 98.14 45.68 45.68 36.90 1.96 F FOFCWT_5
22 NONE -180.0 180.0 321 98.14 4.12 90.23 36.90 1.96 P PHFOFCWT_5
Best wishes
Kevin
--
Kevin Jude, PhD
Structural Biology Research Specialist, Garcia Lab
Howard Hughes Medical Institute
Stanford University School of Medicine
Beckman B177, 279 Campus Drive, Stanford CA 94305
Phone: (650) 723-6431
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe: [email protected]
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb Unsubscribe:[email protected]
-- Engin Özkan, Ph.D. Associate Professor Dept of Biochemistry and Molecular Biology University of Chicago Phone: (773) 834-5498 http://ozkan.uchicago.edu
participants (5)
-
Billy Poon
-
Dr. Kevin M Jude
-
Elke De Zitter
-
Engin Özkan
-
Gerard Bricogne