Re: [phenixbb] Using the Same Test Set in AutoBuild and Phenix.Refine
Hi Dale, Can you try something else: phenix.refine AutoBuild_run_12_/overall_best.pdb \ refinement.input.xray_data.file_name=\ AutoBuild_run_12/exptl_fobs_freeR_flags.mtz \ refinement.main.high_resolution=2.2 refinement.main.low_resolution=20 \ /usr/users/dale/geometry/chromophores/bcl_tnt.cif This differs from your run only by substituting AutoBuild_run_12/exptl_fobs_freeR_flags.mtz for your 2 refinement data files. This is the exact file that is used in refinement by AutoBuild. I agree that you should be able to use your original data file instead. A possible reason why this has failed is that the original data file has a couple reflections for which there is no data...and which were tossed by AutoBuild before creating exptl_fobs_freeR_flags.mtz . Two files that differ only in reflections with no data will still give different checksums, I think. All the best, Tom T
Hi Dale,
1) Why you specify reflection MTZ file twice in phenix.refine script?
I put the mtz in twice because if I put it in once phenix.refine complains that I have no free R flags. It seems to want one file with the amplitudes and another with the flags. Since I have both in the same file I put that file on the line twice and phenix.refine finds everything it needs.
phenix.refine looks for free-R flags in your main data file (1M50-2.mtz). Optionally you can provide a separate file containing free-R flags (I have to write about this in the manual). However, if your 1M50-2.mtz contains free-R flags then you don't need to give it twice. So clearly something is wrong at this step and we need to find out what is wrong before doing anything else. Could you send the result of the command "phenix.mtz.dump 1M50-2.mtz" to see what's inside of your data file? Or I can debug it myself if you send me the data and model.
If the MD5 hash of the test set depends on the resolution then certainly I could be in trouble.
No. It must always use the original files before any processing.
Does the resolution limit affect the MD5 hash of the test set?
No. If it does then it is a very bad bug. I will play with this myself later tonight.
3) Does this work:
(...)
I'll try these but it will take a bit of time.
Don't run it until completion. Just make sure it passed through the processing step.
Pavel.
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Thomas C. Terwilliger wrote:
Hi Dale,
Can you try something else:
phenix.refine AutoBuild_run_12_/overall_best.pdb \ refinement.input.xray_data.file_name=\ AutoBuild_run_12/exptl_fobs_freeR_flags.mtz \ refinement.main.high_resolution=2.2 refinement.main.low_resolution=20 \ /usr/users/dale/geometry/chromophores/bcl_tnt.cif
This differs from your run only by substituting
AutoBuild_run_12/exptl_fobs_freeR_flags.mtz
for your 2 refinement data files. This is the exact file that is used in refinement by AutoBuild.
I tried this command, very similar to yours: phenix.refine AutoBuild_run_12_/overall_best.pdb \ refinement.input.xray_data.file_name=AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz \ refinement.main.high_resolution=2.2 refinement.main.low_resolution=20 \ /usr/users/dale/geometry/chromophores/bcl_tnt.cif output.prefix=junk2 The final output was: F-obs: AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz:FP,SIGFP If previously used R-free flags are available run this command again with the name of the file containing the original flags as an additional input. If the structure was never refined before, or if the original R-free flags are unrecoverable, run this command again with the additional definition: refinement.main.generate_r_free_flags=True If the structure was refined previously using different R-free flags, the values for R-free will become meaningful only after many cycles of refinement. Sorry: Please try again. The output from mtz.dump for your .mtz is Processing: AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz Title: Resolve mtz file. Space group symbol from file: P Space group number from file: 212 Space group from matrices: P 43 3 2 (No. 212) Point group symbol from file: 43 Number of crystals: 2 Number of Miller indices: 38159 Resolution range: 75.6238 2.14896 History: From resolve_huge, 27/12/07 15:07:56 Crystal 1: Name: HKL_base Project: HKL_base Id: 0 Unit cell: (169.1, 169.1, 169.1, 90, 90, 90) Number of datasets: 1 Dataset 1: Name: HKL_base Id: 0 Wavelength: 0 Number of columns: 0 Crystal 2: Name: allen- Project: FMO-ct Id: 2 Unit cell: (169.1, 169.1, 169.1, 90, 90, 90) Number of datasets: 1 Dataset 1: Name: 1 Id: 1 Wavelength: 0 Number of columns: 9 label #valid %valid min max type H 38159 100.00% 0.00 38.00 H: index h,k,l K 38159 100.00% 2.00 78.00 H: index h,k,l L 38159 100.00% 0.00 55.00 H: index h,k,l FP 38159 100.00% 32.00 15171.00 F: amplitude SIGFP 38159 100.00% 23.00 1716.00 Q: standard deviation PHIM 38159 100.00% -90.00 45.00 P: phase angle in degrees FOMM 38159 100.00% 0.00 0.00 W: weight (of some sort) FreeR_flag 38159 100.00% 0.00 19.00 I: integer FC 38159 100.00% 0.00 0.00 F: amplitude Dale Tronrud
I agree that you should be able to use your original data file instead. A possible reason why this has failed is that the original data file has a couple reflections for which there is no data...and which were tossed by AutoBuild before creating exptl_fobs_freeR_flags.mtz . Two files that differ only in reflections with no data will still give different checksums, I think.
All the best, Tom T
Hi Dale,
1) Why you specify reflection MTZ file twice in phenix.refine script?
I put the mtz in twice because if I put it in once phenix.refine complains that I have no free R flags. It seems to want one file with the amplitudes and another with the flags. Since I have both in the same file I put that file on the line twice and phenix.refine finds everything it needs.
phenix.refine looks for free-R flags in your main data file (1M50-2.mtz). Optionally you can provide a separate file containing free-R flags (I have to write about this in the manual). However, if your 1M50-2.mtz contains free-R flags then you don't need to give it twice. So clearly something is wrong at this step and we need to find out what is wrong before doing anything else. Could you send the result of the command "phenix.mtz.dump 1M50-2.mtz" to see what's inside of your data file? Or I can debug it myself if you send me the data and model.
If the MD5 hash of the test set depends on the resolution then certainly I could be in trouble.
No. It must always use the original files before any processing.
Does the resolution limit affect the MD5 hash of the test set?
No. If it does then it is a very bad bug. I will play with this myself later tonight.
3) Does this work:
(...) I'll try these but it will take a bit of time.
Don't run it until completion. Just make sure it passed through the processing step.
Pavel.
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
Hi Dale, this tells me that phenix.refine does not see the free-r flags in exptl_fobs_freeR_flags.mtz file. I'm not sure how Autobuild managed to run phenix.refine in this case. Tom is not available until Jan 21. Meanwhile I can try to debug it myself. To do so I need to reproduce this problem. Could you please send me the model and data and the exact commands (for autobuild and phenix.reifne) you used (send it to my mailbox and not to bb)? If the model / data are confidential feel free to manipulate them somehow (randomize the data, remove half of the model, ...) but make sure the program still fail. Also, I noticed that you are using an old version of PHENIX (I replaced "main.high_resolution=" with "xray_data.high_resolution=" a while ago). Since we keep improving things and many bugs were fixed since that time, it could be a good idea to try to run this with the latest version of PHENIX (I can do this if you send me the data). Thanks! Pavel. On 1/2/2008 4:47 PM, Dale Tronrud wrote:
Thomas C. Terwilliger wrote:
Hi Dale,
Can you try something else:
phenix.refine AutoBuild_run_12_/overall_best.pdb \ refinement.input.xray_data.file_name=\ AutoBuild_run_12/exptl_fobs_freeR_flags.mtz \ refinement.main.high_resolution=2.2 refinement.main.low_resolution=20 \ /usr/users/dale/geometry/chromophores/bcl_tnt.cif
This differs from your run only by substituting
AutoBuild_run_12/exptl_fobs_freeR_flags.mtz
for your 2 refinement data files. This is the exact file that is used in refinement by AutoBuild.
I tried this command, very similar to yours:
phenix.refine AutoBuild_run_12_/overall_best.pdb \ refinement.input.xray_data.file_name=AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz \ refinement.main.high_resolution=2.2 refinement.main.low_resolution=20 \ /usr/users/dale/geometry/chromophores/bcl_tnt.cif output.prefix=junk2
The final output was:
F-obs: AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz:FP,SIGFP
If previously used R-free flags are available run this command again with the name of the file containing the original flags as an additional input. If the structure was never refined before, or if the original R-free flags are unrecoverable, run this command again with the additional definition:
refinement.main.generate_r_free_flags=True
If the structure was refined previously using different R-free flags, the values for R-free will become meaningful only after many cycles of refinement.
Sorry: Please try again.
The output from mtz.dump for your .mtz is
Processing: AutoBuild_run_12_/exptl_fobs_phases_freeR_flags.mtz Title: Resolve mtz file. Space group symbol from file: P Space group number from file: 212 Space group from matrices: P 43 3 2 (No. 212) Point group symbol from file: 43 Number of crystals: 2 Number of Miller indices: 38159 Resolution range: 75.6238 2.14896 History: From resolve_huge, 27/12/07 15:07:56 Crystal 1: Name: HKL_base Project: HKL_base Id: 0 Unit cell: (169.1, 169.1, 169.1, 90, 90, 90) Number of datasets: 1 Dataset 1: Name: HKL_base Id: 0 Wavelength: 0 Number of columns: 0 Crystal 2: Name: allen- Project: FMO-ct Id: 2 Unit cell: (169.1, 169.1, 169.1, 90, 90, 90) Number of datasets: 1 Dataset 1: Name: 1 Id: 1 Wavelength: 0 Number of columns: 9 label #valid %valid min max type H 38159 100.00% 0.00 38.00 H: index h,k,l K 38159 100.00% 2.00 78.00 H: index h,k,l L 38159 100.00% 0.00 55.00 H: index h,k,l FP 38159 100.00% 32.00 15171.00 F: amplitude SIGFP 38159 100.00% 23.00 1716.00 Q: standard deviation PHIM 38159 100.00% -90.00 45.00 P: phase angle in degrees FOMM 38159 100.00% 0.00 0.00 W: weight (of some sort) FreeR_flag 38159 100.00% 0.00 19.00 I: integer FC 38159 100.00% 0.00 0.00 F: amplitude
Dale Tronrud
I agree that you should be able to use your original data file instead. A possible reason why this has failed is that the original data file has a couple reflections for which there is no data...and which were tossed by AutoBuild before creating exptl_fobs_freeR_flags.mtz . Two files that differ only in reflections with no data will still give different checksums, I think.
All the best, Tom T
Hi Dale,
1) Why you specify reflection MTZ file twice in phenix.refine script?
I put the mtz in twice because if I put it in once phenix.refine complains that I have no free R flags. It seems to want one file with the amplitudes and another with the flags. Since I have both in the same file I put that file on the line twice and phenix.refine finds everything it needs.
phenix.refine looks for free-R flags in your main data file (1M50-2.mtz). Optionally you can provide a separate file containing free-R flags (I have to write about this in the manual). However, if your 1M50-2.mtz contains free-R flags then you don't need to give it twice. So clearly something is wrong at this step and we need to find out what is wrong before doing anything else. Could you send the result of the command "phenix.mtz.dump 1M50-2.mtz" to see what's inside of your data file? Or I can debug it myself if you send me the data and model.
If the MD5 hash of the test set depends on the resolution then certainly I could be in trouble.
No. It must always use the original files before any processing.
Does the resolution limit affect the MD5 hash of the test set?
No. If it does then it is a very bad bug. I will play with this myself later tonight.
3) Does this work:
(...)
I'll try these but it will take a bit of time.
Don't run it until completion. Just make sure it passed through the processing step.
Pavel.
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
_______________________________________________ phenixbb mailing list [email protected] http://www.phenix-online.org/mailman/listinfo/phenixbb
participants (3)
-
Dale Tronrud -
Pavel Afonine -
Thomas C. Terwilliger