Pavel wanted some evidence of whether or not it makes a difference to omit very weak reflections. Here's one relevant paper. Hirshfeld and Rabinovich (Acta Cryst. A29: 510-513, 1973) showed in a numerical experiment that, if you omit weak intensities, there is a systematic error in refined scale and ADP parameters. They used least squares so it's possible that the results would be somewhat different with maximum likelihood targets, but at least here is an objective demonstration that the weak data can have a significant influence. Regards, Randy On 7 Dec 2009, at 15:20, Ed Pozharski wrote:
Pavel,
Here is an example of the SCALEPACK log-file. My reading of it is that in the highest resolution shell I have 25% of reflections with I/sigma less than zero, thus negative intensities. Again, it doesn't bother *me*, because I use French&Wilson to convert the negative intensities to positive amplitudes and would like to reiterate that neither phenix.refine nor any other program can be blamed for what is essentially user error in assigning these reflections zero amplitude. However, Joe's point, imho, was that *if* they are assigned zero amplitudes significant portion of the data will be excluded if FOBS>0 is required. They are not a few, in the dataset that I pulled this from they are apparently >12000 out of about 100000 overall.
Shell I/Sigma in resolution shells: Lower Upper % of of reflections with I / Sigma less than limit limit 0 1 2 3 5 10 20 >20 total 50.00 4.52 0.6 1.8 3.4 5.3 8.8 17.4 33.4 66.3 99.8 4.52 3.59 1.0 2.7 5.6 8.6 14.7 28.1 50.4 49.5 99.9 3.59 3.14 2.8 7.2 13.1 19.5 30.2 48.7 70.8 29.2 99.9 3.14 2.85 6.8 16.4 27.9 37.0 50.4 68.9 86.1 13.8 99.9 2.85 2.65 11.5 26.2 41.4 52.0 65.5 81.5 94.0 5.8 99.8 2.65 2.49 15.4 35.0 52.4 63.4 75.3 89.0 97.0 2.7 99.7 2.49 2.37 17.9 41.1 59.0 69.9 80.8 91.8 97.9 1.6 99.6 2.37 2.26 21.1 46.6 64.9 75.0 84.9 93.6 98.5 1.0 99.5 2.26 2.18 22.6 50.5 70.0 79.8 88.7 96.1 98.9 0.3 99.2 2.18 2.10 25.1 56.1 76.0 84.9 92.4 97.8 99.1 0.1 99.3 All hkl 12.4 28.3 41.2 49.4 59.0 71.1 82.4 17.2 99.7
I was just wondering how better (and what is "better" in this case) the refinement results will be if one includes the negative (zero) reflections into the process of refinement, and did anyone clearly demonstrate it. That's all I asked.
This is indeed the key question - how do you determine if your model is getting "better"? R-factors are no good here, as they will definitely improve upon rejection of weak reflections. I guess what most people would accept is the following:
1. Generate synthetic data (e.g. from molecular dynamics trajectories) and use MLFSOM and your favorite data reduction software to obtain a dataset. 2. Convert negative intensities to zero amplitudes. 3. Refine the model with and without FOBS=0 cutoff. 4. Determine the additional error introduced by the data rejection.
My suspicion is that the result will be more complex than a simple yes/no answer. It's possible that rejection of negative intensities plays significant role only in certain resolution domain, but at least the effect should be resolution-dependent (as well as vary from dataset to dataset).
But one thing we can be certain about. Except for the unlikely but real scenario you mentioned when data well beyond I/s=1 resolution cutoff is somehow included, it is my expectation that by including the weak reflections the model quality will *not* be decreased. If I am right (and that is an if), then removing the FOBS=0 cutoff does no harm and has the potential to improve models at least on some occasions.
Cheers,
Ed.
-- Edwin Pozharski, PhD, Assistant Professor University of Maryland, Baltimore ---------------------------------------------- When the Way is forgotten duty and justice appear; Then knowledge and wisdom are born along with hypocrisy. When harmonious relationships dissolve then respect and devotion arise; When a nation falls to chaos then loyalty and patriotism are born. ------------------------------ / Lao Tse /
_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb
------ Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical Research Tel: + 44 1223 336500 Wellcome Trust/MRC Building Fax: + 44 1223 336827 Hills Road E-mail: [email protected] Cambridge CB2 0XY, U.K. www- structmed.cimr.cam.ac.uk