Hi Leif,

I thought this would be simple, but this is nowhere near as clear as finding the values for X-rays and electrons.

it took me a few seconds to find it, although I believe I have an advantage of working with this code for about 11 years -;) Neutron: cctbx_project/cctbx/eltbx/neutron.cpp Electron: same directory, look for files with "electron" in names. Xray: cctbx_project/cctbx/eltbx/xray_scattering Note, for Xray there are 3 choices available: it1992 (form-factors from Intl.Tabl. 1992), wk1995 (D. Waasmaier & A. Kirfel. Acta Cryst. (1995). A51, 416-431. "New analytical scattering-factor functions for free atoms and ions") and n_gaussian, which is a N-gaussian dynamic approximation to the table data and that is used by default.

Where is the neutron_news_1992_table located in the code? Is there anyway to redefine the scattering for specific isotopes?

See my reply to Maxime's email.

And why use these values over those published by Rauch and Waschkowski in the Neutron Data Booklet (2003), other than the horribly small print?

I didn't get this.. Could you please explain what you mean? We added neutron scattering table well before we even planned to add neutron refinement option to phenix.refine. If you think there is a better option then I am happy to add it. Pavel

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear Leif, you can still make use of the X-ray structure if you use external restraints - they can stabilise the non-H atom structure. I don't understand why you need free variables for each of the exchangeable hydrogens. If you were trying to determine the exchange rate that might be a little over what crystallography can do for you. Yet, you find a description of how you can do this from occupancy refinement without the use of an abundant use of free variables in our paper http://dx.doi.org/10.1107/S1600576713027659 There we made reasonable groups, but if you think you can justify it, you could do the same for each hydrogen position. I noticed that there are structures from neutron diffraction in the PDB with hydrogen occupancies of 0.02 and the respective deuterium at 0.98 - I don't think such an accuracy is justifiable from crystallographic data, this is merely overfitting the data. Cheers, Tim On 12/04/2014 02:33 PM, Leif Hanson wrote:

Tim, As I understand joint refinement (although Paul Langan and Marat Mustyakimov can give a better answer), the X-ray data are used to establish the non-H atom positions, and n0 data to establish H and D positions. From a practical standpoint, shelx was wonderful for defining the scattering factors of the atoms. But we had issues with the length of the input file since we had to create free variables for each of the exchangeable hydrogens. With more than 400 residues this got a little crazy. Leif

On Thu, Dec 4, 2014 at 4:14 AM, Tim Gruene wrote:

Hi Maxime,

you could use shelxl for refinement - it uses the values from the Neutron Data Booklet for the most abundant isotopes, and you can mix them with your own scattering values without even looking at the code. You can even take into account incoherent contributions by adjusting the f' and f'' values on the SFAC command like NEUT SFAC C H N O S D SFAC FeX 0 0 0 0 0 0 0 0 4.20 0 0 11.220 1.23 56 SFAC Co

if you have e.g. Fe-54

If you want to have a joint refinement between X-ray and neutron data, I recommend using the X-ray structure by external restraints rather than mixing two different types of experiments. You won't e.g need to worry about different effective hydrogen bond lengths. Published restraints for hydrogen atoms to use with neutron data are available from my web-site, for ligands they can be generated by the grade-server.

Regards, Tim

On 12/03/2014 10:36 PM, Maxime Cuypers wrote:

...

...

...

Hello,

I would like to alter the neutron scattering table for phenix.refine so that it takes into account the correct bcoherent value for the metal isotope present in my structure. the difference is significative between the natural occurence bcoh... i have been looking around in chem_data but could not find the neutron scattering tables. does anyone have any idea where to look please?

cheerios

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

...

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

- -- - -- Dr Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen GPG Key ID = A46BEE1A -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iD8DBQFUgGWhUxlJ7aRr7hoRAvyLAKCBnNE1TGDF4lSNr55GM7fsPFPxEgCggnDQ s8gRaBEy/ZTBQrwGkyI0aQo= =V/m0 -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Leif, the procedure you suggest, i.e. to fix the coordinates from the refinement against X-ray data, is of course possible in shelxl. This means you use the X-ray structure as CONstraint. I would, though, prefer to generate REstraints with prosmart and transcribe its output into shelx format - which is not too difficult. This way you allow room for differences between X-ray and neutron diffractions that we may simply not be aware of (or are aware of like the radiation damage you mention). When I refined against neutron data, I did not consider it necessary to take the X-ray structure into account at all. I guess you are arguing that by using constraints there are more data available to refine B-values AND occupancy. You are probably aware that these to numbers are strongly correlated (>=90%!!) so that it is very tricky to get get reliable numbers anyhow (see e.g. http://www.sciencemag.org/content/346/6207/352). (Very) high resolution data should help to reduce the CC, but I would be cautious to think that mathematically increasing the data to parameter ratio by using constraints improves the CC. But I never tried - shelxl prints the highest CC values if you use 'MORE 3' and L.S. as minimisation method (instead of CGLS). Regards, Tim On 12/04/2014 03:12 PM, Leif Hanson wrote:

Tim, Let me see if I understand this process flow properly. Let us assume that we have measured two datasets from the same crystal, one neutron, one X-ray (neutron always measured first). I use shelx to refine the x-ray structure to 'stability'. I then use that structure and the input cards from X-ray structure to solve the neutron structure. This is normally what we do. We do not allow the structure to move after X-ray positional refinement, refining only b-factors and occupancies. The primary activity at this point is resolving protonation and water orientation. There are some issues with this strategy (what about X-radiation damage to acidic residues and water molecules?), but it has been reliable to this point. The features for neutron refinement in Phenix as I understand it position the H and D atoms at the end of each major cycle, then complete the b and q refinement. Does this differ from what you are suggesting? I agree that the 0.98 occupancy should be considered unity, although it might be valid for Julian Chen's crambin structure. Leif

On Thu, Dec 4, 2014 at 8:46 AM, Tim Gruene wrote:

Dear Leif,

you can still make use of the X-ray structure if you use external restraints - they can stabilise the non-H atom structure.

I don't understand why you need free variables for each of the exchangeable hydrogens. If you were trying to determine the exchange rate that might be a little over what crystallography can do for you. Yet, you find a description of how you can do this from occupancy refinement without the use of an abundant use of free variables in our paper http://dx.doi.org/10.1107/S1600576713027659 There we made reasonable groups, but if you think you can justify it, you could do the same for each hydrogen position.

I noticed that there are structures from neutron diffraction in the PDB with hydrogen occupancies of 0.02 and the respective deuterium at 0.98 - I don't think such an accuracy is justifiable from crystallographic data, this is merely overfitting the data.

Cheers, Tim

On 12/04/2014 02:33 PM, Leif Hanson wrote:

...

...

...

Tim, As I understand joint refinement (although Paul Langan and Marat Mustyakimov can give a better answer), the X-ray data are used to establish the non-H atom positions, and n0 data to establish H and D positions. From a practical standpoint, shelx was wonderful for defining the scattering factors of the atoms. But we had issues with the length of the input file since we had to create free variables for each of the exchangeable hydrogens. With more than 400 residues this got a little crazy. Leif

On Thu, Dec 4, 2014 at 4:14 AM, Tim Gruene wrote:

Hi Maxime,

you could use shelxl for refinement - it uses the values from the Neutron Data Booklet for the most abundant isotopes, and you can mix them with your own scattering values without even looking at the code. You can even take into account incoherent contributions by adjusting the f' and f'' values on the SFAC command like NEUT SFAC C H N O S D SFAC FeX 0 0 0 0 0 0 0 0 4.20 0 0 11.220 1.23 56 SFAC Co

if you have e.g. Fe-54

If you want to have a joint refinement between X-ray and neutron data, I recommend using the X-ray structure by external restraints rather than mixing two different types of experiments. You won't e.g need to worry about different effective hydrogen bond lengths. Published restraints for hydrogen atoms to use with neutron data are available from my web-site, for ligands they can be generated by the grade-server.

Regards, Tim

On 12/03/2014 10:36 PM, Maxime Cuypers wrote:

...

...

> Hello, > > I would like to alter the neutron scattering table for > phenix.refine so that it takes into account the > correct bcoherent value for the metal isotope present > in my structure. the difference is significative > between the natural occurence bcoh... i have been > looking around in chem_data but could not find the > neutron scattering tables. does anyone have any idea > where to look please? > > cheerios > > > > _______________________________________________ > phenixbb mailing list [email protected] > http://phenix-online.org/mailman/listinfo/phenixbb >

...

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

...

- -- - -- Dr Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen GPG Key ID = A46BEE1A -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iD8DBQFUgHFAUxlJ7aRr7hoRAizNAJ9vmKVFJYcLzC4k8rBguFK4+wxFqgCeLDss y+bTbfBGFkEZI41i9HyCIY0= =iqfr -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You haven't quite finished the job Pavel. In your test you are always starting with an atom with occupancy of 1. Those atoms we usually know have an occupancy of 1 so we constrain to that value. The real question come up with the occupancy is less than one, but we don't know either the B or the Occ. Then you need to plot R value in 2-D as both Occ and B are varied. The correlation we all talk of is due to a diagonal line of minimal R in that plot. Since there are a bunch of different combinations of B and Occ that give similar fit we end up with a relationship between the possible values. Certainly when you change EITHER Occ or B alone (as you have done here) you see a change to the fit to the data. Use your CCTBX wizardry to show us the 2-D plot! Dale Tronrud On 12/4/2014 2:29 PM, Pavel Afonine wrote:

Hello,

...

I guess you are arguing that by using constraints there are more data available to refine B-values AND occupancy. You are probably aware that these to numbers are strongly correlated (>=90%!!) so that it is very tricky to get get reliable numbers anyhow

sometimes numbers excite me! So this one caught my attention and I decided to entertain myself.

First off, an obvious statement: occupancy defines peak's height and B-factor defines its shape. Therefore one cannot be entirely compensated with the other.

Now let's see if and how occupancy and B-factor are correlated. For this let's take an atom and plot its electron density distribution with occupancy q=1 and some B value; let's call this density rho_ref (reference map). Then let's vary occupancy from 0.1 to 1.0 (with step 0.1) and for each trial occupancy value find such B_opt that corresponding electron density distribution fits rho_ref as good as possible; let's call it rho_opt (map corresponding to optimal B_opt). In the end we will have ten occupancy values and ten corresponding optimal B values so that we can calculate the correlation between two sets of numbers (q, B_opt). In addition let's calculate correlation and R-factor for rho_ref and rho_opt.

We will repeat the numerical experiment defined above with: a) different starting B values (10, 30, 50, 80), b) different atoms H, C, S, c) exact electron density distribution as well as its Fourier image of 2A resolution.

Attached script does it all in one go. Also it illustrates the beauty of CCTBX that allows to do this so easily!

Here are the numbers:

Resolution: None (exact map)---------------------------------------------------------- atom: H B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 2.00 3.00 5.00 6.00 7.00 9.00 10.00 CC(rho_ref,rho_opt): 0.95 0.95 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 34.94 34.94 34.94 28.30 23.09 14.83 11.36 8.19 2.55 0.00 CC(q,B): 0.97 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 4.00 7.00 11.00 14.00 17.00 21.00 24.00 27.00 30.00 CC(rho_ref,rho_opt): 0.85 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 54.39 42.43 34.42 26.09 20.84 16.16 10.59 6.81 3.29 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 3.00 9.00 15.00 20.00 26.00 31.00 36.00 41.00 45.00 50.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 55.33 42.14 32.91 26.59 20.01 15.15 10.72 6.64 3.59 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 8.00 18.00 27.00 35.00 43.00 51.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.85 0.92 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 53.74 41.14 32.57 26.13 20.44 15.31 10.64 6.86 3.32 0.00 CC(q,B): 1.00 atom: C B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 CC(rho_ref,rho_opt): 0.79 0.88 0.93 0.96 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 62.77 49.83 39.70 31.37 24.32 18.23 12.89 8.14 3.87 0.00 CC(q,B): 1.00 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 4.00 7.00 10.00 13.00 16.00 19.00 21.00 24.00 27.00 30.00 CC(rho_ref,rho_opt): 0.83 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 56.51 43.81 34.49 27.06 20.84 15.45 12.21 7.76 3.72 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 6.00 12.00 17.00 22.00 27.00 32.00 36.00 41.00 46.00 50.00 CC(rho_ref,rho_opt): 0.82 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 57.64 42.58 33.79 26.68 20.63 15.32 11.47 7.06 3.02 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 11.00 20.00 28.00 36.00 44.00 52.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 54.69 41.90 33.51 26.57 20.56 15.22 10.98 7.06 3.41 0.00 CC(q,B): 1.00 atom: S B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 CC(rho_ref,rho_opt): 0.82 0.89 0.93 0.96 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 59.32 47.29 38.28 30.75 24.21 18.39 13.15 8.39 4.03 0.00 CC(q,B): 1.00 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 4.00 8.00 11.00 14.00 17.00 19.00 22.00 25.00 27.00 30.00 CC(rho_ref,rho_opt): 0.82 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 57.49 42.56 33.98 26.75 20.48 16.72 11.57 6.91 4.03 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 8.00 13.00 19.00 23.00 28.00 33.00 37.00 42.00 46.00 50.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 54.25 43.18 32.90 27.17 20.88 15.31 11.27 6.64 3.21 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 13.00 22.00 30.00 38.00 46.00 53.00 60.00 67.00 73.00 80.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 53.98 41.82 33.40 26.32 20.16 15.32 10.91 6.83 3.57 0.00 CC(q,B): 1.00 Resolution: 2.0 ----------------------------------------------------------------------

atom: H

B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 2.00 6.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.28 2.66 0.00 CC(q,B): 0.72 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 4.00 10.00 16.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.17 18.17 18.17 18.17 16.35 12.62 8.83 5.66 2.50 0.00 CC(q,B): 0.95 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 2.00 12.00 20.00 27.00 33.00 39.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 27.44 27.44 26.91 21.35 16.76 12.70 9.30 5.94 2.66 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 6.00 20.00 31.00 41.00 50.00 58.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 36.70 34.44 27.69 22.24 17.34 13.03 9.33 5.80 2.84 0.00 CC(q,B): 1.00 atom: C B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3.00 6.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.96 5.96 5.96 5.96 5.96 5.96 5.96 4.65 2.67 0.00 CC(q,B): 0.75 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 5.00 11.00 16.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.37 18.37 18.37 18.37 15.93 12.16 8.96 5.75 2.54 0.00 CC(q,B): 0.95 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 3.00 12.00 20.00 27.00 33.00 39.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 27.82 27.82 26.75 21.70 17.07 12.95 9.48 6.06 2.71 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 7.00 21.00 32.00 41.00 50.00 58.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 37.14 34.45 27.63 22.10 17.64 13.25 9.48 5.90 2.88 0.00 CC(q,B): 1.00 atom: S B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3.00 7.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.88 5.88 5.88 5.88 5.88 5.88 5.88 4.60 1.99 0.00 CC(q,B): 0.76 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 6.00 11.00 17.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.46 18.46 18.46 18.46 15.43 12.30 8.45 5.84 2.59 0.00 CC(q,B): 0.96 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 4.00 13.00 21.00 28.00 34.00 40.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 28.30 28.30 26.67 21.63 16.92 12.75 9.18 5.68 2.81 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 9.00 22.00 33.00 43.00 51.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 37.91 34.26 27.88 22.25 17.20 13.22 9.35 6.10 2.99 0.00 CC(q,B): 0.99

What we see here is: - correlation of q and B is indeed approaches 100%; - map correlation is greater than 90% in most cases except a few corner cases; - the last column in all tests is an obvious sanity check (CC=1, R=0 if exact B and q are used); - R-factors are greater than zero except a trivial case. This is the key that makes it possible to deconvolute q and B.

All the best, Pavel

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iEYEARECAAYFAlSA6zAACgkQU5C0gGfAG13hmQCffgQcmT94H12EDsNIE6UTdcx/ wFUAn0RWRfce/o8nQXZpKDowZkhS+1Vi =2mMN -----END PGP SIGNATURE-----

I have a question on these occupancies with respect to labile H atoms. During the exchange process, we assume that the ratio of H to D at a given atom will vary from 1 to 0 as deuteration increases. However, since the scattering length varies from negative to positive (-0.3 to 0.6 fm), does this enhance the ability to determine the occupancy? In nCNS this shows up in the q column as -0.5 for H where 1 is a D. For Phenix where both H and D for a given site are listed, the q values vary from 0 to 1, although the fractional values don't necessarily add to 1. I disremember whether the q value goes negative in Shelx. To follow on what Ed said, if one assumes that half of the atoms in a structure are H, and 1/3 of those are labile, then up to 1/6 of the structure has some variability for q. If one examines a His residue and the scattering for one proton position is zero does this mean nothing is there, or does it mean that it has 0.66 occupancy for His? Would I really expect to see a change on R at this site with either no proton, or 0.66 H? Leif On Thu, Dec 4, 2014 at 6:44 PM, Edward A. Berry wrote:

atom: C B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3.00 6.00 10.00 CC(rho_ref,rho_opt) : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.96 5.96 5.96 5.96 5.96 5.96 5.96 4.65 2.67 0.00 CC(q,B): 0.75

In cases where the CC(q,B) is poor like this, it seems to be because the B factor has pegged at 1.0, which it reaches at occupancy 0.7. Then as occupancy continues to decrease to 0.1, B remains the same, so CC(q,B) is low.

And it is not surprising the CC(rho_ref,rho_opt) is very good in all cases, since you hardly change the shape of the map by dropping B by 9A^2. Note that if you set occupancy to 0.01 and didn't change the B-factor at all you would still get CC(rho_ref,rho_opt) =1. This doesn't mean we can't distinguish occupancy 0.01 from 1.0!

It _is_ surprising that R also doesn't change as the occupancy drops from .7 to .1 with no compensation by B. This tells me there is some scaling going on in the calculation of R. In a real case with thousands of atoms, the scale would be fixed by the great majority of atoms at occupancy 1.0, and low occupancy for this atom would make a difference. So i suggest the R factor should be calculated without scaling. R going through the roof would then tell us that B is not successfully compensating for q. But I agree a 3-d plot of R vs B and q is the best way to show this. eab

On 12/04/2014 05:29 PM, Pavel Afonine wrote:

...

Hello,

I guess you are

...

arguing that by using constraints there are more data available to refine B-values AND occupancy. You are probably aware that these to numbers are strongly correlated (>=90%!!) so that it is very tricky to get get reliable numbers anyhow

sometimes numbers excite me! So this one caught my attention and I decided to entertain myself.

First off, an obvious statement: occupancy defines peak's height and B-factor defines its shape. Therefore one cannot be entirely compensated with the other.

Now let's see if and how occupancy and B-factor are correlated. For this let's take an atom and plot its electron density distribution with occupancy q=1 and some B value; let's call this density rho_ref (reference map). Then let's vary occupancy from 0.1 to 1.0 (with step 0.1) and for each trial occupancy value find such B_opt that corresponding electron density distribution fits rho_ref as good as possible; let's call it rho_opt (map corresponding to optimal B_opt). In the end we will have ten occupancy values and ten corresponding optimal B values so that we can calculate the correlation between two sets of numbers (q, B_opt). In addition let's calculate correlation and R-factor for rho_ref and rho_opt.

We will repeat the numerical experiment defined above with: a) different starting B values (10, 30, 50, 80), b) different atoms H, C, S, c) exact electron density distribution as well as its Fourier image of 2A resolution.

Attached script does it all in one go. Also it illustrates the beauty of CCTBX that allows to do this so easily!

Here are the numbers:

Resolution: None (exact map)-------------------------- -------------------------------- atom: H B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 2.00 3.00 5.00 6.00 7.00 9.00 10.00 CC(rho_ref,rho_opt): 0.95 0.95 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 34.94 34.94 34.94 28.30 23.09 14.83 11.36 8.19 2.55 0.00 CC(q,B): 0.97 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 4.00 7.00 11.00 14.00 17.00 21.00 24.00 27.00 30.00 CC(rho_ref,rho_opt): 0.85 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 54.39 42.43 34.42 26.09 20.84 16.16 10.59 6.81 3.29 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 3.00 9.00 15.00 20.00 26.00 31.00 36.00 41.00 45.00 50.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 55.33 42.14 32.91 26.59 20.01 15.15 10.72 6.64 3.59 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 8.00 18.00 27.00 35.00 43.00 51.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.85 0.92 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 53.74 41.14 32.57 26.13 20.44 15.31 10.64 6.86 3.32 0.00 CC(q,B): 1.00 atom: C B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 CC(rho_ref,rho_opt): 0.79 0.88 0.93 0.96 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 62.77 49.83 39.70 31.37 24.32 18.23 12.89 8.14 3.87 0.00 CC(q,B): 1.00 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 4.00 7.00 10.00 13.00 16.00 19.00 21.00 24.00 27.00 30.00 CC(rho_ref,rho_opt): 0.83 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 56.51 43.81 34.49 27.06 20.84 15.45 12.21 7.76 3.72 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 6.00 12.00 17.00 22.00 27.00 32.00 36.00 41.00 46.00 50.00 CC(rho_ref,rho_opt): 0.82 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 57.64 42.58 33.79 26.68 20.63 15.32 11.47 7.06 3.02 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 11.00 20.00 28.00 36.00 44.00 52.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 54.69 41.90 33.51 26.57 20.56 15.22 10.98 7.06 3.41 0.00 CC(q,B): 1.00 atom: S B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 10.00 CC(rho_ref,rho_opt): 0.82 0.89 0.93 0.96 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 59.32 47.29 38.28 30.75 24.21 18.39 13.15 8.39 4.03 0.00 CC(q,B): 1.00 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 4.00 8.00 11.00 14.00 17.00 19.00 22.00 25.00 27.00 30.00 CC(rho_ref,rho_opt): 0.82 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 57.49 42.56 33.98 26.75 20.48 16.72 11.57 6.91 4.03 0.00 CC(q,B): 1.00 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 8.00 13.00 19.00 23.00 28.00 33.00 37.00 42.00 46.00 50.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 0.99 1.00 1.00 1.00 R(%) : 54.25 43.18 32.90 27.17 20.88 15.31 11.27 6.64 3.21 0.00 CC(q,B): 1.00 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 13.00 22.00 30.00 38.00 46.00 53.00 60.00 67.00 73.00 80.00 CC(rho_ref,rho_opt): 0.84 0.91 0.95 0.97 0.98 0.99 1.00 1.00 1.00 1.00 R(%) : 53.98 41.82 33.40 26.32 20.16 15.32 10.91 6.83 3.57 0.00 CC(q,B): 1.00 Resolution: 2.0 ------------------------------ ---------------------------------------- atom: H B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 2.00 6.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.28 2.66 0.00 CC(q,B): 0.72 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 4.00 10.00 16.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.17 18.17 18.17 18.17 16.35 12.62 8.83 5.66 2.50 0.00 CC(q,B): 0.95 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 2.00 12.00 20.00 27.00 33.00 39.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 27.44 27.44 26.91 21.35 16.76 12.70 9.30 5.94 2.66 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 6.00 20.00 31.00 41.00 50.00 58.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 36.70 34.44 27.69 22.24 17.34 13.03 9.33 5.80 2.84 0.00 CC(q,B): 1.00 atom: C B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3.00 6.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.96 5.96 5.96 5.96 5.96 5.96 5.96 4.65 2.67 0.00 CC(q,B): 0.75 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 5.00 11.00 16.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.37 18.37 18.37 18.37 15.93 12.16 8.96 5.75 2.54 0.00 CC(q,B): 0.95 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 3.00 12.00 20.00 27.00 33.00 39.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 27.82 27.82 26.75 21.70 17.07 12.95 9.48 6.06 2.71 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 7.00 21.00 32.00 41.00 50.00 58.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 37.14 34.45 27.63 22.10 17.64 13.25 9.48 5.90 2.88 0.00 CC(q,B): 1.00 atom: S B: 10 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 1.00 1.00 1.00 3.00 7.00 10.00 CC(rho_ref,rho_opt): 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 R(%) : 5.88 5.88 5.88 5.88 5.88 5.88 5.88 4.60 1.99 0.00 CC(q,B): 0.76 B: 30 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 1.00 1.00 6.00 11.00 17.00 21.00 26.00 30.00 CC(rho_ref,rho_opt): 0.99 0.99 0.99 0.99 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 18.46 18.46 18.46 18.46 15.43 12.30 8.45 5.84 2.59 0.00 CC(q,B): 0.96 B: 50 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 1.00 4.00 13.00 21.00 28.00 34.00 40.00 45.00 50.00 CC(rho_ref,rho_opt): 0.97 0.97 0.97 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 28.30 28.30 26.67 21.63 16.92 12.75 9.18 5.68 2.81 0.00 CC(q,B): 0.99 B: 80 trial q : 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 B_opt : 1.00 9.00 22.00 33.00 43.00 51.00 59.00 66.00 73.00 80.00 CC(rho_ref,rho_opt): 0.93 0.94 0.96 0.98 0.99 0.99 1.00 1.00 1.00 1.00 R(%) : 37.91 34.26 27.88 22.25 17.20 13.22 9.35 6.10 2.99 0.00 CC(q,B): 0.99

What we see here is: - correlation of q and B is indeed approaches 100%; - map correlation is greater than 90% in most cases except a few corner cases; - the last column in all tests is an obvious sanity check (CC=1, R=0 if exact B and q are used); - R-factors are greater than zero except a trivial case. This is the key that makes it possible to deconvolute q and B.

All the best, Pavel

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb