Bing! Bet Nat's hit this in one. There may be a resolution dependence for 'F' but F001 (say) should not change, and should not go from 0 to non-0 no matter what. Direct summation much better here... Presumably for a protein structure the effects of gridding will also be much less significant. Thanks, Graeme ________________________________________ From: [email protected] [[email protected]] on behalf of Nathaniel Echols [[email protected]] Sent: 07 December 2012 15:46 To: cctbx mailing list Subject: Re: [cctbxbb] Structure factor calculation weirdness On Fri, Dec 7, 2012 at 7:36 AM, wrote:

Some confusion here at diamond – it seems that the amplitude of F(hkl) depends on dmin

Yes, because it's using the FFT method, which means it first does map sampling on a grid whose dimensions will partly depend on the resolution. I'm not sure if there is resolution dependence for the scattering factors (I thought so, but I don't really know details), but I tried using direct summation and the resolution effect disappears. f_calc = structure.structure_factors( algorithm="direct", d_min = dmin).f_calc() For your toy example the speed difference isn't noticeable - for a protein structure it will be considerable. -Nat _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Hi Graeme, When you pass algorithm=None to xray.structure.structure_factors() it is supposed determine dynamically whether direct summation or FFT is best suited. However in this case (I guess because of the tiny unit cell?), the algorithm for choosing which method clearly fails as in this case direct summation would undoubtedly be better than FFT: (from cctbx_project/cctbx/xray/structure_factors/from_scatterers.py) # rough estimate if ( 4*n_scatterers*self.space_group().order_p()*n_miller_indices < self.crystal_gridding().n_grid_points()): algorithm = "direct" Maybe there should be some special handling for small unit cells here? I don't know if there is a reason why you would ever want to calculate structure factors via FFT for unit cells this tiny? Cheers, Richard On 7 December 2012 09:21, Nicholas Sauter wrote:

Graeme,

Just as a general ballpark figure, for protein work the difference between approximate FFT structure factors and the exact direct sum figures is on the order of 1%.

Nick

...

Bing! Bet Nat's hit this in one.

There may be a resolution dependence for 'F' but F001 (say) should not change, and should not go from 0 to non-0 no matter what. Direct summation much better here...

Presumably for a protein structure the effects of gridding will also be much less significant.

Thanks,

Graeme ________________________________________ From: [email protected] [ [email protected]] on behalf of Nathaniel Echols [ [email protected]] Sent: 07 December 2012 15:46 To: cctbx mailing list Subject: Re: [cctbxbb] Structure factor calculation weirdness

On Fri, Dec 7, 2012 at 7:36 AM, wrote:

...

Some confusion here at diamond – it seems that the amplitude of F(hkl) depends on dmin

Yes, because it's using the FFT method, which means it first does map sampling on a grid whose dimensions will partly depend on the resolution. I'm not sure if there is resolution dependence for the scattering factors (I thought so, but I don't really know details), but I tried using direct summation and the resolution effect disappears.

f_calc = structure.structure_factors( algorithm="direct", d_min = dmin).f_calc()

For your toy example the speed difference isn't noticeable - for a protein structure it will be considerable.

-Nat _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to

...

Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be

On Fri, Dec 7, 2012 at 8:29 AM, wrote: the e-mail. transmitted in or with the message.

...

Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Physical Biosciences Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 64R0121 Berkeley, CA 94720-8118 (510) 486-5713 _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Well, yes. The relationship between direct and FFT methods is very straightforward. Since the direct summation over the grid points gives the exact value of the sampled function AT the grid points, the relationship is clearly (in the one dimensional case) F\sub{FFT}(h) = Sum(-infinity, +infinity, F\sub{exact}(h+j*N) where j is the summation index and N is the number of equally spaced sample points. When I published this method I gave an error analysis that gives a more detailed discussion. Acta Crystallographica Section A, Vol. 33, p. 486-492 (1977). A very quick calculation involving grid size and an assumption of roughly Gaussian fall-off will quickly tell you whether or not the expected error is negligible. I recommend this check for any attempt to automatically decide between quick and accurate structure factor calculations. Lynn Ten Eyck On Dec 7, 2012, at 11:06 AM, Richard Gildea wrote:

I meant to include in the previous email that I add the following lines to the loop over dmin in Graeme's script:

gridding = structure.crystal_symmetry().gridding(d_min=dmin) print "gridding: ", gridding.n_real()

giving the output:

d_min: 5.0 gridding: (3, 3, 3) d_min: 2.5 gridding: (8, 8, 8) d_min: 1.7 gridding: (9, 9, 9)

With so few grid points it is entirely unsurprising that the FFT is such a poor approximation of the direct summation in this case!

Cheers,

Richard

On 7 December 2012 10:49, Richard Gildea wrote: Hi Graeme,

When you pass algorithm=None to xray.structure.structure_factors() it is supposed determine dynamically whether direct summation or FFT is best suited. However in this case (I guess because of the tiny unit cell?), the algorithm for choosing which method clearly fails as in this case direct summation would undoubtedly be better than FFT:

(from cctbx_project/cctbx/xray/structure_factors/from_scatterers.py)

# rough estimate if ( 4*n_scatterers*self.space_group().order_p()*n_miller_indices < self.crystal_gridding().n_grid_points()): algorithm = "direct"

Maybe there should be some special handling for small unit cells here? I don't know if there is a reason why you would ever want to calculate structure factors via FFT for unit cells this tiny?

Cheers,

Richard

On 7 December 2012 09:21, Nicholas Sauter wrote: Graeme,

Just as a general ballpark figure, for protein work the difference between approximate FFT structure factors and the exact direct sum figures is on the order of 1%.

Nick

On Fri, Dec 7, 2012 at 8:29 AM, wrote:

...

Bing! Bet Nat's hit this in one.

There may be a resolution dependence for 'F' but F001 (say) should not change, and should not go from 0 to non-0 no matter what. Direct summation much better here...

Presumably for a protein structure the effects of gridding will also be much less significant.

Thanks,

Graeme ________________________________________ From: [email protected] [[email protected]] on behalf of Nathaniel Echols [[email protected]] Sent: 07 December 2012 15:46 To: cctbx mailing list Subject: Re: [cctbxbb] Structure factor calculation weirdness

On Fri, Dec 7, 2012 at 7:36 AM, wrote:

...

Some confusion here at diamond – it seems that the amplitude of F(hkl) depends on dmin

Yes, because it's using the FFT method, which means it first does map sampling on a grid whose dimensions will partly depend on the resolution. I'm not sure if there is resolution dependence for the scattering factors (I thought so, but I don't really know details), but I tried using direct summation and the resolution effect disappears.

f_calc = structure.structure_factors( algorithm="direct", d_min = dmin).f_calc()

For your toy example the speed difference isn't noticeable - for a protein structure it will be considerable.

-Nat _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Physical Biosciences Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 64R0121 Berkeley, CA 94720-8118 (510) 486-5713 _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb