- cctbxbb - phenix-online.org

Strange behaviour of resolution filter
by Jan Marten Simons 08 Feb '13

08 Feb '13

Hi, I discovered a rather strange behaviour of the miller.index_generator. Sample code showing the weirdness: -----8<--------------------------- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=crystal.symmetry( unit_cell=(3.14098, 4.08327, 5.33966, 83, 109, 129), space_group_symbol="P 1")), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.,0.,0.), u=0.0)])) fc1 = structure.structure_factors(d_min=1.0).f_calc().sort() d_max, d_min = fc1.resolution_range() fc2 = structure.structure_factors(d_min=d_min).f_calc().sort() print(fc1.size(), fc2.size()) #fc1.show_array() fc1.show_comprehensive_summary() print("\nd_min: {}\n".format(d_min)) fc2.show_comprehensive_summary() #fc2.show_array() -----8<--------------------------- fc1 has 105 indices, while fc2 only has 104. Also fc2 has a higher d_min than fc1 although I'd expect both to have the same. To me it seems like the index with d==d_min is not included and only indices with d>d_min are calculated. I wonder if this behaviour is the desired one or rather a bug, as I was quite surprised when I discovered this. With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

2 1

Using cctbx::uctbx::unit_cell in c++ code / boosted to Python
by Graeme.Winter＠diamond.ac.uk 25 Jan '13

25 Jan '13

Hi Folks, Been working on some c++ coding using boost etc (which I have to say is great) however I found one bit of weirdness. If I want to generate a cctbx::uctbx::unit_cell in c++ code then access this from Python I get Traceback (most recent call last): File "tst_x2tbx.py", line 67, in <module> tst_x2tbx(sys.argv[1]) File "tst_x2tbx.py", line 6, in tst_x2tbx import x2tbx File "/Users/graeme/svn/cctbx/sources/x2tbx/__init__.py", line 7, in <module> ext = boost.python.import_ext("x2tbx_ext", optional = False) File "/Users/graeme/svn/cctbx/sources/cctbx_project/boost_adaptbx/boost/python.py", line 36, in import_ext + [" "+p for p in sys.path])) ImportError: __import__("x2tbx_ext"): dlopen(/Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so, 2): Symbol not found: cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny<double, 6ul> const&) Referenced from: /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so Expected in: flat namespace in /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so sys.path: …. unless I have import cctbx.uctbx # explicit import at the top of the Python code - have I done something wrong with the SConscript? I found something similar with import scitbx.array_family.flex # explicit import which libtbx.clean_clutter complains about as I don't reference it (hence the additional comment) Equally I could just leave the additional clutter there but it feels like I should not need it... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 8

Sphinx documentation
by Graeme.Winter＠diamond.ac.uk 21 Jan '13

21 Jan '13

Hi Folks, I installed sphinx and tried building the documentation but I got an error: File ./xfel/xfel.cxi.gfx.rst already exists, skipping. File ./xfel/xfel.cxi.merging.rst already exists, skipping. File ./xfel/xfel.cxi.rst already exists, skipping. File ./xfel/xfel.metrology.rst already exists, skipping. File ./xfel/xfel.rst already exists, skipping. File ./xfel/modules.rst already exists, skipping. sphinx-build -b coverage -d _build/doctrees -D latex_paper_size=a4 . _build/coverage Making output directory... Running Sphinx v1.2pre loading pickled environment... not yet created building [coverage]: coverage overview updating environment: 206 added, 0 changed, 0 removed reading sources... [ 0%] cctbx/cctbx Exception occurred: File "/Users/graeme/svn/cctbx/phenix/phenix-dev-1241/build/mac-intel-osx-x86_64/base/Python.framework/Versions/Current/lib/python2.7/site-packages/Sphinx-1.2predev_20130121-py2.7.egg/sphinx/ext/autodoc.py", line 595, in filter_members membername in self.options.special_members: TypeError: argument of type 'object' is not iterable The full traceback has been saved in /var/folders/j1/cgsqy8fs1lxdyjggcsb3twsw0000gn/T/sphinx-err-jjM5C8.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-users/>, or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks! make: *** [coverage] Error 1 cp: ./_build/coverage/python.txt: No such file or directory cp: ./_build/coverage/c.txt: No such file or directory sphinx-build -b html -d _build/doctrees -D latex_paper_size=a4 . _build/html Making output directory... Running Sphinx v1.2pre loading pickled environment... not yet created building [html]: targets for 206 source files that are out of date updating environment: 206 added, 0 changed, 0 removed reading sources... [ 0%] cctbx/cctbx Exception occurred: File "/Users/graeme/svn/cctbx/phenix/phenix-dev-1241/build/mac-intel-osx-x86_64/base/Python.framework/Versions/Current/lib/python2.7/site-packages/Sphinx-1.2predev_20130121-py2.7.egg/sphinx/ext/autodoc.py", line 595, in filter_members membername in self.options.special_members: TypeError: argument of type 'object' is not iterable The full traceback has been saved in /var/folders/j1/cgsqy8fs1lxdyjggcsb3twsw0000gn/T/sphinx-err-lONSky.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-users/>, or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks! make: *** [html] Error 1 Did I do something wrong? I just grabbed sphinx from the internet. Should I have got it from someplace else? OS X Lion / svn cctbx. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Phil (dumb) question
by Graeme.Winter＠diamond.ac.uk 14 Jan '13

14 Jan '13

Hi Folks, I have a toy example where I should be overriding Phil defaults: phil_defaults = ''' resolutionizer { rmerge = 0.0 .type = float completeness = 0.0 .type = float isigma = 1.0 .type = float misigma = 2.0 .type = float bins = 100 .type = int } ''' more = ''' resolutionizer { isigma = 2.0 misigma = 4.0 } ''' from libtbx.phil import parse working_phil = parse(phil_defaults) working_phil.fetch(parse(more)) working_phil.show() but it seems I am not: Graemes-MacBook-Pro:scale graeme$ cctbx.python toy.py resolutionizer { rmerge = 0.0 completeness = 0.0 isigma = 1.0 misigma = 2.0 bins = 100 } Am I doing something dumb? I hope so as I really cannot find my error. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

Simple cctbx / boost c++ extension example
by Graeme.Winter＠diamond.ac.uk 11 Jan '13

11 Jan '13

Hi Folks, Just say I wanted to write a new from the ground up module for cctbx which wanted to include some C++ code, and a SConscript etc. so I can run libtbx.configure toytbx (say) And then find from toytbx import cplusplus_functions would work.... Is there an example I could follow? (I have not looked through the newsletters, I suppose I should have... but I could not seen anything obvious in the documents) Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 6

Computing E^4 from an mtz file
by Graeme.Winter＠diamond.ac.uk 14 Dec '12

14 Dec '12

Hi Folks, Trying to write some code to return the average E^4 across (some) bins for an mtz file, I got to def main(): import sys from iotbx import reflection_file_reader for argv in sys.argv[1:]: reflection_file = reflection_file_reader.any_reflection_file( file_name = argv) if reflection_file.file_type() is None: continue miller_arrays = reflection_file.as_miller_arrays() for ma in miller_arrays: E4(ma) def E4(ma): f = ma.as_intensity_array() f = f.array(data=f.data()/f.epsilons().data().as_double()) f.set_observation_type_xray_intensity() f.setup_binner(auto_binning = True) sm = f.second_moment(use_binning = True) sm.show() if __name__ == '__main__': main() but I have no idea how to actually extract the information I want from sm as this only has show() which gives Graemes-MacBook-Pro:PlayPen graeme$ cctbx.python E4.py /Users/graeme/projects/sci-397/DEFAULT/scale/AUTOMATIC_DEFAULT_scaled.mtz unused: - 20.8668 [ 0/7 ] bin 1: 20.8668 - 5.8873 [221/235] 2.3863 bin 2: 5.8873 - 4.6904 [216/221] 1.9823 bin 3: 4.6904 - 4.1026 [221/222] 1.9925 <snip> bin 47: 1.6553 - 1.6435 [153/209] 2.2051 bin 48: 1.6435 - 1.6320 [143/198] 2.2021 bin 49: 1.6320 - 1.6208 [141/204] 2.7085 unused: 1.6208 - [ 0/0 ] unused: - 20.8668 [ 0/9 ] bin 1: 20.8668 - 7.1654 [201/225] 2.2405 bin 2: 7.1654 - 5.7261 [202/213] 1.9109 bin 3: 5.7261 - 5.0137 [216/223] 2.0162 bin 4: 5.0137 - 4.5605 [207/212] 2.0043 <snip> bin 87: 1.6456 - 1.6393 [126/192] 2.5521 bin 88: 1.6393 - 1.6330 [122/210] 2.0195 bin 89: 1.6330 - 1.6269 [131/224] 2.6251 bin 90: 1.6269 - 1.6208 [120/212] 3.0912 unused: 1.6208 - [ 0/0 ] Really I would like the value programatically or at least a way of getting back an array of the binned values so I can average them myself (which is obviously trivial) Any suggestions? Thanks, Graeme ps - most of this code is ripped from iotbx.reflection_statistics :o) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 4

Writing a pdb file from an xray_structure object
by Claudia Millán Nebot 12 Dec '12

12 Dec '12

Hi everyone, i have been using the xray_structure_simple() method to get an xray.structure object from a pdb object and use it to perform some calculations on it, one of them being expand it to P1. After, i tried to write a new pdb out of this xray_structure in P1. For that purpose, i used the following method: cctbx.xray.structure.as_pdb_file(self, remark=None, remarks=[], fractional_coordinates=False, resname=None, connect=None) but using it both with the original xray structure and with the one on P1 I get very bad lookings pdbs, with incorrect formating, like that for the case of the one in P1: ATOM 1 PDB= PDB 1 12.330 -6.948 -40.820 1.00 19.26 N ATOM 2 PDB= PDB 2 49.988 55.370 3.948 1.00 19.26 N ATOM 3 PDB= PDB 3 36.092 30.710 93.483 1.00 19.26 N ATOM 4 PDB= PDB 4 55.370 49.988 175.123 1.00 19.26 N ATOM 5 PDB= PDB 5 30.710 36.092 85.587 1.00 19.26 N so i guessed that something must be wrong with the residues.... and went to check the code. That as_pdb_file method calls to iotbx.pdb.xray_structure.as_pdb_file(), and on that code i realised that as i was calling it whithout any resname, it was retrieving it from some information on the scatterers that could be wrong. Putting the value of resname to ALA, as a trial, produced almost the same output except for the name: ATOM 1 PDB= ALA 1 12.330 -6.948 -40.820 1.00 19.26 N ATOM 2 PDB= ALA 2 49.988 55.370 3.948 1.00 19.26 N ATOM 3 PDB= ALA 3 36.092 30.710 93.483 1.00 19.26 N ATOM 4 PDB= ALA 4 55.370 49.988 175.123 1.00 19.26 N ATOM 5 PDB= ALA 5 30.710 36.092 85.587 1.00 19.26 N Up to this point, i don't get what is wrong. It may be that i'm using a too simple method and without the correct parameters (xray_structure_simple()), as i guess part of the error is that the assignation of the types of atoms and residues is going crazy. There are other ways of performing this step from a pdb to an xray_structure and viceversa? Thanks in advance! Best regards, Claudia Millán (cmncri(a)ibmb.csic.es) Institut de Biologia Molecular de Barcelona (IBMB-CSIC) Barcelona, Spain

2 2

Documentation .... again
by Graeme.Winter＠diamond.ac.uk 12 Dec '12

12 Dec '12

Hi Folks, Been thinking some more about the old chestnut of documentation. The lack thereof is a well understood issue, and to be honest one which is not going to change any time soon as no one has the time to maintain a manual - this is fine and well understood. As a kind of "patch" for this I was wondering if it would be possible to have a forum where someone could ask for e.g. documentation for a currently undocumented function, which could then be added to the subversion in the code itself. For example: cctbx_project/cctbx/xray/structure.py ... def structure_factors(self, anomalous_flag=None, d_min=None, algorithm=None, cos_sin_table=False, quality_factor=None, u_base=None, b_base=None, wing_cutoff=None): what is cos_sin_table for? with things like this there is the attitude that it is safe to ignore, however if we're writing code to guide people's experiments it would be good to be sure of this. Some simple documentation explaining this would be very useful. I'm definitely not suggesting (i) that anyone writes a manual or (ii) that all undocumented functions are suddenly documented, only that there is a mechanism set up that someone could ask about this and that someone who knows the answer could commit the changes to cctbx and then reply with the diffs - perhaps the way that stackoverflow works, or an email, or something really lightweight. Even this mailing list could be used I guess? I would be happy to help with this, though the above example is not one where I know the answer! Thoughts? Best wishes, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

8 24

Structure factor calculation weirdness
by Graeme.Winter＠diamond.ac.uk 10 Dec '12

10 Dec '12

Hi Folks, Some confusion here at diamond - it seems that the amplitude of F(hkl) depends on dmin ---------------- from cctbx import xray from cctbx import crystal from cctbx.array_family import flex # 14keV wavelength = 12.3984 / 14.0 spgroup = 'Fd-3m' latt = (5.4309, 5.4309, 5.4309, 90, 90, 90) atoms = [('Si', 0.125, 0.125, 0.125)] allscatterers=[] for atom in atoms: allscatterers.append(xray.scatterer( label = atom[0], site = (atom[1], atom[2], atom[3]), u = 0)) structure = xray.structure( special_position_settings = crystal.special_position_settings( crystal_symmetry = crystal.symmetry( unit_cell = latt, space_group_symbol = spgroup)), scatterers = flex.xray_scatterer(allscatterers)).expand_to_p1() for dmin in 5.0, 2.5, 1.7: print dmin f_calculations = structure.structure_factors( d_min = dmin).f_calc().amplitudes().show_array() --------------- Chopped out only 001, 010, 100 from this... 5.0 (0, 0, 1) 0.290277589693 (0, 1, 0) 0.290277589693 (1, 0, 0) 0.290277589693 2.5 (0, 0, 1) 0.0 (0, 1, 0) 0.0 (1, 0, 0) 0.0 <snip> 1.7 (0, 0, 1) 0.278378587828 (0, 1, 0) 0.278378587828 (1, 0, 0) 0.278378587828 <snip> Have we misunderstood something? Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

7 9

UB matrix in cbf files
by Carlo.Vecchini＠diamond.ac.uk 05 Dec '12

05 Dec '12

Hello, I would like to add the orientation matrix information in the cbf file. Following the description in sections 3.2.2.2.3 of IUCr (G), I added the following lines in the cbf header: _diffrn_orient_matrix_type ub_matrix_test _diffrn_orient_matrix_UB_11 0.083424 _diffrn_orient_matrix_UB_12 -0.076007 _diffrn_orient_matrix_UB_13 -0.078787 _diffrn_orient_matrix_UB_21 -0.103374 _diffrn_orient_matrix_UB_22 -0.075939 _diffrn_orient_matrix_UB_23 -0.01146 _diffrn_orient_matrix_UB_31 -0.034434 _diffrn_orient_matrix_UB_32 0.043832 _diffrn_orient_matrix_UB_33 -0.156476 When I try to read back the values with: get_orientation_matrix() I get the following error: UBm = cbf_handle.get_orientation_matrix() File "~/CCTBX/cctbx_plus_build/lib/pycbf.py", line 3915, in get_orientation_matrix return _pycbf.cbf_handle_struct_get_orientation_matrix(self) Exception: CBFlib Error(s): CBF_NOTFOUND If I use set_orientation_matrix() to write the cbf file, I get a similar error. Does anyone have any suggestion on how to solve this? Thanks in advance, Carlo Vecchini -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1