- cctbxbb - phenix-online.org

Re: [cctbxbb] building sphinx documentation
by Jan Marten Simons 20 Apr '13

20 Apr '13

Am Donnerstag 18 April 2013 19:16:00 schrieben Sie: > Hi Jan, Hi Richard, > I have attempted to build the sphinx documentation using > dox.sphinx/build_docs.sh, and this appears to have run successfully and > produced some html which shows the table of contents etc. However, I can't > seem to find the documentation for any of the significant modules, e.g. > cctbx/xray/structure.py. When I navigate to the link for the structure > module within the xray package > (dox.sphinx/_build/html/cctbx/cctbx.xray.html#structure-module), I see > nothing other than a list of all the modules within the xray package. Am I > missing something here? Have you followed the instructions given in the comments at the top of dox.sphinx/build_docs.sh? Which version of Sphinx do you use? I'm using 1.1.3 regularly and get useful documentation (I've attached the "makescript" I use to build cctbx and the sphinx documentation. If you are interested in it I can also put the output of the sphinx run on my system with current svn on webserver over here.). Has your sphinx build completed successfully or did it abort (e.g. due to out of memory* or due to other problems)? Could you also check if build/dox.sphinx/cctbx/cctbx.xray.rst has all modules listed? > As a novice to using sphinx, I don't like to add significant documentation > in sphinx style without being able to check via generating the > documentation that what I have written is correct sphinx-speak. I can totally understand this ;). Cheers and hopefully we can get this working for you as well, Jan *) For me running sphinx with all those modules selected needs about 3.3 GiB of virtual memory (RAM) and takes a few minutes to actually finish. p.s.: Sorry didn't hit "reply all".

3 6

Finding neighbouring atoms using pair_asu_table
by Lukas Reck 20 Apr '13

20 Apr '13

Dear all, please forgive me if the following is trivial or if I've overlooked something simple, I'm new to cctbx. I'm trying to use the cctbx python libraries to read in a large number of crystal structures and extract some geometric information on the coordination environment of certain atom types in them. I've been following the instructions from the 8/2004 CompComm Newsletter No. 4 at <http://www.iucr.org/__data/assets/pdf_file/0003/6384/iucrcompcomm_aug2004.p…>, pages 22-28 (is that still up to date?) but I cannot seem to get at the correct symmetry elements for each neighbouring atom in the coordination environment of some atom of interest. As far as I understand, the following should be correct: assert(isinstance(structure, cctbx.xray.structure)) # yes, it is a structure asu_mappings = structure.asu_mappings(buffer_thickness = 3) # create the ASU mappings pair_asu_table = cctbx.crystal.pair_asu_table(asu_mappings = asu_mappings) pair_asu_table.add_all_pairs(distance_cutoff = 3) # create the pair ASU table raw_mappings = asu_mappings.mappings() raw_pair_table = pair_asu_table.table() # access to the raw structures for scatterer_index, scatterer in enumerate(structure.scatterers()): # iterate over all atoms in the structure label = scatterer.element_symbol() if label == atom_of_interest: # we are only interested in one atom type for j_seq, j_syms in raw_pair_table[scatterer_index].items(): # j_seq is the neighbouring scatterer's index, j_sym is a list of lists of symmetry indices for sym_group in j_syms: for j_sym in sym_group: # j_sym is an index into raw_mappings[j_seq] print "j_seq:", j_seq, "j_sym:", j_sym print "i_sym_op:", raw_mappings[j_seq][j_sym].i_sym_op() symmetry = asu_mappings.get_rt_mx(j_seq, j_sym) print "%s at %s" % (scatterer.element_symbol(), scatterer.site) print "Neighbour: %s at %s" % (structure.scatterers()[j_seq].element_symbol(), structure.scatterers()[j_seq].site) print "To be transformed by %s (symmetry #%s)" % (symmetry, j_sym) print "into %s" % (symmetry(structure.scatterers()[j_seq].site),) Here's a sample output for one pair of neighbouring atoms: j_seq: 10 j_sym: 1 i_sym_op: 10 V at (0.30204, 0.52919, 0.5) Neighbour: O at (0.36242, 0.49166, 0.5) To be transformed by y-1/2,-x+1/2,z-1/2 (symmetry #1) into (-0.00834, 0.13758, 0.0) As you can see from the coordinates, the neighbouring atom was correctly identified, but the corresponding symmetry as returned by get_rt_mx transforms it into a symmetry equivalent that is very far away. Clearly I misunderstand the function of one or more of the various indices here, but I am not quite clear on where my error lies... Thanks for any pointers! Best regards Lukas Reck -- School of Chemistry Faculty of Engineering, Mathematics and Science Trinity College Dublin Dublin 2 Tel: (+353) 1 896 3452

2 3

building sphinx documentation
by Richard Gildea 19 Apr '13

19 Apr '13

Hi Jan, I have attempted to build the sphinx documentation using dox.sphinx/build_docs.sh, and this appears to have run successfully and produced some html which shows the table of contents etc. However, I can't seem to find the documentation for any of the significant modules, e.g. cctbx/xray/structure.py. When I navigate to the link for the structure module within the xray package (dox.sphinx/_build/html/cctbx/cctbx.xray.html#structure-module), I see nothing other than a list of all the modules within the xray package. Am I missing something here? As a novice to using sphinx, I don't like to add significant documentation in sphinx style without being able to check via generating the documentation that what I have written is correct sphinx-speak. Cheers, Richard

3 2

First svn commit
by James Stroud 18 Apr '13

18 Apr '13

Hello All, Since this is my first commit, I just want to make sure that I have done these things in the appropriate way. Here's what I did: 1. Added a wiki page: https://sourceforge.net/p/cctbx/wiki/Subversion%20set%20up/ 2. Added the method as_miller_arrays_map() to iotbx_mtz_ext.object, which is declared in iotbx/mtz/__init__.py. 3. Added a test for this functionality in the file iotbx/mtz/tst_miller_map.py Please let me know if the manner in which I did anything could use improvement. James

6 6

Euro SciPy
by Graeme.Winter＠diamond.ac.uk 17 Apr '13

17 Apr '13

Hi Folks, Anyone going to this meeting? https://www.euroscipy.org/ Some of the scientific software team here at Diamond have a habit of going to this meeting and remarked that a contribution on CCTBX may be appropriate... (see below for meeting topics) Best wishes, Graeme Topics * Presentations of scientific tools and libraries using the Python language, including but not limited to: * Vector and array manipulation * Parallel computing * Scientific visualization * Scientific data flow and persistence * Algorithms implemented or exposed in Python * Web applications and portals for science and engineering- * Reports on the use of Python in scientific achievements or ongoing projects. * General-purpose Python tools that can be of special interest to the scientific community. -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

structure.replace_sites_... vs. structure.set_sites_...
by Jan Marten Simons 17 Apr '13

17 Apr '13

Hi, while working on a really fast way to calculate the structure factors of a few millions of different modifications of a structure with only the selected scatterers contributing to f_calc I came across this: Why are there two methods for changing the sites_(cart/frac) of the scatterers doing quite different things? set_sites_... seems to simply update the corresponding site information of the scatterer, while replace_sites_... seems to create a copy of the structure. Is this the only difference there is? Also If anybody got a nice idea for my original problem, I'm also very much interested. (I think building a larger structure and working with selections might be faster than creating lots of different structures containing only the desired scatterers, but I might be wrong on this.) Cheers, Jan

3 4

pyyaml - iotbx.pdb incompatibility
by James Stroud 16 Apr '13

16 Apr '13

Hello All, There is a strange incompatibility between pyyaml and iotbx.pdb that depends on the order of import. If yaml is imported first, there is no problem. If iotbx.pdb is imported first, then importing yaml crashes. This is the yaml that comes with enthought python, but I don't think it's using libyaml, which is the C implementation. So the yaml, from what I can tell, is pure python. I'm wondering if anyone else can reproduce this error. James chernev 185% cctbx.python Enthought Python Distribution -- www.enthought.com Version: 7.3-2 (64-bit) Python 2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 11:14:05) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "credits", "demo" or "enthought" for more information. py> import yaml py> from iotbx import pdb py> raise SystemExit() chernev 186% cctbx.python Enthought Python Distribution -- www.enthought.com Version: 7.3-2 (64-bit) Python 2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 11:14:05) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "credits", "demo" or "enthought" for more information. py> from iotbx import pdb py> import yaml show_stack(1): /Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/yaml/constructor.py(256) SafeConstructor show_stack(2): /Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/yaml/constructor.py(161) <module> show_stack(3): /Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/yaml/loader.py(8) <module> show_stack(4): /Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/yaml/__init__.py(8) <module> show_stack(5): <stdin>(1) <module> libc backtrace (49 frames, most recent call last): 50 python2.7 0x0000000100000f54 start + 52 49 Python 0x0000000100108576 Py_Main + 3318 48 Python 0x00000001000f2c6e PyRun_AnyFileExFlags + 126 47 Python 0x00000001000f2b2e PyRun_InteractiveLoopFlags + 78 46 Python 0x00000001000f28eb PyRun_InteractiveOneFlags + 379 45 Python 0x00000001000cd2c6 PyEval_EvalCode + 54 44 Python 0x00000001000ccfc5 PyEval_EvalCodeEx + 1733 43 Python 0x00000001000c77fe PyEval_EvalFrameEx + 9502 42 Python 0x00000001000c4267 PyEval_CallObjectWithKeywords + 87 41 Python 0x000000010000d052 PyObject_Call + 98 40 Python 0x00000001000be823 builtin___import__ + 131 39 Python 0x00000001000e5cf8 PyImport_ImportModuleLevel + 344 38 Python 0x00000001000e536a load_next + 234 37 Python 0x00000001000e511a import_submodule + 314 36 Python 0x00000001000e4f59 load_package + 409 35 Python 0x00000001000e4632 load_source_module + 466 34 Python 0x00000001000e3981 PyImport_ExecCodeModuleEx + 209 33 Python 0x00000001000cd2c6 PyEval_EvalCode + 54 32 Python 0x00000001000ccfc5 PyEval_EvalCodeEx + 1733 31 Python 0x00000001000c77fe PyEval_EvalFrameEx + 9502 30 Python 0x00000001000c4267 PyEval_CallObjectWithKeywords + 87 29 Python 0x000000010000d052 PyObject_Call + 98 28 Python 0x00000001000be823 builtin___import__ + 131 27 Python 0x00000001000e5cf8 PyImport_ImportModuleLevel + 344 26 Python 0x00000001000e536a load_next + 234 25 Python 0x00000001000e511a import_submodule + 314 24 Python 0x00000001000e4632 load_source_module + 466 23 Python 0x00000001000e3981 PyImport_ExecCodeModuleEx + 209 22 Python 0x00000001000cd2c6 PyEval_EvalCode + 54 21 Python 0x00000001000ccfc5 PyEval_EvalCodeEx + 1733 20 Python 0x00000001000c77fe PyEval_EvalFrameEx + 9502 19 Python 0x00000001000c4267 PyEval_CallObjectWithKeywords + 87 18 Python 0x000000010000d052 PyObject_Call + 98 17 Python 0x00000001000be823 builtin___import__ + 131 16 Python 0x00000001000e5cf8 PyImport_ImportModuleLevel + 344 15 Python 0x00000001000e536a load_next + 234 14 Python 0x00000001000e511a import_submodule + 314 13 Python 0x00000001000e4632 load_source_module + 466 12 Python 0x00000001000e3981 PyImport_ExecCodeModuleEx + 209 11 Python 0x00000001000cd2c6 PyEval_EvalCode + 54 10 Python 0x00000001000ccfc5 PyEval_EvalCodeEx + 1733 9 Python 0x00000001000cb0e6 PyEval_EvalFrameEx + 24070 8 Python 0x00000001000ccfc5 PyEval_EvalCodeEx + 1733 7 Python 0x00000001000c7ea0 PyEval_EvalFrameEx + 11200 6 Python 0x0000000100010af8 PyNumber_Multiply + 40 5 Python 0x000000010000c569 binary_op1 + 137 4 ??? 0x0000000000000aa9 0x0 + 2729 3 libsystem_c.dylib 0x00007fff84b0ccfa _sigtramp + 26 2 boost_python_meta_ext.so 0x00000001007013c0 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. -- James Stroud Affiliate Howard Hughes Medical Institute UCLA-DOE Institute for Genomics and Proteomics

3 8

Flex array slice assignment work-alike
by James Stroud 16 Apr '13

16 Apr '13

Hello All, It seems that flex arrays do not support slice assignment: py> from scitbx.array_family import flex py> ary = flex.float(range(5)) py> list(ary) [0.0, 1.0, 2.0, 3.0, 4.0] py> ary[:3] = flex.float([41, 42, 43]) --------------------------------------------------------------------------- ArgumentError Traceback (most recent call last) /usr/local/cctbx/cctbx_sources/mmtbx/dynamics/<ipython-input-13-5ce3d648bc6f> in <module>() ----> 1 ary[:3] = flex.float([41, 42, 43]) ArgumentError: Python argument types in float.__setitem__(float, slice, float) did not match C++ signature: __setitem__(scitbx::af::versa<float, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > {lvalue}, scitbx::af::small<long, 10ul>, float) __setitem__(scitbx::af::versa<float, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > {lvalue}, long, float) Slice assignment is common in python: py> alist = range(5) py> alist[:3] = [41, 42, 43] py> alist [41, 42, 43, 3, 4] Is there a flex array work-alike for this operation? I've found set_selected(): py> sel_ary = flex.bool([True, True, True, False, False]) py> ary.set_selected(sel_ary, flex.float([41, 42, 43])) <scitbx_array_family_flex_ext.float at 0x101df1158> py> list(ary) [41.0, 42.0, 43.0, 3.0, 4.0] But set_selected() is cumbersome for slice assignment because (1) you have to build a flex selection array somehow and (2) you have to put the new values into a flex array of the appropriate type. Is set_selected() the best way? If so, is there something in the cctbx libraries that makes building "slicing" selection arrays more efficient than say a list comprehension? sel_ary = flex.bool([True if i < 3 else False for i in xrange(5)]) James

7 16

cif parser & ccp4 mon_lib incompatibility
by James Stroud 15 Apr '13

15 Apr '13

Hi All, It's a fresh install of the latest-greatest cctbx and of ccp4. Any ideas? % cctbx.python ./tst_sa.py CifParserError: /Applications/ccp4-6.3.0/lib/data/monomers/list/mon_lib_list.cif(line 1) : error 4 : Unexpected token, at offset -1 near global : unexpected input... expected one of : Actually dude, we didn't seem to be expecting anything here, or at least I could not work out what I was expecting, like so many of us these days! Here's the top of the cif file: global _lib_name mon_lib _lib_version 5.36 _lib_update 03/05/12 --------------------------------- # # --- LIST OF MONOMERS --- # data_comp_list loop_ _chem_comp.id _chem_comp.three_letter_code _chem_comp.name _chem_comp.group James

7 24

Re: [cctbxbb] [ccp4bb] CCP4 Update victim of own success
by James Stroud 12 Apr '13

12 Apr '13

On Apr 11, 2013, at 6:17 AM, <eugene.krissinel(a)stfc.ac.uk> <eugene.krissinel(a)stfc.ac.uk> wrote: > Sorry that this was unclear. We assume that updater is used primarily from ccp4i, where nothing changed (and why it should be used from command line at all ?:)). Scriptability. On Apr 11, 2013, at 10:34 AM, Jim Pflugrath wrote: > I think James gets to 'fight' like in the old game of rogue by pressing the h, j, k, l keys on his keyboard (not a detachable one either). While Eugene gets to use a modern game controller or a Wii. > > Ooops, game is already over and James [Holton] has lost. But he wrote the game. James

1 1

bin_centers() and d_star_sq
by James Stroud 10 Apr '13

10 Apr '13

Hello All, I'm genuinely perplexed about what factors should go into choosing an appropriate d_star_sq to calculate bin_centers() in cctbx.miller.binner. This decision may be a nuance because it doesn't seem change the output significantly (except for low resolution bins). But if it doesn't matter significantly, then why have the parameter in the first place? James py> binner = miller.setup_binner(n_bins=10) py> binner.show_summary() unused: - 81.7405 [ 0/0 ] bin 1: 81.7405 - 6.1396 [2341/2341] bin 2: 6.1396 - 4.8734 [2213/2213] bin 3: 4.8734 - 4.2574 [2185/2185] bin 4: 4.2574 - 3.8681 [2168/2168] bin 5: 3.8681 - 3.5909 [2177/2177] bin 6: 3.5909 - 3.3792 [2156/2156] bin 7: 3.3792 - 3.2099 [2137/2137] bin 8: 3.2099 - 3.0702 [2144/2144] bin 9: 3.0702 - 2.9520 [2133/2133] bin 10: 2.9520 - 2.8501 [2135/2135] unused: 2.8501 - [ 0/0 ] py> centers = binner.bin_centers(1) py> list(1 / centers) [11.421375552155686, 5.4336997182888345, 4.544594791289144, 4.053431451445772, 3.7243527058119903, 3.481806175995981, 3.292371389128289, 3.138504394416836, 3.0099330507138475, 2.9001646196929975] py> centers = binner.bin_centers(4) py> list(1 / centers**(1 / 4)) [7.301226942489155, 5.330776416864701, 4.514065417212759, 4.039592121777967, 3.716672320747147, 3.4770032890247498, 3.28912114749567, 3.13617782414047, 3.0081962210988116, 2.8988250671976883]

1 1

match_indices.paired_miller_indices() ungraceful overshoot
by James Stroud 09 Apr '13

09 Apr '13

Hello all, I accidentally gave paired_miller_indices() a value > 1 for i_array, and got a segfault. The problem looks to be in match_indices.cpp, line 94: result.push_back(miller_indices_[i_array][pairs_[i][i_array]]); Maybe there should be a test that raises an exception on the python side, if possible. James

3 3

flex and numpy caveats
by James Stroud 06 Apr '13

06 Apr '13

Hello All, I've noticed some incompatibilities between numpy and cctbx, which is understandable. However, the incompatibilities manifest either as nonsense errors or, worse, not at all. Following are a couple of examples I have found. 1. The first example should really throw an exception, because silent failures like this can be catastrophic: py> from numpy import random py> r = random.randint(2, size=10) py> r array([1, 0, 1, 1, 0, 1, 0, 1, 1, 1]) py> list(flex.bool(r)) [True, False, False, False, False, False, False, False, False, False] This example is clearly due to incorrect assumptions about the internal representations of numpy ndarrays. Much better behavior can be seen when using a sequence of python `int` objects: py> list(flex.bool(range(10))) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/jstroud/Unison/Code/radialx/testdata/<ipython-input-754-dce96b02979e> in <module>() ----> 1 list(flex.bool(range(10))) TypeError: No registered converter was able to produce a C++ rvalue of type bool from this Python object of type int 2. Although not as potentially catastrophic as the first, the second example should (in a perfect world) either work or throw a more meaningful exception: py> flex.int(range(10))[r[0]] --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/jstroud/Unison/Code/radialx/testdata/<ipython-input-753-f14587f0ceeb> in <module>() ----> 1 flex.int(range(10))[r[0]] TypeError: 'numpy.int64' object is not iterable This example is obviously a result of type checking for a python `int`. The usual python approach of duck-type checking would avoid this problem: def __getitem__(self, i): try: i = int(i) except: pass return do_whatever_with_i(i) James

2 9

Wilson Plot
by James Stroud 02 Apr '13

02 Apr '13

Hello All, The docsting for cctbx.miller.array.wilson_plot() does not seem to agree with its function (see below). Am I misinterpreting something? James py> miller.wilson_plot.__doc__ '<data^2>' py> miller.wilson_plot() (0.6109909654097065+4.386371170677923j) py> flex.mean(miller.data()) (0.6109909654097065+4.386371170677923j) py> flex.mean(miller.data() * flex.conj(miller.data())) (366421.87112270086+0j) py> flex.mean(miller.data() * miller.data()) (70775.39818541733-1856.4852414296488j)

2 1

Miller array by index
by James Stroud 28 Mar '13

28 Mar '13

Hello All, I ask this question mostly out of curiosity, but what is the most direct way to access the value for a Miller index? The best I can do is the method below, but it seems rather indirect. James py> miller <cctbx.miller.array at 0x107bfa290> py> idx = (0, 0, 2) py> miller.select_indices([idx]).data()[0] (-26338.974329482127+0j)

2 1

Re: [cctbxbb] [ccp4bb] refinement protein structure
by James Stroud 28 Mar '13

28 Mar '13

On Mar 27, 2013, at 12:32 PM, Anastassis Perrakis wrote: > These said, I am rather easily irritated these days, so I will not comment on the irritable character of the email. ... > we will soon start calling disordered loops > positionally challenged polypeptide segments (*). You might want to embrace the power of positive thinking. Where you see "positionally challenged", an optimist would see "entropically gifted". James

1 0

Rotate & translate a structure
by James Stroud 27 Mar '13

27 Mar '13

Hello all, I have a rotation + translation matrix. Let's say py> mtx array([[ 0.88302225, 0.11697778, 0.45451948, 5. ], [ 0.11697778, 0.88302225, -0.45451948, 10. ], [ -0.45451948, 0.45451948, 0.76604444, 20. ], [ 0. , 0. , 0. , 1. ]]) My usual way of transforming a set of coordinates with a 4x4 matrix is to use numpy: py> xyz_1 array([[ 5., 10., 15., 1.], [ 20., 15., 10., 1.]]) py> numpy.inner(xyz_1, mtx) array([[ 17.40268126, 12.59731919, 33.76326397, 1. ], [ 28.9603065 , 21.03969455, 25.38784698, 1. ]]) What is the equivalent in cctbx? Is there a way to avoid converting coordinates back and fourth between numpy arrays with atoms().extract_xyz() & .set_xyz() (as might be used from a iotbx_pdb_hierarchy_ext.root). Thank you for any help. James

3 2

Left to dry by iotbx_pdb_ext.input.construct_hierarchy()
by James Stroud 26 Mar '13

26 Mar '13

Hello all, This seems like it should qualify as a bug. If I have an instance of iotbx_pdb_ext.input and I create a hierarchy with construct_hierarchy(), then I must retain the reference to that hierarchy or apparently I can never get it back: >>> hierarchy = my_pdb.construct_hierarchy() >>> def some_local_scope_function(a_pdb): local_hierarchy = a_pdb.construct_hierarchy() >>> some_local_scope_function(my_pdb) […] RuntimeError: atom has another parent atom_group already. Wouldn't better behavior just return the previously constructed hierarchy given that multiple hierarchies can't be made? If each atom apparently remembers to which atom_group it belongs (and presumably on up), then shouldn't it make sense that an iotbx_pdb_ext.input be able to remember it's own hierarchy and produce it upon demand? Looking at a iotbx_pdb_hierarchy_ext.root, I don't find a reference to the iotbx_pdb_ext.input from which it was created. So, although passing around a hierarchy might be the obvious work-around, the end-user still needs to do some accounting if said user wants to track which roots belong to which inputs, and vice versa. James

2 1

Build Notes for cctbx on OS X 10.7.5
by James Stroud 23 Mar '13

23 Mar '13

Hello All, I spent some time to build cctbx on an OS X 10.7 box for a non-system interpreter. I ran into some kinks along the way, so I thought I'd share my notes in case anyone wants to do the same. These notes are in reStructuredText, so it was no extra work to turn them into the attached HTML file, which might make reading a little easier. James ==================================== Build Notes for cctbx on OS X 10.7.5 ==================================== These notes describe a successful build of cctbx against a specific python interpreter, namely `Enthought python`_ 7.3-2 for x86_64. Although working, versions of cctbx are available for both the system python or with a dedicated python, these canned solutions are not optimal. First, it's not advisable to modify system utilities. Although cctbx seems well-behaved in this regard, other packages may not be so well behaved. Second, the canned approach requires building many of the numerous packages that come with bundeld python distributions like the Enthought distribution (some such packages exceed even cctbx in their difficulty to build). Third, if every package required its own python, every script would need to fire-up dozens of separate interpreters just to do its job, resulting in significant computational overhead with consummately slower execution times. .. _`Enthought python`: http://www.enthought.com/products/epd.php 1. The default c++ compiler for OS X 10.7 is actually clang (``/usr/bin/c++``). Using clang exposes invalid template code. The first of many errors is:: In file included from /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent_ext.cpp:7: /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent.h:496:21: error: call to function 'r_factor' that is neither visible in the template definition nor found by argument-dependent lookup r.push_back(r_factor(f_obs, f_model.const_ref())); These types of errors are documented at the page http://clang.llvm.org/compatibility.html#dep_lookup .. note:: The installation directory is ``/opt/cctbx``. 2. I tried to use the HPC compilers at http://hpc.sourceforge.net/, but they do not recognize the ``-no-cpp-precomp`` flag. The web page https://svn.boost.org/trac/boost/ticket/6697 suggests the problem is with boost. However, according to the comments therein, this should have been fixed in boost 1.50. I am building against boost 1.52. Perplexed about the source of this compiler flag, I tracked it down to the file ``cctbx_sources/libtbx/SConscript``, after spending much of a day learning how scons goes about its business. There are two possible fixes: - Comment the following line in ``cctbx_sources/libtbx/SConscript``:: # env_etc.ccflags_base.extend(["-no-cpp-precomp"]) - Link ``/usr/bin/g++`` to ``c++`` in a way that it will be found first in the path. For example:: % ln -s /usr/bin/g++ ${HOME}/bin/c++ 3. I first fixed the ``-no-cpp-precomp`` error by the former, commenting the offending line in ``cctbx_sources/libtbx/SConscript`` but this failed with the error:: /opt/cctbx/cctbx_sources/scitbx/array_family/memory.h:35:47: error: operator '&&' has no right operand #elif defined(_XOPEN_SOURCE) && _XOPEN_SOURCE >= 600 && defined(_POSIX_ADVISORY_INFO) ^ The page http://forums.fedoraforum.org/archive/index.php/t-52929.html suggests that the naked ``_XOPEN_SOURCE`` wrecks the conditional when ``_XOPEN_SOURCE`` is not defined. One possible way to overcome this problem is to change the offending line into a nested if/then, hoping that the comparison never gets evaluated under circumstances when ``_XOPEN_SOURCE`` is not defined:: #elif defined(_XOPEN_SOURCE) #if _XOPEN_SOURCE >= 600 && defined(_POSIX_ADVISORY_INFO) #define SCITBX_AF_HAS_ALIGNED_MALLOC 1 #define SCITBX_AF_USE_POSIX_FOR_ALIGNED_MALLOC 1 #endif #endif However, this was unnecessary because the second fix to the ``-no-cpp-precomp`` error (linking ``/usr/bin/g++`` to ``c++``) worked and avoided changing the preprocessor source. 4. Left to its own devices, scons could not find my ``fftw3.h``, which I have installed (via fink) to ``/sw/include/fftw3.h``. To fix this problem, I inserted the second of the following two lines into the file ``cctbx_sources/libtbx/SConscript``:: env_etc.ccflags_base.extend(["-no-cpp-precomp"]) env_etc.ccflags_base.extend(["-I/sw/include"]) Although this allowed scons to find the header, it could not find the actual library, requiring the insertion of the second of the following lines into the file ``cctbx_sources/libtbx/SConscript``:: env_etc.shlinkflags_bpl = [ "-L/sw/lib", "-w", # suppress "source/lib does not exist" warning "-bundle", "-undefined", "dynamic_lookup"]

4 5

structure_factors() "d_max" equivalent
by James Stroud 22 Mar '13

22 Mar '13

Hello All, I'm new to cctbx and trying to get the most rudimentary understanding. I would like to select all of the reflections within a particular resolution shell. This line from the tutorial: f_calc = quartz_structure.structure_factors(d_min=2).f_calc_array() gets all structure factors through the highest resolution of d_min=2 Å (excluding 0 0 0). My question: is there an equivalent to a d_max argument for this function (i.e. a low resolution limit)? James

4 3

cctbx and symmetry: source code
by prasun kumar 22 Mar '13

22 Mar '13

Dear all I am developing a software for which I needed to use the unit cell generation module of the cctbx. The software will be working for small molecule as well as macromolecule. It will be a great help, if some body can suggest me the part of the program to be used as I am getting confused while selecting the .cpp and .h files. Thanx alot in advance Prasun Kumar PRASUN (ASHOKA) Desire + stability = Resolution Resolution + Hard work = Success

4 10

changes to SourceForge project
by Nathaniel Echols 20 Mar '13

20 Mar '13

As promised, the CCTBX SourceForge site has been migrated to their new backend. I am still not sure exactly what this means, but there are two practical changes: 1) The SVN URL has changed - it used to be http://cctbx.svn.sourceforge.net/svnroot/cctbx. Now we have several options. svn checkout --username=natechols svn+ssh://[email protected]/p/cctbx/code/trunk cctbx_project svn checkout --username=natechols https://svn.code.sf.net/p/cctbx/code/trunk cctbx_project svn checkout svn://svn.code.sf.net/p/cctbx/code/trunk cctbx_project The last one is read-only; for the others, replace "natechols" with your SourceForge ID. The old repository is now read-only, so you will need to do a completely new checkout before you can commit any changes you make. Hopefully this will cause minimal disruption; I will change the nightly builds ASAP. 2) As requested, we now have a wiki (completely empty right now): https://sourceforge.net/p/cctbx/wiki/Home/ Feel free to start populating this - if you would like to contribute and don't have access, sign up for a SourceForge account and email me your user ID. -Nat

2 1

Best way to compare two structures using cctbx?
by Jan Marten Simons 13 Mar '13

13 Mar '13

Hi, I think I've seen some code in cctbx which tries to put the similarity of two crystal structures into a number. Am I right on this? I want to have something similar: I'm interested in comparing the direct space atomic distances of two structures sharing the same unit cell parameters and space group and scatterers but different atomic coordinates for those. Now I'd like to have a mapping where I could compare the bond lengths between all scatterers which should eliminate any movements which might also be due to a shift in the origin of a P 1 cell. Also I'd like to have a value for mean, minimal and maximal displacements. The purpose of this is to check how good a structure approximation matches a reference structure. How would you suggest to take on this task? Or is there even some code related to this? Thanks in advance, Jan

5 8

upcoming SourceForge changes
by Nathaniel Echols 27 Feb '13

27 Feb '13

FYI for anyone using CCTBX SVN checkouts: on or about March 9th or 10th, I plan to upgrade our SourceForge project to their new platform, which they claim to want everyone to do in the first quarter of 2013. (I had hoped to do this a month ago but it would have created problems for other projects.) This should give us access to new features including a project wiki, requested by several of you. I am hoping any disruption to SVN access will be minimal and/or quickly resolved (the timing coincides with our Phenix developer workshop so those of us who are attending will be able to fix problems interactively). I'll send out another email after the switch summarizing any changes. -Nat

1 0

pdb shake - dihedral angles
by Dmytro Guzenko 20 Feb '13

20 Feb '13

Hi everyone, I wonder what would it take to write an equivalent to pdb_shake script from iotbx/examples with randomization of the backbone dihedral angles phi and psi, instead of all coordinates perturbation? I want to be able to randomize models in a more "realistic" way. Are there data structures/procedures that can be useful for this in cctbx, or should I extract the coordinates and take it from there? Thanks! Kind regards, Dmytro.

3 2

Re: [cctbxbb] unexpected behaviour of unit_cell.fractionalize
by Jan Marten Simons 14 Feb '13

14 Feb '13

Am Mittwoch 13 Februar 2013 17:01:37 you wrote: > Hi Jan, > > can this method work for you: sites_mod_positive ? I don't remember what > exactly it does but I think it either what you need or at least very close. Hi Pavel, hm, it goes in the right direction, but is not exactly what I'm looking for, as it is bound to scatterers and not to sites in general. And adding a scatterer for each site I want to convert to the structure first, then converting, and than extracting the converted sites from the scatterers afterwards seems quite inefficient to me. Also I think it would be clearer to add the functionality for this to the "fractionalize" function as people trying to convert from direct space coordinates to fractional coordinates will most likely find this functionality first. Cheers, Jan > On 2/13/13 7:17 AM, Jan Marten Simons wrote: > > Hi, > > > > When I try to convert a list of direct space coordinates into fractional > > coordinates like this: > > > > from cctbx import xray > > from cctbx import crystal > > from cctbx.array_family import flex > > > > structure = xray.structure( > > > > special_position_settings=crystal.special_position_settings( > > > > crystal_symmetry=crystal.symmetry( > > > > unit_cell=(3.14098, 4.08327, 5.33966, 83, 109, 129), > > space_group_symbol="P1"))) > > > > sites = flex.vec3_double( ((0.0, 0.1, 0.2), (1.3, 3.0, 2.3), (5.3, 0.1, > > 7.3)) ) sites_frac = structure.unit_cell().fractionalize(sites) > > print(list(sites_frac)) > > > > Now this returns a list of coordinates where x,y,z may be outside ot the > > interval [0,1[. ... quite unexpected. > > > > It would be very nice to have at least an option like "restrict_to_asu" > > in unit_cell.fractionalize which would enforce (x,y,z) to be inside the > > asymmetric unit and/or "restrict_to_cell" which would enforce (x,y,z) to > > be inside the unit cell. > > > > Could any of you more into the inner workings of the cctbx add this > > functionality, please? > > > > Thanks in advance, > > > > Jan

3 4

unexpected behaviour of unit_cell.fractionalize
by Jan Marten Simons 14 Feb '13

14 Feb '13

Hi, When I try to convert a list of direct space coordinates into fractional coordinates like this: from cctbx import xray from cctbx import crystal from cctbx.array_family import flex structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=crystal.symmetry( unit_cell=(3.14098, 4.08327, 5.33966, 83, 109, 129), space_group_symbol="P1"))) sites = flex.vec3_double( ((0.0, 0.1, 0.2), (1.3, 3.0, 2.3), (5.3, 0.1, 7.3)) ) sites_frac = structure.unit_cell().fractionalize(sites) print(list(sites_frac)) Now this returns a list of coordinates where x,y,z may be outside ot the interval [0,1[. ... quite unexpected. It would be very nice to have at least an option like "restrict_to_asu" in unit_cell.fractionalize which would enforce (x,y,z) to be inside the asymmetric unit and/or "restrict_to_cell" which would enforce (x,y,z) to be inside the unit cell. Could any of you more into the inner workings of the cctbx add this functionality, please? Thanks in advance, Jan

2 1

Strange behaviour of resolution filter
by Jan Marten Simons 08 Feb '13

08 Feb '13

Hi, I discovered a rather strange behaviour of the miller.index_generator. Sample code showing the weirdness: -----8<--------------------------- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=crystal.symmetry( unit_cell=(3.14098, 4.08327, 5.33966, 83, 109, 129), space_group_symbol="P 1")), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.,0.,0.), u=0.0)])) fc1 = structure.structure_factors(d_min=1.0).f_calc().sort() d_max, d_min = fc1.resolution_range() fc2 = structure.structure_factors(d_min=d_min).f_calc().sort() print(fc1.size(), fc2.size()) #fc1.show_array() fc1.show_comprehensive_summary() print("\nd_min: {}\n".format(d_min)) fc2.show_comprehensive_summary() #fc2.show_array() -----8<--------------------------- fc1 has 105 indices, while fc2 only has 104. Also fc2 has a higher d_min than fc1 although I'd expect both to have the same. To me it seems like the index with d==d_min is not included and only indices with d>d_min are calculated. I wonder if this behaviour is the desired one or rather a bug, as I was quite surprised when I discovered this. With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

2 1

Using cctbx::uctbx::unit_cell in c++ code / boosted to Python
by Graeme.Winter＠diamond.ac.uk 26 Jan '13

26 Jan '13

Hi Folks, Been working on some c++ coding using boost etc (which I have to say is great) however I found one bit of weirdness. If I want to generate a cctbx::uctbx::unit_cell in c++ code then access this from Python I get Traceback (most recent call last): File "tst_x2tbx.py", line 67, in <module> tst_x2tbx(sys.argv[1]) File "tst_x2tbx.py", line 6, in tst_x2tbx import x2tbx File "/Users/graeme/svn/cctbx/sources/x2tbx/__init__.py", line 7, in <module> ext = boost.python.import_ext("x2tbx_ext", optional = False) File "/Users/graeme/svn/cctbx/sources/cctbx_project/boost_adaptbx/boost/python.py", line 36, in import_ext + [" "+p for p in sys.path])) ImportError: __import__("x2tbx_ext"): dlopen(/Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so, 2): Symbol not found: cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny<double, 6ul> const&) Referenced from: /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so Expected in: flat namespace in /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so sys.path: …. unless I have import cctbx.uctbx # explicit import at the top of the Python code - have I done something wrong with the SConscript? I found something similar with import scitbx.array_family.flex # explicit import which libtbx.clean_clutter complains about as I don't reference it (hence the additional comment) Equally I could just leave the additional clutter there but it feels like I should not need it... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 8

Sphinx documentation
by Graeme.Winter＠diamond.ac.uk 21 Jan '13

21 Jan '13

Hi Folks, I installed sphinx and tried building the documentation but I got an error: File ./xfel/xfel.cxi.gfx.rst already exists, skipping. File ./xfel/xfel.cxi.merging.rst already exists, skipping. File ./xfel/xfel.cxi.rst already exists, skipping. File ./xfel/xfel.metrology.rst already exists, skipping. File ./xfel/xfel.rst already exists, skipping. File ./xfel/modules.rst already exists, skipping. sphinx-build -b coverage -d _build/doctrees -D latex_paper_size=a4 . _build/coverage Making output directory... Running Sphinx v1.2pre loading pickled environment... not yet created building [coverage]: coverage overview updating environment: 206 added, 0 changed, 0 removed reading sources... [ 0%] cctbx/cctbx Exception occurred: File "/Users/graeme/svn/cctbx/phenix/phenix-dev-1241/build/mac-intel-osx-x86_64/base/Python.framework/Versions/Current/lib/python2.7/site-packages/Sphinx-1.2predev_20130121-py2.7.egg/sphinx/ext/autodoc.py", line 595, in filter_members membername in self.options.special_members: TypeError: argument of type 'object' is not iterable The full traceback has been saved in /var/folders/j1/cgsqy8fs1lxdyjggcsb3twsw0000gn/T/sphinx-err-jjM5C8.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-users/>, or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks! make: *** [coverage] Error 1 cp: ./_build/coverage/python.txt: No such file or directory cp: ./_build/coverage/c.txt: No such file or directory sphinx-build -b html -d _build/doctrees -D latex_paper_size=a4 . _build/html Making output directory... Running Sphinx v1.2pre loading pickled environment... not yet created building [html]: targets for 206 source files that are out of date updating environment: 206 added, 0 changed, 0 removed reading sources... [ 0%] cctbx/cctbx Exception occurred: File "/Users/graeme/svn/cctbx/phenix/phenix-dev-1241/build/mac-intel-osx-x86_64/base/Python.framework/Versions/Current/lib/python2.7/site-packages/Sphinx-1.2predev_20130121-py2.7.egg/sphinx/ext/autodoc.py", line 595, in filter_members membername in self.options.special_members: TypeError: argument of type 'object' is not iterable The full traceback has been saved in /var/folders/j1/cgsqy8fs1lxdyjggcsb3twsw0000gn/T/sphinx-err-lONSky.log, if you want to report the issue to the developers. Please also report this if it was a user error, so that a better error message can be provided next time. Either send bugs to the mailing list at <http://groups.google.com/group/sphinx-users/>, or report them in the tracker at <http://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks! make: *** [html] Error 1 Did I do something wrong? I just grabbed sphinx from the internet. Should I have got it from someplace else? OS X Lion / svn cctbx. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Phil (dumb) question
by Graeme.Winter＠diamond.ac.uk 15 Jan '13

15 Jan '13

Hi Folks, I have a toy example where I should be overriding Phil defaults: phil_defaults = ''' resolutionizer { rmerge = 0.0 .type = float completeness = 0.0 .type = float isigma = 1.0 .type = float misigma = 2.0 .type = float bins = 100 .type = int } ''' more = ''' resolutionizer { isigma = 2.0 misigma = 4.0 } ''' from libtbx.phil import parse working_phil = parse(phil_defaults) working_phil.fetch(parse(more)) working_phil.show() but it seems I am not: Graemes-MacBook-Pro:scale graeme$ cctbx.python toy.py resolutionizer { rmerge = 0.0 completeness = 0.0 isigma = 1.0 misigma = 2.0 bins = 100 } Am I doing something dumb? I hope so as I really cannot find my error. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

Simple cctbx / boost c++ extension example
by Graeme.Winter＠diamond.ac.uk 12 Jan '13

12 Jan '13

Hi Folks, Just say I wanted to write a new from the ground up module for cctbx which wanted to include some C++ code, and a SConscript etc. so I can run libtbx.configure toytbx (say) And then find from toytbx import cplusplus_functions would work.... Is there an example I could follow? (I have not looked through the newsletters, I suppose I should have... but I could not seen anything obvious in the documents) Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 6

Computing E^4 from an mtz file
by Graeme.Winter＠diamond.ac.uk 15 Dec '12

15 Dec '12

Hi Folks, Trying to write some code to return the average E^4 across (some) bins for an mtz file, I got to def main(): import sys from iotbx import reflection_file_reader for argv in sys.argv[1:]: reflection_file = reflection_file_reader.any_reflection_file( file_name = argv) if reflection_file.file_type() is None: continue miller_arrays = reflection_file.as_miller_arrays() for ma in miller_arrays: E4(ma) def E4(ma): f = ma.as_intensity_array() f = f.array(data=f.data()/f.epsilons().data().as_double()) f.set_observation_type_xray_intensity() f.setup_binner(auto_binning = True) sm = f.second_moment(use_binning = True) sm.show() if __name__ == '__main__': main() but I have no idea how to actually extract the information I want from sm as this only has show() which gives Graemes-MacBook-Pro:PlayPen graeme$ cctbx.python E4.py /Users/graeme/projects/sci-397/DEFAULT/scale/AUTOMATIC_DEFAULT_scaled.mtz unused: - 20.8668 [ 0/7 ] bin 1: 20.8668 - 5.8873 [221/235] 2.3863 bin 2: 5.8873 - 4.6904 [216/221] 1.9823 bin 3: 4.6904 - 4.1026 [221/222] 1.9925 <snip> bin 47: 1.6553 - 1.6435 [153/209] 2.2051 bin 48: 1.6435 - 1.6320 [143/198] 2.2021 bin 49: 1.6320 - 1.6208 [141/204] 2.7085 unused: 1.6208 - [ 0/0 ] unused: - 20.8668 [ 0/9 ] bin 1: 20.8668 - 7.1654 [201/225] 2.2405 bin 2: 7.1654 - 5.7261 [202/213] 1.9109 bin 3: 5.7261 - 5.0137 [216/223] 2.0162 bin 4: 5.0137 - 4.5605 [207/212] 2.0043 <snip> bin 87: 1.6456 - 1.6393 [126/192] 2.5521 bin 88: 1.6393 - 1.6330 [122/210] 2.0195 bin 89: 1.6330 - 1.6269 [131/224] 2.6251 bin 90: 1.6269 - 1.6208 [120/212] 3.0912 unused: 1.6208 - [ 0/0 ] Really I would like the value programatically or at least a way of getting back an array of the binned values so I can average them myself (which is obviously trivial) Any suggestions? Thanks, Graeme ps - most of this code is ripped from iotbx.reflection_statistics :o) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 4

Writing a pdb file from an xray_structure object
by Claudia Millán Nebot 13 Dec '12

13 Dec '12

Hi everyone, i have been using the xray_structure_simple() method to get an xray.structure object from a pdb object and use it to perform some calculations on it, one of them being expand it to P1. After, i tried to write a new pdb out of this xray_structure in P1. For that purpose, i used the following method: cctbx.xray.structure.as_pdb_file(self, remark=None, remarks=[], fractional_coordinates=False, resname=None, connect=None) but using it both with the original xray structure and with the one on P1 I get very bad lookings pdbs, with incorrect formating, like that for the case of the one in P1: ATOM 1 PDB= PDB 1 12.330 -6.948 -40.820 1.00 19.26 N ATOM 2 PDB= PDB 2 49.988 55.370 3.948 1.00 19.26 N ATOM 3 PDB= PDB 3 36.092 30.710 93.483 1.00 19.26 N ATOM 4 PDB= PDB 4 55.370 49.988 175.123 1.00 19.26 N ATOM 5 PDB= PDB 5 30.710 36.092 85.587 1.00 19.26 N so i guessed that something must be wrong with the residues.... and went to check the code. That as_pdb_file method calls to iotbx.pdb.xray_structure.as_pdb_file(), and on that code i realised that as i was calling it whithout any resname, it was retrieving it from some information on the scatterers that could be wrong. Putting the value of resname to ALA, as a trial, produced almost the same output except for the name: ATOM 1 PDB= ALA 1 12.330 -6.948 -40.820 1.00 19.26 N ATOM 2 PDB= ALA 2 49.988 55.370 3.948 1.00 19.26 N ATOM 3 PDB= ALA 3 36.092 30.710 93.483 1.00 19.26 N ATOM 4 PDB= ALA 4 55.370 49.988 175.123 1.00 19.26 N ATOM 5 PDB= ALA 5 30.710 36.092 85.587 1.00 19.26 N Up to this point, i don't get what is wrong. It may be that i'm using a too simple method and without the correct parameters (xray_structure_simple()), as i guess part of the error is that the assignation of the types of atoms and residues is going crazy. There are other ways of performing this step from a pdb to an xray_structure and viceversa? Thanks in advance! Best regards, Claudia Millán (cmncri(a)ibmb.csic.es) Institut de Biologia Molecular de Barcelona (IBMB-CSIC) Barcelona, Spain

2 2

Documentation .... again
by Graeme.Winter＠diamond.ac.uk 12 Dec '12

12 Dec '12

Hi Folks, Been thinking some more about the old chestnut of documentation. The lack thereof is a well understood issue, and to be honest one which is not going to change any time soon as no one has the time to maintain a manual - this is fine and well understood. As a kind of "patch" for this I was wondering if it would be possible to have a forum where someone could ask for e.g. documentation for a currently undocumented function, which could then be added to the subversion in the code itself. For example: cctbx_project/cctbx/xray/structure.py ... def structure_factors(self, anomalous_flag=None, d_min=None, algorithm=None, cos_sin_table=False, quality_factor=None, u_base=None, b_base=None, wing_cutoff=None): what is cos_sin_table for? with things like this there is the attitude that it is safe to ignore, however if we're writing code to guide people's experiments it would be good to be sure of this. Some simple documentation explaining this would be very useful. I'm definitely not suggesting (i) that anyone writes a manual or (ii) that all undocumented functions are suddenly documented, only that there is a mechanism set up that someone could ask about this and that someone who knows the answer could commit the changes to cctbx and then reply with the diffs - perhaps the way that stackoverflow works, or an email, or something really lightweight. Even this mailing list could be used I guess? I would be happy to help with this, though the above example is not one where I know the answer! Thoughts? Best wishes, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

8 24

Structure factor calculation weirdness
by Graeme.Winter＠diamond.ac.uk 10 Dec '12

10 Dec '12

Hi Folks, Some confusion here at diamond - it seems that the amplitude of F(hkl) depends on dmin ---------------- from cctbx import xray from cctbx import crystal from cctbx.array_family import flex # 14keV wavelength = 12.3984 / 14.0 spgroup = 'Fd-3m' latt = (5.4309, 5.4309, 5.4309, 90, 90, 90) atoms = [('Si', 0.125, 0.125, 0.125)] allscatterers=[] for atom in atoms: allscatterers.append(xray.scatterer( label = atom[0], site = (atom[1], atom[2], atom[3]), u = 0)) structure = xray.structure( special_position_settings = crystal.special_position_settings( crystal_symmetry = crystal.symmetry( unit_cell = latt, space_group_symbol = spgroup)), scatterers = flex.xray_scatterer(allscatterers)).expand_to_p1() for dmin in 5.0, 2.5, 1.7: print dmin f_calculations = structure.structure_factors( d_min = dmin).f_calc().amplitudes().show_array() --------------- Chopped out only 001, 010, 100 from this... 5.0 (0, 0, 1) 0.290277589693 (0, 1, 0) 0.290277589693 (1, 0, 0) 0.290277589693 2.5 (0, 0, 1) 0.0 (0, 1, 0) 0.0 (1, 0, 0) 0.0 <snip> 1.7 (0, 0, 1) 0.278378587828 (0, 1, 0) 0.278378587828 (1, 0, 0) 0.278378587828 <snip> Have we misunderstood something? Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

7 9

UB matrix in cbf files
by Carlo.Vecchini＠diamond.ac.uk 05 Dec '12

05 Dec '12

Hello, I would like to add the orientation matrix information in the cbf file. Following the description in sections 3.2.2.2.3 of IUCr (G), I added the following lines in the cbf header: _diffrn_orient_matrix_type ub_matrix_test _diffrn_orient_matrix_UB_11 0.083424 _diffrn_orient_matrix_UB_12 -0.076007 _diffrn_orient_matrix_UB_13 -0.078787 _diffrn_orient_matrix_UB_21 -0.103374 _diffrn_orient_matrix_UB_22 -0.075939 _diffrn_orient_matrix_UB_23 -0.01146 _diffrn_orient_matrix_UB_31 -0.034434 _diffrn_orient_matrix_UB_32 0.043832 _diffrn_orient_matrix_UB_33 -0.156476 When I try to read back the values with: get_orientation_matrix() I get the following error: UBm = cbf_handle.get_orientation_matrix() File "~/CCTBX/cctbx_plus_build/lib/pycbf.py", line 3915, in get_orientation_matrix return _pycbf.cbf_handle_struct_get_orientation_matrix(self) Exception: CBFlib Error(s): CBF_NOTFOUND If I use set_orientation_matrix() to write the cbf file, I get a similar error. Does anyone have any suggestion on how to solve this? Thanks in advance, Carlo Vecchini -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

CBF compositing
by Frank Murphy 03 Dec '12

03 Dec '12

Hello, Does anyone have a CCTBX-based method for compositing multiple CBFs to a single image a la merge2cbf? Thanks in advance, Frank Murphy

2 3

OS X Lion + CCTBX + scipy
by Graeme.Winter＠diamond.ac.uk 02 Dec '12

02 Dec '12

Hi Folks, Has anyone got scipy to build against the cctbx python binary distribution on OS X Lion? I am finding that it keeps on compiling some bits of the library for PPC architecture which does not work... Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

SVN broken?
by Jan Marten Simons 24 Oct '12

24 Oct '12

Hi, while trying to update to the latest svn revision or when trying a fresh checkout I encounter this error: svn co https://cctbx.svn.sourceforge.net/svnroot/cctbx/trunk cctbx_project svn: Can't find a temporary directory: Internal error According to a quick web search the problem is most likely on the svn server (either no disk space or corrupted .svn dir). Hopefully either sourceforge or any of you can fix this. Cheers, Jan

2 2

Re: [cctbxbb] Question about sigma(I) calculation in merge_equivalents
by Kay Diederichs 22 Oct '12

22 Oct '12

Dear Keitaro, Phil, Luc, after looking into the question of how to merge equivalent reflections (in the context of XDS data processing), I am now convinced that cctbx should _not_ use max(internal sigma, external sigma) by default. Why? - it is not obvious to the "end user" of the data that come out of cctbx that this formula was applied (I am not aware that the above formula is clearly documented) - the data processing programs that I know of do not use this formula, but rather use the "external sigma" . I don't imply that the "external sigma" is better, but it violates the principle of least surprise to deviate from a proven procedure. - if the observations (that are merged in cctbx) come from one of these data processing programs then there is a chance that the error model was already adjusted such that the reduced chi**2 is near 1 (at least this is the case for XDS and SCALA). If the sigmas of the merged data then are adjusted upwards again (by the formula above), I have a strong feeling that this leads to an inconsistency. - using max() seems like an ad-hoc way and it lacks a clear rationale. I would like to see examples of comparison of downstream crystallographic calculations (most importantly, experimental phasing) using different ways of calculating the sigma of the merged data. Until this proves the superiority of the above formula, I believe it should be an option, not a default. thanks, Kay > Keitaro Yamashita yamashita at castor.sci.hokudai.ac.jp > Thu Sep 6 09:20:34 PDT 2012 > > Previous message: [cctbxbb] Question about sigma(I) calculation in > merge_equivalents > Next message: [cctbxbb] Question about sigma(I) calculation in > merge_equivalents > Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] > > Dear Luc, > > Thank you for your explanation! I understand better than before. > > Usually, sigmas given by data processing programs are already > corrected based on their error model. > Sigmas are adjusted to match the actual scatter. > I think using internal variance is re-correction of sigmas. > Is it valid way? If internal variance is bigger, it suggests error > model is not perfect? > > I calculated external/internal variances using lysozyme (standard > sample in protein crystallography) data. > Intensities and sigmas are determined by XDS. > > I attached two plots, where "Imean", "wsigma", "sigma" are averaged > intensity, internal sigma, external sigma, respectively. > One plot is histogram of wsigma/sigma by multiplicity. For lower > multiplicity, we can see extreme discrepancies. (Note that each > vertical axis is not on the same scale.) > The other plot is wsigma/sigma vs intensity. Extreme discrepancies can > be seen in lower intensities. > I hope it could be interesting for you. > > > > Crystals use only the external variance by default, the reasoning > being that the internal variance being based on sample statistics is > almost always too unreliable because groups of equivalent reflections > are too small. > > Then, I think it would be nice if we can choose the way in cctbx. > I mean, option to choose "use bigger one" or "always use > external/internal variance" would be nice to have. > > I am looking forward to your comment. > > Best regards, > Keitaro > > 2012/9/6 Luc Bourhis <luc_j_bourhis at mac.com>: > > Dear Keitaro, > > > >> But it is still unclear to me why it takes the greatest of the > >> "internal" variance and "external" variance. > >> Is it based on some tests using real data? or is it theoretically > >> superior to using always external variance? > > > > Those are good questions and to be honest I do not know for sure the > answer to them. As it seems common in applied statistics, the treatment > starts with by-the-book methods relying on a well defined theory but at > the end there is always a completely heuristic twist. Particularly true > in crystallography I would argue. But let me try to give some rationales. > > > > It seems to me that the internal and external variance should not > differ too much. Let's consider the two ways this may not be true. > > > > 1. The quoted intensities of a group of equivalent reflections have a > small spread, leading to a small internal variance, but the quoted > sigma's are comparatively big, resulting in an external variance > significantly bigger than the internal one. This is a possible event but > an unlikely one: the statistical intuition in that case is to say that > the small internal variance is a fluke and to use the external one instead. > > > > 2. An external variance significantly smaller than the internal one, > should ring an alarm bell. Indeed a small external variance means that > the small quoted sigma's strongly suggests the intensities cannot spread > too much from their assumed common true value whereas the comparatively > bigger internal variance blatantly contradicts that. Thus either the > intensities or the sigma's have not been correctly determined. > Crystallographers seem to err on the side of trusting data here, i.e. to > disregard the sigma's, and therefore to choose the internal variance. > > > >> I would like to know how this method affects further > crystallographic process. > > > > I am afraid I do not have experience with your domain, protein > crystallography. I know that the small molecule program Crystals use > only the external variance by default, the reasoning being that the > internal variance being based on sample statistics is almost always too > unreliable because groups of equivalent reflections are too small. Since > Crystals is as well accepted as ShelXL to produce publishable > structures, it answer your question in at least in one corner of > crystallography, unfortunately not yours. > > > > I think it could be a simple and interesting exercise to take a > representative protein dataset of yours, then to print the redundancy, > the internal, and the external variance. Actually I would be surprised > if such a study has not already been done and published. Perhaps some of > the gurus on this forum can shed more lights onto that subject. > > > > Best wishes, > > > > Luc -- Kay Diederichs http://strucbio.biologie.uni-konstanz.de email: Kay.Diederichs(a)uni-konstanz.de Tel +49 7531 88 4049 Fax 3183 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz This e-mail is digitally signed. If your e-mail client does not have the necessary capabilities, just ignore the attached signature "smime.p7s".

2 1

FYI: The Quaternion Characteristic Polynomial method
by Jeffrey Van Voorst 20 Oct '12

20 Oct '12

Greetings, If you already know about the quaternion characteristic polynomial method for a fast closed-form solution of the least squares superposition problem, this note is mostly noise. While in the process of updating links to my project, I noticed the note at the bottom of the cctbx sourceforge page (http://cctbx.sourceforge.net/). I like the historical nature presented, and I used the historical method to numerically solve quartic polynomials in a previous project at Michigan State University that ranks relatively large datasets of protein small-molecule binding sites (with respect to a given query site). The superposition method I used has a closed form and is due to BKP Horn. Later on (after writing my dissertation), I found that using the historical method to numerically solve quartic polynomials is not numerically stable or robust. Fortunately, there is an easy solution, and if you are interested in a fast closed-form solution for the least squares superposition problem you would do well (in my opinion) to visit http://theobald.brandeis.edu/qcp/. Best regards, Jeff Van Voorst

1 0

--enable-boost-threads flag
by Jeffrey Van Voorst 19 Oct '12

19 Oct '12

Greetings, If someone knows the answer off the top of their head, I would be interested in the answer. Otherwise, feel free to ignore this. Does the --enable-boost-threads flag for cctbx_sources/libtbx/configure.py do anything drastic besides include -pthread in the compiling and linking steps? I am trying to understand why I would get a free() error (for the shared_ptr to root_data) under the following situation: *) the iotbx_pdb shared library is configured without the --enable-boost-threads flag (on a linux box) *) sometimes reproducible from a test C++ program that is linked using -pthread even in the case when the program is as simple as loading a pdb file and creating the hierarchy. *) gcc 4.4.5 --Jeff Van Voorst

4 14

Re: [cctbxbb] Patterson map data structure
by Ten Eyck, Lynn 17 Oct '12

17 Oct '12

Any Fourier map without an F000 term will analytically have a mean value of 0. Lynn Connected by DROID on Verizon Wireless -----Original message----- From: Nathaniel Echols <nechols(a)lbl.gov> To: cctbx mailing list <cctbxbb(a)phenix-online.org> Sent: Tue, Oct 16, 2012 17:22:17 GMT+00:00 Subject: Re: [cctbxbb] Patterson map data structure On Tue, Oct 16, 2012 at 6:47 AM, Dmytro Guzenko <Dmytro.Guzenko(a)pharm.kuleuven.be> wrote: > rmap = pmap.real_map_unpadded() > n_real = pmap.n_real() > unit_cell = pmap.unit_cell() > grid2cart = maptbx.grid2cart(n_real,unit_cell.orthogonalization_matrix()) > > n_points_total = n_real[0]*n_real[1]*n_real[2] > cart_coord = flex.double(flex.grid(3,n_points_total)) > p_weight = flex.double(n_points_total) > > for plain_ind,grid_point in enumerate(flex.nested_loop(n_real)): > cart_coord[0,plain_ind],cart_coord[1,plain_ind],cart_coord[2,plain_ind] = grid2cart(grid_point) > p_weight[plain_ind] = rmap[grid_point] There are better arrays for storing the coordinates, e.g.: cart_coord = flex.vec3_double(flex.grid(n_real)) p_weight = flex.double(rmap.size()) for plain_ind,grid_point in enumerate(flex.nested_loop(n_real)): cart_coord[grid_point] = grid2cart(grid_point) p_weight[plain_ind] = rmap[grid_point] > By the way, I noticed that many weights are negative, is this possible in a patterson map or am I doing something wrong? This is normal - I believe it's a Fourier series termination error, but I still don't fully understand the reason. An Fobs map will have the same problem; the mean will always be approximately zero. Technically, if the map were perfect with no missing data, it would be uniformly positive, but that's never actually the case. There are ways to "fix" the map so it's all-positive, such as maximum entropy cleanup, but in practice I think we just ignore the negative values (at least for Fobs/2mFo-DFc maps - not sure how Patterson maps are usually handled). -Nat _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org http://phenix-online.org/mailman/listinfo/cctbxbb

1 0

Patterson map data structure
by Dmytro Guzenko 17 Oct '12

17 Oct '12

Hello all, I am playing with the Patterson function as calculated by cctbx and I am confused as to how I can extract and manipulate the data. More specifically, miller.patterson_map(...).real_map() returns a one-dimensional array of values. How do I get the corresponding 3D coordinates? I suspect it has to do with the gridding and some kind of fixed order enumeration, but I haven't found an example (the peak_search().sites() seems to be relevant, but I would like the whole thing). On a related note - is there a way to debug the python code of cctbx and step into the c++ code? Debugging is my favorite way to figure things out, but with cctbx most interesting things are hidden. I am using Eclipse + PyDev now. Thank you! Kind regards, Dmytro.

3 4

iotbx.xds
by Graeme.Winter＠diamond.ac.uk 16 Oct '12

16 Oct '12

Hi Folks, Would anyone have any objection if I added some code to iotbx.xds to read things like experimental geometry etc in a straightforward way from the various files that the program produces? I see something to read reflection files in there already. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 4

no more Python 2.4
by Nathaniel Echols 16 Oct '12

16 Oct '12

FYI for those of you who didn't get my original email: the minimum Python version required for CCTBX is now 2.5, so for anyone contributing to CCTBX, you can now use more advanced syntax such as "with" and "try/except/else/finally" (and decorators, which were already supported in 2.4, but we haven't used them so far). Full list of changes in 2.5 here: http://docs.python.org/whatsnew/2.5.html -Nat

1 0

Python documentation
by David Waterman 10 Oct '12

10 Oct '12

Hi, I've got a few questions about the automatically generated Python documentation. 1. Is this done by doxygen like the C++ documentation? 2. What are the rules that determine whether text in source files is included as documentation or not? Clearly docstrings are, but also some comments do (only those at module scope?) 3. Similarly, what determines which packages are documented or not? For example, from http://cctbx.sourceforge.net/current/python/rstbx.html, cftbx is documented but bpcx is not. Is there at some point a manual decision about what should be documented? Cheers -- David

4 7

Going from a pdb file to an x_ray_structure object
by Claudia Millán Nebot 09 Oct '12

09 Oct '12

Hi everyone, i'm trying to get an xray.structure object from a pdb file, as i'm interested on using both this and a Miller array coming from a reflection file to calculate correlation coefficients between Fobs and Fcalc. For this purpose, i was searching for something that will do the same as the iotbx.pdb.xray_structure.as_pdb_file but the other way around. Is there such a tool? I've look for a while but as i'm new to all that maybe i missed something. Thanks in advance and my excuses if is a too evident question, Claudia Millán (cmncri(a)ibmb.csic.es) Institut de Biologia Molecular de Barcelona (IBMB-CSIC) Barcelona, Spain

3 8

Re: [cctbxbb] RE : cctbx debian package: new commit
by Radostan Riedel 04 Oct '12

04 Oct '12

On Wed, 03. Oct 11:11, Baptiste Carvello wrote: > 2) there are a few C++ applications which depend on cctbx, the one I > know is FOX ( http://vincefn.net/Fox/ , objcryst-fox in Debian). For > those use cases, should we still build a libcctbx-dev package with the > headers and static libs (for those we don't need to care about ABI > stability)? I think this is a very important point here. Let's assume I want to extent the cctbx by adding a module, for let's say absorbtion correction and I want to implement a method called "multiscan" and I want to make a fast python boost extension to integrate with my python code. I need some symbols from any of the shared libs in cctbx. How could this be done in Debian or in any other distribution? About the check for version: In my patch I did this with SCons[1]. What this does is to compile and link a test program and check for success. I saw in your code that you are doing similar things and we could write some simple c++ program that for example includes boost/version and check some "BOOST_VERSION"? This way it checks globally for the availability of a shared library. [1] http://www.scons.org/doc/2.0.1/HTML/scons-user/x4146.html

2 1

Explicit import for iotbx.pdb.amino_acid_codes
by Claudia Millán Nebot 02 Oct '12

02 Oct '12

Hello everyone :) I've just started to look into the cctbx facilities, and as i'm a newbie in both object oriented programming and in general to the system, i'm tryng to play around and find by myself how things are working. Using the cctbx.python, i realised that it is necessary to import explicitly the module amino_acid_codes, while for others like atom_name_interpretation this is done straigthforward if you do an import from a lower level (like from iotbx.pdb import *). Is there some good reason because this has to be like that? Thanks in advance for the answer, and excuse me if is it is a question too simple! Claudia Millán (cmncri(a)ibmb.csic.es) Institut de Biologia Molecular de Barcelona (IBMB-CSIC) Barcelona, Spain

2 2

Build failing
by Graeme.Winter＠diamond.ac.uk 01 Oct '12

01 Oct '12

Hi Folks, Following an svn update and several attempts to build, I am still getting this compile error /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:11: error: 'bessel_y_small_z_series' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'template<class T, class Policy> T boost::math::detail::bessel_y_small_z_series(T, T, T*, const Policy&)' declared here, later in the translation unit In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17:0, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:11: error: 'bessel_j_small_z_series' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'template<class T, class Policy> T boost::math::detail::bessel_j_small_z_series(T, T, const Policy&)' declared here, later in the translation unit In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17:0, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:11: error: 'bessel_yn_small_z' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'template<class T, class Policy> T boost::math::detail::bessel_yn_small_z(int, T, T*, const Policy&)' declared here, later in the translation unit scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 26.64 seconds, ticks: 51849408, micro-seconds/tick: 0.514 wall clock time: 8 minutes 5.55 seconds (485.55 seconds total) make: *** [default] Error 2 OS X Lion machine Graemes-MacBook-Pro:build graeme$ uname -a Darwin Graemes-MacBook-Pro.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 Graemes-MacBook-Pro:build graeme$ gcc --version gcc (GCC) 4.7.0 20120314 (prerelease) Copyright (C) 2012 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Have I missed something? Nightly builds seem to be OK. Commands ../../cctbx_plus/base/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ../sources/cctbx_project/libtbx/configure.py --compiler=gcc labelit cctbx bpcx_regression rstbx . setpaths.sh make Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 8

dropping Python 2.4 support?
by Nathaniel Echols 27 Sep '12

27 Sep '12

Hi all-- Will it ruin anyone's work if we make Python 2.5 the minimum platform for using CCTBX? 2.4 is now almost eight years old and the lack of certain newer features (decorators, 'with') is starting to hold us back. thanks, Nat

2 2

Enabling OpenMP for phaser on OS X
by William G. Scott 18 Sep '12

18 Sep '12

Hi folks: I found if I created a symbolic link from a g++4.7 compiler to /usr/bin/g++, I could "trick" phaser into compiling with OpenMP enabled. This is kind of an ugly hack. How do I force it to use /sw/bin/g++-4 ? Is there some place in the source code for cctbx (or phaser) that I can make such a change? Thanks. Bil William G. Scott Professor Department of Chemistry and Biochemistry and The Center for the Molecular Biology of RNA 228 Sinsheimer Laboratories University of California at Santa Cruz Santa Cruz, California 95064 USA

2 4

Quick start info?
by esserlo＠helix.nih.gov 12 Sep '12

12 Sep '12

Hi all, this is a question about how to get started quickly. I have looked at the sourceforge site for cctbx and read diverse documents but so far I have not found documentation that would help me get started fast. I have a crystallographic problem which I can probably program from scratch but then again why bother if there is a crystallographic tool box around. But well despite all the power in it getting it to work seems a bit daunting. Are there really good examples? I did get the "sand box quartz model" script to work but this is still far off of what I want to do. Thanks for all advice and sorry if I missed the 'obvious' one. Lothar

4 4

Question about sigma(I) calculation in merge_equivalents
by Keitaro Yamashita 07 Sep '12

07 Sep '12

Dear CCTBX developers, I have a question about the method in CCTBX to calculate sigma(averaged I) when merging. In cctbx/miller/merge_equivalents.h (line 594 in phenix-dev-1148), sigma(averaged I)^2 will be calculated by max(mv.gsl_stats_wvariance()/values.size(), 1/mv.sum_weights()). It takes maximum value of the two options. I know the latter equation - this is just sigma(averaged I)^2 = 1/sum(w), where w = 1/sigma(I)^2. And I believe this equation is widely used in other crystallographic programs e.g. XDS. The former is, according to the comment in the code, the emulation of gsl_stats_wvariance, and we can find the information from gsl website: http://www.gnu.org/software/gsl/manual/html_node/Weighted-Samples.html I don't know how this "wvariance" is derived. Is it a better estimate of sigma(averaged I)? Why does it take the maximum from the two options? I would like to know the theory of this sigma calculation in CCTBX. Best regards, Keitaro

3 5

Debian Package
by Radostan Riedel 05 Sep '12

05 Sep '12

Dear CCTBX Maintainers, Maybe some of you remember me last year asking on this mailing list[1]. I was asked by the Debian Science Team[2] to contribute my former work to get cctbx finally packaged in Debian. The last month I was working with Baptiste Carvello and Picca Frédéric-Emmanuel to get this done. Our packaging efforts + patches can be found in our git[3]. We organize our packaging efforts in a wiki[4]. But we don't want to overlook you and give some of our work back into the main project. We have a few patches to give back upstream that would really help us and hopefully help to increase the cctbx user community. To explain a little bit what we've done. First of all we tried respectfully not to change too much about your build system. We integrated a few options in libtbx/env_config which shouldn't break anything. The primary changes are: * Option to check and use system libraries (in case any library is already installed no need to compile the bundled shlib) * Install targets for headers and shlibs and binaries: calling "scons install" * Shlib versioning: This is achieved by utilizing libtool (the GNU standard) * --install-prefix, --install-destdir and --libdir option * support for LD_FLAGS in --use_environment_flags option * Fallback to avoid using "LIBTBX_BUILD" env variable after install * Full integration of your scons build system in distutils: setup.py and sconsutils.py. You can just call 'python setup.py install' * Adaption of your test system to be called by distutils * Autogenerate pkg-config files for shlibs What we now need to know is if our changes are appreciated or if we need to spend more work on this. Keep in mind that our work is still work in progress! One big concern is always the ABI/API stability. But if we can count on the shlib versioning as mentioned here[5] it would help us and other developers a lot. Libtool should be available on most systems, works with common compilers and it provides the quasi standard for versioning shlibs. We have other issues too but I guess I won't overkill this mail to much for now. I hope we can discuss and work on this together. Please tell us your honest opinion and critics. kind regards Radostan Riedel P.S: This work is intended to work on other distributions too. We also hope that Fedora, openSUSE, Gentoo etc. can benefit from it too. [1] http://phenix-online.org/pipermail/cctbxbb/2011-June/000192.html [2] http://wiki.debian.org/DebianScience [3] http://anonscm.debian.org/gitweb/?p=debian-science/packages/cctbx.git;a=sum… [4] http://wiki.debian.org/DebianScience/cctbx [5] http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.…

12 84

Boost.Extension plugins in CCTBX
by Graeme.Winter＠diamond.ac.uk 21 Aug '12

21 Aug '12

Hi Folks, It seems that Boost includes (or may include) a mechanism to allow compiled plugins see for example http://boost-extension.redshoelace.com/docs/boost/extension/boost_extension… - is this in cctbx? is anyone interested in it being available? It seems useful from where I am sat. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Removal of Boost.Thread support
by Luc Bourhis 17 Aug '12

17 Aug '12

Hi fellow cctbx developers, I am going to ditch support for Boost.Thread and therefore to remove the SF computation parallelised with Boost.Thread as well. Parallelisation at the C++ level should be done with OpenMP: Boost.Thread was merely an early experimentation that was kept alive for no particularly good reasons. If anybody disagrees with that move, please voice your opinion now. Best wishes, Luc J. Bourhis

6 15

single or double precision ?
by Pascal 17 Aug '12

17 Aug '12

Hi, Is cctbx doing all the calculations in double or single precision? I am intersted in the direct method of the structure factors calculation. >From what I saw it's double precision but I am not sure. Regards, Pascal

2 2

Boost.Extension plugins in CCTBX
by Graeme.Winter＠diamond.ac.uk 16 Aug '12

16 Aug '12

Hi Folks, It seems that Boost includes (or may include) a mechanism to allow compiled plugins see for example http://boost-extension.redshoelace.com/docs/boost/extension/boost_extension… - is this in cctbx? is anyone interested in it being available? It seems useful from where I am sat. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

from __future__ import division has become compulsory
by Luc Bourhis 15 Aug '12

15 Aug '12

Dear cctbx developers, as discussed here a few days ago, it has been decided that every single Python module must feature from __future__ import division A mega commit of mine has just taken care of existing code. For future modules, libtbx.find_clutter will complain if that line is missing. A script libtbx.add_from_future_import_division is provided that can add that line correctly. The rationale is that: i. as you know, we want to use the so-called new division semantics in cctbx modules (i.e. 1/2 == 0.5 instead of 1/2 == 0 as per the old semantics for the record) ii. the current solution passing -Qnew on the Python command line in the dispatchers libtbx.python, cctbx.python, etc is not always appropriate as there are people out there using cctbx as yet-another-library in a context where some code would not correctly run with the new division semantics, therefore barring the use of -Qnew The only solution to reconcile i. and ii. is therefore to add that __future__ line in every single Python module committed to the cctbx. Note however that the aforementioned dispatchers do still use -Qnew so that Phenix developers do not need to put that very line in their Phenix code. Hopefully, we can make everybody reasonably happy with those compromises. Best wishes, Luc

1 0

Deprecation warning?
by Graeme.Winter＠diamond.ac.uk 09 Aug '12

09 Aug '12

Is there a way to avoid this? Using matplotlib.pyplot. /Users/graeme/svn/cctbx_plus/base/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/backends/backend_wx.py:1382: wxPyDeprecationWarning: Using deprecated class PySimpleApp. wxapp = wx.PySimpleApp() Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Problems compiling cctbx on OS X (10.8.0)
by William G. Scott 04 Aug '12

04 Aug '12

Hi folks: I am having the same problem compiling cctbx stand-alone, or within CCP4. I've posted the error log here: http://fennario.ucsc.edu/~wgscott/temp/cctbx_compile_errors.log Any suggestions what to fix? Thanks. Bill William G. Scott Professor Department of Chemistry and Biochemistry and The Center for the Molecular Biology of RNA 228 Sinsheimer Laboratories University of California at Santa Cruz Santa Cruz, California 95064 USA

7 17

Invert hand of shelxd res file from cctbx
by Graeme.Winter＠diamond.ac.uk 23 Jul '12

23 Jul '12

Hi Folks, This should be easy but I can't find anything! I have a .res file: [gw56@ws050 fast_ep]$ cat sad_fa.res REM Best SHELXD solution: CC 41.11 CC(weak) 23.51 CFOM 64.62 REM TITL sad_fa.ins SAD in P422 CELL 0.98000 78.17 78.17 37.87 90.00 90.00 90.00 LATT -1 SYMM -Y+1/2,X+1/2,Z+1/4 SYMM -X,-Y,Z+1/2 SYMM Y+1/2,-X+1/2,Z+3/4 SYMM X+1/2,-Y+1/2,-Z+3/4 SYMM Y,X,-Z SYMM -X+1/2,Y+1/2,-Z+1/4 SYMM -Y,-X,-Z+1/2 SFAC SE UNIT 256 SE01 1 0.600227 0.892868 0.042187 1.0000 0.2 SE02 1 0.627518 0.852661 0.427392 0.8667 0.2 SE03 1 0.867409 0.911667 0.394418 0.8420 0.2 SE04 1 0.489761 0.805634 0.391009 0.7279 0.2 SE05 1 0.642792 0.649048 0.369694 0.7172 0.2 SE06 1 0.678879 0.535027 0.477918 0.4571 0.2 SE07 1 0.609207 0.572754 0.421746 0.4212 0.2 SE08 1 0.584526 0.536499 0.322047 0.4154 0.2 SE09 1 0.843063 0.846794 0.480296 0.3831 0.2 SE10 1 0.622429 0.743332 0.157899 0.3604 0.2 SE11 1 0.626114 0.528595 0.644789 0.3590 0.2 HKLF 3 END which I would like to read, invert the hand of and write as a pdb file. I got stuck trying to work through the example / test code. Can someone point me in the right direction? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 4

rt_mx inconsistencies
by Luc Bourhis 19 Jul '12

19 Jul '12

Hi fellow cctbx developers, class rt_mx, contrarily to the documentation, does not only hold rotation + translation but any transformation y = M x + t where M is an invertible matrix. As a result, rt_mx can be used to store both space group elements and change-of-basis operators (the latter are in general not rotations). Not the purest design but apart from the recurrent need of improving on the documentation, it has been all right. (1) However, revision 15082 has introduced an inconsistency in the comparison of rt_mx instances related to that divide between rotations and non-rotations. Let's r1 and r2 be rotations and m1 and m2 be non-rotations. After that revision, r1 == r2 works all right r1 == m1 throws an exception m1 == m2 throws an exception As far as I know the problem has been first reported by Phil Evans who provided a very clear regression test. The reason is simple: that revision changed operator== to compare the geometric elements, assuming the compared operators are rotations (and therefore that they have an invariant axis or plane). (2) Another side effect of revision 15082 seems to result in r1 == r2 being true even if r1.t() and r2.t() do not have the same denominator. Before that revision, this comparison used to return false. As part of that revision, one test has been changed from != to == for that very reason. (3) revision 15082 follows revision 14990 that introduces an order on rt_mx intances. That order is used to implement a new symmetry-equivalent pair interaction table and rightly so op1 == op2 is made consistent with op1 < op2 and op1 <= op2 by revision 15082. So I am coming to you to discuss what to do. I am of the opinion that (1) needs fixing. I am also of the opinion that (2) is a good move. Here is what I propose to do: (i) Remove operator <=, >=, <, > from class rt_mx and revert operator== to what it used to be. (ii) Change rot_mx and tr_vec operator== to disregard the value of den() (iii) Introduce a proxy class that wraps around rt_mx and that obeys the ordering introduced by revision 14990 (iv) Rewrite the implementation of the symmetry-equivalent pair interaction table generation to use that proxy instead of directly comparing space group symmetry elements. Basically I propose a very orthogonal design that keeps the core sgtbx pristine except for the fixing of a small but long-standing glitch while encapsulating the feature needed by one single algorithm along with that algorithm. What do you think? Best wishes, Luc Bourhis

3 8

cctbx.miller.complete_array() bug?
by Jan Marten Simons 13 Jul '12

13 Jul '12

Hi, I think I found a bug or missing feature in cctbx.miller.complete_array: As I suppose the complete_array function should check if a set of reflections is complete and add missing indices with the specified values. The bug is now when a set contains symmetrically equivalent reflections e.g. in P4 (1, 0, 2) and (0, 1, 2). The sample code below generates this output for the atached data file: --- f_obs read in: Miller array info: None Observation type: xray.amplitude Type of data: double, size=8 Type of sigmas: None Number of Miller indices: 8 Anomalous flag: False Unit cell: (4, 4, 4.44, 90, 90, 90) Space group: P 4 (No. 75) Systematic absences: 0 Centric reflections: 3 Resolution range: 4.44 1.94109 Completeness in resolution range: 1 Completeness with d_max=infinity: 1 (0, 0, 1) 4.10852771683 (0, 1, 0) 4.34856298103 (0, 1, 1) 10.0 (1, 1, 0) 0.911043357914 (1, 1, 1) 9.12523972288 (0, 0, 2) 5.97578446733 (0, 2, 0) 7.10492786733 (1, 0, 2) 2.17485631709 --- f_obs completed: Miller array info: None Observation type: xray.amplitude Type of data: double, size=9 Type of sigmas: None Number of Miller indices: 9 Anomalous flag: False Unit cell: (4, 4, 4.44, 90, 90, 90) Space group: P 4 (No. 75) Systematic absences: 0 Centric reflections: 3 Resolution range: 4.44 1.94109 (0, 0, 1) 4.10852771683 (0, 1, 0) 4.34856298103 (0, 1, 1) 10.0 (1, 1, 0) 0.911043357914 (1, 1, 1) 9.12523972288 (0, 0, 2) 5.97578446733 (0, 2, 0) 7.10492786733 (1, 0, 2) 2.17485631709 (0, 1, 2) 0.0 ---- sample code ------- obs_file = "Model02.hkl" xtal_symm = crystal.symmetry( unit_cell=(4.000, 4.0, 4.44, 90.0, 90.0, 90.0), space_group_symbol="P 4") # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("intensities="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() I_obs.merge_equivalents() f_obs = I_obs.as_amplitude_array() print("\n--- f_obs read in:") f_obs.show_comprehensive_summary() f_obs.show_array() d_max, d_min = f_obs.resolution_range() # add missing reflections with 0.0 intensity (= not observed) f_obs = f_obs.complete_array(d_min=d_min, d_max=d_max, new_data_value=0.0, new_sigmas_value=1.0).sort() print("\n--- f_obs completed:") f_obs.show_comprehensive_summary() f_obs.show_array() ---- end sample code ------- As you can see the index (0, 1, 2) is added with zero amplitude where instead the index (1, 0, 2) should be renamed to (0, 1, 2) as those two are symmetrically equivalent for any tetragonal spacegroup (like P4). So to me it looks like the complete_array method does not honor symmetrically equivalent indices at all, where it should imho. What do you think on this issue? With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

7 8

Getting r1_factor of miller arrays with different count of miller indices
by Jan Marten Simons 07 Jul '12

07 Jul '12

Hi, I'm facing a new challenge while working with cctbx: Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem: # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2") trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)])) # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array() f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents() R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False) gives: " assert other.indices().size() == self.indices().size() AssertionError " Is there some way to get the R1_factor for this kind of scenario? Cheers and thanks in advance, Jan

4 8

Fast way to enable/disable individual scatterers in structure factor calculation
by Jan Marten Simons 06 Jul '12

06 Jul '12

Hi, I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation. As I see from the source (structure_factors_direct.h) the selection toggle does not seem to be evaluated for this. Of course an additional check for the selection flag would harm performance in most standard cases, so I'd rather like to have a separate c++ function for this, which could be swapped in for the normal f_calc calculation by a boolean toggle in the python part of f_calc. As I'm not that much into the c++ part of cctbx, I'd really appreciate if somebody more knowledgeable could monkey patch this. Cheers, Jan

2 5

angle derivative of rotation matrix
by David Waterman 28 Jun '12

28 Jun '12

Hi, I'm using the axis_and_angle_as_r3_rotation_matrix method of scitbx.matrix.col in some Python code. Now I need to calculate the first derivative of the rotation matrix with respect to the rotation angle. Is there a function to do that somewhere in scitbx (or elsewhere in cctbx)? I had a brief look around but couldn't find anything that looked hopeful. If there isn't something there already I have some Fortran code from David Thomas that I could adapt to do this. Of course it wouldn't be very efficient in pure Python, and calculating the matrix and its derivative separately, but that's okay for now as the code is just for testing out some ideas. By the way, links to http://skal.planet-d.net/demo/matrixfaq.htm in the documentation (e.g. here<http://cctbx.sourceforge.net/current/c_plus_plus/namespacescitbx_1_1math_1_…>) are broken. I found a document with the same name here: http://vamos.sourceforge.net/matrixfaq.htm, but I have no idea if this has the same content as the original. Cheers -- David

3 4

Problem with direct_space_asu
by Jan Marten Simons 27 Jun '12

27 Jun '12

Hi, as there seems to be no documentation on the direct_space_asu module I just tried to use it like I thought the function names imply. The task I want to accomplish is checking whether a point in direct space is inside the asymmetric unit of a given unit cell. A small testcase I wrote reveals that apparently .is_inside(point) only uses fractional coordinates instead of cartesian coordinates. Is there a nice/easy way to get the desired behaviour? # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex import numpy as np asu = xray.structure( crystal_symmetry=crystal.symmetry( unit_cell=(5.01,5.01,5.47,95,85,90), space_group_symbol="P 1")).direct_space_asu() print(asu.show_comprehensive_summary()) print(asu.is_inside((0.0, 0.0, 0.0))) #should be True OK print(asu.is_inside((0.5, 0.5, 0.5))) #should be True OK print(asu.is_inside((0.9, 0.9, 0.9))) #should be True OK print(asu.is_inside((1.0, 0.9, 0.9))) #should be True is False! print(asu.is_inside((1.0, 1.0, 1.0))) #should be True is False! print(asu.is_inside((2.0, 2.0, 2.0))) #should be True is False! print(asu.is_inside((3.0, 3.0, 3.0))) #should be True is False! print(asu.is_inside((4.0, 4.0, 4.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 5.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 6.0))) #should be False OK print(asu.is_inside((6.0, 6.0, 6.0))) #should be False OK Cheers, Jan

2 1

compilation errors under gcc-4.7
by Justin Lecher 19 Jun '12

19 Jun '12

Hi, when compiling cctbx with gcc-4.7 I see following errors which I am not able to fix: cctbx_sources/scitbx/math/boost_python/distributions.cpp:112:59: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:86:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp: In instantiation of 'static void scitbx::math::{anonymous}::non_member_function_wrappers<FloatType, Distribution>::wrap() [with FloatType = double; Distribution = boost::math::students_t_distribution<double>]': cctbx_sources/scitbx/math/boost_python/distributions.cpp:116:63: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:85:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information Any suggestions to fix? Thanks, justin -- Justin Lecher Institute of Complex Systems ICS-6 Structural Biochemistry Research Centre Juelich 52425 Juelich, Germany phone: +49 2461 61 2117

5 8

Re: [cctbxbb] compilation errors under gcc-4.7
by Luc Bourhis 19 Jun '12

19 Jun '12

> I take it you're interested in fixing this? Yes. I've got a Fedora 17 virtual machine. Luc

1 0

proposal to remove iotbx.shelx.from_ins
by Mirko Scholz 12 Jun '12

12 Jun '12

Hello, since I am new here it seems an odd proposal, but could you not remove iotbx.shelx.from_ins.from_ins since its functionality is covered much better by xray.structure.from_shelx (at least it seems to me). The following is roughly identical with the first statement being inaccurate (anisotropic vs isotropic displacements parameters) and wrong (structures containing disorder) xs0 = iotbx.shelx.from_ins.from_ins(file_name='foo.ins') fc0 = xs0.structure_factors(d_min=0.7, algorithm='direct').f_calc() xs1 = cctbx.xray.structure.from_shelx(file=open('foo.ins')) fc1 = xs1.structure_factors(d_min=0.7, algorithm='direct').f_calc() As someone who (seldom) uses cctbx I stumbled first over iotbx.shelx.from_ins and only noticed after a while that it does not handle disorder properly (problem is in ins_record.if_10_added). from_ins is used only here ./iotbx/command_line/emma.py ./cctbx/adp_restraints/tst_ext.py Where it should be possible to substitute it though I have not tested it. Cordially Mirko Scholz

4 4

controlling types over the C++/Python interface
by David Waterman 16 May '12

16 May '12

Hi, I'm starting to explore the use of boost.python in cctbx. I've written a simple C++ class and exposed its accessor methods as attributes using the .add_property syntax in the wrapper. In the C++ class my member data and accessor return types are, for example, scitbx::vec3<double>. When I access the attribute in Python with the usual dot operator I get a tuple. What I'd really like though is an object of type <class 'scitbx.matrix.col'>, which is what I passed to the constructor of the object in Python. So the conversion from Python to C++ does what I want, but not the other way round. How do I take control of the type of the attribute in Python, without having to explicitly construct a scitbx.matrix.col from the tuple? It's the same case if I use .def and expose the accessor as a method instead of an attribute. I admit I don't fully understand the details of conversions in either direction. I found I had to add from cctbx.array_family import flex to the package __init__.py where my C++ extension is imported, so that I could pass scitbx.matrix.col objects (or scitbx.matrix.rec, or tuples) to the constructor in Python and not get a Boost.Python.ArgumentError. I've not yet figured out what this import is doing though. Apologies for the remedial question. These are still early days for me. Thanks -- David

5 12

Niggli-reduced cell C++ implementation
by Martin Uhrin 25 Apr '12

25 Apr '12

Dear Cctbx community, Firstly I'd like to say thank you to Rolf, Nicholas and Paul for their expertly thought through implementation of the reduced cell algorithm. I've found it to be extremely useful for my work. My code is all in C++ and I'd like to be able to use the Krivy-Gruber algorithm. My understanding is that only the reduced (Buerger) unit cell algorithm is implemented in C++ [1] which guarantees shortest lengths but not unique angles. From my understanding the Krivy-Gruber would also guarantee me uniqueness of unit cell angles, however this is only implemented in Python [2]. Sorry to be so verbose, I just wanted to check that I was on the right page. Would it be possible for me to implement the Krivy-Gruber in C++ by adding in the epsilon_relative to the parameter and following the procedure found in the python version? Many thanks, -Martin [1] http://cctbx.sourceforge.net/current/c_plus_plus/classcctbx_1_1uctbx_1_1fas… [2] http://cctbx.sourceforge.net/current/python/cctbx.uctbx.krivy_gruber_1976.h… -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

3 8

Handling unmerged reflections from MTZ
by Graeme.Winter＠Diamond.ac.uk 25 Apr '12

25 Apr '12

Hi Folks, I have been writing very bad Python code to handle unmerged MTZ files using CCTBX - in essence copying all of the measurements to lists for each Miller index. Inefficient. Has anyone looked a neater way of handling these files perhaps with doing the "hard work" from C++? Reading the file is very easy, but managing the data is harder. Essentially I would like to be able to read in a scaled-summed-but-unmerged MTZ file (equivalent to unmerged scale pack no-merge-original-index) and get back all observations of a given HKL, which I can then subsequently analyse… I could crack on and write something but it strikes me as a good idea to ask first. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Branches
by Graeme.Winter＠Diamond.ac.uk 23 Apr '12

23 Apr '12

Hi Folks, Quick question - does anyone ever use branches with the CCTBX repository? Just at the moment I'm up to a few things which I would like to be able to commit, but I don't want to break the existing code... however as a project it does not seem CCTBX uses branches much... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 9

Type checking in Phil...
by Graeme.Winter＠Diamond.ac.uk 11 Apr '12

11 Apr '12

… is there any? This does not give an error: from libtbx.phil import parse working_phil = parse(""" parameter { toy = 10 .type = int } """) if __name__ == '__main__': working_phil.show() # this should crash right? working_phil = working_phil.fetch(parse('parameter.toy=foo')) working_phil.show() Output: cctbx.python toy_phil.py parameter { toy = 10 } parameter { toy = foo } Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Build on OS X Lion
by Graeme.Winter＠Diamond.ac.uk 24 Feb '12

24 Feb '12

Hi Folks, Just ran the cctbx_svn_getting_started script to do a build on a new lion system and I got this error - had anyone else got this working correctly on such a system? Template definition not found is a little beyond my usual C++ debugging skills. Many thanks, Graeme c++ -o scitbx/math/boost_python/tensor_rank_2.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/tensor_rank_2.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:339:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:119:4: note: in instantiation of function template specialization 'boost::math::detail::bessel_jy<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here bessel_jy(v, x, &j, &y, need_j, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:165:9: note: in instantiation of function template specialization 'boost::math::detail::cyl_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here * cyl_bessel_j_imp(T(T(n)+T(0.5f)), x, bessel_no_int_tag(), pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:449:65: note: in instantiation of function template specialization 'boost::math::detail::sph_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return policies::checked_narrowing_cast<result_type, Policy>(detail:... ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:455:11: note: in instantiation of function template specialization 'boost::math::sph_bessel<double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return sph_bessel(v, x, policies::policy<>()); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:291:13: note: in instantiation of function template specialization 'boost::math::sph_bessel<double>' requested here return( sph_bessel(l,x) ); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:363:9: note: in instantiation of function template specialization 'scitbx::math::bessel::spherical_bessel<double>' requested here bessel::spherical_bessel); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:16: error: call to function 'bessel_y_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_y_small_z_series(v, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'bessel_y_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_y_small_z_series(T v, T x, T* pscale, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:16: error: call to function 'bessel_yn_small_z' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_yn_small_z(n, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'bessel_yn_small_z' should be declared prior to the call site or in namespace 'boost::math::policies' T bessel_yn_small_z(int n, T z, T* scale, const Policy& pol) ^ 4 errors generated. c++ -o scitbx/math/boost_python/chebyshev.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/chebyshev.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 23.10 seconds wall clock time: 3 minutes 39.02 seconds (219.02 seconds total) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6

Comparison of Crystal Structures
by David Lonie 14 Jan '12

14 Jan '12

Hi list, I have written an open-source (BSD-licensed) algorithm to compare two crystal structure descriptions and identify whether they describe the same underlying structure. The algorithm is capable of detecting degenerate descriptions (meaning unit cell vectors + atomic positions and types) even in the presence of the "real-world" issues confronting computational crystallographers (choice of unit cell representation and offset, rotations, reflections, and numerical noise). The algorithm returns a boolean match using customizable tolerances. I just wanted to inform the mailing list of this, in case such functionality is needed by your user community. The code, XtalComp, is written in C++ and is easy to interface with, having no external dependencies. The liberal license would allow it to be included with the cctbx sources and interfaced with cctbx code if desired. The code uses the CMake build system, but the simple nature of the sources would allow trivial porting to SCons. There is a web interface to the code here: http://xtalopt.openmolecules.net/xtalcomp/xtalcomp.html Also on that page are links to the journal article describing the algorithm, as well as a free download of the preprint version of the article. The code is available on Github (http://www.github.com/dlonie/XtalComp). Thanks for your time, and I hope that someone out there will find this code useful :-) Keep up the good work! Dave

2 1

Attitude of rotation, Python to C++
by Graeme.Winter＠Diamond.ac.uk 13 Jan '12

13 Jan '12

Hi Folks, Just started moving code from Python to C++ - and seeing a 40x speedup which is nice - however I noticed something odd. scitbx.matrix.col.rotate(axis, angle) has the reversed attitude to scitbx::vec3<double>.rotate(scitbx::vec3<double>, double angle) viz: this python code s = (self._ub * hkl).rotate(self._axis, angle) q = (s + self._s0).normalize() # check if diffracted ray parallel to detector face q_dot_n = q.dot(detector_normal) if q_dot_n == 0: continue r = (q * distance / q_dot_n) - self._detector_origin x = r.dot(self._detector_fast) y = r.dot(self._detector_slow) if x < self._limits[0] or y < self._limits[2]: continue if x > self._limits[1] or y > self._limits[3]: continue observed_reflection_positions.append((hkl, x, y, angle)) gives the same result as scitbx::vec3<double> s, q, r; double x, y, q_dot_n; /* this is rather weird - the attitude of rotation is back to front in the c++ code */ s = (ub * hkl).rotate(axis, - angle); q = (s + s0).normalize(); q_dot_n = q * normal; if (q_dot_n == 0) return false; r = (q * distance / q_dot_n) - origin; x = r * fast; y = r * slow; if (x < limits[0]) return false; if (y < limits[2]) return false; if (x > limits[1]) return false; if (y > limits[3]) return false; prediction[0] = x; prediction[1] = y; where I had to use -angle. Code is linked into same routines - used identically. With -angle I get numerically identical results (which is nice) Anyone got any clues on this one? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

How to use the binner in cctbx.miller?
by Jan Marten Simons 04 Jan '12

04 Jan '12

Hi, as documentation is lacking on this: How can I use the binner from cctbx.miller to bin reflections into different groups according to their d-value and would it be possible afterwards to get integral intesities of each binned group easily? Thanks, Jan

3 3

Inconsistency between numpy and flex (sclice notation)
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, I've found a minor inconveniance using the flex type: test case: import numpy as np np_grid=np.double(np.zeros((3,7))) np_grid[1,3]=3.0 print(np_grid) # works print(np_grid[1,:]) # works from cctbx.array_family import flex grid=flex.double(flex.grid(3,7)) grid[1,3]=3.0 flex.show(grid) #works flex.show(grid[1,:]) #gives error (see below) "TypeError: All items must be of same type." Is there another simple way to do sclices with flex arrays or could this be implemented the same way as in numpy by someone who is more into c++ than myself? Thanks, Jan

3 5

Determining crystal space group using C++ libraries
by Martin Uhrin 30 Dec '11

30 Dec '11

Dear Cctbx community, I'm trying to determine the space group for my crystal structures using the C++ libraries. I haven't found a way to determine the full space group (i.e. lattice + atomic sites), does anyone have any experience of doing this with cctbx? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 1

iotbx::pdb and boost::unit_test::framework
by vanv0059＠umn.edu 15 Dec '11

15 Dec '11

Hi, Is it reasonable to assume that the boost unit test framework (boost::test) is compatible with cctbx's iotbx::pdb? Or might boost::test's handling of memory clash with that of cctbx? I can get simple examples with iotbx::pdb::input by itself or a print statement wrapped by boost test to work fine on Linux (gcc 4.1.2). What I am having issues with is replacing the print statement & include file with iotbx::pdb::input (even if no file is opened). If the answer to the above question is: "it is reasonable", then would it be better to post my question to the c++ sig group? The simple example is: #define BOOST_TEST_DYN_LINK #define BOOST_TEST_MODULE test_iotbx_pdb_input #include <boost/test/unit_test.hpp> #include <iotbx/pdb/input.h> BOOST_AUTO_TEST_CASE(test_iotbx_pdb_input) { iotbx::pdb::input p; } Both gdb and valgrind complain about the example program trying to access addr 0x1 gdb traceback: #0 0x00002aaaab74be75 in free () from /lib64/libc.so.6 #1 0x00002aaaaad1a768 in boost::checked_array_delete<char> ( x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:41 #2 0x00002aaaaad1a783 in boost::checked_array_deleter<char>::operator() ( this=0x5103e8, x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:63 #3 0x00002aaaaad1a8b1 in boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose (this=0x5103d0) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/smart_ptr/detail/sp_counted_impl.hpp:148 #4 0x00002aaaab028d78 in boost::runtime::cla::parser::parse(int&, char**) () from /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/stage/lib/libboost_unit_test_framework.so.1.48.0 #5 0x0000000000000000 in ?? () valgrind note: ==26968== Invalid free() / delete / delete[] / realloc() ==26968== at 0x4A20E72: operator delete[](void*) (vg_replace_malloc.c:490) ==26968== by 0x4C79767: void boost::checked_array_delete<char>(char*) (checked_delete.hpp:41) ==26968== by 0x4C79782: boost::checked_array_deleter<char>::operator()(char*) const (checked_delete.hpp:63) ==26968== by 0x4C798B0: boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose() (sp_counted_impl.hpp:148) ==26968== by 0x4F86D77: boost::runtime::cla::parser::parse(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F65E6F: boost::unit_test::runtime_config::init(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libb oost_unit_test_framework.so.1.48.0) ==26968== by 0x4F455E0: boost::unit_test::framework::init(bool (*)(), int, ch ar**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/li b/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F59D7E: boost::unit_test::unit_test_main(bool (*)(), int, cha r**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib /libboost_unit_test_framework.so.1.48.0) ==26968== by 0x407377: main (unit_test.hpp:59) ==26968== Address 0x1 is not stack'd, malloc'd or (recently) free'd

2 7

Using iotbx.pdb as C++ only dynamic (shared) library
by vanv0059＠umn.edu 13 Dec '11

13 Dec '11

Hi, The iotbx.pdb Python module is very useful and fast. I am trying to use the C++ library and have gotten an example program to run correctly when compiling in the object files in cctbx_build/iotbx/pdb: atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o However, I would like to have a shared library that wraps all of the objects into shared library (or dynamic on Mac OSX). I have created such a library on my Mac using the command: g++ -dynamiclib -fPIC -O3 --fast-math -o libiotbx_pdb.dylib atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o -install_name /Users/jvanvoorst/Python_VEs/Lore30/lib/libiotbx_pdb.dylib -Wl,-single_module When I link my program to that library I always get a segment fault in the deconstructor of the last instance of iotbx::pdb::hierarchy::root::~root(). gdb traceback is: Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x000000000000001a 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () (gdb) where #0 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () #1 0x00000001000035fe in iotbx::pdb::hierarchy::root_data::~root_data () #2 0x0000000100003662 in boost::checked_delete<iotbx::pdb::hierarchy::root_data> () #3 0x0000000100003687 in boost::detail::sp_counted_impl_p<iotbx::pdb::hierarchy::root_data>::dispose () #4 0x0000000100001342 in boost::detail::sp_counted_base::release () #5 0x0000000100001372 in boost::detail::shared_count::~shared_count () #6 0x000000010000149b in boost::shared_ptr<iotbx::pdb::hierarchy::root_data>::~shared_ptr () #7 0x00000001000014e9 in iotbx::pdb::hierarchy::root::~root () #8 0x00000001000053c9 in FooBar::~FooBar () #9 0x0000000100001150 in main () I can send you my program but I have pruned FooBar down to only two class variables (one for a root object and the other for the input). All the class does is read the input from a file and construct the hierarchy. The seg fault only happens on the cleanup of the last object. That is if I vary the number of FooBar instances, the program faults on the cleanup of the last root object. I tried numerous combinations of flags to the linker (g++), but (as expected) the result is always the same. I am using "g++ -I{include for sources} -I{include for build} test.C libiotbx_pdb.dylib" to compile the program. I am likely overlooking something. Also, I am not proficient in debugging dynamic libraries via gdb. Therefore, any tips or pointers would be much appreciated. If there is an edit I can make to my Sconscript to automagically generate the shared library I could use that as well. The libtbx/configure.py output looks right (Static libraries: False). However, when running scons using -Q the source files are getting compiled with -fPIC (good), but the libraries are getting created using ar (implies static libraries) -- unless I am looking in the wrong directory for the shared libraries (cctbx_build/lib). Thanks, Jeff Van Voorst

2 8

Problems building scitbx.random
by Jan Marten Simons 29 Nov '11

29 Nov '11

Hi, After doing a fresh checkout with these commands: > #! /bin/csh -fe > set verbose > curl http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx -o cctbx_bundle.selfx > perl cctbx_bundle.selfx 0 > rm cctbx_bundle.selfx > rm cctbx_install_script.csh > mv cctbx_sources sources > cd sources > svn co https://cctbx.svn.sourceforge.net/svnroot/cctbx/trunk cctbx_project > cctbx_project/libtbx/development/move_obsolete_from_bundle.csh And using these commands to configue and build cctbx: > mkdir -p build > cd build > python ../sources/cctbx_project/libtbx/configure.py -- enable_openmp_if_possible=True --scan_boost --build=release boost_adaptbx cctbx chiltbx crys3d iotbx mmtbx rstbx ucif smtbx clipper_adaptbx fable scitbx > source setpaths.sh > make || ( echo "Make failed!"; exit 1 ) I encounter these errors: Processing: "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py" Traceback (most recent call last): File "../sources/cctbx_project/libtbx/configure.py", line 21, in <module> run() File "../sources/cctbx_project/libtbx/configure.py", line 17, in run libtbx.env_config.cold_start(sys.argv) File "/home/marten/cctbx/cctbx- dev/sources/cctbx_project/libtbx/env_config.py", line 2061, in cold_start env.refresh() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1401, in refresh module.process_libtbx_refresh_py() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1678, in process_libtbx_refresh_py execfile(abs(custom_refresh), {}, {"self": self}) File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py", line 3, in <module> from crys3d import qttbx File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/qttbx/__init__.py", line 4, in <module> import gltbx.util File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/gltbx/util.py", line 1, in <module> import scitbx.array_family.flex # import dependency File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/array_family/flex.py", line 10, in <module> import scitbx.random File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/random/__init__.py", line 24, in <module> mt19937 = mersenne_twister_19937(value=get_random_seed()) NameError: name 'mersenne_twister_19937' is not defined To me these look like scitbx.random-ext is not built. Is this a regression from the changes to the build system or am I doining something wrong here? Cheers, Jan

2 3

Using cctbx as a C++ only library
by Martin Uhrin 26 Nov '11

26 Nov '11

Dear Cctbx community, thank you very much for the great library, full of wonderful algorithms. I'm writing a structure searching engine in C++ to be used as a high throughput method to do energy landscape sampling with a view to finding new ground state crystal structures. Key requirements of this project are efficiency and ease of development & deployment. Cctbx has many useful algorithms and data structures that I would like to avoid having to re-code if possible. Thus I've downloaded and played with the source as it is but have found it difficult to integrate with my project (for understandably reasons given the goals of cctbx). My main goals regarding ease of development & deployment mean that it would be very inconvenient to distribute the entire cctbx with my code. Ideally I'd like to achieve the following: * strip out everything python related (apart from maybe the build system) * strip out all modules (namespaces) that are unused by my code leaving a fairly bare-bones C++ library that is quick and easy to build. Is anyone able to comment on how feasible this might be? Is this even worth attempting? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 3

cctbx + PyPy?
by Jan Marten Simons 25 Nov '11

25 Nov '11

Hi, PyPy 1.7 was released just a few days ago: http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html After taking a look at http://speed.pypy.org/ I'm wondering if it would be possible to get cctbx to work with PyPy-jit instead of CPython. Perhaps it's a little bit early, as they are still working on a complete numpy integration and I'm not sure how well boost.python fits into this. But in general speeding up the python parts looks nice if one does not have to invest too much time into this. Has any of you already done some tests with PyPy? Cheers, Jan

2 1

crystal packing/unit cell question
by Jessica Grant 08 Nov '11

08 Nov '11

I am new to cctbx and crystallography in general. I am trying to take a pdb file and produce a representation of the crystal, similar to the symmetry mates produced in pymol. I have used cctbx to apply the symmetry operators, but this gives atomic coordinates that are quite spread out. I think what I want to do is to fill the unit cell, or in some way extract crystal packing information. I would like, in the end, to be able to build files with a physically realistic representation of different sized crystals, so I can go on to study the interactions of molecules within those crystals. Is there a way to do this in cctbx? Thanks! Jessica

5 11

flex index
by Monarin Uervirojnangkoorn 18 Oct '11

18 Oct '11

Hi, I'm using flex and 'd like to get an access to elements fall beyond/ below certain value. eg. a=[1,4,5,7,5,6,6,1] anything beyond 5 should be set to 5. (in matlab then a(a>5)=5;) if you could tell me how to do this quickly for flex, that would be great. for now i use loop and that is... pretty slow. many thanks. mona ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^ Institute of Biochemistry, Institut für Neuro- und Bioinformatik, Graduate School for Computing in Medicine and Life Sciences University of Lübeck, Ratzeburger Allee 160 Lübeck 23538 Germany Tel: +49451-5004072 Fax: +49451-5004068

2 3

CCTBX comprehensive overview
by Graeme.Winter＠Diamond.ac.uk 12 Oct '11

12 Oct '11

Hi Folks, Is there somewhere a comprehensive overview of what is available within cctbx? I have had a request for this from someone who may be interested in using it, but does not want to start looking until they know what is in there, without inspecting any code. Specifically, an idea of what is available from Python would hit the spot. Is there a way to generate this or does it already exist on the web? Thanks in advance, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

cropping out a fragment
by Monarin Uervirojnangkoorn 04 Oct '11

04 Oct '11

Hi all, I'd like to crop out a portion of a fragment (say a helix) from real_map by specifying fractional coordinates. would be nice to know if there is any specific routine for this. (oh .. & ++ option like crop only 2 angstroms from the location of the atom would be really nice). thankx Best, Mona ~.~.~.~><~.~~.~.~. Institute of Biochemistry, Institut of Neuro- and Bioinformatics Graduate School for Computing in Medicine and Life Sciences University of Lübeck, Ratzeburger Allee 160 Lübeck 23538 Germany

2 1

Deleting batch from mtz batch header via iotbx.mtz
by Graeme.Winter＠Diamond.ac.uk 27 Sep '11

27 Sep '11

Hi Folks, Is there a way to delete a batch from an mtz file batch header? Been trying to do this and it looks like it is impossible - though you can add batches. Clearly there is the worry that the subsequent batch column may then refer to a missing batch, however... it would be handy to be able to do this (providing that the reflections were also removed elsewhere in the calling code) Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Lattice options & changes of basis
by Graeme.Winter＠Diamond.ac.uk 20 Sep '11

20 Sep '11

Hi Folks, I have a very simple question which I *know* can be done with cctbx, using the Python API, but I cannot for the life of me figure out how to do it. I have even read the code, it's just not clear. I have a Bravais lattice and cell, say oP and 20, 30, 40, 90, 90, 90 - what I would like back is a list of pointgroup, cell, reindexing operation for all compatible unique settings i.e. 1 x P1, 3 x P2, 1 x P222 - from a Python loop. At the moment I run iotbx.lattice symmetry and grep the results, which is not a great idea. It would also be useful to give the reindex operations to centred lattices. I have also stared at phenix.explore_metric_symmetry and remain puzzled despite my attempt to essentially duplicate the code. I just can't get the reindex operation out. Has anyone some simple code to do this? I tried piggybacking on pointgroup tools but that did not lead to enlightenment. I suspect using elementary ish operations would be better... Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Conversion from symmetry operation to matrix
by Graeme.Winter＠Diamond.ac.uk 13 Jul '11

13 Jul '11

Dear All, For a little while now I have been using some cctbx jiffy code to convert from symmetry operations to matrices to help with testing reindexing data. This works as def symop_to_mat(symop): return matrix.sqr(sgtbx.rt_mx( sgtbx.parse_string(symop)).as_double_array()[:9]).transpose().elems Which is plenty simple. However it just exploded when given "1/2*z,-y+1/2*z,x" - reasonably really as this is not a rotation. It is however a valid reindexing operation to go from C2 to P1. Does anyone have code which will handle this general transformation from a reindexing operation to a matrix, which will cope with the fact that the resulting matrix is not RT? I had a little jiffy I wrote using ccp4f lib (symop2mat - you can probably see it!) but I am trying to do away with things like that and do everything properly... Many thanks in advance, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 6

Bug in structure.expand_to_p1?
by Jan Marten Simons 22 Jun '11

22 Jun '11

Hi, I'm quite puzzled at the moment. using this code: if __name__ == '__main__': quartz_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=crystal.symmetry( unit_cell=(5.01,5.01,5.47,90,90,120), space_group_symbol="P6222")), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(1/2.,1/2.,1/3.), u=0.2), xray.scatterer( label="O", site=(0.197,-0.197,0.83333), u=0)])) ref_structure = quartz_structure.expand_to_p1( append_number_to_labels=True) print(ref_structure.as_py_code()) I get this result: xray.structure( crystal_symmetry=crystal.symmetry( unit_cell=(5.01, 5.01, 5.47, 90, 90, 120), space_group_symbol="P 1"), scatterers=flex.xray_scatterer([ xray.scatterer( #0 label="Si_0", site=(0.500000, 0.500000, 0.333333), u=0.200000), xray.scatterer( #1 label="Si_1", site=(0.000000, 0.500000, 0.666667), u=0.200000), xray.scatterer( #2 label="Si_2", site=(0.500000, 0.000000, 1.000000), u=0.200000), xray.scatterer( #3 label="O_0", site=(0.197000, -0.197000, 0.833333), u=0.000000), xray.scatterer( #4 label="O_1", site=(0.394000, 0.197000, 1.166667), u=0.000000), xray.scatterer( #5 label="O_2", site=(-0.197000, -0.394000, 1.500000), u=0.000000), xray.scatterer( #6 label="O_3", site=(0.197000, 0.394000, 1.500000), u=0.000000), xray.scatterer( #7 label="O_4", site=(-0.394000, -0.197000, 1.166667), u=0.000000), xray.scatterer( #8 label="O_5", site=(-0.197000, 0.197000, 0.833333), u=0.000000)])) It bothers me that there are x,y,z not in [0, 1[. Shouldn't the newly expanded cell only contain atoms within the unit cell? I realize, that the quarz structure (from the sample code from the website) also violates this with Oy beeing negative, but still I think expand_to_p1 should return a regular structure. Also is there a method already implemented to change a structure from a non- conventional description into a conventional one (with all atoms within its unit cell)? With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

3 4

CCTBX for Debian + PATCH
by Radostan Riedel 14 Jun '11

14 Jun '11

Hello, I'm about to make a (multi binary) Debian source package of CCTBX-Project. The plan is to build a few small Packages that depends on each other. So the python stuff would go to /usr/lib/python*.*/dist-packages/ and the libraries would go to /usr/lib and the Headers to /usr/include. Example: ''' python-iotbx: installs iotbx python module and extensions to /usr/lib/python*.*/dist-packages/ libcctbx0: installs libcctbx.so.0 etc. to /usr/lib/ libcctbx-dev: installs a symlink libcctbx.so ---> libcctbx.so.0 etc. and header files to /usr/include/cctbx ''' This would follow strictly the Debian Policy, so maybe when this will be accepted we could see CCTBX in the official Repos of Debian and Ubuntu etc. But for that I still have a few problems: 1.) The SONAME and SOVERSION is missing from the shared libraries. I'm attaching a patch that can fix this for GNU/Linux. That produces files like this: ''' ls lib/libcctbx* lib/libcctbx_sgtbx_asu.so lib/libcctbx_sgtbx_asu.so.0 lib/libcctbx.so lib/libcctbx.so.0 ''' 2.) PyCifRW: I had to remove it from the sources because it ain't free software because of some lines in their license. It wasn't accepted to Fedora because of that, so I guess I can forget Debian either. 3.) In your changelog I read you wanna get rid of the "from __future__ import division" and change to "python -Qnew". This would be a problem for me, because than I couldn't use it from sys.path anymore. I have a Django project at work that imports some cctbx modules directly from pythonpath. And I can't really call "cctbx.python" for that, otherwise I would have to patch my Apache and mod_wsgi, respectively cctbx. Wouldn't it be better to install all python modules to the standard python location of the Distribution than making dispatchers with hardcoded paths? Also, when the libraries, headers and python stuff would be in their standard location, everyone could just import it from anywhere? I will be finished soon with packaging and then I will upload it to my PPA on Lauchpad https://launchpad.net/~raybuntu/+archive/xray which is empty by now. And I think this could also be adopted to other Distributions like Fedora. I'd be glad to hear some critical opinions about this from you. regards R.R.

2 6

Best way to do 3d xyz plots with cctbx?
by Jan Marten Simons 08 Jun '11

08 Jun '11

Hi, I noticed cctbx has some custom wx based graphic windows to display various specialized plots and visualisations. I'm interested in generating some simple 3d xyz plots like in [1]. Is functionality for this included in cctbx or should I rather use another python based plotter for this? [1] http://pyx.sourceforge.net/examples/3dgraphs/surface.html Thanks in advance, Jan

2 1