- cctbxbb - phenix-online.org

Niggli-reduced cell C++ implementation
by Martin Uhrin 24 Apr '12

24 Apr '12

Dear Cctbx community, Firstly I'd like to say thank you to Rolf, Nicholas and Paul for their expertly thought through implementation of the reduced cell algorithm. I've found it to be extremely useful for my work. My code is all in C++ and I'd like to be able to use the Krivy-Gruber algorithm. My understanding is that only the reduced (Buerger) unit cell algorithm is implemented in C++ [1] which guarantees shortest lengths but not unique angles. From my understanding the Krivy-Gruber would also guarantee me uniqueness of unit cell angles, however this is only implemented in Python [2]. Sorry to be so verbose, I just wanted to check that I was on the right page. Would it be possible for me to implement the Krivy-Gruber in C++ by adding in the epsilon_relative to the parameter and following the procedure found in the python version? Many thanks, -Martin [1] http://cctbx.sourceforge.net/current/c_plus_plus/classcctbx_1_1uctbx_1_1fas… [2] http://cctbx.sourceforge.net/current/python/cctbx.uctbx.krivy_gruber_1976.h… -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

3 8

Handling unmerged reflections from MTZ
by Graeme.Winter＠Diamond.ac.uk 24 Apr '12

24 Apr '12

Hi Folks, I have been writing very bad Python code to handle unmerged MTZ files using CCTBX - in essence copying all of the measurements to lists for each Miller index. Inefficient. Has anyone looked a neater way of handling these files perhaps with doing the "hard work" from C++? Reading the file is very easy, but managing the data is harder. Essentially I would like to be able to read in a scaled-summed-but-unmerged MTZ file (equivalent to unmerged scale pack no-merge-original-index) and get back all observations of a given HKL, which I can then subsequently analyse… I could crack on and write something but it strikes me as a good idea to ask first. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Branches
by Graeme.Winter＠Diamond.ac.uk 22 Apr '12

22 Apr '12

Hi Folks, Quick question - does anyone ever use branches with the CCTBX repository? Just at the moment I'm up to a few things which I would like to be able to commit, but I don't want to break the existing code... however as a project it does not seem CCTBX uses branches much... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 9

Type checking in Phil...
by Graeme.Winter＠Diamond.ac.uk 11 Apr '12

11 Apr '12

… is there any? This does not give an error: from libtbx.phil import parse working_phil = parse(""" parameter { toy = 10 .type = int } """) if __name__ == '__main__': working_phil.show() # this should crash right? working_phil = working_phil.fetch(parse('parameter.toy=foo')) working_phil.show() Output: cctbx.python toy_phil.py parameter { toy = 10 } parameter { toy = foo } Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Build on OS X Lion
by Graeme.Winter＠Diamond.ac.uk 23 Feb '12

23 Feb '12

Hi Folks, Just ran the cctbx_svn_getting_started script to do a build on a new lion system and I got this error - had anyone else got this working correctly on such a system? Template definition not found is a little beyond my usual C++ debugging skills. Many thanks, Graeme c++ -o scitbx/math/boost_python/tensor_rank_2.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/tensor_rank_2.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:339:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:119:4: note: in instantiation of function template specialization 'boost::math::detail::bessel_jy<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here bessel_jy(v, x, &j, &y, need_j, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:165:9: note: in instantiation of function template specialization 'boost::math::detail::cyl_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here * cyl_bessel_j_imp(T(T(n)+T(0.5f)), x, bessel_no_int_tag(), pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:449:65: note: in instantiation of function template specialization 'boost::math::detail::sph_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return policies::checked_narrowing_cast<result_type, Policy>(detail:... ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:455:11: note: in instantiation of function template specialization 'boost::math::sph_bessel<double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return sph_bessel(v, x, policies::policy<>()); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:291:13: note: in instantiation of function template specialization 'boost::math::sph_bessel<double>' requested here return( sph_bessel(l,x) ); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:363:9: note: in instantiation of function template specialization 'scitbx::math::bessel::spherical_bessel<double>' requested here bessel::spherical_bessel); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:16: error: call to function 'bessel_y_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_y_small_z_series(v, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'bessel_y_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_y_small_z_series(T v, T x, T* pscale, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:16: error: call to function 'bessel_yn_small_z' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_yn_small_z(n, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'bessel_yn_small_z' should be declared prior to the call site or in namespace 'boost::math::policies' T bessel_yn_small_z(int n, T z, T* scale, const Policy& pol) ^ 4 errors generated. c++ -o scitbx/math/boost_python/chebyshev.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/chebyshev.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 23.10 seconds wall clock time: 3 minutes 39.02 seconds (219.02 seconds total) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6

Comparison of Crystal Structures
by David Lonie 13 Jan '12

13 Jan '12

Hi list, I have written an open-source (BSD-licensed) algorithm to compare two crystal structure descriptions and identify whether they describe the same underlying structure. The algorithm is capable of detecting degenerate descriptions (meaning unit cell vectors + atomic positions and types) even in the presence of the "real-world" issues confronting computational crystallographers (choice of unit cell representation and offset, rotations, reflections, and numerical noise). The algorithm returns a boolean match using customizable tolerances. I just wanted to inform the mailing list of this, in case such functionality is needed by your user community. The code, XtalComp, is written in C++ and is easy to interface with, having no external dependencies. The liberal license would allow it to be included with the cctbx sources and interfaced with cctbx code if desired. The code uses the CMake build system, but the simple nature of the sources would allow trivial porting to SCons. There is a web interface to the code here: http://xtalopt.openmolecules.net/xtalcomp/xtalcomp.html Also on that page are links to the journal article describing the algorithm, as well as a free download of the preprint version of the article. The code is available on Github (http://www.github.com/dlonie/XtalComp). Thanks for your time, and I hope that someone out there will find this code useful :-) Keep up the good work! Dave

2 1

Attitude of rotation, Python to C++
by Graeme.Winter＠Diamond.ac.uk 13 Jan '12

13 Jan '12

Hi Folks, Just started moving code from Python to C++ - and seeing a 40x speedup which is nice - however I noticed something odd. scitbx.matrix.col.rotate(axis, angle) has the reversed attitude to scitbx::vec3<double>.rotate(scitbx::vec3<double>, double angle) viz: this python code s = (self._ub * hkl).rotate(self._axis, angle) q = (s + self._s0).normalize() # check if diffracted ray parallel to detector face q_dot_n = q.dot(detector_normal) if q_dot_n == 0: continue r = (q * distance / q_dot_n) - self._detector_origin x = r.dot(self._detector_fast) y = r.dot(self._detector_slow) if x < self._limits[0] or y < self._limits[2]: continue if x > self._limits[1] or y > self._limits[3]: continue observed_reflection_positions.append((hkl, x, y, angle)) gives the same result as scitbx::vec3<double> s, q, r; double x, y, q_dot_n; /* this is rather weird - the attitude of rotation is back to front in the c++ code */ s = (ub * hkl).rotate(axis, - angle); q = (s + s0).normalize(); q_dot_n = q * normal; if (q_dot_n == 0) return false; r = (q * distance / q_dot_n) - origin; x = r * fast; y = r * slow; if (x < limits[0]) return false; if (y < limits[2]) return false; if (x > limits[1]) return false; if (y > limits[3]) return false; prediction[0] = x; prediction[1] = y; where I had to use -angle. Code is linked into same routines - used identically. With -angle I get numerically identical results (which is nice) Anyone got any clues on this one? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

How to use the binner in cctbx.miller?
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, as documentation is lacking on this: How can I use the binner from cctbx.miller to bin reflections into different groups according to their d-value and would it be possible afterwards to get integral intesities of each binned group easily? Thanks, Jan

3 3

Inconsistency between numpy and flex (sclice notation)
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, I've found a minor inconveniance using the flex type: test case: import numpy as np np_grid=np.double(np.zeros((3,7))) np_grid[1,3]=3.0 print(np_grid) # works print(np_grid[1,:]) # works from cctbx.array_family import flex grid=flex.double(flex.grid(3,7)) grid[1,3]=3.0 flex.show(grid) #works flex.show(grid[1,:]) #gives error (see below) "TypeError: All items must be of same type." Is there another simple way to do sclices with flex arrays or could this be implemented the same way as in numpy by someone who is more into c++ than myself? Thanks, Jan

3 5

Determining crystal space group using C++ libraries
by Martin Uhrin 30 Dec '11

30 Dec '11

Dear Cctbx community, I'm trying to determine the space group for my crystal structures using the C++ libraries. I haven't found a way to determine the full space group (i.e. lattice + atomic sites), does anyone have any experience of doing this with cctbx? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 1

iotbx::pdb and boost::unit_test::framework
by vanv0059＠umn.edu 14 Dec '11

14 Dec '11

Hi, Is it reasonable to assume that the boost unit test framework (boost::test) is compatible with cctbx's iotbx::pdb? Or might boost::test's handling of memory clash with that of cctbx? I can get simple examples with iotbx::pdb::input by itself or a print statement wrapped by boost test to work fine on Linux (gcc 4.1.2). What I am having issues with is replacing the print statement & include file with iotbx::pdb::input (even if no file is opened). If the answer to the above question is: "it is reasonable", then would it be better to post my question to the c++ sig group? The simple example is: #define BOOST_TEST_DYN_LINK #define BOOST_TEST_MODULE test_iotbx_pdb_input #include <boost/test/unit_test.hpp> #include <iotbx/pdb/input.h> BOOST_AUTO_TEST_CASE(test_iotbx_pdb_input) { iotbx::pdb::input p; } Both gdb and valgrind complain about the example program trying to access addr 0x1 gdb traceback: #0 0x00002aaaab74be75 in free () from /lib64/libc.so.6 #1 0x00002aaaaad1a768 in boost::checked_array_delete<char> ( x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:41 #2 0x00002aaaaad1a783 in boost::checked_array_deleter<char>::operator() ( this=0x5103e8, x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:63 #3 0x00002aaaaad1a8b1 in boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose (this=0x5103d0) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/smart_ptr/detail/sp_counted_impl.hpp:148 #4 0x00002aaaab028d78 in boost::runtime::cla::parser::parse(int&, char**) () from /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/stage/lib/libboost_unit_test_framework.so.1.48.0 #5 0x0000000000000000 in ?? () valgrind note: ==26968== Invalid free() / delete / delete[] / realloc() ==26968== at 0x4A20E72: operator delete[](void*) (vg_replace_malloc.c:490) ==26968== by 0x4C79767: void boost::checked_array_delete<char>(char*) (checked_delete.hpp:41) ==26968== by 0x4C79782: boost::checked_array_deleter<char>::operator()(char*) const (checked_delete.hpp:63) ==26968== by 0x4C798B0: boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose() (sp_counted_impl.hpp:148) ==26968== by 0x4F86D77: boost::runtime::cla::parser::parse(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F65E6F: boost::unit_test::runtime_config::init(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libb oost_unit_test_framework.so.1.48.0) ==26968== by 0x4F455E0: boost::unit_test::framework::init(bool (*)(), int, ch ar**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/li b/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F59D7E: boost::unit_test::unit_test_main(bool (*)(), int, cha r**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib /libboost_unit_test_framework.so.1.48.0) ==26968== by 0x407377: main (unit_test.hpp:59) ==26968== Address 0x1 is not stack'd, malloc'd or (recently) free'd

2 7

Using iotbx.pdb as C++ only dynamic (shared) library
by vanv0059＠umn.edu 13 Dec '11

13 Dec '11

Hi, The iotbx.pdb Python module is very useful and fast. I am trying to use the C++ library and have gotten an example program to run correctly when compiling in the object files in cctbx_build/iotbx/pdb: atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o However, I would like to have a shared library that wraps all of the objects into shared library (or dynamic on Mac OSX). I have created such a library on my Mac using the command: g++ -dynamiclib -fPIC -O3 --fast-math -o libiotbx_pdb.dylib atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o -install_name /Users/jvanvoorst/Python_VEs/Lore30/lib/libiotbx_pdb.dylib -Wl,-single_module When I link my program to that library I always get a segment fault in the deconstructor of the last instance of iotbx::pdb::hierarchy::root::~root(). gdb traceback is: Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x000000000000001a 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () (gdb) where #0 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () #1 0x00000001000035fe in iotbx::pdb::hierarchy::root_data::~root_data () #2 0x0000000100003662 in boost::checked_delete<iotbx::pdb::hierarchy::root_data> () #3 0x0000000100003687 in boost::detail::sp_counted_impl_p<iotbx::pdb::hierarchy::root_data>::dispose () #4 0x0000000100001342 in boost::detail::sp_counted_base::release () #5 0x0000000100001372 in boost::detail::shared_count::~shared_count () #6 0x000000010000149b in boost::shared_ptr<iotbx::pdb::hierarchy::root_data>::~shared_ptr () #7 0x00000001000014e9 in iotbx::pdb::hierarchy::root::~root () #8 0x00000001000053c9 in FooBar::~FooBar () #9 0x0000000100001150 in main () I can send you my program but I have pruned FooBar down to only two class variables (one for a root object and the other for the input). All the class does is read the input from a file and construct the hierarchy. The seg fault only happens on the cleanup of the last object. That is if I vary the number of FooBar instances, the program faults on the cleanup of the last root object. I tried numerous combinations of flags to the linker (g++), but (as expected) the result is always the same. I am using "g++ -I{include for sources} -I{include for build} test.C libiotbx_pdb.dylib" to compile the program. I am likely overlooking something. Also, I am not proficient in debugging dynamic libraries via gdb. Therefore, any tips or pointers would be much appreciated. If there is an edit I can make to my Sconscript to automagically generate the shared library I could use that as well. The libtbx/configure.py output looks right (Static libraries: False). However, when running scons using -Q the source files are getting compiled with -fPIC (good), but the libraries are getting created using ar (implies static libraries) -- unless I am looking in the wrong directory for the shared libraries (cctbx_build/lib). Thanks, Jeff Van Voorst

2 8

Problems building scitbx.random
by Jan Marten Simons 29 Nov '11

29 Nov '11

Hi, After doing a fresh checkout with these commands: > #! /bin/csh -fe > set verbose > curl http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx -o cctbx_bundle.selfx > perl cctbx_bundle.selfx 0 > rm cctbx_bundle.selfx > rm cctbx_install_script.csh > mv cctbx_sources sources > cd sources > svn co https://cctbx.svn.sourceforge.net/svnroot/cctbx/trunk cctbx_project > cctbx_project/libtbx/development/move_obsolete_from_bundle.csh And using these commands to configue and build cctbx: > mkdir -p build > cd build > python ../sources/cctbx_project/libtbx/configure.py -- enable_openmp_if_possible=True --scan_boost --build=release boost_adaptbx cctbx chiltbx crys3d iotbx mmtbx rstbx ucif smtbx clipper_adaptbx fable scitbx > source setpaths.sh > make || ( echo "Make failed!"; exit 1 ) I encounter these errors: Processing: "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py" Traceback (most recent call last): File "../sources/cctbx_project/libtbx/configure.py", line 21, in <module> run() File "../sources/cctbx_project/libtbx/configure.py", line 17, in run libtbx.env_config.cold_start(sys.argv) File "/home/marten/cctbx/cctbx- dev/sources/cctbx_project/libtbx/env_config.py", line 2061, in cold_start env.refresh() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1401, in refresh module.process_libtbx_refresh_py() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1678, in process_libtbx_refresh_py execfile(abs(custom_refresh), {}, {"self": self}) File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py", line 3, in <module> from crys3d import qttbx File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/qttbx/__init__.py", line 4, in <module> import gltbx.util File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/gltbx/util.py", line 1, in <module> import scitbx.array_family.flex # import dependency File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/array_family/flex.py", line 10, in <module> import scitbx.random File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/random/__init__.py", line 24, in <module> mt19937 = mersenne_twister_19937(value=get_random_seed()) NameError: name 'mersenne_twister_19937' is not defined To me these look like scitbx.random-ext is not built. Is this a regression from the changes to the build system or am I doining something wrong here? Cheers, Jan

2 3

Using cctbx as a C++ only library
by Martin Uhrin 26 Nov '11

26 Nov '11

Dear Cctbx community, thank you very much for the great library, full of wonderful algorithms. I'm writing a structure searching engine in C++ to be used as a high throughput method to do energy landscape sampling with a view to finding new ground state crystal structures. Key requirements of this project are efficiency and ease of development & deployment. Cctbx has many useful algorithms and data structures that I would like to avoid having to re-code if possible. Thus I've downloaded and played with the source as it is but have found it difficult to integrate with my project (for understandably reasons given the goals of cctbx). My main goals regarding ease of development & deployment mean that it would be very inconvenient to distribute the entire cctbx with my code. Ideally I'd like to achieve the following: * strip out everything python related (apart from maybe the build system) * strip out all modules (namespaces) that are unused by my code leaving a fairly bare-bones C++ library that is quick and easy to build. Is anyone able to comment on how feasible this might be? Is this even worth attempting? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 3

cctbx + PyPy?
by Jan Marten Simons 25 Nov '11

25 Nov '11

Hi, PyPy 1.7 was released just a few days ago: http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html After taking a look at http://speed.pypy.org/ I'm wondering if it would be possible to get cctbx to work with PyPy-jit instead of CPython. Perhaps it's a little bit early, as they are still working on a complete numpy integration and I'm not sure how well boost.python fits into this. But in general speeding up the python parts looks nice if one does not have to invest too much time into this. Has any of you already done some tests with PyPy? Cheers, Jan

2 1

crystal packing/unit cell question
by Jessica Grant 08 Nov '11

08 Nov '11

I am new to cctbx and crystallography in general. I am trying to take a pdb file and produce a representation of the crystal, similar to the symmetry mates produced in pymol. I have used cctbx to apply the symmetry operators, but this gives atomic coordinates that are quite spread out. I think what I want to do is to fill the unit cell, or in some way extract crystal packing information. I would like, in the end, to be able to build files with a physically realistic representation of different sized crystals, so I can go on to study the interactions of molecules within those crystals. Is there a way to do this in cctbx? Thanks! Jessica

5 11

flex index
by Monarin Uervirojnangkoorn 18 Oct '11

18 Oct '11

Hi, I'm using flex and 'd like to get an access to elements fall beyond/ below certain value. eg. a=[1,4,5,7,5,6,6,1] anything beyond 5 should be set to 5. (in matlab then a(a>5)=5;) if you could tell me how to do this quickly for flex, that would be great. for now i use loop and that is... pretty slow. many thanks. mona ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^ Institute of Biochemistry, Institut für Neuro- und Bioinformatik, Graduate School for Computing in Medicine and Life Sciences University of Lübeck, Ratzeburger Allee 160 Lübeck 23538 Germany Tel: +49451-5004072 Fax: +49451-5004068

2 3

CCTBX comprehensive overview
by Graeme.Winter＠Diamond.ac.uk 12 Oct '11

12 Oct '11

Hi Folks, Is there somewhere a comprehensive overview of what is available within cctbx? I have had a request for this from someone who may be interested in using it, but does not want to start looking until they know what is in there, without inspecting any code. Specifically, an idea of what is available from Python would hit the spot. Is there a way to generate this or does it already exist on the web? Thanks in advance, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

cropping out a fragment
by Monarin Uervirojnangkoorn 04 Oct '11

04 Oct '11

Hi all, I'd like to crop out a portion of a fragment (say a helix) from real_map by specifying fractional coordinates. would be nice to know if there is any specific routine for this. (oh .. & ++ option like crop only 2 angstroms from the location of the atom would be really nice). thankx Best, Mona ~.~.~.~><~.~~.~.~. Institute of Biochemistry, Institut of Neuro- and Bioinformatics Graduate School for Computing in Medicine and Life Sciences University of Lübeck, Ratzeburger Allee 160 Lübeck 23538 Germany

2 1

Deleting batch from mtz batch header via iotbx.mtz
by Graeme.Winter＠Diamond.ac.uk 27 Sep '11

27 Sep '11

Hi Folks, Is there a way to delete a batch from an mtz file batch header? Been trying to do this and it looks like it is impossible - though you can add batches. Clearly there is the worry that the subsequent batch column may then refer to a missing batch, however... it would be handy to be able to do this (providing that the reflections were also removed elsewhere in the calling code) Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Lattice options & changes of basis
by Graeme.Winter＠Diamond.ac.uk 20 Sep '11

20 Sep '11

Hi Folks, I have a very simple question which I *know* can be done with cctbx, using the Python API, but I cannot for the life of me figure out how to do it. I have even read the code, it's just not clear. I have a Bravais lattice and cell, say oP and 20, 30, 40, 90, 90, 90 - what I would like back is a list of pointgroup, cell, reindexing operation for all compatible unique settings i.e. 1 x P1, 3 x P2, 1 x P222 - from a Python loop. At the moment I run iotbx.lattice symmetry and grep the results, which is not a great idea. It would also be useful to give the reindex operations to centred lattices. I have also stared at phenix.explore_metric_symmetry and remain puzzled despite my attempt to essentially duplicate the code. I just can't get the reindex operation out. Has anyone some simple code to do this? I tried piggybacking on pointgroup tools but that did not lead to enlightenment. I suspect using elementary ish operations would be better... Thanks, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Conversion from symmetry operation to matrix
by Graeme.Winter＠Diamond.ac.uk 13 Jul '11

13 Jul '11

Dear All, For a little while now I have been using some cctbx jiffy code to convert from symmetry operations to matrices to help with testing reindexing data. This works as def symop_to_mat(symop): return matrix.sqr(sgtbx.rt_mx( sgtbx.parse_string(symop)).as_double_array()[:9]).transpose().elems Which is plenty simple. However it just exploded when given "1/2*z,-y+1/2*z,x" - reasonably really as this is not a rotation. It is however a valid reindexing operation to go from C2 to P1. Does anyone have code which will handle this general transformation from a reindexing operation to a matrix, which will cope with the fact that the resulting matrix is not RT? I had a little jiffy I wrote using ccp4f lib (symop2mat - you can probably see it!) but I am trying to do away with things like that and do everything properly... Many thanks in advance, Graeme Dr. Graeme Winter Senior Software Scientist Diamond Light Source +44 1235 778091 (work) +44 7786 662784 (work mobile) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 6

Bug in structure.expand_to_p1?
by Jan Marten Simons 21 Jun '11

21 Jun '11

Hi, I'm quite puzzled at the moment. using this code: if __name__ == '__main__': quartz_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=crystal.symmetry( unit_cell=(5.01,5.01,5.47,90,90,120), space_group_symbol="P6222")), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(1/2.,1/2.,1/3.), u=0.2), xray.scatterer( label="O", site=(0.197,-0.197,0.83333), u=0)])) ref_structure = quartz_structure.expand_to_p1( append_number_to_labels=True) print(ref_structure.as_py_code()) I get this result: xray.structure( crystal_symmetry=crystal.symmetry( unit_cell=(5.01, 5.01, 5.47, 90, 90, 120), space_group_symbol="P 1"), scatterers=flex.xray_scatterer([ xray.scatterer( #0 label="Si_0", site=(0.500000, 0.500000, 0.333333), u=0.200000), xray.scatterer( #1 label="Si_1", site=(0.000000, 0.500000, 0.666667), u=0.200000), xray.scatterer( #2 label="Si_2", site=(0.500000, 0.000000, 1.000000), u=0.200000), xray.scatterer( #3 label="O_0", site=(0.197000, -0.197000, 0.833333), u=0.000000), xray.scatterer( #4 label="O_1", site=(0.394000, 0.197000, 1.166667), u=0.000000), xray.scatterer( #5 label="O_2", site=(-0.197000, -0.394000, 1.500000), u=0.000000), xray.scatterer( #6 label="O_3", site=(0.197000, 0.394000, 1.500000), u=0.000000), xray.scatterer( #7 label="O_4", site=(-0.394000, -0.197000, 1.166667), u=0.000000), xray.scatterer( #8 label="O_5", site=(-0.197000, 0.197000, 0.833333), u=0.000000)])) It bothers me that there are x,y,z not in [0, 1[. Shouldn't the newly expanded cell only contain atoms within the unit cell? I realize, that the quarz structure (from the sample code from the website) also violates this with Oy beeing negative, but still I think expand_to_p1 should return a regular structure. Also is there a method already implemented to change a structure from a non- conventional description into a conventional one (with all atoms within its unit cell)? With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

3 4

CCTBX for Debian + PATCH
by Radostan Riedel 13 Jun '11

13 Jun '11

Hello, I'm about to make a (multi binary) Debian source package of CCTBX-Project. The plan is to build a few small Packages that depends on each other. So the python stuff would go to /usr/lib/python*.*/dist-packages/ and the libraries would go to /usr/lib and the Headers to /usr/include. Example: ''' python-iotbx: installs iotbx python module and extensions to /usr/lib/python*.*/dist-packages/ libcctbx0: installs libcctbx.so.0 etc. to /usr/lib/ libcctbx-dev: installs a symlink libcctbx.so ---> libcctbx.so.0 etc. and header files to /usr/include/cctbx ''' This would follow strictly the Debian Policy, so maybe when this will be accepted we could see CCTBX in the official Repos of Debian and Ubuntu etc. But for that I still have a few problems: 1.) The SONAME and SOVERSION is missing from the shared libraries. I'm attaching a patch that can fix this for GNU/Linux. That produces files like this: ''' ls lib/libcctbx* lib/libcctbx_sgtbx_asu.so lib/libcctbx_sgtbx_asu.so.0 lib/libcctbx.so lib/libcctbx.so.0 ''' 2.) PyCifRW: I had to remove it from the sources because it ain't free software because of some lines in their license. It wasn't accepted to Fedora because of that, so I guess I can forget Debian either. 3.) In your changelog I read you wanna get rid of the "from __future__ import division" and change to "python -Qnew". This would be a problem for me, because than I couldn't use it from sys.path anymore. I have a Django project at work that imports some cctbx modules directly from pythonpath. And I can't really call "cctbx.python" for that, otherwise I would have to patch my Apache and mod_wsgi, respectively cctbx. Wouldn't it be better to install all python modules to the standard python location of the Distribution than making dispatchers with hardcoded paths? Also, when the libraries, headers and python stuff would be in their standard location, everyone could just import it from anywhere? I will be finished soon with packaging and then I will upload it to my PPA on Lauchpad https://launchpad.net/~raybuntu/+archive/xray which is empty by now. And I think this could also be adopted to other Distributions like Fedora. I'd be glad to hear some critical opinions about this from you. regards R.R.

2 6

Best way to do 3d xyz plots with cctbx?
by Jan Marten Simons 07 Jun '11

07 Jun '11

Hi, I noticed cctbx has some custom wx based graphic windows to display various specialized plots and visualisations. I'm interested in generating some simple 3d xyz plots like in [1]. Is functionality for this included in cctbx or should I rather use another python based plotter for this? [1] http://pyx.sourceforge.net/examples/3dgraphs/surface.html Thanks in advance, Jan

2 1