- cctbxbb - phenix-online.org

Comparison of Crystal Structures
by David Lonie 13 Jan '12

13 Jan '12

Hi list, I have written an open-source (BSD-licensed) algorithm to compare two crystal structure descriptions and identify whether they describe the same underlying structure. The algorithm is capable of detecting degenerate descriptions (meaning unit cell vectors + atomic positions and types) even in the presence of the "real-world" issues confronting computational crystallographers (choice of unit cell representation and offset, rotations, reflections, and numerical noise). The algorithm returns a boolean match using customizable tolerances. I just wanted to inform the mailing list of this, in case such functionality is needed by your user community. The code, XtalComp, is written in C++ and is easy to interface with, having no external dependencies. The liberal license would allow it to be included with the cctbx sources and interfaced with cctbx code if desired. The code uses the CMake build system, but the simple nature of the sources would allow trivial porting to SCons. There is a web interface to the code here: http://xtalopt.openmolecules.net/xtalcomp/xtalcomp.html Also on that page are links to the journal article describing the algorithm, as well as a free download of the preprint version of the article. The code is available on Github (http://www.github.com/dlonie/XtalComp). Thanks for your time, and I hope that someone out there will find this code useful :-) Keep up the good work! Dave

2 1

Attitude of rotation, Python to C++
by Graeme.Winter＠Diamond.ac.uk 13 Jan '12

13 Jan '12

Hi Folks, Just started moving code from Python to C++ - and seeing a 40x speedup which is nice - however I noticed something odd. scitbx.matrix.col.rotate(axis, angle) has the reversed attitude to scitbx::vec3<double>.rotate(scitbx::vec3<double>, double angle) viz: this python code s = (self._ub * hkl).rotate(self._axis, angle) q = (s + self._s0).normalize() # check if diffracted ray parallel to detector face q_dot_n = q.dot(detector_normal) if q_dot_n == 0: continue r = (q * distance / q_dot_n) - self._detector_origin x = r.dot(self._detector_fast) y = r.dot(self._detector_slow) if x < self._limits[0] or y < self._limits[2]: continue if x > self._limits[1] or y > self._limits[3]: continue observed_reflection_positions.append((hkl, x, y, angle)) gives the same result as scitbx::vec3<double> s, q, r; double x, y, q_dot_n; /* this is rather weird - the attitude of rotation is back to front in the c++ code */ s = (ub * hkl).rotate(axis, - angle); q = (s + s0).normalize(); q_dot_n = q * normal; if (q_dot_n == 0) return false; r = (q * distance / q_dot_n) - origin; x = r * fast; y = r * slow; if (x < limits[0]) return false; if (y < limits[2]) return false; if (x > limits[1]) return false; if (y > limits[3]) return false; prediction[0] = x; prediction[1] = y; where I had to use -angle. Code is linked into same routines - used identically. With -angle I get numerically identical results (which is nice) Anyone got any clues on this one? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

How to use the binner in cctbx.miller?
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, as documentation is lacking on this: How can I use the binner from cctbx.miller to bin reflections into different groups according to their d-value and would it be possible afterwards to get integral intesities of each binned group easily? Thanks, Jan

3 3

Inconsistency between numpy and flex (sclice notation)
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, I've found a minor inconveniance using the flex type: test case: import numpy as np np_grid=np.double(np.zeros((3,7))) np_grid[1,3]=3.0 print(np_grid) # works print(np_grid[1,:]) # works from cctbx.array_family import flex grid=flex.double(flex.grid(3,7)) grid[1,3]=3.0 flex.show(grid) #works flex.show(grid[1,:]) #gives error (see below) "TypeError: All items must be of same type." Is there another simple way to do sclices with flex arrays or could this be implemented the same way as in numpy by someone who is more into c++ than myself? Thanks, Jan

3 5

Determining crystal space group using C++ libraries
by Martin Uhrin 30 Dec '11

30 Dec '11

Dear Cctbx community, I'm trying to determine the space group for my crystal structures using the C++ libraries. I haven't found a way to determine the full space group (i.e. lattice + atomic sites), does anyone have any experience of doing this with cctbx? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 1

iotbx::pdb and boost::unit_test::framework
by vanv0059＠umn.edu 14 Dec '11

14 Dec '11

Hi, Is it reasonable to assume that the boost unit test framework (boost::test) is compatible with cctbx's iotbx::pdb? Or might boost::test's handling of memory clash with that of cctbx? I can get simple examples with iotbx::pdb::input by itself or a print statement wrapped by boost test to work fine on Linux (gcc 4.1.2). What I am having issues with is replacing the print statement & include file with iotbx::pdb::input (even if no file is opened). If the answer to the above question is: "it is reasonable", then would it be better to post my question to the c++ sig group? The simple example is: #define BOOST_TEST_DYN_LINK #define BOOST_TEST_MODULE test_iotbx_pdb_input #include <boost/test/unit_test.hpp> #include <iotbx/pdb/input.h> BOOST_AUTO_TEST_CASE(test_iotbx_pdb_input) { iotbx::pdb::input p; } Both gdb and valgrind complain about the example program trying to access addr 0x1 gdb traceback: #0 0x00002aaaab74be75 in free () from /lib64/libc.so.6 #1 0x00002aaaaad1a768 in boost::checked_array_delete<char> ( x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:41 #2 0x00002aaaaad1a783 in boost::checked_array_deleter<char>::operator() ( this=0x5103e8, x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:63 #3 0x00002aaaaad1a8b1 in boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose (this=0x5103d0) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/smart_ptr/detail/sp_counted_impl.hpp:148 #4 0x00002aaaab028d78 in boost::runtime::cla::parser::parse(int&, char**) () from /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/stage/lib/libboost_unit_test_framework.so.1.48.0 #5 0x0000000000000000 in ?? () valgrind note: ==26968== Invalid free() / delete / delete[] / realloc() ==26968== at 0x4A20E72: operator delete[](void*) (vg_replace_malloc.c:490) ==26968== by 0x4C79767: void boost::checked_array_delete<char>(char*) (checked_delete.hpp:41) ==26968== by 0x4C79782: boost::checked_array_deleter<char>::operator()(char*) const (checked_delete.hpp:63) ==26968== by 0x4C798B0: boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose() (sp_counted_impl.hpp:148) ==26968== by 0x4F86D77: boost::runtime::cla::parser::parse(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F65E6F: boost::unit_test::runtime_config::init(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libb oost_unit_test_framework.so.1.48.0) ==26968== by 0x4F455E0: boost::unit_test::framework::init(bool (*)(), int, ch ar**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/li b/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F59D7E: boost::unit_test::unit_test_main(bool (*)(), int, cha r**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib /libboost_unit_test_framework.so.1.48.0) ==26968== by 0x407377: main (unit_test.hpp:59) ==26968== Address 0x1 is not stack'd, malloc'd or (recently) free'd

2 7

Using iotbx.pdb as C++ only dynamic (shared) library
by vanv0059＠umn.edu 13 Dec '11

13 Dec '11

Hi, The iotbx.pdb Python module is very useful and fast. I am trying to use the C++ library and have gotten an example program to run correctly when compiling in the object files in cctbx_build/iotbx/pdb: atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o However, I would like to have a shared library that wraps all of the objects into shared library (or dynamic on Mac OSX). I have created such a library on my Mac using the command: g++ -dynamiclib -fPIC -O3 --fast-math -o libiotbx_pdb.dylib atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o -install_name /Users/jvanvoorst/Python_VEs/Lore30/lib/libiotbx_pdb.dylib -Wl,-single_module When I link my program to that library I always get a segment fault in the deconstructor of the last instance of iotbx::pdb::hierarchy::root::~root(). gdb traceback is: Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x000000000000001a 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () (gdb) where #0 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () #1 0x00000001000035fe in iotbx::pdb::hierarchy::root_data::~root_data () #2 0x0000000100003662 in boost::checked_delete<iotbx::pdb::hierarchy::root_data> () #3 0x0000000100003687 in boost::detail::sp_counted_impl_p<iotbx::pdb::hierarchy::root_data>::dispose () #4 0x0000000100001342 in boost::detail::sp_counted_base::release () #5 0x0000000100001372 in boost::detail::shared_count::~shared_count () #6 0x000000010000149b in boost::shared_ptr<iotbx::pdb::hierarchy::root_data>::~shared_ptr () #7 0x00000001000014e9 in iotbx::pdb::hierarchy::root::~root () #8 0x00000001000053c9 in FooBar::~FooBar () #9 0x0000000100001150 in main () I can send you my program but I have pruned FooBar down to only two class variables (one for a root object and the other for the input). All the class does is read the input from a file and construct the hierarchy. The seg fault only happens on the cleanup of the last object. That is if I vary the number of FooBar instances, the program faults on the cleanup of the last root object. I tried numerous combinations of flags to the linker (g++), but (as expected) the result is always the same. I am using "g++ -I{include for sources} -I{include for build} test.C libiotbx_pdb.dylib" to compile the program. I am likely overlooking something. Also, I am not proficient in debugging dynamic libraries via gdb. Therefore, any tips or pointers would be much appreciated. If there is an edit I can make to my Sconscript to automagically generate the shared library I could use that as well. The libtbx/configure.py output looks right (Static libraries: False). However, when running scons using -Q the source files are getting compiled with -fPIC (good), but the libraries are getting created using ar (implies static libraries) -- unless I am looking in the wrong directory for the shared libraries (cctbx_build/lib). Thanks, Jeff Van Voorst

2 8

Problems building scitbx.random
by Jan Marten Simons 29 Nov '11

29 Nov '11

Hi, After doing a fresh checkout with these commands: > #! /bin/csh -fe > set verbose > curl http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx -o cctbx_bundle.selfx > perl cctbx_bundle.selfx 0 > rm cctbx_bundle.selfx > rm cctbx_install_script.csh > mv cctbx_sources sources > cd sources > svn co https://cctbx.svn.sourceforge.net/svnroot/cctbx/trunk cctbx_project > cctbx_project/libtbx/development/move_obsolete_from_bundle.csh And using these commands to configue and build cctbx: > mkdir -p build > cd build > python ../sources/cctbx_project/libtbx/configure.py -- enable_openmp_if_possible=True --scan_boost --build=release boost_adaptbx cctbx chiltbx crys3d iotbx mmtbx rstbx ucif smtbx clipper_adaptbx fable scitbx > source setpaths.sh > make || ( echo "Make failed!"; exit 1 ) I encounter these errors: Processing: "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py" Traceback (most recent call last): File "../sources/cctbx_project/libtbx/configure.py", line 21, in <module> run() File "../sources/cctbx_project/libtbx/configure.py", line 17, in run libtbx.env_config.cold_start(sys.argv) File "/home/marten/cctbx/cctbx- dev/sources/cctbx_project/libtbx/env_config.py", line 2061, in cold_start env.refresh() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1401, in refresh module.process_libtbx_refresh_py() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1678, in process_libtbx_refresh_py execfile(abs(custom_refresh), {}, {"self": self}) File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py", line 3, in <module> from crys3d import qttbx File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/qttbx/__init__.py", line 4, in <module> import gltbx.util File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/gltbx/util.py", line 1, in <module> import scitbx.array_family.flex # import dependency File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/array_family/flex.py", line 10, in <module> import scitbx.random File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/random/__init__.py", line 24, in <module> mt19937 = mersenne_twister_19937(value=get_random_seed()) NameError: name 'mersenne_twister_19937' is not defined To me these look like scitbx.random-ext is not built. Is this a regression from the changes to the build system or am I doining something wrong here? Cheers, Jan

2 3

Using cctbx as a C++ only library
by Martin Uhrin 26 Nov '11

26 Nov '11

Dear Cctbx community, thank you very much for the great library, full of wonderful algorithms. I'm writing a structure searching engine in C++ to be used as a high throughput method to do energy landscape sampling with a view to finding new ground state crystal structures. Key requirements of this project are efficiency and ease of development & deployment. Cctbx has many useful algorithms and data structures that I would like to avoid having to re-code if possible. Thus I've downloaded and played with the source as it is but have found it difficult to integrate with my project (for understandably reasons given the goals of cctbx). My main goals regarding ease of development & deployment mean that it would be very inconvenient to distribute the entire cctbx with my code. Ideally I'd like to achieve the following: * strip out everything python related (apart from maybe the build system) * strip out all modules (namespaces) that are unused by my code leaving a fairly bare-bones C++ library that is quick and easy to build. Is anyone able to comment on how feasible this might be? Is this even worth attempting? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 3

cctbx + PyPy?
by Jan Marten Simons 25 Nov '11

25 Nov '11

Hi, PyPy 1.7 was released just a few days ago: http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html After taking a look at http://speed.pypy.org/ I'm wondering if it would be possible to get cctbx to work with PyPy-jit instead of CPython. Perhaps it's a little bit early, as they are still working on a complete numpy integration and I'm not sure how well boost.python fits into this. But in general speeding up the python parts looks nice if one does not have to invest too much time into this. Has any of you already done some tests with PyPy? Cheers, Jan

2 1