- cctbxbb - phenix-online.org

Getting r1_factor of miller arrays with different count of miller indices
by Jan Marten Simons 06 Jul '12

06 Jul '12

Hi, I'm facing a new challenge while working with cctbx: Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem: # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2") trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)])) # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array() f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents() R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False) gives: " assert other.indices().size() == self.indices().size() AssertionError " Is there some way to get the R1_factor for this kind of scenario? Cheers and thanks in advance, Jan

4 8

Fast way to enable/disable individual scatterers in structure factor calculation
by Jan Marten Simons 06 Jul '12

06 Jul '12

Hi, I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation. As I see from the source (structure_factors_direct.h) the selection toggle does not seem to be evaluated for this. Of course an additional check for the selection flag would harm performance in most standard cases, so I'd rather like to have a separate c++ function for this, which could be swapped in for the normal f_calc calculation by a boolean toggle in the python part of f_calc. As I'm not that much into the c++ part of cctbx, I'd really appreciate if somebody more knowledgeable could monkey patch this. Cheers, Jan

2 5

angle derivative of rotation matrix
by David Waterman 28 Jun '12

28 Jun '12

Hi, I'm using the axis_and_angle_as_r3_rotation_matrix method of scitbx.matrix.col in some Python code. Now I need to calculate the first derivative of the rotation matrix with respect to the rotation angle. Is there a function to do that somewhere in scitbx (or elsewhere in cctbx)? I had a brief look around but couldn't find anything that looked hopeful. If there isn't something there already I have some Fortran code from David Thomas that I could adapt to do this. Of course it wouldn't be very efficient in pure Python, and calculating the matrix and its derivative separately, but that's okay for now as the code is just for testing out some ideas. By the way, links to http://skal.planet-d.net/demo/matrixfaq.htm in the documentation (e.g. here<http://cctbx.sourceforge.net/current/c_plus_plus/namespacescitbx_1_1math_1_…>) are broken. I found a document with the same name here: http://vamos.sourceforge.net/matrixfaq.htm, but I have no idea if this has the same content as the original. Cheers -- David

3 4

Problem with direct_space_asu
by Jan Marten Simons 26 Jun '12

26 Jun '12

Hi, as there seems to be no documentation on the direct_space_asu module I just tried to use it like I thought the function names imply. The task I want to accomplish is checking whether a point in direct space is inside the asymmetric unit of a given unit cell. A small testcase I wrote reveals that apparently .is_inside(point) only uses fractional coordinates instead of cartesian coordinates. Is there a nice/easy way to get the desired behaviour? # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex import numpy as np asu = xray.structure( crystal_symmetry=crystal.symmetry( unit_cell=(5.01,5.01,5.47,95,85,90), space_group_symbol="P 1")).direct_space_asu() print(asu.show_comprehensive_summary()) print(asu.is_inside((0.0, 0.0, 0.0))) #should be True OK print(asu.is_inside((0.5, 0.5, 0.5))) #should be True OK print(asu.is_inside((0.9, 0.9, 0.9))) #should be True OK print(asu.is_inside((1.0, 0.9, 0.9))) #should be True is False! print(asu.is_inside((1.0, 1.0, 1.0))) #should be True is False! print(asu.is_inside((2.0, 2.0, 2.0))) #should be True is False! print(asu.is_inside((3.0, 3.0, 3.0))) #should be True is False! print(asu.is_inside((4.0, 4.0, 4.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 5.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 6.0))) #should be False OK print(asu.is_inside((6.0, 6.0, 6.0))) #should be False OK Cheers, Jan

2 1

compilation errors under gcc-4.7
by Justin Lecher 19 Jun '12

19 Jun '12

Hi, when compiling cctbx with gcc-4.7 I see following errors which I am not able to fix: cctbx_sources/scitbx/math/boost_python/distributions.cpp:112:59: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:86:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp: In instantiation of 'static void scitbx::math::{anonymous}::non_member_function_wrappers<FloatType, Distribution>::wrap() [with FloatType = double; Distribution = boost::math::students_t_distribution<double>]': cctbx_sources/scitbx/math/boost_python/distributions.cpp:116:63: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:85:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information Any suggestions to fix? Thanks, justin -- Justin Lecher Institute of Complex Systems ICS-6 Structural Biochemistry Research Centre Juelich 52425 Juelich, Germany phone: +49 2461 61 2117

5 8

Re: [cctbxbb] compilation errors under gcc-4.7
by Luc Bourhis 19 Jun '12

19 Jun '12

> I take it you're interested in fixing this? Yes. I've got a Fedora 17 virtual machine. Luc

1 0

proposal to remove iotbx.shelx.from_ins
by Mirko Scholz 12 Jun '12

12 Jun '12

Hello, since I am new here it seems an odd proposal, but could you not remove iotbx.shelx.from_ins.from_ins since its functionality is covered much better by xray.structure.from_shelx (at least it seems to me). The following is roughly identical with the first statement being inaccurate (anisotropic vs isotropic displacements parameters) and wrong (structures containing disorder) xs0 = iotbx.shelx.from_ins.from_ins(file_name='foo.ins') fc0 = xs0.structure_factors(d_min=0.7, algorithm='direct').f_calc() xs1 = cctbx.xray.structure.from_shelx(file=open('foo.ins')) fc1 = xs1.structure_factors(d_min=0.7, algorithm='direct').f_calc() As someone who (seldom) uses cctbx I stumbled first over iotbx.shelx.from_ins and only noticed after a while that it does not handle disorder properly (problem is in ins_record.if_10_added). from_ins is used only here ./iotbx/command_line/emma.py ./cctbx/adp_restraints/tst_ext.py Where it should be possible to substitute it though I have not tested it. Cordially Mirko Scholz

4 4

controlling types over the C++/Python interface
by David Waterman 16 May '12

16 May '12

Hi, I'm starting to explore the use of boost.python in cctbx. I've written a simple C++ class and exposed its accessor methods as attributes using the .add_property syntax in the wrapper. In the C++ class my member data and accessor return types are, for example, scitbx::vec3<double>. When I access the attribute in Python with the usual dot operator I get a tuple. What I'd really like though is an object of type <class 'scitbx.matrix.col'>, which is what I passed to the constructor of the object in Python. So the conversion from Python to C++ does what I want, but not the other way round. How do I take control of the type of the attribute in Python, without having to explicitly construct a scitbx.matrix.col from the tuple? It's the same case if I use .def and expose the accessor as a method instead of an attribute. I admit I don't fully understand the details of conversions in either direction. I found I had to add from cctbx.array_family import flex to the package __init__.py where my C++ extension is imported, so that I could pass scitbx.matrix.col objects (or scitbx.matrix.rec, or tuples) to the constructor in Python and not get a Boost.Python.ArgumentError. I've not yet figured out what this import is doing though. Apologies for the remedial question. These are still early days for me. Thanks -- David

5 12

Niggli-reduced cell C++ implementation
by Martin Uhrin 24 Apr '12

24 Apr '12

Dear Cctbx community, Firstly I'd like to say thank you to Rolf, Nicholas and Paul for their expertly thought through implementation of the reduced cell algorithm. I've found it to be extremely useful for my work. My code is all in C++ and I'd like to be able to use the Krivy-Gruber algorithm. My understanding is that only the reduced (Buerger) unit cell algorithm is implemented in C++ [1] which guarantees shortest lengths but not unique angles. From my understanding the Krivy-Gruber would also guarantee me uniqueness of unit cell angles, however this is only implemented in Python [2]. Sorry to be so verbose, I just wanted to check that I was on the right page. Would it be possible for me to implement the Krivy-Gruber in C++ by adding in the epsilon_relative to the parameter and following the procedure found in the python version? Many thanks, -Martin [1] http://cctbx.sourceforge.net/current/c_plus_plus/classcctbx_1_1uctbx_1_1fas… [2] http://cctbx.sourceforge.net/current/python/cctbx.uctbx.krivy_gruber_1976.h… -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

3 8

Handling unmerged reflections from MTZ
by Graeme.Winter＠Diamond.ac.uk 24 Apr '12

24 Apr '12

Hi Folks, I have been writing very bad Python code to handle unmerged MTZ files using CCTBX - in essence copying all of the measurements to lists for each Miller index. Inefficient. Has anyone looked a neater way of handling these files perhaps with doing the "hard work" from C++? Reading the file is very easy, but managing the data is harder. Essentially I would like to be able to read in a scaled-summed-but-unmerged MTZ file (equivalent to unmerged scale pack no-merge-original-index) and get back all observations of a given HKL, which I can then subsequently analyse… I could crack on and write something but it strikes me as a good idea to ask first. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Branches
by Graeme.Winter＠Diamond.ac.uk 22 Apr '12

22 Apr '12

Hi Folks, Quick question - does anyone ever use branches with the CCTBX repository? Just at the moment I'm up to a few things which I would like to be able to commit, but I don't want to break the existing code... however as a project it does not seem CCTBX uses branches much... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 9

Type checking in Phil...
by Graeme.Winter＠Diamond.ac.uk 11 Apr '12

11 Apr '12

… is there any? This does not give an error: from libtbx.phil import parse working_phil = parse(""" parameter { toy = 10 .type = int } """) if __name__ == '__main__': working_phil.show() # this should crash right? working_phil = working_phil.fetch(parse('parameter.toy=foo')) working_phil.show() Output: cctbx.python toy_phil.py parameter { toy = 10 } parameter { toy = foo } Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Build on OS X Lion
by Graeme.Winter＠Diamond.ac.uk 23 Feb '12

23 Feb '12

Hi Folks, Just ran the cctbx_svn_getting_started script to do a build on a new lion system and I got this error - had anyone else got this working correctly on such a system? Template definition not found is a little beyond my usual C++ debugging skills. Many thanks, Graeme c++ -o scitbx/math/boost_python/tensor_rank_2.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/tensor_rank_2.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:339:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:119:4: note: in instantiation of function template specialization 'boost::math::detail::bessel_jy<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here bessel_jy(v, x, &j, &y, need_j, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:165:9: note: in instantiation of function template specialization 'boost::math::detail::cyl_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here * cyl_bessel_j_imp(T(T(n)+T(0.5f)), x, bessel_no_int_tag(), pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:449:65: note: in instantiation of function template specialization 'boost::math::detail::sph_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return policies::checked_narrowing_cast<result_type, Policy>(detail:... ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:455:11: note: in instantiation of function template specialization 'boost::math::sph_bessel<double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return sph_bessel(v, x, policies::policy<>()); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:291:13: note: in instantiation of function template specialization 'boost::math::sph_bessel<double>' requested here return( sph_bessel(l,x) ); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:363:9: note: in instantiation of function template specialization 'scitbx::math::bessel::spherical_bessel<double>' requested here bessel::spherical_bessel); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:16: error: call to function 'bessel_y_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_y_small_z_series(v, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'bessel_y_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_y_small_z_series(T v, T x, T* pscale, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:16: error: call to function 'bessel_yn_small_z' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_yn_small_z(n, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'bessel_yn_small_z' should be declared prior to the call site or in namespace 'boost::math::policies' T bessel_yn_small_z(int n, T z, T* scale, const Policy& pol) ^ 4 errors generated. c++ -o scitbx/math/boost_python/chebyshev.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/chebyshev.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 23.10 seconds wall clock time: 3 minutes 39.02 seconds (219.02 seconds total) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6

Comparison of Crystal Structures
by David Lonie 13 Jan '12

13 Jan '12

Hi list, I have written an open-source (BSD-licensed) algorithm to compare two crystal structure descriptions and identify whether they describe the same underlying structure. The algorithm is capable of detecting degenerate descriptions (meaning unit cell vectors + atomic positions and types) even in the presence of the "real-world" issues confronting computational crystallographers (choice of unit cell representation and offset, rotations, reflections, and numerical noise). The algorithm returns a boolean match using customizable tolerances. I just wanted to inform the mailing list of this, in case such functionality is needed by your user community. The code, XtalComp, is written in C++ and is easy to interface with, having no external dependencies. The liberal license would allow it to be included with the cctbx sources and interfaced with cctbx code if desired. The code uses the CMake build system, but the simple nature of the sources would allow trivial porting to SCons. There is a web interface to the code here: http://xtalopt.openmolecules.net/xtalcomp/xtalcomp.html Also on that page are links to the journal article describing the algorithm, as well as a free download of the preprint version of the article. The code is available on Github (http://www.github.com/dlonie/XtalComp). Thanks for your time, and I hope that someone out there will find this code useful :-) Keep up the good work! Dave

2 1

Attitude of rotation, Python to C++
by Graeme.Winter＠Diamond.ac.uk 13 Jan '12

13 Jan '12

Hi Folks, Just started moving code from Python to C++ - and seeing a 40x speedup which is nice - however I noticed something odd. scitbx.matrix.col.rotate(axis, angle) has the reversed attitude to scitbx::vec3<double>.rotate(scitbx::vec3<double>, double angle) viz: this python code s = (self._ub * hkl).rotate(self._axis, angle) q = (s + self._s0).normalize() # check if diffracted ray parallel to detector face q_dot_n = q.dot(detector_normal) if q_dot_n == 0: continue r = (q * distance / q_dot_n) - self._detector_origin x = r.dot(self._detector_fast) y = r.dot(self._detector_slow) if x < self._limits[0] or y < self._limits[2]: continue if x > self._limits[1] or y > self._limits[3]: continue observed_reflection_positions.append((hkl, x, y, angle)) gives the same result as scitbx::vec3<double> s, q, r; double x, y, q_dot_n; /* this is rather weird - the attitude of rotation is back to front in the c++ code */ s = (ub * hkl).rotate(axis, - angle); q = (s + s0).normalize(); q_dot_n = q * normal; if (q_dot_n == 0) return false; r = (q * distance / q_dot_n) - origin; x = r * fast; y = r * slow; if (x < limits[0]) return false; if (y < limits[2]) return false; if (x > limits[1]) return false; if (y > limits[3]) return false; prediction[0] = x; prediction[1] = y; where I had to use -angle. Code is linked into same routines - used identically. With -angle I get numerically identical results (which is nice) Anyone got any clues on this one? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

How to use the binner in cctbx.miller?
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, as documentation is lacking on this: How can I use the binner from cctbx.miller to bin reflections into different groups according to their d-value and would it be possible afterwards to get integral intesities of each binned group easily? Thanks, Jan

3 3

Inconsistency between numpy and flex (sclice notation)
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, I've found a minor inconveniance using the flex type: test case: import numpy as np np_grid=np.double(np.zeros((3,7))) np_grid[1,3]=3.0 print(np_grid) # works print(np_grid[1,:]) # works from cctbx.array_family import flex grid=flex.double(flex.grid(3,7)) grid[1,3]=3.0 flex.show(grid) #works flex.show(grid[1,:]) #gives error (see below) "TypeError: All items must be of same type." Is there another simple way to do sclices with flex arrays or could this be implemented the same way as in numpy by someone who is more into c++ than myself? Thanks, Jan

3 5

Determining crystal space group using C++ libraries
by Martin Uhrin 30 Dec '11

30 Dec '11

Dear Cctbx community, I'm trying to determine the space group for my crystal structures using the C++ libraries. I haven't found a way to determine the full space group (i.e. lattice + atomic sites), does anyone have any experience of doing this with cctbx? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 1

iotbx::pdb and boost::unit_test::framework
by vanv0059＠umn.edu 14 Dec '11

14 Dec '11

Hi, Is it reasonable to assume that the boost unit test framework (boost::test) is compatible with cctbx's iotbx::pdb? Or might boost::test's handling of memory clash with that of cctbx? I can get simple examples with iotbx::pdb::input by itself or a print statement wrapped by boost test to work fine on Linux (gcc 4.1.2). What I am having issues with is replacing the print statement & include file with iotbx::pdb::input (even if no file is opened). If the answer to the above question is: "it is reasonable", then would it be better to post my question to the c++ sig group? The simple example is: #define BOOST_TEST_DYN_LINK #define BOOST_TEST_MODULE test_iotbx_pdb_input #include <boost/test/unit_test.hpp> #include <iotbx/pdb/input.h> BOOST_AUTO_TEST_CASE(test_iotbx_pdb_input) { iotbx::pdb::input p; } Both gdb and valgrind complain about the example program trying to access addr 0x1 gdb traceback: #0 0x00002aaaab74be75 in free () from /lib64/libc.so.6 #1 0x00002aaaaad1a768 in boost::checked_array_delete<char> ( x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:41 #2 0x00002aaaaad1a783 in boost::checked_array_deleter<char>::operator() ( this=0x5103e8, x=0x1 <Address 0x1 out of bounds>) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/checked_delete.hpp:63 #3 0x00002aaaaad1a8b1 in boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose (this=0x5103d0) at /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/boost/smart_ptr/detail/sp_counted_impl.hpp:148 #4 0x00002aaaab028d78 in boost::runtime::cla::parser::parse(int&, char**) () from /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost/stage/lib/libboost_unit_test_framework.so.1.48.0 #5 0x0000000000000000 in ?? () valgrind note: ==26968== Invalid free() / delete / delete[] / realloc() ==26968== at 0x4A20E72: operator delete[](void*) (vg_replace_malloc.c:490) ==26968== by 0x4C79767: void boost::checked_array_delete<char>(char*) (checked_delete.hpp:41) ==26968== by 0x4C79782: boost::checked_array_deleter<char>::operator()(char*) const (checked_delete.hpp:63) ==26968== by 0x4C798B0: boost::detail::sp_counted_impl_pd<char*, boost::checked_array_deleter<char> >::dispose() (sp_counted_impl.hpp:148) ==26968== by 0x4F86D77: boost::runtime::cla::parser::parse(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F65E6F: boost::unit_test::runtime_config::init(int&, char**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib/libb oost_unit_test_framework.so.1.48.0) ==26968== by 0x4F455E0: boost::unit_test::framework::init(bool (*)(), int, ch ar**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/li b/libboost_unit_test_framework.so.1.48.0) ==26968== by 0x4F59D7E: boost::unit_test::unit_test_main(bool (*)(), int, cha r**) (in /home/msi/vanvoors/code/test_cctbx/cctbx_sources/boost_1_48_0/stage/lib /libboost_unit_test_framework.so.1.48.0) ==26968== by 0x407377: main (unit_test.hpp:59) ==26968== Address 0x1 is not stack'd, malloc'd or (recently) free'd

2 7

Using iotbx.pdb as C++ only dynamic (shared) library
by vanv0059＠umn.edu 13 Dec '11

13 Dec '11

Hi, The iotbx.pdb Python module is very useful and fast. I am trying to use the C++ library and have gotten an example program to run correctly when compiling in the object files in cctbx_build/iotbx/pdb: atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o However, I would like to have a shared library that wraps all of the objects into shared library (or dynamic on Mac OSX). I have created such a library on my Mac using the command: g++ -dynamiclib -fPIC -O3 --fast-math -o libiotbx_pdb.dylib atom_selection.o construct_hierarchy.o hierarchy.o hierarchy_atoms.o hierarchy_select.o hierarchy_write.o hybrid_36_c.o hybrid_36_cpp.o input.o input_write.o overall_counts.o utils.o write_utils.o -install_name /Users/jvanvoorst/Python_VEs/Lore30/lib/libiotbx_pdb.dylib -Wl,-single_module When I link my program to that library I always get a segment fault in the deconstructor of the last instance of iotbx::pdb::hierarchy::root::~root(). gdb traceback is: Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x000000000000001a 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () (gdb) where #0 0x00000001000030e6 in std::vector<iotbx::pdb::hierarchy::model, std::allocator<iotbx::pdb::hierarchy::model> >::~vector () #1 0x00000001000035fe in iotbx::pdb::hierarchy::root_data::~root_data () #2 0x0000000100003662 in boost::checked_delete<iotbx::pdb::hierarchy::root_data> () #3 0x0000000100003687 in boost::detail::sp_counted_impl_p<iotbx::pdb::hierarchy::root_data>::dispose () #4 0x0000000100001342 in boost::detail::sp_counted_base::release () #5 0x0000000100001372 in boost::detail::shared_count::~shared_count () #6 0x000000010000149b in boost::shared_ptr<iotbx::pdb::hierarchy::root_data>::~shared_ptr () #7 0x00000001000014e9 in iotbx::pdb::hierarchy::root::~root () #8 0x00000001000053c9 in FooBar::~FooBar () #9 0x0000000100001150 in main () I can send you my program but I have pruned FooBar down to only two class variables (one for a root object and the other for the input). All the class does is read the input from a file and construct the hierarchy. The seg fault only happens on the cleanup of the last object. That is if I vary the number of FooBar instances, the program faults on the cleanup of the last root object. I tried numerous combinations of flags to the linker (g++), but (as expected) the result is always the same. I am using "g++ -I{include for sources} -I{include for build} test.C libiotbx_pdb.dylib" to compile the program. I am likely overlooking something. Also, I am not proficient in debugging dynamic libraries via gdb. Therefore, any tips or pointers would be much appreciated. If there is an edit I can make to my Sconscript to automagically generate the shared library I could use that as well. The libtbx/configure.py output looks right (Static libraries: False). However, when running scons using -Q the source files are getting compiled with -fPIC (good), but the libraries are getting created using ar (implies static libraries) -- unless I am looking in the wrong directory for the shared libraries (cctbx_build/lib). Thanks, Jeff Van Voorst

2 8

Problems building scitbx.random
by Jan Marten Simons 29 Nov '11

29 Nov '11

Hi, After doing a fresh checkout with these commands: > #! /bin/csh -fe > set verbose > curl http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx -o cctbx_bundle.selfx > perl cctbx_bundle.selfx 0 > rm cctbx_bundle.selfx > rm cctbx_install_script.csh > mv cctbx_sources sources > cd sources > svn co https://cctbx.svn.sourceforge.net/svnroot/cctbx/trunk cctbx_project > cctbx_project/libtbx/development/move_obsolete_from_bundle.csh And using these commands to configue and build cctbx: > mkdir -p build > cd build > python ../sources/cctbx_project/libtbx/configure.py -- enable_openmp_if_possible=True --scan_boost --build=release boost_adaptbx cctbx chiltbx crys3d iotbx mmtbx rstbx ucif smtbx clipper_adaptbx fable scitbx > source setpaths.sh > make || ( echo "Make failed!"; exit 1 ) I encounter these errors: Processing: "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py" Traceback (most recent call last): File "../sources/cctbx_project/libtbx/configure.py", line 21, in <module> run() File "../sources/cctbx_project/libtbx/configure.py", line 17, in run libtbx.env_config.cold_start(sys.argv) File "/home/marten/cctbx/cctbx- dev/sources/cctbx_project/libtbx/env_config.py", line 2061, in cold_start env.refresh() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1401, in refresh module.process_libtbx_refresh_py() File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/libtbx/env_config.py", line 1678, in process_libtbx_refresh_py execfile(abs(custom_refresh), {}, {"self": self}) File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/libtbx_refresh.py", line 3, in <module> from crys3d import qttbx File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/crys3d/qttbx/__init__.py", line 4, in <module> import gltbx.util File "/home/mu/cctbx/cctbx-dev/sources/cctbx_project/gltbx/util.py", line 1, in <module> import scitbx.array_family.flex # import dependency File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/array_family/flex.py", line 10, in <module> import scitbx.random File "/home/mu/cctbx/cctbx- dev/sources/cctbx_project/scitbx/random/__init__.py", line 24, in <module> mt19937 = mersenne_twister_19937(value=get_random_seed()) NameError: name 'mersenne_twister_19937' is not defined To me these look like scitbx.random-ext is not built. Is this a regression from the changes to the build system or am I doining something wrong here? Cheers, Jan

2 3

Using cctbx as a C++ only library
by Martin Uhrin 26 Nov '11

26 Nov '11

Dear Cctbx community, thank you very much for the great library, full of wonderful algorithms. I'm writing a structure searching engine in C++ to be used as a high throughput method to do energy landscape sampling with a view to finding new ground state crystal structures. Key requirements of this project are efficiency and ease of development & deployment. Cctbx has many useful algorithms and data structures that I would like to avoid having to re-code if possible. Thus I've downloaded and played with the source as it is but have found it difficult to integrate with my project (for understandably reasons given the goals of cctbx). My main goals regarding ease of development & deployment mean that it would be very inconvenient to distribute the entire cctbx with my code. Ideally I'd like to achieve the following: * strip out everything python related (apart from maybe the build system) * strip out all modules (namespaces) that are unused by my code leaving a fairly bare-bones C++ library that is quick and easy to build. Is anyone able to comment on how feasible this might be? Is this even worth attempting? Many thanks, -Martin -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

2 3

cctbx + PyPy?
by Jan Marten Simons 25 Nov '11

25 Nov '11

Hi, PyPy 1.7 was released just a few days ago: http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html After taking a look at http://speed.pypy.org/ I'm wondering if it would be possible to get cctbx to work with PyPy-jit instead of CPython. Perhaps it's a little bit early, as they are still working on a complete numpy integration and I'm not sure how well boost.python fits into this. But in general speeding up the python parts looks nice if one does not have to invest too much time into this. Has any of you already done some tests with PyPy? Cheers, Jan

2 1

crystal packing/unit cell question
by Jessica Grant 08 Nov '11

08 Nov '11

I am new to cctbx and crystallography in general. I am trying to take a pdb file and produce a representation of the crystal, similar to the symmetry mates produced in pymol. I have used cctbx to apply the symmetry operators, but this gives atomic coordinates that are quite spread out. I think what I want to do is to fill the unit cell, or in some way extract crystal packing information. I would like, in the end, to be able to build files with a physically realistic representation of different sized crystals, so I can go on to study the interactions of molecules within those crystals. Is there a way to do this in cctbx? Thanks! Jessica

5 11

flex index
by Monarin Uervirojnangkoorn 18 Oct '11

18 Oct '11

Hi, I'm using flex and 'd like to get an access to elements fall beyond/ below certain value. eg. a=[1,4,5,7,5,6,6,1] anything beyond 5 should be set to 5. (in matlab then a(a>5)=5;) if you could tell me how to do this quickly for flex, that would be great. for now i use loop and that is... pretty slow. many thanks. mona ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^ Institute of Biochemistry, Institut für Neuro- und Bioinformatik, Graduate School for Computing in Medicine and Life Sciences University of Lübeck, Ratzeburger Allee 160 Lübeck 23538 Germany Tel: +49451-5004072 Fax: +49451-5004068

2 3