- cctbxbb - phenix-online.org

Explicit import for iotbx.pdb.amino_acid_codes
by Claudia Millán Nebot 01 Oct '12

01 Oct '12

Hello everyone :) I've just started to look into the cctbx facilities, and as i'm a newbie in both object oriented programming and in general to the system, i'm tryng to play around and find by myself how things are working. Using the cctbx.python, i realised that it is necessary to import explicitly the module amino_acid_codes, while for others like atom_name_interpretation this is done straigthforward if you do an import from a lower level (like from iotbx.pdb import *). Is there some good reason because this has to be like that? Thanks in advance for the answer, and excuse me if is it is a question too simple! Claudia Millán (cmncri(a)ibmb.csic.es) Institut de Biologia Molecular de Barcelona (IBMB-CSIC) Barcelona, Spain

2 2

Build failing
by Graeme.Winter＠diamond.ac.uk 01 Oct '12

01 Oct '12

Hi Folks, Following an svn update and several attempts to build, I am still getting this compile error /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:11: error: 'bessel_y_small_z_series' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'template<class T, class Policy> T boost::math::detail::bessel_y_small_z_series(T, T, T*, const Policy&)' declared here, later in the translation unit In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17:0, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:11: error: 'bessel_j_small_z_series' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'template<class T, class Policy> T boost::math::detail::bessel_j_small_z_series(T, T, const Policy&)' declared here, later in the translation unit In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17:0, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:11: error: 'bessel_yn_small_z' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jn.hpp:17:0, from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:18, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9, from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'template<class T, class Policy> T boost::math::detail::bessel_yn_small_z(int, T, T*, const Policy&)' declared here, later in the translation unit scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 26.64 seconds, ticks: 51849408, micro-seconds/tick: 0.514 wall clock time: 8 minutes 5.55 seconds (485.55 seconds total) make: *** [default] Error 2 OS X Lion machine Graemes-MacBook-Pro:build graeme$ uname -a Darwin Graemes-MacBook-Pro.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 Graemes-MacBook-Pro:build graeme$ gcc --version gcc (GCC) 4.7.0 20120314 (prerelease) Copyright (C) 2012 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Have I missed something? Nightly builds seem to be OK. Commands ../../cctbx_plus/base/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ../sources/cctbx_project/libtbx/configure.py --compiler=gcc labelit cctbx bpcx_regression rstbx . setpaths.sh make Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 8

dropping Python 2.4 support?
by Nathaniel Echols 27 Sep '12

27 Sep '12

Hi all-- Will it ruin anyone's work if we make Python 2.5 the minimum platform for using CCTBX? 2.4 is now almost eight years old and the lack of certain newer features (decorators, 'with') is starting to hold us back. thanks, Nat

2 2

Enabling OpenMP for phaser on OS X
by William G. Scott 17 Sep '12

17 Sep '12

Hi folks: I found if I created a symbolic link from a g++4.7 compiler to /usr/bin/g++, I could "trick" phaser into compiling with OpenMP enabled. This is kind of an ugly hack. How do I force it to use /sw/bin/g++-4 ? Is there some place in the source code for cctbx (or phaser) that I can make such a change? Thanks. Bil William G. Scott Professor Department of Chemistry and Biochemistry and The Center for the Molecular Biology of RNA 228 Sinsheimer Laboratories University of California at Santa Cruz Santa Cruz, California 95064 USA

2 4

Quick start info?
by esserlo＠helix.nih.gov 11 Sep '12

11 Sep '12

Hi all, this is a question about how to get started quickly. I have looked at the sourceforge site for cctbx and read diverse documents but so far I have not found documentation that would help me get started fast. I have a crystallographic problem which I can probably program from scratch but then again why bother if there is a crystallographic tool box around. But well despite all the power in it getting it to work seems a bit daunting. Are there really good examples? I did get the "sand box quartz model" script to work but this is still far off of what I want to do. Thanks for all advice and sorry if I missed the 'obvious' one. Lothar

4 4

Question about sigma(I) calculation in merge_equivalents
by Keitaro Yamashita 07 Sep '12

07 Sep '12

Dear CCTBX developers, I have a question about the method in CCTBX to calculate sigma(averaged I) when merging. In cctbx/miller/merge_equivalents.h (line 594 in phenix-dev-1148), sigma(averaged I)^2 will be calculated by max(mv.gsl_stats_wvariance()/values.size(), 1/mv.sum_weights()). It takes maximum value of the two options. I know the latter equation - this is just sigma(averaged I)^2 = 1/sum(w), where w = 1/sigma(I)^2. And I believe this equation is widely used in other crystallographic programs e.g. XDS. The former is, according to the comment in the code, the emulation of gsl_stats_wvariance, and we can find the information from gsl website: http://www.gnu.org/software/gsl/manual/html_node/Weighted-Samples.html I don't know how this "wvariance" is derived. Is it a better estimate of sigma(averaged I)? Why does it take the maximum from the two options? I would like to know the theory of this sigma calculation in CCTBX. Best regards, Keitaro

3 5

Debian Package
by Radostan Riedel 05 Sep '12

05 Sep '12

Dear CCTBX Maintainers, Maybe some of you remember me last year asking on this mailing list[1]. I was asked by the Debian Science Team[2] to contribute my former work to get cctbx finally packaged in Debian. The last month I was working with Baptiste Carvello and Picca Frédéric-Emmanuel to get this done. Our packaging efforts + patches can be found in our git[3]. We organize our packaging efforts in a wiki[4]. But we don't want to overlook you and give some of our work back into the main project. We have a few patches to give back upstream that would really help us and hopefully help to increase the cctbx user community. To explain a little bit what we've done. First of all we tried respectfully not to change too much about your build system. We integrated a few options in libtbx/env_config which shouldn't break anything. The primary changes are: * Option to check and use system libraries (in case any library is already installed no need to compile the bundled shlib) * Install targets for headers and shlibs and binaries: calling "scons install" * Shlib versioning: This is achieved by utilizing libtool (the GNU standard) * --install-prefix, --install-destdir and --libdir option * support for LD_FLAGS in --use_environment_flags option * Fallback to avoid using "LIBTBX_BUILD" env variable after install * Full integration of your scons build system in distutils: setup.py and sconsutils.py. You can just call 'python setup.py install' * Adaption of your test system to be called by distutils * Autogenerate pkg-config files for shlibs What we now need to know is if our changes are appreciated or if we need to spend more work on this. Keep in mind that our work is still work in progress! One big concern is always the ABI/API stability. But if we can count on the shlib versioning as mentioned here[5] it would help us and other developers a lot. Libtool should be available on most systems, works with common compilers and it provides the quasi standard for versioning shlibs. We have other issues too but I guess I won't overkill this mail to much for now. I hope we can discuss and work on this together. Please tell us your honest opinion and critics. kind regards Radostan Riedel P.S: This work is intended to work on other distributions too. We also hope that Fedora, openSUSE, Gentoo etc. can benefit from it too. [1] http://phenix-online.org/pipermail/cctbxbb/2011-June/000192.html [2] http://wiki.debian.org/DebianScience [3] http://anonscm.debian.org/gitweb/?p=debian-science/packages/cctbx.git;a=sum… [4] http://wiki.debian.org/DebianScience/cctbx [5] http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.…

12 84

Boost.Extension plugins in CCTBX
by Graeme.Winter＠diamond.ac.uk 20 Aug '12

20 Aug '12

Hi Folks, It seems that Boost includes (or may include) a mechanism to allow compiled plugins see for example http://boost-extension.redshoelace.com/docs/boost/extension/boost_extension… - is this in cctbx? is anyone interested in it being available? It seems useful from where I am sat. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Removal of Boost.Thread support
by Luc Bourhis 17 Aug '12

17 Aug '12

Hi fellow cctbx developers, I am going to ditch support for Boost.Thread and therefore to remove the SF computation parallelised with Boost.Thread as well. Parallelisation at the C++ level should be done with OpenMP: Boost.Thread was merely an early experimentation that was kept alive for no particularly good reasons. If anybody disagrees with that move, please voice your opinion now. Best wishes, Luc J. Bourhis

6 15

single or double precision ?
by Pascal 17 Aug '12

17 Aug '12

Hi, Is cctbx doing all the calculations in double or single precision? I am intersted in the direct method of the structure factors calculation. >From what I saw it's double precision but I am not sure. Regards, Pascal

2 2

Boost.Extension plugins in CCTBX
by Graeme.Winter＠diamond.ac.uk 16 Aug '12

16 Aug '12

Hi Folks, It seems that Boost includes (or may include) a mechanism to allow compiled plugins see for example http://boost-extension.redshoelace.com/docs/boost/extension/boost_extension… - is this in cctbx? is anyone interested in it being available? It seems useful from where I am sat. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

from __future__ import division has become compulsory
by Luc Bourhis 15 Aug '12

15 Aug '12

Dear cctbx developers, as discussed here a few days ago, it has been decided that every single Python module must feature from __future__ import division A mega commit of mine has just taken care of existing code. For future modules, libtbx.find_clutter will complain if that line is missing. A script libtbx.add_from_future_import_division is provided that can add that line correctly. The rationale is that: i. as you know, we want to use the so-called new division semantics in cctbx modules (i.e. 1/2 == 0.5 instead of 1/2 == 0 as per the old semantics for the record) ii. the current solution passing -Qnew on the Python command line in the dispatchers libtbx.python, cctbx.python, etc is not always appropriate as there are people out there using cctbx as yet-another-library in a context where some code would not correctly run with the new division semantics, therefore barring the use of -Qnew The only solution to reconcile i. and ii. is therefore to add that __future__ line in every single Python module committed to the cctbx. Note however that the aforementioned dispatchers do still use -Qnew so that Phenix developers do not need to put that very line in their Phenix code. Hopefully, we can make everybody reasonably happy with those compromises. Best wishes, Luc

1 0

Deprecation warning?
by Graeme.Winter＠diamond.ac.uk 09 Aug '12

09 Aug '12

Is there a way to avoid this? Using matplotlib.pyplot. /Users/graeme/svn/cctbx_plus/base/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/backends/backend_wx.py:1382: wxPyDeprecationWarning: Using deprecated class PySimpleApp. wxapp = wx.PySimpleApp() Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Problems compiling cctbx on OS X (10.8.0)
by William G. Scott 03 Aug '12

03 Aug '12

Hi folks: I am having the same problem compiling cctbx stand-alone, or within CCP4. I've posted the error log here: http://fennario.ucsc.edu/~wgscott/temp/cctbx_compile_errors.log Any suggestions what to fix? Thanks. Bill William G. Scott Professor Department of Chemistry and Biochemistry and The Center for the Molecular Biology of RNA 228 Sinsheimer Laboratories University of California at Santa Cruz Santa Cruz, California 95064 USA

7 17

Invert hand of shelxd res file from cctbx
by Graeme.Winter＠diamond.ac.uk 23 Jul '12

23 Jul '12

Hi Folks, This should be easy but I can't find anything! I have a .res file: [gw56@ws050 fast_ep]$ cat sad_fa.res REM Best SHELXD solution: CC 41.11 CC(weak) 23.51 CFOM 64.62 REM TITL sad_fa.ins SAD in P422 CELL 0.98000 78.17 78.17 37.87 90.00 90.00 90.00 LATT -1 SYMM -Y+1/2,X+1/2,Z+1/4 SYMM -X,-Y,Z+1/2 SYMM Y+1/2,-X+1/2,Z+3/4 SYMM X+1/2,-Y+1/2,-Z+3/4 SYMM Y,X,-Z SYMM -X+1/2,Y+1/2,-Z+1/4 SYMM -Y,-X,-Z+1/2 SFAC SE UNIT 256 SE01 1 0.600227 0.892868 0.042187 1.0000 0.2 SE02 1 0.627518 0.852661 0.427392 0.8667 0.2 SE03 1 0.867409 0.911667 0.394418 0.8420 0.2 SE04 1 0.489761 0.805634 0.391009 0.7279 0.2 SE05 1 0.642792 0.649048 0.369694 0.7172 0.2 SE06 1 0.678879 0.535027 0.477918 0.4571 0.2 SE07 1 0.609207 0.572754 0.421746 0.4212 0.2 SE08 1 0.584526 0.536499 0.322047 0.4154 0.2 SE09 1 0.843063 0.846794 0.480296 0.3831 0.2 SE10 1 0.622429 0.743332 0.157899 0.3604 0.2 SE11 1 0.626114 0.528595 0.644789 0.3590 0.2 HKLF 3 END which I would like to read, invert the hand of and write as a pdb file. I got stuck trying to work through the example / test code. Can someone point me in the right direction? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 4

rt_mx inconsistencies
by Luc Bourhis 19 Jul '12

19 Jul '12

Hi fellow cctbx developers, class rt_mx, contrarily to the documentation, does not only hold rotation + translation but any transformation y = M x + t where M is an invertible matrix. As a result, rt_mx can be used to store both space group elements and change-of-basis operators (the latter are in general not rotations). Not the purest design but apart from the recurrent need of improving on the documentation, it has been all right. (1) However, revision 15082 has introduced an inconsistency in the comparison of rt_mx instances related to that divide between rotations and non-rotations. Let's r1 and r2 be rotations and m1 and m2 be non-rotations. After that revision, r1 == r2 works all right r1 == m1 throws an exception m1 == m2 throws an exception As far as I know the problem has been first reported by Phil Evans who provided a very clear regression test. The reason is simple: that revision changed operator== to compare the geometric elements, assuming the compared operators are rotations (and therefore that they have an invariant axis or plane). (2) Another side effect of revision 15082 seems to result in r1 == r2 being true even if r1.t() and r2.t() do not have the same denominator. Before that revision, this comparison used to return false. As part of that revision, one test has been changed from != to == for that very reason. (3) revision 15082 follows revision 14990 that introduces an order on rt_mx intances. That order is used to implement a new symmetry-equivalent pair interaction table and rightly so op1 == op2 is made consistent with op1 < op2 and op1 <= op2 by revision 15082. So I am coming to you to discuss what to do. I am of the opinion that (1) needs fixing. I am also of the opinion that (2) is a good move. Here is what I propose to do: (i) Remove operator <=, >=, <, > from class rt_mx and revert operator== to what it used to be. (ii) Change rot_mx and tr_vec operator== to disregard the value of den() (iii) Introduce a proxy class that wraps around rt_mx and that obeys the ordering introduced by revision 14990 (iv) Rewrite the implementation of the symmetry-equivalent pair interaction table generation to use that proxy instead of directly comparing space group symmetry elements. Basically I propose a very orthogonal design that keeps the core sgtbx pristine except for the fixing of a small but long-standing glitch while encapsulating the feature needed by one single algorithm along with that algorithm. What do you think? Best wishes, Luc Bourhis

3 8

cctbx.miller.complete_array() bug?
by Jan Marten Simons 13 Jul '12

13 Jul '12

Hi, I think I found a bug or missing feature in cctbx.miller.complete_array: As I suppose the complete_array function should check if a set of reflections is complete and add missing indices with the specified values. The bug is now when a set contains symmetrically equivalent reflections e.g. in P4 (1, 0, 2) and (0, 1, 2). The sample code below generates this output for the atached data file: --- f_obs read in: Miller array info: None Observation type: xray.amplitude Type of data: double, size=8 Type of sigmas: None Number of Miller indices: 8 Anomalous flag: False Unit cell: (4, 4, 4.44, 90, 90, 90) Space group: P 4 (No. 75) Systematic absences: 0 Centric reflections: 3 Resolution range: 4.44 1.94109 Completeness in resolution range: 1 Completeness with d_max=infinity: 1 (0, 0, 1) 4.10852771683 (0, 1, 0) 4.34856298103 (0, 1, 1) 10.0 (1, 1, 0) 0.911043357914 (1, 1, 1) 9.12523972288 (0, 0, 2) 5.97578446733 (0, 2, 0) 7.10492786733 (1, 0, 2) 2.17485631709 --- f_obs completed: Miller array info: None Observation type: xray.amplitude Type of data: double, size=9 Type of sigmas: None Number of Miller indices: 9 Anomalous flag: False Unit cell: (4, 4, 4.44, 90, 90, 90) Space group: P 4 (No. 75) Systematic absences: 0 Centric reflections: 3 Resolution range: 4.44 1.94109 (0, 0, 1) 4.10852771683 (0, 1, 0) 4.34856298103 (0, 1, 1) 10.0 (1, 1, 0) 0.911043357914 (1, 1, 1) 9.12523972288 (0, 0, 2) 5.97578446733 (0, 2, 0) 7.10492786733 (1, 0, 2) 2.17485631709 (0, 1, 2) 0.0 ---- sample code ------- obs_file = "Model02.hkl" xtal_symm = crystal.symmetry( unit_cell=(4.000, 4.0, 4.44, 90.0, 90.0, 90.0), space_group_symbol="P 4") # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("intensities="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() I_obs.merge_equivalents() f_obs = I_obs.as_amplitude_array() print("\n--- f_obs read in:") f_obs.show_comprehensive_summary() f_obs.show_array() d_max, d_min = f_obs.resolution_range() # add missing reflections with 0.0 intensity (= not observed) f_obs = f_obs.complete_array(d_min=d_min, d_max=d_max, new_data_value=0.0, new_sigmas_value=1.0).sort() print("\n--- f_obs completed:") f_obs.show_comprehensive_summary() f_obs.show_array() ---- end sample code ------- As you can see the index (0, 1, 2) is added with zero amplitude where instead the index (1, 0, 2) should be renamed to (0, 1, 2) as those two are symmetrically equivalent for any tetragonal spacegroup (like P4). So to me it looks like the complete_array method does not honor symmetrically equivalent indices at all, where it should imho. What do you think on this issue? With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University

7 8

Getting r1_factor of miller arrays with different count of miller indices
by Jan Marten Simons 06 Jul '12

06 Jul '12

Hi, I'm facing a new challenge while working with cctbx: Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem: # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2") trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)])) # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array() f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents() R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False) gives: " assert other.indices().size() == self.indices().size() AssertionError " Is there some way to get the R1_factor for this kind of scenario? Cheers and thanks in advance, Jan

4 8

Fast way to enable/disable individual scatterers in structure factor calculation
by Jan Marten Simons 06 Jul '12

06 Jul '12

Hi, I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation. As I see from the source (structure_factors_direct.h) the selection toggle does not seem to be evaluated for this. Of course an additional check for the selection flag would harm performance in most standard cases, so I'd rather like to have a separate c++ function for this, which could be swapped in for the normal f_calc calculation by a boolean toggle in the python part of f_calc. As I'm not that much into the c++ part of cctbx, I'd really appreciate if somebody more knowledgeable could monkey patch this. Cheers, Jan

2 5

angle derivative of rotation matrix
by David Waterman 28 Jun '12

28 Jun '12

Hi, I'm using the axis_and_angle_as_r3_rotation_matrix method of scitbx.matrix.col in some Python code. Now I need to calculate the first derivative of the rotation matrix with respect to the rotation angle. Is there a function to do that somewhere in scitbx (or elsewhere in cctbx)? I had a brief look around but couldn't find anything that looked hopeful. If there isn't something there already I have some Fortran code from David Thomas that I could adapt to do this. Of course it wouldn't be very efficient in pure Python, and calculating the matrix and its derivative separately, but that's okay for now as the code is just for testing out some ideas. By the way, links to http://skal.planet-d.net/demo/matrixfaq.htm in the documentation (e.g. here<http://cctbx.sourceforge.net/current/c_plus_plus/namespacescitbx_1_1math_1_…>) are broken. I found a document with the same name here: http://vamos.sourceforge.net/matrixfaq.htm, but I have no idea if this has the same content as the original. Cheers -- David

3 4

Problem with direct_space_asu
by Jan Marten Simons 26 Jun '12

26 Jun '12

Hi, as there seems to be no documentation on the direct_space_asu module I just tried to use it like I thought the function names imply. The task I want to accomplish is checking whether a point in direct space is inside the asymmetric unit of a given unit cell. A small testcase I wrote reveals that apparently .is_inside(point) only uses fractional coordinates instead of cartesian coordinates. Is there a nice/easy way to get the desired behaviour? # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex import numpy as np asu = xray.structure( crystal_symmetry=crystal.symmetry( unit_cell=(5.01,5.01,5.47,95,85,90), space_group_symbol="P 1")).direct_space_asu() print(asu.show_comprehensive_summary()) print(asu.is_inside((0.0, 0.0, 0.0))) #should be True OK print(asu.is_inside((0.5, 0.5, 0.5))) #should be True OK print(asu.is_inside((0.9, 0.9, 0.9))) #should be True OK print(asu.is_inside((1.0, 0.9, 0.9))) #should be True is False! print(asu.is_inside((1.0, 1.0, 1.0))) #should be True is False! print(asu.is_inside((2.0, 2.0, 2.0))) #should be True is False! print(asu.is_inside((3.0, 3.0, 3.0))) #should be True is False! print(asu.is_inside((4.0, 4.0, 4.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 5.0))) #should be True is False! print(asu.is_inside((5.0, 5.0, 6.0))) #should be False OK print(asu.is_inside((6.0, 6.0, 6.0))) #should be False OK Cheers, Jan

2 1

compilation errors under gcc-4.7
by Justin Lecher 19 Jun '12

19 Jun '12

Hi, when compiling cctbx with gcc-4.7 I see following errors which I am not able to fix: cctbx_sources/scitbx/math/boost_python/distributions.cpp:112:59: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:86:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp: In instantiation of 'static void scitbx::math::{anonymous}::non_member_function_wrappers<FloatType, Distribution>::wrap() [with FloatType = double; Distribution = boost::math::students_t_distribution<double>]': cctbx_sources/scitbx/math/boost_python/distributions.cpp:116:63: required from here cctbx_sources/scitbx/math/boost_python/distributions.cpp:83:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:85:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:90:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:91:7: error: overloaded function with no contextual type information cctbx_sources/scitbx/math/boost_python/distributions.cpp:92:7: error: overloaded function with no contextual type information Any suggestions to fix? Thanks, justin -- Justin Lecher Institute of Complex Systems ICS-6 Structural Biochemistry Research Centre Juelich 52425 Juelich, Germany phone: +49 2461 61 2117

5 8

Re: [cctbxbb] compilation errors under gcc-4.7
by Luc Bourhis 19 Jun '12

19 Jun '12

> I take it you're interested in fixing this? Yes. I've got a Fedora 17 virtual machine. Luc

1 0

proposal to remove iotbx.shelx.from_ins
by Mirko Scholz 12 Jun '12

12 Jun '12

Hello, since I am new here it seems an odd proposal, but could you not remove iotbx.shelx.from_ins.from_ins since its functionality is covered much better by xray.structure.from_shelx (at least it seems to me). The following is roughly identical with the first statement being inaccurate (anisotropic vs isotropic displacements parameters) and wrong (structures containing disorder) xs0 = iotbx.shelx.from_ins.from_ins(file_name='foo.ins') fc0 = xs0.structure_factors(d_min=0.7, algorithm='direct').f_calc() xs1 = cctbx.xray.structure.from_shelx(file=open('foo.ins')) fc1 = xs1.structure_factors(d_min=0.7, algorithm='direct').f_calc() As someone who (seldom) uses cctbx I stumbled first over iotbx.shelx.from_ins and only noticed after a while that it does not handle disorder properly (problem is in ins_record.if_10_added). from_ins is used only here ./iotbx/command_line/emma.py ./cctbx/adp_restraints/tst_ext.py Where it should be possible to substitute it though I have not tested it. Cordially Mirko Scholz

4 4

controlling types over the C++/Python interface
by David Waterman 16 May '12

16 May '12

Hi, I'm starting to explore the use of boost.python in cctbx. I've written a simple C++ class and exposed its accessor methods as attributes using the .add_property syntax in the wrapper. In the C++ class my member data and accessor return types are, for example, scitbx::vec3<double>. When I access the attribute in Python with the usual dot operator I get a tuple. What I'd really like though is an object of type <class 'scitbx.matrix.col'>, which is what I passed to the constructor of the object in Python. So the conversion from Python to C++ does what I want, but not the other way round. How do I take control of the type of the attribute in Python, without having to explicitly construct a scitbx.matrix.col from the tuple? It's the same case if I use .def and expose the accessor as a method instead of an attribute. I admit I don't fully understand the details of conversions in either direction. I found I had to add from cctbx.array_family import flex to the package __init__.py where my C++ extension is imported, so that I could pass scitbx.matrix.col objects (or scitbx.matrix.rec, or tuples) to the constructor in Python and not get a Boost.Python.ArgumentError. I've not yet figured out what this import is doing though. Apologies for the remedial question. These are still early days for me. Thanks -- David

5 12