- cctbxbb - phenix-online.org

Re: [cctbxbb] Adding columns to new *unmerged* MTZ object
by Graeme.Winter＠diamond.ac.uk 13 Feb '14

13 Feb '14

In related news…. Should void object::adjust_column_array_sizes(int new_nref) { CMtz::MTZ* p = ptr(); if (!p->refs_in_memory) return; if (new_nref > p->nref) { reserve(new_nref); for(int i=0;i<p->nxtal;i++) { for(int j=0;j<p->xtal[i]->nset;j++) { for(int k=0;k<p->xtal[i]->set[j]->ncol;k++) { CMtz::MTZCOL* col_k = p->xtal[i]->set[j]->col[k]; int old_size = column_array_size(col_k); if (new_nref > old_size) { ccp4array_resize(col_k->ref, new_nref); for(int iref=old_size;iref<new_nref;iref++) { *(reinterpret_cast<union float_uint_uchar*>( &col_k->ref[iref])) = not_a_number_value_; } } } } } } } assign nref? otherwise repeated calling of this will keep on re-allocating the arrays Thanks Graeme On 12 Feb 2014, at 10:40, Graeme Winter <graeme.winter(a)diamond.ac.uk> wrote: > All, > > By adding > > m_out.adjust_column_array_sizes(len(mi)) > > to the bpl wrapper and defining > > m_out.set_n_reflections(len(mi)) > > in the object.h and bpl wrapper I can do what I want… > > I will add some tests and some warnings that this may make nonsense of the values… however before committing any comments on the interface? or behaviour? > > Thanks Graeme > > > > On 12 Feb 2014, at 09:11, <Graeme.Winter(a)diamond.ac.uk> <Graeme.Winter(a)diamond.ac.uk> wrote: > >> Hi Folks, >> >> I am trying to create a new MTZ file containing unmerged and it seems to be less simple than I would think - set_reals() on a column requires a set of Miller indices and values and does a lookup internally (which I don’t want for unmerged values) and set_values() requires that the array is already the right size - when it starts off as size 0 and I have no idea how to make enough room in there from the Python API… >> >> Any clues? I have done quite a chunk of grepping around to try and figure this one out and have run out of ideas… >> >> Thanks Graeme_______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> http://phenix-online.org/mailman/listinfo/cctbxbb >

3 11

Adding columns to new *unmerged* MTZ object
by Graeme.Winter＠diamond.ac.uk 12 Feb '14

12 Feb '14

Hi Folks, I am trying to create a new MTZ file containing unmerged and it seems to be less simple than I would think - set_reals() on a column requires a set of Miller indices and values and does a lookup internally (which I don’t want for unmerged values) and set_values() requires that the array is already the right size - when it starts off as size 0 and I have no idea how to make enough room in there from the Python API… Any clues? I have done quite a chunk of grepping around to try and figure this one out and have run out of ideas… Thanks Graeme

1 1

building with external boost
by Marcin Wojdyr 30 Jan '14

30 Jan '14

Hi all, would it be ok if I added an option to use installed boost libraries? Debian and gentoo patches add boolean --use_system_libs flag. I could use the same, but with a list of arguments (at first only --use_system_libs=boost). Marcin

2 2

boost python and metaclass inheritance
by David Waterman 10 Jan '14

10 Jan '14

Hi, Does anyone know how to use __metaclass__ specification alongside inheritance from boost python classes? This code illustrates the problem: import abc > from scitbx.lstbx import normal_eqns > class Refinery(object): > __metaclass__ = abc.ABCMeta > pass > class AdaptLstbx(Refinery, normal_eqns.non_linear_ls): > pass The result is Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python2.7/abc.py", line 87, in __new__ > cls = super(ABCMeta, mcls).__new__(mcls, name, bases, namespace) > TypeError: Error when calling the metaclass bases > metaclass conflict: the metaclass of a derived class must be a > (non-strict) subclass of the metaclasses of all its bases The real Refinery is a class that has one method that must be supplied by a concrete subclass. I like the idea of making it impossible to instantiate a base Refinery rather than relying on e.g. raise NotImplementedError() in that method definition. This is what ABCMeta would give me, if only I could get it to work here. Google found me some answers that require me to know the __metaclass__ of the bases (as suggested by the error message), but normal_eqns.non_linear_ls.__metaclass__ does not exist. I don't terribly mind the raise NotImplementedError() alternative, but I thought that if I can easily use ABCMeta, I'd like to. Cheers -- David

2 5

cctbx.python unnecessarily unfriendly
by James Stroud 10 Jan '14

10 Jan '14

Hello CCTBX developers, I noticed in the cctbx.python script the following lines that call the python interpreter if [ -n "$LIBTBX__VALGRIND_FLAG__" ]; then exec $LIBTBX_VALGRIND "$LIBTBX_PYEXE" -Qnew "$@" elif [ $# -eq 0 ]; then exec "$LIBTBX_PYEXE" -Qnew else exec "$LIBTBX_PYEXE" -Qnew "$@" fi Here, the “-Qnew” flag forces the python interpreter to work with new division (where “/“ does not strictly mean integer division). This flag basically overrides the expected python behavior to which python programmers have been writing code for over a decade, wrecks some dependencies, and stifles new code that itself depends on cctbx. It is highly frustrating to work around this highly unnecessary flag!! Fortunately, python has a better way to handle this problem by just having “from __future__ import division” at the top of .py files. In fact, I have spot-checked several files in the cctbx source tree and noticed that the ALL have this line at the top. So why does the -Qnew flag still exist in the cctbx.python script? It is entirely unnecessary and requires any and all dependencies to be programmed to a behavior that is not standard for the language. James p.s. It is not reasonable to ask users who install your packages to patch the cctbx.python script or to patch dependencies that don’t conform to this non-standard behavior.

6 13

building cctbx with external boost
by Marcin Wojdyr 19 Dec '13

19 Dec '13

Hi I vaguely remember a discussion some time ago about building cctbx with stand-alone boost libraries. Do you know what boost versions are compatible? Is using the --scan-boost option enough? I see that Gentoo has extra patch to unbundle boost from cctbx: http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sci-libs/cctbx/file… Is it necessary? I'm asking because at this moment we have in CCP4 two versions of boost, one in cctbx and the other for building DSSP (it uses different subset of boost libs than cctbx). It would make sense to use a single version for both, if possible. Cheers Marcin

2 3

Re: [cctbxbb] cctbxbb Digest, Vol 59, Issue 1
by Zhenwei Luo 08 Dec '13

08 Dec '13

Hi Nat, Thanks for your reply! I have tried the first method you mentioned--adding an index value to the return method. It did work if I only needed a single array for future calculation. But I found it's not very convenient for my program. It's convenient and more efficient if other classes in it could process all the one-dimensional arrays at the same time. Otherwise, I needed to write more control code in python, and the c++ part could only deal with a single array now. Therefore, I hoped that I could pass such a multidimensional array in my program. Sincerely, Zhenwei On Sun, Dec 8, 2013 at 2:00 PM, <cctbxbb-request(a)phenix-online.org> wrote: > Send cctbxbb mailing list submissions to > cctbxbb(a)phenix-online.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://phenix-online.org/mailman/listinfo/cctbxbb > or, via email, send a message with subject or body 'help' to > cctbxbb-request(a)phenix-online.org > > You can reach the person managing the list at > cctbxbb-owner(a)phenix-online.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of cctbxbb digest..." > > > Today's Topics: > > 1. Passing array between python and C++ (Zhenwei Luo) > 2. Re: Passing array between python and C++ (Nathaniel Echols) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 7 Dec 2013 22:25:08 -0600 > From: Zhenwei Luo <tluozhenwei(a)gmail.com> > To: cctbxbb(a)phenix-online.org > Subject: [cctbxbb] Passing array between python and C++ > Message-ID: <90518A19-AC1B-493B-8B27-79D31EDDF055(a)gmail.com> > Content-Type: text/plain; charset=us-ascii > > Hello Everyone, > > I am writing a refinement program using the cctbx. I have > encountered with a problem when I wanted to pass an array between c++ and > python. The type of array I used in my c++ source file is > af::shared<af::shared<vec3> >, I would like to pass it into my python > script, thus I could do some other operations to this array. > I wrote a method in my class to return this array. > > af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } > > I wrote another method in the *_ext file to wrap this method to make it > callable from python. > > .def("return_modes", &nm_init::return_modes) > > But when I use it in python, > eigenvector = nm_init_manager.return_modes() > > It gave me an error: > > TypeError: No to_python (by-value) converter found for c++ type: > scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > > > What I want to realize is just creating an two dimensional array using > vec3 as its element. I can also using the select method in this array to > select some elements from its column, just like in the > af::shared<vec3<double> > array. Namely, the column of this array should > have some same properties like af::shared<vec3<double> >. Can anyone help > me figure out what I should do? > > Thank you very much! Any help will be greatly appreciated. > > Sincerely, > Zhenwei Luo > Rice University > > > ------------------------------ > > Message: 2 > Date: Sat, 7 Dec 2013 20:45:10 -0800 > From: Nathaniel Echols <nechols(a)lbl.gov> > To: cctbx mailing list <cctbxbb(a)phenix-online.org> > Subject: Re: [cctbxbb] Passing array between python and C++ > Message-ID: > <CALeAa1MOL1wbW5Vb=M= > jXsnGTaCBQNJjcOzDwMhM27x5PP+XNA(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > On Sat, Dec 7, 2013 at 8:25 PM, Zhenwei Luo <tluozhenwei(a)gmail.com> wrote: > > > I am writing a refinement program using the cctbx. I have > > encountered with a problem when I wanted to pass an array between c++ and > > python. The type of array I used in my c++ source file is > > af::shared<af::shared<vec3> >, I would like to pass it into my python > > script, thus I could do some other operations to this array. > > I wrote a method in my class to return this array. > > > > af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } > > > > I wrote another method in the *_ext file to wrap this method to make it > > callable from python. > > > > .def("return_modes", &nm_init::return_modes) > > > > But when I use it in python, > > eigenvector = nm_init_manager.return_modes() > > It gave me an error: > > TypeError: No to_python (by-value) converter found for c++ type: > > scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > > > > > What that error means is that boost.python doesn't know how to present this > new type to Python. For the data structures that we pass back and forth, > we have to explicitly provide wrappers for them - including new > instantiations of scitbx::array_family templates. > > What I want to realize is just creating an two dimensional array using vec3 > > as its element. I can also using the select method in this array to > select > > some elements from its column, just like in the af::shared<vec3<double> > > > array. Namely, the column of this array should have some same properties > > like af::shared<vec3<double> >. Can anyone help me figure out what I > should > > do? > > > > In general the scitbx::array_family templates aren't really intended to > every be multidimensional; for inherently 2D and 3D data types like images > and maps, we use flex.grid to provide a multi-dimensional accessor on top > of the 1D array. The grid may not be the most appropriate in your case but > if I'm making the correct assumptions about the data you're trying to pass > back and forth, I think it might be easiest to simply define a custom class > that contains a std::vector of af::shared<vec3> arrays, and add an index > parameter to the return_modes method, which would be modified to return > only a single array of modes at a time. You can also do things like > wrapping a method as __getitem__ with boost.python, which would mimic a 2D > array. (I can probably dig up/cook up some code examples tomorrow if this > is unclear, since it was certainly not intuitive to me when I started > working with CCTBX and it would be nice to have documentation for the > future.) > > -Nat >

1 0

Passing array between python and C++
by Zhenwei Luo 07 Dec '13

07 Dec '13

Hello Everyone, I am writing a refinement program using the cctbx. I have encountered with a problem when I wanted to pass an array between c++ and python. The type of array I used in my c++ source file is af::shared<af::shared<vec3> >, I would like to pass it into my python script, thus I could do some other operations to this array. I wrote a method in my class to return this array. af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } I wrote another method in the *_ext file to wrap this method to make it callable from python. .def("return_modes", &nm_init::return_modes) But when I use it in python, eigenvector = nm_init_manager.return_modes() It gave me an error: TypeError: No to_python (by-value) converter found for c++ type: scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > What I want to realize is just creating an two dimensional array using vec3 as its element. I can also using the select method in this array to select some elements from its column, just like in the af::shared<vec3<double> > array. Namely, the column of this array should have some same properties like af::shared<vec3<double> >. Can anyone help me figure out what I should do? Thank you very much! Any help will be greatly appreciated. Sincerely, Zhenwei Luo Rice University

2 1

libtbx.run_tests_parallel
by Graeme.Winter＠diamond.ac.uk 14 Nov '13

14 Nov '13

Folks, Would anyone mind if I added an option to libtbx.run_tests_parallel to automatically make a temporary directory, chdir to there and run? It would mean I could write a little update / build / run tests script more easily. auto_run_dir=true springs to mind as a Phil option. Any thoughts? Thanks & best wishes, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 3

iotbx newbie question
by wtempel 07 Nov '13

07 Nov '13

Hello all, please consider the scenario of a manuscript that, among other things, describes 20 crystal structures. Suppose the journal editor requires a Table 1. I now looking for a tool to prepare such table with high accuracy and little effort. My PDB deposition workflow includes the preparation of mmcif coordinates with pdb_extract. The mmcif file is complete in the sense that after upload to RCSB-ADIT, virtually no additional information is required for deposition. My intuition was to combine CSV and IOTBX python modules and produce the table via input of a list of mmcif files. The current phenix table_1 tool provides some, but not all of that functionality, and I thought once familiar with the python modules, I would quickly write the script. After confirming that the example at http://cctbx.sourceforge.net/iotbx_cif/ works with my cctbx installation, I attempted to transform that code to read in PDB mmcif coordinates: <code> test_structure = iotbx.cif.reader( file_path="/tmp/1NXC.cif").build_crystal_structures()["1NXC"] </code> which throws an exception: <output> Traceback (most recent call last): File "1NXC.py", line 3, in <module> test_structure = iotbx.cif.reader(file_path="/tmp/1NXC.cif").build_crystal_structures()["1NXC"] KeyError: '1NXC' </output> How do I need to write the code so that it returns a python object with access to all the cif items? Thank you, Wolfram Tempel

2 1