- cctbxbb - phenix-online.org

Re: [cctbxbb] [Dials-support] DIALS source tarball
by markus.gerstel＠diamond.ac.uk 07 Apr '15

07 Apr '15

Cc'ing cctbx folks David Waterman wrote: > So maybe once we have a source tarball up at dials.diamond.ac.uk (..) Speaking of which: our DIALS builds are currently failing at the installer testing stage. This is due to cctbx/regression/tst_geometry_restraints_2.py failing deep in mmtbx territory. So far I could not figure out how to get this running properly. Call: libtbx.python tst_geometry_restraints_2.py Standard Output Skipping exercise_with_zeolite(): input file not available Skipping exercise_with_pdb(): chem_data directory not available Standard Error Traceback (most recent call last): File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 866, in <module> exercise_all(sys.argv[1:]) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 862, in exercise_all exercise_na_restraints_output_to_geo(verbose=verbose) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 805, in exercise_na_restraints_output_to_geo log=out1) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 5270, in run log=log) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 4876, in __init__ restraints_loading_flags=restraints_loading_flags, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2902, in __init__ log=log, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2287, in __init__ m_j=mm) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 1513, in get_lib_link return mon_lib_srv.link_link_id_dict["rna3p"] KeyError: 'rna3p' (On a side note: We really should call multiple tests within the same python file separately, instead of running them together within a signle run() method. This file contains 4 tests, but these will result in only a single return value for any upstream processing of the test results, such as Jenkins.) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

programmatic access to PDB through cctbx?
by Kaushik Hatti 01 Apr '15

01 Apr '15

Hello, Sorry if this is not relevant to this mailing list. I was wondering if there are any modules available under cctbx to query PDB database programmatically. There was an option to SQL-query PDBe (from here : http://www.ebi.ac.uk/pdbe-as/pdbedatabase/ ). But, this service has been retired since Mar 2015. How else could complex query on PDB possible? An example query could be: list of PDBid with SCOPid="1hba" and (Resolution < 2 and R/Rfree < 15/20) or (Resolution < 3 and R/Rfree < 20/25 ) and Completeness < 95% and structureFactorAvailability="Y" Such a query was possible via above mentioned SQL link. However, I am unable to find any other means now. Thanks in advance, Kaushik Hatti, Molecular Biophysics Unit, Indian Institute of Science, Bangalore, India -- People living deeply have no fear of death - Anais Nin Caution: I am still the dumbest person I have ever known :-)

2 2

dxtbx test failure
by markus.gerstel＠diamond.ac.uk 25 Mar '15

25 Mar '15

Hello everyone, Our last nightly build on RHEL linux x64 failed because the test dxtbx/tests/tst_dxtbx.py , which normally takes roughly 3 seconds to run on the build machine, got stuck for over 5 minutes. All the other tests (cctbx/scitbx/libtbx/iotbx/rstbx and remaining dxtbx) completed successfully. Tests were run via libtbx.run_tests_parallel with nproc=4. As far as I can tell there is no debug output. The libtbx testing suite does not support timeouts like some timeout decorators on python unittest? As far as I can tell tst_dxtbx.py at the moment runs 5 separate tests, so for future debugging it may make sense to split these up. -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

compilation issues
by Dave Lodowski 16 Mar '15

16 Mar '15

Hi, sorry if this is a repeat, but an email sent to the cctbx(a)cci.lbl.gov bounced back to me. We are attempting to install the DIALS package and CCTBX is required for compilation. During compilation of CCTBX, Xrender fails to find the X windows server. OS is Ubuntu LTS 12.04.5 text from console upon failure: Installing Xrender... log file is /home/dave/cctbx/build/base_tmp/Xrender_install_log getting package xrender-0.8.3.tar.gz... downloading from http://cci.lbl.gov/cctbx_dependencies : local copy is current using ./xrender-0.8.3.tar.gz (cached) installing xrender-0.8.3.tar.gz... Traceback (most recent call last): File "../auto_build/install", line 9, in <module> install_base_packages.installer(args=sys.argv, log=sys.stdout) File "../auto_build/install_base_packages.py", line 161, in __init__ self.build_dependencies(packages=packages) File "../auto_build/install_base_packages.py", line 505, in build_dependencies getattr(self, 'build_%s'%i)() File "../auto_build/install_base_packages.py", line 781, in build_render self.build_compiled_package_simple(pkg_name=pkg, pkg_name_label=name) File "../auto_build/install_base_packages.py", line 407, in build_compiled_package_simple log=pkg_log) File "../auto_build/install_base_packages.py", line 392, in configure_and_build self.call("./configure %s" % " ".join(list(config_args)), log=log) File "../auto_build/install_base_packages.py", line 243, in call return call(args, log=log, verbose=self.verbose, **kwargs) File "../auto_build/installer_utils.py", line 71, in call raise RuntimeError("Call to '%s' failed with exit code %d" % (args, rc)) RuntimeError: Call to './configure --prefix="/home/dave/cctbx/build/base"' failed with exit code 1 Log file indicates everything fine till X is required then : checking for X... no configure: error: X is required, but it was either disabled or not found. any suggestions as to how to proceed would be appreciated thanks, --David Lodowski

2 1

test 2
by Nicholas Sauter 05 Mar '15

05 Mar '15

-- Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Physical Biosciences Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 64R0121 Berkeley, CA 94720-8118 (510) 486-5713

1 0

Atom selections for distance (embarrassingly simple)
by Morten Grøftehauge 18 Feb '15

18 Feb '15

Hi everyone, I'm trying to do some distance extraction of hetero atoms in pdbs. I've got for model in pdb_obj.hierarchy.models(): for chain in model.chains(): for rg in chain.residue_groups(): for ag in rg.atom_groups(): for atom in ag.atoms(): if atom.hetero == True: print atom.distance(atom) Which prints out 0.0 as expected. But I can't figure out how to, say, measure the distance to an atom in the previous residue (if there is a previous residue). I also noticed something weird in this tutorial: http://cctbx.sourceforge.net/sbgrid2008/tutorial.html They truncate a non-standard amino acid to the beta-carbon but this only works because there's an alternate conformation with a standard amino acid (Cys). So it wouldn't work with a seleno-met for instance (I know there's a convert-to-ala function in mmtbx - I said it was weird, not terrible). Anyway, I hope someone can explain the distances thing for me. I'm sure there are multiple ways of doing it (should I just do a selection of "hetatm" and iterate over them and try to get some distanes?). Cheers, Morten

2 1

Partitioning of Grid/Unit Cell
by Nicholas Pearce 15 Feb '15

15 Feb '15

Hi Everyone, I’m trying to partition a grid/space based on smallest distance to atomic sites - as in a voronoi diagram (http://en.wikipedia.org/wiki/Voronoi_diagram <http://en.wikipedia.org/wiki/Voronoi_diagram>). Is there any code in cctbx that could be used to accomplish this? Thanks, Nick Nicholas Pearce, DPhil Student, Structural Genomics Consortium, University of Oxford

3 2

iotbx xray.reconstructed_amplitude and changed SIGF
by Huw Jenkins 12 Feb '15

12 Feb '15

Hi, This is my first post to this list and I hope I can explain my confusion properly. I have been using iotbx to manipulate data stored in mtz files and found that values in columns that should not have been altered by my code are different in the output mtz file. This seems only to occur when the input mtz file contains DANO, SIGDANO columns. When I run the code below F and DANO in input.mtz and output.mtz are the same but SIGF and SIGDANO are different (shown for 5 reflections listed). Is this expected behaviour? Given that in my understanding that code should just read a mtz file, convert the data to a miller array then write the array back out to a new mtz I would expect the values in the input and output to be identical (within rounding error). Thanks, Huw from __future__ import division import sys from iotbx import mtz from iotbx import reflection_file_reader from iotbx import reflection_file_utils def run(args): mtz_file = reflection_file_reader.any_reflection_file("input.mtz") mtz_as_miller_array = mtz_file.as_miller_arrays() f_obs = mtz_as_miller_array[0] mtz_out = f_obs.as_mtz_dataset(column_root_label="F_iotbx") mtz_out.mtz_object().write("output.mtz") if (__name__ == "__main__"): run(sys.argv[1:]) some reflections from input.mtz are: H K L F_ccp4 SIGF_ccp4 DANO_ccp4 SIGDANO_ccp4 ISYM_ccp4 1 2 2 4.90461 2.31536 0 3.27441 0 1 2 3 186.172 1.86381 119.745 3.84862 0 1 2 4 289.31 3.06709 -5.42197 6.46531 0 1 2 5 378.928 2.7118 87.3381 5.56933 0 1 2 6 540.866 4.38513 46.4155 8.8053 0 The same reflections in output.mtz are: H K L F_iotbx SIGF_iotbx DANOF_iotbx SIGDANOF_iotbx ISYMF_iotbx 1 2 2 4.90461 2.00516 0 4.01032 0 1 2 3 186.172 1.8943 119.745 3.7886 0 1 2 4 289.31 3.15096 -5.42197 6.30192 0 1 2 5 378.928 2.74848 87.3381 5.49695 0 1 2 6 540.866 4.3939 46.4155 8.7878 0

2 2

Jenkins spam imminent (but not now)
by markus.gerstel＠diamond.ac.uk 03 Feb '15

03 Feb '15

Hello everyone, We are currently building a continuous integration setup for cctbx using Jenkins at Diamond. This means Jenkins will run all tests for every source code change. If any test (or the build itself) fails, it will send a notification email to this list. This could result in a number of mails. However they follow a pretty obvious format, so should be easy to filter (and we can fine-tune the spamminess if needed). Mails will start soonishly. -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Best place to get ccp4io, boost, etc?
by Luc Bourhis 29 Jan '15

29 Jan '15

Hi, I noticed that the nightly build get their dependencies from http://cci.lbl.gov/repositories/ccp4io.gz and the like. Is this the preferred way to make sure one is in synch with trunk then? Best wishes, Luc

2 1

directory of cctbx tutorials
by David Waterman 14 Jan '15

14 Jan '15

Hi all, I've just spent a little time with Google collecting together tutorials for CCTBX and related open-source projects. I'm sure this is not a complete list, so I thought I'd share in case anyone knows of some other sources to add to it. I'm collecting this because the DIALS project is involved in workshops and teaching. My main interest is in diffraction data processing, but all tutorials that use core modules and types from cctbx are of interest, for potential updating/cannibalisation. General/developer - Tour of the cctbx http://cctbx.sourceforge.net/current/tour.html - Making compiled extensions. cctbx_introduction_3 at https://sourceforge.net/p/dials/code/HEAD/tree/trunk/doc/developer/ Data processing - Various tutorials on free-electron laser data processing (cctbx.xfel) at http://viper.lbl.gov/cctbx.xfel/index.php/Tutorials - CCTBX for diffraction calculations. cctbx_introduction_{1,2} at https://sourceforge.net/p/dials/code/HEAD/tree/trunk/doc/developer/ - Powder diffraction unit cell refinement ( <http://cctbx.sourceforge.net/siena2005/>Siena 2005) http://cctbx.sourceforge.net/siena2005/unit_cell_refinement.html - Data processing with DIALS. 4 tutorials at http://dials.sourceforge.net/doc/index.html Coordinate handling - PDB truncate to Ala (SBGrid 2008) (iotbx.pdb) at http://cctbx.sourceforge.net/sbgrid2008/tutorial.html Phasing - Direct methods (Siena 2005) http://cctbx.sourceforge.net/siena2005/direct_methods_light.html Model refinement - Rigid body refinement (IUCr 2008) at http://cctbx.sourceforge.net/iucr2008/rigid_body_refinement_core.html -- David

2 1

Boost Import Error
by Garrett Thompson 12 Jan '15

12 Jan '15

Hello, I recently looking at the cctbxbb mailing archive and I came across your email response a few months ago : http://phenix-online.org/pipermail/cctbxbb/2014-June/000933.html I was wondering if you could elaborate a bit on how to use the wrappers cctbx.python? I consider myself to be quite a beginner in this area and can't seem to import the same module, import scitbx.array_family.flex *ImportError: No module named boost.optional* When I do I receive the same error about having boost.optional being an import dependency. If you are able to, please be as explicit as possible, I am pretty bad with common conventions and terminology, as I have never studied computer science or any related area formally. Any help would be much appreciated! Sincerely, Garrett Thompson

2 1

e density from atomic model?
by TJ Lane 06 Jan '15

06 Jan '15

Crystallography pros, Field-outsider asking for advice here. Is there a standard way to estimate the electron density on a grid from an atomic model (e.g. PDB coordinates of atoms)? One thing that occurred to me was to simply FT the Cromer-Mann atom-centered Gaussians commonly used to model atomic form factors. Perhaps there is a more standard way, though? And possible code to mooch...? Thanks much! TJ

2 1

building on Mac OSX
by Roger Martin 05 Jan '15

05 Jan '15

Hi, I'm attempting to build on Mac. Via jenkins. The scons finds /usr/bin/c++ but my builds are based on gcc located at /opt/gcc/current/bin. A which c++ returns /opt/gcc/current/bin/c++. I tried to tell it by making a site_init.py according to http://www.scons.org/doc/production/HTML/scons-user.html#idm12919064 --------------- cd build mkdir -p site_scons cat <<EOF >./site_scons/site_init.py env = DefaultEnvironment(tools = ['gcc', 'gnulink'], CC = '/opt/gcc/current/bin/gcc', CXX = '/opt/gcc/currentbin/g++') EOF -------------- Do you have a way to tell it which compiler version to find that is compatible with the source code level? Regards, Roger

2 3

building from cctbx SVN repository
by Roger Martin 11 Dec '14

11 Dec '14

Hi, http://cctbx.sourceforge.net/current/installation.html shows svn export svn://svn.code.sf.net/p/cctbx/code/trunk/libtbx/development/cctbx_svn_getti… ./cctbx_svn_getting_started.csh where cctbx_svn_getting_started.csh starts with a curl: curl http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx -o cctbx_bundle.selfx yet currently http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx doesn't exist. http://cci.lbl.gov/cctbx_build/results/current/ and has the Nov 13 bundle. Has the build changed or this url http://cci.lbl.gov/cctbx_build/results/current/cctbx_bundle.selfx really missing? Thank you, I'm wanting to be at svn HEAD for development in case I find something to add or patch. Roger

1 0

Correlation coefficient for symmetry-related reflections
by Vanbenschoten, Andrew 08 Dec '14

08 Dec '14

Hi everyone, I'm analyzing a diffraction dataset with very weak intensities and I'm getting extremely high R-linear values. Though this may simply be due to poor data, I'd like to calculate the correlation coefficient between sets of symmetry-related reflections to make sure the low intensity values aren't artificially inflating the observed R-linears. Is there a way to do this in Phenix or cctbx? Thanks! -Andrew

1 0

mean weighted phase error by resolution bin
by Mooers, Blaine H.M. (HSC) 05 Dec '14

05 Dec '14

Thanks to Pavel's suggestion, I was able to calculate the overall mean weighted phase error between to sets of phases in miller_arrays 2 and 5. I am stumped on how to retrieve the mean weighted phase error by resolution bin. #! /usr/bin/env cctbx.python import sys import iotbx mtz_filename = sys.argv[1] mtz_file = mtz.object(mtz_filename) miller_arrays = mtz_file.as_miller_arrays() binner=miller_array.setup_binner_d_star_sq_step(d_star_sq_step=0.01) mwpe = miller_arrays[2].mean_weighted_phase_error(miller_arrays[5]) Best regards, Blaine Blaine Mooers, Ph.D. Assistant Professor Department of Biochemistry and Molecular Biology University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center Rm. 466 Shipping address: 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 Letter address: P.O. Box 26901, BRC 466 Oklahoma City, OK 73190 office: (405) 271-8300 lab: (405) 271-8313 fax: (405) 271-3910 e-mail: blaine-mooers(a)ouhsc.edu Faculty webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/… Small Angle Scattering webpage: http://www.oumedicine.com/docs/default-source/ad-biochemistry-workfiles/sma… X-ray lab webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/… ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Wednesday, December 03, 2014 2:00 PM To: cctbxbb(a)phenix-online.org Subject: cctbxbb Digest, Vol 69, Issue 3 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. Re: computing mean phase error between final refined model and experimental phases (Pavel Afonine) ---------------------------------------------------------------------- Message: 1 Date: Tue, 02 Dec 2014 12:06:08 -0800 From: Pavel Afonine <pafonine(a)lbl.gov> To: cctbx mailing list <cctbxbb(a)phenix-online.org>, Blaine-Mooers(a)ouhsc.edu Subject: Re: [cctbxbb] computing mean phase error between final refined model and experimental phases Message-ID: <547E1BB0.3030103(a)lbl.gov> Content-Type: text/plain; charset="windows-1252"; Format="flowed" Hi Blaine, key functionality for this is the method of Miller array class called mean_phase_error, see: cctbx/miller/__init__.py . Unit and regression tests are best usage examples most of the time. In this case see: cctbx/regression/tst_miller.py . Let me know if you have any questions. I don't have a good grasp of phase error value as a quality measure. For example, I do not know what number is good, what is poor, etc. I think to some extent what you are trying to do can be converted to Fourier map comparison: just compute two maps, each one using identical amplitudes and two different sets of phases, and then compare maps. To me at least the map CC is a more telling metric. However, map comparison is tricky too (usual map CC may be misleading!) and you need to use proper metrics, as explained in details here: Acta Cryst. (2014). D70, 2593-2606. Metrics for comparison of crystallographic maps A. Urzhumtsev, P. V. Afonine, V. Y. Lunin, T. C. Terwilliger and P. D. Adams Pavel On 12/2/14 9:20 AM, Mooers, Blaine H.M. (HSC) wrote: > Dear CCTBXBB, > > I want to compare the phases from the final refined model and a set of > experimental phases > to calculate the weighted mean phase error. > > How do I this using python? > > Best regards, > > Blaine > > Blaine Mooers, Ph.D. > Assistant Professor > Department of Biochemistry and Molecular Biology > University of Oklahoma Health Sciences Center > S.L. Young Biomedical Research Center Rm. 466 > > Shipping address: > > > office: (405) 271-8300 lab: (405) 271-8313 fax: (405) 271-3910 > e-mail: blaine-mooers(a)ouhsc.edu > > Faculty webpage: > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oumedicine.com_depa… > > Small Angle Scattering webpage: > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oumedicine.com_docs… > > X-ray lab webpage: > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.oumedicine.com_depa… > > > > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai…

2 1

computing mean phase error between final refined model and experimental phases
by Mooers, Blaine H.M. (HSC) 02 Dec '14

02 Dec '14

Dear CCTBXBB, I want to compare the phases from the final refined model and a set of experimental phases to calculate the weighted mean phase error. How do I this using python? Best regards, Blaine Blaine Mooers, Ph.D. Assistant Professor Department of Biochemistry and Molecular Biology University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center Rm. 466 Shipping address: office: (405) 271-8300 lab: (405) 271-8313 fax: (405) 271-3910 e-mail: blaine-mooers(a)ouhsc.edu Faculty webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/… Small Angle Scattering webpage: http://www.oumedicine.com/docs/default-source/ad-biochemistry-workfiles/sma… X-ray lab webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/…

2 1

structure factors from map
by castro＠biochem.mpg.de 01 Dec '14

01 Dec '14

Hi, Could somebody please help to find the function to transform electron density map into structures factors? I found a function "from_map()" within cctbx/maptbx/structure_factor.h, But it don't know how to transform the map into a "complex_map" format. Thanks a lot in advance, Marcelino

3 3

Problem with executing Pymol with Tk (I build it with cctbx)
by Tam＠em.uni-frankfurt.de 17 Nov '14

17 Nov '14

Dear all, I have a problem to execute Pymol that it is without external GUI and just only the Viewer window. This is the error message I got after executing the pymol. Traceback (most recent call last): File "/home/mauricetam/software/cctbx/cctbx_plus_build/base/lib/python2.7/site-packages/pymol/__init__.py", line 401, in launch_gui __import__(self.invocation.options.gui) File "/home/mauricetam/software/cctbx/cctbx_plus_build/base/lib/python2.7/site-packages/pmg_tk/__init__.py", line 22, in <module> from PMGApp import * File "/home/mauricetam/software/cctbx/cctbx_plus_build/base/lib/python2.7/site-packages/pmg_tk/PMGApp.py", line 28, in <module> from Tkinter import * File "/home/mauricetam/software/cctbx/cctbx_plus_build/base/lib/python2.7/lib-tk/Tkinter.py", line 39, in <module> import _tkinter # If this fails your Python may not be configured for Tk ImportError: No module named _tkinter Thanks for the help. Regards HK

1 0

using the c++ fft_to_real_map_unpadded to make maps
by Roger Martin 14 Nov '14

14 Nov '14

Hi, I posted to the sourceforge help forum: http://sourceforge.net/p/cctbx/discussion/76274/thread/16122dcc/ about using fft_to_real_map_unpadded. Have had great success with it making maps for visualizing results. What I'm not understanding is the shape and properties of maps for 'P 32 2 1' and other space groups that have translations of 1/3, 1/4 or 1/6 in symmetry operators. Is there something special or extra that needs done for the fourier coefficients, n_real dimensions or the returned versa map? cctbx::sgtbx::space_group spaceGroup(cctbx::sgtbx::space_group_symbols(154)); const scitbx::af::int3 n_real(n, m, l); scitbx::af::versa< double, scitbx::af::c_grid<3>> pDelta=cctbx::maptbx::fft_to_real_map_unpadded(spaceGroup, n_real, miller_indices.const_ref(), structure_factors.const_ref()); Looking inside I see it does tha padding for me. So running out of things to try. Any ideas would be appreciated --Roger

1 0

Electron density hitogram from MTZ file
by castro＠biochem.mpg.de 28 Oct '14

28 Oct '14

Hi everybody, I am new in crystallogrphy and in the use of cctbx. I am interested in building a electron density histogram from a mtz file. Could somebody give a hint on how to accomplish this task using cctbx libraries (I am scripting in python)? Let's say that I already have the columns of interest as "miller arrays". Then (I think), a grid (on the real space) needs to be created based on unit cell parameters, to finally make the fft. I think that most of my doubts are about the grid. How small the grid should be with respect to the low-resolution limit? Does the space group information needed or it is enough with the the cell parameters? Is there all ready a routine doing this? Thanks a lot for your help. Marcelino

3 4

which cctbx version is recommended now?
by Marcin Wojdyr 15 Oct '14

15 Oct '14

Hi, we need to pick a version of cctbx for CCP4 for use in phaser and DIALS. The latest logs at cci.lbl.gov/cctbx_build/show_error_summary.cgi?build_tag=2014_10_05_2205 show in all builds: [Errno 2] No such file or directory Traceback (most recent call last): ImportError: __import__("boost_python_meta_ext"): No module named boost_python_meta_ext Is it better to pick older version? Marcin

2 3

Miller arrays from lists of indices
by Oliver Zeldin 07 Oct '14

07 Oct '14

Dear cctbx devs, Does anyone know how to make a miller set from a list of (h, k, l) indices? e.g. [(1,2,3), (34, 1, 45), .... ] Cheers, Oliver -- Dr. Oliver B. Zeldin Brunger Group Stanford University

2 2

new cctbx-plus installers
by Nathaniel Echols 23 Sep '14

23 Sep '14

Hi all-- We now have improved installers for CCTBX with all optional third-party dependencies, including numpy, wxPython, and matplotlib; these are basically just a simplified Phenix installer (minus anything that isn't open-source, obviously). For anyone doing visualization these are probably preferable to the default packages. More useful for doing active development is that since these include the pre-build Python, if you are working directly with the CCTBX sources you can bootstrap from these packages. Yesterday's build is here: http://cci.lbl.gov/cctbx_build/plus_bundles/2014_09_08_0005/ and in general the latest version can be obtained here: http://cci.lbl.gov/cctbx_build/plus_bundles/current although today's build isn't finished yet. The next build will also incorporate IPython. I still need to fix the download page so these are listed alongside the old bundles. Once this is done we'll probably declare an official "release". -Nat

3 5

Tests which randomly pass or fail (nag)
by Graeme.Winter＠diamond.ac.uk 16 Sep '14

16 Sep '14

Hi Folks Quick nag if I may... if people have tests which employ random inputs (e.g. random unit cell, random amplitudes and phases) please could they set a seed or select from some seeds which should give random values which work, or fix their tests so that all random values work (ideally!) That way when tests fail people know something is wrong, rather than everything is fine but the random values drawn are troublesome... Sorry for sounding grumpy! Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

order bias in flex.random_permutation
by Keitaro Yamashita 11 Sep '14

11 Sep '14

Dear cctbx developers, I happened to notice that the implementation of flex.random_permutation (scitbx/random.h) results in biased order because each element is always swapped with the element randomly selected from entire range of array. For details, please see the second paragraph and the right figure of http://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle#Implementation_er… I reproduced the plot using flex.random_permutation. Actually this seems to have less impact because flex.random_permutation is rarely used in cctbx or phenix as far as I know. If needed, I would commit the corrected version (using the algorithm in the wikipedia article above). Best regards, Keitaro

2 1

IPython with cctbx
by Oliver Zeldin 09 Sep '14

09 Sep '14

Dear Devs, Has anyone tried to get IPython working with cctbx? I'd really like to be able to use IPython notebooks for examples/tutorials/documented experiments, and was wondering if anyone else had thought the same thing. Cheers, Oli

3 10

Re: [cctbxbb] Sphinx docs for CCTBX
by Nigel Moriarty 20 Aug '14

20 Aug '14

These links don't work for me. On Mon, Aug 18, 2014 at 3:16 PM, Ian Rees <irees(a)lbl.gov> wrote: > I'm using a separate build configuration for the docs. You can view > the nightly results here: > > http://cci-vm-6.lbl.gov:8010/builders/cctbx%2Bdocs-linux-64 > > If you click the first build, you can view the log files for each > step, and view the Sphinx log file by clicking "docs:cctbx stdio", > e.g. > > > http://cci-vm-6.lbl.gov:8010/builders/cctbx%2Bdocs-linux-64/builds/6/steps/… > > I'm working through the warnings now. Mostly missing references and > import errors. > > Ian > > On Fri, Aug 15, 2014 at 8:37 AM, Nathaniel Echols <nechols(a)lbl.gov> wrote: > > On Fri, Aug 15, 2014 at 8:06 AM, Richard Gildea <rgildea(a)gmail.com> > wrote: > >> > >> I had a go at starting to document cctbx.uctbx.unit_cell, which as you > >> know is mainly a Boost.Python extension > >> (https://sourceforge.net/p/cctbx/code/20467/). Before I travel too much > >> further down this route, is this an appropriate way of documenting > >> Boost.Python extensions or is there a better way we can do this? > > > > > > I haven't had enough time to experiment to decide on the best approach. > I > > see three choices - the first is what you've done: > > > > class my_class (ext.my_class) : > > """doc""" > > > > def boosted_function (self, some_argument) : > > """function doc""" > > super(ext.my_class, self).boosted_function(some_argument) > > > > OR (2): > > > > ext.my_class.__doc__ = """doc""" > > ext.my_class.boosted_function.__func__.__doc__ = """function doc""" > > > > OR (3): > > > > put the docstrings in C++ code, for instance: > > > > .def("boosted_function", &my_class::boosted_function, "function doc") > > > > (not sure what the equivalent is for classes but it's probably pretty > > similar) > > > > My opinion is that (1) is much too prone to confusion and API conflicts > > unless we're already using this code structure (and FYI, you broke the > call > > signatures for a couple of methods). (3) is just gross. (2) is also > gross > > but has the virtue of being the least likely to break, and doesn't > require > > modifying C++ files. So I'm inclined to take that approach > > > > (For what it's worth, when modifying a class using the boost.python > > injector, we appear to be able to set __doc__ inside there; see > > iotbx.pdb.hierarchy for an example.) > > > >> It would be good to get the documentation for the core cctbx classes up > >> and running, however several of the core classes (e.g. > >> cctbx.sgtbx.space_group, cctbx.uctbx.unit_cell, flex arrays) are mostly > >> Boost.Python extensions, so it would be good to arrive at a sensible > way to > >> document these extensions. > > > > > > Agreed. I think for flex arrays we will be stuck writing the docstrings > in > > C++ - however since these are relatively stable APIs it should be > reasonably > > straightforward. > > > > -Nat > -- Nigel W. Moriarty Building 64R0246B, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

2 1

real-space symmetry operators
by Nathaniel Echols 18 Aug '14

18 Aug '14

Hi all-- I'm trying to figure out how to orthogonalize symmetry operators given a unit cell, which doesn't seem to be something we have code for. These appear in PDB files in the REMARK 290 section but I haven't been able to find any other software that generates them, and the relationship between the symops and the real-space equivalents is unclear. I assume this is conceptually similar to orthogonalizing fractional coordinates, but unit_cell.orthogonalize() does not quite do what I wanted. In the PDB, for P63 it has this (excerpted): REMARK 290 2555 -Y,X-Y,Z ... REMARK 290 SMTRY1 2 -0.500000 -0.866025 0.000000 0.00000 REMARK 290 SMTRY2 2 0.866025 -0.500000 0.000000 0.00000 REMARK 290 SMTRY3 2 0.000000 0.000000 1.000000 0.00000 which looks like it should be relatively straightforward, but I'm not very good with matrices. Does anyone know the correct math for this? The corresponding rt_mx object from sgtbx looks like this: r=(0.0, -1.0, 0.0, 1.0, -1.0, 0.0, 0.0, 0.0, 1.0) t=(0.0, 0.0, 0.0) (Yes, I realize that there are much easier ways to apply symmetry operators to coordinates, but I want to limit the number of matrix multiplications involved when starting from Cartesian coordinates.) thanks, Nat

3 3

Sphinx docs for CCTBX
by Nathaniel Echols 15 Aug '14

15 Aug '14

Thanks to some superb work by Nader Morshed, Oliver Zeldin, and Ian Rees, we now have automatically updated Sphinx docs for CCTBX: http://cci.lbl.gov/cctbx_docs/ Obviously these are still pretty sparse - right now most of the formal documentation is clustered here: libtbx libtbx.utils libtbx.phil (mostly re-using Ralf's old documentation) cctbx.miller iotbx.pdb (built on Ralf's newsletter article) mmtbx.command_line These are a combination of inline docstrings and separate restructured text files in cctbx_project/sphinx. We have automatically generated rst files for some modules, but others will need to be added manually. In the end I think we'll want to curate every file, but the current layout will get us started. To facilitate additional contributions, I've modifed the installation script for base packages to include a --sphinx option. After setting up CCTBX you can then do this: cd $BUILD libtbx.configure sphinx cd $SRC/cctbx_project/sphinx make html and you will get the full documentation built locally. I am still figuring out how to use Sphinx effectively; it is definitely possible to document Boost.Python extensions but it's not quite as obvious. I don't expect us to retroactively document several hundred thousand lines of code written over a decade, but if we can get the core parts thoroughly covered that will greatly reduce the barrier for new developers. Suggestions for particular modules to focus on are appreciated; I am going to try to cover the basic PDB+reflection file functionality. I encourage anyone doing CCTBX-related development to contribute as you go. (They don't need to be your own APIs, just modules you're reasonably familiar with.) Those of you who already have a ton of undocumented code can expect me to harass you periodically about fixing this. (Note that I've cannibalized at least one newsletter article for this, and continue to do so; we should re-use existing prose descriptions as much as possible without violating copyrights.) -Nat

4 5

Re: [cctbxbb] resend
by Mustyakimov, Marat 24 Jul '14

24 Jul '14

Loop through symops: apply each symop to the original asu, then apply unit translations to minimize its distance to the original asu, test if new asu intersects with the target 27-bricks-large box, use it, if it does. Keep in mind that asu is not always a brick, it is always a convex polyhedron in cctbx. I do not see any good reason for doing all that. I do not read cctbxbb. On Jul 24, 2014, at 2:43 PM, Alastair Fyfe <afyfe(a)soe.ucsc.edu> wrote: >> Hi Marat, my apologies if this is a duplicate. From Pavel's comment it occurred to me you might not follow posts to the cctxbb >> Alastair >> ------ >> >> Many thanks for your replies. >> >> Pavel : by "slow" I meant to (a) apply sym ops to asu and refer results >> back to unit cell thus obtaining a "full" unit cell (I expect >> "expand_to_p1()" does this?) (b) translate the cell and (c) find overlap >> with the 26-brick neighborhood of the starting brick. This should work >> but seems inefficient. >> >> Marat: applying SymOps to the basis vectors seems a great idea. >> However, it wasn't clear to me how to combine sg SymOps to yield the >> 26 adjacent bricks. Does the map expansion code do this? >> >> >

1 0

Re: [cctbxbb] 26-fold symm expansion of asu?
by Alastair Fyfe 24 Jul '14

24 Jul '14

Many thanks for your replies. Pavel : by "slow" I meant to (a) apply sym ops to asu and refer results back to unit cell thus obtaining a "full" unit cell (I expect "expand_to_p1()" does this?) (b) translate the cell and (c) find overlap with the 26-brick neighborhood of the starting brick. This should work but seems inefficient. Marat: applying SymOps to the basis vectors seems a great idea. However, it wasn't clear to me how to combine sg SymOps to yield the 26 adjacent bricks. Does the map expansion code do this? On 07/23/2014 12:00 PM, cctbxbb-request(a)phenix-online.org wrote: > Send cctbxbb mailing list submissions to > cctbxbb(a)phenix-online.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://phenix-online.org/mailman/listinfo/cctbxbb > or, via email, send a message with subject or body 'help' to > cctbxbb-request(a)phenix-online.org > > You can reach the person managing the list at > cctbxbb-owner(a)phenix-online.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of cctbxbb digest..." > > > Today's Topics: > > 1. Re: 26-fold symm expansion of asu? (Pavel Afonine) > 2. Re: 26-fold symm expansion of asu? (Mustyakimov, Marat) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 22 Jul 2014 18:58:32 -0700 > From: Pavel Afonine <pafonine(a)lbl.gov> > To: cctbx mailing list <cctbxbb(a)phenix-online.org> > Cc: "Mustyakimov, Marat" <mustyakimovm(a)ornl.gov> > Subject: Re: [cctbxbb] 26-fold symm expansion of asu? > Message-ID: <53CF16C8.5040504(a)lbl.gov> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi Alastair, > > I think I can write a piece of code using cctbx that would do what you > want and probably that would be what you call "slow". Could you please > explain some more what it is for and why, and may be give some context? > Perhaps that helps me to see if we have something similar already > available.. > > For instance we have very efficient code that computes maps in ASU, and > also that expands map in ASU into the whole P1 box. This is something > Marat wrote recently and is very efficient. May be you can reuse that or > at least harvest ideas? > > I'm copying to Marat in case he has comments. > > Pavel > > On 7/21/14, 2:14 PM, Alastair Fyfe wrote: >> Hi, >> I'd be grateful for any pointers to clipper or cctbx code that >> addresses the following: given a "brick" of map grid coordinates that >> enclose the asu, obtain the 26 symmetry-related "bricks" which contact >> it. Calculating this by successively generating symmetry mates and >> testing seems slow; I was wondering whether there was a better way. >> thank you, >> Alastair Fyfe >> UCSC >> _______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> http://phenix-online.org/mailman/listinfo/cctbxbb > > > ------------------------------ > > Message: 2 > Date: Wed, 23 Jul 2014 14:01:13 +0000 > From: "Mustyakimov, Marat" <mustyakimovm(a)ornl.gov> > To: Pavel Afonine <pafonine(a)lbl.gov> > Cc: cctbx mailing list <cctbxbb(a)phenix-online.org> > Subject: Re: [cctbxbb] 26-fold symm expansion of asu? > Message-ID: <9457BEEC-4821-4BBB-8D91-E903AAFAE7F0(a)ornl.gov> > Content-Type: text/plain; charset="iso-8859-1" > > The fast way to generate symmetry equivalent ASU: > Apply symmetry operation to unit vectors: ex = SymOp*(1,0,0), ey=SymOp*(0,1,0) and ez=SymOp*(0,0,1); > Then every map point of the symmetry related asu is obtained from the original one: > new_ijk = i*ex + j*ey + k*ez where (i,j,k) is the grid point in the original ASU. > > Marat Mustyakimov > P.O. Box 2008, Bldg. 8600, MS-6475 > Oak Ridge National Lab > Oak Ridge TN 37831 USA > e-mail: mustyakimovm(a)ornl.gov<mailto:[email protected]> > e-mail: marat.mustyakimov(a)gmail.com<mailto:[email protected]> > Tel: +1-865-253-3924 > > On Jul 22, 2014, at 9:58 PM, Pavel Afonine <pafonine(a)lbl.gov<mailto:[email protected]>> wrote: > > Hi Alastair, > > I think I can write a piece of code using cctbx that would do what you want and probably that would be what you call "slow". Could you please explain some more what it is for and why, and may be give some context? Perhaps that helps me to see if we have something similar already available.. > > For instance we have very efficient code that computes maps in ASU, and also that expands map in ASU into the whole P1 box. This is something Marat wrote recently and is very efficient. May be you can reuse that or at least harvest ideas? > > I'm copying to Marat in case he has comments. > > Pavel > > On 7/21/14, 2:14 PM, Alastair Fyfe wrote: > Hi, > I'd be grateful for any pointers to clipper or cctbx code that addresses the following: given a "brick" of map grid coordinates that enclose the asu, obtain the 26 symmetry-related "bricks" which contact it. Calculating this by successively generating symmetry mates and testing seems slow; I was wondering whether there was a better way. > thank you, > Alastair Fyfe > UCSC > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org<mailto:[email protected]> > http://phenix-online.org/mailman/listinfo/cctbxbb > > > > > ------------------------------ > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > http://phenix-online.org/mailman/listinfo/cctbxbb > > > End of cctbxbb Digest, Vol 64, Issue 2 > **************************************

1 0

26-fold symm expansion of asu?
by Alastair Fyfe 23 Jul '14

23 Jul '14

Hi, I'd be grateful for any pointers to clipper or cctbx code that addresses the following: given a "brick" of map grid coordinates that enclose the asu, obtain the 26 symmetry-related "bricks" which contact it. Calculating this by successively generating symmetry mates and testing seems slow; I was wondering whether there was a better way. thank you, Alastair Fyfe UCSC

3 2

Phenix dependencies in CCTBX code
by Graeme.Winter＠diamond.ac.uk 22 Jun '14

22 Jun '14

Dear All, In cctbx_project/iotbx/pdb/multimer_reconstruction.py there is a dependency on Phenix from phenix.command_line.simple_ncs_from_pdb import simple_ncs_from_pdb from phenix.command_line.simple_ncs_from_pdb import ncs_master_params Please could someone remove this? I just had a look at cleaning things up so that this was encapsulated but was not obvious and figured risk of breaking was more substantial. This is breaking our nightly build tests at Diamond as we don’t check out phenix :o( Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Phenix dependencies in CCTBX code
by Graeme.Winter＠diamond.ac.uk 22 Jun '14

22 Jun '14

Dear All, In cctbx_project/iotbx/pdb/multimer_reconstruction.py there is a dependency on Phenix from phenix.command_line.simple_ncs_from_pdb import simple_ncs_from_pdb from phenix.command_line.simple_ncs_from_pdb import ncs_master_params Please could someone remove this? I just had a look at cleaning things up so that this was encapsulated but was not obvious and figured risk of breaking was more substantial. This is breaking our nightly build tests at Diamond as we don’t check out phenix :o( Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Re: [cctbxbb] Boost Import Error
by Magnus Andersson 19 Jun '14

19 Jun '14

Hi all, In case anyone else runs into the same problems (which is not unlikely), the following worked for me: > svn co https://pymol.svn.sourceforge.net/svnroot/pymol/trunk/pymol > cd pymol > cctbx.python setup.py build > cctbx.python setup.py install > Thanks to Luc & Nat for being helpful and persistent! Cheers, Magnus > > On 17 Jun 2014, at 21:34, Luc Bourhis <luc_j_bourhis(a)mac.com> wrote: > >> Hi, >> >> it's a bit late here in Paris but I can at least fix one point: >> >>> port select --set python27 >>> Error: The 'set' command expects two arguments: <group>, <version> >> >>> I tried to figure this out but without success... sorry to bug you with syntax issues. >> >> I'm the one to be sorry here as I pulled it from the top of my head and got it wrong. It's >> >> port select --set python python27 >> >> As for your original problem, I'll try to regroup and think through our former exchange to see what I may have missed. >> >> Best wishes, >> >> Luc >> >

1 0

Boost Import Error
by Magnus Andersson 17 Jun '14

17 Jun '14

Hi, I've installed the cctbx package from the selfextracting file found in http://cci.lbl.gov/cctbx_build/ but I run into problems when importing cctbx in my python script: import iotbx.pdb File "/Users/magnus/Downloads/cctbx_sources/iotbx/pdb/__init__.py", line 2, in <module> from cctbx.array_family import flex File "/Users/magnus/Downloads/cctbx_sources/cctbx/array_family/flex.py", line 2, in <module> import scitbx.array_family.flex File "/Users/magnus/Downloads/cctbx_sources/scitbx/array_family/flex.py", line 2, in <module> import boost.optional # import dependency ImportError: No module named boost.optional It is flex.py that wants to import boost.optional and does not find it. However, I find these three files: optional.hpp optional_fwd.hpp optional_io.hpp in the following directory: /Users/magnus/Downloads/cctbx_sources/boost/boost/optional which suggests that it's present. What am I missing here? I'm using Mac OS X Version 10.9.3 and Python2.7. Cheers, Magnus

4 26

sparse matrix
by David Waterman 10 Jun '14

10 Jun '14

Hi, I have a problem where the use of a sparse matrix is appropriate, so I was pretty happy to grep cctbx for "sparse" and discover the code in scitbx/sparse. I'm just browsing through the code and tests to get a feel for how to use these types. In my existing Python code I set the values for some columns using flex.double vectors, then ultimately fill a flex.double(flex.grid(n,m)) by these columns using matrix_paste_column_in_place. It seems to me I can use a very similar pattern by replacing the flex.doubles with sparse.matrix_columns and setting columns in the sparse.matrix using slice notation: mat[:,0] = column. So far so good. The issue is that I'm not sure how best to go about setting the values in the matrix_column in the first place. At the moment I'm making a lot of use of flex.size_t selections and the set_selected method of flex.double to efficiently (from Python) set elements of one flex.double from another. It appears that there is no similar functionality for sparse.matrix_column. Am I right? Possibly the best I can do is construct a Python dictionary consisting of the indices from my size_t selection and the values from my flex.double array and use these to construct the matrix_column, as done in tst_sparse, e.g. sparse.matrix_column(m, {1:0.1, 2:0.2}). Without having tried this, I sense it might be a burden. I have a lot of values to set and don't really want to convert from flex.double via a dictionary. Are there any philosophical/technical/other reasons for not writing a set_selected method for sparse.vector? If not, I might try to do this. Thanks -- David

2 5

where is the mmtbx.mp_geo command
by Sun, Miao 13 Mar '14

13 Mar '14

Hi, I have installed cctbx software, however, I cant find the command mmtbx.mp_geo. I wanna to validate the backbone bond and angles. Is there any other command could do that? Thanks. -Miao

3 2

Re: [cctbxbb] Adding columns to new *unmerged* MTZ object
by Graeme.Winter＠diamond.ac.uk 13 Feb '14

13 Feb '14

In related news…. Should void object::adjust_column_array_sizes(int new_nref) { CMtz::MTZ* p = ptr(); if (!p->refs_in_memory) return; if (new_nref > p->nref) { reserve(new_nref); for(int i=0;i<p->nxtal;i++) { for(int j=0;j<p->xtal[i]->nset;j++) { for(int k=0;k<p->xtal[i]->set[j]->ncol;k++) { CMtz::MTZCOL* col_k = p->xtal[i]->set[j]->col[k]; int old_size = column_array_size(col_k); if (new_nref > old_size) { ccp4array_resize(col_k->ref, new_nref); for(int iref=old_size;iref<new_nref;iref++) { *(reinterpret_cast<union float_uint_uchar*>( &col_k->ref[iref])) = not_a_number_value_; } } } } } } } assign nref? otherwise repeated calling of this will keep on re-allocating the arrays Thanks Graeme On 12 Feb 2014, at 10:40, Graeme Winter <graeme.winter(a)diamond.ac.uk> wrote: > All, > > By adding > > m_out.adjust_column_array_sizes(len(mi)) > > to the bpl wrapper and defining > > m_out.set_n_reflections(len(mi)) > > in the object.h and bpl wrapper I can do what I want… > > I will add some tests and some warnings that this may make nonsense of the values… however before committing any comments on the interface? or behaviour? > > Thanks Graeme > > > > On 12 Feb 2014, at 09:11, <Graeme.Winter(a)diamond.ac.uk> <Graeme.Winter(a)diamond.ac.uk> wrote: > >> Hi Folks, >> >> I am trying to create a new MTZ file containing unmerged and it seems to be less simple than I would think - set_reals() on a column requires a set of Miller indices and values and does a lookup internally (which I don’t want for unmerged values) and set_values() requires that the array is already the right size - when it starts off as size 0 and I have no idea how to make enough room in there from the Python API… >> >> Any clues? I have done quite a chunk of grepping around to try and figure this one out and have run out of ideas… >> >> Thanks Graeme_______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> http://phenix-online.org/mailman/listinfo/cctbxbb >

3 11

Adding columns to new *unmerged* MTZ object
by Graeme.Winter＠diamond.ac.uk 12 Feb '14

12 Feb '14

Hi Folks, I am trying to create a new MTZ file containing unmerged and it seems to be less simple than I would think - set_reals() on a column requires a set of Miller indices and values and does a lookup internally (which I don’t want for unmerged values) and set_values() requires that the array is already the right size - when it starts off as size 0 and I have no idea how to make enough room in there from the Python API… Any clues? I have done quite a chunk of grepping around to try and figure this one out and have run out of ideas… Thanks Graeme

1 1

building with external boost
by Marcin Wojdyr 30 Jan '14

30 Jan '14

Hi all, would it be ok if I added an option to use installed boost libraries? Debian and gentoo patches add boolean --use_system_libs flag. I could use the same, but with a list of arguments (at first only --use_system_libs=boost). Marcin

2 2

boost python and metaclass inheritance
by David Waterman 10 Jan '14

10 Jan '14

Hi, Does anyone know how to use __metaclass__ specification alongside inheritance from boost python classes? This code illustrates the problem: import abc > from scitbx.lstbx import normal_eqns > class Refinery(object): > __metaclass__ = abc.ABCMeta > pass > class AdaptLstbx(Refinery, normal_eqns.non_linear_ls): > pass The result is Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python2.7/abc.py", line 87, in __new__ > cls = super(ABCMeta, mcls).__new__(mcls, name, bases, namespace) > TypeError: Error when calling the metaclass bases > metaclass conflict: the metaclass of a derived class must be a > (non-strict) subclass of the metaclasses of all its bases The real Refinery is a class that has one method that must be supplied by a concrete subclass. I like the idea of making it impossible to instantiate a base Refinery rather than relying on e.g. raise NotImplementedError() in that method definition. This is what ABCMeta would give me, if only I could get it to work here. Google found me some answers that require me to know the __metaclass__ of the bases (as suggested by the error message), but normal_eqns.non_linear_ls.__metaclass__ does not exist. I don't terribly mind the raise NotImplementedError() alternative, but I thought that if I can easily use ABCMeta, I'd like to. Cheers -- David

2 5

cctbx.python unnecessarily unfriendly
by James Stroud 10 Jan '14

10 Jan '14

Hello CCTBX developers, I noticed in the cctbx.python script the following lines that call the python interpreter if [ -n "$LIBTBX__VALGRIND_FLAG__" ]; then exec $LIBTBX_VALGRIND "$LIBTBX_PYEXE" -Qnew "$@" elif [ $# -eq 0 ]; then exec "$LIBTBX_PYEXE" -Qnew else exec "$LIBTBX_PYEXE" -Qnew "$@" fi Here, the “-Qnew” flag forces the python interpreter to work with new division (where “/“ does not strictly mean integer division). This flag basically overrides the expected python behavior to which python programmers have been writing code for over a decade, wrecks some dependencies, and stifles new code that itself depends on cctbx. It is highly frustrating to work around this highly unnecessary flag!! Fortunately, python has a better way to handle this problem by just having “from __future__ import division” at the top of .py files. In fact, I have spot-checked several files in the cctbx source tree and noticed that the ALL have this line at the top. So why does the -Qnew flag still exist in the cctbx.python script? It is entirely unnecessary and requires any and all dependencies to be programmed to a behavior that is not standard for the language. James p.s. It is not reasonable to ask users who install your packages to patch the cctbx.python script or to patch dependencies that don’t conform to this non-standard behavior.

6 13

building cctbx with external boost
by Marcin Wojdyr 19 Dec '13

19 Dec '13

Hi I vaguely remember a discussion some time ago about building cctbx with stand-alone boost libraries. Do you know what boost versions are compatible? Is using the --scan-boost option enough? I see that Gentoo has extra patch to unbundle boost from cctbx: http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sci-libs/cctbx/file… Is it necessary? I'm asking because at this moment we have in CCP4 two versions of boost, one in cctbx and the other for building DSSP (it uses different subset of boost libs than cctbx). It would make sense to use a single version for both, if possible. Cheers Marcin

2 3

Re: [cctbxbb] cctbxbb Digest, Vol 59, Issue 1
by Zhenwei Luo 09 Dec '13

09 Dec '13

Hi Nat, Thanks for your reply! I have tried the first method you mentioned--adding an index value to the return method. It did work if I only needed a single array for future calculation. But I found it's not very convenient for my program. It's convenient and more efficient if other classes in it could process all the one-dimensional arrays at the same time. Otherwise, I needed to write more control code in python, and the c++ part could only deal with a single array now. Therefore, I hoped that I could pass such a multidimensional array in my program. Sincerely, Zhenwei On Sun, Dec 8, 2013 at 2:00 PM, <cctbxbb-request(a)phenix-online.org> wrote: > Send cctbxbb mailing list submissions to > cctbxbb(a)phenix-online.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://phenix-online.org/mailman/listinfo/cctbxbb > or, via email, send a message with subject or body 'help' to > cctbxbb-request(a)phenix-online.org > > You can reach the person managing the list at > cctbxbb-owner(a)phenix-online.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of cctbxbb digest..." > > > Today's Topics: > > 1. Passing array between python and C++ (Zhenwei Luo) > 2. Re: Passing array between python and C++ (Nathaniel Echols) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 7 Dec 2013 22:25:08 -0600 > From: Zhenwei Luo <tluozhenwei(a)gmail.com> > To: cctbxbb(a)phenix-online.org > Subject: [cctbxbb] Passing array between python and C++ > Message-ID: <90518A19-AC1B-493B-8B27-79D31EDDF055(a)gmail.com> > Content-Type: text/plain; charset=us-ascii > > Hello Everyone, > > I am writing a refinement program using the cctbx. I have > encountered with a problem when I wanted to pass an array between c++ and > python. The type of array I used in my c++ source file is > af::shared<af::shared<vec3> >, I would like to pass it into my python > script, thus I could do some other operations to this array. > I wrote a method in my class to return this array. > > af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } > > I wrote another method in the *_ext file to wrap this method to make it > callable from python. > > .def("return_modes", &nm_init::return_modes) > > But when I use it in python, > eigenvector = nm_init_manager.return_modes() > > It gave me an error: > > TypeError: No to_python (by-value) converter found for c++ type: > scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > > > What I want to realize is just creating an two dimensional array using > vec3 as its element. I can also using the select method in this array to > select some elements from its column, just like in the > af::shared<vec3<double> > array. Namely, the column of this array should > have some same properties like af::shared<vec3<double> >. Can anyone help > me figure out what I should do? > > Thank you very much! Any help will be greatly appreciated. > > Sincerely, > Zhenwei Luo > Rice University > > > ------------------------------ > > Message: 2 > Date: Sat, 7 Dec 2013 20:45:10 -0800 > From: Nathaniel Echols <nechols(a)lbl.gov> > To: cctbx mailing list <cctbxbb(a)phenix-online.org> > Subject: Re: [cctbxbb] Passing array between python and C++ > Message-ID: > <CALeAa1MOL1wbW5Vb=M= > jXsnGTaCBQNJjcOzDwMhM27x5PP+XNA(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > On Sat, Dec 7, 2013 at 8:25 PM, Zhenwei Luo <tluozhenwei(a)gmail.com> wrote: > > > I am writing a refinement program using the cctbx. I have > > encountered with a problem when I wanted to pass an array between c++ and > > python. The type of array I used in my c++ source file is > > af::shared<af::shared<vec3> >, I would like to pass it into my python > > script, thus I could do some other operations to this array. > > I wrote a method in my class to return this array. > > > > af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } > > > > I wrote another method in the *_ext file to wrap this method to make it > > callable from python. > > > > .def("return_modes", &nm_init::return_modes) > > > > But when I use it in python, > > eigenvector = nm_init_manager.return_modes() > > It gave me an error: > > TypeError: No to_python (by-value) converter found for c++ type: > > scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > > > > > What that error means is that boost.python doesn't know how to present this > new type to Python. For the data structures that we pass back and forth, > we have to explicitly provide wrappers for them - including new > instantiations of scitbx::array_family templates. > > What I want to realize is just creating an two dimensional array using vec3 > > as its element. I can also using the select method in this array to > select > > some elements from its column, just like in the af::shared<vec3<double> > > > array. Namely, the column of this array should have some same properties > > like af::shared<vec3<double> >. Can anyone help me figure out what I > should > > do? > > > > In general the scitbx::array_family templates aren't really intended to > every be multidimensional; for inherently 2D and 3D data types like images > and maps, we use flex.grid to provide a multi-dimensional accessor on top > of the 1D array. The grid may not be the most appropriate in your case but > if I'm making the correct assumptions about the data you're trying to pass > back and forth, I think it might be easiest to simply define a custom class > that contains a std::vector of af::shared<vec3> arrays, and add an index > parameter to the return_modes method, which would be modified to return > only a single array of modes at a time. You can also do things like > wrapping a method as __getitem__ with boost.python, which would mimic a 2D > array. (I can probably dig up/cook up some code examples tomorrow if this > is unclear, since it was certainly not intuitive to me when I started > working with CCTBX and it would be nice to have documentation for the > future.) > > -Nat >

1 0

Passing array between python and C++
by Zhenwei Luo 08 Dec '13

08 Dec '13

Hello Everyone, I am writing a refinement program using the cctbx. I have encountered with a problem when I wanted to pass an array between c++ and python. The type of array I used in my c++ source file is af::shared<af::shared<vec3> >, I would like to pass it into my python script, thus I could do some other operations to this array. I wrote a method in my class to return this array. af::shared<af::shared<vec3<double> > return_modes( ) { return modes; } I wrote another method in the *_ext file to wrap this method to make it callable from python. .def("return_modes", &nm_init::return_modes) But when I use it in python, eigenvector = nm_init_manager.return_modes() It gave me an error: TypeError: No to_python (by-value) converter found for c++ type: scitbx::af::shared<scitbx::af::shared<scitbx::vec3<double> > > What I want to realize is just creating an two dimensional array using vec3 as its element. I can also using the select method in this array to select some elements from its column, just like in the af::shared<vec3<double> > array. Namely, the column of this array should have some same properties like af::shared<vec3<double> >. Can anyone help me figure out what I should do? Thank you very much! Any help will be greatly appreciated. Sincerely, Zhenwei Luo Rice University

2 1

libtbx.run_tests_parallel
by Graeme.Winter＠diamond.ac.uk 14 Nov '13

14 Nov '13

Folks, Would anyone mind if I added an option to libtbx.run_tests_parallel to automatically make a temporary directory, chdir to there and run? It would mean I could write a little update / build / run tests script more easily. auto_run_dir=true springs to mind as a Phil option. Any thoughts? Thanks & best wishes, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 3

iotbx newbie question
by wtempel 07 Nov '13

07 Nov '13

Hello all, please consider the scenario of a manuscript that, among other things, describes 20 crystal structures. Suppose the journal editor requires a Table 1. I now looking for a tool to prepare such table with high accuracy and little effort. My PDB deposition workflow includes the preparation of mmcif coordinates with pdb_extract. The mmcif file is complete in the sense that after upload to RCSB-ADIT, virtually no additional information is required for deposition. My intuition was to combine CSV and IOTBX python modules and produce the table via input of a list of mmcif files. The current phenix table_1 tool provides some, but not all of that functionality, and I thought once familiar with the python modules, I would quickly write the script. After confirming that the example at http://cctbx.sourceforge.net/iotbx_cif/ works with my cctbx installation, I attempted to transform that code to read in PDB mmcif coordinates: <code> test_structure = iotbx.cif.reader( file_path="/tmp/1NXC.cif").build_crystal_structures()["1NXC"] </code> which throws an exception: <output> Traceback (most recent call last): File "1NXC.py", line 3, in <module> test_structure = iotbx.cif.reader(file_path="/tmp/1NXC.cif").build_crystal_structures()["1NXC"] KeyError: '1NXC' </output> How do I need to write the code so that it returns a python object with access to all the cif items? Thank you, Wolfram Tempel

2 1