- cctbxbb - phenix-online.org

Re: [cctbxbb] dials install error
by markus.gerstel＠diamond.ac.uk 12 Jan '16

12 Jan '16

Dear Sid, Unfortunately I don't know what the proper solution to your problem is. We had issues with pango/glibc reported just a few days ago, but I don't know what the outcome to that was. Dear David, Nick, Billy, Following up on the discussion in the thread 'bootstrap.py build on Ubuntu', here is an installer failure on CentOS5 that comes down to pango/libfontconfig/libglib missing some libraries, trace attached. -Markus From: Sidhu, Khushwant (Dr.) [mailto:[email protected]] Sent: 12 January 2016 11:26 To: Gerstel, Markus (DLSLtd,RAL,LSCI) Cc: Sidhu, Khushwant (Dr.) Subject: Re: dials install error Dear Markus, using the --verbose flag, I get the following .... ./install --verbose (snip) : Regenerating module files in /usr/local/dials-dev20160112/base : generating pangorc file at /usr/local/dials-dev20160112/base/etc/pango/pangorc : generating pango.modules file at /usr/local/dials-dev20160112/base/etc/pango/pango.modules : /usr/local/dials-dev20160112/base/bin/pango-querymodules: /lib64/libc.so.6: version `GLIBC_2.7' not found (required by /usr/local/dials-dev20160112/base/lib/libfontconfig.so.1) : /usr/local/dials-dev20160112/base/bin/pango-querymodules: /lib64/libc.so.6: version `GLIBC_2.7' not found (required by /usr/local/dials-dev20160112/base/lib/libglib-2.0.so.0) : Traceback (most recent call last): : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/configure.py", line 22, in <module> : if not run(): : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/configure.py", line 17, in run : libtbx.env_config.cold_start(sys.argv) : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/env_config.py", line 2267, in cold_start : env.refresh() : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/env_config.py", line 1544, in refresh : regenerate_module_files.run(libtbx.env.under_base('.'), only_if_needed=True) : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/auto_build/regenerate_module_files.py", line 153, in run : fix_pango(base_dir, out) : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/auto_build/regenerate_module_files.py", line 59, in fix_pango : call(("%s/bin/pango-querymodules > %s") % (base_dir, pangomodules), log=out) : File "/usr/local/dials-dev20160112/modules/cctbx_project/libtbx/auto_build/installer_utils.py", line 81, in call : raise RuntimeError("Call to '%s' failed with exit code %d" % (args, rc)) : RuntimeError: Call to '/usr/local/dials-dev20160112/base/bin/pango-querymodules > /usr/local/dials-dev20160112/base/etc/pango/pango.modules' failed with exit code 1 Traceback (most recent call last): File "/usr/local/dials-installer-dev/bin/install.py", line 46, in ? installer(sys.argv[1:]).install() File "/usr/local/dials-installer-dev/lib/libtbx/auto_build/install_distribution.py", line 197, in install self.install_from_binary() File "/usr/local/dials-installer-dev/lib/libtbx/auto_build/install_distribution.py", line 318, in install_from_binary self.reconfigure(log=log) File "/usr/local/dials-installer-dev/lib/libtbx/auto_build/install_distribution.py", line 484, in reconfigure call(args=args, log=log, verbose=self.options.verbose) File "/usr/local/dials-installer-dev/lib/libtbx/auto_build/installer_utils.py", line 81, in call raise RuntimeError("Call to '%s' failed with exit code %d" % (args, rc)) RuntimeError: Call to '/usr/local/dials-dev20160112/base/bin/python /usr/local/dials-dev20160112/modules/cctbx_project/libtbx/configure.py --current_working_directory /usr/local/dials-dev20160112/build dials xia2' failed with exit code 1 regards Sid -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

status of mmtbx/phenix.molprobity
by Marcin Wojdyr 04 Jan '16

04 Jan '16

Dear All, we'd like to use mmtbx validation code. Is all the code in mmtbx meant to be open-source, even if the dispatchers have names starting with phenix? In particular I'm thinking about phenix.molprobity. (the names are not a problem, I just want to double check that it's ok if we use it) All the best in 2016, Marcin

2 1

Failure in phenix.map_box when working with map files....
by Ezra Peisach 11 Dec '15

11 Dec '15

Has anyone observed a failure of phenix.map_box when working with map files? When working with a structure factor file with map coefficients, it works fine. When a map file is used - the following occurs: Traceback (most recent call last): File "/local/scratch/CCTBX/cctbx_build/../cctbx_sources/mmtbx/command_line/map_box.py", line 194, in <module> run(args=sys.argv[1:]) File "/local/scratch/CCTBX/cctbx_build/../cctbx_sources/mmtbx/command_line/map_box.py", line 158, in run selection = selection) File "/local/scratch/CCTBX/cctbx_sources/mmtbx/utils/__init__.py", line 2938, in __init__ self.map_box = maptbx.copy(map_data, gridding_first, gridding_last) Boost.Python.ArgumentError: Python argument types in cctbx_maptbx_ext.copy(double, list, list) did not match C++ signature: copy(scitbx::af::const_ref<double, cctbx::maptbx::c_grid_padded_p1<3ul> > map_unit_cell, scitbx::af::tiny<int, 3ul> first, scitbx::af::tiny<int, 3ul> last) copy(scitbx::af::const_ref<float, cctbx::maptbx::c_grid_padded_p1<3ul> > map_unit_cell, scitbx::af::tiny<int, 3ul> first, scitbx::af::tiny<int, 3ul> last) copy(scitbx::af::const_ref<double, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > map, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > result_grid) copy(scitbx::af::const_ref<float, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > map, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > result_grid) Any suggestions? The map_data is the same format in the call....

4 5

crystallography Q&A site proposal on StackExchange
by Marcin Wojdyr 06 Nov '15

06 Nov '15

Hello, I'm writing about it first to developer mailing lists assuming that developers know StackOverflow. SO has now a number of sister sites on StackExchange and some Q&A sites in the same format are self-hosted, e.g. biostars.org. Here is a list of sites in the science category on SE: http://stackexchange.com/sites#science-traffic I though that such a site for crystallography would be useful, mostly because it has lower psychological and technical barrier for asking questions. I'm not sure how it's on phenixbb and other mailing lists, but on ccp4bb maybe a few percent of all 6000+ subscribers have asked anything at all. I assume people have questions in mind, they are just not eager to ask it by mailing 6000 other people. I don't know if a Q&A site would help, but just to try here is proposal for Crystallography site on SE: http://area51.stackexchange.com/proposals/91841/crystallography Using SE is simpler than hosting it, and even if the proposal doesn't make it at least we'll have an idea what was the interest. Technicalities. New sites are created basically when enough people is interested. Currently the proposal is in the "definition" stage, which means coming up with example questions - to show what kind of questions could be asked when the site exists. To complete this stage we need 40 question with 10+ votes. Everyone has 5 up-votes for selecting good questions. Votes can be changed at any time. So at least 80 voting people would be needed for this stage. Having more example questions would also help. Marcin

1 0

(no subject)
by Nigel Moriarty 06 Nov '15

06 Nov '15

Folks Calling for articles and short communications of interest to structural biologists. The deadline for submission in the January 2016 issue is 18 December 2015 (with some leeway if you are still working on the text, especially as it can be polished over the break). The Computational Crystallography Newsletter (CCN) is an electronic newsletter for structural biologists, and is published online every 6 months. Feature articles, meeting announcements and reports can be submitted to me at any time for consideration. Submission of text by email or word-processing files using the CCN templates is requested. Past newsletters and a Microsoft Word template are available at www.phenix-online.org/newsletter. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

2 2

Unused/old modules on the top-level of cctbx
by Oleg Sobolev 18 Oct '15

18 Oct '15

Hi, I went through top-level directories of cctbx and discovered the following: compcomm - two papers from IUCr computational crystallography authored by Ralf with some scripts. These scripts are referenced in these papers. One paper in LaTeX by Luc Bourhis "Small-molecule toolbox". dox - probably was meant as documentation, actually contains some documentation in RST, Ralf's talks from Siena, 2005, some tutorial in LaTeX, the content for the cctbx website. fftw3tbx - probably is not used anywhere ipctbx - most likely is not used anywhere, wrapper for sys/ipc.h - C library for interprocess communication access structure. rpmbuild - not sure whether it is still needed/working, last change to the only file was on 7-29-2009. setup - not clear whether the only .sh script in this directory is still needed/working. I suggest to move compcomm to dox despite the reference in 2009 paper and remove fftw3tbx, ipctbx, rpmbuild, setup if nobody has an idea why they should stay where they are/be moved somewhere. Best regards, Oleg Sobolev.

3 3

cctbx documentation
by Paul Adams 13 Oct '15

13 Oct '15

Dear Colleagues, we are looking at the cctbx documentation. Currently the world is directed to the documentation on the Sourceforge site, which hasn’t been updated for a long time and is looking somewhat old. Last year, Oliver and Ian (and others) worked on Sphinx-based documentation for cctbx (the last published version of which can be found here: http://cci.lbl.gov/cctbx_docs/). The question for the cctbx developer community therefore is whether we should move over to the Sphinx system for cctbx documentation. The benefits are: a more consistent and modern interface, and automated documentation generation from source code header file information. As part of making this move we would need to integrate some of the additional information from the original cctbx site into the Sphinx documentation. We would also move the hosting of the cctbx documentation to the CCI web server in Berkeley (and off the Sourceforge site). Another email related to that topic soon. I’d appreciate any feedback on this topic. Cheers, Paul -- Paul Adams Interim Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ LBenvenue(a)lbl.gov ][ 1-510-495-2506 ] --

2 1

cctbx and Github
by Paul Adams 11 Oct '15

11 Oct '15

Dear Colleagues, there has been some discussion in this bulletin board about moving the cctbx repository from Sourceforge to Github. We are seriously considering this option, especially in light of the Sourgeforge outage earlier in the year. One concern is what impact this will have on the cctbx developer community - which goes beyond Phenix and DIALS. We’d appreciate any feedback about this proposed move. Note that this will take some time to orchestrate and will involve contacting any known users of the cctbx package to inform them of the change. Cheers, Paul -- Paul Adams Interim Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ LBenvenue(a)lbl.gov ][ 1-510-495-2506 ] --

1 0

CentOS7 - fixed
by markus.gerstel＠diamond.ac.uk 30 Sep '15

30 Sep '15

Thanks for your suggestions. They did not do the trick though. Some more detective work got me down to ../base/bin/python ../modules/cctbx_project/libtbx/configure.py dials failing and both libtbx.python ../modules/cctbx_project/libtbx/configure.py dials and LD_LIBRARY_PATH=lib ../base/bin/python ../modules/cctbx_project/libtbx/configure.py dials succeeding. A run with LD_DEBUG=files identified the problem: /lib64/libboost_python.so: error: symbol lookup error: undefined symbol: PyUnicodeUCS4_FromEncodedObject (fatal) CentOS7 comes with its own set of incompatible boost libraries. I've put a fix in: http://sourceforge.net/p/cctbx/code/22986/ Installation on CentOS7 now works for me. As far as I could tell this does not break other things. -Markus ________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of Nigel Moriarty [nwmoriarty(a)lbl.gov] Sent: Tuesday, September 29, 2015 18:36 To: cctbx mailing list Subject: Re: [cctbxbb] CentOS7 bugs (cont.) libtbx.configure --scan-boost phenix Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> On Tue, Sep 29, 2015 at 10:20 AM, Nicholas Sauter <nksauter(a)lbl.gov<mailto:[email protected]>> wrote: Nigel, Do remember how to force libtbx.scons to scan the boost directory? Nick On Tue, Sep 29, 2015 at 9:25 AM, Nigel Moriarty <nwmoriarty(a)lbl.gov<mailto:[email protected]>> wrote: Markus The default usage of libtbx.scons in the build directory does not scan the boost directory so maybe that has something to do with the .so not behaving on Centos7 - because it was built on Centos5/6? Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709<tel:510-486-5709> Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909<tel:510-486-5909> Web : CCI.LBL.gov<http://CCI.LBL.gov> On Tue, Sep 29, 2015 at 8:29 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> wrote: Hello cctbx folks, We are having problems running, building and/or installing DIALS on CentOS7. Running the linux installer unceremoniously ends in a segfault. The installer is even broken when it is build on the same system using bootstrapped cctbx/dials. The broken installer can be traced to the DIALS extension loader, which uses python’s pkgutil loader, which triggers some library scanning routine within python, which in turn falls over boost_python_meta_ext.so and dies with the strace shown in the email below. The installer can be successfully “fixed” by deleting build/lib/boost_python_meta_ext.so during the installation. So I see two issues here: 1. What is broken with build/lib/boost_python_meta_ext.so that it causes segfaults on CentOS7? 2. Why is this not covered by any test? Neither of the following caught anything: a. python bootstrap.py builder=dials tests b. libtbx.run_tests_parallel module=cctbx module=dials module=boost_adaptbx [NB: The previously reported 3 broken tests on CentOS7 are red herrings and are all fixed by running ‘yum update’] I don’t know how to proceed from here. -Markus From: markus.gerstel(a)diamond.ac.uk<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] Sent: 29 September 2015 13:29 To: dials-support(a)lists.sourceforge.net<mailto:[email protected]> Subject: Re: [Dials-support] CentOS7 bugs (cont.) Hello everyone, (..) Sorry, I hit ctrl+enter by accident… In addition to these failed tests the installer dies a boost-related death according to strace: (..) poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 1 ([{fd=13, revents=POLLIN}]) read(13, "\tlinux-vdso.so.1 => (0x00007fff"..., 4096) = 486 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7648, si_status=0, si_utime=0, si_stime=0} --- poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 2 ([{fd=13, revents=POLLHUP}, {fd=15, revents=POLLHUP}]) close(13) = 0 close(15) = 0 wait4(7648, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 7648 stat("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext", 0x7fff6421d3f0) = -1 ENOENT (No such file or directory) open("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext.so", O_RDONLY) = 11 fstat(11, {st_mode=S_IFREG|0775, st_size=200624, ...}) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18a86} --- +++ killed by SIGSEGV (core dumped) +++ Segmentation fault (core dumped) Not sure if relevant, but CentOS7 compiler is GCC 4.8.3 20140911 (Red Hat 4.8.3-9) compared to RHEL6 GCC 4.4.7 20120313 (Red Hat 4.4.7-11) Anybody have any ideas? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb -- Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Physical Biosciences Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 64R0121 Berkeley, CA 94720-8118 (510) 486-5713<tel:%28510%29%20486-5713> _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

1 0

Re: [cctbxbb] CentOS7 bugs (cont.)
by markus.gerstel＠diamond.ac.uk 29 Sep '15

29 Sep '15

Hello cctbx folks, We are having problems running, building and/or installing DIALS on CentOS7. Running the linux installer unceremoniously ends in a segfault. The installer is even broken when it is build on the same system using bootstrapped cctbx/dials. The broken installer can be traced to the DIALS extension loader, which uses python's pkgutil loader, which triggers some library scanning routine within python, which in turn falls over boost_python_meta_ext.so and dies with the strace shown in the email below. The installer can be successfully "fixed" by deleting build/lib/boost_python_meta_ext.so during the installation. So I see two issues here: 1. What is broken with build/lib/boost_python_meta_ext.so that it causes segfaults on CentOS7? 2. Why is this not covered by any test? Neither of the following caught anything: a. python bootstrap.py builder=dials tests b. libtbx.run_tests_parallel module=cctbx module=dials module=boost_adaptbx [NB: The previously reported 3 broken tests on CentOS7 are red herrings and are all fixed by running 'yum update'] I don't know how to proceed from here. -Markus From: markus.gerstel(a)diamond.ac.uk [mailto:[email protected]] Sent: 29 September 2015 13:29 To: dials-support(a)lists.sourceforge.net Subject: Re: [Dials-support] CentOS7 bugs (cont.) Hello everyone, (..) Sorry, I hit ctrl+enter by accident... In addition to these failed tests the installer dies a boost-related death according to strace: (..) poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 1 ([{fd=13, revents=POLLIN}]) read(13, "\tlinux-vdso.so.1 => (0x00007fff"..., 4096) = 486 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7648, si_status=0, si_utime=0, si_stime=0} --- poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 2 ([{fd=13, revents=POLLHUP}, {fd=15, revents=POLLHUP}]) close(13) = 0 close(15) = 0 wait4(7648, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 7648 stat("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext", 0x7fff6421d3f0) = -1 ENOENT (No such file or directory) open("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext.so", O_RDONLY) = 11 fstat(11, {st_mode=S_IFREG|0775, st_size=200624, ...}) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18a86} --- +++ killed by SIGSEGV (core dumped) +++ Segmentation fault (core dumped) Not sure if relevant, but CentOS7 compiler is GCC 4.8.3 20140911 (Red Hat 4.8.3-9) compared to RHEL6 GCC 4.4.7 20120313 (Red Hat 4.4.7-11) Anybody have any ideas? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Phi-Psi Angles
by Nicholas Pearce 21 Sep '15

21 Sep '15

Hi Everyone, Phenomenally stupid question, but is there an easy way to calculate phi-psi angles for a protein using cctbx? And, following on from that, Chi-1, Chi-2 etc angles? I’ve had a poke around, but couldn’t find any obvious (at least to me) way that worked… I presume this must be implemented rigorously somewhere, and I didn’t want to waste time reimplementing the wheel. Thanks, Nick

2 2

Move to github
by Graeme.Winter＠diamond.ac.uk 15 Sep '15

15 Sep '15

Hi Folks A while back we asked about moving cctbx (along with DIALS) to github; postponement was asked until Phenix release is out. Phenix release is out. How do people feel about this? Enthusiastic endorsement of this project from DIALS-East & xia2 developer camps. Latter has already migrated… Go? No go? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Moving to github
by Graeme.Winter＠diamond.ac.uk 08 Sep '15

08 Sep '15

Hi Folks Looks like phenix release has happened… should we begin the conversation about moving cctbx and DIALS over to the happy world of github? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 3

tst_miller failing
by markus.gerstel＠diamond.ac.uk 26 Aug '15

26 Aug '15

Hi everyone, particularly Pavel I guess, I had a look at one intermittently failing test: cctbx/regression/tst_miller.py, run with parameter P31 The relevant piece of code is: def exercise_randomize_amplitude_and_phase(): xrs = random_structure.xray_structure( space_group_info=sgtbx.space_group_info(number=1), elements=["C"]*300) fc = xrs.structure_factors(d_min=1).f_calc() def run(a,p): fc_ = fc.randomize_amplitude_and_phase(amplitude_error=a, phase_error_deg=p, random_seed=1312425) d1, d2 = flex.abs(fc.data()), flex.abs(fc_.data()) r = flex.sum(flex.abs(d1-d2))/flex.sum(flex.abs(d1+d2))*2 return (r, fc.mean_phase_error(phase_source=fc_)) # assert approx_equal(run(0,0), (0,0)) for v in list(xrange(0,91, 10)): r = run(v/100.,v) assert approx_equal(r[0], v/100., 0.01) assert approx_equal(r[1], v, 1) The test fails in the last line for varying values of v. I saw that the values of r[] increase proportionally to v, so I ran the test 2000 times for v=90, plotting the values of r[0] and r[1]; results attached. With the current statistics the r[1] test is expected to fail in 1 run out of every 80. Could you please have a look at this, and either increase the tolerance on the test or identify whether these values are true unexpected outliers indicating a problem in the code? Don't know what the r[1] distribution should look like, but that's clearly not normal. :) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Structure factor calculation for Si
by Allan Lyckegaard 26 Aug '15

26 Aug '15

Dear CCTBX-list I am trying to reproduce the (scaled) intensities for Si reported here: http://rruff.geo.arizona.edu/AMS/xtal_data/DIFfiles/15285.txt based on this information: http://rruff.geo.arizona.edu/AMS/CIF_text_files/15285_cif.txt I am using CCTBX in Python: from cctbx.array_family import flex from cctbx import sgtbx, crystal, miller, xray sgnum = 227 sg = sgtbx.space_group_info(sgnum).group() latticepar = [5.43070, 5.43070, 5.43070, 90, 90, 90] dmin = 1.0 atompos = [['Si',(0.0, 0.0, 0.0)]] scatterers = flex.xray_scatterer() scatterers.append(xray.scatterer(label=atompos[0][0], site=atompos[0][1])) pg = sg.build_derived_point_group() sym = crystal.symmetry(space_group=sg,unit_cell=latticepar) sps = crystal.special_position_settings(crystal_symmetry=sym) structure = xray.structure(special_position_settings=sps,scatterers=scatterers) F = structure.structure_factors(d_min=dmin,algorithm='direct').f_calc() list(F.amplitudes()) This gives: [((0, 2, 2), 0.0), ((0, 4, 0), 120.21765812638594), ((1, 1, 1), 84.18849550806966), ((1, 3, 1), 65.48371727205928), ((1, 3, 3), 57.45438308324155), ((1, 5, 1), 51.48759686950681), ((2, 2, 2), 128.54540698861973), ((2, 4, 2), 0.0), ((3, 3, 3), 51.48759686950681)] How come CCTBX give an amplitude of 0.0 for (0,2,2) and (2,4,2)? The measured intensities in the DIFfile above does not show this. Am I missing something? Cheers, /Allan

4 3

Computational Crystallography Newsletter - Volume 6, Number 2
by Nigel Moriarty 25 Aug '15

25 Aug '15

I am pleased to announce the publication of the latest issue of the Computational Crystallography Newsletter: http://www.phenix-online.org/newsletter/ A listing of the articles and short communications is given below. Please note that the newsletter accepts articles of a general nature of interest to all crystallographers. Please send any articles to me at NWMoriarty(a)lbl.gov noting that there is a Word Template on the website to streamline production. July 2015 <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf> Articles - A context-sensitive guide to RNA & DNA base-pair & base-stack geometry <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=22> Short Communications - New Phaser-MR search panel in Phenix <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=7> - Quantum chemical techniques for minimising ligand geometries in the active site <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=10> - 13 typical occupancy refinement scenarios and available options in *phenix.refine* <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=12> Fitting tips #10 – How do your base pairs touch and twist? <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=3> Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

1 0

Shelx HKL file with original indices from unmerged MTZ
by Graeme.Winter＠diamond.ac.uk 06 Aug '15

06 Aug '15

Hi Folks Using reader = any_reflection_file(hklin) intensities = [ma for ma in reader.as_miller_arrays(merge_equivalents=False) if ma.info().labels == ['I', 'SIGI']][0] help(intensities) with open('%s.hkl' % prefix, 'wb') as f: intensities.export_as_shelx_hklf(f, normalise_if_format_overflow=True) to get a file for input to shelx and it appears to have only indices in the ASU not the Fridel pairs. What do I need to do to output with original indices (ie. P1 indices?) I assume that this has been solved before... Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Tonights build/testing failures
by markus.gerstel＠diamond.ac.uk 05 Aug '15

05 Aug '15

Hi everyone, tonights test (cctbx) or build (dials) failures were caused by changes to the CBFlib in r568. http://sourceforge.net/p/cbflib/code-0/commit_browser Error messages may include ImportError: (..)/build/lib/_pycbf.so: undefined symbol: cbf_airy_disk_volume Do we have a red phone to cbflib folks? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

cctbx developers: sf outage has lead to local corruption of your cctbx_project
by Aaron Brewster 27 Jul '15

27 Jul '15

Hi folks, because of the recent parallel commits to the cctbx repository due to the sourceforge outage, you will likely need to move your old cctbx_project directory aside and check it out fresh again. The 'too long didn't read' version of this email is you need to do this or several files will remain out of sync. This email only applies to developers or users who use svn directly. How to proceed: $ cd <your sources/modules directory> $ mv cctbx_project cctbx_project.old $ svn co https://svn.code.sf.net/p/cctbx/code/trunk cctbx_project If you have local edits in your cctbx_project directory, you'll need to copy them over to your new folder. More detail for those interested. The following example illustrates the problem: [phenix-src/cctbx_project/scitbx/lstbx]$ svn update [phenix-src/cctbx_project/scitbx/lstbx]$ svn status According to svn, this directory is up to date [phenix-src/cctbx_project/scitbx/lstbx]$ svn info normal_equations.h Path: normal_equations.h Name: normal_equations.h URL: https://svn.code.sf.net/p/cctbx/code/trunk/scitbx/lstbx/normal_equations.h Repository Root: https://svn.code.sf.net/p/cctbx/code Repository UUID: 3c1a6239-ac2c-0410-bc49-92500e444112 Revision: 22630 Node Kind: file Schedule: normal Last Changed Author: luc_j_bourhis Last Changed Rev: 15039 Last Changed Date: 2012-03-12 10:03:05 -0700 (Mon, 12 Mar 2012) Text Last Updated: 2015-07-27 10:24:16 -0700 (Mon, 27 Jul 2015) Checksum: ec2e555a9ad1d2553bb37bc04f677ef5 Note the revision number is 22630, and it lists the last revision for this file as 15039. After I move my directory aside and re-check out the directory: [phenix-src/cctbx_project/scitbx/lstbx]$ svn info normal_equations.h Path: normal_equations.h Name: normal_equations.h URL: https://svn.code.sf.net/p/cctbx/code/trunk/scitbx/lstbx/normal_equations.h Repository Root: https://svn.code.sf.net/p/cctbx/code Repository UUID: 3c1a6239-ac2c-0410-bc49-92500e444112 Revision: 22630 Node Kind: file Schedule: normal Last Changed Author: nksauter Last Changed Rev: 22624 Last Changed Date: 2015-07-25 17:16:55 -0700 (Sat, 25 Jul 2015) Text Last Updated: 2015-07-27 10:46:36 -0700 (Mon, 27 Jul 2015) Checksum: e5e5e623ecdde362afadf321d37b2c5e The correct revision is now retrieved. Thanks, -Aaron

1 0

Re: [cctbxbb] git on BuildBot machines
by markus.gerstel＠diamond.ac.uk 26 Jul '15

26 Jul '15

Hi Aaron, Is there some documentation about the buildbot objects I could use to figure out a sensible way to determine whether git is available? I could of course just do try-run-catch... -Markus On 26 Jul 2015 16:35, Aaron Brewster <asbrewster(a)lbl.gov> wrote: Moving the cctbx_project directory to cctbx_project.old forced svn to get the latest version from scratch so buildbot at LBL has no versioning issues like Jenkins at Diamond does. Further, svn update on my local sources got the latest version, so I'm inclined to agree with Markus that that's a Jenkins issue. I installed git on the build nodes at LBL to accommodate the changes to bootstrap.py, however reconfig.sh still fails. This new line from r22623 and r22626 is the culprit: git_available = self.shell(command=['git', '--version'], haltOnFailure=False, quiet=True, silent=True).run() == 0 While this clearly works on Jenkins, when ran by buildbot, self.shell() is returning a buildbot shell object instead of a Builder shell object, so the .run call fails (not implemented). I'm out of time until this afternoon to look deeper. I've left the build bot server running using r22622 of bootstrap.py for now. -Aaron On Sun, Jul 26, 2015 at 7:00 AM, Nigel Moriarty <nwmoriarty(a)lbl.gov<mailto:[email protected]>> wrote: Aaron Recent changes to include git in bootstrap.py have broken the buildbot master (won't start) because git is not available on the VM. I don't have time before I fly to ACA, so please install git when you can. One reason I was hoping to see the buildbot website was because it has an interface to see all the commits to cctbx. This is likely on the buildbot master in some other format. It may be useful that I just moved the old cctbx_project to cctbx_project.old on all the builds. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

lost cctbx commits
by Aaron Brewster 26 Jul '15

26 Jul '15

Hi folks, after more investigation I now agree we have parallel commits. The following commits have been lost, all done on July 15th: nwmoriatry: r22622 phyy-nx: r22623-4 tmmclark: r22625 olegsobolev, r22626-7, r22629 alyubimov: r22628 These revisions number have been reused by svn and are no longer valid. The authors should recommit their changes. -Aaron

2 2

git on BuildBot machines
by Nigel Moriarty 26 Jul '15

26 Jul '15

Aaron Recent changes to include git in bootstrap.py have broken the buildbot master (won't start) because git is not available on the VM. I don't have time before I fly to ACA, so please install git when you can. One reason I was hoping to see the buildbot website was because it has an interface to see all the commits to cctbx. This is likely on the buildbot master in some other format. It may be useful that I just moved the old cctbx_project to cctbx_project.old on all the builds. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

2 1

SVN is back on SourceForge
by Nigel Moriarty 26 Jul '15

26 Jul '15

Folks The return of SVN on SF has caused a few problems on the BuildBot but I moved the cctbx_project directory and forced some builds. I'm hoping things will progress but will check later tonight. There have been some strange error messages causing some emails to be send but I'm hoping things will settle soon. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

7 9

console
by Nigel Moriarty 26 Jul '15

26 Jul '15

I reverted the version of bootstrap.py on cci-vm-6 and got the attached console GUI. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

1 0

Re: [cctbxbb] after SourceForge awakens
by Graeme.Winter＠diamond.ac.uk 22 Jul '15

22 Jul '15

Hi Nigel We're thinking along the same lines moving xia2 across first then maybe dials, depending on how people feel. Moving all of it across in the longer term would work well. Thanks Graeme On 20 Jul 2015 20:42, Nigel Moriarty <nwmoriarty(a)lbl.gov> wrote: Folks I have not read the emails regarding the current SourceForge situation. However, we had been planning to move to github in the near future. SF has been very reliable so I don't think it was too much of pain given the long use we've had from them. It does, however, come at a time when we are trying to get out a major release of Phenix. To that end, it would be helpful if any commits were bug fixes only to minimise the risk of breaking to the codebase. We will be actively evaluating the options regarding to github after Phenix 1.10 is safely away... Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2