September 2015 - cctbxbb - phenix-online.org

dependency on phenix
by Roger Martin 26 Jan '16

26 Jan '16

Hi, looking through http://sourceforge.net/p/cctbx/code/HEAD/tree/trunk/mmtbx/refinement/minimi… shows at line 61 from phenix.refinement import weight_xray_chem and http://sourceforge.net/p/cctbx/code/HEAD/tree/trunk/mmtbx/refinement/occupa… line 134 Is weight_xray_chem.py code that could move to cctbx? (cctbx_project/mmtbx/refinement/weight_xray_chem.py) Are there other cross dependencies?

2 2

Failure in phenix.map_box when working with map files....
by Ezra Peisach 12 Dec '15

12 Dec '15

Has anyone observed a failure of phenix.map_box when working with map files? When working with a structure factor file with map coefficients, it works fine. When a map file is used - the following occurs: Traceback (most recent call last): File "/local/scratch/CCTBX/cctbx_build/../cctbx_sources/mmtbx/command_line/map_box.py", line 194, in <module> run(args=sys.argv[1:]) File "/local/scratch/CCTBX/cctbx_build/../cctbx_sources/mmtbx/command_line/map_box.py", line 158, in run selection = selection) File "/local/scratch/CCTBX/cctbx_sources/mmtbx/utils/__init__.py", line 2938, in __init__ self.map_box = maptbx.copy(map_data, gridding_first, gridding_last) Boost.Python.ArgumentError: Python argument types in cctbx_maptbx_ext.copy(double, list, list) did not match C++ signature: copy(scitbx::af::const_ref<double, cctbx::maptbx::c_grid_padded_p1<3ul> > map_unit_cell, scitbx::af::tiny<int, 3ul> first, scitbx::af::tiny<int, 3ul> last) copy(scitbx::af::const_ref<float, cctbx::maptbx::c_grid_padded_p1<3ul> > map_unit_cell, scitbx::af::tiny<int, 3ul> first, scitbx::af::tiny<int, 3ul> last) copy(scitbx::af::const_ref<double, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > map, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > result_grid) copy(scitbx::af::const_ref<float, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > map, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > result_grid) Any suggestions? The map_data is the same format in the call....

4 5

Unused/old modules on the top-level of cctbx
by Oleg Sobolev 18 Oct '15

18 Oct '15

Hi, I went through top-level directories of cctbx and discovered the following: compcomm - two papers from IUCr computational crystallography authored by Ralf with some scripts. These scripts are referenced in these papers. One paper in LaTeX by Luc Bourhis "Small-molecule toolbox". dox - probably was meant as documentation, actually contains some documentation in RST, Ralf's talks from Siena, 2005, some tutorial in LaTeX, the content for the cctbx website. fftw3tbx - probably is not used anywhere ipctbx - most likely is not used anywhere, wrapper for sys/ipc.h - C library for interprocess communication access structure. rpmbuild - not sure whether it is still needed/working, last change to the only file was on 7-29-2009. setup - not clear whether the only .sh script in this directory is still needed/working. I suggest to move compcomm to dox despite the reference in 2009 paper and remove fftw3tbx, ipctbx, rpmbuild, setup if nobody has an idea why they should stay where they are/be moved somewhere. Best regards, Oleg Sobolev.

3 3

CentOS7 - fixed
by markus.gerstel＠diamond.ac.uk 30 Sep '15

30 Sep '15

Thanks for your suggestions. They did not do the trick though. Some more detective work got me down to ../base/bin/python ../modules/cctbx_project/libtbx/configure.py dials failing and both libtbx.python ../modules/cctbx_project/libtbx/configure.py dials and LD_LIBRARY_PATH=lib ../base/bin/python ../modules/cctbx_project/libtbx/configure.py dials succeeding. A run with LD_DEBUG=files identified the problem: /lib64/libboost_python.so: error: symbol lookup error: undefined symbol: PyUnicodeUCS4_FromEncodedObject (fatal) CentOS7 comes with its own set of incompatible boost libraries. I've put a fix in: http://sourceforge.net/p/cctbx/code/22986/ Installation on CentOS7 now works for me. As far as I could tell this does not break other things. -Markus ________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of Nigel Moriarty [nwmoriarty(a)lbl.gov] Sent: Tuesday, September 29, 2015 18:36 To: cctbx mailing list Subject: Re: [cctbxbb] CentOS7 bugs (cont.) libtbx.configure --scan-boost phenix Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> On Tue, Sep 29, 2015 at 10:20 AM, Nicholas Sauter <nksauter(a)lbl.gov<mailto:[email protected]>> wrote: Nigel, Do remember how to force libtbx.scons to scan the boost directory? Nick On Tue, Sep 29, 2015 at 9:25 AM, Nigel Moriarty <nwmoriarty(a)lbl.gov<mailto:[email protected]>> wrote: Markus The default usage of libtbx.scons in the build directory does not scan the boost directory so maybe that has something to do with the .so not behaving on Centos7 - because it was built on Centos5/6? Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709<tel:510-486-5709> Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909<tel:510-486-5909> Web : CCI.LBL.gov<http://CCI.LBL.gov> On Tue, Sep 29, 2015 at 8:29 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> wrote: Hello cctbx folks, We are having problems running, building and/or installing DIALS on CentOS7. Running the linux installer unceremoniously ends in a segfault. The installer is even broken when it is build on the same system using bootstrapped cctbx/dials. The broken installer can be traced to the DIALS extension loader, which uses python’s pkgutil loader, which triggers some library scanning routine within python, which in turn falls over boost_python_meta_ext.so and dies with the strace shown in the email below. The installer can be successfully “fixed” by deleting build/lib/boost_python_meta_ext.so during the installation. So I see two issues here: 1. What is broken with build/lib/boost_python_meta_ext.so that it causes segfaults on CentOS7? 2. Why is this not covered by any test? Neither of the following caught anything: a. python bootstrap.py builder=dials tests b. libtbx.run_tests_parallel module=cctbx module=dials module=boost_adaptbx [NB: The previously reported 3 broken tests on CentOS7 are red herrings and are all fixed by running ‘yum update’] I don’t know how to proceed from here. -Markus From: markus.gerstel(a)diamond.ac.uk<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] Sent: 29 September 2015 13:29 To: dials-support(a)lists.sourceforge.net<mailto:[email protected]> Subject: Re: [Dials-support] CentOS7 bugs (cont.) Hello everyone, (..) Sorry, I hit ctrl+enter by accident… In addition to these failed tests the installer dies a boost-related death according to strace: (..) poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 1 ([{fd=13, revents=POLLIN}]) read(13, "\tlinux-vdso.so.1 => (0x00007fff"..., 4096) = 486 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7648, si_status=0, si_utime=0, si_stime=0} --- poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 2 ([{fd=13, revents=POLLHUP}, {fd=15, revents=POLLHUP}]) close(13) = 0 close(15) = 0 wait4(7648, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 7648 stat("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext", 0x7fff6421d3f0) = -1 ENOENT (No such file or directory) open("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext.so", O_RDONLY) = 11 fstat(11, {st_mode=S_IFREG|0775, st_size=200624, ...}) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18a86} --- +++ killed by SIGSEGV (core dumped) +++ Segmentation fault (core dumped) Not sure if relevant, but CentOS7 compiler is GCC 4.8.3 20140911 (Red Hat 4.8.3-9) compared to RHEL6 GCC 4.4.7 20120313 (Red Hat 4.4.7-11) Anybody have any ideas? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb -- Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Physical Biosciences Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 64R0121 Berkeley, CA 94720-8118 (510) 486-5713<tel:%28510%29%20486-5713> _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

1 0

Re: [cctbxbb] CentOS7 bugs (cont.)
by markus.gerstel＠diamond.ac.uk 29 Sep '15

29 Sep '15

Hello cctbx folks, We are having problems running, building and/or installing DIALS on CentOS7. Running the linux installer unceremoniously ends in a segfault. The installer is even broken when it is build on the same system using bootstrapped cctbx/dials. The broken installer can be traced to the DIALS extension loader, which uses python's pkgutil loader, which triggers some library scanning routine within python, which in turn falls over boost_python_meta_ext.so and dies with the strace shown in the email below. The installer can be successfully "fixed" by deleting build/lib/boost_python_meta_ext.so during the installation. So I see two issues here: 1. What is broken with build/lib/boost_python_meta_ext.so that it causes segfaults on CentOS7? 2. Why is this not covered by any test? Neither of the following caught anything: a. python bootstrap.py builder=dials tests b. libtbx.run_tests_parallel module=cctbx module=dials module=boost_adaptbx [NB: The previously reported 3 broken tests on CentOS7 are red herrings and are all fixed by running 'yum update'] I don't know how to proceed from here. -Markus From: markus.gerstel(a)diamond.ac.uk [mailto:[email protected]] Sent: 29 September 2015 13:29 To: dials-support(a)lists.sourceforge.net Subject: Re: [Dials-support] CentOS7 bugs (cont.) Hello everyone, (..) Sorry, I hit ctrl+enter by accident... In addition to these failed tests the installer dies a boost-related death according to strace: (..) poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 1 ([{fd=13, revents=POLLIN}]) read(13, "\tlinux-vdso.so.1 => (0x00007fff"..., 4096) = 486 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=7648, si_status=0, si_utime=0, si_stime=0} --- poll([{fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}], 2, 4294967295) = 2 ([{fd=13, revents=POLLHUP}, {fd=15, revents=POLLHUP}]) close(13) = 0 close(15) = 0 wait4(7648, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 7648 stat("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext", 0x7fff6421d3f0) = -1 ENOENT (No such file or directory) open("$prefix/dials-dev20150928/build/lib/boost_python_meta_ext.so", O_RDONLY) = 11 fstat(11, {st_mode=S_IFREG|0775, st_size=200624, ...}) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x18a86} --- +++ killed by SIGSEGV (core dumped) +++ Segmentation fault (core dumped) Not sure if relevant, but CentOS7 compiler is GCC 4.8.3 20140911 (Red Hat 4.8.3-9) compared to RHEL6 GCC 4.4.7 20120313 (Red Hat 4.4.7-11) Anybody have any ideas? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Phi-Psi Angles
by Nicholas Pearce 21 Sep '15

21 Sep '15

Hi Everyone, Phenomenally stupid question, but is there an easy way to calculate phi-psi angles for a protein using cctbx? And, following on from that, Chi-1, Chi-2 etc angles? I’ve had a poke around, but couldn’t find any obvious (at least to me) way that worked… I presume this must be implemented rigorously somewhere, and I didn’t want to waste time reimplementing the wheel. Thanks, Nick

2 2

Move to github
by Graeme.Winter＠diamond.ac.uk 16 Sep '15

16 Sep '15

Hi Folks A while back we asked about moving cctbx (along with DIALS) to github; postponement was asked until Phenix release is out. Phenix release is out. How do people feel about this? Enthusiastic endorsement of this project from DIALS-East & xia2 developer camps. Latter has already migrated… Go? No go? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Moving to github
by Graeme.Winter＠diamond.ac.uk 08 Sep '15

08 Sep '15

Hi Folks Looks like phenix release has happened… should we begin the conversation about moving cctbx and DIALS over to the happy world of github? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 3