- cctbxbb - phenix-online.org

cctbx has acquired a hidden dependency on iotbx
by Luc Bourhis 27 May '17

27 May '17

Hi, C:\Users\luc\Documents\Developer\cctbx\modules\cctbx_project\cctbx\maptbx\SConscript, line 5-8: env = env_base.Clone( SHLINKFLAGS=env_etc.shlinkflags, LIBS=["iotbx_xplor", "cctbx_sgtbx_asu", "cctbx", "omptbx", env_etc.libm] ) This is not supposed to be: the cctbx shall be usable with only what’s defined in its libtbx_config. Best wishes, Luc

3 2

shall iotbx really depends on dxtbx?
by Luc Bourhis 26 May '17

26 May '17

Hi, iotbx/libtbx_config: { "modules_required_for_use": ["cctbx", "fable", "ucif", "smtbx", "dxtbx"], "optional_modules": ["ccp4io", "cbflib"], } A search for dxtbx in the source code of iotbx does not return anything. Did I miss something? Moreover dxtbx/libtbx_config: { "modules_required_for_use" : [ "iotbx" ], } The problem is that dxtbx pulls in HDF5 and that it fails to compiles on Windows 64 with a 64-bit Python, a configuration we need for Olex 2. If the solution was as simple as dropping the dependence of iotbx on dxtbx, that would make my day! Thoughts? Best wishes, Luc Bourhis

3 4

probe/reduce repositories
by Hattne, Johan 04 May '17

04 May '17

Dear all; I appear to be missing probe in my source build. On running phenix.model_vs_data I get the traceback posted below. The error message appears to be a tad misleading, because having probe in PATH is not enough: the actual test in the code is if (not libtbx.env.has_module(name="probe")): which leads me to believe that I’m supposed to have something probe-like in my source tree, but the duke-server I used to get these kinds of things from (quidditch?) seems unresponsive from here. This is the first time in weeks that I’m doing this, so it is very much likely that whatever happened to break my setup happened quite some time ago. Cluebat, anyone? // Cheers; Johan Traceback (most recent call last): File "/mnt/gonen/home/hattnej/projects/phenix-build/../phenix-src/phenix/phenix/command_line/model_vs_data.py", line 6, in <module> show_geometry_statistics = True) File "/mnt/gonen/home/hattnej/projects/phenix-src/cctbx_project/mmtbx/model_vs_data.py", line 710, in run atom_selections = atom_selections) File "/mnt/gonen/home/hattnej/projects/phenix-src/cctbx_project/mmtbx/model_vs_data.py", line 346, in show_geometry restraints_manager = restraints_manager) File "/mnt/gonen/home/hattnej/projects/phenix-src/cctbx_project/mmtbx/model_statistics.py", line 138, in __init__ molprobity_scores=molprobity_scores) File "/mnt/gonen/home/hattnej/projects/phenix-src/cctbx_project/mmtbx/model_statistics.py", line 51, in __init__ self.clashscore = clashscore(pdb_hierarchy=pdb_hierarchy).get_clashscore() File "/mnt/gonen/home/hattnej/projects/phenix-src/cctbx_project/mmtbx/validation/clashscore.py", line 90, in __init__ "http://kinemage.biochem.duke.edu/") RuntimeError: Probe could not be detected on your system. Please make sure Probe is in your path. Probe is available at http://kinemage.biochem.duke.edu/ Research Specialist @ Gonen Lab ____________________________________________________ Janelia Research Campus * 19700 Helix Drive Ashburn, VA 20147 * +1 (571) 209-4000 extension 3376

2 5

normalize() is not safe
by Oleg Sobolev 02 May '17

02 May '17

Dear colleagues, I recently discovered that normalize() function of scitbx::vec3<double> is not safe, i.e. this case scitbx::vec3<double> a(0,0,0); a.normalize(); it will produce division by zero and fail. If function contains this being called from python the traceback would be rather vague: <...> Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. The code that generates this error sits in scitbx/vec3.h, lines 143-153 with a comment that this was done intentionally and a pointer to an appropriate function to do safe normalization. This function is not wrapped to be available in python directly. Similar functionality (but coded in a different place) accessible via python by: >>> from scitbx import matrix >>> b = matrix.col([0,0,0]) >>> b.normalize() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/net/anaconda/raid1/olegs/phenix_queue/modules/cctbx_project/scitbx/matrix/__init__.py", line 270, in normalize return self / abs(self) File "/net/anaconda/raid1/olegs/phenix_queue/modules/cctbx_project/scitbx/matrix/__init__.py", line 158, in __truediv__ return rec([e/other for e in self.elems], self.n) ZeroDivisionError: float division by zero Fast grepping shows that normalize() is widely used in xfel, smtbx, rstbx, dxtbx, mmtbx/hydrogens, scitbx/rigid_body, and in some other places. My questions to CCTBX community are: 1. Were you aware of this issue? 2. Should we do something about it: - make the function more safe and slow; - leave it like it is and let everybody know about the issue and let developers deal with it individually; 3. Any other input on the matter? Best regards, Oleg Sobolev.

5 5

Boost Python 1.56
by Luc Bourhis 14 Apr '17

14 Apr '17

Hi, I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56. So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling. Best wishes, Luc

14 36

Re: [cctbxbb] [git/cctbx] master: Replacing illegal C++ code with proper dynamic array allocation. (a31eb2bf0)
by Nicholas Sauter 12 Apr '17

12 Apr '17

Rob, Just as a general thing, "new" statements in C++ need to be paired with "delete" statements (delete pk, delete a_sum, and delete akml). Unless this is done there will be a memory leak. Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Tue, Apr 11, 2017 at 3:31 AM, CCTBX Commit via DLS Jenkins < graeme.winter(a)gmail.com> wrote: > Repository : ssh://g18-sc-serv-04.diamond.ac.uk/cctbx > On branch : master > > ------------------------------ > > > commit a31eb2bf0d5bb129425bb759466f45622c773fd4 > Author: Robert Oeffner <rdo20(a)cam.ac.uk> > Date: Tue Apr 11 11:31:24 2017 +0100 > > Replacing illegal C++ code with proper dynamic array allocation. > > > ------------------------------ > > > a31eb2bf0d5bb129425bb759466f45622c773fd4 > scitbx/lbfgs.h | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/scitbx/lbfgs.h b/scitbx/lbfgs.h > index 41be8e8cc..f0000143d 100755 > --- a/scitbx/lbfgs.h > +++ b/scitbx/lbfgs.h > @@ -917,9 +917,9 @@ namespace lbfgs { > int info = 5; > brackt = false; > FloatType sxnorm = std::sqrt(ddot(SizeType(n), sx, sx)); > - FloatType pk[n]; > - FloatType a_sum[n]; > - FloatType akm1[n]; > + FloatType* pk = new FloatType[n]; > + FloatType* a_sum = new FloatType[n]; > + FloatType* akm1 = new FloatType[n]; > for (SizeType i=0; i < n; i++) { > pk[i] = sx[i]/sxnorm; > akm1[i] = -gx[i]/sxnorm; >

3 2

libtiff dependency on macOS
by Billy Poon 11 Apr '17

11 Apr '17

Hi everyone, I just updated install_base_packages.py to build libtiff generally instead of just for Linux. So if you update your sources and get build errors relating to TIFF, please rebuild your base directory. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

Re: [cctbxbb] bug fix in cctbx_project/scitbx/lbfgs.h
by R.D. Oeffner 11 Apr '17

11 Apr '17

There is now a fatal warning in libtbx/SConscript if gcc encounters variable length arrays as this is not portable code and we've had broken builds on more than one occasion due to this recently. Rob On 11/04/2017 12:01, Pavel Afonine wrote: > Hi Rob, > > thanks for the fix. This is was Min's code (remember her from the past workshop?). I simply applied her patch without looking into it too much, and made sure all t96 pass. > > Again, thanks for fixing it. > > Pavel > > On 4/11/17 03:48, R.D. Oeffner wrote: > >> Hi Pavel, I have made a change to that file as it didn't compile on Windows. In fact I can't understand how it would compile with other compilers before as the syntax to me seemed illegal given that n is not known at compile time. I'm not sure how a dynamically created FloatType should be destroyed as I couldn't find any examples of that being done throughout this file. Rob

2 1

bootstrap.py on windows
by Horst Puschmann 07 Apr '17

07 Apr '17

Hello I am trying to install the cctbx on a Windows 7 64 bit machine using the following bootstrap.py file: https://raw.githubusercontent.com/cctbx/cctbx_project/master /libtbx/auto_build/bootstrap.py The script will fail, unless I comment out lines 1101 and 1102: #if self.isPlatformWindows(): #tarurl, arxname, dirpath = MODULES.get_module(module)(). get_tarauthenticated(auth=self.get_auth()) If it tries to execute the commented lines, the error will be "KeyError: 'cciuser' in line 590" After commenting this out, it will fail in line 271, with "AttributeError: 'module' object has no attribute '_create_unverified_context'" I can get round that with disabling line 247: if sys.platform == "win32": (i.e. change this to if sys.platform == "xxx":) After that, things start downloading. All appears well until the same thing happens again in another bootstrap.py file in \modules\cctbx_project\ libtbx\auto_build\bootstrap.py If I disable *that*, it goes further but fails finally with ===== Running in build: run configure.py Traceback (most recent call last): File "bootstrap.py", line 2113, in <module> run() File "bootstrap.py", line 2108, in run enable_shared=options.enable_shared, File "bootstrap.py", line 1036, in run i.run() File "bootstrap.py", line 189, in run raise RuntimeError("Could not run %s: File not found" % executable) RuntimeError: Could not run base\bin\python: File not found I guess the automatic tests haven't picked this up, because no authentication is needed, maybe? Greetings Horst

6 16

Fw: cctbx developer guidelines
by R. D. Oeffner 06 Apr '17

06 Apr '17

Hi Pavel and others, Attached is the original word document you sent out if that's useful for anyone. Rob -----Original Message----- From: Pavel Afonine Sent: Saturday, July 11, 2015 7:12 PM To: Paul Adams ; developers(a)phenix-online.org Subject: Re: cctbx developer guidelines Hi Paul, looks great! Attached is slightly edited copy. Pavel On 7/10/15 11:14, Paul Adams wrote: > Hi Everyone, > > we have developed a cctbx developer guidance document (much of which > also applies to Phenix development). I’m attaching it for your review and > comments. If you have any suggestions let me know. The goal is to > distribute this to all developers who are contributing to the cctbx and to > put it somewhere prominent on the web. > > Cheers, > Paul >

1 0