[cctbxbb] Revisiting cctbx/iotbx/dxtbx dependencies

Aaron Brewster asbrewster at lbl.gov
Wed Feb 7 15:08:47 PST 2018

Hey folks, especially Rob and Luc.  So after a quick change to
iotbx/libtbx_config, it turns out there's a way to get a super minimum
version of cctbx, without configuring dxtbx, and it takes advantage of
existing mechanisms:

   - python bootstrap.py --builder=cctbx hot update
   - python
   modules/cctbx_project/libtbx/auto_build/install_base_packages.py openssl
   python python_compatibility certifi numpy cython png libsvm pytest
   pythonextra jinja2 junitxml biopython docutils sphinx pyopengl misc
   freetype matplotlib pillow gettext glib expat fontconfig render pixman tiff
   cairo gtk fonts wxpython
   - mkdir build; cd build
   - ../base/bin/python ../modules/cctbx_project/libtbx/configure.py cctbx
   - make

Now, some details as to why this works.  First, the cctbx builder in
boostrap.py doesn't download cbflib so cbflib is never configured.  Next,
the base step of bootstrap.py is simply a call to install_base_packages.py
with a list of required packages.  You can just run it standalone and get
the packages you need.  Here, I ran "python bootstrap.py --builder=cctbx
base", killed it after I saw the list of packages it was going to install,
then ran install_base_packages manually after removing hdf5 and lz4 (which
depends on hdf5) from the list.  One could imagine removing other packages
depending on your use case.  With that, I could run configure.py directly.
I can't use cctbx by itself as the cctbx/maptbx/SConscript file has a
dependency on iotbx, but configuring cctbx and iotbx together seems to work.

The key trick was to make iotbx only depend on dxtbx if dxtbx is explicitly
configured.  I used a keyword in libtbx_configure added by Luc a bit ago:
optional_modules_only_if_explicit_request.  I committed that change just

Outstanding questions:

   1. Is there interest in making this a builder? It would be
   straightforward to add a builder named cctbx_minimal which would just do
   the above steps
   2. Is there further need for making cbflib and hdf5 optional in dxtbx?
   I purport there is not as the above procedure seems to satisfy the use
   case, but I can be persuaded otherwise.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20180207/713059ca/attachment.htm>

More information about the cctbxbb mailing list