[cctbxbb] Revisiting cctbx/iotbx/dxtbx dependencies

[R. D. Oeffner]

Hi Aaron,

Thank you for those amendments to bootstrap. I just tested cctbxlite with 
amendments to also build phaser from bootstrap. It simplifies the build 
steps for building phaser so sometime soon I'd like to add a separate 
builder to bootstrap that builds phaser only. Perhaps cctbxlite could form 
the basis for a python wheel installer.

Rob

From: Aaron Brewster
Sent: Saturday, February 10, 2018 12:58 AM
To: Asmit Bhowmick
Cc: cctbx mailing list ; R.D. Oeffner ; Luc Bourhis
Subject: Re: [cctbxbb] Revisiting cctbx/iotbx/dxtbx dependencies

Hi folks, you can now run python bootstrap.py --builder=cctbxlite to get a 
lite version of cctbx.  It will follow the steps listed above, skipping 
hdf5, cbflib and dxtbx.

I'd like to close issues 125 and 126 unless someone objects.
https://github.com/cctbx/cctbx_project/issues/125
https://github.com/cctbx/cctbx_project/issues/126

Thanks!
-Aaron

On Thu, Feb 8, 2018 at 8:50 PM, Asmit Bhowmick <abhowmick at lbl.gov> wrote:

For the 2nd step that Aaron mentioned, if certain packages are to be skipped 
during the base step, you can also run bootstrap.py with the 
flag --skip-base-packages and specify the packages to be skipped separated 
by a comma. For example

python bootstrap.py base --builder=cctbx --skip-base-packages 
hdf5,lz4_plugin

Best Wishes
Asmit

On Wed, Feb 7, 2018 at 3:08 PM, Aaron Brewster <asbrewster at lbl.gov> wrote:

Hey folks, especially Rob and Luc.  So after a quick change to 
iotbx/libtbx_config, it turns out there's a way to get a super minimum 
version of cctbx, without configuring dxtbx, and it takes advantage of 
existing mechanisms:

python bootstrap.py --builder=cctbx hot update

python modules/cctbx_project/libtbx/auto_build/install_base_packages.py 
openssl python python_compatibility certifi numpy cython png libsvm pytest 
pythonextra jinja2 junitxml biopython docutils sphinx pyopengl misc freetype 
matplotlib pillow gettext glib expat fontconfig render pixman tiff cairo gtk 
fonts wxpython

mkdir build; cd build

../base/bin/python ../modules/cctbx_project/libtbx/configure.py cctbx iotbx

make

Now, some details as to why this works.  First, the cctbx builder in 
boostrap.py doesn't download cbflib so cbflib is never configured.  Next, 
the base step of bootstrap.py is simply a call to install_base_packages.py 
with a list of required packages.  You can just run it standalone and get 
the packages you need.  Here, I ran "python bootstrap.py --builder=cctbx 
base", killed it after I saw the list of packages it was going to install, 
then ran install_base_packages manually after removing hdf5 and lz4 (which 
depends on hdf5) from the list.  One could imagine removing other packages 
depending on your use case.  With that, I could run configure.py directly. 
I can't use cctbx by itself as the cctbx/maptbx/SConscript file has a 
dependency on iotbx, but configuring cctbx and iotbx together seems to work.

The key trick was to make iotbx only depend on dxtbx if dxtbx is explicitly 
configured.  I used a keyword in libtbx_configure added by Luc a bit ago: 
optional_modules_only_if_explicit_request.  I committed that change just 
now.

Outstanding questions:

Is there interest in making this a builder? It would be straightforward to 
add a builder named cctbx_minimal which would just do the above steps
Is there further need for making cbflib and hdf5 optional in dxtbx?  I 
purport there is not as the above procedure seems to satisfy the use case, 
but I can be persuaded otherwise.


Thanks,
-Aaron

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb






-- 

Asmit Bhowmick, Ph.D
Molecular Biophysics & Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33, Room 349B
Berkeley, CA 94720
Cell: 510.684.6011




Virus-free. www.avg.com