[cctbxbb] Misuse of libtbx/configure.py

Wed Apr 4 10:37:40 PDT 2018

Hi Billy,

orderedset is a requirement for dials, specifically dials.image_viewer, and has been added to the base build on the 19th of March. Are you updating your base build as part of the nightly build?

https://github.com/cctbx/cctbx_project/commit/f748d1d1d5ea5d64b92b7cdf44e8307d6de00573

We now list our package dependencies analogous to the standard python packaging guidelines in libtbx_refresh.py. Ordinarly you would find them in a setup.py, but we do not have one at this time.

To disable the automatic installation and updating of dependencies you can set the environment variable LIBTBX_DISABLE_UPDATES but be aware that this means that your build is still broken since you are missing required dependencies.

Some discussion of this happened in/around #151.

-Markus

________________________________
From: cctbxbb-bounces at phenix-online.org <cctbxbb-bounces at phenix-online.org> on behalf of Robert Oeffner <rdo20 at cam.ac.uk>
Sent: Wednesday, April 4, 2018 17:40
To: cctbx mailing list
Subject: [cctbxbb] Misuse of libtbx/configure.py

Hi,

I'm not sure when this has been put into configure.py as I have had
taken my eyes of the ball lately regarding Windows builds. But it seems
that during bootstrap build the configure.py step is now doing more than
just configuring modules. On my PC the excerpt below from stdout shows
an error during the configure.py step

Processing:
"c:\busers\oeffner\nightlybuild\x32\current\modules\dials\libtbx_refresh.py"
requires mock>=2.0, has 2.0.0
requirement orderedset is not currently met, package not installed
attempting install of orderedset...
Collecting orderedset
   Using cached orderedset-2.0.1.tar.gz
Installing collected packages: orderedset
   Running setup.py install for orderedset: started
     Running setup.py install for orderedset: finished with status 'error'
     Complete output from command
C:\Busers\oeffner\NightlyBuild\x32\Current\base\bin\python\python.exe -u
-c "import setuptools,
tokenize;__file__='c:\\users\\oeffner\\appdata\\local\\temp\\pip-build-smhifw\\orderedset\\setup.py';f=getattr(tokenize,
'open', open)(__file__);code=f.read().replace('\r\n',
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
c:\users\oeffner\appdata\local\temp\pip-0seb8q-record\install-record.txt
--single-version-externally-managed --compile:
     running install
     running build
     running build_py
     creating build
     creating build\lib.win32-2.7
     creating build\lib.win32-2.7\orderedset
     copying lib\orderedset\__init__.py -> build\lib.win32-2.7\orderedset
     running build_ext
     building 'orderedset._orderedset' extension
     error: Microsoft Visual C++ 9.0 is required (Unable to find
vcvarsall.bat). Get it from http://aka.ms/vcpython27

----------------------------------------
Command
"C:\Busers\oeffner\NightlyBuild\x32\Current\base\bin\python\python.exe
-u -c "import setuptools,
tokenize;__file__='c:\\users\\oeffner\\appdata\\local\\temp\\pip-build-smhifw\\orderedset\\setup.py';f=getattr(tokenize,
'open', open)(__file__);code=f.read().replace('\r\n',
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
c:\users\oeffner\appdata\local\temp\pip-0seb8q-record\install-record.txt
--single-version-externally-managed --compile" failed with error code 1
in c:\users\oeffner\appdata\local\temp\pip-build-smhifw\orderedset\
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'python -m pip install --upgrade
pip' command.
install failed. please check manually

My first concern with this approach is that the bootstrap process
doesn't stop. It carries on and eventually builds the Phenix software
suite. This may or may not be a broken build as it is not clear whether
the above failure to build orderedset was a showstopper for something.

Secondly, I thought that modules would not be installed anywhere else
than when building base components or when downloading sources. For a
small project it is a nice and sneaky solution to have certain build
steps second guessing what dependencies are missing and then installing
them "on the fly", if not covertly. But for a large project such as
CCTBX with several teams using that as a common framework this becomes
messy I would argue.

A quick and dirty fix for the above error might be just to put
vcvarsall.bat in the PATH environment. But that is not a solution since
bootstrap.py has been fashioned to use different versions of Visual C++,
some of which are not invoked through a vcvarsall.bat file.

I appreciate the need to install additional component modules for
python. But would it not be more tidy to install these during the base
step of the bootstrap process? Is there any reason why this cannot happen?

Regards,

Rob

--
Robert Oeffner, Ph.D.
Research Associate, The Read Group
Department of Haematology,
Cambridge Institute for Medical Research
University of Cambridge
Cambridge Biomedical Campus
Wellcome Trust/MRC Building
Hills Road
Cambridge CB2 0XY

www.cimr.cam.ac.uk/investigators/read/index.html<http://www.cimr.cam.ac.uk/investigators/read/index.html>
tel: +44(0)1223 763234

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom