[cctbxbb] Debian Package

[Baptiste Carvello]

Hi everybody,

I'm working with Radostan, more specifically on the distutils
integration. I'm the author of the patches 0010-adding-setup_py and
0016-adapt-test_utils-in-libtbx-for-setup_py-test, so I'll discuss only
them.

Le 08/08/2012 01:31, Luc Bourhis a écrit :
> 0003-correct-paths-in-dispatcher-creation,
> 0008-Fix-to-skip-pycbf-build,
> 0016-adapt-test_utils-in-libtbx-for-setup_py-test,
> 0018-Fix-to-use-systems-include-path,
> 0019-Fix-to-skip-build-of-clipper-examples
> I have put those together because they participate to the same philosophy. They make sense only in the Debian environment you are designing, where the cctbx will depend on other packages, that will therefore be installed in standard locations if the cctbx is installed. But in agnostic an environment, where the cctbx dynamic libraries and Python modules are not in standard places, those patches break the build system and part of the runtime system. 
> For example, 0018 assumes there is gpp4/ccp4 somewhere on the header paths: that would require changing the packaging of Phenix to match that. This is so obvious that you can't have missed that. So am I missing something here?

The patch 0016-adapt-test_utils-in-libtbx-for-setup_py-test is meant to
allow the caller of the tests to pass the location of the test routines
("builddir" and "distdir") to the test runner.

I needed this feature to set "distdir" to the distutils build directory,
in order to make sure that all the tests and their data are correctly
copied by my setup.py script (conclusion: they are not yet, see below).
It would also be necessary in order to run the tests on the installed
debian system.

It may be, though, that this can be achieved more cleanly by
appropriately reconfiguring the pickled Environment object.

Radostan told me that this patch no longer applies on the newer nightly
builds of cctbx, but I didn't look into this problem yet.

> 0010-adding-setup_py: pending discussions
> I don't quite understand your code but this is orthogonal to our existing code.

This is indeed quite orthogonal to your code. The goal of this whole
distutils integration is twofold:

1) we need to be able to actually install the python part of cctbx,
which is normally just run from the source tree.

2) we need to make it easy for the Debian build system ("dh") to trigger
a rebuild for each python version supported by Debian. It makes sense to
use distutils because the Debian build system knows how to handle it.

So the "build_ext", "install_lib" and "clean" classes are really just
part of our "plumbing". They could probably be useful to other distros
as well.

The "install_data" class is mostly used to reconfigure the pickled
Environment object, by replacing all the build-time paths with the
run-time paths on the installed system.

> What do you need class build_py for e.g.?

The "build_py" class is an experiment at adding "from __future__ import
division" lines to python files, in case it would be needed to use them
without the "-Qnew" option of python. Indeed, in Debian, cctbx will have
to be compatible with the system python, without needing a global option.

Whether this "build_py" class is needed actually depends on you: if the
cctbx community is committed to keeping cctbx working also without the
"-Qnew" option, we don't need it, which is much better for us.

The "test" command is a first try at running the tests. Right now only
half of the tests work, because distutils only copies the importable
python packages, and many tests either are not importable, or need data
files. I should be able to solve this by fixing my setup.py script.

Other projects I have regarding tests are:

* make sure the test runner returns with an error code in case of test
failures, so that the Debian build system knows it must abort the build.

* make it possible to run the tests on installed systems.

Cheers,
Baptiste

PS: please continue to CC me in replies