[cctbxbb] RE : Debian Package

Wed Aug 8 08:37:45 PDT 2012

Hello,

to present myself first, I am the Debian Developper who "mentor" the
work of Radostan and Baptiste. I am also working in the French
synchrotron SOLEIL [1]. I am in charge of the diffractometer's
calculation library [2] used with the tango control system [3]. I am
also preparing what we call a Debian Pure Blend [4] related to Photon
and Neutron facilities [5] (work in progress). During the last Debian
Science Sprint at the ESRF [6], I met one guyes of the laboratory
where I did my PhD (material science). This guy is working with
Vincent Fabre Nicolin and use his pynx software [7] which optionaly
use cctbx. This is why I started to interest myself to cctbx. Looking
in your mailing liste I discovert that Radostan was interested in the
cctbx Debian packaging. so I contact him and voilà...

> I appreciate the effort of yours. Making the cctbx installable with
> aptitude is great indeed. Then, as you mentioned, it would be great
> if it became installable with yum on Fedora-like system, etc. Making
> it installable with distutils may open the possibility to use
> easy_install or pip to install the cctbx. More of a gimmick imho but
> the average Python user seems to expect that these days.

thanks, we are doing this work in the spirit to ease as much as
possible the installation and maintainability of cctbx for
non-specialist and indeed also for system administrators. As you may
not know the ESRF as decided to use Debian as it first OS on most of
it's infrastructur. So packaging software they are using seems to be
important for long term maintainability of the facility. Nevertheless
in the Debian social contract [8] we try as much as possible to
forward our work to the upsteam and to be non-debian specific. So all
the FLOSS should beneficiate of our work.

> First a general comment: you have been using git in a manner that I
> find suboptimal. It would have been much easier for us (and much more
> in the spirit of git) if you had asked us to make a public git
> repository (I exclusively work with git for the record, using git svn
> to interact with sourceforge, so I could have provided one on no
> time), and then forked it. Indeed we would have been able to simply
> check out your repo into a branch of our public repo, and then
> immediately test your changes, and eventually apply those that passes
> the trial of fire. Actually, as pointed out in my comments below, we
> can't even apply your patches because some seem to be missing.

Here I would like to explain why I asked Radostand to work the way it
is now. We as "Debian Science team" use a common infrastructure for
our team work (mostly packaging effort). We are packaging only
released (stable) software, so we started the cctbx packaging from the
latest released version 2012.05.08.2305. It is important for us to
host the packaging repository (every member of the Debian-Science team
can commit on this repository in case there is a problem with the
current maintainer, vacation etc...). Now that we manage to package
this latest stable release (still some work to do), we decided to
discuss with you about integration of part of our patch series. Now we
completly agreed that it would be a lot easyer for all of us if the
patch series provided were generated on top of your latest git
repository. As a first contact it is important for us to know your
opinion about the proposed pacthes before working on the git head of
the cctbx repository. Can you give us the URL of this repository ?

> 0002-fix-opengl-header-missing-gltbx: rejected
> Do you really want to force all cctbx users to install OpenGL? Even if
> they don't need it because e.g. they run cctbx-based scripts as the
> back end of a web server?

the packaging of cctbx is done like this (for now):
- 2 binary package per shared library libxxx, libxxx-dbg
- 1 binary package for all the devel files
- 1 binary package for all the python modules and extensions.
  python-cctbx_2012.05.08.2305-1_i386.deb

So yes for now apt-get install python-cctbx pull also openGL libraries
(<30 Mo on my computer). The room on a server is no more a problem
nowaday. we can find harddisk of 1To for less than 250euros. Indeed we
should also split python-cctbx under finer grain packages, but is it
worth the effort ? This can indeed be discussed.

> 0003-correct-paths-in-dispatcher-creation,
> 0008-Fix-to-skip-pycbf-build,
> 0016-adapt-test_utils-in-libtbx-for-setup_py-test,
> 0018-Fix-to-use-systems-include-path,
> 0019-Fix-to-skip-build-of-clipper-examples
> I have put those together because they participate to the same
> philosophy. They make sense only in the Debian environment you are
> designing, where the cctbx will depend on other packages, that will
> therefore be installed in standard locations if the cctbx is
> installed. But in agnostic an environment, where the cctbx dynamic
> libraries and Python modules are not in standard places, those patches
> break the build system and part of the runtime system.  For example,
> 0018 assumes there is gpp4/ccp4 somewhere on the header paths: that
> would require changing the packaging of Phenix to match that. This is
> so obvious that you can't have missed that. So am I missing something
> here?

So in a first time for this 0018 patch it should be possible to fix it
providing during the build the right -I arguments.  Let's rework this
part on our side before going further.

> 0006-options-for-system-libs-installtarget-and-prefix: needs thorough testing
> I approve the spirit of it but this patch introduces a truckload of
> changes and that needs to stand the trial of our nightly tests.  Note
> that you use a couple of new methods, e.g. env_etc.check_syslib, that
> none of the patches define as far as I can tell.

let's see with Radostan

> 0007-adding-shlib-versioning: accepted
> The new build option libtoolize seems properly introduced. Beyond that
> I must admit I am rather clueless about libtool.  Anyway, if configure
> is not run with --libtoolize, this won't impact us!

great, now your just need to learn about so verisonning :) [9]

> 0009-build-libann-statically: pending explanations
> could you explain why you need to build this one statically only?

You can find the explaination on the wiki [10]

> 0011-fix-missing-python-lib-during-linking: needs tidying up
> Why don't you append to env_etc.libs_python instead of created the
> string env_etc.py_lib? We try to use list as much as possible in the
> SConscript.

I let others answer this question

> 0012-fix-to-remove-cctbx.python-interpreter: rationale? And trunk has moved on anyway
> Why do you need to remove cctbx.python?
> uc1_2_reeke.py has been removed and there is now uc1_2_a.py that features cctbx.python too

this is part of the questions we would like to ask you.  We want to
use the default Debian python interpreter, so we need to change all
#!/usr/bin/env xxxx.python with #!/usr/bin/env python in all your
files.  If I remember correctly this job is also done by distutils
[11] is declare as a script.

we do not know now how to deal with the mutiple scripts that you are
provinding in your bin directory. we call them dispatcher scripts
(look at the wiki [10]).my concerne is with third party softwares
which are relying on them in the PATH. Policy of Debian says that for
each binary in /usr/bin we should provide a man pages so in our case
this should be an issue...

So for now we mostly worked on the python modules and to provide the
libraries.

> 0013-fix-to-support-LDFLAGS-in-use_enviroment_flags: not sure
> This seems done in orthodox a manner. However, this has the potential
> of wrecking havoc to Phenix on some machines where LDFLAGS is set in
> fancy ways.

This is also true with the other flag also, why do you treat LDFLAGS
differently than others ?

In that case as explained by Radostan, Debian need to tune the build
process by providing their own build flags. The trade-off would be to
add a config flag that should allow or not to use the LDFLAGS
--use-also-LDFLAGS

what is your opinion ?

> 0015-fix-cif-parser-to-work-with-antlr3c-3.2: for Richard's eyes
> Richard (Gildea) is the expert when it comes to ANTLR

this should be problematic as Debian provide only 3.2 for now.  We
shoul dask for the packaging of 3.4, but as you told us you also
pacthed it. Did you forwarded you changed to the antlr3c upstream ?
If not our last solution is to compile it statically for now.

> 0017-autogenerate-pkgconfig-files: accepted
> Your business!

ok, lets doit.

Thanks for your attention

Frédéric

PS: please CC me also

[1] http://www.synchrotron-soleil.fr/
[2] http://people.debian.org/~picca/hkl/
[3] http://tango-controls.org/
[4] http://wiki.debian.org/DebianPureBlends
[5] http://blends.debian.net/pan/tasks/
[6] http://www.esrf.eu/events/conferences/debian-for-scientific-facilities-days-1/Provisional%20Programme
[7] http://sourceforge.net/apps/mediawiki/pynx/index.php?title=Main_Page
[8] http://www.debian.org/social_contract
[9] http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
[10] http://wiki.debian.org/DebianScience/cctbx
[11] http://docs.python.org/distutils/setupscript.html#installing-scripts