[cctbxbb] Module versioning (was Re: Debian Package)

[Radostan Riedel]

On Tue, 04. Sep 21:51, Luc Bourhis wrote:
> So, Radostan, I reckon that the list of shared libraries we would version is as follow:
> 
> libcctbx.so                             
> libcctbx_sgtbx_asu.so                   
> libiotbx_mtz.so                         
> libiotbx_pdb.so                         
> libmmtbx_masks.so       
> libomptbx.so                
> libscitbx_boost_python.so
> libsimdtbx_memory_allocation_central.so (this one is a work of mine that I will commit soon-ish: just for the record)
> libsmtbx_refinement_constraints.so
> 
> I have excluded libccp4io.so because your plan is to rely on Debian libraries instead. I have also excluded libscitbx_minpack.so
> and libscitbx_slatec.so for the same reason, although I don't know your objectives for those two. I would like to emphasise again, and I am sure all the other cctbx developers will agree with me here, that you have to make sure that, after replacing the libraries we ship by the Debian ones, a significant subset of the Phenix test cases does still pass with flying colours. What is now called t12 should be enough.
I'd like to include libscitbx_slatec and libscitbx_minpack too as of my
understanding this are c++ ports of those fortran routines and maybe some
developers can use them too.

> 
> Regarding your libtool patch, it appears that for reasons that have nothing to do with my arguments in this thread of discussion,  you have set up env_no_includes_boost_python_ext not to use libtool to version the shlibs. That's how we want it and you should therefore just change the comment explaining the rationales.
Reason is that libtool does not support python-extension. It checks if the file
is called lib*.so. I don't know why it's so picky but as so-versioning is not
required for python-extensions I just unset it for env_no_includes_boost_python_ext.
To be honest I wanned to do this with a python method first but 2 problems I see
here. Libtool supports most common compilers and platforms. Libtool itself is
just a hackish bash script but it works fine and is the quasi standard for that
purpose.

> However, we are jumping the guns here as this patch will only make sense of cctbx developers are willing to update so-versions as they change the C++ code. Some libraries in the list above are either tiny, or mature enough that the work involve will be trivial (libscitbx_boost_python.so, libsimdtbx_memory_allocation_central.so and libomptbx.so come to mind). On the contrary, libcctbx.so has 70 source files contributing to it and a huge ABI/API.
> 
> So, dear cctbx developers, what do you think of versioning those shlibs? Do you think you can commit yourself to update one line in the relevant SConscript after a bunch of changes to the C++ headers or sources (according to the rules laid out by Radostan earlier in this thread)? Again I know that as far as our immediate work (Phenix, Olex 2, etc) is concerned, this is basically useless to us. But having a Debian distribution and easing the adoption of the cctbx by the beam lines at Soleil is worth an effort I reckon. But this is not for me to say alone.
I found Freds idea very nice where we (debian maintainers) would check the
API/ABI after a new upstream version we pack (seems that there are debian helper
scripts for that) and contact this mailing list if there are any problems and try to
collaborate with the cctbx developers. Hopefully this leads to more acceptance of the need for so-versioning.

Regarding the versioning of python extensions/modules I'll check the numpy source
code as Fred suggested. Maybe there is an easy way to do
this in cctbx too without too much work for the cctbx-developers. 

I was also thinking as there are debian debhelper scripts for checking API/ABI
problems in shlibs maybe I can have a look into the scripts and contribute an
automatic version checking and setting script.
I can imagine to take the last compiled public release as a reference
for an automatic script that takes care of the versioning for the
cctbx-developers, this way you could save work. And an automatic script for detecting
and setting python ext/modules API/ABI versions would be nice too. 
You could run it during your test builds and set the versions strings.
What do you think?

kind regards
Radi