[cctbxbb] Tidy up CCTBX

Aaron Brewster asbrewster at lbl.gov
Fri Jan 26 11:52:08 PST 2018


@Graeme, replying to your points

" - I am not sure that these address the biggest problem i.e. iotbx
depends_on dxtbx"

Luc's original problem with iotbx depending on dxtbx was bringing in the
hdf5 dependency.  ISo making hdf5 optional for dxtbx would be helpful.  The
secondary complaint of a cyclical dependency of iotbx on dxtbx and dxtbx on
iotbx is valid.  Merging iotbx and dxtbx has been discussed previously and
is probably the right solution, but that would take a bit of doing...

" - Will we end up getting bug reports where DIALS programs can’t read e.g.
Eiger, CBF images?"

This depends on distribution.  We are distributing builds of phenix,
labelit and dials for linux and mac created by bootstrap that include hdf5
and cbflib, so there are no issues there.  I don't know how or if Windows
builds will be affected as I thought they were distributing HDF5 and cbflib
too.  I believe this change only opens up possibilities for easier
distribution for applications outside of phenix, labelit and dials.
Please, if Rob or Luc want to correct me, feel free :)

-Aaron

On Fri, Jan 26, 2018 at 4:00 AM, <Graeme.Winter at diamond.ac.uk> wrote:

> This sounds like “pip install libtbx” “pip install scitbx”
>
> Since both of these are relatively static & self contained, would make
> sense
>
> ISTR there was a suggestion earlier in the thread to this effect, which
> perhaps got lost…
>
> Best wishes Graeme
>
> > On 25 Jan 2018, at 20:41, Tristan Croll <tic20 at cam.ac.uk> wrote:
> >
> > My 2 cents: when talking about creating PyPI CCTBX modules it may be
> worthwhile to start by thinking smaller. For instance, while planning out
> how to interface my (largely NumPy-based by necessity) application to
> PHENIX, it would be an enormous help if I could, say, 'pip install
> flex_array' and 'pip install phil'.
> >
> > On 2018-01-25 19:20, Aaron Brewster wrote:
> >> I've created the issues:
> >> https://github.com/cctbx/cctbx_project/issues/125 [6]
> >> https://github.com/cctbx/cctbx_project/issues/126 [7]
> >> -Aaron
> >> On Tue, Jan 23, 2018 at 10:50 AM, Paul Adams <pdadams at lbl.gov> wrote:
> >>> Hi Aaron,
> >>> thanks for the email. I think your proposal to remove the compile
> >>> time dependencies by moving code to extension modules is great. I
> >>> think this would be greatly appreciated by Rob and others.
> >>> Fable, contrary to recent rumors, is actually often used by
> >>> others outside of the cctbx community. While it is true that we
> >>> wrote it for mainly one task, it turns out there are lots of people
> >>> with old f77 code they’d like to translate. Given that we don’t
> >>> seem to have any dependency or support issues with the code I’d
> >>> recommend it stays - making and supporting a separate package would
> >>> be too much.
> >>> Cheers,
> >>> Paul
> >>>> On Jan 23, 2018, at 9:53 AM, Aaron Brewster <asbrewster at lbl.gov>
> >>> wrote:
> >>>> Hi folks,
> >>>> There are lots of threads in this email. Pulling a couple of
> >>> them...
> >>>> • The LBL team runs the libtbx/phil/tst_interface.py test
> >>> nightly in buildbot as part of testing phenix, so the test is
> >>> exercised (and it passed last night). Whether it should be moved to
> >>> phenix_regression is another issue (I don't think it's necessary,
> >>> the has_module call seems clear to me).
> >>>> • I think there are many projects that shouldn't be moved
> >>> out of cctbx_project. IOTA, xfel, prime, simtbx, and dxtbx are all
> >>> under active development and are useful to many crystallographers.
> >>> cctbx is a toolbox after all. It just has a lot of tools.
> >>>> • That said, It may be worth having a discussion about
> >>> fable, and maybe one or two of the others. That can be done in
> >>> separate threads for each project?
> >>>> • I would like to break compile-time dependencies on
> >>> external packages in dxtbx as identified by Rob. If the C++ code in
> >>> dxtbx/iotbx that imports cbf.h and hdf5.h is moved to separate
> >>> boost-python extension modules (some of this is done already), then
> >>> the SConscript can be made smarter and compile only the libraries it
> >>> can find headers for. I'd be in favor of making two issues to track
> >>> this work, one for cbflib and one for hdf5. That would be a step in
> >>> the direction restoring the 'standaloneness' of the cctbx package.
> >>>> -Aaron
> >>>> On Tue, Jan 23, 2018 at 3:16 AM, <markus.gerstel at diamond.ac.uk>
> >>> wrote:
> >>>> Hi Marcin
> >>>> Hmm, so in other words "his free / open source software cannot be
> >>> fully tested without agreeing to this license"?
> >>>> No, still doesn't make sense to me. It only raises more
> >>> questions. Such as: does Travis have a phenix license?
> >>>> Currently we don't run these tests at all - although we *should*
> >>> run all tests on libtbx - because we don't have phenix installed
> >>> during cctbx tests.
> >>>> Either the required test data should fully go into libtbx or the
> >>> test should fully go into phenix. Distributing a test in two parts
> >>> is no good to anyone.
> >>>> -Markus
> >>>> ________________________________________
> >>>> From: cctbxbb-bounces at phenix-online.org
> >>> [cctbxbb-bounces at phenix-online.org] on behalf of Marcin Wojdyr
> >>> [wojdyr at gmail.com]
> >>>> Sent: Tuesday, January 23, 2018 10:55
> >>>> To: cctbx mailing list
> >>>> Cc: Randy Read
> >>>> Subject: Re: [cctbxbb] Tidy up CCTBX
> >>>> Hi Graeme,
> >>>> > Turns out that the assert that libtbx has no dependencies is
> >>> not strictly true:
> >>>> >
> >>>> > if (not libtbx.env.has_module(name="phenix")):
> >>>> > print "phenix module not available: skipping advanced
> >>> tests"
> >>>> > return
> >>>> This is optional dependency (cctbx optionally uses Phenix for
> >>> testing
> >>>> in various places) and doesn't cause problems.
> >>>> If you'd like to do disentangling, dxtbx-iotbx would be a good
> >>>> starting point as it really causes problems.
> >>>> _______________________________________________
> >>>> cctbxbb mailing list
> >>>> cctbxbb at phenix-online.org
> >>>> http://phenix-online.org/mailman/listinfo/cctbxbb [1]
> >>>> --
> >>>> This e-mail and any attachments may contain confidential,
> >>> copyright and or privileged material, and are for the use of the
> >>> intended addressee only. If you are not the intended addressee or an
> >>> authorised recipient of the addressee please notify us of receipt by
> >>> returning the e-mail and do not use, copy, retain, distribute or
> >>> disclose the information in or attached to the e-mail.
> >>>> Any opinions expressed within this e-mail are those of the
> >>> individual and not necessarily of Diamond Light Source Ltd.
> >>>> Diamond Light Source Ltd. cannot guarantee that this e-mail or
> >>> any attachments are free from viruses and we cannot accept liability
> >>> for any damage which you may sustain as a result of software viruses
> >>> which may be transmitted in or with the message.
> >>>> Diamond Light Source Limited (company no. 4375679). Registered in
> >>> England and Wales with its registered office at Diamond House,
> >>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
> >>> 0DE, United Kingdom
> >>>> _______________________________________________
> >>>> cctbxbb mailing list
> >>>> cctbxbb at phenix-online.org
> >>>> http://phenix-online.org/mailman/listinfo/cctbxbb [1]
> >>>> _______________________________________________
> >>>> cctbxbb mailing list
> >>>> cctbxbb at phenix-online.org
> >>>> http://phenix-online.org/mailman/listinfo/cctbxbb [1]
> >>> --
> >>> Paul Adams
> >>> Division Director, Molecular Biophysics & Integrated Bioimaging,
> >>> Lawrence Berkeley Lab
> >>> Division Deputy for Biosciences, Advanced Light Source, Lawrence
> >>> Berkeley Lab
> >>> Adjunct Professor, Department of Bioengineering, U.C. Berkeley
> >>> Vice President for Technology, the Joint BioEnergy Institute
> >>> Laboratory Research Manager, ENIGMA Science Focus Area
> >>> Building 33, Room 347
> >>> Building 978, Room 4126
> >>> Tel: 1-510-486-4225 [2], Fax: 1-510-486-5909 [3]
> >>> http://cci.lbl.gov/paul [4]
> >>> Lawrence Berkeley Laboratory
> >>> 1 Cyclotron Road
> >>> BLDG 33R0345
> >>> Berkeley, CA 94720, USA.
> >>> Executive Assistant: Louise Benvenue [ LBenvenue at lbl.gov ][
> >>> 1-510-495-2506 [5] ]
> >>> --
> >>> _______________________________________________
> >>> cctbxbb mailing list
> >>> cctbxbb at phenix-online.org
> >>> http://phenix-online.org/mailman/listinfo/cctbxbb [1]
> >> Links:
> >> ------
> >> [1] http://phenix-online.org/mailman/listinfo/cctbxbb
> >> [2] tel:1-510-486-4225
> >> [3] tel:1-510-486-5909
> >> [4] http://cci.lbl.gov/paul
> >> [5] tel:%5B%201-510-495-2506
> >> [6] https://github.com/cctbx/cctbx_project/issues/125
> >> [7] https://github.com/cctbx/cctbx_project/issues/126
> >> _______________________________________________
> >> cctbxbb mailing list
> >> cctbxbb at phenix-online.org
> >> http://phenix-online.org/mailman/listinfo/cctbxbb
> >
> >
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org
> > http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> --
> This e-mail and any attachments may contain confidential, copyright and or
> privileged material, and are for the use of the intended addressee only. If
> you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
> not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
> and Wales with its registered office at Diamond House, Harwell Science and
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20180126/cd1acf74/attachment-0001.htm>


More information about the cctbxbb mailing list