[cctbxbb] simtbx

Graeme.Winter at diamond.ac.uk Graeme.Winter at diamond.ac.uk
Wed May 3 09:03:03 PDT 2017


Aaron

Last command exceedingly useful thank you

Will explore

Think the dependency was indirect will verify

Cheers Graeme

On 3 May 2017, at 16:59, Aaron Brewster <asbrewster at lbl.gov<mailto:asbrewster at lbl.gov>> wrote:

Hi, sorry I thought things were good based on my local tests.  However, it looks like we need to fix some compile errors on different platforms.  I've reverted the change for now, but I'll be putting it back soon (probably later today) after we've done some more testing.

Regarding Graeme's original point: "simtbx, part of the core cctbx now, depends on dials.  This means if you do not have a dials build your cctbx will be broken".  This is untrue.  A core cctbx build means you run bootstrap.py --builder=cctbx.  You'll get 'cctbx', 'cbflib', 'scitbx', 'libtbx', 'iotbx', 'mmtbx', 'smtbx', 'dxtbx', 'gltbx', 'wxtbx'.  This does means several packages in cctbx_project would not be configured.  If instead you ran python /path/to/libtbx/configure.py cctbx in an empty folder, you would get a similar list of packages.

About dependencies, there is a circular dependency between xfel and dials.  xfel depends on dials and dials depends on xfel.  It's been there for a while.  The root cause is dials.stills_process depends on xfel code, specifically  xfel.mono_simulation.  It would be nice to break this cycle, but it would involve shuffling a lot of code around.  Instead, xfel is an optional dependency of dials and dials is a required dependency of xfel.  simtbx will be an optional dependency of xfel.

If you'd like to see the dependency chains, use
libtbx.show_module_dependencies xfel
libtbx.show_module_dependencies dials
libtbx.show_module_dependencies cctbx

-Aaron








On Wed, May 3, 2017 at 8:28 AM, James Holton <jmholton at lbl.gov<mailto:jmholton at lbl.gov>> wrote:

"simtbx" is the name we are giving to my diffraction image simulation codes.  You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg.  Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages.  There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx.  Until recently, all this was standalone c code, with home pages here:

http://bl831.als.lbl.gov/~jamesh/mlfsom/
http://bl831.als.lbl.gov/~jamesh/nanoBragg/
http://bl831.als.lbl.gov/~jamesh/nearBragg/
http://bl831.als.lbl.gov/~jamesh/nonBragg/

I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx.  I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation.

Again, sorry this broke everything.  It's been in there since February and I was surprised to find that a new developer here couldn't access it.  This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught.

-James


On 5/3/2017 7:57 AM, Pavel Afonine wrote:
Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it?
Pavel

On 5/3/17 07:53, markus.gerstel at diamond.ac.uk<mailto:markus.gerstel at diamond.ac.uk> wrote:
Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.
Why should we ship dead code?

-Markus


From: cctbxbb-bounces at phenix-online.org<mailto:cctbxbb-bounces at phenix-online.org> [mailto:cctbxbb-bounces at phenix-online.org] On Behalf Of Nicholas Sauter
Sent: 03 May 2017 15:52
To: cctbx mailing list <cctbxbb at phenix-online.org><mailto:cctbxbb at phenix-online.org>
Subject: Re: [cctbxbb] simtbx

Markus,

We want option c.  simtbx should be in the cctbx_project directory.  No separate git repository.  We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo.  James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D.
Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33R0345
Berkeley, CA 94720
(510) 486-5713<tel:(510)%20486-5713>

On Wed, May 3, 2017 at 7:49 AM, <markus.gerstel at diamond.ac.uk<mailto:markus.gerstel at diamond.ac.uk>> wrote:
Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.
Simtbx repo: https://github.com/dials/simtbx
Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a.       move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b.       leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c.       leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

From: cctbxbb-bounces at phenix-online.org<mailto:cctbxbb-bounces at phenix-online.org> [mailto:cctbxbb-bounces at phenix-online.org<mailto:cctbxbb-bounces at phenix-online.org>] On Behalf Of James Holton
Sent: 03 May 2017 15:40
To: Winter, Graeme (DLSLtd,RAL,LSCI) <Graeme.Winter at Diamond.ac.uk<mailto:Graeme.Winter at Diamond.ac.uk>>; cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
Subject: Re: [cctbxbb] simtbx


Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton.  It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date.  I asked Aaron and Nick for help with this.

  I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials?  I did not expect that.

Sorry, didn't mean to break anything!  Would much appreciate any help on this.

-James
On 5/3/2017 1:41 AM, Graeme.Winter at Diamond.ac.uk<mailto:Graeme.Winter at Diamond.ac.uk> wrote:
Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme



--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom





--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom


_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb




_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb





_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb



_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb


_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb



More information about the cctbxbb mailing list