[cctbxbb] Should we enable boost threads in bootstrap?

Thu Aug 3 09:32:12 PDT 2017

Gaeme

We were evacuated yesterday ~2pm but the lab is reopened now and the
servers are coming on line.

Cheers

Nigel

---
Nigel W. Moriarty
Building 33R0349, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
Berkeley, CA 94720-8235
Phone : 510-486-5709     Email : NWMoriarty at LBL.gov
Fax   : 510-486-5909       Web  : CCI.LBL.gov

On Wed, Aug 2, 2017 at 10:29 PM, <Graeme.Winter at diamond.ac.uk> wrote:

> Hi Rob
>
> For data processing we are very interested in classic threading because
> our calculations include the word “if” - we have looked at pushing some of
> the calculations which do not include if (i.e. summed area tables) and
> there we get clobbered by the need to push the data of GPU memory also…
>
> Re: Python3
>
> Well there is a question. ISTR people have looked (with some success,
> apparently) at building cctbx with cmake, which could be an alternative to
> using SCons? Ergo not a p3 blocker (other blockers may exist, ymmv etc)
>
> Cheers Graeme
>
> PS - any ideas how long the server outage is likely to last? Appreciate
> fires are a pretty good reason to take out kit. Not a problem in the UK
> ATM, flooding more likely :-\
>
>
>
>
>
> > On 2 Aug 2017, at 22:13, Dr. Robert Oeffner <rdo20 at cam.ac.uk> wrote:
> >
> > If efficient threading is desired I would have thought that these days
> GPUs are all the rage and that it would be worth looking into openCL and
> CUDA implementations for doing this.
> >
> > On an unrelated note are there any thoughts on moving CCTBX to Python3?
> One issue, which may not be insurmountable is that SCons does not yet
> support Python3.
> >
> > Rob
> >
> >
> >
> > -----Original Message----- From: Graeme.Winter at diamond.ac.uk
> > Sent: Tuesday, August 1, 2017 10:16 PM
> > To: cctbxbb at phenix-online.org
> > Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?
> >
> > Lee,
> >
> > End game for us is moving to “proper” threading i.e. lots of threads /
> cores working on one problem in one address space - be it regular 20 core
> xeon or 64 core KNL
> >
> > Boost threads came up in conversation today as a C++11 like threading
> model, so I wondered if it would be a stepping stone...
> >
> > Don’t have this book, maybe should get it….
> >
> > Cheers Graeme
> >
> >
> >
> > On 1 Aug 2017, at 18:14, Lee O'Riordan <loriordan at lbl.gov<mailto:lori
> ordan at lbl.gov>> wrote:
> >
> > Graeme, Nigel,
> >
> > I would be a little bit worried about Boost threads when it comes to our
> KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at
> least accordingly to Intel docs) would be optimal over boost threads (or
> pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL
> edition P155 ---  no ebook, sorry]. That being said, there is no way to
> know unless we try it out first, but it isn't something we can test right
> now.
> >
> > As for Threads vs MP, this again falls into our KNL port, where threads
> would be better suited (and become a necessity for optimal performance)
> when running on high-core count devices. If the OpenMP functionality
> exists, then maybe this would be a more portable way of taking advantage of
> all cores/hyperthreads.
> >
> > In this instance I think turning boost threads on for a build-by-build
> basis would be better, rather than as a default? Though if I am wrong, feel
> free to correct me.
> >
> > Best,
> > Lee.
> >
> > On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty <nwmoriarty at lbl.gov
> <mailto:nwmoriarty at lbl.gov>> wrote:
> > Graeme
> >
> > The short answer is "Why?" but that may start a very long discussion.
> There are a number of multiprocessing modules in easy_mp that seem to cover
> all the bases. Are there situations where threading is "better" to
> multiprocessing?
> >
> > Articles on multiprocessing in cctbx.
> >
> > https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6
> >
> > https://www.phenix-online.org/newsletter/CCN_2013_07.pdf
> >
> > https://www.phenix-online.org/newsletter/CCN_2013_01.pdf
> >
> >
> > Cheers
> >
> > Nigel
> >
> > ---
> > Nigel W. Moriarty
> > Building 33R0349, Molecular Biophysics and Integrated Bioimaging
> > Lawrence Berkeley National Laboratory
> > Berkeley, CA 94720-8235
> > Phone : 510-486-5709<tel:(510)%20486-5709>     Email :
> NWMoriarty at LBL.gov<mailto:NWMoriarty at LBL.gov>
> > Fax   : 510-486-5909<tel:(510)%20486-5909>       Web  : CCI.LBL.gov<
> http://cci.lbl.gov/>
> >
> > On Tue, Aug 1, 2017 at 7:54 AM, <Graeme.Winter at diamond.ac.uk<mailto:
> Graeme.Winter at diamond.ac.uk>> wrote:
> > Afternoon all,
> >
> > Should we do this? Any opinions? Could be useful for threads in a
> semi-portable way...
> >
> > Thanks & cheerio Graeme
> >
> > --
> > This e-mail and any attachments may contain confidential, copyright and
> or privileged material, and are for the use of the intended addressee only.
> If you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> > Any opinions expressed within this e-mail are those of the individual
> and not necessarily of Diamond Light Source Ltd.
> > Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> > Diamond Light Source Limited (company no. 4375679). Registered in
> England and Wales with its registered office at Diamond House, Harwell
> Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
> >
> >
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> > http://phenix-online.org/mailman/listinfo/cctbxbb
> >
> >
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> > http://phenix-online.org/mailman/listinfo/cctbxbb
> >
> >
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> > http://phenix-online.org/mailman/listinfo/cctbxbb
> >
> >
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org
> > http://phenix-online.org/mailman/listinfo/cctbxbb
> >
> >
> > ---
> > This email has been checked for viruses by AVG.
> > http://www.avg.com
> > _______________________________________________
> > cctbxbb mailing list
> > cctbxbb at phenix-online.org
> > http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20170803/5ad3a7f0/attachment-0001.htm>