[cctbxbb] Should we enable boost threads in bootstrap?

Wed Aug 2 15:20:22 PDT 2017

One other thought.  Helen Ginn's CCPXFEL project does activate the
boost::threads library for the build.  Luc Bourhis made this possible.
Some documentation for this should exist on the cci.lbl.gov/xfel but the
power is currently out at LBNL due to a wild land fire.  The lab is
evacuated!  No web access right now.

Having said that I'm not too keen on the boost::threads avenue for moving
to KNL.  We are investigating a combination of using the existing
cctbx/python multiprocessing tools as Nigel described, plus MPI and OpenMP.

Nick

Nicholas K. Sauter, Ph. D.
Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33R0345
Berkeley, CA 94720
(510) 486-5713

On Wed, Aug 2, 2017 at 2:13 PM, Dr. Robert Oeffner <rdo20 at cam.ac.uk> wrote:

> If efficient threading is desired I would have thought that these days
> GPUs are all the rage and that it would be worth looking into openCL and
> CUDA implementations for doing this.
>
> On an unrelated note are there any thoughts on moving CCTBX to Python3?
> One issue, which may not be insurmountable is that SCons does not yet
> support Python3.
>
> Rob
>
>
>
> -----Original Message----- From: Graeme.Winter at diamond.ac.uk
> Sent: Tuesday, August 1, 2017 10:16 PM
> To: cctbxbb at phenix-online.org
> Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?
>
>
> Lee,
>
> End game for us is moving to “proper” threading i.e. lots of threads /
> cores working on one problem in one address space - be it regular 20 core
> xeon or 64 core KNL
>
> Boost threads came up in conversation today as a C++11 like threading
> model, so I wondered if it would be a stepping stone...
>
> Don’t have this book, maybe should get it….
>
> Cheers Graeme
>
>
>
> On 1 Aug 2017, at 18:14, Lee O'Riordan <loriordan at lbl.gov<mailto:lori
> ordan at lbl.gov>> wrote:
>
> Graeme, Nigel,
>
> I would be a little bit worried about Boost threads when it comes to our
> KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at
> least accordingly to Intel docs) would be optimal over boost threads (or
> pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL
> edition P155 ---  no ebook, sorry]. That being said, there is no way to
> know unless we try it out first, but it isn't something we can test right
> now.
>
> As for Threads vs MP, this again falls into our KNL port, where threads
> would be better suited (and become a necessity for optimal performance)
> when running on high-core count devices. If the OpenMP functionality
> exists, then maybe this would be a more portable way of taking advantage of
> all cores/hyperthreads.
>
> In this instance I think turning boost threads on for a build-by-build
> basis would be better, rather than as a default? Though if I am wrong, feel
> free to correct me.
>
> Best,
> Lee.
>
> On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty <nwmoriarty at lbl.gov<mailto:
> nwmoriarty at lbl.gov>> wrote:
> Graeme
>
> The short answer is "Why?" but that may start a very long discussion.
> There are a number of multiprocessing modules in easy_mp that seem to cover
> all the bases. Are there situations where threading is "better" to
> multiprocessing?
>
> Articles on multiprocessing in cctbx.
>
> https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6
>
> https://www.phenix-online.org/newsletter/CCN_2013_07.pdf
>
> https://www.phenix-online.org/newsletter/CCN_2013_01.pdf
>
>
> Cheers
>
> Nigel
>
> ---
> Nigel W. Moriarty
> Building 33R0349, Molecular Biophysics and Integrated Bioimaging
> Lawrence Berkeley National Laboratory
> Berkeley, CA 94720-8235
> Phone : 510-486-5709<tel:(510)%20486-5709>     Email : NWMoriarty at LBL.gov
> <mailto:NWMoriarty at LBL.gov>
> Fax   : 510-486-5909<tel:(510)%20486-5909>       Web  : CCI.LBL.gov<
> http://cci.lbl.gov/>
>
> On Tue, Aug 1, 2017 at 7:54 AM, <Graeme.Winter at diamond.ac.uk<mailto:
> Graeme.Winter at diamond.ac.uk>> wrote:
> Afternoon all,
>
> Should we do this? Any opinions? Could be useful for threads in a
> semi-portable way...
>
> Thanks & cheerio Graeme
>
> --
> This e-mail and any attachments may contain confidential, copyright and or
> privileged material, and are for the use of the intended addressee only. If
> you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
> not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
> and Wales with its registered office at Diamond House, Harwell Science and
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20170802/89f16715/attachment.htm>