Graeme, Nigel, I would be a little bit worried about Boost threads when it comes to our KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at least accordingly to Intel docs) would be optimal over boost threads (or pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL edition P155 --- no ebook, sorry]. That being said, there is no way to know unless we try it out first, but it isn't something we can test right now. As for Threads vs MP, this again falls into our KNL port, where threads would be better suited (and become a necessity for optimal performance) when running on high-core count devices. If the OpenMP functionality exists, then maybe this would be a more portable way of taking advantage of all cores/hyperthreads. In this instance I think turning boost threads on for a build-by-build basis would be better, rather than as a default? Though if I am wrong, feel free to correct me. Best, Lee. On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty wrote:

Graeme

The short answer is "Why?" but that may start a very long discussion. There are a number of multiprocessing modules in easy_mp that seem to cover all the bases. Are there situations where threading is "better" to multiprocessing?

Articles on multiprocessing in cctbx.

https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6

https://www.phenix-online.org/newsletter/CCN_2013_07.pdf

https://www.phenix-online.org/newsletter/CCN_2013_01.pdf

Cheers

Nigel

--- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 <(510)%20486-5709> Email : [email protected] Fax : 510-486-5909 <(510)%20486-5909> Web : CCI.LBL.gov

On Tue, Aug 1, 2017 at 7:54 AM, wrote:

...

Afternoon all,

Should we do this? Any opinions? Could be useful for threads in a semi-portable way...

Thanks & cheerio Graeme

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

If efficient threading is desired I would have thought that these days GPUs are all the rage and that it would be worth looking into openCL and CUDA implementations for doing this. On an unrelated note are there any thoughts on moving CCTBX to Python3? One issue, which may not be insurmountable is that SCons does not yet support Python3. Rob -----Original Message----- From: [email protected] Sent: Tuesday, August 1, 2017 10:16 PM To: [email protected] Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap? Lee, End game for us is moving to “proper” threading i.e. lots of threads / cores working on one problem in one address space - be it regular 20 core xeon or 64 core KNL Boost threads came up in conversation today as a C++11 like threading model, so I wondered if it would be a stepping stone... Don’t have this book, maybe should get it…. Cheers Graeme On 1 Aug 2017, at 18:14, Lee O'Riordan mailto:[email protected]> wrote: Graeme, Nigel, I would be a little bit worried about Boost threads when it comes to our KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at least accordingly to Intel docs) would be optimal over boost threads (or pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL edition P155 --- no ebook, sorry]. That being said, there is no way to know unless we try it out first, but it isn't something we can test right now. As for Threads vs MP, this again falls into our KNL port, where threads would be better suited (and become a necessity for optimal performance) when running on high-core count devices. If the OpenMP functionality exists, then maybe this would be a more portable way of taking advantage of all cores/hyperthreads. In this instance I think turning boost threads on for a build-by-build basis would be better, rather than as a default? Though if I am wrong, feel free to correct me. Best, Lee. On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty mailto:[email protected]> wrote: Graeme The short answer is "Why?" but that may start a very long discussion. There are a number of multiprocessing modules in easy_mp that seem to cover all the bases. Are there situations where threading is "better" to multiprocessing? Articles on multiprocessing in cctbx. https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6 https://www.phenix-online.org/newsletter/CCN_2013_07.pdf https://www.phenix-online.org/newsletter/CCN_2013_01.pdf Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709tel:(510)%20486-5709 Email : [email protected]mailto:[email protected] Fax : 510-486-5909tel:(510)%20486-5909 Web : CCI.LBL.govhttp://cci.lbl.gov/ On Tue, Aug 1, 2017 at 7:54 AM, mailto:[email protected]> wrote: Afternoon all, Should we do this? Any opinions? Could be useful for threads in a semi-portable way... Thanks & cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb --- This email has been checked for viruses by AVG. http://www.avg.com

Hi Rob For data processing we are very interested in classic threading because our calculations include the word “if” - we have looked at pushing some of the calculations which do not include if (i.e. summed area tables) and there we get clobbered by the need to push the data of GPU memory also… Re: Python3 Well there is a question. ISTR people have looked (with some success, apparently) at building cctbx with cmake, which could be an alternative to using SCons? Ergo not a p3 blocker (other blockers may exist, ymmv etc) Cheers Graeme PS - any ideas how long the server outage is likely to last? Appreciate fires are a pretty good reason to take out kit. Not a problem in the UK ATM, flooding more likely :-\

On 2 Aug 2017, at 22:13, Dr. Robert Oeffner wrote:

If efficient threading is desired I would have thought that these days GPUs are all the rage and that it would be worth looking into openCL and CUDA implementations for doing this.

On an unrelated note are there any thoughts on moving CCTBX to Python3? One issue, which may not be insurmountable is that SCons does not yet support Python3.

Rob

-----Original Message----- From: [email protected] Sent: Tuesday, August 1, 2017 10:16 PM To: [email protected] Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?

Lee,

End game for us is moving to “proper” threading i.e. lots of threads / cores working on one problem in one address space - be it regular 20 core xeon or 64 core KNL

Boost threads came up in conversation today as a C++11 like threading model, so I wondered if it would be a stepping stone...

Don’t have this book, maybe should get it….

Cheers Graeme

On 1 Aug 2017, at 18:14, Lee O'Riordan mailto:[email protected]> wrote:

Graeme, Nigel,

I would be a little bit worried about Boost threads when it comes to our KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at least accordingly to Intel docs) would be optimal over boost threads (or pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL edition P155 --- no ebook, sorry]. That being said, there is no way to know unless we try it out first, but it isn't something we can test right now.

As for Threads vs MP, this again falls into our KNL port, where threads would be better suited (and become a necessity for optimal performance) when running on high-core count devices. If the OpenMP functionality exists, then maybe this would be a more portable way of taking advantage of all cores/hyperthreads.

In this instance I think turning boost threads on for a build-by-build basis would be better, rather than as a default? Though if I am wrong, feel free to correct me.

Best, Lee.

On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty mailto:[email protected]> wrote: Graeme

The short answer is "Why?" but that may start a very long discussion. There are a number of multiprocessing modules in easy_mp that seem to cover all the bases. Are there situations where threading is "better" to multiprocessing?

Articles on multiprocessing in cctbx.

https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6

https://www.phenix-online.org/newsletter/CCN_2013_07.pdf

https://www.phenix-online.org/newsletter/CCN_2013_01.pdf

Cheers

Nigel

--- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709tel:(510)%20486-5709 Email : [email protected]mailto:[email protected] Fax : 510-486-5909tel:(510)%20486-5909 Web : CCI.LBL.govhttp://cci.lbl.gov/

On Tue, Aug 1, 2017 at 7:54 AM, mailto:[email protected]> wrote: Afternoon all,

Should we do this? Any opinions? Could be useful for threads in a semi-portable way...

Thanks & cheerio Graeme

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

--- This email has been checked for viruses by AVG. http://www.avg.com _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

I just chatted with Tristan Croll from Cambridge at the Phenix developer workshop. Would the Global Interpreter Lock be an issue? https://docs.python.org/2.7/c-api/init.html#thread-state-and-the-global-inte... It sounds like we should be releasing the lock before doing any threading and then reacquiring the lock afterwards. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Thu, Aug 3, 2017 at 9:32 AM, Nigel Moriarty wrote:

Gaeme

We were evacuated yesterday ~2pm but the lab is reopened now and the servers are coming on line.

Cheers

Nigel

--- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 <(510)%20486-5709> Email : [email protected] Fax : 510-486-5909 <(510)%20486-5909> Web : CCI.LBL.gov

On Wed, Aug 2, 2017 at 10:29 PM, wrote:

...

Hi Rob

For data processing we are very interested in classic threading because our calculations include the word “if” - we have looked at pushing some of the calculations which do not include if (i.e. summed area tables) and there we get clobbered by the need to push the data of GPU memory also…

Re: Python3

Well there is a question. ISTR people have looked (with some success, apparently) at building cctbx with cmake, which could be an alternative to using SCons? Ergo not a p3 blocker (other blockers may exist, ymmv etc)

Cheers Graeme

PS - any ideas how long the server outage is likely to last? Appreciate fires are a pretty good reason to take out kit. Not a problem in the UK ATM, flooding more likely :-\

...

On 2 Aug 2017, at 22:13, Dr. Robert Oeffner wrote:

If efficient threading is desired I would have thought that these days GPUs are all the rage and that it would be worth looking into openCL and CUDA implementations for doing this.

On an unrelated note are there any thoughts on moving CCTBX to Python3? One issue, which may not be insurmountable is that SCons does not yet support Python3.

Rob

-----Original Message----- From: [email protected] Sent: Tuesday, August 1, 2017 10:16 PM To: [email protected] Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?

Lee,

End game for us is moving to “proper” threading i.e. lots of threads / cores working on one problem in one address space - be it regular 20 core xeon or 64 core KNL

Boost threads came up in conversation today as a C++11 like threading model, so I wondered if it would be a stepping stone...

Don’t have this book, maybe should get it….

Cheers Graeme

On 1 Aug 2017, at 18:14, Lee O'Riordan > wrote:

Graeme, Nigel,

I would be a little bit worried about Boost threads when it comes to our KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at least accordingly to Intel docs) would be optimal over boost threads (or pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL edition P155 --- no ebook, sorry]. That being said, there is no way to know unless we try it out first, but it isn't something we can test right now.

As for Threads vs MP, this again falls into our KNL port, where threads would be better suited (and become a necessity for optimal performance) when running on high-core count devices. If the OpenMP functionality exists, then maybe this would be a more portable way of taking advantage of all cores/hyperthreads.

In this instance I think turning boost threads on for a build-by-build basis would be better, rather than as a default? Though if I am wrong, feel free to correct me.

Best, Lee.

On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty mailto:[email protected]> wrote: Graeme

The short answer is "Why?" but that may start a very long discussion. There are a number of multiprocessing modules in easy_mp that seem to cover all the bases. Are there situations where threading is "better" to multiprocessing?

Articles on multiprocessing in cctbx.

https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6

https://www.phenix-online.org/newsletter/CCN_2013_07.pdf

https://www.phenix-online.org/newsletter/CCN_2013_01.pdf

Cheers

Nigel

--- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709tel:(510)%20486-5709 Email : [email protected]mailto:[email protected] Fax : 510-486-5909tel:(510)%20486-5909 Web : CCI.LBL.gov< http://cci.lbl.gov/>

On Tue, Aug 1, 2017 at 7:54 AM, > wrote: Afternoon all,

Should we do this? Any opinions? Could be useful for threads in a semi-portable way...

Thanks & cheerio Graeme

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

--- This email has been checked for viruses by AVG. http://www.avg.com _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Billy,

On 26 Sep 2017, at 17:11, Billy Poon wrote:

I just chatted with Tristan Croll from Cambridge at the Phenix developer workshop. Would the Global Interpreter Lock be an issue?

https://docs.python.org/2.7/c-api/init.html#thread-state-and-the-global-inte... https://docs.python.org/2.7/c-api/init.html#thread-state-and-the-global-inte...

It sounds like we should be releasing the lock before doing any threading and then reacquiring the lock afterwards.

So that means you plan your threaded code to call back into Python, then? This would be quite unusual in the context of the cctbx but I haven’t followed development closely for a long while, so I might have missed something. Best wishes, Luc

In principle boost threads is all we need – if we *are* calling back from Python then I imagine it would be something like: at start of block: read data in Python (as hack) split of threads, do many calculations join threads move to next block I would hope that anyone who has decided to use threads and uses Python has heard of the GIL, so the fact that this can be a problem should not block us from enabling boost threads PS we would like this over this side of the pond ☺ Thanks & cheerio Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Billy Poon Sent: 26 September 2017 23:04 To: cctbx mailing list Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap? Hi Luc, I'm not sure. It was requested and I'm not familiar enough with Python threading. Is this something the data processing folks need? Or is threading in the C++ layer sufficient? -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Tue, Sep 26, 2017 at 2:35 PM, Luc Bourhis mailto:[email protected]> wrote: Hi Billy, On 26 Sep 2017, at 17:11, Billy Poon mailto:[email protected]> wrote: I just chatted with Tristan Croll from Cambridge at the Phenix developer workshop. Would the Global Interpreter Lock be an issue? https://docs.python.org/2.7/c-api/init.html#thread-state-and-the-global-inte... It sounds like we should be releasing the lock before doing any threading and then reacquiring the lock afterwards. So that means you plan your threaded code to call back into Python, then? This would be quite unusual in the context of the cctbx but I haven’t followed development closely for a long while, so I might have missed something. Best wishes, Luc _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

HI Luc Re: But you seem to want to write a Python interface to Boost::Thread. First, I would advise against that. I don’t see any need for it but please feel free to convince me otherwise!! Nope thought had not occurred to me! Our use case is to split into threads in C++ land and rejoin still in C++ land, to make the C++ code exercise more cores sharing one address space / one set of input read-only data objects. We may gather inputs in Python land, but that is a different problem Still, I can see no reason not to start using boost threading here… Cheers Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Luc Bourhis Sent: 28 September 2017 08:46 To: cctbx mailing list Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap? On 27 Sep 2017, at 09:21, [email protected]mailto:[email protected] wrote: In principle boost threads is all we need – if we *are* calling back from Python then I imagine it would be something like: at start of block: read data in Python (as hack) split of threads, do many calculations join threads move to next block I would hope that anyone who has decided to use threads and uses Python has heard of the GIL, so the fact that this can be a problem should not block us from enabling boost threads Well, the standard cctbx pattern goes as follow. From the Python side, from scitbx.array_family import flex py_arr = flex.double((1,2,3,4)) some_function(a) and then on the C++ side, void some_function(af::const_ref const &scitbx_arr) { // work on scitbx_arr } The C++ code does not call back into the Python interpreter here. There are two separate reference counting: one for the Python object py_arr and one for the C++ handle scitbx_arr. For example, even if py_arr disappears after the call `some_function(a)`, scitbx_arr will not go away because the reference counter of the data underneath is unaffected and it is not zero by construction of the automatic conversions performed through the Boost::Python bridge. The only way something can go wrong is if from the C++ `some_function`, one does explicitly call into the C API of the Python interpreter, or it’s Boost::Python interface. There are hardly any place like that. I can think of the bridge between Python and C++ I/O I wrote in boost_adaptbx but of nothing else actually. Note of course that if we use Boost::Thread only in `some_function`, breaking the work into blocks in there and distributing it to thread, this is the end of the discussion. But you seem to want to write a Python interface to Boost::Thread. First, I would advise against that. I don’t see any need for it but please feel free to convince me otherwise!! But for the sake of discussion, let’s see whether this can be done. I guess you (when I say you, I don’t mean you Graeme in particular) want to be able to use it like so, on the Python side, for block_index in xrange(…): thread_me(some_function, py_arr, block_index, block_index + block_size) and on the C++ side, void some_function(af::const_ref const &scitbx_arr, unsigned int block_start, unsigned int block_end) { // work on the block } where `some_function` would be auto-magically run in its own thread. The issue here is how to pass the function from Python to C++. Is it possible to write a `thread_me` with the following signature and then have wrap it with Boost::Python? typedef void (*worker_type)(af::const_ref const &scitbx_arr, unsigned int block_start, unsigned int block_end); void thread_me(worker_type const f) { // fire a Boost::Thread and execute f in it } I would not think so. So dead end I would say. Best wishes, Luc -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Hi Graeme, The Berkeley servers are now available. Berkeley lab and the UC Berkeley campus were evacuated yesterday as a precaution since the power company was planning on cutting the power. As for SCons, based on the more recent commits on bitbucket, it looks like they are working on Python 3 support. So I think we can wait a little longer to see if the next SCons release supports Python 3. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Wed, Aug 2, 2017 at 10:29 PM, wrote:

Hi Rob

For data processing we are very interested in classic threading because our calculations include the word “if” - we have looked at pushing some of the calculations which do not include if (i.e. summed area tables) and there we get clobbered by the need to push the data of GPU memory also…

Re: Python3

Well there is a question. ISTR people have looked (with some success, apparently) at building cctbx with cmake, which could be an alternative to using SCons? Ergo not a p3 blocker (other blockers may exist, ymmv etc)

Cheers Graeme

PS - any ideas how long the server outage is likely to last? Appreciate fires are a pretty good reason to take out kit. Not a problem in the UK ATM, flooding more likely :-\

...

On 2 Aug 2017, at 22:13, Dr. Robert Oeffner wrote:

If efficient threading is desired I would have thought that these days GPUs are all the rage and that it would be worth looking into openCL and CUDA implementations for doing this.

On an unrelated note are there any thoughts on moving CCTBX to Python3? One issue, which may not be insurmountable is that SCons does not yet support Python3.

Rob

-----Original Message----- From: [email protected] Sent: Tuesday, August 1, 2017 10:16 PM To: [email protected] Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?

Lee,

End game for us is moving to “proper” threading i.e. lots of threads / cores working on one problem in one address space - be it regular 20 core xeon or 64 core KNL

Boost threads came up in conversation today as a C++11 like threading model, so I wondered if it would be a stepping stone...

Don’t have this book, maybe should get it….

Cheers Graeme

On 1 Aug 2017, at 18:14, Lee O'Riordan > wrote:

Graeme, Nigel,

I would be a little bit worried about Boost threads when it comes to our KNL port of cctbx. In this instance the use of OpenMP or Intel TBB (at least accordingly to Intel docs) would be optimal over boost threads (or pthreads, etc.)[see Intel Xeon Phi High Performance Programming, KNL edition P155 --- no ebook, sorry]. That being said, there is no way to know unless we try it out first, but it isn't something we can test right now.

As for Threads vs MP, this again falls into our KNL port, where threads would be better suited (and become a necessity for optimal performance) when running on high-core count devices. If the OpenMP functionality exists, then maybe this would be a more portable way of taking advantage of all cores/hyperthreads.

In this instance I think turning boost threads on for a build-by-build basis would be better, rather than as a default? Though if I am wrong, feel free to correct me.

Best, Lee.

On Tue, Aug 1, 2017 at 9:36 AM, Nigel Moriarty mailto:[email protected]> wrote: Graeme

The short answer is "Why?" but that may start a very long discussion. There are a number of multiprocessing modules in easy_mp that seem to cover all the bases. Are there situations where threading is "better" to multiprocessing?

Articles on multiprocessing in cctbx.

https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6

https://www.phenix-online.org/newsletter/CCN_2013_07.pdf

https://www.phenix-online.org/newsletter/CCN_2013_01.pdf

Cheers

Nigel

--- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709tel:(510)%20486-5709 Email : [email protected]mailto:[email protected] Fax : 510-486-5909tel:(510)%20486-5909 Web : CCI.LBL.gov< http://cci.lbl.gov/>

On Tue, Aug 1, 2017 at 7:54 AM, > wrote: Afternoon all,

Should we do this? Any opinions? Could be useful for threads in a semi-portable way...

Thanks & cheerio Graeme

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

--- This email has been checked for viruses by AVG. http://www.avg.com _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb