[cctbxbb] Should we enable boost threads in bootstrap?

[Graeme.Winter at diamond.ac.uk]

In principle boost threads is all we need – if we *are* calling back from Python then I imagine it would be something like:

at start of block: read data in Python (as hack)
split of threads, do many calculations
join threads
move to next block

I would hope that anyone who has decided to use threads and uses Python has heard of the GIL, so the fact that this can be a problem should not block us from enabling boost threads

PS we would like this over this side of the pond ☺

Thanks & cheerio Graeme


From: cctbxbb-bounces at phenix-online.org [mailto:cctbxbb-bounces at phenix-online.org] On Behalf Of Billy Poon
Sent: 26 September 2017 23:04
To: cctbx mailing list
Subject: Re: [cctbxbb] Should we enable boost threads in bootstrap?

Hi Luc,

I'm not sure. It was requested and I'm not familiar enough with Python threading. Is this something the data processing folks need? Or is threading in the C++ layer sufficient?

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Tel: (510) 486-5709
Fax: (510) 486-5909
Web: https://phenix-online.org

On Tue, Sep 26, 2017 at 2:35 PM, Luc Bourhis <luc_j_bourhis at mac.com<mailto:luc_j_bourhis at mac.com>> wrote:
Hi Billy,

On 26 Sep 2017, at 17:11, Billy Poon <BKPoon at lbl.gov<mailto:BKPoon at lbl.gov>> wrote:

I just chatted with Tristan Croll from Cambridge at the Phenix developer workshop. Would the Global Interpreter Lock be an issue?

https://docs.python.org/2.7/c-api/init.html#thread-state-and-the-global-interpreter-lock

It sounds like we should be releasing the lock before doing any threading and then reacquiring the lock afterwards.

So that means you plan your threaded code to call back into Python, then? This would be quite unusual in the context of the cctbx but I haven’t followed development closely for a long while, so I might have missed something.

Best wishes,

Luc


_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb


-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20170927/a52a27d5/attachment-0001.htm>