[cctbxbb] Simple C++11 async example

[David Waterman]

Thanks Tristan,

This does look useful indeed. I note you use pybind11 in your example.
Should this work just as easily with Boost.Python?

On that topic, I've heard many good things about pybind11. Does anyone have
a feeling for how much work it would be to convert cctbx to pybind11
bindings rather than Boost.Python. Is this feasible? Is it desirable?

Cheers
-- David


On Thu, 1 Oct 2020 at 16:45, Tristan Croll <tic20 at cam.ac.uk> wrote:

> Hi all,
>
> For anyone who's interested: following on from my quick demo of the use of
> std::async to simplify threading of long-running tasks called from Python,
> I've put some example code at
> https://github.com/tristanic/async-hello-world/tree/master/src (implemented
> as a ChimeraX plugin for demo purposes, but the C++ code and bindings are
> entirely generic).
>
> To re-iterate what this is about, and why I think it's important: in
> brief, for most real-world problems Python-level threading is not
> particularly useful since (except in specific cases) the Global Interpreter
> Lock limits the overall performance to no better than - and often much
> worse than - single-threaded code. While multiprocessing can be an option
> for some tasks, it's (a) slow to get started, (b) memory-hungry due to the
> need for each process to have a complete copy of all the data, (c) rather
> difficult to organise with respect to intercommunication, and (d) not very
> cross-platform friendly (I'm looking at you, Windows!). The point of this
> example is to show that with C++11 organising the threading at the C++
> level can often end up being both the simplest *and* fastest approach -
> it's only a dozen or so extra lines of fairly boilerplate code required to
> allow the Python programmer to choose between running the exact same
> function either in the main thread or asynchronously in a new one.
>
> Why I think it's important: I've said in the past, and continue to
> believe, that we should be aiming for (interactive) model building,
> validation and refinement to merge together into a single integrated
> process - as much as anything, ISOLDE is meant to be the proof of principle
> that this is a possibility. But for that to work, high-performance and
> cross-platform parallelism is a real necessity. The approach described here
> is how I'm currently getting ChimeraX to maintain 30+ fps graphics
> performance while simultaneously running interactive molecular dynamics and
> structure factor calculations. I suspect there are many functions in CCTBX
> that could potentially benefit from this approach even when not considering
> GUI applications.
>
> Best regards,
>
> Tristan
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20201002/913190df/attachment.htm>