[cctbxbb] cctbx + PyPy?

[Ralf Grosse-Kunstleve]

Hi Jan,
My relatively uneducated first impression is "interesting, but will
probably consume a lot of developer time, for a relatively moderate speed
gain".
In general, most performance-critical parts of cctbx are implemented in
C++. Most application spend only a relatively small amount of time in
Python. Let's try to overestimate: 30%. Even if you make that 10x faster
(don't know if that's realistic) the total runtime for that application
will still be 73%. Which is nice, but I believe many people won't even
notice.
In the bigger picture, I'm most excited about the large numbers of cores
now available in one machine (e.g. we have one machine with 64 cores, a few
more with 48 cores). For certain things you can get large speed increases
with a relatively small effort. This is more the direction we're moving in
in general.
If you want to start playing around with PyPy, to see how it could be
integrated into the cctbx environment, I'm happy to answer your questions
if you run into problems. Maybe a good limited scope for a start is to get
some trivial module in boost_adaptbx to work, or if that's too simple,
scitbx. There isn't any performance critical Python code in there, but
should be good enough to develop the framework. Then it should be
relatively straightforward to move to e.g. smtbx.refine of even
phenix.refine. The latter probably has some Python bottlenecks.
A different idea is to make the fable module in cctbx work with PyPy, as a
proof of concept. The Boost.Python dependency is optional. It also works
with pure Python. So you could first get an impression how much PyPy does
for pure Python code, then try to see how it works in combination with
Boost.Python extensions. fable is relatively small, which means you can
move quickly.
Ralf


2011/11/24 Jan Marten Simons <marten at xtal.rwth-aachen.de>

> Hi,
>
> PyPy 1.7 was released just a few days ago:
> http://morepypy.blogspot.com/2011/11/pypy-17-widening-sweet-spot.html
>
> After taking a look at http://speed.pypy.org/ I'm wondering if it would be
> possible to get cctbx to work with PyPy-jit instead of CPython.
>
> Perhaps it's a little bit early, as they are still working on a complete
> numpy
> integration and I'm not sure how well boost.python fits into this. But in
> general speeding up the python parts looks nice if one does not have to
> invest
> too much time into this.
>
> Has any of you already done some tests with PyPy?
>
> Cheers,
> Jan
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20111124/7a02b633/attachment.htm>