[cctbxbb] flex.int.as_double() performance

richard.gildea at diamond.ac.uk richard.gildea at diamond.ac.uk
Tue Oct 11 06:11:29 PDT 2016


Hi,

Whilst looking into performance of various DIALS routines recently, I noticed that the flex.int.as_double() conversion takes roughly 3x as long as the equivalent functionality in numpy:

(the choice of a 2527x2463 array is simply because this is the size of the raw data array for a Pilatus 6M detector)

import time
from dials.array_family import flex
n = 100
a = flex.int(flex.grid(2527, 2463))
t0 = time.time()
for i in range(n):
  b = a.as_double()
t1 = time.time()
t = t1-t0
print "flex.int.as_double(): %.2fs (%.3fs/call)" %(t, t/n)

import numpy as np
a = np.random.randint(100, size=a.all())
t0 = time.time()
for i in range(n):
  b = a.astype(np.float64)
t1 = time.time()
t = t1-t0
print "numpy.astype(np.float64): %.2fs (%.3fs/call)" %(t, t/n)


$ libtbx.python time_as_double.py

flex.int.as_double(): 5.76s (0.058s/call)

numpy.astype(np.float64): 1.86s (0.019s/call)


This is the current version of the as_double function:


    static flex_double

    as_double(f_t const& a)

    {

      shared_plain<double> result(a.size(), init_functor_null<double>());

      for(std::size_t i=0;i<a.size();i++) result[i] = a[i];

      return flex_double(result, a.accessor());

    }


Replacing the above code with the following, I see a significant runtime improvement, now flex.int.as_double() is marginally quicker than the equivalent numpy routine:

    static flex_double
    as_double(f_t const& a)
    {
      shared_plain<double> result(a.begin(), a.end());
      return flex_double(result, a.accessor());
    }

$ libtbx.python time_as_double.py

flex.int.as_double(): 1.64s (0.016s/call)

numpy.astype(np.float64): 1.82s (0.018s/call)

Is there any obvious reason why making this change would be A Bad Thing?

In the context of dials spot-finding, this change results in a 22s saving (out of a total 172s with the original code) when running dials.find_spots on a 540 image Pilatus 6M dataset using a single processor, so it is definitely a noticeable speed-up.

All timings done on a macbook pro.

Cheers,

Richard



Dr Richard Gildea
Data Analysis Scientist
Tel: +441235 77 8078

Diamond Light Source Ltd.
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom




More information about the cctbxbb mailing list