[cctbxbb] flex and numpy caveats

[James Stroud]

On Apr 4, 2013, at 3:15 AM, Luc Bourhis wrote:
>> 1. The first example should really throw an exception, because silent failures like this can be catastrophic:
>> 
>> py> from numpy import random
>> py> r = random.randint(2, size=10)
>> py> r
>> array([1, 0, 1, 1, 0, 1, 0, 1, 1, 1])
>> py> list(flex.bool(r))
>> [True, False, False, False, False, False, False, False, False, False]
>> 
>> This example is clearly due to incorrect assumptions about the internal representations of numpy ndarrays.
> 
> As far as I can tell, this is much worse than that as the numpy-to-flex conversion assumes the same element type in the source and target array. Thus converting a numpy array of int's into a flex array of bools is illegal but this precondition is not asserted unfortunately.
> Nasty indeed. I do not understand numpy well enough to propose a solution, I am afraid.

On the python side, the correct types for which to check have names that mirror the python names:

py> numpy.complex is type(complex())
True
py> numpy.bool is type(bool())
True
py> numpy.int is type(int())
True
py> numpy.float is type(float())
True
py> numpy.long is type(long())
True

It is unfortunate that array creation by numpy does not respect this naming symmetry:

py> numpy.array([1, 2, 3]).dtype is int
False
py> numpy.array([1, 2, 3], dtype=int).dtype is int
False
py> # same as
py> numpy.array([1, 2, 3], dtype=numpy.int).dtype is int
False

So, even if numpy-to-flex conversion catches wrongly [1] typed arrays, users still need to know which numpy dtypes to use for which flex constructors. Here is what I have determined:

flex.bool : numpy.bool
flex.int : numpy.int32
flex.long : numpy.int64, numpy.int
flex.float : numpy.float32
flex.double : numpy.float, numpy.double

In all cases, if a wrongly typed numpy array is used then flex will fail silently as of build 2013_02_27_0005. Here's one more example:

py> numpy.arange(4)
py> array([0, 1, 2, 3])
py> list(flex.int(numpy.arange(4)))
[0, 0, 1, 0]

James


[1] The concept of "wrongly" typed objects doesn't truly exist in python.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20130404/8ddf5080/attachment-0001.htm>