<div dir="ltr"><div dir="ltr">Hi Rob,<div><br></div><div>We looked a little into this and it seems like the byte string interface will preserve NAN. We're going to dig deeper into the serialization code for a fix.</div><div><br></div><div>Anyway, here is a code snippet for using the byte string interface. The byte string can be pickled, so a workaround for flex would be to add an extra conversion step to byte string before pickling.</div><div><br></div><div><div><font face="courier new, monospace">from __future__ import print_function</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">import pickle</font></div><div><font face="courier new, monospace">from scitbx.array_family import flex</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># NAN in flex.double</font></div><div><font face="courier new, monospace">a = flex.random_double(5)</font></div><div><font face="courier new, monospace">n = float('nan')</font></div><div><font face="courier new, monospace">a[3] = n</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># pickle test</font></div><div><font face="courier new, monospace">p = pickle.dumps(a)</font></div><div><font face="courier new, monospace">ap = pickle.loads(p)</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># byte string test</font></div><div><font face="courier new, monospace">b = a.copy_to_byte_str()</font></div><div><font face="courier new, monospace">ab = flex.double_from_byte_str(b)</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># pickled byte string test</font></div><div><font face="courier new, monospace">pb = pickle.dumps(b)</font></div><div><font face="courier new, monospace">apb = flex.double_from_byte_str(pickle.loads(pb))</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># check arrays</font></div><div><font face="courier new, monospace">print('Original:', list(a))</font></div><div><font face="courier new, monospace">print('Pickle: ', list(ap), 'Oh no!')</font></div><div><font face="courier new, monospace">print('Bytes: ', list(ab))</font></div><div><font face="courier new, monospace">print('Bytes2: ', list(apb))</font></div></div><div><br></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div>--</div><div><span style="font-size:12.8000001907349px">Billy K. Poon</span><br></div></div><div>Research Scientist, Molecular Biophysics and Integrated Bioimaging</div><div>Lawrence Berkeley National Laboratory</div><div>1 Cyclotron Road, M/S 33R0345</div><div>Berkeley, CA 94720</div><div>Tel: (510) 486-5709</div><div>Fax: (510) 486-5909</div><div>Web: <a href="https://phenix-online.org" target="_blank">https://phenix-online.org</a></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jun 10, 2019 at 7:10 AM Robert Oeffner <<a href="mailto:rdo20@cam.ac.uk">rdo20@cam.ac.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Hi all,<br>
<br>
It seems that a flex.double array containing "nan" values doesn't <br>
survive pickling as in:<br>
<br>
oeffner@grove:~$ cctbx.python<br>
Python 2.7.15 (default, Sep 11 2018, 23:48:45)<br>
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux2<br>
Type "help", "copyright", "credits" or "license" for more information.<br>
>>><br>
>>> from cctbx.array_family import flex<br>
>>> import pickle<br>
>>> flex_nans = flex.double(3, float('nan'))<br>
>>> list(flex_nans)<br>
[nan, nan, nan]<br>
>>> list(pickle.loads( pickle.dumps(flex_nans)))<br>
[0.0, 0.0, 0.0]<br>
>>><br>
>>><br>
<br>
If the flex double is cast into a python list then it does survive <br>
pickling as in:<br>
<br>
>>><br>
>>> list(pickle.loads( pickle.dumps(list( flex_nans ))))<br>
[nan, nan, nan]<br>
>>><br>
<br>
I would like to use nan values to indicate missing or invalid data <br>
values. Would anyone have a suggestion how to go about this?<br>
<br>
<br>
Many thanks,<br>
<br>
<br>
Rob<br>
<br>
<br>
-- <br>
Robert Oeffner, Ph.D.<br>
Research Associate, The Read Group<br>
Department of Haematology,<br>
Cambridge Institute for Medical Research<br>
University of Cambridge<br>
Cambridge Biomedical Campus<br>
The Keith Peters Building<br>
Hills Road<br>
Cambridge CB2 0XY<br>
<br>
<a href="http://www.cimr.cam.ac.uk/investigators/read/index.html" rel="noreferrer" target="_blank">www.cimr.cam.ac.uk/investigators/read/index.html</a><br>
tel: +44(0)1223 763234<br>
<br>
_______________________________________________<br>
cctbxbb mailing list<br>
<a href="mailto:cctbxbb@phenix-online.org" target="_blank">cctbxbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/cctbxbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailman/listinfo/cctbxbb</a><br>
</blockquote></div>