[cctbxbb] Unstable platform-dependent sort order for flex.sort_permutation

[Nicholas Sauter]

Richard & Oleg,

I'm not in office right now, but I think the concept you are looking for is
the "stable sort", which would sort items but give the fewest
rearrangements.  I believe the C++ standard library exposes this function,
so it would be possible to create a corresponding
flex.stable_sort_permuation for our array family.

Nick

Nicholas K. Sauter, Ph. D.
Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33R0345
Berkeley, CA 94720
(510) 486-5713

On Fri, Nov 18, 2016 at 8:10 AM, <richard.gildea at diamond.ac.uk> wrote:

> Hi Oleg,
>
> The permutations would lead to the same sequence of miller indices, but in
> the context of an unmerged reflection array (i.e.
> miller.array.sort_permutation), then the data associated with those
> miller indices would not necessarily be in the same order. As the dataset
> is then split into two half-datasets, this difference in sort order leads
> to a different value of the calculate correlation coefficient between those
> two half datasets:
>
> https://github.com/cctbx/cctbx_project/blob/master/
> cctbx/miller/__init__.py#L4700
>
> Cheers,
>
> Richard
>
> Dr Richard Gildea
> Data Analysis Scientist
> Tel: +441235 77 8078
>
> Diamond Light Source Ltd.
> Diamond House
> Harwell Science & Innovation Campus
> Didcot
> Oxfordshire
> OX11 0DE
> ________________________________
> From: cctbxbb-bounces at phenix-online.org [cctbxbb-bounces at phenix-online.org]
> on behalf of oleg at olexsys.org [oleg at olexsys.org]
> Sent: 18 November 2016 15:59
> To: cctbx mailing list
> Subject: Re: [cctbxbb] Unstable platform-dependent sort order for
> flex.sort_permutation
>
>
> Hi Richard,
>
>
> it is a bug only if the permutations do not lead to the same sequence.
> Otherwise you cannot expect to get the same sorting permutations for
> collections with redundant data on different platforms or even between
> different versions of compilers.
>
>
> Cheers,
>
> Oleg.
>
> ________________________________
> From: cctbxbb-bounces at phenix-online.org <cctbxbb-bounces at phenix-online.org>
> on behalf of richard.gildea at diamond.ac.uk <richard.gildea at diamond.ac.uk>
> Sent: 18 November 2016 14:12:29
> To: cctbxbb at phenix-online.org
> Subject: [cctbxbb] Unstable platform-dependent sort order for
> flex.sort_permutation
>
> Hi,
>
> I've been trying to diagnose why the miller array cc_anom calculations
> seem to be platform dependent. I've narrowed this down to a variation in
> the sort order returned by flex.sort_permutation which is called from
> within miller.array.sort("packed_indices"). The following code
> demonstrates the problem and the different output I get on mac and Linux:
>
> Mac:
> >>> from scitbx.array_family import flex
> >>> a = flex.size_t([7, 1, 1, 5, 1, 3, 3, 7, 1, 7, 3, 3, 5, 7, 5, 5, 5, 1,
> 1, 1])
> >>> print list(flex.sort_permutation(a))
> [1, 2, 4, 8, 17, 18, 19, 5, 6, 10, 11, 3, 12, 14, 15, 16, 0, 7, 9, 13]
>
> Linux:
> >>> from scitbx.array_family import flex
> >>> a = flex.size_t([7, 1, 1, 5, 1, 3, 3, 7, 1, 7, 3, 3, 5, 7, 5, 5, 5, 1,
> 1, 1])
> >>> print list(flex.sort_permutation(a))
> [19, 1, 2, 18, 4, 17, 8, 11, 10, 6, 5, 12, 14, 15, 16, 3, 9, 7, 13, 0]
>
> Is it known/expected for the sort order to be platform dependent, or is
> this a bug?
>
> Here is the relevant code for flex.sort_permutation():
>
> https://github.com/cctbx/cctbx_project/blob/master/
> scitbx/array_family/sort.h
>
> Cheers,
>
> Richard
>
> Dr Richard Gildea
> Data Analysis Scientist
> Tel: +441235 77 8078
>
> Diamond Light Source Ltd.
> Diamond House
> Harwell Science & Innovation Campus
> Didcot
> Oxfordshire
> OX11 0DE
>
> --
> This e-mail and any attachments may contain confidential, copyright and or
> privileged material, and are for the use of the intended addressee only. If
> you are not the intended addressee or an authorised recipient of the
> addressee please notify us of receipt by returning the e-mail and do not
> use, copy, retain, distribute or disclose the information in or attached to
> the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
> not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
> attachments are free from viruses and we cannot accept liability for any
> damage which you may sustain as a result of software viruses which may be
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
> and Wales with its registered office at Diamond House, Harwell Science and
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20161118/a7afde44/attachment-0001.htm>