[cctbxbb] [Dials-support] indexing success rates

[Aaron Brewster]

Right, typically if I don't know the cell, I'll index a bunch of images
with no target, then use cluster.unit_cell on the results to get a
consensus cell, then re-index with that cell.

For dials.stills_process, the default of fft1d works pretty well generally,
but to get the best results, indexing.method_list=fft1d,fft3d might be
better.  There, if the first method fails the second is tried.
real_space_grid_search can also be used.

Getting a specific image in a composite file (such as hdf5) isn't a use
case I've hit often, so I don't have an easy solution there.  It's possible
an image range parameter to dials.import would be useful?

-Aaron

On Sat, Feb 3, 2018 at 1:02 AM, Graeme Winter <graeme.winter at gmail.com>
wrote:

> Hi Aaron
>
> (and DIALS)
>
> For stills / single images - should dials.index switch by magic to default
> fft1d?
>
> Of is that the responsibility of e.g. xia2 / cctbx.xfel to worry about?
>
> It has to be said, more often than not, particularly for screening, we do
> not know cell / symmetry a priori (at diamond anyways)
>
> A related question
>
> If I have a data block / strong pickle from 1000 stills, how can I index
> e.g. image 672? Do I have to split this into 1000 separate things?
>
> @JH - thanks for this - I think some rigorous testing will be useful here.
>
> Cheers Graeme
>
> On 3 Feb 2018, at 08:37, Aaron Brewster <asbrewster at lbl.gov> wrote:
>
> Hi James, thanks.  I looked at random image 15 and after trying a bunch of
> stuff it indexes fine if I use indexing.method=fft3d.  The default for
> stills processing is fft1d, so I find this super interesting.
>
> For what it's worth, I pretty much never process stills without a target
> cell.  I generated ~30 random images using your script and indexed them,
> then I got the unit cells with "dials.show *refined*.json | grep cell".
> They were all about the same.  So I re-indexed random image 15 using fft1d
> but I also specified known_symmetry.cell=50,60,70,90,90,90 and it indexed
> fine.
>
> So, either indexing.method=fft3d or known_symmetry.cell=50,60,70,90,90,90
> is sufficient to index random image 15.  Certainly for experimental data I
> would recommend using known_symmetry as the results are always better.
> However, I'd like to look deeper at fft1d vs fft3d at some point.
>
> Cc'ing dials-support.
>
> Thanks!
> -Aaron
>
>
>
> On Fri, Feb 2, 2018 at 1:35 PM, James Holton <jmholton at lbl.gov> wrote:
>
>> While testing the new simTBX diffraction image simulator Aaron and I
>> found that some rather beautiful-looking images still won't index.
>> Subsequently, I followed up with mosflm, which does succeed in these cases,
>> even without prior knowledge of the cell or space group.
>>
>> I have now run this 1000 times, varying only the crystal orientation, and
>> found 171 cases of dials.stills_process failing to index the simulated
>> image whereas mosflm had no trouble, and 5 cases where mosflm failed and
>> dials.stills_process succeeded.  There were no overlapping cases where both
>> programs failed.
>>
>> I have tarballed up the relevant files here:
>> http://bl831.als.lbl.gov/~jamesh/bugreports/dials_index_020218.tgz
>>
>> The tarball contains a "runme.com" shell script for reproducing these
>> results, and also the tst_nanoBragg_forindex.py jiffy for making a test
>> image given a provided random-number seed.  For example:
>>
>> libtbx.python ./tst_nanoBragg_forindex.py random 15
>>
>> will create a file called noiseimage_001.cbf that dials.stills_process
>> cannot index (at least in my hands).  For those of you who don't have
>> mosflm installed, the tarball incluses a copy, and the "autoindex.com"
>> script should run on any linux system.
>>
>> Any ideas as to why there is this discrepancy?  Feels like an opportunity
>> for a non-trivial improvement.
>>
>> Something to ponder over the weekend, I suppose.  Note that I recently
>> checked in a few bug fixes to simTBX, but this analysis is unaffected by
>> them, so you shouldn't have to update your build to reproduce this.
>>
>> Cheers,
>>
>> -James Holton
>> MAD Scientist
>>
>>
>>
>> _______________________________________________
>> cctbxbb mailing list
>> cctbxbb at phenix-online.org
>> http://phenix-online.org/mailman/listinfo/cctbxbb
>>
>>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
> _________________________________________
> DIALS-support mailing list
> DIALS-support at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dials-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20180203/8135d63f/attachment-0001.htm>