[cctbxbb] [Dials-support] indexing success rates
James Holton
jmholton at lbl.gov
Mon Feb 5 14:14:17 PST 2018
I can definitely see the advantages of a second pass with a consensus
cell. There will always be weak, split or contaminated images that
could benefit from the prior information. But in this case I cranked up
the spot intensity to be just shy of overloads and we have beautiful,
clearly-single-crystal patterns. If you can't index these then
something is wrong. I don't like comparing software packages like this
because it is hard not to sound insulting to at least one group of
authors, but I hope we can all accept that if we have a clear and
high-quality image and both mosflm and XDS can get the correct cell from
it, then there is no reason why DIALS shouldn't.
I have now prepared a 2-image example based on seed 56. In this case
rotating the crystal by 0.01 degrees makes the difference between
dials.stills_process going from confidently arriving at a very wrong
cell in fft1d,fft3d,real_space_grid_search mode to instead arriving at
the correct one. In both orientations mosflm and xds arrive at the
correct cell. Files here:
http://bl831.als.lbl.gov/~jamesh/bugreports/dials_index_020418.tgz
I believe the synthetic images do help nail down where the problem might
be. It can't be finding too few spots, because the spots are clear. It
can't be splitting, ice, zingers or other outlier spots, because I
didn't put those in. It can't be the beam center because its in the
center of the image and the spot separation is nice and wide. It has to
be something in the indexing or cell reduction algorithms.
Comparing the logs dials_000.log vs dials_001.log, the problem seems to
arrive as early as the "Candidate basis vectors". A 4th ~8 A basis
vector is found and seems to win out over the others for reasons that
aren't really obvious to me. Where is the code for this step?
-James
On 2/3/2018 1:11 AM, Aaron Brewster wrote:
> Right, typically if I don't know the cell, I'll index a bunch of
> images with no target, then use cluster.unit_cell on the results to
> get a consensus cell, then re-index with that cell.
>
> For dials.stills_process, the default of fft1d works pretty well
> generally, but to get the best results,
> indexing.method_list=fft1d,fft3d might be better. There, if the first
> method fails the second is tried. real_space_grid_search can also be used.
>
> Getting a specific image in a composite file (such as hdf5) isn't a
> use case I've hit often, so I don't have an easy solution there. It's
> possible an image range parameter to dials.import would be useful?
>
> -Aaron
>
> On Sat, Feb 3, 2018 at 1:02 AM, Graeme Winter <graeme.winter at gmail.com
> <mailto:graeme.winter at gmail.com>> wrote:
>
> Hi Aaron
>
> (and DIALS)
>
> For stills / single images - should dials.index switch by magic to
> default fft1d?
>
> Of is that the responsibility of e.g. xia2 / cctbx.xfel to worry
> about?
>
> It has to be said, more often than not, particularly for
> screening, we do not know cell / symmetry a priori (at diamond
> anyways)
>
> A related question
>
> If I have a data block / strong pickle from 1000 stills, how can I
> index e.g. image 672? Do I have to split this into 1000 separate
> things?
>
> @JH - thanks for this - I think some rigorous testing will be
> useful here.
>
> Cheers Graeme
>
>> On 3 Feb 2018, at 08:37, Aaron Brewster <asbrewster at lbl.gov
>> <mailto:asbrewster at lbl.gov>> wrote:
>>
>> Hi James, thanks. I looked at random image 15 and after trying a
>> bunch of stuff it indexes fine if I use indexing.method=fft3d.
>> The default for stills processing is fft1d, so I find this super
>> interesting.
>>
>> For what it's worth, I pretty much never process stills without a
>> target cell. I generated ~30 random images using your script and
>> indexed them, then I got the unit cells with "dials.show
>> *refined*.json | grep cell". They were all about the same. So I
>> re-indexed random image 15 using fft1d but I also specified
>> known_symmetry.cell=50,60,70,90,90,90 and it indexed fine.
>>
>> So, either indexing.method=fft3d or
>> known_symmetry.cell=50,60,70,90,90,90 is sufficient to index
>> random image 15. Certainly for experimental data I would
>> recommend using known_symmetry as the results are always better.
>> However, I'd like to look deeper at fft1d vs fft3d at some point.
>>
>> Cc'ing dials-support.
>>
>> Thanks!
>> -Aaron
>>
>>
>>
>> On Fri, Feb 2, 2018 at 1:35 PM, James Holton <jmholton at lbl.gov
>> <mailto:jmholton at lbl.gov>> wrote:
>>
>> While testing the new simTBX diffraction image simulator
>> Aaron and I found that some rather beautiful-looking images
>> still won't index. Subsequently, I followed up with mosflm,
>> which does succeed in these cases, even without prior
>> knowledge of the cell or space group.
>>
>> I have now run this 1000 times, varying only the crystal
>> orientation, and found 171 cases of dials.stills_process
>> failing to index the simulated image whereas mosflm had no
>> trouble, and 5 cases where mosflm failed and
>> dials.stills_process succeeded. There were no overlapping
>> cases where both programs failed.
>>
>> I have tarballed up the relevant files here:
>> http://bl831.als.lbl.gov/~jamesh/bugreports/dials_index_020218.tgz
>> <http://bl831.als.lbl.gov/%7Ejamesh/bugreports/dials_index_020218.tgz>
>>
>> The tarball contains a "runme.com <http://runme.com/>" shell
>> script for reproducing these results, and also the
>> tst_nanoBragg_forindex.py jiffy for making a test image given
>> a provided random-number seed. For example:
>>
>> libtbx.python ./tst_nanoBragg_forindex.py random 15
>>
>> will create a file called noiseimage_001.cbf that
>> dials.stills_process cannot index (at least in my hands). For
>> those of you who don't have mosflm installed, the tarball
>> incluses a copy, and the "autoindex.com
>> <http://autoindex.com/>" script should run on any linux system.
>>
>> Any ideas as to why there is this discrepancy? Feels like an
>> opportunity for a non-trivial improvement.
>>
>> Something to ponder over the weekend, I suppose. Note that I
>> recently checked in a few bug fixes to simTBX, but this
>> analysis is unaffected by them, so you shouldn't have to
>> update your build to reproduce this.
>>
>> Cheers,
>>
>> -James Holton
>> MAD Scientist
>>
>>
>>
>> _______________________________________________
>> cctbxbb mailing list
>> cctbxbb at phenix-online.org <mailto:cctbxbb at phenix-online.org>
>> http://phenix-online.org/mailman/listinfo/cctbxbb
>> <http://phenix-online.org/mailman/listinfo/cctbxbb>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org <http://Slashdot.org>!
>> http://sdm.link/slashdot_______________________________________________
>> <http://sdm.link/slashdot_______________________________________________>
>> DIALS-support mailing list
>> DIALS-support at lists.sourceforge.net
>> <mailto:DIALS-support at lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/dials-support
>> <https://lists.sourceforge.net/lists/listinfo/dials-support>
>
>
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/cctbxbb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20180205/f75ddadc/attachment-0001.htm>
More information about the cctbxbb
mailing list