Fast way to enable/disable individual scatterers in structure factor calculation
Hi, I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation. As I see from the source (structure_factors_direct.h) the selection toggle does not seem to be evaluated for this. Of course an additional check for the selection flag would harm performance in most standard cases, so I'd rather like to have a separate c++ function for this, which could be swapped in for the normal f_calc calculation by a boolean toggle in the python part of f_calc. As I'm not that much into the c++ part of cctbx, I'd really appreciate if somebody more knowledgeable could monkey patch this. Cheers, Jan
Hi Jan,
I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation.
I recommend not to do that. Instead I would create a new instance of xray.structure with those scatterers you want, using the the methods xxx_selection and then the method select, and then use it for structure factor calculations. Best wishes, Luc
Am Freitag 29 Juni 2012 10:27:19 schrieb Luc Bourhis:
Hi Jan,
I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation.
I recommend not to do that. Instead I would create a new instance of xray.structure with those scatterers you want, using the the methods xxx_selection and then the method select, and then use it for structure factor calculations.
I'm concerned about overhead here, as I'd like to quickly check a lot (some millions) of different variations of a basically identical structure against a set of given f_obs. I only need to calculate f_calc once for each of the slightly modified structures. So I think simply masking one or some of the scatterers instead of creating a new (copy of the) structure would save a huge ammount of cpu time. Cheers, Jan
On 29 Jun 2012, at 11:47, Jan Marten Simons wrote:
Am Freitag 29 Juni 2012 10:27:19 schrieb Luc Bourhis:
Hi Jan,
I'm looking for a simple and fast way to set individual scatterers to be either included or excluded from structure factor calculation.
I recommend not to do that. Instead I would create a new instance of xray.structure with those scatterers you want, using the the methods xxx_selection and then the method select, and then use it for structure factor calculations.
I'm concerned about overhead here, as I'd like to quickly check a lot (some millions) of different variations of a basically identical structure against a set of given f_obs. I only need to calculate f_calc once for each of the slightly modified structures. So I think simply masking one or some of the scatterers instead of creating a new (copy of the) structure would save a huge ammount of cpu time.
The structure factor calculations will typically dwarf the copies. As usual, try it, time it and if it is too slow then profile it to ascertain where you should spend your time optimising. Best wishes, Luc Bourhis
Am Freitag 06 Juli 2012 11:15:41 schrieb Luc Bourhis:
Hi Jan,
The structure factor calculations will typically dwarf the copies. As usual, try it, time it and if it is too slow then profile it to ascertain where you should spend your time optimising.
Was my assumption correct in your case?
Hi Luc, Well, I did some profiling and cctbx is doing quite a good job with not all too much overhead here. Still I think some of this could be cut down for the task I described in my original post. Here is some exerpt from a profile run: (I did 4686825 different structure evaluations for this test) 1096744290 function calls in 12304.826 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) ... 4686825 86.526 0.000 9898.006 0.002 from_scatterers.py:11(__call__) 4686825 86.526 0.000 9898.006 0.002 from_scatterers.py:11(__call__) 4686825 7442.794 0.002 9476.525 0.002 from_scatterers_fft.py:11(__init__) 4686825 16.761 0.000 65.343 0.000 from_scatterers_fft.py:66(f_calc) 468 0.001 0.000 0.002 0.000 genericpath.py:85(_splitext) 4686825 7.850 0.000 7.850 0.000 global_counters.py:15(process) 4686825 82.863 0.000 439.964 0.000 introspection.py:114(__init__) 4686825 23.952 0.000 110.286 0.000 introspection.py:120(virtual_memory_peak_size) 4686825 13.638 0.000 61.437 0.000 introspection.py:128(virtual_memory_size) 4686825 12.718 0.000 56.038 0.000 introspection.py:134(resident_set_size) 4686825 12.295 0.000 54.883 0.000 introspection.py:140(stack_size) 4686825 22.563 0.000 305.207 0.000 introspection.py:146(update_max) 18747300 132.635 0.000 203.779 0.000 introspection.py:85(get_bytes) 18747300 96.497 0.000 100.365 0.000 manager.py:102(rfft) 4686825 70.220 0.000 73.808 0.000 manager.py:108(cfft) 9373650 31.406 0.000 38.115 0.000 manager.py:114(u_base) 4686825 13.829 0.000 213.898 0.000 manager.py:122(setup_fft) 4686825 5.525 0.000 22.302 0.000 manager.py:128(have_good_timing_estimates) 4686825 68.036 0.000 348.953 0.000 manager.py:13(__init__) 4686825 8.770 0.000 8.770 0.000 manager.py:134(__init__) 4686825 16.777 0.000 16.777 0.000 manager.py:140(have_good_estimate) 4686825 16.077 0.000 16.077 0.000 manager.py:162(__init__) 4686825 19.558 0.000 19.558 0.000 manager.py:181(register) 4686825 84.479 0.000 185.810 0.000 manager.py:229(__init__) 4686825 2.366 0.000 2.366 0.000 manager.py:245(miller_set) 14060475 5.893 0.000 5.893 0.000 manager.py:53(d_min) 9373650 4.043 0.000 4.043 0.000 manager.py:59(grid_resolution_factor) 4686825 2.078 0.000 2.078 0.000 manager.py:62(symmetry_flags) 4686825 2.065 0.000 2.065 0.000 manager.py:65(mandatory_grid_factors) 4686825 3.168 0.000 3.168 0.000 manager.py:68(quality_factor) 4686825 4.619 0.000 4.619 0.000 manager.py:71(wing_cutoff) 4686825 2.562 0.000 2.562 0.000 manager.py:74(exp_table_one_over_step_size) 4686825 1.906 0.000 1.906 0.000 manager.py:77(max_prime) 4686825 2.028 0.000 2.028 0.000 manager.py:80(force_complex) 4686825 2.103 0.000 2.103 0.000 manager.py:83(sampled_density_must_be_positive) 4686825 3.513 0.000 3.513 0.000 manager.py:86(tolerance_positive_definite) 14060475 57.552 0.000 291.970 0.000 manager.py:89(crystal_gridding) 4686825 8.547 0.000 10.303 0.000 misc.py:3(quality_factor_from_any) 468 0.001 0.000 0.001 0.000 posixpath.py:110(basename) 936 0.000 0.000 0.000 0.000 posixpath.py:43(normcase) 468 0.001 0.000 0.003 0.000 posixpath.py:95(splitext) 1 0.000 0.000 0.000 0.000 random_fill.py:2(<module>) 1 0.000 0.000 0.000 0.000 random_fill.py:9(RandomFill) 42181425 421.610 0.000 421.610 0.000 scatterer.py:13(__init__) 1 0.000 0.000 0.000 0.000 structure.py:109(deep_copy_scatterers) 10 0.000 0.000 0.000 0.000 structure.py:1209(special_position_indices) 4686825 466.303 0.000 474.725 0.000 structure.py:1212(scattering_type_registry) 4686825 366.871 0.000 384.748 0.000 structure.py:1357(add_scatterers) <--- 4686825 97.723 0.000 14262.277 0.003 structure.py:1440(structure_factors) 10 0.000 0.000 0.000 0.000 structure.py:1489(show_summary) 10 0.000 0.000 0.002 0.000 structure.py:1498(show_scatterers) 18747320 8.940 0.000 8.940 0.000 structure.py:150(scatterers) 4686825 5.531 0.000 5.531 0.000 structure.py:26(__init__) 4686827 49.952 0.000 518.240 0.000 structure.py:50(__init__) 4686827 50.261 0.000 50.261 0.000 structure.py:99(erase_scatterers) <--- 469 0.003 0.000 0.004 0.000 threading.py:120(acquire) 469 0.008 0.000 0.008 0.000 threading.py:140(release) 938 0.001 0.000 0.001 0.000 threading.py:62(_note) 469 0.001 0.000 0.001 0.000 threading.py:681(name) 469 0.001 0.000 0.001 0.000 threading.py:825(currentThread) At least all calls to add_scatterers and erase_scatterers could be skipped if one could easily mask individual scatterers. possibly some other calls as well. As I'll have to work with really many structure evaluations even a gain of a few % in runtime is very welcome. With regards, Dipl. Phys. Jan M. Simons Institute of Crystallography RWTH Aachen University
participants (2)
-
Jan Marten Simons -
Luc Bourhis