[cctbxbb] Experimental gradients

[Pavel Afonine]

Hi Kristoffer,

you got it almost right! Attached is the working example (using 1yjp 
model from PDB).

Note:

- choosing f_obs as "f_obs = miller_arrays[1]"  assumes you know it is 
indexed as "1" in the list of miller arrays (which isn't always the case!);

- once you made fmodel you need to do "update_all_scales()" in order to 
be able to compute Fmodel = ktotal * (Fcalc + kmask*Fmask) because 
ktotal=1 and kmask=0 otherwise (the way it is done described here: 
https://journals.iucr.org/d/issues/2013/04/00/dz5273/dz5273.pdf);

- choice of target_name means the target that you get the gradients of.

Let me know if you have any questions or need any help!

Pavel

On 10/10/23 03:47, Kristoffer Lundgren wrote:
>
> Hi Pavel,
>
> Thank you for your answer and hints. Trying to just test and 
> understand how the libraries works, this is as far as I have come:
>
> import iotbx.pdb
>
> import mmtbx.model
>
> import mmtbx.f_model
>
> from iotbx import reflection_file_reader
>
> pdb_file = '1gwd.pdb'
>
> mtz_file = '1gwd.mtz'
>
> pdb_inp = iotbx.pdb.input(file_name=pdb_file)
>
> model = mmtbx.model.manager(model_input=pdb_inp)
>
> miller_arrays = 
> reflection_file_reader.any_reflection_file(file_name=mtz_file).as_miller_arrays()
>
> f_obs = miller_arrays[1]
>
> fmodel = mmtbx.f_model.manager(f_obs=f_obs, 
> xray_structure=model.get_xray_structure(), target_name='ls_wunit_k1')
>
> gradients = fmodel.one_time_gradients_wrt_atomic_parameters()
>
> print(list(gradients.d_target_d_site_cart()))
>
> >>> [(0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 0.0), (0.0, 0.0, 
> 0.0), […]
>
> I would expect the gradients to not be zero for all scatterers so 
> obviously I am doing something wrong – help on how to get further 
> would be very much appreciated.
>
> Best regards
>
> Kristoffer
>
> *From: *Pavel Afonine <pafonine at lbl.gov>
> *Date: *Friday, 6 October 2023 at 02:35
> *To: *cctbx mailing list <cctbxbb at phenix-online.org>, Kristoffer 
> Lundgren <kristoffer.lundgren at compchem.lu.se>
> *Subject: *Re: [cctbxbb] Experimental gradients
>
> Hi Kristoffer,
>
> CCTBX is a source of building blocks for Phenix, and Phenix has real- 
> and reciprocal-space refinement programs, phenix.real_space_refine and 
> phenix.refine, so the answer to your question is yes.
>
> You have not specified, but I'm assuming you are looking for gradients 
> of reciprocal space refinement target (LS, ML, MLHL, etc) and 
> real-space target (we have only one described here: 
> https://pubmed.ncbi.nlm.nih.gov/29872004/) with respect to model 
> parameters such as coordinates, ADPs (isotropic, anisotropic), 
> occupancies, f' and f'', etc.
>
> Here are a few places:
>
> - reciprocal space:
>
> cctbx_project/mmtbx/refinement
>
> files:
>
> minimization.py
> occupancies.py
> rigid_body.py
> group.py
>
> regression/tst_xray_fast_gradients.py
> xray/boost_python/tst_xray.py
> regression/tst_xray_derivatives.py
>
> - real space:
>
> cctbx_project/mmtbx/refinement/real_space/individual_sites.py
> cctbx_project/cctbx/maptbx/tst_real_space_refinement_simple.py
> cctbx_project/cctbx/maptbx/target_and_gradients.h
> cctbx_project/cctbx/maptbx/real_space_refinement_simple.py
>
> Note, these are examples of high-level code that uses target function 
> and its gradients to perform various kind of refinement. You'll need 
> to do some detective work to trace to the low-level code that actually 
> calculates gradients. Hope that's good enough starting point for you.
>
> Also, try grep for "gradient" and "target"! Usually tests (files 
> tst*.py serve as good examples).
>
> Good luck!
> Pavel
>
> On 10/3/23 08:13, Kristoffer Lundgren wrote:
>
>     Hello all,
>
>     I am wondering if it is possible to use cctbx to calculate
>     gradients from experimental data? Both in reciprocal and real
>     space is of interest.
>
>     Are there any code examples available somewhere showcasing how
>     this can be achieved?
>
>     Best regards
>
>     Kristoffer Lundgren
>
>
>
>     _______________________________________________
>
>     cctbxbb mailing list
>
>     cctbxbb at phenix-online.org
>
>     http://phenix-online.org/mailman/listinfo/cctbxbb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20231010/7fb1d026/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: run.py
Type: text/x-python-script
Size: 1375 bytes
Desc: not available
URL: <http://phenix-online.org/pipermail/cctbxbb/attachments/20231010/7fb1d026/attachment.bin>