adding together

James Holton

12 Sep 2025 12 Sep '25

11:28 a.m.

Greetings Phenix developers! I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this? I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations. Thank you, -James Holton MAD Scientist

Show replies by date

Petrus Zwart

12 Sep 12 Sep

11:34 a.m.

https://claude.ai/public/artifacts/f390c669-b8c7-4776-a4bc-1761b110bba0 On Fri, Sep 12, 2025 at 11:29 AM James Holton wrote:

...

Greetings Phenix developers!

I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this?

I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations.

Thank you,

-James Holton MAD Scientist

_______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- ------------------------------------------------------------------------------------------ Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246 ------------------------------------------------------------------------------------------

Tom Terwilliger

11:36 a.m.

Hi James, Sounds like the chat did not do well on this one! You could try: phenix.average_map_coeffs map_coeffs_1.mtz map_coeffs_2.mtz All the best, Tom On Fri, Sep 12, 2025 at 12:29 PM James Holton wrote:

...

Greetings Phenix developers!

I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this?

I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations.

Thank you,

-James Holton MAD Scientist

_______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- Thomas C Terwilliger Laboratory Fellow, Los Alamos National Laboratory Senior Scientist, New Mexico Consortium 100 Entrada Dr, Los Alamos, NM 87544 Email: [email protected] Tel: 505-431-0010

James Holton

11:56 a.m.

Thanks Tom! And if I wanted to square them first? This is for a diffuse scatter calculation, BTW. Need phased and unphased sums. On 9/12/2025 11:36 AM, Tom Terwilliger wrote:

...

Hi James, Sounds like the chat did not do well on this one!

You could try:

phenix.average_map_coeffs map_coeffs_1.mtz map_coeffs_2.mtz

All the best, Tom

On Fri, Sep 12, 2025 at 12:29 PM James Holton wrote:

Greetings Phenix developers!

I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this?

I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations.

Thank you,

-James Holton MAD Scientist

_______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- Thomas C Terwilliger Laboratory Fellow, Los Alamos National Laboratory Senior Scientist, New Mexico Consortium 100 Entrada Dr, Los Alamos, NM 87544 Email: [email protected] Tel: 505-431-0010

Pavel Afonine

11:57 a.m.

Hi James, attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 ) It uses 1yjp as a source of sample files to run the example, you can get it using phenix.fetch_pdb 1yjp action=all convert_to_mtz=true Pavel On 9/12/25 11:28, James Holton wrote:

...

Greetings Phenix developers!

I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this?

I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations.

Thank you,

-James Holton MAD Scientist

Petrus Zwart

12:04 p.m.

Hi Pavel, You say 25 lines, but there are zero - can you attach the code? P On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine wrote:

...

Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote:

...
Greetings Phenix developers!

I have what I hope is a quick question. I'm trying to add a stack of phased structure factors together. Specifically from separate runs of phenix.fmodel. Normally, I'd use sftools for this, but the distributed version has a size limitation. According to Tom's phenix chatbot, I can use phenix.maps for adding two sets of structure factors together, but it seems to only want to combine an mtz with a pdb. There is iotbx.reflection_file_editor, but it only works from the GUI (and I want to add 1000 mtz files together). phenix.xmanip looks promising too, but again, seems to want a pdb and an mtz. Is there a recommended way to do this?

I've already asked this on CCP4BB, but I wanted to get the Phenix take on these kinds of manipulations.

Thank you,

-James Holton MAD Scientist

_______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

Pavel Afonine

12:12 p.m.

Oh, does phenixbb really strip off attachments under 1 KB in size?! Terrible. Anyway, I’m inlining it below (make sure your email client doesn’t mess up the indentation): ******** from iotbx import reflection_file_reader import iotbx.pdb def run(): # Read your reflection data miller_arrays = reflection_file_reader.any_reflection_file(file_name = "1yjp.mtz").as_miller_arrays() for ma in miller_arrays: print (ma.info().label_string()) if(ma.info().label_string()=="FOBS,SIGFOBS"): f_obs = ma # Get phases by computing Fcalc (for example) pdb_inp = iotbx.pdb.input(file_name = "1yjp.pdb") xrs = pdb_inp.xray_structure_simple() f_calc = f_obs.structure_factors_from_scatterers(xray_structure=xrs).f_calc() # Transfer phased from f_calc to f_obs f_obs_cmpl = f_obs.phase_transfer(phase_source = f_calc.phases()) print(f_obs_cmpl.data()) # just to see the data are now complex array! # Write into MTZ mtz_dataset = f_calc.as_mtz_dataset(column_root_label = "Fcmpl") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "data.mtz") if __name__ == '__main__': run() ******** Pavel On 9/12/25 12:04, Petrus Zwart wrote:

...

Hi Pavel,

You say 25 lines, but there are zero - can you attach the code?

P

On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine wrote:

Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote: > Greetings Phenix developers! > > I have what I hope is a quick question. I'm trying to add a stack of > phased structure factors together. Specifically from separate runs of > phenix.fmodel. Normally, I'd use sftools for this, but the > distributed version has a size limitation. According to Tom's phenix > chatbot, I can use phenix.maps for adding two sets of structure > factors together, but it seems to only want to combine an mtz with a > pdb. There is iotbx.reflection_file_editor, but it only works from the > GUI (and I want to add 1000 mtz files together). phenix.xmanip looks > promising too, but again, seems to want a pdb and an mtz. Is there a > recommended way to do this? > > I've already asked this on CCP4BB, but I wanted to get the Phenix take > on these kinds of manipulations. > > Thank you, > > -James Holton > MAD Scientist > _______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- ------------------------------------------------------------------------------------------ Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246 ------------------------------------------------------------------------------------------

James Holton

12:36 p.m.

Thank you for this Pavel, Funny, I got the attachment just fine? Maybe its on P's email client side. I'm using thunderbird. However, the code you shared seems to be taking an amplitude from one place and attaching it to a phase from another. What I want to do is the complex sum of two phased structure factors. Specifically, I have is a stack of 1000 mtz files with FMODEL PHIMODEL in each. What I want is the phased sum of all of them, as well as the unphased sum of their squares. I've been doing the additions two at a time because that is parallelizable. My script is here: https://github.com/jmholton/altloc_diffuse/blob/main/addup_mtzs_diffuse.com It is currently using sftools for these operations: calc ( COL Fsum PHIsum ) = ( COL F1 P1 ) ( COL F2 P2 ) + calc COL I1 = COL F1 2 ** calc COL I2 = COL F2 2 ** calc J COL Isum = COL I1 COL I2 + There does not seem to be a general-purpose math toolkit like sftools in phenix? Much appreciate all the quick responses! -James On 9/12/2025 12:12 PM, Pavel Afonine wrote:

...

Oh, does phenixbb really strip off attachments under 1 KB in size?! Terrible. Anyway, I’m inlining it below (make sure your email client doesn’t mess up the indentation):

******** from iotbx import reflection_file_reader import iotbx.pdb

def run(): # Read your reflection data miller_arrays = reflection_file_reader.any_reflection_file(file_name = "1yjp.mtz").as_miller_arrays() for ma in miller_arrays: print (ma.info().label_string()) if(ma.info().label_string()=="FOBS,SIGFOBS"): f_obs = ma # Get phases by computing Fcalc (for example) pdb_inp = iotbx.pdb.input(file_name = "1yjp.pdb") xrs = pdb_inp.xray_structure_simple() f_calc = f_obs.structure_factors_from_scatterers(xray_structure=xrs).f_calc() # Transfer phased from f_calc to f_obs f_obs_cmpl = f_obs.phase_transfer(phase_source = f_calc.phases()) print(f_obs_cmpl.data()) # just to see the data are now complex array! # Write into MTZ mtz_dataset = f_calc.as_mtz_dataset(column_root_label = "Fcmpl") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "data.mtz")

if __name__ == '__main__': run()

********

Pavel

On 9/12/25 12:04, Petrus Zwart wrote:

...
Hi Pavel,

You say 25 lines, but there are zero - can you attach the code?

P

On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine wrote:

Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote: > Greetings Phenix developers! > > I have what I hope is a quick question. I'm trying to add a stack of > phased structure factors together. Specifically from separate runs of > phenix.fmodel. Normally, I'd use sftools for this, but the > distributed version has a size limitation. According to Tom's phenix > chatbot, I can use phenix.maps for adding two sets of structure > factors together, but it seems to only want to combine an mtz with a > pdb. There is iotbx.reflection_file_editor, but it only works from the > GUI (and I want to add 1000 mtz files together). phenix.xmanip looks > promising too, but again, seems to want a pdb and an mtz. Is there a > recommended way to do this? > > I've already asked this on CCP4BB, but I wanted to get the Phenix take > on these kinds of manipulations. > > Thank you, > > -James Holton > MAD Scientist > _______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- ------------------------------------------------------------------------------------------ Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246 ------------------------------------------------------------------------------------------

Tom Terwilliger

1:08 p.m.

Hi James, Have a look at the tool: phenix.get_struct_fact_from_md I have not used this in some time but it does pretty much what you are talking about. It is not a supported tool however so it may not be easy to use. All the best, Tom T On Fri, Sep 12, 2025 at 1:37 PM James Holton wrote:

...

Thank you for this Pavel,

Funny, I got the attachment just fine? Maybe its on P's email client side. I'm using thunderbird.

However, the code you shared seems to be taking an amplitude from one place and attaching it to a phase from another. What I want to do is the complex sum of two phased structure factors.

Specifically, I have is a stack of 1000 mtz files with FMODEL PHIMODEL in each. What I want is the phased sum of all of them, as well as the unphased sum of their squares. I've been doing the additions two at a time because that is parallelizable. My script is here: https://github.com/jmholton/altloc_diffuse/blob/main/addup_mtzs_diffuse.com It is currently using sftools for these operations: calc ( COL Fsum PHIsum ) = ( COL F1 P1 ) ( COL F2 P2 ) + calc COL I1 = COL F1 2 ** calc COL I2 = COL F2 2 ** calc J COL Isum = COL I1 COL I2 +

There does not seem to be a general-purpose math toolkit like sftools in phenix?

Much appreciate all the quick responses!

-James

On 9/12/2025 12:12 PM, Pavel Afonine wrote:

...
Oh, does phenixbb really strip off attachments under 1 KB in size?! Terrible. Anyway, I’m inlining it below (make sure your email client doesn’t mess up the indentation):

******** from iotbx import reflection_file_reader import iotbx.pdb

def run(): # Read your reflection data miller_arrays = reflection_file_reader.any_reflection_file(file_name = "1yjp.mtz").as_miller_arrays() for ma in miller_arrays: print (ma.info().label_string()) if(ma.info().label_string()=="FOBS,SIGFOBS"): f_obs = ma # Get phases by computing Fcalc (for example) pdb_inp = iotbx.pdb.input(file_name = "1yjp.pdb") xrs = pdb_inp.xray_structure_simple() f_calc = f_obs.structure_factors_from_scatterers(xray_structure=xrs).f_calc() # Transfer phased from f_calc to f_obs f_obs_cmpl = f_obs.phase_transfer(phase_source = f_calc.phases()) print(f_obs_cmpl.data()) # just to see the data are now complex array! # Write into MTZ mtz_dataset = f_calc.as_mtz_dataset(column_root_label = "Fcmpl") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "data.mtz")

if __name__ == '__main__': run()

********

Pavel

On 9/12/25 12:04, Petrus Zwart wrote:

...
Hi Pavel,

You say 25 lines, but there are zero - can you attach the code?

P

On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine

wrote:

...
...
Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote: > Greetings Phenix developers! > > I have what I hope is a quick question. I'm trying to add a stack of > phased structure factors together. Specifically from separate runs of > phenix.fmodel. Normally, I'd use sftools for this, but the > distributed version has a size limitation. According to Tom's phenix > chatbot, I can use phenix.maps for adding two sets of structure > factors together, but it seems to only want to combine an mtz with a > pdb. There is iotbx.reflection_file_editor, but it only works from the > GUI (and I want to add 1000 mtz files together). phenix.xmanip looks > promising too, but again, seems to want a pdb and an mtz. Is there a > recommended way to do this? > > I've already asked this on CCP4BB, but I wanted to get the Phenix take > on these kinds of manipulations. > > Thank you, > > -James Holton > MAD Scientist > _______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

--

...
...
Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246

...
...

phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- Thomas C Terwilliger Laboratory Fellow, Los Alamos National Laboratory Senior Scientist, New Mexico Consortium 100 Entrada Dr, Los Alamos, NM 87544 Email: [email protected] Tel: 505-431-0010

Pavel Afonine

1:23 p.m.

Hi James, script pasted below should do it. This time it is 54 lines mostly because I was assuming arrays you are summing do not have matching indices. **** from iotbx import reflection_file_reader import iotbx.pdb import os from scitbx.array_family import flex def run(): # Collect all data arrays from input files arrays = [] for f in os.listdir("."): if not (f.startswith("fmodel") and f.endswith(".mtz")): continue miller_arrays = reflection_file_reader.any_reflection_file(file_name = f).as_miller_arrays() for ma in miller_arrays: #print (ma.info().label_string()) if(ma.info().label_string()=="FMODEL,PHIFMODEL"): arrays.append(ma) break # Assume arrays may not have matching indices (e.g., different resolution), # so make them all to have matching indices for ma in arrays: common_ref_array = arrays[0] aligned_arrays = [common_ref_array] new_arrays = [] for i in range(1, len(arrays)): next_array = arrays[i] common_ref_array, next_array_aligned = common_ref_array.common_sets(next_array) aligned_arrays[0] = common_ref_array aligned_arrays.append(next_array_aligned) new_arrays = aligned_arrays # sum_cmpl = None sum_sq = None for ma in new_arrays: data = ma.data() data_abs = flex.abs(data) data_abs_sq = data_abs * data_abs if sum_cmpl is None: sum_cmpl = data sum_sq = data_abs_sq else: sum_cmpl = sum_cmpl + data sum_sq = sum_sq + data_abs_sq f_sum = ma.array(data = sum_cmpl) f_sq = ma.array(data = sum_sq) # Write into MTZ mtz_dataset = f_sum.as_mtz_dataset(column_root_label = "FSUM") mtz_dataset.add_miller_array( miller_array = f_sq, column_root_label = "FSQ") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "result.mtz") if __name__ == '__main__': run() **** Pavel On 9/12/25 12:36, James Holton wrote:

...

Thank you for this Pavel,

Funny, I got the attachment just fine? Maybe its on P's email client side. I'm using thunderbird.

However, the code you shared seems to be taking an amplitude from one place and attaching it to a phase from another. What I want to do is the complex sum of two phased structure factors.

Specifically, I have is a stack of 1000 mtz files with FMODEL PHIMODEL in each. What I want is the phased sum of all of them, as well as the unphased sum of their squares. I've been doing the additions two at a time because that is parallelizable. My script is here: https://github.com/jmholton/altloc_diffuse/blob/main/addup_mtzs_diffuse.com It is currently using sftools for these operations: calc ( COL Fsum PHIsum ) = ( COL F1 P1 ) ( COL F2 P2 ) + calc COL I1 = COL F1 2 ** calc COL I2 = COL F2 2 ** calc J COL Isum = COL I1 COL I2 +

There does not seem to be a general-purpose math toolkit like sftools in phenix?

Much appreciate all the quick responses!

-James

On 9/12/2025 12:12 PM, Pavel Afonine wrote:

...
Oh, does phenixbb really strip off attachments under 1 KB in size?! Terrible. Anyway, I’m inlining it below (make sure your email client doesn’t mess up the indentation):

******** from iotbx import reflection_file_reader import iotbx.pdb

def run(): # Read your reflection data miller_arrays = reflection_file_reader.any_reflection_file(file_name = "1yjp.mtz").as_miller_arrays() for ma in miller_arrays: print (ma.info().label_string()) if(ma.info().label_string()=="FOBS,SIGFOBS"): f_obs = ma # Get phases by computing Fcalc (for example) pdb_inp = iotbx.pdb.input(file_name = "1yjp.pdb") xrs = pdb_inp.xray_structure_simple() f_calc = f_obs.structure_factors_from_scatterers(xray_structure=xrs).f_calc() # Transfer phased from f_calc to f_obs f_obs_cmpl = f_obs.phase_transfer(phase_source = f_calc.phases()) print(f_obs_cmpl.data()) # just to see the data are now complex array! # Write into MTZ mtz_dataset = f_calc.as_mtz_dataset(column_root_label = "Fcmpl") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "data.mtz")

if __name__ == '__main__': run()

********

Pavel

On 9/12/25 12:04, Petrus Zwart wrote:

...
Hi Pavel,

You say 25 lines, but there are zero - can you attach the code?

P

On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine wrote:

Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote: > Greetings Phenix developers! > > I have what I hope is a quick question. I'm trying to add a stack of > phased structure factors together. Specifically from separate runs of > phenix.fmodel. Normally, I'd use sftools for this, but the > distributed version has a size limitation. According to Tom's phenix > chatbot, I can use phenix.maps for adding two sets of structure > factors together, but it seems to only want to combine an mtz with a > pdb. There is iotbx.reflection_file_editor, but it only works from the > GUI (and I want to add 1000 mtz files together). phenix.xmanip looks > promising too, but again, seems to want a pdb and an mtz. Is there a > recommended way to do this? > > I've already asked this on CCP4BB, but I wanted to get the Phenix take > on these kinds of manipulations. > > Thank you, > > -James Holton > MAD Scientist > _______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

-- ------------------------------------------------------------------------------------------ Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246 ------------------------------------------------------------------------------------------

Billy Poon

4:19 p.m.

Hi Pavel, Yes, we generally filter out attachments, but if you use a .txt extension, it should go through. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org On Fri, Sep 12, 2025 at 1:23 PM Pavel Afonine wrote:

...

Hi James,

script pasted below should do it. This time it is 54 lines mostly because I was assuming arrays you are summing do not have matching indices.

**** from iotbx import reflection_file_reader import iotbx.pdb import os from scitbx.array_family import flex

def run(): # Collect all data arrays from input files arrays = [] for f in os.listdir("."): if not (f.startswith("fmodel") and f.endswith(".mtz")): continue miller_arrays = reflection_file_reader.any_reflection_file(file_name = f).as_miller_arrays() for ma in miller_arrays: #print (ma.info().label_string()) if(ma.info().label_string()=="FMODEL,PHIFMODEL"): arrays.append(ma) break # Assume arrays may not have matching indices (e.g., different resolution), # so make them all to have matching indices for ma in arrays: common_ref_array = arrays[0] aligned_arrays = [common_ref_array] new_arrays = [] for i in range(1, len(arrays)): next_array = arrays[i] common_ref_array, next_array_aligned = common_ref_array.common_sets(next_array) aligned_arrays[0] = common_ref_array aligned_arrays.append(next_array_aligned) new_arrays = aligned_arrays # sum_cmpl = None sum_sq = None for ma in new_arrays: data = ma.data() data_abs = flex.abs(data) data_abs_sq = data_abs * data_abs if sum_cmpl is None: sum_cmpl = data sum_sq = data_abs_sq else: sum_cmpl = sum_cmpl + data sum_sq = sum_sq + data_abs_sq f_sum = ma.array(data = sum_cmpl) f_sq = ma.array(data = sum_sq) # Write into MTZ mtz_dataset = f_sum.as_mtz_dataset(column_root_label = "FSUM") mtz_dataset.add_miller_array( miller_array = f_sq, column_root_label = "FSQ") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "result.mtz")

if __name__ == '__main__': run()

****

Pavel

On 9/12/25 12:36, James Holton wrote:

...
Thank you for this Pavel,

Funny, I got the attachment just fine? Maybe its on P's email client side. I'm using thunderbird.

However, the code you shared seems to be taking an amplitude from one place and attaching it to a phase from another. What I want to do is the complex sum of two phased structure factors.

Specifically, I have is a stack of 1000 mtz files with FMODEL PHIMODEL in each. What I want is the phased sum of all of them, as well as the unphased sum of their squares. I've been doing the additions two at a time because that is parallelizable. My script is here:

https://github.com/jmholton/altloc_diffuse/blob/main/addup_mtzs_diffuse.com

...
It is currently using sftools for these operations: calc ( COL Fsum PHIsum ) = ( COL F1 P1 ) ( COL F2 P2 ) + calc COL I1 = COL F1 2 ** calc COL I2 = COL F2 2 ** calc J COL Isum = COL I1 COL I2 +

There does not seem to be a general-purpose math toolkit like sftools in phenix?

Much appreciate all the quick responses!

-James

On 9/12/2025 12:12 PM, Pavel Afonine wrote:

...
Oh, does phenixbb really strip off attachments under 1 KB in size?! Terrible. Anyway, I’m inlining it below (make sure your email client doesn’t mess up the indentation):

******** from iotbx import reflection_file_reader import iotbx.pdb

def run(): # Read your reflection data miller_arrays = reflection_file_reader.any_reflection_file(file_name = "1yjp.mtz").as_miller_arrays() for ma in miller_arrays: print (ma.info().label_string()) if(ma.info().label_string()=="FOBS,SIGFOBS"): f_obs = ma # Get phases by computing Fcalc (for example) pdb_inp = iotbx.pdb.input(file_name = "1yjp.pdb") xrs = pdb_inp.xray_structure_simple() f_calc = f_obs.structure_factors_from_scatterers(xray_structure=xrs).f_calc() # Transfer phased from f_calc to f_obs f_obs_cmpl = f_obs.phase_transfer(phase_source = f_calc.phases()) print(f_obs_cmpl.data()) # just to see the data are now complex array! # Write into MTZ mtz_dataset = f_calc.as_mtz_dataset(column_root_label = "Fcmpl") mtz_object = mtz_dataset.mtz_object() mtz_object.write(file_name = "data.mtz")

if __name__ == '__main__': run()

********

Pavel

On 9/12/25 12:04, Petrus Zwart wrote:

...
Hi Pavel,

You say 25 lines, but there are zero - can you attach the code?

P

On Fri, Sep 12, 2025 at 11:57 AM Pavel Afonine

wrote:

...
...
Hi James,

attached script shows how to do that in cctbx. ( And it is just 25 lines including comments compared Claud's 173 )

It uses 1yjp as a source of sample files to run the example, you can get it using

phenix.fetch_pdb 1yjp action=all convert_to_mtz=true

Pavel

On 9/12/25 11:28, James Holton wrote: > Greetings Phenix developers! > > I have what I hope is a quick question. I'm trying to add a stack of > phased structure factors together. Specifically from separate runs of > phenix.fmodel. Normally, I'd use sftools for this, but the > distributed version has a size limitation. According to Tom's phenix > chatbot, I can use phenix.maps for adding two sets of structure > factors together, but it seems to only want to combine an mtz with a > pdb. There is iotbx.reflection_file_editor, but it only works from the > GUI (and I want to add 1000 mtz files together). phenix.xmanip looks > promising too, but again, seems to want a pdb and an mtz. Is there a > recommended way to do this? > > I've already asked this on CCP4BB, but I wanted to get the Phenix take > on these kinds of manipulations. > > Thank you, > > -James Holton > MAD Scientist > _______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

--

...
...
...
Peter Zwart Staff Scientist, Molecular Biophysics and Integrated Bioimaging Berkeley Synchrotron Infrared Structural Biology Biosciences Lead, Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246

...
...
...
_______________________________________________ phenixbb mailing list -- [email protected] To unsubscribe send an email to [email protected] Unsubscribe: phenixbb-leave@%(host_name)s

Age (days ago)

Last active (days ago)

List overview

Download

10 comments

5 participants

participants (5)

Billy Poon
James Holton
Pavel Afonine
Petrus Zwart
Tom Terwilliger

adding together

tags

participants (5)