[phenixbb] iotbx.merging_statistics on Python 3: too memory consuming

[Martin Malý]

Dear PHENIX & CCTBX developers and users,

I tried to calculate merging statistics with CCTBX tools using Python 3.
I realized that the memory requirements are much higher comparing with
Python 2... I closed all programs firstly so I had just 1.6 GB RAM used
of 7.6 GB total RAM. Then I ran these two lines of code in the
cctbx.python shell:

import iotbx.merging_statistics
i_obs = iotbx.merging_statistics.select_data(file_name="XDS_ASCII.HKL",
data_labels=None)

Python 2.7: The RAM usage went to 4.8 GB and then I was able to
calculate merging statistics.
Python 3.7: The module import was successful. Then the RAM usage went to
total 7.6 GB and then the process was killed by the operating system
(CentOS 7).

Please, do you have any suggestion how to use this module "more
carefully" and save memory?
Thank you!
Best regards,
Martin Malý
-----

Upozornění: Není-li v této zprávě výslovně uvedeno jinak, má tato e-mailová zpráva nebo její přílohy pouze informativní charakter. Tato zpráva ani její přílohy v žádném ohledu Biotechnologický ústav AV ČR, v. v. i. k ničemu nezavazují. Text této zprávy nebo jejích příloh není návrhem na uzavření smlouvy, ani přijetím případného návrhu na uzavření smlouvy, ani jiným právním jednáním směřujícím k uzavření jakékoliv smlouvy a nezakládá předsmluvní odpovědnost Biotechnologického ústavu AV ČR, v. v. i.

Disclaimer: If not expressly stated otherwise, this e-mail message (including any attached files) is intended purely for informational purposes and does not represent a binding agreement on the part of Institute of Biotechnology of the Czech Academy of Sciences. The text of this message and its attachments cannot be considered as a proposal to conclude a contract, nor the acceptance of a proposal to conclude a contract, nor any other legal act leading to concluding any contract, nor does it create any pre-contractual liability on the part of Institute of Biotechnology of the Czech Academy of Sciences.