- cctbxbb - phenix-online.org

Fwd: [git/cctbx] master: Python 3 syntax changes (790d9e315)
by Graeme.Winter＠Diamond.ac.uk 21 Sep '18

21 Sep '18

Folks, I expressed some surprise at this & friends however - - this is necessary to make the bootstrap process Python 3 compatible - the timetable for which said this should have been finished 9 months ago (https://github.com/cctbx/cctbx_project/wiki/Python-3-roadmap) - these changes were discussed and agreed (https://github.com/cctbx/cctbx_project/pull/199) The only part of this which was not approved was reverted before the request was merged (I felt that this annotation was helpful as the changes themselves go back ~6 months so people have possibly forgotten the context) Best wishes Graeme Begin forwarded message: From: CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM<mailto:[email protected]>> Subject: [git/cctbx] master: Python 3 syntax changes (790d9e315) Date: 21 September 2018 at 14:25:29 BST To: <CCTBX-COMMIT(a)JISCMAIL.AC.UK<mailto:[email protected]>> Reply-To: <cctbxbb(a)PHENIX-ONLINE.ORG<mailto:[email protected]>> Repository : ssh://ws133.diamond.ac.uk/cctbx On branch : master ________________________________ commit 790d9e315a2a6ae5a875ba86b461ed570c77280d Author: Markus Gerstel <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> Date: Thu May 24 12:16:38 2018 +0100 Python 3 syntax changes Replace the reduce(add, ...) function by sum(). reduce() is no longer in the global namespace in python 3. ________________________________ 790d9e315a2a6ae5a875ba86b461ed570c77280d scitbx/math/__init__.py | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/scitbx/math/__init__.py b/scitbx/math/__init__.py index a13405f92..49e295eb0 100644 --- a/scitbx/math/__init__.py +++ b/scitbx/math/__init__.py @@ -1,11 +1,13 @@ -from __future__ import division, absolute_import +from __future__ import absolute_import, division, print_function + +import math + from scitbx.math.ext import * import scitbx.linalg.eigensystem import scitbx.math.gaussian # implicit import from scitbx import matrix from scitbx.array_family import flex from boost import rational # implicit import -import math gaussian_fit_1d_analytical.__doc__ = """ Fits a gaussian function to a list of points. @@ -88,16 +90,14 @@ def r3_rotation_average_rotation_matrix_from_matrices(*matrices): """ if not matrices: - raise TypeError, "Average of empty sequence" + raise TypeError("Average of empty sequence") - import operator import scitbx.matrix - average = reduce( - operator.add, - [ scitbx.matrix.col( r3_rotation_matrix_as_unit_quaternion( e ) ) - for e in matrices ] - ) + average = sum( + scitbx.matrix.col( r3_rotation_matrix_as_unit_quaternion( e ) ) + for e in matrices + ) return r3_rotation_unit_quaternion_as_matrix( average.normalize() ) @@ -111,13 +111,12 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver """ if not matrices: - raise TypeError, "Average of empty sequence" + raise TypeError("Average of empty sequence") if maxiter < 0 or convergence <= 0: - raise ValueError, "Invalid iteration parameters" + raise ValueError("Invalid iteration parameters") from scitbx.math import so3_lie_algebra - import operator conv_sq = convergence * convergence norm = 1.0 / len( matrices ) @@ -125,10 +124,9 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver for i in range( maxiter ): inverse = current.inverse() - log_diff = norm * reduce( - operator.add, - [ so3_lie_algebra.element.from_rotation_matrix( matrix = inverse * m ) - for m in matrices ] + log_diff = norm * sum( + so3_lie_algebra.element.from_rotation_matrix( matrix = inverse * m ) + for m in matrices ) if log_diff.norm_sq() < conv_sq: @@ -136,7 +134,7 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver current *= log_diff.exponential() - raise RuntimeError, "Iteration limit exceeded" + raise RuntimeError("Iteration limit exceeded") def euler_angles_as_matrix(angles, deg=False): if (deg): @@ -174,7 +172,7 @@ class erf_verification(object): if (self.max_delta < delta): self.max_delta = delta if (delta > self.tolerance): - print x, expected_result, result, delta + print(x, expected_result, result, delta) class minimum_covering_sphere_nd(object): ________________________________ To unsubscribe from the CCTBX-COMMIT list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1 -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Re: [cctbxbb] [git/cctbx] master: Python 3 syntax changes (790d9e315)
by Graeme Winter 21 Sep '18

21 Sep '18

Folks, I expressed some surprise at this & friends however - - this is necessary to make the bootstrap process Python 3 compatible - the timetable for which said this should have been finished 9 months ago (https://github.com/cctbx/cctbx_project/wiki/Python-3-roadmap <https://github.com/cctbx/cctbx_project/wiki/Python-3-roadmap>) - these changes were discussed and agreed (https://github.com/cctbx/cctbx_project/pull/199 <https://github.com/cctbx/cctbx_project/pull/199>) The only part of this which was not approved was reverted before the request was merged (I felt that this annotation was helpful as the changes themselves go back ~6 months so people have possibly forgotten the context) Best wishes Graeme > On 21 Sep 2018, at 14:25, CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM> wrote: > > Repository : ssh://ws133.diamond.ac.uk/cctbx > On branch : master > > > > commit 790d9e315a2a6ae5a875ba86b461ed570c77280d > Author: Markus Gerstel <markus.gerstel(a)diamond.ac.uk> > Date: Thu May 24 12:16:38 2018 +0100 > > Python 3 syntax changes > > Replace the reduce(add, ...) function by sum(). > reduce() is no longer in the global namespace in python 3. > > > > > 790d9e315a2a6ae5a875ba86b461ed570c77280d > scitbx/math/__init__.py | 34 ++++++++++++++++------------------ > 1 file changed, 16 insertions(+), 18 deletions(-) > > diff --git a/scitbx/math/__init__.py b/scitbx/math/__init__.py > index a13405f92..49e295eb0 100644 > --- a/scitbx/math/__init__.py > +++ b/scitbx/math/__init__.py > @@ -1,11 +1,13 @@ > -from __future__ import division, absolute_import > +from __future__ import absolute_import, division, print_function > + > +import math > + > from scitbx.math.ext import * > import scitbx.linalg.eigensystem > import scitbx.math.gaussian # implicit import > from scitbx import matrix > from scitbx.array_family import flex > from boost import rational # implicit import > -import math > > gaussian_fit_1d_analytical.__doc__ = """ > Fits a gaussian function to a list of points. > @@ -88,16 +90,14 @@ def r3_rotation_average_rotation_matrix_from_matrices(*matrices): > """ > > if not matrices: > - raise TypeError, "Average of empty sequence" > + raise TypeError("Average of empty sequence") > > - import operator > import scitbx.matrix > > - average = reduce( > - operator.add, > - [ scitbx.matrix.col( r3_rotation_matrix_as_unit_quaternion( e ) ) > - for e in matrices ] > - ) > + average = sum( > + scitbx.matrix.col( r3_rotation_matrix_as_unit_quaternion( e ) ) > + for e in matrices > + ) > > return r3_rotation_unit_quaternion_as_matrix( average.normalize() ) > > @@ -111,13 +111,12 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver > """ > > if not matrices: > - raise TypeError, "Average of empty sequence" > + raise TypeError("Average of empty sequence") > > if maxiter < 0 or convergence <= 0: > - raise ValueError, "Invalid iteration parameters" > + raise ValueError("Invalid iteration parameters") > > from scitbx.math import so3_lie_algebra > - import operator > > conv_sq = convergence * convergence > norm = 1.0 / len( matrices ) > @@ -125,10 +124,9 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver > > for i in range( maxiter ): > inverse = current.inverse() > - log_diff = norm * reduce( > - operator.add, > - [ so3_lie_algebra.element.from_rotation_matrix( matrix = inverse * m ) > - for m in matrices ] > + log_diff = norm * sum( > + so3_lie_algebra.element.from_rotation_matrix( matrix = inverse * m ) > + for m in matrices > ) > > if log_diff.norm_sq() < conv_sq: > @@ -136,7 +134,7 @@ def r3_rotation_average_rotation_via_lie_algebra(matrices, maxiter = 100, conver > > current *= log_diff.exponential() > > - raise RuntimeError, "Iteration limit exceeded" > + raise RuntimeError("Iteration limit exceeded") > > def euler_angles_as_matrix(angles, deg=False): > if (deg): > @@ -174,7 +172,7 @@ class erf_verification(object): > if (self.max_delta < delta): > self.max_delta = delta > if (delta > self.tolerance): > - print x, expected_result, result, delta > + print(x, expected_result, result, delta) > > class minimum_covering_sphere_nd(object): > > > To unsubscribe from the CCTBX-COMMIT list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1 <https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1>

1 0

Post-PHENIX changes
by Graeme.Winter＠Diamond.ac.uk 21 Sep '18

21 Sep '18

Morning All, I see from http://phenix-online.org/ That PHENIX 1.14 has been released Now, I see at https://github.com/cctbx/cctbx_project/pulls we have a few fairly hefty pull requests parked as “hold for phenix release” - I would certainly not suggest merging all these in one go however probably we should give some thought to stepping through some of them, deciding which ones we should look at first and starting to merge things? I for one would like Nick’s allocator code to come back into play (so we can make use of it for DIALS) Thoughts / opinions? For info: PHENIX folks did you tag the version which was released in PHENIX 1.14 to allow a release branch for .1 etc. releases? Many thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 1

repr for flex arrays
by Graeme.Winter＠Diamond.ac.uk 14 Sep '18

14 Sep '18

Hi Folks, Is there a reason for not having a helpful __repr__ for flex arrays i.e. Grey-Area:002-decide-when graeme$ dials.python Python 2.7.14 (default, Apr 17 2018, 15:02:10) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from dials.array_family import flex >>> a = flex.double([1, 2, 3]) >>> print a <scitbx_array_family_flex_ext.double object at 0x1095673c0> (I have a feeling I asked this before but cannot find reference) For example >>> print a.as_numpy_array() [ 1. 2. 3.] is more useful… but annoying to type I appreciate a could be “big” but numpy seems to handle this OK even - could __repr__ be punned to as_numpy_array(),__repr__()? Currently it prints the type of the array and its location in ram, which are adequately served by >>> type(a) <class 'scitbx_array_family_flex_ext.double'> >>> print id(a) 4451627968 If I print an array I more often want to see the content and less often the storage class and location (I humbly assert) Thoughts? I know this tends to confuse and annoy people who are new to flex arrays (and veterans who are busy debugging stuff…) Many thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 9

Python 3 status
by Graeme.Winter＠Diamond.ac.uk 06 Sep '18

06 Sep '18

Hi Folks, Are there any updates on the progress to Python 3 compatibility? I know there is a project here https://github.com/cctbx/cctbx_project/projects/2 but I have lost the thread of activities since I have been out of the office much of the summer. At the very least I know no progress has been made from this side of the pond as we’ve put this work down for now. i would guess that a lot is on hold while the PHENIX release is done - however I thought I should check in case I have missed anything? Part of why I ask is that there is a PHENIX meeting in Cambridge real soon now (mid September) and I wonder if it is worth folks from DIALS and PHENIX getting together at that time to discuss this? Any thoughts? Thanks & best wishes Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

7 12

Re: [cctbxbb] [git/cctbx] master: re-defining the shift/est convergence limit in terms of crystallographic rather than independent parameters to make it easier to relate to the convergence reported in the CIF; also caching the shifts for further analysis and reporting; (9339fd5f7)
by Nicholas Sauter 05 Sep '18

05 Sep '18

I'm worried about making changes like this to a fundamental toolbox component like normal equations solving. We use this for processing xfel data, and would like the opportunity to test any changes through a pull request, prior to having them committed to the main branch. Nick Sauter Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, Sep 5, 2018 at 5:29 AM, CCTBX commit < diamondlightsource.jenkins(a)gmail.com> wrote: > Repository : ssh://g18-sc-serv-04.diamond.ac.uk/cctbx > On branch : master > > ------------------------------ > > > commit 9339fd5f7d240f0d6f85dd57ce42e796d19608f1 > Author: pcxod <oleg_dolomanov(a)hotmail.com> > Date: Wed Sep 5 13:29:29 2018 +0100 > > re-defining the shift/est convergence limit in terms of > crystallographic rather than independent parameters to make it easier to > relate to the convergence reported in the CIF; also caching the shifts for > further analysis and reporting; > > > ------------------------------ > > > 9339fd5f7d240f0d6f85dd57ce42e796d19608f1 > scitbx/lstbx/normal_eqns_solving.py | 17 +++++++++++------ > 1 file changed, 11 insertions(+), 6 deletions(-) > > diff --git a/scitbx/lstbx/normal_eqns_solving.py > b/scitbx/lstbx/normal_eqns_solving.py > index 660304eab..45f070622 100644 > --- a/scitbx/lstbx/normal_eqns_solving.py > +++ b/scitbx/lstbx/normal_eqns_solving.py > @@ -140,15 +140,20 @@ class iterations(object): > a = self.non_linear_ls.normal_matrix_packed_u() > a.matrix_packed_u_diagonal_add_in_place(value*a. > matrix_packed_u_diagonal()) > > - def do_scale_shifts(self, max_shift_over_esd): > + def do_scale_shifts(self, limit_shift_over_su): > x = self.non_linear_ls.step() > esd = self.non_linear_ls.covariance_matrix().matrix_packed_u_ > diagonal() > - x_over_esd = flex.abs(x/flex.sqrt(esd)) > - max_val = flex.max(x_over_esd) > - if max_val < self.convergence_as_shift_over_esd: > + ls_shifts_over_su = flex.abs(x/flex.sqrt(esd)) > + #max shift for the LS > + self.max_ls_shift_over_su = flex.max(ls_shifts_over_su) > + jac_tr = self.non_linear_ls.actual.\ > + reparametrisation.jacobian_transpose_matching_grad_fc() > + self.shifts_over_su = jac_tr.transpose() * ls_shifts_over_su > + self.max_shift_over_su = flex.max(self.shifts_over_su) > + if self.max_shift_over_su < self.convergence_as_shift_over_esd: > return True > - if max_val > max_shift_over_esd: > - shift_scale = max_shift_over_esd/max_val > + if self.max_ls_shift_over_su > limit_shift_over_su: > + shift_scale = limit_shift_over_su/self.max_ls_shift_over_su > x *= shift_scale > return False > > > ------------------------------ > > To unsubscribe from the CCTBX-COMMIT list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1 >

2 3

Adding more parameters to xray::scatterer for anharmonic thermal displacements
by Luc J. Bourhis 22 Aug '18

22 Aug '18

Hi, we are working with Michal Chodkiewicz (Warsaw University) to add anharmonic thermal tensors to smtbx refinement. I was thinking to add one pointer to xray::scatterer, which can be NULL. When it is not so, it would point to an object which can compute the value and the derivatives for that thermal factor. That means we would have a base class with virtual methods, but I would say that is our problem! The issue for the rest of the cctbx community is the extra storage of that pointer in that class. Would that be all right? I remember quite a few years ago Ralf saying he wanted to replace the ADP's by a pointer to an external storage to save space. Our idea is along the same line. Best wishes, Luc

3 2

model selection
by Pavel Afonine 16 Aug '18

16 Aug '18

Currently model class has three selection methods: def selection(self, selstr, optional=True): def select(self, selection): def iselection(self, selstr): which is hideously confusing with zero chance to remember, and also inconsistent with the rest. Most objects in cctbx have ".select(self, selection)" methods, while being inconsistent about what they expect: iselection or boolean selection as "selection" argument. Some smart ones though can take both int or bool selections (figure out type automatically, which is trivial). Can we consolidate all three into one ".select()" that would allow taking bool or int or str? Or you think there are philosophical arguments against it? Pavel On 8/16/18 15:00, CCTBX commit wrote: > Repository : ssh://g18-sc-serv-04.diamond.ac.uk/cctbx > On branch : master > > ------------------------------------------------------------------------ > > > commit c98b44997e8eb7bf53ee828f510b933cd5167758 > Author: Oleg Sobolev <osobolev(a)lbl.gov> > Date: Thu Aug 16 15:00:20 2018 -0700 > > Proper use of model class > > > ------------------------------------------------------------------------ > > > c98b44997e8eb7bf53ee828f510b933cd5167758 > mmtbx/model/model.py | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/mmtbx/model/model.py b/mmtbx/model/model.py > index 6f2247452..87412f95c 100644 > --- a/mmtbx/model/model.py > +++ b/mmtbx/model/model.py > @@ -2312,8 +2312,7 @@ class manager(object): > sizes = flex.int() > h = self.get_hierarchy() > if(macro_molecule_only): > - asc = h.atom_selection_cache() > - s = asc.selection("protein or nucleotide") > + s = self.selection("protein or nucleotide") > h = h.select(s) > for r in h.residue_groups(): > sizes.append(r.atoms().size())

2 3

Extracting data from MTZ
by David Waterman 14 Aug '18

14 Aug '18

Hi all, I have a script that extracts Fobs and Fcalc from an MTZ file produced by Refmac for some analysis. I was just doing this in a trivial way: fobs = m.extract_reals("F").data fc = m.extract_reals("FC_ALL").data and this worked fine, until I came across a file with missing observations, so that the lengths of fobs and fc do not match. Can anybody advise a better way to do this please? I'm inexperienced with iotbx.mtz. I don't need anything other than the values of the structure factors, but I need each element of fobs and fc to contain the equivalent values. Thanks -- David

3 3

How does Jenkins work?
by Takanori Nakane 28 Jul '18

28 Jul '18

Hi, Why does Jenkins still run tests on my old commits on my deleted (dxtbx-sacla) branch? The test failure is probably caused by my commit dc17165 but I committed a fix 5c7416b and confirmed that it passes the tests. So I merged my topic branch to master and deleted the branch from GitHub. I know how git works but I am new to the practice in the cctbx repository (like using rebase instead of merge). If I am doing something wrong, could you please point it to me? Best regards, Takanori Nakane > <html><head></head><body><table style="width:100%; background-color: > #F59400; color: #FFF; border-radius: 0.5em; margin-bottom: 0.5em"><tr><td > style="margin: 0.5em 1.25em 0.5em 0.7em; font-weight: bold; font-family: > sans-serif; font-size: 110%"><a > href="http://jenkins.diamond.ac.uk:8080/job/cctbx_branch_tests/145/" > style="color:#FFF; text-decoration: none"><big>Build CCTBX Branch Tests is > unstable</big></a></td></tr></table><br/><table style="width:100%; > background-color: #F59400; color: #FFF; border-radius: 0.5em; > margin-bottom: 0.5em"><tr><td style="margin: 0.5em 1.25em 0.5em 0.7em; > font-weight: bold; font-family: sans-serif; font-size: 110%"><a > href="http://jenkins.diamond.ac.uk:8080/job/cctbx_branch_tests/145/testReport" > style="color:#FFF; text-decoration: none">3 out of 940 tests failed (132 > skipped)</a></td></tr></table><table style="margin: 0.5em 0 1em > 2em"><tr><td><pre style="font-size:80%">3 tests failed. > FAILED: > dxtbx.build_dials.modules.cctbx_project.dxtbx.tests.test_regression_images.test_no_multiple_format_understanding[SACLA_MPCCD_Cheetah/run266702-0-subset.h5] > Error Message: > exit code 1 > Stack Trace: > FAILED: > dxtbx.build_dials.modules.cctbx_project.dxtbx.tests.test_regression_images.test_no_multiple_format_understanding[putative_imgCIF_HDF5_mapping/minicbf.h5] > Error Message: > exit code 1 > Stack Trace: > FAILED: > dxtbx.build_dials.modules.cctbx_project.dxtbx.tests.test_regression_images.test_no_multiple_format_understanding[putative_imgCIF_HDF5_mapping/X4_wide_M1S4_1.nxs] > Error Message: > exit code 1 > Stack Trace: > </pre></td></tr></table><table style="width:100%; background-color: > #F59400; color: #FFF; border-radius: 0.5em; margin-bottom: 0.5em"><tr><td > style="margin: 0.5em 1.25em 0.5em 0.7em; font-weight: bold; font-family: > sans-serif; font-size: 110%">Changes since the last successful > build</td></tr></table><table style="margin: 0.5em 0 1em > 2em"><tr><td><table style="padding: 0.1em 0.4em; background-color: > #F59400; color:#fff"><tr><td>145</td></tr></table></td><td>No changes > </td></tr></table><table style="width:100%; background-color: #CCC; color: > #000; border-radius: 0.5em; margin-bottom: 0.5em"><tr><td style="margin: > 0.5em 1.25em 0.5em 0.7em; font-weight: bold; font-family: sans-serif; > font-size: 110%">Build environment</td></tr></table><table style="margin: > 0.5em 0 1em 2em"><tr><td><span > style="font-size:130%"> [ </span>cctbx <strong>71a4ab94a > 2018-07-25 06:47:08 +0900</strong><span > style="font-size:130%"> | </span>DIALS regression > <strong>1924</strong>, PHENIX regression <strong>4419</strong>, DLS > Jenkins scripts <strong>8de8b58 2018-07-04 23:29:22 +0100</strong><span > style="font-size:130%"> ] </span></td></tr></table><span > style="color:#CCC;font-size:80%">mail template revision > 2487</span></body></html> > <br> > <hr> > <p align="center">To unsubscribe from the CCTBX-COMMIT list, click the > following link:<br> > <a > href="https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1" > target="_blank">https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1</a> > </p>

4 5

mmtbx wilson_scaling in a small molecule context
by ben.williams＠diamond.ac.uk 23 Jul '18

23 Jul '18

Hello any and all, I was looking to make use of the functionality of wilson_scaling (Wilson plot, etc.) from mmtbx.scaling.data_statistics, in the context of small molecule data. I'm new to mmtbx, so here are two (groups of) questions for the enlightened: 1. Designed, as it is, with MX in mind, wilson_scaling's __init__ demands as an argument the number of protein residues, in order to form an estimate of the contents of the asymmetric unit. If this class were to be used in a small molecule context, what would be a sensible value to use? Would spoofing n_residues with an arbitrary small number (1?) be a robust bodge? Is it ever likely to howl? 2. Being crystals of small molecules, the samples in question may have inversion centres, mirror planes. Having been designed as a tool for MX, will wilson_scaling throw a wobbly when it sees my data? Many thanks, Ben Williams Diamond Light Source -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

subset of reflections from mtz object
by Robert Oeffner 18 Jul '18

18 Jul '18

Hi, I'm not as well versed using iotbx.mtz as I ought to be. I'd like to make a subset of reflections from an mtz file bounded by a chosen dmin and dmax, i.e. a resolution shell of reflections. What is the best way of doing this in python? Many thanks, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234

2 1

Citation for CCTBX
by Thomas, Jens 17 Jul '18

17 Jul '18

Hi, I'm updating the references printed by AMPLE and would like to cite CCTBX. Is there a recommended citation or way to acknowledge the use of CCTBX in a program? Best wishes, Jens

2 2

cctbx debug build windows
by Pascal 10 Jul '18

10 Jul '18

Hi, How do you build a debug version of cctbx on windows? After getting an environement working with bootstrap, I reconfigured with configure.py --build=debug and it is now failing when linking against python27_d.lib Any advice? Pascal

3 2

Deprecating legacy support (CentOS 5, Windows 32-bit) in CCTBX and Phenix
by Billy Poon 06 Jul '18

06 Jul '18

Hi everyone, The upcoming Phenix 1.14 official release will still have installers for CentOS 5 and Windows 32-bit, but since CentOS is no longer supporting CentOS 5, and RedHat is only supporting RHEL5 through their Extended Life Cycle Support Add-On (until November 30, 2020), we are planning on deprecating CentOS 5 support (32-bit and 64-bit). In addition, we are planning on deprecating 32-bit support for Windows because only a few percent (< 5% for both Windows and CentOS 5) of our Phenix downloads are for 32-bit versions. Please let us know if support for CentOS 5 (32-bit and/or 64-bit) or 32-bit Windows is a requirement for your systems. If so, also please describe your setup. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

2 4

Install OpenBLAS or MKL in base
by Luc Bourhis 22 Jun '18

22 Jun '18

Hi, years ago, I had wrote on this discussion list about a project of mine to dramatically accelerate scitbx.lstbx by using optimised BLAS libraries. That work had been dormant on a branch but now we are planning to move forward with the help of Pascal Parlois who will do the actual coding in the coming months. In order for this new code to be exercised by nightly tests, we, the smtbx people, need to write code so that that optimised BLAS library is installed during bootstrap. Right now, as a stopgap, I used conda to install either OpenBLAS and MKL but that won’t do with the cctbx philosophy. There are basically two choices: either OpenBLAS or MKL. When I started this project, I chose OpenBLAS because MKL was not freely redistributable, and I had got the green light to install it as needed. However now MKL has become completely free, thus becoming a possible choice. MKL is more performant on Intel but OpenBLAS is better on AMD (OpenBLAS actually runs on pretty much any processor out there but we don’t care in the context of cctbx): c.f. Julia people conclusions <https://discourse.julialang.org/t/openblas-is-faster-than-intel-mkl-on-amd-…> Now I realise that most of you may not care one way or another!?! But we won’t move forward before we got the green light… Note that I am taking about installing a BLAS usable from C++ here, not getting a BLAS-accelerated numpy. The latter could be an alternative though. But that would require to modify the bootstrap code to compile a MKL- or OpenBLAS-enabled numpy anyway. Best wishes, Luc J Bourhis

6 22

Fwd: buildbot failure in Phenix on cctbx-ci-ci
by Graeme.Winter＠Diamond.ac.uk 05 Jun '18

05 Jun '18

Hi Folks, Am I alone in finding this kind of message ** very ** annoying I did not touch this file I have no business touching this file If I do fix this then it may cause conflicts for others, which will no doubt annoy them far more than the odd extra space in the code However I am also now to blame for breaking the Phenix build, in the run up to a release This is no a happy place Suggestions? Thanks & cheerio Graeme Begin forwarded message: From: phenix(a)phenix-online.org<mailto:[email protected]> Subject: buildbot failure in Phenix on cctbx-ci-ci Date: 4 June 2018 at 12:11:19 BST To: graeme.winter(a)gmail.com<mailto:[email protected]> Cc: ASBrewster(a)LBL.Gov<mailto:[email protected]>, NWMoriarty(a)LBL.Gov<mailto:[email protected]> Build status: FAILURE Buildslave for this Build: ci Complete logs for all build steps: http://cci-vm-6.lbl.gov:8010/builders/cctbx-ci-ci/builds/5580 Build Reason: The SingleBranchScheduler scheduler named 'cctbx-ci.poll' triggered this build Build Source Stamp: clipper: [branch master] HEAD cbflib: [branch master] HEAD cctbx_project: [branch master] 4de5e7f82b0302632282f78c6dda8b999cf6abf6 annlib_adaptbx: [branch master] HEAD tntbx: [branch master] HEAD ccp4io_adaptbx: [branch master] HEAD boost: [branch master] HEAD gui_resources: [branch master] HEAD Blamelist: graeme-winter If you got an email and you didn't touch this file, it may because someone commited this file before you with an error and hasn't fixed it yet. You might have noticed the error if you ran libtbx.find_clutter on the entire tree. Detailed log of last build step: http://cci-vm-6.lbl.gov:8010/builders/cctbx-ci-ci/builds/5580/steps/test ALL clutter/logs/stdio<http://cci-vm-6.lbl.gov:8010/builders/cctbx-ci-ci/builds/5580/steps/test%20…> Last 80 lines of "test ALL clutter.stdio" iota/command_line/iota_single_image.py: tabs or trailing whitespace=2, #10, #11 *** To clean up please use: libtbx.clean_clutter *** -The Buildbot -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 5

Re: [cctbxbb] [git/cctbx] master: Generate libtbx.* wrappers from correct directory (ad736e2c8)
by Billy Poon 01 Jun '18

01 Jun '18

I don't think this is working on macOS. The configuration step is generating error messages like, Ignored: [Errno 2] No such file or directory: <command path> In lines 1640-1643, self.write_dispatcher( source_file=libtbx.env.under_base(os.path.join('bin', ep.name)), target_file=os.path.join('bin', 'libtbx.' + ep.name), ) the source_file argument is always being constructed as <base>/bin/<command> instead of using the directory from self.get_setuptools_script_dir(). Should it be the bin_directory variable instead (e.g. source_file=os.path.join(bin_directory, ep.name))? As a side note, conda will put everything in bin because there is no framework. For access to the window manager, you would call pythonw, which points to a python inside a .app bundle. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Fri, Jun 1, 2018 at 3:15 AM CCTBX commit < diamondlightsource.jenkins(a)gmail.com> wrote: > Repository : ssh://g18-sc-serv-04.diamond.ac.uk/cctbx > On branch : master > > ------------------------------ > > > commit ad736e2c8e65407615a7e6604b0544687ed8058c > Author: Markus Gerstel <markus.gerstel(a)diamond.ac.uk> > Date: Thu May 31 08:27:09 2018 +0100 > > Generate libtbx.* wrappers from correct directory > > Python console scripts don't necessarily land in base/bin, specifically > on MacOS this is not the case. Use setuptools etc. to find the actual > Python console script location and use that directory instead. Fixes > #177 > > Add a function to regenerate all Python console scripts which can be > called from an installer to relocate paths. > > > ------------------------------ > > > ad736e2c8e65407615a7e6604b0544687ed8058c > libtbx/env_config.py | 74 +++++++++++++++++++++++++----- > libtbx/fastentrypoints.py | 112 > ++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 176 insertions(+), 10 deletions(-) > > diff --git a/libtbx/env_config.py b/libtbx/env_config.py > index de5fc72dd..25daa2cb3 100644 > --- a/libtbx/env_config.py > +++ b/libtbx/env_config.py > @@ -1562,23 +1562,77 @@ selfx: > source_file=source_file, > target_file=module_name+"."+command) > > + def get_setuptools_script_dir(): > + ''' > + Find the location of python entry point console_scripts, ie. things > like > + 'pip', 'pytest', ... > + This is different from simple /base/bin, eg. on MacOS. > + > + > https://stackoverflow.com/questions/25066084/get-entry-point-script-file-lo… > + ''' > + from setuptools import Distribution > + from setuptools.command.install import install > + class OnlyGetScriptPath(install): > + def run(self): > + # does not call install.run() by design > + self.distribution.install_scripts = self.install_scripts > + dist = Distribution({'cmdclass': {'install': OnlyGetScriptPath}}) > + dist.dry_run = True # not sure if necessary, but to be safe > + dist.parse_config_files() > + command = dist.get_command_obj('install') > + command.ensure_finalized() > + command.run() > + return dist.install_scripts > + > + def regenerate_entry_point_console_scripts(self, verbose=True): > + ''' > + Creates all console_scripts entry point scripts from scratch and > overwrites existing ones. > + This is intended to be used by installers to relocate the entry point > script paths. > + ''' > + try: > + import distutils.dist > + import libtbx.fastentrypoints # monkeypatches setuptools > + import pkg_resources > + import setuptools.command.easy_install > + except ImportError: > + return > + > + # Prepare generic script generator > + distribution = distutils.dist.Distribution({'name': 'setuptools'}) > + command = setuptools.command.easy_install.easy_install(distribution) > + command.args = ['wheel'] # dummy argument > + command.finalize_options() > + > + # Force regeneration of all known console_scripts > + for pkg_resources_dist in pkg_resources.working_set: > + console_scripts = > pkg_resources_dist.get_entry_map().get('console_scripts') > + if console_scripts: > + if verbose: > + print("Regenerating commands for %s: %s" % ( > + pkg_resources_dist, > + list(console_scripts), > + )) > + command.install_wrapper_scripts(pkg_resources_dist) > + > def generate_entry_point_dispatchers(self): > - # Write indirect dispatcher scripts for all console_scripts entry > points > - # that have existing dispatcher scripts in the base/bin directory, but > - # add a 'libtbx.' prefix. > - base_bin_directory = libtbx.env.under_base('bin') > - if not os.path.isdir(base_bin_directory): > + ''' > + Write indirect dispatcher scripts for all console_scripts entry points > + that have existing dispatcher scripts in the base/bin directory, but > + add a 'libtbx.' prefix. > + ''' > + try: > + import pkg_resources > + bin_directory = get_setuptools_script_dir() > + except ImportError: > + return > + if not os.path.isdir(bin_directory): > return # do not create console_scripts dispatchers, only point to > them > > - base_bin_dispatchers = set(os.listdir(base_bin_directory)) > + base_bin_dispatchers = set(os.listdir(bin_directory)) > existing_dispatchers = filter(lambda f: f.startswith('libtbx.'), > self.bin_path.listdir()) > existing_dispatchers = set(map(lambda f: f[7:], existing_dispatchers)) > entry_point_candidates = base_bin_dispatchers - existing_dispatchers > > - try: > - import pkg_resources > - except ImportError: > - return > entry_points = pkg_resources.iter_entry_points('console_scripts') > entry_points = filter(lambda ep: ep.name in entry_point_candidates, > entry_points) > for ep in entry_points: > diff --git a/libtbx/fastentrypoints.py b/libtbx/fastentrypoints.py > new file mode 100644 > index 000000000..9707f74a3 > --- /dev/null > +++ b/libtbx/fastentrypoints.py > @@ -0,0 +1,112 @@ > +# noqa: D300,D400 > +# Copyright (c) 2016, Aaron Christianson > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions are > +# met: > +# > +# 1. Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# > +# 2. Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in the > +# documentation and/or other materials provided with the distribution. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS > +# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED > +# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A > +# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED > +# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR > +# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF > +# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING > +# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS > +# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > +''' > +Monkey patch setuptools to write faster console_scripts with this format: > + > + import sys > + from mymodule import entry_function > + sys.exit(entry_function()) > + > +This is better. > + > +(c) 2016, Aaron Christianson > +http://github.com/ninjaaron/fast-entry_points > +''' > +from setuptools.command import easy_install > +import re > +TEMPLATE = '''\ > +# -*- coding: utf-8 -*- > +# EASY-INSTALL-ENTRY-SCRIPT: '{3}','{4}','{5}' > +__requires__ = '{3}' > +import re > +import sys > + > +from {0} import {1} > + > +if __name__ == '__main__': > + sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) > + sys.exit({2}())''' > + > + > +@classmethod > +def get_args(cls, dist, header=None): # noqa: D205,D400 > + """ > + Yield write_script() argument tuples for a distribution's > + console_scripts and gui_scripts entry points. > + """ > + if header is None: > + # pylint: disable=E1101 > + header = cls.get_header() > + spec = str(dist.as_requirement()) > + for type_ in 'console', 'gui': > + group = type_ + '_scripts' > + for name, ep in dist.get_entry_map(group).items(): > + # ensure_safe_name > + if re.search(r'[\\/]', name): > + raise ValueError("Path separators not allowed in script > names") > + script_text = TEMPLATE.format( > + ep.module_name, ep.attrs[0], '.'.join(ep.attrs), > + spec, group, name) > + # pylint: disable=E1101 > + args = cls._get_script_args(type_, name, header, script_text) > + for res in args: > + yield res > + > + > +# pylint: disable=E1101 > +easy_install.ScriptWriter.get_args = get_args > + > + > +def main(): > + import os > + import re > + import shutil > + import sys > + dests = sys.argv[1:] or ['.'] > + filename = re.sub('\.pyc$', '.py', __file__) > + > + for dst in dests: > + shutil.copy(filename, dst) > + manifest_path = os.path.join(dst, 'MANIFEST.in') > + setup_path = os.path.join(dst, 'setup.py') > + > + # Insert the include statement to MANIFEST.in if not present > + with open(manifest_path, 'a+') as manifest: > + manifest.seek(0) > + manifest_content = manifest.read() > + if 'include fastentrypoints.py' not in manifest_content: > + manifest.write(('\n' if manifest_content else '') + > + 'include fastentrypoints.py') > + > + # Insert the import statement to setup.py if not present > + with open(setup_path, 'a+') as setup: > + setup.seek(0) > + setup_content = setup.read() > + if 'import fastentrypoints' not in setup_content: > + setup.seek(0) > + setup.truncate() > + setup.write('import fastentrypoints\n' + setup_content) > > ------------------------------ > > To unsubscribe from the CCTBX-COMMIT list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCTBX-COMMIT&A=1 >

1 0

show() not __str__ - why?
by Graeme.Winter＠Diamond.ac.uk 01 Jun '18

01 Jun '18

Folks, Came up in conversation - how come the motif for printing / providing object info is to call show() and pass a file handle rather than the more pythonic __str__? See e.g. https://github.com/dials/dials/blob/b499c6191caa18e04ddaf190321a3da6c14af55… I know in dials we could do it differently however we inherited the motif from cctbx & I am not sure why things work that way - figured there must be a reason... This is also under discussion under https://github.com/dials/dials/pull/567 but I thought it was worth making sure we knew the reasons why cctbx uses it's own way of doing something... (also it's quite annoying that printing things will give you say space_group object i.e. much like printing type(object), when you want to see it's contents - like printing j and being told it is an int!) Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Berkeley Server Downtown (5/29 - 5/31)
by Billy Poon 30 May '18

30 May '18

Dear all, On Tuesday, May 29, our servers will be shut down starting at 5:00 pm Pacific for scheduled maintenance in our building. We expect that the servers will be available by 10:00 am Pacific on Thursday (5/31) at the latest. This will affect, - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 1

Automated Travis CI for phenix
by Elliot Nelson 25 May '18

25 May '18

Hi I'm looking to run automated integration tests for software, which currently runs from CCTBX, but also calls phenix scripts. These are phenix.fmodel and phenix.maps. Currently I have a docker image which sources CCP4 to get access to CCTBX, but this will not allow me to test stuff reliant on the phenix scripts. As Phenix is not downloadable on command line (CCP4 build can be got through wget command), it is not obvious as the best way to setup up such a docker build on travis CI. I wouldn't want to source a local download of the software as i am unsure of the licesining implications of hosting a copy of the software. Any suggestions would be greatly appreciated? Software (with dockerfile): https://github.com/nelse003/exhaustive_search <https://github.com/nelse003/exhaustive_search>Current docker image: https://hub.docker.com/r/nelse003/ccp4_phenix_docker/ Thanks Elliot DPhil Student, Systems Approaches to Biomedical Science<http://www.sabsidc.ox.ac.uk/> Protein Crystallography<http://www.thesgc.org/groupprofile/9489>, Structural Genomics Consortium, NDM, University of Oxford Oxford Protein Informatics Group<http://opig.stats.ox.ac.uk/>, Department of Statistics, University of Oxford

3 3

mmtbx.f_model.manager.deep_copy() does not work correctly when twinned?
by Keitaro Yamashita 15 May '18

15 May '18

Dear CCTBX developers, I found phenix.maps did not recognize twinning. Clearly, "fmodel" in the following code should be "fmodel_". (underscore missing) https://github.com/cctbx/cctbx_project/blob/41df8498857c186a3f83351090683b3… However, this did not solve the problem. This is due to fmodel_.deep_copy() in line 1530. For example, fmodel_.info().show_rfactors_targets_scales_overall() fmodel_.deep_copy().info().show_rfactors_targets_scales_overall() These two lines show different stats in case of twinning (fmodel_ is mmtbx.twinning.twin_f_model.twin_model_manager object). So deep_copy() is not working correctly. I found that something was wrong with fmodel_ts1.deep_copy() here. https://github.com/cctbx/cctbx_project/blob/master/mmtbx/twinning/twin_f_mo… Does anybody know how to solve this copying issue? Best regards, Keitaro

2 3

mmtbx/examples/simulate_experimental_data.py
by Elliot Nelson 10 May '18

10 May '18

Hi All, I'm trying to generate some simulated mtz files. My main aim is to test a script which looks to determine ligand occupancy (https://github.com/nelse003/exhaustive_search/blob/master/exhaustive_search…) by looking at |Fo-Fc| over the ligand/ ground-state the ligand replaces. To test this I would like to generate mtz files from pdb models with a ligand occupancy that has varied between 0-1, which the above script would hopefully recover the occupancy. I have tried to use mmtbx/examples/simulate_experimental_data.py (https://github.com/cctbx/cctbx-playground/blob/8f23c190a9f8d2ed2b5845385f56… ) to generate an mtz file. So far I have been able to generate mtz files using the simulate_experimental_data.py, these have the columns: H K L FOBS(+) SIGFOBS(+) FOBS(-) SIGFOBS(-) I can use this outputted mtz for running tests of my scripts ,However as this has no PHI columns, I cannot display this in coot. I have tried to add the phi column from the refined mtz of the orignal data using CAD, but have not yet managed to do this succesfully. I feel that this may be an overcomplicated, and potentially superseded way of generating simulated mtz data. Any advice on alternatives would be really useful? Apologies if this is not very clear. Thanks Elliot Nelson

3 4

Jenkins testing on cctbx branches other than master
by markus.gerstel＠diamond.ac.uk 09 May '18

09 May '18

Hello everyone, As requested we now also run Jenkins tests on all branches that are not master. The current settings are as follows: (this is currently exactly the same we do for testing on the master branch) We test the following modules: libtbx, scitbx, cctbx, iotbx, rstbx, fable and dxtbx The following other modules are present and configured: dials_regression, phenix_regression The following modules are not configured: dials, xia2, xfel, iota, prime, simtbx Let me know if you want any changes here. Results will look like this: https://github.com/cctbx/cctbx_project/commits/testbranch If you hover over the green tick mark (or the red cross) of the top 3 commits you see that these are results from cctbx-branch testing. We do generate emails for failed tests and failed builds as usual, but there will be no 'this build is now fixed'/'back to normal' emails. -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

Re: [cctbxbb] [Dials-support] Issue with hierarchy.as_pdb_string
by Graeme.Winter＠Diamond.ac.uk 28 Apr '18

28 Apr '18

Morning all, Mike came to the DIALS list with some pdb-reading related issues - however in DIALS land we usually don’t mess with this kind of stuff - is there someone who can help him out here please? Many thanks Graeme On 27 Apr 2018, at 18:14, Wall, Michael E <mewall(a)lanl.gov<mailto:[email protected]>> wrote: Hi, I'm seeing the following issue with a fresh build of dials development source, downloaded within the last hour. There's no error using dials v1-9-1. Is this a bug? Something that's being revised right now? Thanks, Mike Traceback (most recent call last): File "../../src/xenm_new.py", line 609, in <module> ftmp.write(pdb_in.hierarchy.as_pdb_string(crystal_symmetry=pdb_in.input.crystal_symmetry())) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/__init__.py", line 1191, in crystal_symmetry self_symmetry = self.crystal_symmetry_from_cryst1() File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/__init__.py", line 1168, in crystal_symmetry_from_cryst1 return cryst1_interpretation.crystal_symmetry(cryst1_record=line) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/cryst1_interpretation.py", line 138, in crystal_symmetry unit_cell=u) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/cryst1_interpretation.py", line 87, in space_group_info try: return sgtbx.space_group_info(self.symbol) File "/home/mewall/packages/dials_devel/modules/cctbx_project/cctbx/sgtbx/__init__.py", line 97, in __init__ symbols = space_group_symbols(symbol) Boost.Python.ArgumentError: Python argument types in space_group_symbols.__init__(space_group_symbols, str) did not match C++ signature: __init__(_object*, int space_group_number) __init__(_object*, int space_group_number, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > extension='') __init__(_object*, int space_group_number, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > extension='', std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > table_id='') __init__(_object*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > symbol) __init__(_object*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > symbol, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > table_id='') ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org<http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ DIALS-support mailing list DIALS-support(a)lists.sourceforge.net<mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/dials-support -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1