- cctbxbb - phenix-online.org

tst_miller failing
by markus.gerstel＠diamond.ac.uk 26 Aug '15

26 Aug '15

Hi everyone, particularly Pavel I guess, I had a look at one intermittently failing test: cctbx/regression/tst_miller.py, run with parameter P31 The relevant piece of code is: def exercise_randomize_amplitude_and_phase(): xrs = random_structure.xray_structure( space_group_info=sgtbx.space_group_info(number=1), elements=["C"]*300) fc = xrs.structure_factors(d_min=1).f_calc() def run(a,p): fc_ = fc.randomize_amplitude_and_phase(amplitude_error=a, phase_error_deg=p, random_seed=1312425) d1, d2 = flex.abs(fc.data()), flex.abs(fc_.data()) r = flex.sum(flex.abs(d1-d2))/flex.sum(flex.abs(d1+d2))*2 return (r, fc.mean_phase_error(phase_source=fc_)) # assert approx_equal(run(0,0), (0,0)) for v in list(xrange(0,91, 10)): r = run(v/100.,v) assert approx_equal(r[0], v/100., 0.01) assert approx_equal(r[1], v, 1) The test fails in the last line for varying values of v. I saw that the values of r[] increase proportionally to v, so I ran the test 2000 times for v=90, plotting the values of r[0] and r[1]; results attached. With the current statistics the r[1] test is expected to fail in 1 run out of every 80. Could you please have a look at this, and either increase the tolerance on the test or identify whether these values are true unexpected outliers indicating a problem in the code? Don't know what the r[1] distribution should look like, but that's clearly not normal. :) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Structure factor calculation for Si
by Allan Lyckegaard 26 Aug '15

26 Aug '15

Dear CCTBX-list I am trying to reproduce the (scaled) intensities for Si reported here: http://rruff.geo.arizona.edu/AMS/xtal_data/DIFfiles/15285.txt based on this information: http://rruff.geo.arizona.edu/AMS/CIF_text_files/15285_cif.txt I am using CCTBX in Python: from cctbx.array_family import flex from cctbx import sgtbx, crystal, miller, xray sgnum = 227 sg = sgtbx.space_group_info(sgnum).group() latticepar = [5.43070, 5.43070, 5.43070, 90, 90, 90] dmin = 1.0 atompos = [['Si',(0.0, 0.0, 0.0)]] scatterers = flex.xray_scatterer() scatterers.append(xray.scatterer(label=atompos[0][0], site=atompos[0][1])) pg = sg.build_derived_point_group() sym = crystal.symmetry(space_group=sg,unit_cell=latticepar) sps = crystal.special_position_settings(crystal_symmetry=sym) structure = xray.structure(special_position_settings=sps,scatterers=scatterers) F = structure.structure_factors(d_min=dmin,algorithm='direct').f_calc() list(F.amplitudes()) This gives: [((0, 2, 2), 0.0), ((0, 4, 0), 120.21765812638594), ((1, 1, 1), 84.18849550806966), ((1, 3, 1), 65.48371727205928), ((1, 3, 3), 57.45438308324155), ((1, 5, 1), 51.48759686950681), ((2, 2, 2), 128.54540698861973), ((2, 4, 2), 0.0), ((3, 3, 3), 51.48759686950681)] How come CCTBX give an amplitude of 0.0 for (0,2,2) and (2,4,2)? The measured intensities in the DIFfile above does not show this. Am I missing something? Cheers, /Allan

4 3

Computational Crystallography Newsletter - Volume 6, Number 2
by Nigel Moriarty 25 Aug '15

25 Aug '15

I am pleased to announce the publication of the latest issue of the Computational Crystallography Newsletter: http://www.phenix-online.org/newsletter/ A listing of the articles and short communications is given below. Please note that the newsletter accepts articles of a general nature of interest to all crystallographers. Please send any articles to me at NWMoriarty(a)lbl.gov noting that there is a Word Template on the website to streamline production. July 2015 <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf> Articles - A context-sensitive guide to RNA & DNA base-pair & base-stack geometry <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=22> Short Communications - New Phaser-MR search panel in Phenix <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=7> - Quantum chemical techniques for minimising ligand geometries in the active site <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=10> - 13 typical occupancy refinement scenarios and available options in *phenix.refine* <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=12> Fitting tips #10 – How do your base pairs touch and twist? <http://www.phenix-online.org/newsletter/CCN_2015_07.pdf#page=3> Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

1 0

Shelx HKL file with original indices from unmerged MTZ
by Graeme.Winter＠diamond.ac.uk 06 Aug '15

06 Aug '15

Hi Folks Using reader = any_reflection_file(hklin) intensities = [ma for ma in reader.as_miller_arrays(merge_equivalents=False) if ma.info().labels == ['I', 'SIGI']][0] help(intensities) with open('%s.hkl' % prefix, 'wb') as f: intensities.export_as_shelx_hklf(f, normalise_if_format_overflow=True) to get a file for input to shelx and it appears to have only indices in the ASU not the Fridel pairs. What do I need to do to output with original indices (ie. P1 indices?) I assume that this has been solved before... Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Tonights build/testing failures
by markus.gerstel＠diamond.ac.uk 05 Aug '15

05 Aug '15

Hi everyone, tonights test (cctbx) or build (dials) failures were caused by changes to the CBFlib in r568. http://sourceforge.net/p/cbflib/code-0/commit_browser Error messages may include ImportError: (..)/build/lib/_pycbf.so: undefined symbol: cbf_airy_disk_volume Do we have a red phone to cbflib folks? -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

cctbx developers: sf outage has lead to local corruption of your cctbx_project
by Aaron Brewster 27 Jul '15

27 Jul '15

Hi folks, because of the recent parallel commits to the cctbx repository due to the sourceforge outage, you will likely need to move your old cctbx_project directory aside and check it out fresh again. The 'too long didn't read' version of this email is you need to do this or several files will remain out of sync. This email only applies to developers or users who use svn directly. How to proceed: $ cd <your sources/modules directory> $ mv cctbx_project cctbx_project.old $ svn co https://svn.code.sf.net/p/cctbx/code/trunk cctbx_project If you have local edits in your cctbx_project directory, you'll need to copy them over to your new folder. More detail for those interested. The following example illustrates the problem: [phenix-src/cctbx_project/scitbx/lstbx]$ svn update [phenix-src/cctbx_project/scitbx/lstbx]$ svn status According to svn, this directory is up to date [phenix-src/cctbx_project/scitbx/lstbx]$ svn info normal_equations.h Path: normal_equations.h Name: normal_equations.h URL: https://svn.code.sf.net/p/cctbx/code/trunk/scitbx/lstbx/normal_equations.h Repository Root: https://svn.code.sf.net/p/cctbx/code Repository UUID: 3c1a6239-ac2c-0410-bc49-92500e444112 Revision: 22630 Node Kind: file Schedule: normal Last Changed Author: luc_j_bourhis Last Changed Rev: 15039 Last Changed Date: 2012-03-12 10:03:05 -0700 (Mon, 12 Mar 2012) Text Last Updated: 2015-07-27 10:24:16 -0700 (Mon, 27 Jul 2015) Checksum: ec2e555a9ad1d2553bb37bc04f677ef5 Note the revision number is 22630, and it lists the last revision for this file as 15039. After I move my directory aside and re-check out the directory: [phenix-src/cctbx_project/scitbx/lstbx]$ svn info normal_equations.h Path: normal_equations.h Name: normal_equations.h URL: https://svn.code.sf.net/p/cctbx/code/trunk/scitbx/lstbx/normal_equations.h Repository Root: https://svn.code.sf.net/p/cctbx/code Repository UUID: 3c1a6239-ac2c-0410-bc49-92500e444112 Revision: 22630 Node Kind: file Schedule: normal Last Changed Author: nksauter Last Changed Rev: 22624 Last Changed Date: 2015-07-25 17:16:55 -0700 (Sat, 25 Jul 2015) Text Last Updated: 2015-07-27 10:46:36 -0700 (Mon, 27 Jul 2015) Checksum: e5e5e623ecdde362afadf321d37b2c5e The correct revision is now retrieved. Thanks, -Aaron

1 0

Re: [cctbxbb] git on BuildBot machines
by markus.gerstel＠diamond.ac.uk 27 Jul '15

27 Jul '15

Hi Aaron, Is there some documentation about the buildbot objects I could use to figure out a sensible way to determine whether git is available? I could of course just do try-run-catch... -Markus On 26 Jul 2015 16:35, Aaron Brewster <asbrewster(a)lbl.gov> wrote: Moving the cctbx_project directory to cctbx_project.old forced svn to get the latest version from scratch so buildbot at LBL has no versioning issues like Jenkins at Diamond does. Further, svn update on my local sources got the latest version, so I'm inclined to agree with Markus that that's a Jenkins issue. I installed git on the build nodes at LBL to accommodate the changes to bootstrap.py, however reconfig.sh still fails. This new line from r22623 and r22626 is the culprit: git_available = self.shell(command=['git', '--version'], haltOnFailure=False, quiet=True, silent=True).run() == 0 While this clearly works on Jenkins, when ran by buildbot, self.shell() is returning a buildbot shell object instead of a Builder shell object, so the .run call fails (not implemented). I'm out of time until this afternoon to look deeper. I've left the build bot server running using r22622 of bootstrap.py for now. -Aaron On Sun, Jul 26, 2015 at 7:00 AM, Nigel Moriarty <nwmoriarty(a)lbl.gov<mailto:[email protected]>> wrote: Aaron Recent changes to include git in bootstrap.py have broken the buildbot master (won't start) because git is not available on the VM. I don't have time before I fly to ACA, so please install git when you can. One reason I was hoping to see the buildbot website was because it has an interface to see all the commits to cctbx. This is likely on the buildbot master in some other format. It may be useful that I just moved the old cctbx_project to cctbx_project.old on all the builds. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org<mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

lost cctbx commits
by Aaron Brewster 27 Jul '15

27 Jul '15

Hi folks, after more investigation I now agree we have parallel commits. The following commits have been lost, all done on July 15th: nwmoriatry: r22622 phyy-nx: r22623-4 tmmclark: r22625 olegsobolev, r22626-7, r22629 alyubimov: r22628 These revisions number have been reused by svn and are no longer valid. The authors should recommit their changes. -Aaron

2 2

git on BuildBot machines
by Nigel Moriarty 26 Jul '15

26 Jul '15

Aaron Recent changes to include git in bootstrap.py have broken the buildbot master (won't start) because git is not available on the VM. I don't have time before I fly to ACA, so please install git when you can. One reason I was hoping to see the buildbot website was because it has an interface to see all the commits to cctbx. This is likely on the buildbot master in some other format. It may be useful that I just moved the old cctbx_project to cctbx_project.old on all the builds. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

2 1

SVN is back on SourceForge
by Nigel Moriarty 26 Jul '15

26 Jul '15

Folks The return of SVN on SF has caused a few problems on the BuildBot but I moved the cctbx_project directory and forced some builds. I'm hoping things will progress but will check later tonight. There have been some strange error messages causing some emails to be send but I'm hoping things will settle soon. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

7 9