October 2016 - cctbxbb - phenix-online.org

Boost Python 1.56
by Luc Bourhis 14 Apr '17

14 Apr '17

Hi, I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56. So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling. Best wishes, Luc

14 36

Re: [cctbxbb] Git status update
by markus.gerstel＠diamond.ac.uk 04 Nov '16

04 Nov '16

Hello everyone, since I have received no complaints and heard nothing to the contrary, so I thought I would let you know on the current state of the git migration: Until *Thursday, 1 September 2016*, 07:59 BST/08:59 CEST/11:59 pm PDT Cctbx development happens on Sourceforge. Note that cctbx development is "frozen" in preparation for the phenix release. If you are actively developing cctbx you should *now* sign up for a GitHub account. If you are not among the 27 people that have already joined the cctbx project at https://github.com/cctbx then please send me a mail with your github account name so I can invite you. In preparation for the move to github you should also install git on your machines. Set up your local installation of git on each machine you are using by running > git config --global pull.rebase true > git config --global user.name "John Doe" > git config --global user.email "your_email(a)example.com" Use an email address you have set up in your GitHub account at https://github.com/settings/emails. Note that this address will be forever publically readable! If you do not want your mail address to be public, you need to verify any (private) mail address with GitHub and then check the /Keep my email address private/ box at the link above. You can then use the dummy address <yourgithubname>@users.noreply.github.com instead. Other useful settings may be > git config --global alias.up "pull --rebase" To enable a *git up* command working much like svn up does. > git config --global alias.unstage "reset HEAD" *git unstage* to easily undo *git add*/*git rm*. > git config --global alias.mkremotebranch "push origin -u" *git mkremotebranch* will copy your current local branch to GitHub. > git config --global alias.rmremotebranch "push origin --delete" *git rmremotebranch <name>* will delete said branch from GitHub. > git config --global color.ui auto This adds color to the git output, which some may find useful for eg. *git status*. On the 1st of September the transitional period starts, and we will begin changing scripts to point to the new code locations. The transition may be delayed somewhat if the phenix release encounters any delays. I will send you further updates closer to the day. You can find and comment on the transition plan at https://docs.google.com/document/d/1bTiGbnvw8lNUrBLbRmhEdnd22McDqBbiKxcpyga… Best wishes -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

6 16

Monthly "how far are we from moving to git" message
by Graeme.Winter＠diamond.ac.uk 21 Oct '16

21 Oct '16

Good afternoon / morning / evening (depending on your locale) I'm just running a bootstrap build and having to check out the whole of cctbx one ... file ... at ... a ... time from svn & it reminded me we were going to move over to git at some point - ISTR was determined by the PHENIX release. Just wondering how this is going & how long it is likely to be until we can get this ball rolling :) Thanks & best wishes Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6

sginfo and sgtbx
by Pascal 21 Oct '16

21 Oct '16

Hi, Just out of curiosity, sginfo website mentions that it has been superseeded by sgtbx. Is that a brand new implementation? I don't recognize sginfo code at all. Also, anyone can confirm that sgtbx is tightly embedded is cctbx and cannot be used standalone? Regards, Pascal

4 6

flex.int.as_double() performance
by richard.gildea＠diamond.ac.uk 11 Oct '16

11 Oct '16

Hi, Whilst looking into performance of various DIALS routines recently, I noticed that the flex.int.as_double() conversion takes roughly 3x as long as the equivalent functionality in numpy: (the choice of a 2527x2463 array is simply because this is the size of the raw data array for a Pilatus 6M detector) import time from dials.array_family import flex n = 100 a = flex.int(flex.grid(2527, 2463)) t0 = time.time() for i in range(n): b = a.as_double() t1 = time.time() t = t1-t0 print "flex.int.as_double(): %.2fs (%.3fs/call)" %(t, t/n) import numpy as np a = np.random.randint(100, size=a.all()) t0 = time.time() for i in range(n): b = a.astype(np.float64) t1 = time.time() t = t1-t0 print "numpy.astype(np.float64): %.2fs (%.3fs/call)" %(t, t/n) $ libtbx.python time_as_double.py flex.int.as_double(): 5.76s (0.058s/call) numpy.astype(np.float64): 1.86s (0.019s/call) This is the current version of the as_double function: static flex_double as_double(f_t const& a) { shared_plain<double> result(a.size(), init_functor_null<double>()); for(std::size_t i=0;i<a.size();i++) result[i] = a[i]; return flex_double(result, a.accessor()); } Replacing the above code with the following, I see a significant runtime improvement, now flex.int.as_double() is marginally quicker than the equivalent numpy routine: static flex_double as_double(f_t const& a) { shared_plain<double> result(a.begin(), a.end()); return flex_double(result, a.accessor()); } $ libtbx.python time_as_double.py flex.int.as_double(): 1.64s (0.016s/call) numpy.astype(np.float64): 1.82s (0.018s/call) Is there any obvious reason why making this change would be A Bad Thing? In the context of dials spot-finding, this change results in a 22s saving (out of a total 172s with the original code) when running dials.find_spots on a 540 image Pilatus 6M dataset using a single processor, so it is definitely a noticeable speed-up. All timings done on a macbook pro. Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3