[cctbxbb] Boost Python 1.56

Thu Apr 13 00:29:22 PDT 2017

Okay, let me correct myself here since libtbx.bootstrap fails after deleting the build directory (surprise)...

cd $(libtbx.show_build_path)/..
rm -rv build
cp modules/cctbx_project/libtbx/auto_build/bootstrap.py .
python bootstrap.py hot build --builder=dials

you may then have to do 

cd build
libtbx.configure xia2 xia2_regression dials_regression # (...)
make
# and, assuming you are in the DLS network:
modules/xia2_regression/command_line/fetch_test_data_local.sh

and that should be it.

-Markus

-----Original Message-----
From: cctbxbb-bounces at phenix-online.org [mailto:cctbxbb-bounces at phenix-online.org] On Behalf Of markus.gerstel at diamond.ac.uk
Sent: 13 April 2017 08:20
To: cctbxbb at phenix-online.org
Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

But Billy has essentially done this :)

rm -rv $(libtbx.show_build_path)
libtbx.bootstrap hot build --builder=dials

in your case. No warranty for anything though.

Jenkins seems to have mostly picked up the new library with no issues. A few jobs apparently have not yet picked it up, but should do so soon by themselves.

-Markus

-----Original Message-----
From: cctbxbb-bounces at phenix-online.org [mailto:cctbxbb-bounces at phenix-online.org] On Behalf Of Graeme.Winter at diamond.ac.uk
Sent: 13 April 2017 08:11
To: cctbxbb at phenix-online.org
Subject: Re: [cctbxbb] Boost Python 1.56

Colleagues

For those of us who are elderly and hard of thinking 😉 please could you send a relatively complete set of instructions on how best to update i.e. Using bootstrap or whatever.

5 line shell is fine

My usual recipe of redo from start is kind of annoying

Thanks Graeme

On 12 Apr 2017, at 21:48, Billy Poon <bkpoon at lbl.gov<mailto:bkpoon at lbl.gov>> wrote:

Hi everyone,

Boost 1.56 is now available and the tests have been updated. To update an existing installtion, you have to delete the "build" directory first. If you don't, I guess some files are not recompiled with the new Boost headers, which will cause  cctbx_project/scitbx/random/tests/tst_random.py to expect the old values. Then run bootstrap.py with hot, update, and build to rebuild.

cd <installation directory>
rm -fr build
python bootstrap.py hot update build --builder=cctbx

Let me know if there are any issues. Thanks!

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Tel: (510) 486-5709
Fax: (510) 486-5909
Web: https://phenix-online.org

On Fri, Apr 7, 2017 at 1:08 PM, Billy Poon <bkpoon at lbl.gov<mailto:bkpoon at lbl.gov>> wrote:
I can do the switch to Boost 1.56 next Wednesday now that Dials 1.5 has been released. I'm just double-checking some other Phenix tests.

Once the update is live, bootstrap.py should delete the existing modules/boost directory and replace it with the new one. And then scons should recompile anything that depends on boost. If in doubt, you can manually delete the module/boost and build directories.

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Tel: (510) 486-5709<tel:(510)%20486-5709>
Fax: (510) 486-5909<tel:(510)%20486-5909>
Web: https://phenix-online.org

On Thu, Apr 6, 2017 at 9:32 AM, Billy Poon <bkpoon at lbl.gov<mailto:bkpoon at lbl.gov>> wrote:
Hi James,

I agree with Graeme's tests and I would add

cctbx_regression.test_nightly

That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)?

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Tel: (510) 486-5709<tel:(510)%20486-5709>
Fax: (510) 486-5909<tel:(510)%20486-5909>
Web: https://phenix-online.org

On Thu, Apr 6, 2017 at 1:37 AM, <richard.gildea at diamond.ac.uk<mailto:richard.gildea at diamond.ac.uk>> wrote:
I've made a start on transcribing this document here:

https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance

It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn).

Cheers,

Richard

Dr Richard Gildea
Data Analysis Scientist
Tel: +441235 77 8078<tel:%2B441235%2077%208078>

Diamond Light Source Ltd.
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE

________________________________________
From: cctbxbb-bounces at phenix-online.org<mailto:cctbxbb-bounces at phenix-online.org> [cctbxbb-bounces at phenix-online.org<mailto:cctbxbb-bounces at phenix-online.org>] on behalf of Pavel Afonine [pafonine at lbl.gov<mailto:pafonine at lbl.gov>]
Sent: 06 April 2017 09:07
To: cctbx mailing list
Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file?

Thanks,
Pavel

On 4/6/17 13:45, Graeme.Winter at diamond.ac.uk<mailto:Graeme.Winter at diamond.ac.uk> wrote:
> Hi Pavel
>
> These all seem sensible
>
> If you have the original non pdf document it may be easier to transcribe this over..
>
> I also note that it lacks the actual detail on how to run tests! 
> However would be happy to add this once on wiki
>
> Best wishes Graeme
>
> On 6 Apr 2017, at 04:00, Pavel Afonine <pafonine at lbl.gov<mailto:pafonine at lbl.gov><mailto:pafonine at lbl.gov<mailto:pafonine at lbl.gov>>> wrote:
>
> Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version).
> Unfortunately, nobody bothered to put it in some central place.
>
> Pavel
>
> On 4/6/17 10:51, James Holton wrote:
> Hey Billy,
>
> On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?
>
> -James
>
> On Apr 5, 2017, at 3:38 PM, Billy Poon <bkpoon at lbl.gov<mailto:bkpoon at lbl.gov><mailto:bkpoon at lbl.gov<mailto:bkpoon at lbl.gov>>> wrote:
>
> Hi all,
>
> I tested Boost 1.56 on our buildbot servers and got some new test 
> failures with
>
> cctbx_project/scitbx/array_family/boost_python/tst_flex.py
> cctbx_project/scitbx/random/tests/tst_random.py
>
> The full log for CentOS 6 can be found at
>
> http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x
> 86_64-centos6/builds/601/steps/test%20cctbx_regression.test_nightly/lo
> gs/stdio
>
> It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.
>
> Are these tests failing for others as well?
>
> --
> Billy K. Poon
> Research Scientist, Molecular Biophysics and Integrated Bioimaging 
> Lawrence Berkeley National Laboratory
> 1 Cyclotron Road, M/S 33R0345
> Berkeley, CA 94720
> Tel: (510) 486-5709<tel:%28510%29%20486-5709>
> Fax: (510) 486-5909<tel:%28510%29%20486-5909>
> Web: https://phenix-online.org<https://phenix-online.org/>
>
> On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard <charles.ballard at stfc.ac.uk<mailto:charles.ballard at stfc.ac.uk><mailto:charles.ballard at stfc.ac.uk<mailto:charles.ballard at stfc.ac.uk>>> wrote:
> FYI,  we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues.  Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).
>
> Charles
>
> On 3 Apr 2017, at 14:31, Luc Bourhis wrote:
>
> Hi all,
>
> everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.
>
> Thanks,
>
> Luc
>
> On 10 Feb 2016, at 15:17, Nicholas Sauter <nksauter at lbl.gov<mailto:nksauter at lbl.gov><mailto:nksauter at lbl.gov<mailto:nksauter at lbl.gov>>> wrote:
>
> Nigel, Billy & Aaron,
>
> I completely endorse this move to Boost 1.56.  Can we update our build?
>
> Nick
>
> Nicholas K. Sauter, Ph. D.
> Computer Staff Scientist, Molecular Biophysics and Integrated 
> Bioimaging Division Lawrence Berkeley National Laboratory
> 1 Cyclotron Rd., Bldg. 33R0345
> Berkeley, CA 94720
> (510) 486-5713<tel:%28510%29%20486-5713><tel:%28510%29%20486-5713>
>
> On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis <luc_j_bourhis at mac.com<mailto:luc_j_bourhis at mac.com><mailto:luc_j_bourhis at mac.com<mailto:luc_j_bourhis at mac.com>>> wrote:
> Hi,
>
> I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.
>
> So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.
>
> Best wishes,
>
> Luc
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
>
> _______________________________________________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>
> <cctbx-developer-guidance-08-2015.pdf>________________________________
> _______________
> cctbxbb mailing list
> cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org><mailto:cct
> bxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>>
> http://phenix-online.org/mailman/listinfo/cctbxbb
>
>

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org<mailto:cctbxbb at phenix-online.org>
http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________
cctbxbb mailing list
cctbxbb at phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb