Hi Nick, Billy and I will work on this today, -Aaron On Wed, Feb 10, 2016 at 6:40 AM, Marcin Wojdyr wrote:

FWIW the latest version of Boost is 1.60. CCP4 6.5 was using cctbx compiled with Boost 1.56 and CCP4 7.0 uses Boost 1.58.

Marcin

On Wed, Feb 10, 2016 at 2:17 PM, Nicholas Sauter wrote:

...

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote:

...

Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose

support

...

I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

+1 On 10 Feb 2016, at 16:48, Richard Gildea mailto:[email protected]> wrote: Hi, Is there any good reason not to move straight to the latest release, 1.60.0, assuming that everything builds/tests pass etc with that version? If we’re putting in the effort to move to a new version it makes sense to try the latest version, rather than some intermediate. Cheers, Richard On 10 Feb 2016, at 16:44, Aaron Brewster mailto:[email protected]> wrote: Hi Nick, Billy and I will work on this today, -Aaron On Wed, Feb 10, 2016 at 6:40 AM, Marcin Wojdyr mailto:[email protected]> wrote: FWIW the latest version of Boost is 1.60. CCP4 6.5 was using cctbx compiled with Boost 1.56 and CCP4 7.0 uses Boost 1.58. Marcin On Wed, Feb 10, 2016 at 2:17 PM, Nicholas Sauter mailto:[email protected]> wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote:

...

Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

To help a bit I went through compilation of cctbx/Dials with Boost 1.60. There is a lot of warnings from Boost.Python, about deprecated header: https://github.com/boostorg/python/commit/0a4c76b9ac16974d7d4f164cf617909521... and unused typedefs. After adding -Wno-unused-local-typedefs and removing the deprecated #include almost all warnings are gone. Then there is one error, copy&pasted below. I tracked it down to this change in boost.rational: https://github.com/boostorg/rational/commit/5fddb3f889cd2a2fe59cdcae182f8b3f... I just reverted this change to finish compilation, but I suppose a proper fix is also not difficult. I don't know if this is relevant, but I'm using Boost compiled outside of cctbx. I'm attaching a patch that we use in CCP4 for this. It's based on Debian/Gentoo patches. cheers, Marcin g++ -o boost_adaptbx/rational_ext.o -c -I/home/marcin/dials/boost_160/include -Wno-unused-local-typedefs -DBOOST_PYTHON_MAX_BASES=2 -fPIC -fno-strict-aliasing -w -DNDEBUG -O3 -ffast-math -DBOOST_ALL_NO_LIB -I/home/marcin/dials/build/include -I/home/marcin/miniconda2/include/python2.7 /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp In file included from /home/marcin/dials/boost_160/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:47:0, from /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp:63, from /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:16, from /home/marcin/dials/boost_160/include/boost/python/object/function_handle.hpp:8, from /home/marcin/dials/boost_160/include/boost/python/converter/arg_to_python.hpp:19, from /home/marcin/dials/boost_160/include/boost/python/call.hpp:15, from /home/marcin/dials/boost_160/include/boost/python/object_core.hpp:14, from /home/marcin/dials/boost_160/include/boost/python/args.hpp:25, from /home/marcin/dials/boost_160/include/boost/python/make_function.hpp:11, from /home/marcin/dials/boost_160/include/boost/python/def.hpp:11, from /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:2: /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_, const RC&, F&, TC&) [with RC = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning; F = const int& (boost::rational<int>::*)()const; TC = boost::python::arg_from_python; PyObject = _object]': /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:223:13: required from 'PyObject* boost::python::detail::caller_arity<1u>::impl::operator()(PyObject*, PyObject*) [with F = const int& (boost::rational<int>::*)()const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2; PyObject = _object]' /home/marcin/dials/boost_160/include/boost/python/object/py_function.hpp:38:33: required from 'PyObject* boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller >; PyObject = _object]' /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:231:1: required from here /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp:88:90: error: no match for call to '(const boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning) (const int&)' return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) ); ^ In file included from /home/marcin/dials/boost_160/include/boost/python/object/function_handle.hpp:8:0, from /home/marcin/dials/boost_160/include/boost/python/converter/arg_to_python.hpp:19, from /home/marcin/dials/boost_160/include/boost/python/call.hpp:15, from /home/marcin/dials/boost_160/include/boost/python/object_core.hpp:14, from /home/marcin/dials/boost_160/include/boost/python/args.hpp:25, from /home/marcin/dials/boost_160/include/boost/python/make_function.hpp:11, from /home/marcin/dials/boost_160/include/boost/python/def.hpp:11, from /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:2: /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject* boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter = boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning; PyTypeObject = _typeobject]': /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:240:19: required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl::signature() [with F = const int& (boost::rational<int>::*)()const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2]' /home/marcin/dials/boost_160/include/boost/python/object/py_function.hpp:48:35: required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller >]' /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:231:1: required from here /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:102:109: error: 'struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning' has no member named 'get_pytype' return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype(); ^ scons: *** [boost_adaptbx/rational_ext.o] Error 1 scons: building terminated because of errors. On Wed, Feb 10, 2016 at 4:44 PM, Aaron Brewster wrote:

Hi Nick, Billy and I will work on this today,

-Aaron

On Wed, Feb 10, 2016 at 6:40 AM, Marcin Wojdyr wrote:

...

FWIW the latest version of Boost is 1.60. CCP4 6.5 was using cctbx compiled with Boost 1.56 and CCP4 7.0 uses Boost 1.58.

Marcin

On Wed, Feb 10, 2016 at 2:17 PM, Nicholas Sauter wrote:

...

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote:

...

Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Aaron, yes, I didn't run test yesterday. I did run it today and at first 20 failed. It's mostly this problem: https://github.com/boostorg/python/issues/56 Other failures I got were probably not related to the boost update. I've committed 3 fixes. I started looking at 4th, which was: File "/home/marcin/dials/modules/cctbx_project/iotbx/mtz/tst_ext.py", line 1093, in exercise_modifiers assert approx_equal(v, [-1]*4) but I must leave now. Does this test also fail for you? Marcin On Wed, Feb 10, 2016 at 11:20 PM, Aaron Brewster wrote:

Hi Marcin, thanks for the detective work. I've reproduced your results on Centos 7 using the cctbx bootstrap script (not sure from your email if that's what you used):

svn export svn://svn.code.sf.net/p/cctbx/code/trunk/libtbx/auto_build/bootstrap.py python bootstrap.py --builder=dials hot update cd modules mv boost oldboost wget https://sourceforge.net/projects/boost/files/latest/download?source=files -O ./boost_1_60_0.tar.gz mv boost_1_60_0 boost vi boost/boost/rational.hpp (revert https://github.com/boostorg/rational/commit/5fddb3f889cd2a2fe59cdcae182f8b3f..., this is necessary to compile) cd .. python bootstrap.py --builder=dials base build

I then exercised the cctbx regression tests:

source build/setpaths.sh mkdir tests cd tests cctbx_regression.test_nightly

17 tests failed, but I think they are fixable, most of them being interface changes in boost. I ran the tests again after restoring the original boost and recompiling. There were still 3 errors, but I need to verify those errors from a completely clean system that hadn't seen boost 1.60 before I'd think they were real.

Anyhow the next step is to track down those 17 errors, fix them, then try the whole process again with the phenix builder and the full set of dials and phenix tests.

-Aaron

On Wed, Feb 10, 2016 at 10:45 AM, Marcin Wojdyr wrote:

...

To help a bit I went through compilation of cctbx/Dials with Boost 1.60.

There is a lot of warnings from Boost.Python, about deprecated header:

https://github.com/boostorg/python/commit/0a4c76b9ac16974d7d4f164cf617909521... and unused typedefs. After adding -Wno-unused-local-typedefs and removing the deprecated #include almost all warnings are gone.

Then there is one error, copy&pasted below. I tracked it down to this change in boost.rational:

https://github.com/boostorg/rational/commit/5fddb3f889cd2a2fe59cdcae182f8b3f... I just reverted this change to finish compilation, but I suppose a proper fix is also not difficult.

I don't know if this is relevant, but I'm using Boost compiled outside of cctbx. I'm attaching a patch that we use in CCP4 for this. It's based on Debian/Gentoo patches.

cheers, Marcin

g++ -o boost_adaptbx/rational_ext.o -c -I/home/marcin/dials/boost_160/include -Wno-unused-local-typedefs -DBOOST_PYTHON_MAX_BASES=2 -fPIC -fno-strict-aliasing -w -DNDEBUG -O3 -ffast-math -DBOOST_ALL_NO_LIB -I/home/marcin/dials/build/include -I/home/marcin/miniconda2/include/python2.7 /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp In file included from

/home/marcin/dials/boost_160/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:47:0, from /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp:63, from /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:16, from

/home/marcin/dials/boost_160/include/boost/python/object/function_handle.hpp:8, from

/home/marcin/dials/boost_160/include/boost/python/converter/arg_to_python.hpp:19, from /home/marcin/dials/boost_160/include/boost/python/call.hpp:15, from /home/marcin/dials/boost_160/include/boost/python/object_core.hpp:14, from /home/marcin/dials/boost_160/include/boost/python/args.hpp:25, from /home/marcin/dials/boost_160/include/boost/python/make_function.hpp:11, from /home/marcin/dials/boost_160/include/boost/python/def.hpp:11, from /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:2: /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp: In instantiation of 'PyObject* boost::python::detail::invoke(boost::python::detail::invoke_tag_, const RC&, F&, TC&) [with RC =

boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning; F = const int& (boost::rational<int>::*)()const; TC = boost::python::arg_from_python; PyObject = _object]':

/home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:223:13: required from 'PyObject* boost::python::detail::caller_arity<1u>::impl::operator()(PyObject*, PyObject*) [with F = const int& (boost::rational<int>::*)()const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2; PyObject = _object]'

/home/marcin/dials/boost_160/include/boost/python/object/py_function.hpp:38:33: required from 'PyObject*

boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, PyObject*) [with Caller = boost::python::detail::caller >; PyObject = _object]'

/home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:231:1: required from here /home/marcin/dials/boost_160/include/boost/python/detail/invoke.hpp:88:90: error: no match for call to '(const

boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning) (const int&)' return rc( (tc().*f)(BOOST_PP_ENUM_BINARY_PARAMS_Z(1, N, ac, () BOOST_PP_INTERCEPT)) );

^ In file included from

/home/marcin/dials/boost_160/include/boost/python/object/function_handle.hpp:8:0, from

/home/marcin/dials/boost_160/include/boost/python/converter/arg_to_python.hpp:19, from /home/marcin/dials/boost_160/include/boost/python/call.hpp:15, from /home/marcin/dials/boost_160/include/boost/python/object_core.hpp:14, from /home/marcin/dials/boost_160/include/boost/python/args.hpp:25, from /home/marcin/dials/boost_160/include/boost/python/make_function.hpp:11, from /home/marcin/dials/boost_160/include/boost/python/def.hpp:11, from /home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:2: /home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp: In instantiation of 'static const PyTypeObject*

boost::python::detail::converter_target_type<ResultConverter>::get_pytype() [with ResultConverter =

boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning; PyTypeObject = _typeobject]':

/home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:240:19: required from 'static boost::python::detail::py_func_sig_info boost::python::detail::caller_arity<1u>::impl::signature() [with F = const int& (boost::rational<int>::*)()const; Policies = boost::python::default_call_policies; Sig = boost::mpl::vector2]'

/home/marcin/dials/boost_160/include/boost/python/object/py_function.hpp:48:35: required from 'boost::python::detail::py_func_sig_info boost::python::objects::caller_py_function_impl<Caller>::signature() const [with Caller = boost::python::detail::caller >]'

/home/marcin/dials/modules/cctbx_project/boost_adaptbx/rational_ext.cpp:231:1: required from here

/home/marcin/dials/boost_160/include/boost/python/detail/caller.hpp:102:109: error: 'struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning' has no member named 'get_pytype' return create_result_converter((PyObject*)0, (ResultConverter *)0, (ResultConverter *)0).get_pytype();

^ scons: *** [boost_adaptbx/rational_ext.o] Error 1 scons: building terminated because of errors.

On Wed, Feb 10, 2016 at 4:44 PM, Aaron Brewster wrote:

...

Hi Nick, Billy and I will work on this today,

-Aaron

On Wed, Feb 10, 2016 at 6:40 AM, Marcin Wojdyr wrote:

...

FWIW the latest version of Boost is 1.60. CCP4 6.5 was using cctbx compiled with Boost 1.56 and CCP4 7.0 uses Boost 1.58.

Marcin

On Wed, Feb 10, 2016 at 2:17 PM, Nicholas Sauter wrote:

...

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote:

...

Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

, boost::mpl::vector8 const&, cctbx::sgtbx::space_group const&, bool, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref > >::execute(_object*, scitbx::af::tiny const&, cctbx::sgtbx::space_group const&, bool, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref)+0x91) [0x7fbae12f81f1] /home/marcin/dials/build/lib/cctbx_translation_search_ext.so(cctbx::translation_search::fast_nv1995<double>::fast_nv1995(scitbx::af::tiny const&, cctbx::sgtbx::space_group const&, bool, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&)+0xa37) [0x7fbae12f7d57] /home/marcin/dials/build/lib/cctbx_translation_search_ext.so(void cctbx::translation_search::fast_nv1995_detail::combination_eq12<double>(scitbx::af::const_ref > const&, scitbx::af::ref > const&, double const&)+0x169) [0x7fbae12eece9] /lib/x86_64-linux-gnu/libc.so.6(+0x36d40) [0x7fbafe2a4d40] /home/marcin/dials/build/lib/boost_python_meta_ext.so(+0x18140) [0x7fbafcfdc140] Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical.

To summarize: I did this exercise on Ubuntu 14.04 with python from Anaconda. Using Boost from boost.gz from LBL I get 3 errors when running cctbx_regression.test_nightly (copied below - I don't know what's the reason). Boost 1.60 has a regression seemingly caused by changes in boost.traits. It may be trivial to fix for someone familiar with boost.python internals, but I'm not such person and boost.python is not actively maintained nowadays, so I gave up. Boost 1.59 also has one regression in boost python, but it's already fixed: https://github.com/boostorg/python/issues/39 With this version (1.59 + patch from #39) I get additional two test failures caused by changed (improved) boost/random/normal_distribution.hpp: https://github.com/boostorg/random/commit/f0ec97ba36c05ef00f2d29dcf66094e3f4... The failing tests just expect different pseudo-random values, so it shouldn't matter. You may consider using this version. Now I'm updating CCP4 development builds to this and we'll see if we see other problems. Marcin libtbx.python "/home/marcin/dials/modules/cctbx_project/cctbx/translation_search/boost_python/tst_translation_search.py" [FAIL] Time: 0.38 Return code: 1 OKs: 0 Standard error: show_stack(1): /home/marcin/dials/modules/cctbx_project/cctbx/translation_search/boost_python/tst_translation_search.py(31) exercise_fast_nv1995 show_stack(2): /home/marcin/dials/modules/cctbx_project/cctbx/translation_search/boost_python/tst_translation_search.py(51) run show_stack(3): /home/marcin/dials/modules/cctbx_project/cctbx/translation_search/boost_python/tst_translation_search.py(56) <module> libc backtrace (28 frames, most recent call last): /home/marcin/dials/build/../../miniconda2/bin/python2.7() [0x400649] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fbafe28fec5] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(Py_Main+0xca4) [0x7fbaff091b74] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xef) [0x7fbaff07c22f] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyRun_FileExFlags+0xb0) [0x7fbaff07c050] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32) [0x7fbaff05bb42] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e) [0x7fbaff05ba2e] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8525) [0x7fbaff05aa55] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e) [0x7fbaff05ba2e] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x8525) [0x7fbaff05aa55] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x89e) [0x7fbaff05ba2e] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6a24) [0x7fbaff058f54] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fbafefa7d23] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(+0xacdd8) [0x7fbaff00add8] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(+0xb6320) [0x7fbaff014320] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fbafefa7d23] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(+0x5c4bf) [0x7fbafefba4bf] /home/marcin/miniconda2/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fbafefa7d23] /home/marcin/dials/build/lib/libboost_python.so(+0x280d3) [0x7fbafcd9e0d3] /home/marcin/dials/build/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x53) [0x7fbafcda96d3] /home/marcin/dials/build/lib/libboost_python.so(+0x29918) [0x7fbafcd9f918] /home/marcin/dials/build/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0xca) [0x7fbafcd9f5aa] /home/marcin/dials/build/lib/cctbx_translation_search_ext.so(boost::python::detail::caller_arity<9u>::impl const&, cctbx::sgtbx::space_group const&, bool, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref), boost::python::default_call_policies, boost::mpl::vector10 const&, cctbx::sgtbx::space_group const&, bool, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref const&, scitbx::af::const_ref > >::operator()(_object*, _object*)+0x3d8) [0x7fbae12f8628] /home/marcin/dials/build/lib/cctbx_translation_search_ext.so(boost::python::objects::make_holder<8>::apply

Hi Luc et al., no concerns from DIALS East about this. May I ask though that this is delayed for at least a few days, as we are currently in the process of branching DIALS 1.5? Thanks -Markus ________________________________ From: [email protected] [[email protected]] on behalf of Luc Bourhis [[email protected]] Sent: Monday, April 03, 2017 14:31 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56 Hi all, everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2. Thanks, Luc On 10 Feb 2016, at 15:17, Nicholas Sauter mailto:[email protected]> wrote: Nigel, Billy & Aaron, I completely endorse this move to Boost 1.56. Can we update our build? Nick Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi, I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56. So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling. Best wishes, Luc _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary). Charles On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

...

On 10 Feb 2016, at 15:17, Nicholas Sauter wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

On 5 April 2017 at 23:37, Billy Poon wrote:

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

Hi Billy, perhaps it's this change in boost/random/normal_distribution.hpp: https://github.com/boostorg/random/commit/f0ec97ba36c05ef00f2d29dcf66094e3f4...

Hey Billy, On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find? -James On Apr 5, 2017, at 3:38 PM, Billy Poon wrote: Hi all, I tested Boost 1.56 on our buildbot servers and got some new test failures with cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py The full log for CentOS 6 can be found at http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-... It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers. Are these tests failing for others as well? -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard wrote:

FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote:

...

Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

To Pavel’s comment. Now that we have a new location for cctbx maybe we can add this document somewhere prominent on the web site.

On Apr 5, 2017, at 8:00 PM, Pavel Afonine wrote:

Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote:

...

Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon wrote:

...

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-...

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

...

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

...

On 10 Feb 2016, at 15:17, Nicholas Sauter wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list

[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ [email protected] ][ 1-510-495-2506 ] --

Morning all I would think putting the content directly onto the giithub page perhaps at https://github.com/cctbx/cctbx_project/wiki could be the way to go - a copy / paste job would still put us in a better position than we are now (assuming whatever commands are in there still work) James - FWIW the tests I run are libtbx.run_tests_parallel \ run_in_tmp_dir=true \ module=xia2 module=cctbx module=rstbx module=dxtbx module=dials \ module=dials_regression \ nproc=6 this should work for you… though you probably don’t want to bother with xia2 tests Best wishes Graeme On 6 Apr 2017, at 06:14, Pavel Afonine mailto:[email protected]> wrote: Excellent idea! Let's do it! Who is in capacity to do it? Or shell we make a round or reading the document and updating it up to current realities? Pavel On 4/6/17 11:55, Paul Adams wrote: To Pavel’s comment. Now that we have a new location for cctbx maybe we can add this document somewhere prominent on the web site. On Apr 5, 2017, at 8:00 PM, Pavel Afonine mailto:[email protected]> wrote: Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). Unfortunately, nobody bothered to put it in some central place. Pavel On 4/6/17 10:51, James Holton wrote: Hey Billy, On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find? -James On Apr 5, 2017, at 3:38 PM, Billy Poon mailto:[email protected]> wrote: Hi all, I tested Boost 1.56 on our buildbot servers and got some new test failures with cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py The full log for CentOS 6 can be found at http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-... It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers. Are these tests failing for others as well? -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary). Charles On 3 Apr 2017, at 14:31, Luc Bourhis wrote: Hi all, everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2. Thanks, Luc On 10 Feb 2016, at 15:17, Nicholas Sauter wrote: Nigel, Billy & Aaron, I completely endorse this move to Boost 1.56. Can we update our build? Nick Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis wrote: Hi, I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56. So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling. Best wishes, Luc _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Hi Graeme, hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file? Thanks, Pavel On 4/6/17 13:45, [email protected] wrote:

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to transcribe this over..

I also note that it lacks the actual detail on how to run tests! However would be happy to add this once on wiki

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine mailto:[email protected]> wrote:

Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon mailto:[email protected]> wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-...

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.orghttps://phenix-online.org/

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard mailto:[email protected]> wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter mailto:[email protected]> wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi James, I agree with Graeme's tests and I would add cctbx_regression.test_nightly That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)? -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Thu, Apr 6, 2017 at 1:37 AM, wrote:

I've made a start on transcribing this document here:

https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance

It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn).

Cheers,

Richard

Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078

Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE

________________________________________ From: [email protected] [[email protected]] on behalf of Pavel Afonine [[email protected]] Sent: 06 April 2017 09:07 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file?

Thanks, Pavel

...

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to transcribe

On 4/6/17 13:45, [email protected] wrote: this over..

...

I also note that it lacks the actual detail on how to run tests! However

would be happy to add this once on wiki

...

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine

[email protected]>> wrote:

...

Not sure if that answers your questions but once upon a time we here at

...

Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon > wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-

Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). linux-2.6-x86_64-centos6/builds/601/steps/test%20cctbx_ regression.test_nightly/logs/stdio

...

It looks like the errors are related to random number generation. For a

given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

...

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.orghttps://phenix-online.org/

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard <

...

FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for

[email protected]mailto:[email protected]> wrote: the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

...

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to

Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

...

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter

[email protected]>> wrote:

...

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging

Division

...

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_____________________

---

...

cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi everyone, Boost 1.56 is now available and the tests have been updated. To update an existing installtion, you have to delete the "build" directory first. If you don't, I guess some files are not recompiled with the new Boost headers, which will cause cctbx_project/scitbx/random/tests/tst_random.py to expect the old values. Then run bootstrap.py with hot, update, and build to rebuild. cd <installation directory> rm -fr build python bootstrap.py hot update build --builder=cctbx Let me know if there are any issues. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Fri, Apr 7, 2017 at 1:08 PM, Billy Poon wrote:

I can do the switch to Boost 1.56 next Wednesday now that Dials 1.5 has been released. I'm just double-checking some other Phenix tests.

Once the update is live, bootstrap.py should delete the existing modules/boost directory and replace it with the new one. And then scons should recompile anything that depends on boost. If in doubt, you can manually delete the module/boost and build directories.

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

On Thu, Apr 6, 2017 at 9:32 AM, Billy Poon wrote:

...

Hi James,

I agree with Graeme's tests and I would add

cctbx_regression.test_nightly

That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

On Thu, Apr 6, 2017 at 1:37 AM, wrote:

...

I've made a start on transcribing this document here:

https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance

It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn).

Cheers,

Richard

Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078

Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE

________________________________________ From: [email protected] [cctbxbb-bounces@phenix-online .org] on behalf of Pavel Afonine [[email protected]] Sent: 06 April 2017 09:07 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file?

Thanks, Pavel

...

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to

On 4/6/17 13:45, [email protected] wrote: transcribe this over..

...

I also note that it lacks the actual detail on how to run tests!

However would be happy to add this once on wiki

...

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine

[email protected]>> wrote:

...

Not sure if that answers your questions but once upon a time we here

...

Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon > wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-l inux-2.6-x86_64-centos6/builds/601/steps/test%20cctbx_regres sion.test_nightly/logs/stdio

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.orghttps://phenix-online.org/

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard < [email protected]mailto:[email protected]> wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter > wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of

at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

...

So, would it be possible to move to Boost 1.56? Otherwise, I will need

to backport that function. Not too difficult but not thrilling.

...

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

______________________

_________________________

...

cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Graeme, But Billy has essentially done this :) rm -rv $(libtbx.show_build_path) libtbx.bootstrap hot build --builder=dials in your case. No warranty for anything though. Jenkins seems to have mostly picked up the new library with no issues. A few jobs apparently have not yet picked it up, but should do so soon by themselves. -Markus -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: 13 April 2017 08:11 To: [email protected] Subject: Re: [cctbxbb] Boost Python 1.56 Colleagues For those of us who are elderly and hard of thinking 😉 please could you send a relatively complete set of instructions on how best to update i.e. Using bootstrap or whatever. 5 line shell is fine My usual recipe of redo from start is kind of annoying Thanks Graeme On 12 Apr 2017, at 21:48, Billy Poon mailto:[email protected]> wrote: Hi everyone, Boost 1.56 is now available and the tests have been updated. To update an existing installtion, you have to delete the "build" directory first. If you don't, I guess some files are not recompiled with the new Boost headers, which will cause cctbx_project/scitbx/random/tests/tst_random.py to expect the old values. Then run bootstrap.py with hot, update, and build to rebuild. cd <installation directory> rm -fr build python bootstrap.py hot update build --builder=cctbx Let me know if there are any issues. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Fri, Apr 7, 2017 at 1:08 PM, Billy Poon mailto:[email protected]> wrote: I can do the switch to Boost 1.56 next Wednesday now that Dials 1.5 has been released. I'm just double-checking some other Phenix tests. Once the update is live, bootstrap.py should delete the existing modules/boost directory and replace it with the new one. And then scons should recompile anything that depends on boost. If in doubt, you can manually delete the module/boost and build directories. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709tel:(510)%20486-5709 Fax: (510) 486-5909tel:(510)%20486-5909 Web: https://phenix-online.org On Thu, Apr 6, 2017 at 9:32 AM, Billy Poon mailto:[email protected]> wrote: Hi James, I agree with Graeme's tests and I would add cctbx_regression.test_nightly That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)? -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709tel:(510)%20486-5709 Fax: (510) 486-5909tel:(510)%20486-5909 Web: https://phenix-online.org On Thu, Apr 6, 2017 at 1:37 AM, mailto:[email protected]> wrote: I've made a start on transcribing this document here: https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn). Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078tel:%2B441235%2077%208078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE ________________________________________ From: [email protected]mailto:[email protected] [[email protected]mailto:[email protected]] on behalf of Pavel Afonine [[email protected]mailto:[email protected]] Sent: 06 April 2017 09:07 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56 Hi Graeme, hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file? Thanks, Pavel On 4/6/17 13:45, [email protected]mailto:[email protected] wrote:

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to transcribe this over..

I also note that it lacks the actual detail on how to run tests! However would be happy to add this once on wiki

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine mailto:[email protected]mailto:[email protected]>> wrote:

Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon mailto:[email protected]mailto:[email protected]>> wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x 86_64-centos6/builds/601/steps/test%20cctbx_regression.test_nightly/lo gs/stdio

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709tel:%28510%29%20486-5709 Fax: (510) 486-5909tel:%28510%29%20486-5909 Web: https://phenix-online.orghttps://phenix-online.org/

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard mailto:[email protected]mailto:[email protected]>> wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter mailto:[email protected]mailto:[email protected]>> wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]mailto:[email protected]>> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick’s request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

________________________________ _______________ cctbxbb mailing list [email protected]mailto:[email protected]mailto:[email protected]> http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Billy, As you'll know the Windows builds failed with the new boost libraries. this is due to some filenames exceeding the 255 char limit on Windows. It worked fine on my nightly build because the build folder isn't as deeply nested as on pulse and flux. On pulse and flux it dies in tarfile.TarFile._extract_member() which I have made a monkey patch for which appears to work by ignoring the offending files. However, as all the long filenames seem to come from html folders within boost presumably these are just documentation and it would be better to exclude these altogether in the cci subset of boost. People can always go online if they want to read up on the documentation for boost. Besides bootstrap.py is already quite bloated. Let me know what you think, Rob On 12/04/2017 21:47, Billy Poon wrote:

Hi everyone,

Boost 1.56 is now available and the tests have been updated. To update an existing installtion, you have to delete the "build" directory first. If you don't, I guess some files are not recompiled with the new Boost headers, which will cause cctbx_project/scitbx/random/tests/tst_random.py to expect the old values. Then run bootstrap.py with hot, update, and build to rebuild.

cd <installation directory> rm -fr build python bootstrap.py hot update build --builder=cctbx

Let me know if there are any issues. Thanks!

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org [1]

On Fri, Apr 7, 2017 at 1:08 PM, Billy Poon wrote:

I can do the switch to Boost 1.56 next Wednesday now that Dials 1.5 has been released. I'm just double-checking some other Phenix tests.

Once the update is live, bootstrap.py should delete the existing modules/boost directory and replace it with the new one. And then scons should recompile anything that depends on boost. If in doubt, you can manually delete the module/boost and build directories.

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 [2] Fax: (510) 486-5909 [3] Web: https://phenix-online.org [1]

On Thu, Apr 6, 2017 at 9:32 AM, Billy Poon wrote:

Hi James,

I agree with Graeme's tests and I would add

cctbx_regression.test_nightly

That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 [2] Fax: (510) 486-5909 [3] Web: https://phenix-online.org [1]

On Thu, Apr 6, 2017 at 1:37 AM, wrote: I've made a start on transcribing this document here:

https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance [4]

It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn).

Cheers,

Richard

Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 [5]

Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE

________________________________________ From: [email protected] [[email protected]] on behalf of Pavel Afonine [[email protected]] Sent: 06 April 2017 09:07 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file?

Thanks, Pavel

On 4/6/17 13:45, [email protected] wrote:

...

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to transcribe this over..

I also note that it lacks the actual detail on how to run tests! However would be happy to add this once on wiki

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine mailto:[email protected]> wrote:

Not sure if that answers your questions but once upon a time we here at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon mailto:[email protected]> wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-... [6]

It looks like the errors are related to random number generation. For a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 [7] Fax: (510) 486-5909 [8] Web: https://phenix-online.org [1]<https://phenix-online.org/ [9]>

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard mailto:[email protected]> wrote: FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter mailto:[email protected]> wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 [10]tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick's request. I need the function boost::thread::physical_concurrency which returns the number of physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

So, would it be possible to move to Boost 1.56? Otherwise, I will need to backport that function. Not too difficult but not thrilling.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11]

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [11] Links: ------ [1] https://phenix-online.org [2] tel:(510)%20486-5709 [3] tel:(510)%20486-5909 [4] https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance [5] tel:%2B441235%2077%208078 [6] http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-linux-2.6-x86_64-... [7] tel:%28510%29%20486-5709 [8] tel:%28510%29%20486-5909 [9] https://phenix-online.org/ [10] tel:%28510%29%20486-5713 [11] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Graeme, I hope Markus's commands were helpful. The only tricky thing was the deletion of the build directory. If you had set up the cctbx environment, then python would have pointed to something in the build directory. You would have need to explicitly use ./base/bin/python or the system python with bootstrap.py instead. Also, I stripped extraneous stuff from the Boost tarball according to how earlier boost tarballs were built. This will improve download times and fix the Windows installation. Basically, only the following things are kept, LICENSE_1_0.txt boost/ libs/date_time libs/detail libs/filesystem libs/program_options libs/python libs/system libs/thread It's available now if you use authenticated access (--cciuser flag for bootstrap.py) and will propagate for unauthenticated access ( http://cci.lbl.gov/repositories) by 9:30 pm Pacific time (or 5:30 am UK time). Let me know of any issues. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Thu, Apr 13, 2017 at 8:02 AM, R.D. Oeffner wrote:

Sorry this was to Billy only. Please ignore.

Rob

On 13/04/2017 16:00, R.D. Oeffner wrote:

Hi Billy,

As you'll know the Windows builds failed with the new boost libraries. this is due to some filenames exceeding the 255 char limit on Windows. It worked fine on my nightly build because the build folder isn't as deeply nested as on pulse and flux. On pulse and flux it dies in tarfile.TarFile._extract_member() which I have made a monkey patch for which appears to work by ignoring the offending files.

However, as all the long filenames seem to come from html folders within boost presumably these are just documentation and it would be better to exclude these altogether in the cci subset of boost. People can always go online if they want to read up on the documentation for boost. Besides bootstrap.py is already quite bloated.

Let me know what you think,

Rob

On 12/04/2017 21:47, Billy Poon wrote:

Hi everyone,

Boost 1.56 is now available and the tests have been updated. To update an existing installtion, you have to delete the "build" directory first. If you don't, I guess some files are not recompiled with the new Boost headers, which will cause cctbx_project/scitbx/random/tests/tst_random.py to expect the old values. Then run bootstrap.py with hot, update, and build to rebuild.

cd <installation directory> rm -fr build python bootstrap.py hot update build --builder=cctbx

Let me know if there are any issues. Thanks!

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

On Fri, Apr 7, 2017 at 1:08 PM, Billy Poon wrote:

...

I can do the switch to Boost 1.56 next Wednesday now that Dials 1.5 has been released. I'm just double-checking some other Phenix tests.

Once the update is live, bootstrap.py should delete the existing modules/boost directory and replace it with the new one. And then scons should recompile anything that depends on boost. If in doubt, you can manually delete the module/boost and build directories.

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Thu, Apr 6, 2017 at 9:32 AM, Billy Poon wrote:

...

Hi James,

I agree with Graeme's tests and I would add

cctbx_regression.test_nightly

That command is a shortcut for running the test modules for libtbx, boost_adaptbx, scitbx, cctbx, iotbx, dxtbx, and smtbx. The mmtbx module is also tested if chem_data is available. Should we add rstbx to the shortcut (cctbx_project/cctbx/command_line/cctbx_test_nightly.py)?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org On Thu, Apr 6, 2017 at 1:37 AM, wrote:

...

I've made a start on transcribing this document here:

https://github.com/cctbx/cctbx_project/wiki/cctbx-Developer-Guidance

It probably still needs cleaning up a bit (e.g. I couldn't figure out quickly how to do 3-level list nesting, 1.a.i etc.) and updating to reflect current practice (e.g. git not svn).

Cheers,

Richard

Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078

Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE

________________________________________ From: [email protected] [cctbxbb-bounces@phenix- online.org] on behalf of Pavel Afonine [[email protected]] Sent: 06 April 2017 09:07 To: cctbx mailing list Subject: Re: [cctbxbb] Boost Python 1.56

Hi Graeme,

hm.. this is a good question. We've been through back-and-forth iterations of editing this file and I think the latest I have is from Paul. But I can't find a non-PDF version of it. Paul: do you have an editable version of this file?

Thanks, Pavel

...

Hi Pavel

These all seem sensible

If you have the original non pdf document it may be easier to

On 4/6/17 13:45, [email protected] wrote: transcribe this over..

...

I also note that it lacks the actual detail on how to run tests!

However would be happy to add this once on wiki

...

Best wishes Graeme

On 6 Apr 2017, at 04:00, Pavel Afonine

[email protected]>> wrote:

...

Not sure if that answers your questions but once upon a time we here

...

Unfortunately, nobody bothered to put it in some central place.

Pavel

On 4/6/17 10:51, James Holton wrote: Hey Billy,

On a related note. How do I run these regression tests before committing something into Git? Is there a document on dials regression testing I can't find?

-James

On Apr 5, 2017, at 3:38 PM, Billy Poon > wrote:

Hi all,

I tested Boost 1.56 on our buildbot servers and got some new test failures with

cctbx_project/scitbx/array_family/boost_python/tst_flex.py cctbx_project/scitbx/random/tests/tst_random.py

The full log for CentOS 6 can be found at

http://cci-vm-6.lbl.gov:8010/builders/phenix-nightly-intel-

at Berkeley tried to write a some sort of document that was supposed to answer questions like this. Attached. By no means it is complete, up-to-date, etc, but it might be worth reading for anyone who contributes to cctbx. (Even not sure if I'm sending the latest version). linux-2.6-x86_64-centos6/builds/601/steps/test%20cctbx_ regression.test_nightly/logs/stdio

...

It looks like the errors are related to random number generation. For

a given seed, would the sequence of numbers change when Boost is changed? I did a diff between Boost 1.56 and the current Boost and could not see any changes that immediately stood out as being related to random numbers.

...

Are these tests failing for others as well?

-- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.orghttps://phenix-online.org/

On Wed, Apr 5, 2017 at 8:12 AM, Charles Ballard <

...

FYI, we (CCP4) have been using 1.56 for building cctbx/phaser/dials for the last while with no issues. Don't know about 1.60, but 1.59 causes issues with the boost python make_getter and make_setter (initialisation of none const reference if the passed type is a temporary).

Charles

On 3 Apr 2017, at 14:31, Luc Bourhis wrote:

Hi all,

everybody seemed to agree but then it was proposed to move straight to Boost 1.60, and this caused troubles. Could we consider again to move to at least 1.56? As far as I can tell, this does not cause any issue and as stated one year ago, it would help me and Olex 2.

Thanks,

Luc

On 10 Feb 2016, at 15:17, Nicholas Sauter > wrote:

Nigel, Billy & Aaron,

I completely endorse this move to Boost 1.56. Can we update our build?

Nick

Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:%28510%29%20486-5713

On Wed, Feb 10, 2016 at 2:41 PM, Luc Bourhis mailto:[email protected]> wrote: Hi,

I have improvements to the smtbx on their way to be committed which require Boost version 1.56. This is related to Boost.Threads, whose support I re-activated a few months ago on Nick's request. I need the function boost::thread::physical_concurrency which returns the number of

[email protected]mailto:[email protected]> wrote: physical cores on the machine, as opposed to virtual cores when hyperthreading is enabled (which it is by default on any Intel machine). That function is not available in Boost 1.55 which is the version currently used in the nightly tests: it appeared in 1.56.

...

So, would it be possible to move to Boost 1.56? Otherwise, I will

need to backport that function. Not too difficult but not thrilling.

...

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_____________________

__________________________

...

cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

---

cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb