Good morning Nigel, We throw away the base directory (and nearly everything else) each night, and then run incremental builds throughout the day. This way we can test the installer but still get build feedback in less than 10 minutes. Cheers Markus From: [email protected] [mailto:[email protected]] On Behalf Of Nigel Moriarty Sent: 11 April 2015 00:39 To: cctbx mailing list; Paul Adams; Aaron Brewster Subject: Re: [cctbxbb] scitbx test crashing on mac Markus I'm not sure what's going wrong. Our build system using building and testing on 10.9 but my copy on 10.10 is working fine. I have noticed a similar behaviour when the build system does not delete both base and base_tmp directories. This rebuilds the python each time which takes a little time but it better parallels the user experience. I have been working on changing bootstrap.py but not sure the best way forward. Cheers Nigel --- Nigel W. Moriarty Building 64R0246B, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709tel:510-486-5709 Email : [email protected]mailto:[email protected] Fax : 510-486-5909tel:510-486-5909 Web : CCI.LBL.govhttp://CCI.LBL.gov On Fri, Apr 10, 2015 at 12:46 AM, mailto:[email protected]> wrote: Good morning, The test scitbx.rigid_body.proto.tst_singular has been failing consistently on our MacOS 10.10 system for a day now. Last known good svn rev. is 21989. It passes on the linux systems. Apologies if this is not the right place to report scitbx problems to. -Markus Stacktrace 11 Python 0x0000000100010d03 PyObject_Call + 99 10 libboost_python.dylib 0x0000000101a0f893 boost::python::objects::function_call(_object*, _object*, _object*) + 83 9 libboost_python.dylib 0x0000000101a147f1 boost::python::handle_exception_impl(boost::function0<void>) + 81 8 libboost_python.dylib 0x0000000101a0fe5a boost::detail::function::void_function_ref_invoker0::invoke(boost::detail::function::function_buffer&) + 26 7 libboost_python.dylib 0x0000000101a0db6a boost::python::objects::function::call(_object*, _object*) const + 826 6 scitbx_linalg_ext.so 0x000000010518b521 boost::python::objects::caller_py_function_impl > >::operator()(_object*, _object*) + 65 5 scitbx_linalg_ext.so 0x000000010518b6be _object* boost::python::detail::invoke, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::arg_from_python >(boost::python::detail::invoke_tag_, boost::python::to_python_value const&, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*&)() const, boost::python::arg_from_python&) + 46 4 ??? 0x000000010183fc20 0x0 + 4320394272 3 libsystem_platform.dylib 0x00007fff86f16f1a _sigtramp + 26 2 boost_python_meta_ext.so 0x00000001005bdf80 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. Standard Error show_stack(1): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/featherstone.py(38) generalized_inverse show_stack(2): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/featherstone.py(373) FDab show_stack(3): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/test_simulation.py(28) energies_and_accelerations_update show_stack(4): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/test_simulation.py(16) __init__ show_stack(5): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(72) exercise_six_dof show_stack(6): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(146) run show_stack(7): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(162) <module> libc backtrace (43 frames, most recent call last): 44 ??? 0x0000000000000003 0x0 + 3 43 Python 0x0000000100000d71 Python + 3441 42 Python 0x0000000100000e58 Python + 3672 41 Python 0x00000001000edd44 Py_Main + 3076 40 Python 0x00000001000d64a7 PyRun_SimpleFileExFlags + 711 39 Python 0x00000001000d6984 PyRun_FileExFlags + 164 38 Python 0x00000001000ad596 PyEval_EvalCode + 54 37 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 36 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 35 Python 0x00000001000b5556 fast_function + 118 34 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 33 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 32 Python 0x00000001000b5556 fast_function + 118 31 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 30 Python 0x00000001000b295b PyEval_EvalFrameEx + 18651 29 Python 0x0000000100010d03 PyObject_Call + 99 28 Python 0x0000000100069e23 type_call + 339 27 Python 0x000000010006fccd slot_tp_init + 141 26 Python 0x0000000100010d03 PyObject_Call + 99 25 Python 0x000000010001e207 instancemethod_call + 247 24 Python 0x0000000100010d03 PyObject_Call + 99 23 Python 0x0000000100036a6c function_call + 364 22 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 21 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 20 Python 0x00000001000b5556 fast_function + 118 19 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 18 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 17 Python 0x00000001000b5556 fast_function + 118 16 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 15 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 14 Python 0x00000001000b5556 fast_function + 118 13 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 12 Python 0x00000001000b295b PyEval_EvalFrameEx + 18651 11 Python 0x0000000100010d03 PyObject_Call + 99 10 libboost_python.dylib 0x0000000101a0f893 boost::python::objects::function_call(_object*, _object*, _object*) + 83 9 libboost_python.dylib 0x0000000101a147f1 boost::python::handle_exception_impl(boost::function0<void>) + 81 8 libboost_python.dylib 0x0000000101a0fe5a boost::detail::function::void_function_ref_invoker0::invoke(boost::detail::function::function_buffer&) + 26 7 libboost_python.dylib 0x0000000101a0db6a boost::python::objects::function::call(_object*, _object*) const + 826 6 scitbx_linalg_ext.so 0x000000010518b521 boost::python::objects::caller_py_function_impl > >::operator()(_object*, _object*) + 65 5 scitbx_linalg_ext.so 0x000000010518b6be _object* boost::python::detail::invoke, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::arg_from_python >(boost::python::detail::invoke_tag_, boost::python::to_python_value const&, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*&)() const, boost::python::arg_from_python&) + 46 4 ??? 0x000000010183fc20 0x0 + 4320394272 3 libsystem_platform.dylib 0x00007fff86f16f1a _sigtramp + 26 2 boost_python_meta_ext.so 0x00000001005bdf80 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Luc, I managed to 'fix' the error with this commit: https://sourceforge.net/p/cctbx/code/22003/ However, I have no idea why the previous code was problematic. Adding a SCITBX_ASSERT(vi != 0); in the 'else' statement also made the error go away. I can have another look and try out your suggestions later today to see if that results in any more insight into the underlying problem. Cheers, Richard On 14 April 2015 at 00:17, Luc Bourhis wrote:

Hi Markus,

could you do a debug build and then run that very same test script? Moreover could you run each of the 3 following variants in turn,

BOOST_ADAPTBX_FE_OVERFLOW_DEFAULT=1 libtbx.python scitbx/rigid_body.proto/tst_singular.py

BOOST_ADAPTBX_FE_INVALID_DEFAULT=1 libtbx.python scitbx/rigid_body.proto/tst_singular.py

BOOST_ADAPTBX_FE_DIVBYZERO_DEFAULT libtbx.python scitbx/rigid_body.proto/tst_singular.py

and tell me the result?

Thanks,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Hi Luc, It was strange, because it only appeared after I updated our mac-mini nightly build machine with the latest OS X and Xcode updates the other day. However mine and Graeme's mac book pros both have the latest Apple updates installed and it appears to work fine on our machines (although I'm not certain either of us actually tried a full build from scratch running the bootstrap.py script). Cheers, Richard $ clang --version Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn) Target: x86_64-apple-darwin14.3.0 Thread model: posix $ sw_vers -productVersion 10.10.3 Xcode must be latest version (whatever that is). On 14 April 2015 at 08:22, Luc Bourhis wrote:

On 14 Apr 2015, at 08:44, Richard Gildea wrote:

I managed to 'fix' the error with this commit:

https://sourceforge.net/p/cctbx/code/22003/

However, I have no idea why the previous code was problematic. Adding a SCITBX_ASSERT(vi != 0); in the 'else' statement also made the error go away. I can have another look and try out your suggestions later today to see if that results in any more insight into the underlying problem.

My suggestions only aimed at zeroing on the problem. Now that you have found a fix, you won't learn more by following my suggestions. It looks like a bug in the optimiser of whatever version of clang/llvm Apple uses in the version of Xcode installed on that machine. Note clang optimiser had troubles with that file a couple of years ago: it was missing some optimisations and the produced code was too slow. So now it looks too eager! So classic unfortunately…

For the record, could you paste the outcome of "clang --version" on that machine? And tell us the version of Xcode as well.

I will move to Yosemite soon-ish but for the time being, I can't help much.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

$ xcodebuild -version Xcode 6.3 Build version 6D570 On 14 April 2015 at 08:57, Richard Gildea wrote:

Hi Luc,

It was strange, because it only appeared after I updated our mac-mini nightly build machine with the latest OS X and Xcode updates the other day. However mine and Graeme's mac book pros both have the latest Apple updates installed and it appears to work fine on our machines (although I'm not certain either of us actually tried a full build from scratch running the bootstrap.py script).

Cheers,

Richard

$ clang --version Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn) Target: x86_64-apple-darwin14.3.0 Thread model: posix

$ sw_vers -productVersion 10.10.3

Xcode must be latest version (whatever that is).

On 14 April 2015 at 08:22, Luc Bourhis wrote:

...

On 14 Apr 2015, at 08:44, Richard Gildea wrote:

I managed to 'fix' the error with this commit:

https://sourceforge.net/p/cctbx/code/22003/

However, I have no idea why the previous code was problematic. Adding a SCITBX_ASSERT(vi != 0); in the 'else' statement also made the error go away. I can have another look and try out your suggestions later today to see if that results in any more insight into the underlying problem.

My suggestions only aimed at zeroing on the problem. Now that you have found a fix, you won't learn more by following my suggestions. It looks like a bug in the optimiser of whatever version of clang/llvm Apple uses in the version of Xcode installed on that machine. Note clang optimiser had troubles with that file a couple of years ago: it was missing some optimisations and the produced code was too slow. So now it looks too eager! So classic unfortunately…

For the record, could you paste the outcome of "clang --version" on that machine? And tell us the version of Xcode as well.

I will move to Yosemite soon-ish but for the time being, I can't help much.

Best wishes,

Luc

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb