- cctbxbb - phenix-online.org

Re: [cctbxbb] Fw: Build constraints in SConscript?
by Graeme.Winter＠diamond.ac.uk 16 May '15

16 May '15

Nigel If you export the environment variable I sent before before kicking off a build you should make fully portable binaries. We've been doing this in dials for a while. Best wishes Graeme On 15 May 2015 18:00, Nigel Moriarty <nwmoriarty(a)lbl.gov> wrote: Greame Did you try replacing 10.8 in the SConscript file with 10.7? Please do so and let me know if this works. I was working on getting the mechanism to be OS specific on the buildbot and bootstrap when Rob solved the problem independently. If you don't tell me if changing 10.8 to 10.7 works I will do something to get it working here and we can worry about your 2011 vintage OS next week. Cheers Nigel --- Nigel W. Moriarty Building 64R0246B, Physical Biosciences Division Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> On Fri, May 15, 2015 at 2:31 AM, <Graeme.Winter(a)diamond.ac.uk<mailto:[email protected]>> wrote: Dear Rob Given that the tests are currently not working right anyway, and there are no build / deployment tests for testing builds on an older version of OS X than they are compiled on, this is an easy problem to get in to. It's also a collaborative project where we're all working hard on making things work. This will happen from time to time. I had thought the same environment variable based mechanism for the builds was already in place for PHENIX - am surprised it is not! So, no worries. Perhaps I came across more grumpy than I meant to below. Occurs to me it would be very handy to have one email address for all cctbx related developers where (someone) could ask (something like this) and get an answer perhaps suggesting a more optimum solution? Thoughts anyone? (lots of mail lists in send: list!) Thanks & best wishes Graeme -----Original Message----- From: R.D. Oeffner [mailto:[email protected]<mailto:[email protected]>] Sent: 15 May 2015 10:26 To: Winter, Graeme (DLSLtd,RAL,LSCI); cctbxbb(a)phenix-online.org<mailto:[email protected]>; dials-support(a)lists.sourceforge.net<mailto:[email protected]>; dials-dev(a)cci.lbl.gov<mailto:[email protected]>; Nigel W Moriarty Subject: Re: Fw: Build constraints in SConscript? Dear Graeme, I'm sorry for the trouble my change caused and the time taken to identify it. Ironically we had the same issue as you with Phenix not installing on anything lower than OSX 10.9. As nothing appeared to be specified in our build system to that effect my change was a quick fix to address the problem. It was genuinely well intended. Nigel, I defer to you for implementing the minimum OSX platform for Phenix builds. Rob > FROM: Graeme.Winter(a)Diamond.ac.uk<mailto:[email protected]> > SENT: Friday, May 15, 2015 9:09 AM > TO: Graeme.Winter(a)Diamond.ac.uk<mailto:[email protected]> ; cctbxbb(a)phenix-online.org<mailto:[email protected]> ; > robert(a)oeffner.net<mailto:[email protected]> > CC: dials-support(a)lists.sourceforge.net<mailto:[email protected]> ; dials-dev(a)cci.lbl.gov<mailto:[email protected]> > SUBJECT: RE: Build constraints in SConscript? > > [gw56@ws133 libtbx]$ svn commit -m "Reverting with extreme prejudice > hard coded 10.8 requirement, build no longer works on 10.7 and > earlier; please use ENV VAR method to achieve this kind of constraint; > reverted revision=22202" SConscript > > Sending SConscript > > Transmitting file data . > > Committed revision 22223. > > Please use MACOSX_DEPLOYMENT_TARGET=(version) locally for your > personal requirement, export this before kicking off bootstrap.py > > Best wishes Graeme > > ==================== DIFFS FOLLOW ==================== > > Revision: 22223 > http://sourceforge.net/p/cctbx/code/22223 [1] > Author: graeme_winter > Date: 2015-05-15 08:06:10 +0000 (Fri, 15 May 2015) > Log Message: > ----------- > Reverting with extreme prejudice hard coded 10.8 requirement, build no > longer works on 10.7 and earlier; please use ENV VAR method to achieve > this kind of constraint; reverted revision=22202 > > Modified Paths: > -------------- > trunk/libtbx/SConscript > > Modified: trunk/libtbx/SConscript > =================================================================== > --- trunk/libtbx/SConscript 2015-05-15 02:13:42 UTC (rev 22222) > +++ trunk/libtbx/SConscript 2015-05-15 08:06:10 UTC (rev 22223) > @@ -803,7 +803,7 @@ > env_etc.static_libraries = 1 > env_etc.static_bpl = 0 > > - link_flags = ["-w", "-mmacosx-version-min=10.8"] # suppress > "source/lib does not exist" warning > + link_flags = ["-w"] # suppress "source/lib does not exist" warning > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > link_flags.extend(["-arch", "i386"]) > @@ -811,8 +811,7 @@ > env_etc.c_link = "cc" > env_etc.ccflags_base = [ > "-fPIC", > - "-fno-strict-aliasing", > - "-mmacosx-version-min=10.8"] > + "-fno-strict-aliasing"] > if env_etc.clang_version: > setup_clang_warning(env_etc, libtbx.env.build_options.warning_level) > else: > @@ -828,7 +827,7 @@ > env_etc.ccflags_base.extend(["-no-cpp-precomp"]) > if (env_etc.gcc_version < 40201): > env_etc.ccflags_base.append("-Wno-long-double") > - env_etc.cxxflags_base = ["-mmacosx-version-min=10.8"] > + env_etc.cxxflags_base = [] > if env_etc.clang_version is None and env_etc.gcc_version < 40000: > env_etc.cxxflags_base.append("-fcoalesce-templates") > env_etc.cxxflags_base.append("-DBOOST_ALL_NO_LIB") > @@ -861,11 +860,10 @@ > if env_etc.apple_gcc_builds.gcc < 5646: > # fixed on 10.6 (Snow Leopard) whose Dev Tools start with that build > opts.insert(0, "-gstabs+") > - env_etc.shlinkflags = ["-mmacosx-version-min=10.8"] > + env_etc.shlinkflags = [] > env_etc.shlinkflags_bpl = [ > "-w", # suppress "source/lib does not exist" warning > - "-bundle", "-mmacosx-version-min=10.8", > - "-undefined", "dynamic_lookup"] > + "-bundle", "-undefined", "dynamic_lookup"] > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > env_etc.ccflags_base.extend(["-arch", "i386"]) > > FROM: Graeme.Winter(a)diamond.ac.uk<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] > > SENT: 15 May 2015 09:01 > TO: cctbxbb(a)phenix-online.org<mailto:[email protected]>; robert(a)oeffner.net<mailto:[email protected]> > CC: dials-support(a)lists.sourceforge.net<mailto:[email protected]>; dials-dev(a)cci.lbl.gov<mailto:[email protected]> > SUBJECT: Build constraints in SConscript? > > Hi Folks > > A while back we worked on making DIALS work right built on 10.10 but > executed on 10.6 or later – this is very useful as lots of people > have 10.6/7 macs & are not keen to update. This is done by the very > flexible mechanism of exporting an environment variable prior to > compilation > > Then this commit appears > > env_etc.static_bpl = 0 > @@ -802,7 +803,7 @@ > env_etc.static_libraries = 1 > env_etc.static_bpl = 0 > > - link_flags = ["-w"] # suppress "source/lib does not exist" warning > + link_flags = ["-w", "-mmacosx-version-min=10.8"] # suppress > "source/lib does not exist" warning > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > link_flags.extend(["-arch", "i386"]) > @@ -810,7 +811,8 @@ > env_etc.c_link = "cc" > env_etc.ccflags_base = [ > "-fPIC", > - "-fno-strict-aliasing"] > + "-fno-strict-aliasing", > + "-mmacosx-version-min=10.8"] > if env_etc.clang_version: > setup_clang_warning(env_etc, libtbx.env.build_options.warning_level) > else: > @@ -826,7 +828,7 @@ > env_etc.ccflags_base.extend(["-no-cpp-precomp"]) > if (env_etc.gcc_version < 40201): > env_etc.ccflags_base.append("-Wno-long-double") > - env_etc.cxxflags_base = [] > + env_etc.cxxflags_base = ["-mmacosx-version-min=10.8"] > if env_etc.clang_version is None and env_etc.gcc_version < 40000: > env_etc.cxxflags_base.append("-fcoalesce-templates") > env_etc.cxxflags_base.append("-DBOOST_ALL_NO_LIB") > @@ -859,10 +861,10 @@ > > to > > Modified Paths: > -------------- > trunk/libtbx/SConscript > > and now our build no longer works on 10.7. This is *REALLY* annoying. > The head of the commit message follows below. > > Please could someone fix this? The environment variable method is > *FAR* better, works for the dependencies and also allows dials to work > on older machines. You broke our build. Before this it worked fine. > > (we can’t automate this test as we are not supposed to deploy out of > support machines on lab network, so this was missed) > > Thanks Graeme > > EFFNER > > to cctbx-cvs > > May 13 > > Revision: 22202 > http://sourceforge.net/p/cctbx/code/22202 [2] > Author: oeffner > Date: 2015-05-13 15:48:24 +0000 (Wed, 13 May 2015) > Log Message: > ----------- > ensure installers can run on MacOSX 10.8 or higher > > -- > > This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended > addressee only. If you are not the intended addressee or an authorised > recipient of the addressee please notify us of receipt by returning > the e-mail and do not use, copy, retain, distribute or disclose the > information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for > any damage which you may sustain as a result of software viruses which > may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United > Kingdom > > No virus found in this message. > Checked by AVG - www.avg.com<http://www.avg.com> [3] > Version: 2015.0.5941 / Virus Database: 4342/9780 - Release Date: > 05/15/15 > > Links: > ------ > [1] http://sourceforge.net/p/cctbx/code/22223 > [2] http://sourceforge.net/p/cctbx/code/22202 > [3] http://www.avg.com -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Re: [cctbxbb] Fw: Build constraints in SConscript?
by Graeme.Winter＠diamond.ac.uk 16 May '15

16 May '15

Dear Rob Given that the tests are currently not working right anyway, and there are no build / deployment tests for testing builds on an older version of OS X than they are compiled on, this is an easy problem to get in to. It's also a collaborative project where we're all working hard on making things work. This will happen from time to time. I had thought the same environment variable based mechanism for the builds was already in place for PHENIX - am surprised it is not! So, no worries. Perhaps I came across more grumpy than I meant to below. Occurs to me it would be very handy to have one email address for all cctbx related developers where (someone) could ask (something like this) and get an answer perhaps suggesting a more optimum solution? Thoughts anyone? (lots of mail lists in send: list!) Thanks & best wishes Graeme -----Original Message----- From: R.D. Oeffner [mailto:[email protected]] Sent: 15 May 2015 10:26 To: Winter, Graeme (DLSLtd,RAL,LSCI); cctbxbb(a)phenix-online.org; dials-support(a)lists.sourceforge.net; dials-dev(a)cci.lbl.gov; Nigel W Moriarty Subject: Re: Fw: Build constraints in SConscript? Dear Graeme, I'm sorry for the trouble my change caused and the time taken to identify it. Ironically we had the same issue as you with Phenix not installing on anything lower than OSX 10.9. As nothing appeared to be specified in our build system to that effect my change was a quick fix to address the problem. It was genuinely well intended. Nigel, I defer to you for implementing the minimum OSX platform for Phenix builds. Rob > FROM: Graeme.Winter(a)Diamond.ac.uk > SENT: Friday, May 15, 2015 9:09 AM > TO: Graeme.Winter(a)Diamond.ac.uk ; cctbxbb(a)phenix-online.org ; > robert(a)oeffner.net > CC: dials-support(a)lists.sourceforge.net ; dials-dev(a)cci.lbl.gov > SUBJECT: RE: Build constraints in SConscript? > > [gw56@ws133 libtbx]$ svn commit -m "Reverting with extreme prejudice > hard coded 10.8 requirement, build no longer works on 10.7 and > earlier; please use ENV VAR method to achieve this kind of constraint; > reverted revision=22202" SConscript > > Sending SConscript > > Transmitting file data . > > Committed revision 22223. > > Please use MACOSX_DEPLOYMENT_TARGET=(version) locally for your > personal requirement, export this before kicking off bootstrap.py > > Best wishes Graeme > > ==================== DIFFS FOLLOW ==================== > > Revision: 22223 > http://sourceforge.net/p/cctbx/code/22223 [1] > Author: graeme_winter > Date: 2015-05-15 08:06:10 +0000 (Fri, 15 May 2015) > Log Message: > ----------- > Reverting with extreme prejudice hard coded 10.8 requirement, build no > longer works on 10.7 and earlier; please use ENV VAR method to achieve > this kind of constraint; reverted revision=22202 > > Modified Paths: > -------------- > trunk/libtbx/SConscript > > Modified: trunk/libtbx/SConscript > =================================================================== > --- trunk/libtbx/SConscript 2015-05-15 02:13:42 UTC (rev 22222) > +++ trunk/libtbx/SConscript 2015-05-15 08:06:10 UTC (rev 22223) > @@ -803,7 +803,7 @@ > env_etc.static_libraries = 1 > env_etc.static_bpl = 0 > > - link_flags = ["-w", "-mmacosx-version-min=10.8"] # suppress > "source/lib does not exist" warning > + link_flags = ["-w"] # suppress "source/lib does not exist" warning > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > link_flags.extend(["-arch", "i386"]) > @@ -811,8 +811,7 @@ > env_etc.c_link = "cc" > env_etc.ccflags_base = [ > "-fPIC", > - "-fno-strict-aliasing", > - "-mmacosx-version-min=10.8"] > + "-fno-strict-aliasing"] > if env_etc.clang_version: > setup_clang_warning(env_etc, libtbx.env.build_options.warning_level) > else: > @@ -828,7 +827,7 @@ > env_etc.ccflags_base.extend(["-no-cpp-precomp"]) > if (env_etc.gcc_version < 40201): > env_etc.ccflags_base.append("-Wno-long-double") > - env_etc.cxxflags_base = ["-mmacosx-version-min=10.8"] > + env_etc.cxxflags_base = [] > if env_etc.clang_version is None and env_etc.gcc_version < 40000: > env_etc.cxxflags_base.append("-fcoalesce-templates") > env_etc.cxxflags_base.append("-DBOOST_ALL_NO_LIB") > @@ -861,11 +860,10 @@ > if env_etc.apple_gcc_builds.gcc < 5646: > # fixed on 10.6 (Snow Leopard) whose Dev Tools start with that build > opts.insert(0, "-gstabs+") > - env_etc.shlinkflags = ["-mmacosx-version-min=10.8"] > + env_etc.shlinkflags = [] > env_etc.shlinkflags_bpl = [ > "-w", # suppress "source/lib does not exist" warning > - "-bundle", "-mmacosx-version-min=10.8", > - "-undefined", "dynamic_lookup"] > + "-bundle", "-undefined", "dynamic_lookup"] > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > env_etc.ccflags_base.extend(["-arch", "i386"]) > > FROM: Graeme.Winter(a)diamond.ac.uk [mailto:[email protected]] > > SENT: 15 May 2015 09:01 > TO: cctbxbb(a)phenix-online.org; robert(a)oeffner.net > CC: dials-support(a)lists.sourceforge.net; dials-dev(a)cci.lbl.gov > SUBJECT: Build constraints in SConscript? > > Hi Folks > > A while back we worked on making DIALS work right built on 10.10 but > executed on 10.6 or later – this is very useful as lots of people > have 10.6/7 macs & are not keen to update. This is done by the very > flexible mechanism of exporting an environment variable prior to > compilation > > Then this commit appears > > env_etc.static_bpl = 0 > @@ -802,7 +803,7 @@ > env_etc.static_libraries = 1 > env_etc.static_bpl = 0 > > - link_flags = ["-w"] # suppress "source/lib does not exist" warning > + link_flags = ["-w", "-mmacosx-version-min=10.8"] # suppress > "source/lib does not exist" warning > if libtbx.env.build_options.force_32bit: > if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or > env_etc.clang_version: > link_flags.extend(["-arch", "i386"]) > @@ -810,7 +811,8 @@ > env_etc.c_link = "cc" > env_etc.ccflags_base = [ > "-fPIC", > - "-fno-strict-aliasing"] > + "-fno-strict-aliasing", > + "-mmacosx-version-min=10.8"] > if env_etc.clang_version: > setup_clang_warning(env_etc, libtbx.env.build_options.warning_level) > else: > @@ -826,7 +828,7 @@ > env_etc.ccflags_base.extend(["-no-cpp-precomp"]) > if (env_etc.gcc_version < 40201): > env_etc.ccflags_base.append("-Wno-long-double") > - env_etc.cxxflags_base = [] > + env_etc.cxxflags_base = ["-mmacosx-version-min=10.8"] > if env_etc.clang_version is None and env_etc.gcc_version < 40000: > env_etc.cxxflags_base.append("-fcoalesce-templates") > env_etc.cxxflags_base.append("-DBOOST_ALL_NO_LIB") > @@ -859,10 +861,10 @@ > > to > > Modified Paths: > -------------- > trunk/libtbx/SConscript > > and now our build no longer works on 10.7. This is *REALLY* annoying. > The head of the commit message follows below. > > Please could someone fix this? The environment variable method is > *FAR* better, works for the dependencies and also allows dials to work > on older machines. You broke our build. Before this it worked fine. > > (we can’t automate this test as we are not supposed to deploy out of > support machines on lab network, so this was missed) > > Thanks Graeme > > EFFNER > > to cctbx-cvs > > May 13 > > Revision: 22202 > http://sourceforge.net/p/cctbx/code/22202 [2] > Author: oeffner > Date: 2015-05-13 15:48:24 +0000 (Wed, 13 May 2015) > Log Message: > ----------- > ensure installers can run on MacOSX 10.8 or higher > > -- > > This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended > addressee only. If you are not the intended addressee or an authorised > recipient of the addressee please notify us of receipt by returning > the e-mail and do not use, copy, retain, distribute or disclose the > information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for > any damage which you may sustain as a result of software viruses which > may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United > Kingdom > > No virus found in this message. > Checked by AVG - www.avg.com [3] > Version: 2015.0.5941 / Virus Database: 4342/9780 - Release Date: > 05/15/15 > > Links: > ------ > [1] http://sourceforge.net/p/cctbx/code/22223 > [2] http://sourceforge.net/p/cctbx/code/22202 > [3] http://www.avg.com -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Build constraints in SConscript?
by Graeme.Winter＠diamond.ac.uk 15 May '15

15 May '15

Hi Folks A while back we worked on making DIALS work right built on 10.10 but executed on 10.6 or later - this is very useful as lots of people have 10.6/7 macs & are not keen to update. This is done by the very flexible mechanism of exporting an environment variable prior to compilation Then this commit appears env_etc.static_bpl = 0 @@ -802,7 +803,7 @@ env_etc.static_libraries = 1 env_etc.static_bpl = 0 - link_flags = ["-w"] # suppress "source/lib does not exist" warning + link_flags = ["-w", "-mmacosx-version-min=10.8"] # suppress "source/lib does not exist" warning if libtbx.env.build_options.force_32bit: if (env_etc.gcc_version and env_etc.gcc_version >= 40201) or env_etc.clang_version: link_flags.extend(["-arch", "i386"]) @@ -810,7 +811,8 @@ env_etc.c_link = "cc" env_etc.ccflags_base = [ "-fPIC", - "-fno-strict-aliasing"] + "-fno-strict-aliasing", + "-mmacosx-version-min=10.8"] if env_etc.clang_version: setup_clang_warning(env_etc, libtbx.env.build_options.warning_level) else: @@ -826,7 +828,7 @@ env_etc.ccflags_base.extend(["-no-cpp-precomp"]) if (env_etc.gcc_version < 40201): env_etc.ccflags_base.append("-Wno-long-double") - env_etc.cxxflags_base = [] + env_etc.cxxflags_base = ["-mmacosx-version-min=10.8"] if env_etc.clang_version is None and env_etc.gcc_version < 40000: env_etc.cxxflags_base.append("-fcoalesce-templates") env_etc.cxxflags_base.append("-DBOOST_ALL_NO_LIB") @@ -859,10 +861,10 @@ to Modified Paths: -------------- trunk/libtbx/SConscript and now our build no longer works on 10.7. This is *really* annoying. The head of the commit message follows below. Please could someone fix this? The environment variable method is *far* better, works for the dependencies and also allows dials to work on older machines. You broke our build. Before this it worked fine. (we can't automate this test as we are not supposed to deploy out of support machines on lab network, so this was missed) Thanks Graeme effner to cctbx-cvs [https://ssl.gstatic.com/bt/C3341AA7A1A076756462EE2E5CD71C11/2x/ic_expand_sm…] May 13 Revision: 22202 http://sourceforge.net/p/cctbx/code/22202 Author: oeffner Date: 2015-05-13 15:48:24 +0000 (Wed, 13 May 2015) Log Message: ----------- ensure installers can run on MacOSX 10.8 or higher -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 1

Does cctbx have an easy to access spherical harmonic table?
by Graeme.Winter＠diamond.ac.uk 10 May '15

10 May '15

Hi Folks, Looked and something is used from boost but they don’t appear to be clearly wrapped to Python. in scitbx.math.nss_spherical_harmonics found nearly what I was looking for but it seems to be for some other purpose & not really do what I want (and needs mystery things in constructor….) >>> _sh = sh(2, 1) Traceback (most recent call last): File "<stdin>", line 1, in <module> Boost.Python.ArgumentError: Python argument types in nss_spherical_harmonics.__init__(nss_spherical_harmonics, int, int) did not match C++ signature: __init__(_object*, int l_max, int mangle, scitbx::math::zernike::log_factorial_generator<double> lgf) Any clues would be welcome, else wise most sensible thing seems to be to wrap up to Python the boost spherical harmonic code… Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

bin data by a Miller index
by Mooers, Blaine H.M. (HSC) 25 Apr '15

25 Apr '15

Dear cctbxbb: I have a simple question. I have a data set with a strong off origin Patterson peak (0,0,0.11). I want to plot the average I/sig per zone l = n. I expect to see enhanced values at l=9n. How do you bin data by the L Miller index in iotbx? Best regards, Blaine Blaine Mooers, Ph.D. Assistant Professor Director of the Laboratory of Biomolecular Structure and Function Department of Biochemistry and Molecular Biology University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center Rm. 466 Shipping address: 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 Letter address: P.O. Box 26901, BRC 466 Oklahoma City, OK 73190 office: (405) 271-8300 lab: (405) 271-8313 fax: (405) 271-3910 e-mail: blaine-mooers(a)ouhsc.edu Faculty webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/… Small Angle Scattering webpage: http://www.oumedicine.com/docs/default-source/ad-biochemistry-workfiles/sma… X-ray lab webpage: http://www.oumedicine.com/department-of-biochemistry-and-molecular-biology/…

2 1

scitbx test crashing on mac
by markus.gerstel＠diamond.ac.uk 14 Apr '15

14 Apr '15

Good morning, The test scitbx.rigid_body.proto.tst_singular has been failing consistently on our MacOS 10.10 system for a day now. Last known good svn rev. is 21989. It passes on the linux systems. Apologies if this is not the right place to report scitbx problems to. -Markus Stacktrace 11 Python 0x0000000100010d03 PyObject_Call + 99 10 libboost_python.dylib 0x0000000101a0f893 boost::python::objects::function_call(_object*, _object*, _object*) + 83 9 libboost_python.dylib 0x0000000101a147f1 boost::python::handle_exception_impl(boost::function0<void>) + 81 8 libboost_python.dylib 0x0000000101a0fe5a boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 26 7 libboost_python.dylib 0x0000000101a0db6a boost::python::objects::function::call(_object*, _object*) const + 826 6 scitbx_linalg_ext.so 0x000000010518b521 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::default_call_policies, boost::mpl::vector2<scitbx::af::shared<double>, scitbx::matrix::eigensystem::real_symmetric<double>&> > >::operator()(_object*, _object*) + 65 5 scitbx_linalg_ext.so 0x000000010518b6be _object* boost::python::detail::invoke<boost::python::to_python_value<scitbx::af::shared<double> const&>, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::arg_from_python<scitbx::matrix::eigensystem::real_symmetric<double>&> >(boost::python::detail::invoke_tag_<false, true>, boost::python::to_python_value<scitbx::af::shared<double> const&> const&, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*&)() const, boost::python::arg_from_python<scitbx::matrix::eigensystem::real_symmetric<double>&>&) + 46 4 ??? 0x000000010183fc20 0x0 + 4320394272 3 libsystem_platform.dylib 0x00007fff86f16f1a _sigtramp + 26 2 boost_python_meta_ext.so 0x00000001005bdf80 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. Standard Error show_stack(1): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/featherstone.py(38) generalized_inverse show_stack(2): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/featherstone.py(373) FDab show_stack(3): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/test_simulation.py(28) energies_and_accelerations_update show_stack(4): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/test_simulation.py(16) __init__ show_stack(5): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(72) exercise_six_dof show_stack(6): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(146) run show_stack(7): /Users/dlshudson/jenkins_slave/workspace/cctbx_tests_platforms/label/dials-mac-mini/build_dials/modules/cctbx_project/scitbx/rigid_body/proto/tst_singular.py(162) <module> libc backtrace (43 frames, most recent call last): 44 ??? 0x0000000000000003 0x0 + 3 43 Python 0x0000000100000d71 Python + 3441 42 Python 0x0000000100000e58 Python + 3672 41 Python 0x00000001000edd44 Py_Main + 3076 40 Python 0x00000001000d64a7 PyRun_SimpleFileExFlags + 711 39 Python 0x00000001000d6984 PyRun_FileExFlags + 164 38 Python 0x00000001000ad596 PyEval_EvalCode + 54 37 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 36 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 35 Python 0x00000001000b5556 fast_function + 118 34 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 33 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 32 Python 0x00000001000b5556 fast_function + 118 31 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 30 Python 0x00000001000b295b PyEval_EvalFrameEx + 18651 29 Python 0x0000000100010d03 PyObject_Call + 99 28 Python 0x0000000100069e23 type_call + 339 27 Python 0x000000010006fccd slot_tp_init + 141 26 Python 0x0000000100010d03 PyObject_Call + 99 25 Python 0x000000010001e207 instancemethod_call + 247 24 Python 0x0000000100010d03 PyObject_Call + 99 23 Python 0x0000000100036a6c function_call + 364 22 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 21 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 20 Python 0x00000001000b5556 fast_function + 118 19 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 18 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 17 Python 0x00000001000b5556 fast_function + 118 16 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 15 Python 0x00000001000b179f PyEval_EvalFrameEx + 14111 14 Python 0x00000001000b5556 fast_function + 118 13 Python 0x00000001000adc22 PyEval_EvalCodeEx + 1666 12 Python 0x00000001000b295b PyEval_EvalFrameEx + 18651 11 Python 0x0000000100010d03 PyObject_Call + 99 10 libboost_python.dylib 0x0000000101a0f893 boost::python::objects::function_call(_object*, _object*, _object*) + 83 9 libboost_python.dylib 0x0000000101a147f1 boost::python::handle_exception_impl(boost::function0<void>) + 81 8 libboost_python.dylib 0x0000000101a0fe5a boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 26 7 libboost_python.dylib 0x0000000101a0db6a boost::python::objects::function::call(_object*, _object*) const + 826 6 scitbx_linalg_ext.so 0x000000010518b521 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::default_call_policies, boost::mpl::vector2<scitbx::af::shared<double>, scitbx::matrix::eigensystem::real_symmetric<double>&> > >::operator()(_object*, _object*) + 65 5 scitbx_linalg_ext.so 0x000000010518b6be _object* boost::python::detail::invoke<boost::python::to_python_value<scitbx::af::shared<double> const&>, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*)() const, boost::python::arg_from_python<scitbx::matrix::eigensystem::real_symmetric<double>&> >(boost::python::detail::invoke_tag_<false, true>, boost::python::to_python_value<scitbx::af::shared<double> const&> const&, scitbx::af::shared<double> (scitbx::matrix::eigensystem::real_symmetric<double>::*&)() const, boost::python::arg_from_python<scitbx::matrix::eigensystem::real_symmetric<double>&>&) + 46 4 ??? 0x000000010183fc20 0x0 + 4320394272 3 libsystem_platform.dylib 0x00007fff86f16f1a _sigtramp + 26 2 boost_python_meta_ext.so 0x00000001005bdf80 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

5 9

bootstrap script broken - more rotamer goodness
by markus.gerstel＠diamond.ac.uk 08 Apr '15

08 Apr '15

Hi, We received an error report from a dials user that he could not install dials from source. This appears to be caused by the mmtbx.rebuild_rotarama_cache command run after the compilation, which has been introduced in http://sourceforge.net/p/cctbx/code/21921/ I guess this needs either to be run conditional on the library being installed or be changed to fail gracefully. -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 3

Re: [cctbxbb] [Dials-support] DIALS source tarball
by Graeme.Winter＠diamond.ac.uk 08 Apr '15

08 Apr '15

Sadly this also does not fix anything :o( not sure why this was not flagged when I configured chem_data... libtbx.python "/Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py" [FAIL] Time: 6.44 Return code: 1 OKs: 0 Standard error: Traceback (most recent call last): File "/Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 866, in <module> exercise_all(sys.argv[1:]) File "/Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 862, in exercise_all exercise_na_restraints_output_to_geo(verbose=verbose) File "/Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 818, in exercise_na_restraints_output_to_geo log=out2) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 5274, in run nonbonded_distance_threshold=nonbonded_distance_threshold) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 4952, in geometry_restraints_manager ss_manager.initialize(log=self.log) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/secondary_structure/__init__.py", line 280, in initialize self.find_automatically(log=log) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/secondary_structure/__init__.py", line 313, in find_automatically self.sec_str_from_pdb_file = self.find_sec_str(log=log) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/secondary_structure/__init__.py", line 369, in find_sec_str (records, stderr) = run_ksdssp_direct(pdb_str) File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/secondary_structure/__init__.py", line 728, in run_ksdssp_direct exe_path = get_ksdssp_exe_path() File "/Users/graeme/svn/cctbx/modules/cctbx_project/mmtbx/secondary_structure/__init__.py", line 709, in get_ksdssp_exe_path raise RuntimeError("ksdssp module is not configured") RuntimeError: ksdssp module is not configured On 8 Apr 2015, at 11:02, <richard.gildea(a)diamond.ac.uk<mailto:[email protected]>> <richard.gildea(a)diamond.ac.uk<mailto:[email protected]>> wrote: $ du -h --max-depth=0 chem_data 1.6G chem_data Not sure we really want to include this in the dials builds, just to make one test happy, especially when we have no use for any of the monomer library functionality in dials. Cheers, Richard ________________________________ From: Graeme Winter [graeme.winter(a)gmail.com<mailto:[email protected]>] Sent: 08 April 2015 10:58 To: Gildea, Richard (DLSLtd,RAL,LSCI); Gerstel, Markus (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov<mailto:[email protected]>; cctbxbb(a)phenix-online.org<mailto:[email protected]> Cc: dials-support(a)lists.sourceforge.net<mailto:[email protected]> Subject: Re: [Dials-support] [cctbxbb] DIALS source tarball can we add this to the dials build through some kind of tarball download which does not need a cci account? Thanks graeme On Wed, Apr 8, 2015 at 10:52 AM <richard.gildea(a)diamond.ac.uk<mailto:[email protected]><mailto:[email protected]>> wrote: Hi Markus, I agree that it would be better if the tests would either work gracefully with CCP4 monomer library, or (possibly better?) only run if chem_data is available. In the latter case, a simple libtbx.env.has_module("chem_data") could be used in the test to check whether chem_data is available. As a workaround, we developers should be able to add the chem_data repository to our sources after which the tests should pick up the correct version of the monomer library: $ svn co svn+ssh://[email protected]/chem_data/trunk<http://[email protected]/chem_data/trunk> chem_data $ libtbx.configure chem_data Cheers, Richard ________________________________________ From: Gerstel, Markus (DLSLtd,RAL,LSCI) Sent: 08 April 2015 09:33 To: Gildea, Richard (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov<mailto:[email protected]><mailto:[email protected]>; cctbxbb(a)phenix-online.org<mailto:[email protected]><mailto:[email protected]> Cc: dials-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]> Subject: RE: [Dials-support] [cctbxbb] DIALS source tarball Hi Richard, I think you've found it. I have unset CLIBD_MON, and now the test skips cleanly. Of course this also means that users should not run cctbx tests on their machines... Is there no central 'mmtbx_skip_test_if_chem_data_db_missing' function, that just goes 'prints "Skip this test"; exit' if the library isn't present? Or a boolean _is_available() function if you prefer. Either would resolve this problem as well as the current untestability of mmtbx. -Markus -----Original Message----- From: Gildea, Richard (DLSLtd,RAL,LSCI) Sent: 08 April 2015 09:15 To: Gerstel, Markus (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov<mailto:[email protected]><mailto:[email protected]>; cctbxbb(a)phenix-online.org<mailto:[email protected]><mailto:[email protected]> Cc: dials-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]> Subject: RE: [Dials-support] [cctbxbb] DIALS source tarball Hi Markus, Presumably the test is looking for a monomer that is present in chem_data but not in the ccp4 monomer library? Could we unset the CLIBD_MON variable for the purposes of testing the dials installers? Cheers, Richard ________________________________________ From: Gerstel, Markus (DLSLtd,RAL,LSCI) Sent: 08 April 2015 09:07 To: Gildea, Richard (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov<mailto:[email protected]><mailto:[email protected]>; cctbxbb(a)phenix-online.org<mailto:[email protected]><mailto:[email protected]> Cc: dials-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]> Subject: RE: [Dials-support] [cctbxbb] DIALS source tarball Hi Richard, Aaron, This is correct, we do not currently run mmtbx tests at all, as up to 107 of them do not fail gracefully, ie. skip properly. The test in question works fine (ie: is skipped because CCP4 is not made available) on our cctbx test system, but breaks in our final dials installer tests (where CCP4 is made available). CLIBD_MON is set to the /lib/data/monomers folder in the shared DLS CCP4 6.5 update 1 installation. Best wishes, Markus -----Original Message----- From: richard.gildea(a)diamond.ac.uk<mailto:[email protected]><mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] Sent: 07 April 2015 21:36 To: asbrewster(a)lbl.gov<mailto:[email protected]><mailto:[email protected]>; cctbxbb(a)phenix-online.org<mailto:[email protected]><mailto:[email protected]> Cc: dials-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]> Subject: Re: [Dials-support] [cctbxbb] DIALS source tarball Hi Aaron, I'm not sure your fix as I understand it actually helps. The test that was failing was in the cctbx module, not mmtbx, and as far as I am aware we are running libtbx.run_tests_parallel module=cctbx module=dxtbx etc in our nightly builds (and that is what we as developers are running locally on our own development machines). If individual tests (particularly those in the more general modules such as cctbx, iotbx, etc.) rely on external dependencies (such as chem_data) being available, then the individual tests should test for presence of said dependencies and skip gracefully if not available. I have a feeling that we're not even including mmtbx in our nightly build tests (too many fail due to missing dependencies such as chem_data). Cheers, Richard ________________________________ From: Aaron Brewster [asbrewster(a)lbl.gov<mailto:[email protected]><mailto:[email protected]>] Sent: 07 April 2015 20:29 To: cctbx mailing list Cc: Waterman, David (STFC,RAL,SC); Gildea, Richard (DLSLtd,RAL,LSCI); Johan Hattne; <dials-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]>> Subject: Re: [cctbxbb] [Dials-support] DIALS source tarball Hi Markus, two points. First, you are correct that the dials builds are not passing tests related to the chem_data folder which includes monomer libraries. This is because chem_data is required by mmtbx but isn't included in the dials build. Partly because we're unsure if pieces of it are proprietary. In the meantime, I've worked with Nigel to change the cctbx_regression.test_nightly to look for the chem_data repository and skip the mmtbx tests if it's not present. mmtbx will still be tested nightly as part of the standalone cctbx and phenix packages. Because of this change, perhaps you could revert your recent changes to cctbx/regression/tst_geometry_restraints_2.py? It isn't necessary to check for chem_data in every sub test as it is tested for at the top level now. Second, the log you provided is not the one on buildbot. Your log indicates you have a CCP4 installation that includes a monomer library that is being picked up by the tests. Is one of the MMTBX_CCP4_MONOMER_LIB, or CLIBD_MON environment variables set on the system in question? Thanks, -Aaron On Tue, Apr 7, 2015 at 6:41 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]><mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: Cc'ing cctbx folks David Waterman wrote: So maybe once we have a source tarball up at dials.diamond.ac.uk<http://dials.diamond.ac.uk><http://dials.diamond.ac.uk><http://dials.diamond.ac.uk> (..) Speaking of which: our DIALS builds are currently failing at the installer testing stage. This is due to cctbx/regression/tst_geometry_restraints_2.py failing deep in mmtbx territory. So far I could not figure out how to get this running properly. Call: libtbx.python tst_geometry_restraints_2.py Standard Output Skipping exercise_with_zeolite(): input file not available Skipping exercise_with_pdb(): chem_data directory not available Standard Error Traceback (most recent call last): File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 866, in <module> exercise_all(sys.argv[1:]) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 862, in exercise_all exercise_na_restraints_output_to_geo(verbose=verbose) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 805, in exercise_na_restraints_output_to_geo log=out1) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 5270, in run log=log) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 4876, in __init__ restraints_loading_flags=restraints_loading_flags, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2902, in __init__ log=log, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2287, in __init__ m_j=mm) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 1513, in get_lib_link return mon_lib_srv.link_link_id_dict["rna3p"] KeyError: 'rna3p' (On a side note: We really should call multiple tests within the same python file separately, instead of running them together within a signle run() method. This file contains 4 tests, but these will result in only a single return value for any upstream processing of the test results, such as Jenkins.) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom ------------------------------------------------------------------------------ BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF _______________________________________________ DIALS-support mailing list DIALS-support(a)lists.sourceforge.net<mailto:[email protected]><mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/dials-support

1 0

Re: [cctbxbb] [Dials-support] DIALS source tarball
by markus.gerstel＠diamond.ac.uk 08 Apr '15

08 Apr '15

Hi Richard, I think you've found it. I have unset CLIBD_MON, and now the test skips cleanly. Of course this also means that users should not run cctbx tests on their machines... Is there no central 'mmtbx_skip_test_if_chem_data_db_missing' function, that just goes 'prints "Skip this test"; exit' if the library isn't present? Or a boolean _is_available() function if you prefer. Either would resolve this problem as well as the current untestability of mmtbx. -Markus -----Original Message----- From: Gildea, Richard (DLSLtd,RAL,LSCI) Sent: 08 April 2015 09:15 To: Gerstel, Markus (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov; cctbxbb(a)phenix-online.org Cc: dials-support(a)lists.sourceforge.net Subject: RE: [Dials-support] [cctbxbb] DIALS source tarball Hi Markus, Presumably the test is looking for a monomer that is present in chem_data but not in the ccp4 monomer library? Could we unset the CLIBD_MON variable for the purposes of testing the dials installers? Cheers, Richard ________________________________________ From: Gerstel, Markus (DLSLtd,RAL,LSCI) Sent: 08 April 2015 09:07 To: Gildea, Richard (DLSLtd,RAL,LSCI); asbrewster(a)lbl.gov; cctbxbb(a)phenix-online.org Cc: dials-support(a)lists.sourceforge.net Subject: RE: [Dials-support] [cctbxbb] DIALS source tarball Hi Richard, Aaron, This is correct, we do not currently run mmtbx tests at all, as up to 107 of them do not fail gracefully, ie. skip properly. The test in question works fine (ie: is skipped because CCP4 is not made available) on our cctbx test system, but breaks in our final dials installer tests (where CCP4 is made available). CLIBD_MON is set to the /lib/data/monomers folder in the shared DLS CCP4 6.5 update 1 installation. Best wishes, Markus -----Original Message----- From: richard.gildea(a)diamond.ac.uk [mailto:[email protected]] Sent: 07 April 2015 21:36 To: asbrewster(a)lbl.gov; cctbxbb(a)phenix-online.org Cc: dials-support(a)lists.sourceforge.net Subject: Re: [Dials-support] [cctbxbb] DIALS source tarball Hi Aaron, I'm not sure your fix as I understand it actually helps. The test that was failing was in the cctbx module, not mmtbx, and as far as I am aware we are running libtbx.run_tests_parallel module=cctbx module=dxtbx etc in our nightly builds (and that is what we as developers are running locally on our own development machines). If individual tests (particularly those in the more general modules such as cctbx, iotbx, etc.) rely on external dependencies (such as chem_data) being available, then the individual tests should test for presence of said dependencies and skip gracefully if not available. I have a feeling that we're not even including mmtbx in our nightly build tests (too many fail due to missing dependencies such as chem_data). Cheers, Richard ________________________________ From: Aaron Brewster [asbrewster(a)lbl.gov] Sent: 07 April 2015 20:29 To: cctbx mailing list Cc: Waterman, David (STFC,RAL,SC); Gildea, Richard (DLSLtd,RAL,LSCI); Johan Hattne; <dials-support(a)lists.sourceforge.net> Subject: Re: [cctbxbb] [Dials-support] DIALS source tarball Hi Markus, two points. First, you are correct that the dials builds are not passing tests related to the chem_data folder which includes monomer libraries. This is because chem_data is required by mmtbx but isn't included in the dials build. Partly because we're unsure if pieces of it are proprietary. In the meantime, I've worked with Nigel to change the cctbx_regression.test_nightly to look for the chem_data repository and skip the mmtbx tests if it's not present. mmtbx will still be tested nightly as part of the standalone cctbx and phenix packages. Because of this change, perhaps you could revert your recent changes to cctbx/regression/tst_geometry_restraints_2.py? It isn't necessary to check for chem_data in every sub test as it is tested for at the top level now. Second, the log you provided is not the one on buildbot. Your log indicates you have a CCP4 installation that includes a monomer library that is being picked up by the tests. Is one of the MMTBX_CCP4_MONOMER_LIB, or CLIBD_MON environment variables set on the system in question? Thanks, -Aaron On Tue, Apr 7, 2015 at 6:41 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> wrote: Cc'ing cctbx folks David Waterman wrote: > So maybe once we have a source tarball up at > dials.diamond.ac.uk<http://dials.diamond.ac.uk> (..) Speaking of which: our DIALS builds are currently failing at the installer testing stage. This is due to cctbx/regression/tst_geometry_restraints_2.py failing deep in mmtbx territory. So far I could not figure out how to get this running properly. Call: libtbx.python tst_geometry_restraints_2.py Standard Output Skipping exercise_with_zeolite(): input file not available Skipping exercise_with_pdb(): chem_data directory not available Standard Error Traceback (most recent call last): File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 866, in <module> exercise_all(sys.argv[1:]) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 862, in exercise_all exercise_na_restraints_output_to_geo(verbose=verbose) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 805, in exercise_na_restraints_output_to_geo log=out1) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 5270, in run log=log) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 4876, in __init__ restraints_loading_flags=restraints_loading_flags, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2902, in __init__ log=log, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2287, in __init__ m_j=mm) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 1513, in get_lib_link return mon_lib_srv.link_link_id_dict["rna3p"] KeyError: 'rna3p' (On a side note: We really should call multiple tests within the same python file separately, instead of running them together within a signle run() method. This file contains 4 tests, but these will result in only a single return value for any upstream processing of the test results, such as Jenkins.) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Re: [cctbxbb] [Dials-support] DIALS source tarball
by markus.gerstel＠diamond.ac.uk 08 Apr '15

08 Apr '15

Hi Richard, Aaron, This is correct, we do not currently run mmtbx tests at all, as up to 107 of them do not fail gracefully, ie. skip properly. The test in question works fine (ie: is skipped because CCP4 is not made available) on our cctbx test system, but breaks in our final dials installer tests (where CCP4 is made available). CLIBD_MON is set to the /lib/data/monomers folder in the shared DLS CCP4 6.5 update 1 installation. Best wishes, Markus -----Original Message----- From: richard.gildea(a)diamond.ac.uk [mailto:[email protected]] Sent: 07 April 2015 21:36 To: asbrewster(a)lbl.gov; cctbxbb(a)phenix-online.org Cc: dials-support(a)lists.sourceforge.net Subject: Re: [Dials-support] [cctbxbb] DIALS source tarball Hi Aaron, I'm not sure your fix as I understand it actually helps. The test that was failing was in the cctbx module, not mmtbx, and as far as I am aware we are running libtbx.run_tests_parallel module=cctbx module=dxtbx etc in our nightly builds (and that is what we as developers are running locally on our own development machines). If individual tests (particularly those in the more general modules such as cctbx, iotbx, etc.) rely on external dependencies (such as chem_data) being available, then the individual tests should test for presence of said dependencies and skip gracefully if not available. I have a feeling that we're not even including mmtbx in our nightly build tests (too many fail due to missing dependencies such as chem_data). Cheers, Richard ________________________________ From: Aaron Brewster [asbrewster(a)lbl.gov] Sent: 07 April 2015 20:29 To: cctbx mailing list Cc: Waterman, David (STFC,RAL,SC); Gildea, Richard (DLSLtd,RAL,LSCI); Johan Hattne; <dials-support(a)lists.sourceforge.net> Subject: Re: [cctbxbb] [Dials-support] DIALS source tarball Hi Markus, two points. First, you are correct that the dials builds are not passing tests related to the chem_data folder which includes monomer libraries. This is because chem_data is required by mmtbx but isn't included in the dials build. Partly because we're unsure if pieces of it are proprietary. In the meantime, I've worked with Nigel to change the cctbx_regression.test_nightly to look for the chem_data repository and skip the mmtbx tests if it's not present. mmtbx will still be tested nightly as part of the standalone cctbx and phenix packages. Because of this change, perhaps you could revert your recent changes to cctbx/regression/tst_geometry_restraints_2.py? It isn't necessary to check for chem_data in every sub test as it is tested for at the top level now. Second, the log you provided is not the one on buildbot. Your log indicates you have a CCP4 installation that includes a monomer library that is being picked up by the tests. Is one of the MMTBX_CCP4_MONOMER_LIB, or CLIBD_MON environment variables set on the system in question? Thanks, -Aaron On Tue, Apr 7, 2015 at 6:41 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> wrote: Cc'ing cctbx folks David Waterman wrote: > So maybe once we have a source tarball up at > dials.diamond.ac.uk<http://dials.diamond.ac.uk> (..) Speaking of which: our DIALS builds are currently failing at the installer testing stage. This is due to cctbx/regression/tst_geometry_restraints_2.py failing deep in mmtbx territory. So far I could not figure out how to get this running properly. Call: libtbx.python tst_geometry_restraints_2.py Standard Output Skipping exercise_with_zeolite(): input file not available Skipping exercise_with_pdb(): chem_data directory not available Standard Error Traceback (most recent call last): File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 866, in <module> exercise_all(sys.argv[1:]) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 862, in exercise_all exercise_na_restraints_output_to_geo(verbose=verbose) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/cctbx/regression/tst_geometry_restraints_2.py", line 805, in exercise_na_restraints_output_to_geo log=out1) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 5270, in run log=log) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 4876, in __init__ restraints_loading_flags=restraints_loading_flags, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2902, in __init__ log=log, File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 2287, in __init__ m_j=mm) File "/scratch/jenkins_slave/workspace/dials_test_installer/dials-dev/modules/cctbx_project/mmtbx/monomer_library/pdb_interpretation.py", line 1513, in get_lib_link return mon_lib_srv.link_link_id_dict["rna3p"] KeyError: 'rna3p' (On a side note: We really should call multiple tests within the same python file separately, instead of running them together within a signle run() method. This file contains 4 tests, but these will result in only a single return value for any upstream processing of the test results, such as Jenkins.) -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0