October 2020 - cctbxbb - phenix-online.org

Re: [cctbxbb] [git/cctbx_project] master: de-clutter (e9be98075a)
by Robert Oeffner 07 Oct '20

07 Oct '20

Hi Graeme, It's just a simple call to the libtbx.clean_clutter script. But with the path set explicitly so it only works for this clone of the repo. Rob [hook "precommit"] cmdline = C:\\Users\\Oeffner\\Work\\Phenix\\phenix-installer-dev-3981-intel-windows-x86_64\\build\\bin\\libtbx.clean_clutter.bat wait = true show = true On 07-10-2020 14:33, Winter, Graeme (DLSLtd,RAL,LSCI) wrote: > Hi Rob, > > No worries - the deluge of commits last night with the same fricken > message have made my hyper sensitive and I have had a pet hate of > clean clutter for a long while, and your commit happened to come along > while I had it in my head > > I really think universal pre-commit hooks would be _a good thing_ > > if you wanted to circulate the hook in question you may also help > others… though I seem to recall there are some who dislike things like > pre-commit hooks as somehow impinging on their freedom or similar… > > Cheers Graeme > >> On 7 Oct 2020, at 14:23, Robert Oeffner <rdo20(a)cam.ac.uk >> <mailto:[email protected]>> wrote: >> >> Sorry about the noise in the commit history. I have now put a >> pre-commit hook in my git settings that runs clean_clutter on my >> clone of the cctbx repo. >> >> Rob >> >> >> On 07-10-2020 13:36, Winter, Graeme (DLSLtd,RAL,LSCI) wrote: >>> Hi Folks >>> >>> Would anyone be offended if we had a pre-commit hook which ran the >>> clutter / unused imports tests to stop the history fulling up with >>> clean clutter commits? >>> >>> If clutter is important, probably should not let it in in the first >>> place? >>> >>> I’m not suggesting anything extreme like black, like we use for >>> dxtbx and dials, just running the tools that developers are supposed >>> to run before commits >>> >>> Cheers Graeme >>> >>>> Begin forwarded message: >>>> >>>> *From: *CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM >>>> <mailto:[email protected]>> >>>> *Subject: **[git/cctbx_project] master: de-clutter (e9be98075a)* >>>> *Date: *7 October 2020 at 12:56:50 BST >>>> *To: *CCTBX-COMMIT(a)JISCMAIL.AC.UK <mailto:[email protected]> >>>> *Reply-To: *cctbxbb(a)PHENIX-ONLINE.ORG >>>> <mailto:[email protected]> >>>> >>>> Repository : https://github.com/cctbx/cctbx_project/ >>>> <https://github.com/cctbx/cctbx_project/> >>>> On branch : master >>>> Link : >>>> https://github.com/cctbx/cctbx_project/commit/e9be98075a31f4584f1eca41f9eba… >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> commit e9be98075a31f4584f1eca41f9eba5e408d0cb8f >>>> Author: Robert Oeffner <rdo20(a)cam.ac.uk <mailto:[email protected]>> >>>> Date: Wed Oct 7 12:56:50 2020 +0100 >>>> >>>> de-clutter >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> e9be98075a31f4584f1eca41f9eba5e408d0cb8f >>>> cctbx/miller/__init__.py | 0 >>>> 1 file changed, 0 insertions(+), 0 deletions(-) >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> To unsubscribe from the CCTBX-COMMIT list, click the following link: >>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1 >>>> >>> >>> >>> -- >>> >>> This e-mail and any attachments may contain confidential, copyright >>> and or privileged material, and are for the use of the intended >>> addressee only. If you are not the intended addressee or an >>> authorised recipient of the addressee please notify us of receipt by >>> returning the e-mail and do not use, copy, retain, distribute or >>> disclose the information in or attached to the e-mail. >>> Any opinions expressed within this e-mail are those of the >>> individual and not necessarily of Diamond Light Source Ltd. >>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any >>> attachments are free from viruses and we cannot accept liability for >>> any damage which you may sustain as a result of software viruses >>> which may be transmitted in or with the message. >>> Diamond Light Source Limited (company no. 4375679). Registered in >>> England and Wales with its registered office at Diamond House, >>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 >>> 0DE, United Kingdom >>> >>> >>> _______________________________________________ >>> cctbxbb mailing list >>> cctbxbb(a)phenix-online.org >>> http://phenix-online.org/mailman/listinfo/cctbxbb >> -- >> Robert Oeffner, Ph.D. >> Research Associate, The Read Group >> Department of Haematology, >> Cambridge Institute for Medical Research >> University of Cambridge >> Cambridge Biomedical Campus >> The Keith Peters Building >> Hills Road >> Cambridge CB2 0XY >> www.cimr.cam.ac.uk/investigators/read/index.html >> tel: +44(0)1223 763234 >> >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> >> Virus-free. www.avg.com >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> >> >> > > -- > > This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended > addressee only. If you are not the intended addressee or an authorised > recipient of the addressee please notify us of receipt by returning > the e-mail and do not use, copy, retain, distribute or disclose the > information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for > any damage which you may sustain as a result of software viruses which > may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United > Kingdom > -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

1 0

Fwd: [git/cctbx_project] master: de-clutter (e9be98075a)
by Winter, Graeme (DLSLtd,RAL,LSCI) 07 Oct '20

07 Oct '20

Hi Folks Would anyone be offended if we had a pre-commit hook which ran the clutter / unused imports tests to stop the history fulling up with clean clutter commits? If clutter is important, probably should not let it in in the first place? I’m not suggesting anything extreme like black, like we use for dxtbx and dials, just running the tools that developers are supposed to run before commits Cheers Graeme Begin forwarded message: From: CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM<mailto:[email protected]>> Subject: [git/cctbx_project] master: de-clutter (e9be98075a) Date: 7 October 2020 at 12:56:50 BST To: CCTBX-COMMIT(a)JISCMAIL.AC.UK<mailto:[email protected]> Reply-To: cctbxbb(a)PHENIX-ONLINE.ORG<mailto:[email protected]> Repository : https://github.com/cctbx/cctbx_project/ On branch : master Link : https://github.com/cctbx/cctbx_project/commit/e9be98075a31f4584f1eca41f9eba… ________________________________ commit e9be98075a31f4584f1eca41f9eba5e408d0cb8f Author: Robert Oeffner <rdo20(a)cam.ac.uk<mailto:[email protected]>> Date: Wed Oct 7 12:56:50 2020 +0100 de-clutter ________________________________ e9be98075a31f4584f1eca41f9eba5e408d0cb8f cctbx/miller/__init__.py | 0 1 file changed, 0 insertions(+), 0 deletions(-) ________________________________ To unsubscribe from the CCTBX-COMMIT list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1 -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Clutter etc.
by Winter, Graeme (DLSLtd,RAL,LSCI) 03 Oct '20

03 Oct '20

Hi Folks Please could we make less of a big deal about clutter? I have been hacking away for weeks at a PR & errors such as 2020-10-03T09:39:50.9328925Z Generating script. 2020-10-03T09:39:50.9348357Z ========================== Starting Command Output =========================== 2020-10-03T09:39:50.9378363Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/2787f38a-d1cc-4c10-b0bd-c9b041b41631.sh 2020-10-03T09:40:32.0565443Z 2020-10-03T09:40:32.0566065Z mmtbx/model/model.py: unused imports=1 2020-10-03T09:40:32.0566273Z 2020-10-03T09:40:32.0566573Z *** To clean up please use: libtbx.find_unused_imports_crude *** 2020-10-03T09:40:32.0566824Z 2020-10-03T09:40:32.0711947Z 2020-10-03T09:40:32.0785277Z ##[error]Bash exited with code '1'. 2020-10-03T09:40:32.0802183Z ##[section]Finishing: Check for clutter 2020-10-03T09:40:32.0831426Z ##[section]Starting: Checkout cctbx/cctbx_project@refs/pull/533/merge to ./modules/cctbx_project 2020-10-03T09:40:32.0841670Z ============================================================================== 2020-10-03T09:40:32.0842063Z Task : Get sources 2020-10-03T09:40:32.0842444Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories. 2020-10-03T09:40:32.0842850Z Version : 1.0.0 2020-10-03T09:40:32.0843106Z Author : Microsoft 2020-10-03T09:40:32.0843503Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199) 2020-10-03T09:40:32.0843947Z ============================================================================== 2020-10-03T09:40:32.4896538Z Cleaning any cached credential from repository: cctbx/cctbx_project (GitHub) 2020-10-03T09:40:32.4948059Z ##[section]Finishing: Checkout cctbx/cctbx_project@refs/pull/533/merge to ./modules/cctbx_project 2020-10-03T09:40:32.5010291Z ##[section]Starting: Finalize Job 2020-10-03T09:40:32.5043249Z Cleaning up task key 2020-10-03T09:40:32.5044549Z Start cleaning up orphan processes. 2020-10-03T09:40:32.5217555Z ##[section]Finishing: Finalize Job 2020-10-03T09:40:32.5261526Z ##[section]Finishing: syntax python2.7 are making it hard to actually make the tests pass (this change is something I have not touched) Is it really worth breaking builds in PR’s because of some white space / unused imports in unrelated files? Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 2

Simple C++11 async example
by Tristan Croll 02 Oct '20

02 Oct '20

Hi all, For anyone who's interested: following on from my quick demo of the use of std::async to simplify threading of long-running tasks called from Python, I've put some example code at https://github.com/tristanic/async-hello-world/tree/master/src (implemented as a ChimeraX plugin for demo purposes, but the C++ code and bindings are entirely generic). To re-iterate what this is about, and why I think it's important: in brief, for most real-world problems Python-level threading is not particularly useful since (except in specific cases) the Global Interpreter Lock limits the overall performance to no better than - and often much worse than - single-threaded code. While multiprocessing can be an option for some tasks, it's (a) slow to get started, (b) memory-hungry due to the need for each process to have a complete copy of all the data, (c) rather difficult to organise with respect to intercommunication, and (d) not very cross-platform friendly (I'm looking at you, Windows!). The point of this example is to show that with C++11 organising the threading at the C++ level can often end up being both the simplest and fastest approach - it's only a dozen or so extra lines of fairly boilerplate code required to allow the Python programmer to choose between running the exact same function either in the main thread or asynchronously in a new one. Why I think it's important: I've said in the past, and continue to believe, that we should be aiming for (interactive) model building, validation and refinement to merge together into a single integrated process - as much as anything, ISOLDE is meant to be the proof of principle that this is a possibility. But for that to work, high-performance and cross-platform parallelism is a real necessity. The approach described here is how I'm currently getting ChimeraX to maintain 30+ fps graphics performance while simultaneously running interactive molecular dynamics and structure factor calculations. I suspect there are many functions in CCTBX that could potentially benefit from this approach even when not considering GUI applications. Best regards, Tristan

5 6