<div dir="ltr">Thanks Rob. I've switched dials.stills_process to using easy_mp.multi_core_run. I tested raising an error in one of the processes and confirmed the program finishes, and that I can print out the error message raised in the process. This is great.<div><br></div><div>Thanks,</div><div>-Aaron</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 13, 2017 at 4:13 AM, R.D. Oeffner <span dir="ltr"><<a href="mailto:rdo20@cam.ac.uk" target="_blank">rdo20@cam.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Graeme,<br>
<br>
As far as I can tell from stills_process.py you're using easy_mp.parallel_map() for your multiprocessing. I should use easy_mp.multi_core_run() instead. It is a different version of easy_mp.parallel_map().<br>
See <a href="https://www.phenix-online.org/newsletter/CCN_2017_01.pdf#page=6" rel="noreferrer" target="_blank">https://www.phenix-online.org/<wbr>newsletter/CCN_2017_01.pdf#pag<wbr>e=6</a><br>
<br>
One of its features is that it preserves stacktraces from individual child processes should they crash which is helpful for debugging. If a child process crashes in parallel_map() however you will not get a stacktrace from the offending child which makes correcting the misbehaviour more difficult.<br>
<br>
Rob<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
On 13/12/2017 09:34, <a href="mailto:Graeme.Winter@diamond.ac.uk" target="_blank">Graeme.Winter@diamond.ac.uk</a> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Folks<br>
<br>
Once again, lack of stack traces in easy_mp is a big annoyance<br>
<br>
I forget the reasons why printing the actual error was a bad thing,<br>
but I do remember a discussion some 18 mo past<br>
<br>
Not wishing to replay the entire transaction, but could we revisit this?<br>
<br>
Thanks Graeme<br>
<br>
Begin forwarded message:<br>
<br>
From: <<a href="mailto:Danny.Axford@Diamond.ac.uk" target="_blank">Danny.Axford@Diamond.ac.uk</a><ma<wbr>ilto:<a href="mailto:Danny.Axford@Diamond.ac.uk" target="_blank">Danny.Axford@Diamond.ac.<wbr>uk</a>>><br>
Subject: [Dials-support] dials.stills_process crash report<br>
Date: 12 December 2017 at 17:43:41 GMT<br>
To:<br>
<<a href="mailto:dials-support@lists.sourceforge.net" target="_blank">dials-support@lists.sourcefor<wbr>ge.net</a><mailto:<a href="mailto:dials-support@lists.sourceforge.net" target="_blank">dials-support@<wbr>lists.sourceforge.net</a>>><br>
Cc: <a href="mailto:martin.appleby@diamond.ac.uk" target="_blank">martin.appleby@diamond.ac.uk</a><m<wbr>ailto:<a href="mailto:martin.appleby@diamond.ac.uk" target="_blank">martin.appleby@diamond.a<wbr>c.uk</a>><br>
<br>
Perhaps this means something to someone. This is the SACLA data again,<br>
it seems to run happily and then hit a problematic frame? Martin and I<br>
see the error at pretty much the same point in the dataset, we were<br>
both running on local machines rather than the cluster.<br>
<br>
Cheers,<br>
Danny<br>
<br>
<br>
<br>
Detector 1 RMSDs by panel:<br>
------------------------------<wbr>----------------------<br>
| Panel | Nref | RMSD_X | RMSD_Y | RMSD_DeltaPsi |<br>
| id | | (px) | (px) | (deg) |<br>
------------------------------<wbr>----------------------<br>
| 1 | 89 | 0.37442 | 0.48949 | 0.2192 |<br>
| 2 | 69 | 0.49627 | 0.4847 | 0.21385 |<br>
| 5 | 90 | 0.48267 | 0.59909 | 0.22839 |<br>
| 6 | 70 | 0.6094 | 0.55592 | 0.20944 |<br>
------------------------------<wbr>----------------------<br>
Traceback (most recent call last):<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/build/../modules/dials/<wbr>command_line/stills_process.<wbr>py",<br>
line 870, in <module><br>
halraiser(e)<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/build/../modules/dials/<wbr>command_line/stills_process.<wbr>py",<br>
line 868, in <module><br>
script.run()<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/build/../modules/dials/<wbr>command_line/stills_process.<wbr>py",<br>
line 383, in run<br>
preserve_exception_message=Tr<wbr>ue)<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/modules/cctbx_project/<wbr>libtbx/easy_mp.py",<br>
line 623, in parallel_map<br>
result = res()<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/modules/cctbx_project/<wbr>libtbx/scheduling/result.py",<br>
line 119, in __call__<br>
self.traceback( exception = self.exception() )<br>
File<br>
"/dls/science/groups/scisoft/D<wbr>IALS/CD/latest/dials-dev201712<wbr>12/modules/cctbx_project/<wbr>libtbx/scheduling/stacktrace.<wbr>py",<br>
line 86, in __call__<br>
raise exception<br>
RuntimeError: Please report this error to<br>
<a href="mailto:dials-support@lists.sourceforge.net" target="_blank">dials-support@lists.sourceforg<wbr>e.net</a><mailto:<a href="mailto:dials-support@lists.sourceforge.net" target="_blank">dials-support@<wbr>lists.sourceforge.net</a>>:<br>
exit code = -9<br>
<br>
<br>
--<br>
This e-mail and any attachments may contain confidential, copyright<br>
and or privileged material, and are for the use of the intended<br>
addressee only. If you are not the intended addressee or an authorised<br>
recipient of the addressee please notify us of receipt by returning<br>
the e-mail and do not use, copy, retain, distribute or disclose the<br>
information in or attached to the e-mail.<br>
Any opinions expressed within this e-mail are those of the individual<br>
and not necessarily of Diamond Light Source Ltd.<br>
Diamond Light Source Ltd. cannot guarantee that this e-mail or any<br>
attachments are free from viruses and we cannot accept liability for<br>
any damage which you may sustain as a result of software viruses which<br>
may be transmitted in or with the message.<br>
Diamond Light Source Limited (company no. 4375679). Registered in<br>
England and Wales with its registered office at Diamond House, Harwell<br>
Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United<br>
Kingdom<br>
<br>
<br>
------------------------------<wbr>------------------------------<wbr>------------------<br>
Check out the vibrant tech community on one of the world's most<br>
engaging tech sites, Slashdot.org<<a href="http://slashdot.org/" rel="noreferrer" target="_blank">http://slashdot.o<wbr>rg/</a>>!<br>
<a href="http://sdm.link/slashdot" rel="noreferrer" target="_blank">http://sdm.link/slashdot</a><br>
______________________________<wbr>_________________<br>
DIALS-support mailing list<br>
<a href="mailto:DIALS-support@lists.sourceforge.net" target="_blank">DIALS-support@lists.sourceforg<wbr>e.net</a><mailto:<a href="mailto:DIALS-support@lists.sourceforge.net" target="_blank">DIALS-support@<wbr>lists.sourceforge.net</a>><br>
<a href="https://lists.sourceforge.net/lists/listinfo/dials-support" rel="noreferrer" target="_blank">https://lists.sourceforge.net/<wbr>lists/listinfo/dials-support</a><br>
<br>
<br>
______________________________<wbr>_________________<br>
cctbxbb mailing list<br>
<a href="mailto:cctbxbb@phenix-online.org" target="_blank">cctbxbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/cctbxbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailm<wbr>an/listinfo/cctbxbb</a><br>
</blockquote>
<br>
______________________________<wbr>_________________<br>
cctbxbb mailing list<br>
<a href="mailto:cctbxbb@phenix-online.org" target="_blank">cctbxbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/cctbxbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailm<wbr>an/listinfo/cctbxbb</a><br>
</div></div></blockquote></div><br></div>