April 2018 - cctbxbb - phenix-online.org

mmtbx/examples/simulate_experimental_data.py
by Elliot Nelson 10 May '18

10 May '18

Hi All, I'm trying to generate some simulated mtz files. My main aim is to test a script which looks to determine ligand occupancy (https://github.com/nelse003/exhaustive_search/blob/master/exhaustive_search…) by looking at |Fo-Fc| over the ligand/ ground-state the ligand replaces. To test this I would like to generate mtz files from pdb models with a ligand occupancy that has varied between 0-1, which the above script would hopefully recover the occupancy. I have tried to use mmtbx/examples/simulate_experimental_data.py (https://github.com/cctbx/cctbx-playground/blob/8f23c190a9f8d2ed2b5845385f56… ) to generate an mtz file. So far I have been able to generate mtz files using the simulate_experimental_data.py, these have the columns: H K L FOBS(+) SIGFOBS(+) FOBS(-) SIGFOBS(-) I can use this outputted mtz for running tests of my scripts ,However as this has no PHI columns, I cannot display this in coot. I have tried to add the phi column from the refined mtz of the orignal data using CAD, but have not yet managed to do this succesfully. I feel that this may be an overcomplicated, and potentially superseded way of generating simulated mtz data. Any advice on alternatives would be really useful? Apologies if this is not very clear. Thanks Elliot Nelson

3 4

Re: [cctbxbb] [Dials-support] Issue with hierarchy.as_pdb_string
by Graeme.Winter＠Diamond.ac.uk 28 Apr '18

28 Apr '18

Morning all, Mike came to the DIALS list with some pdb-reading related issues - however in DIALS land we usually don’t mess with this kind of stuff - is there someone who can help him out here please? Many thanks Graeme On 27 Apr 2018, at 18:14, Wall, Michael E <mewall(a)lanl.gov<mailto:[email protected]>> wrote: Hi, I'm seeing the following issue with a fresh build of dials development source, downloaded within the last hour. There's no error using dials v1-9-1. Is this a bug? Something that's being revised right now? Thanks, Mike Traceback (most recent call last): File "../../src/xenm_new.py", line 609, in <module> ftmp.write(pdb_in.hierarchy.as_pdb_string(crystal_symmetry=pdb_in.input.crystal_symmetry())) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/__init__.py", line 1191, in crystal_symmetry self_symmetry = self.crystal_symmetry_from_cryst1() File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/__init__.py", line 1168, in crystal_symmetry_from_cryst1 return cryst1_interpretation.crystal_symmetry(cryst1_record=line) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/cryst1_interpretation.py", line 138, in crystal_symmetry unit_cell=u) File "/home/mewall/packages/dials_devel/modules/cctbx_project/iotbx/pdb/cryst1_interpretation.py", line 87, in space_group_info try: return sgtbx.space_group_info(self.symbol) File "/home/mewall/packages/dials_devel/modules/cctbx_project/cctbx/sgtbx/__init__.py", line 97, in __init__ symbols = space_group_symbols(symbol) Boost.Python.ArgumentError: Python argument types in space_group_symbols.__init__(space_group_symbols, str) did not match C++ signature: __init__(_object*, int space_group_number) __init__(_object*, int space_group_number, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > extension='') __init__(_object*, int space_group_number, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > extension='', std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > table_id='') __init__(_object*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > symbol) __init__(_object*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > symbol, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > table_id='') ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org<http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ DIALS-support mailing list DIALS-support(a)lists.sourceforge.net<mailto:[email protected]> https://lists.sourceforge.net/lists/listinfo/dials-support -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

scitbx without numpy - still supported?
by Nicholas Devenish 24 Apr '18

24 Apr '18

Hi All, I've been tweaking the code to allow versions of boost higher than 1.63, which has involved poking at the scitbx-numpy bridge code. There seems to be some level of branching in the code for supporting building without numpy - but on the other hand numpy seems to be used exclusively for all flex.type.__init__ functions that I can find? Is building without numpy still supported? Thanks, Nick

5 6

Misuse of libtbx/configure.py
by Robert Oeffner 09 Apr '18

09 Apr '18

Hi, I'm not sure when this has been put into configure.py as I have had taken my eyes of the ball lately regarding Windows builds. But it seems that during bootstrap build the configure.py step is now doing more than just configuring modules. On my PC the excerpt below from stdout shows an error during the configure.py step Processing: "c:\busers\oeffner\nightlybuild\x32\current\modules\dials\libtbx_refresh.py" requires mock>=2.0, has 2.0.0 requirement orderedset is not currently met, package not installed attempting install of orderedset... Collecting orderedset Using cached orderedset-2.0.1.tar.gz Installing collected packages: orderedset Running setup.py install for orderedset: started Running setup.py install for orderedset: finished with status 'error' Complete output from command C:\Busers\oeffner\NightlyBuild\x32\Current\base\bin\python\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\oeffner\\appdata\\local\\temp\\pip-build-smhifw\\orderedset\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\oeffner\appdata\local\temp\pip-0seb8q-record\install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build\lib.win32-2.7 creating build\lib.win32-2.7\orderedset copying lib\orderedset\__init__.py -> build\lib.win32-2.7\orderedset running build_ext building 'orderedset._orderedset' extension error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27 ---------------------------------------- Command "C:\Busers\oeffner\NightlyBuild\x32\Current\base\bin\python\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\oeffner\\appdata\\local\\temp\\pip-build-smhifw\\orderedset\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\oeffner\appdata\local\temp\pip-0seb8q-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\oeffner\appdata\local\temp\pip-build-smhifw\orderedset\ You are using pip version 9.0.1, however version 9.0.3 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. install failed. please check manually My first concern with this approach is that the bootstrap process doesn't stop. It carries on and eventually builds the Phenix software suite. This may or may not be a broken build as it is not clear whether the above failure to build orderedset was a showstopper for something. Secondly, I thought that modules would not be installed anywhere else than when building base components or when downloading sources. For a small project it is a nice and sneaky solution to have certain build steps second guessing what dependencies are missing and then installing them "on the fly", if not covertly. But for a large project such as CCTBX with several teams using that as a common framework this becomes messy I would argue. A quick and dirty fix for the above error might be just to put vcvarsall.bat in the PATH environment. But that is not a solution since bootstrap.py has been fashioned to use different versions of Visual C++, some of which are not invoked through a vcvarsall.bat file. I appreciate the need to install additional component modules for python. But would it not be more tidy to install these during the base step of the bootstrap process? Is there any reason why this cannot happen? Regards, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234

5 5

simtbx - Pilatus example available
by James Holton 08 Apr '18

08 Apr '18

Took most of last week, but I finally got it to work. tst_nanoBragg_pilatus.py will create a Pilatus-like image, complete with bad pixels, window panes, thickness effects, module-to-module misalignment and 0.1 deg of phi rotation. It takes one optional argument, which is the image number in a sequence of 3600. I've processed this simulated data set with xia2 and XDS and found the anomalous signal has the right sign. This required some edits and additions to the simTBX code as well as FormatCBFMini.as_file() that I think fixed a beam center consistency bug. Now if you dxtbx.load() and FormatCBFMiniPilatus.as_file() you get the same beam center in both input and output headers. I also enhanced the header to make it look more like the ones in DIALS regression. A new "SIM.F000" value is now available for marking the "direct beam" on the image itself. I spent most of Friday and all of this morning trying to follow the check-in procedure we laid out during the DIALS workshop. Not sure how, but it suddenly started working just now. I believe I have checked in this code into the simtbx_sandbox branch of cctbx_project. Is Jenkins looking at this yet? Is anyone looking at this yet? -James Holton MAD Scientist

3 3

print dxtbx.model.Scan()
by James Holton 05 Apr '18

05 Apr '18

The following sequence of commands: % source build/setpaths.csh % libtbx.python >>> import dxtbx >>> from dxtbx import model >>> print dxtbx.model.Scan() seems to cause Python itself to core dump. Is this expected? show_stack(1): <stdin>(1) <module> libc backtrace (25 frames, most recent call last): /home/jamesh/projects/cctbx-dev/build/../base/bin/python2.7() [0x400659] /lib64/libc.so.6(__libc_start_main+0xfd) [0x34e961ed1d] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(Py_Main+0xc74) [0x7f93eb5c2b74] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyRun_AnyFileExFlags+0x4c) [0x7f93eb5ad78c] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyRun_InteractiveLoopFlags+0x4e) [0x7f93eb5ad04e] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyRun_InteractiveOneFlags+0x195) [0x7f93eb5ace35] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32) [0x7f93eb58d4e2] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x88e) [0x7f93eb58d3ce] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x3ef0) [0x7f93eb589de0] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyFile_WriteObject+0x78) [0x7f93eb503b58] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(+0x958f2) [0x7f93eb5288f2] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyObject_Str+0xa) [0x7f93eb52879a] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(_PyObject_Str+0x7b) [0x7f93eb5286ab] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(+0xb5a1e) [0x7f93eb548a1e] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x43) [0x7f93eb584fd3] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7f93eb4dc093] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(+0x5b6cf) [0x7f93eb4ee6cf] /home/jamesh/projects/cctbx-dev/base/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7f93eb4dc093] /home/jamesh/projects/cctbx-dev/build/lib/libboost_python.so(+0x2d088) [0x7f93e3f23088] /home/jamesh/projects/cctbx-dev/build/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x43) [0x7f93e3f34013] /home/jamesh/projects/cctbx-dev/build/lib/libboost_python.so(+0x2fe08) [0x7f93e3f25e08] /home/jamesh/projects/cctbx-dev/build/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0xef) [0x7f93e3f251ff] /home/jamesh/projects/cctbx-dev/build/lib/dxtbx_model_ext.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<std::string (*)(dxtbx::model::Scan const&), boost::python::default_call_policies, boost::mpl::vector2<std::string, dxtbx::model::Scan const&> > >::operator()(_object*, _object*)+0x83) [0x7f93db88a7a3] /home/jamesh/projects/cctbx-dev/build/lib/dxtbx_model_ext.so(dxtbx::model::boost_python::scan_to_string(dxtbx::model::Scan const&)+0x319) [0x7f93db871929] /lib64/libc.so.6() [0x34e9632510] Segmentation fault (Python and libc call stacks above)

2 1

Exceptions squashed by easy_mp (revenge of)
by Graeme.Winter＠Diamond.ac.uk 03 Apr '18

03 Apr '18

Folks, Following up on user reports again of errors within easy_mp - all that gets logged is “something went wrong” i.e. Using multiprocessing with 10 parallel job(s) Traceback (most recent call last): File "/home/user/bin/dials-installer/build/../modules/dials/command_line/integrate.py", line 613, in <module> halraiser(e) File "/home/user/bin/dials-installer/build/../modules/dials/command_line/integrate.py", line 611, in <module> script.run() File "/home/user/bin/dials-installer/build/../modules/dials/command_line/integrate.py", line 341, in run reflections = integrator.integrate() File "/home/user/bin/dials-installer/modules/dials/algorithms/integration/integrator.py", line 1214, in integrate self.reflections, _, time_info = processor.process() File "/home/user/bin/dials-installer/modules/dials/algorithms/integration/processor.py", line 271, in process preserve_exception_message = True) File "/home/user/bin/dials-installer/modules/dials/util/mp.py", line 171, in multi_node_parallel_map preserve_exception_message = preserve_exception_message) File "/home/user/bin/dials-installer/modules/dials/util/mp.py", line 53, in parallel_map preserve_exception_message = preserve_exception_message) File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/easy_mp.py", line 627, in parallel_map result = res() File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/scheduling/result.py", line 119, in __call__ self.traceback( exception = self.exception() ) File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/scheduling/stacktrace.py", line 115, in __call__ self.raise_handler( exception = exception ) File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/scheduling/mainthread.py", line 100, in poll value = target( *args, **kwargs ) File "/home/user/bin/dials-installer/modules/dials/util/mp.py", line 91, in __call__ preserve_exception_message = self.preserve_exception_message) File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/easy_mp.py", line 627, in parallel_map result = res() File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/scheduling/result.py", line 119, in __call__ self.traceback( exception = self.exception() ) File "/home/user/bin/dials-installer/modules/cctbx_project/libtbx/scheduling/stacktrace.py", line 86, in __call__ raise exception RuntimeError: Please report this error to dials-support(a)lists.sourceforge.net: exit code = -9 I forget why it was decided that keeping the proper stack trace was a bad thing, but could this be revisited? It would greatly help to see it in the output of the program (if as is the case here I do not have the user data) My email-fu is not strong enough to dig out the previous conversation Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6