- cctbxbb - phenix-online.org

Windows 10 Retirement Date
by Billy Poon 27 May '25

27 May '25

Hi everyone, Microsoft will be dropping support for Windows 10 later this year on October 14, 2025. https://learn.microsoft.com/en-us/lifecycle/products/windows-10-home-and-pro Phenix intends to drop Windows 10 support after that date. Are there users where Windows 10 support is required after that date? Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

reduce2 and cell construction from python script
by Enrico Ravera 14 Feb '25

14 Feb '25

Hi, I apologize for the naive post. I am trying to figure out how to perform the following tasks: 1) read in a mmcif file (which I think I can do following the tutorial); 2) trim the solvent and hydrogen atoms (probably I can do it following the tutorial); 3) run reduce2 to get hydrogens in the nuclear positions (no clue); 4) get the full content of the unit cell (no clue). Any help is appreciated. All the best, Enrico

3 11

getting indices of selected grid elements of a map
by robert＠oeffner.net 21 Jan '25

21 Jan '25

Hi, I am wondering if there is an efficient way to retrieve the 3D indices of a selection from a map. In the example on https://cci.lbl.gov/docs/cctbx/doc_low_flex_advanced/#getting-indices-of-se… it is shown how to list the indices of a subset of grid elements of the map as in print(list(isel)) But isel is a 1D representation of 3d map indices. What if I want to get hold of the list of 3D indices corresponding to the indices in the isel array? Is there a fast boost-python way of doing this or would I have to do a 3D loop over the unit cell grid numbers to see if enumerated grid elements matches any of the elements in the isel array? Many thanks, Rob

2 4

atomic density profile at finite resolution
by robert＠oeffner.net 13 Dec '24

13 Dec '24

I came across this article https://doi.org/10.1107/S160057672201144X. Although I have yet to read it in details I wonder if the CCTBX can Fcalc maps at finite resolution, i.e. not using Gaussian atomic density profiles. It looks like the function in cctbx.maptbx.atom_curves.bcr_approx(...) is suited to do so. If so, how would I use this for generating a real space map as in https://cci.lbl.gov/docs/cctbx/script_1/#create-a-map-from-a-pdb-file but now using atomic density profiles at finite resolution? Many thanks, Robert

4 8

cluster coordinates to asymmetric unit
by robert＠oeffner.net 06 Dec '24

06 Dec '24

Hi, I'm trying to work out how to cluster atomic coordinates into one asymmetric unit in a crystal for a given spacegroup rather than the atomic coordinates placed in arbitrary asymmetric unit copies in the unit cell. I think the tools to be used must be in cctbx.crystal.direct_space_asu and I have been trying but failed to understand how the test tst_crystal_asu_clusters.py works. Is there already a phenix or cctbx program that can do this? Many thanks, Robert

3 2

chiral validation
by James Holton 04 Nov '24

04 Nov '24

Can anyone tell me the units of the numbers being displayed by phenix.chiral_validation? The --help documentation doesn't mention anything about chirality. Seems to be a copy of the documentation for omegalyze? Just thought that might be of interest, and I also want to know what the numbers mean. Cheers, -James

2 1

electron density value near an atom
by robert＠oeffner.net 15 Oct '24

15 Oct '24

Dear CCTBX users/developers I would like to calculate the theoretical electron density value at a point in space as a function of an atom specified by its coordinate and its form factor. I’ll assume it’s for a certain X-ray wavelength, say 1.0Å. If someone would have the time to give me a few pointers on how to do this using the CCTBX I'd be very grateful. Thanks in advance, Rob

3 2

Hardware migration
by Billy Poon 19 Jun '24

19 Jun '24

Hi everyone, We are in the process of migrating these bulletin boards to new hardware over the next few weeks. I will make another post once the migration is completed. Nothing will be changed until then. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org

1 2

Test thread from website
by bkpoon＠lbl.gov 19 Jun '24

19 Jun '24

The bulletin board update has also enabled posting from the website

1 0

test post
by bkpoon＠lbl.gov 17 Jun '24

17 Jun '24

This is a test message

1 0

test post
by bkpoon＠lbl.gov 17 Jun '24

17 Jun '24

This is a test message

1 0

test post
by bkpoon＠lbl.gov 17 Jun '24

17 Jun '24

This is a test message

1 0

Python 3.12 release schedule
by Luc Bourhis 16 Mar '24

16 Mar '24

Hi, I see in the commit that you guys are working on supporting Python 3.12. With the latest version of Python 3.11 on Anaconda, we have a compatibility issue between Qt that we use and ICU 70.1 that cctbx requires. The problem is solved by moving to Python 3.12, hence my question: when can we hope to see a release with Python 3.12? No pressure! Just trying to plan ahead. Best wishes, Luc

2 6

Experimental gradients
by Kristoffer Lundgren 11 Oct '23

11 Oct '23

Hello all, I am wondering if it is possible to use cctbx to calculate gradients from experimental data? Both in reciprocal and real space is of interest. Are there any code examples available somewhere showcasing how this can be achieved? Best regards Kristoffer Lundgren

3 4

iterating over crystals
by David Waterman 17 Sep '23

17 Sep '23

Hi folks, I'm testing some code changes and would like to iterate over every space group and for each generate a random unit cell compatible with the Bravais symmetry. I've got the iteration sorted with space_group_symbol_iterator, but I'm not sure how best to generate some compatible unit cell from the space_group object. Is this something I could do easily with cctbx? Cheers, -- David

4 4

Unexpected (to me) behavior of flex.bool
by Pavel Afonine 17 Jun '23

17 Jun '23

Hi All, today it took me several hours to debug something that ended up being the result of this behavior: >>> from scitbx.array_family import flex >>> x = flex.bool(True, 5) >>> y = flex.bool(5, True) >>> list(x) [True] >>> list(y) [True, True, True, True, True] >>> I needed to pass a flex.bool array (with all True) into a C++ function and I was doing it as "x" above. In that C++ function there was no size check for incoming arrays (bad idea!) and then later in the code when accessing elements of this flex.bool array I was getting "random" numbers! I was pretty sure that flex.bool(True, 5) should lead to an assertion error! Was this always like this (and I was just super-lucky to never run into this issue in the past 20 years) or something has changed? Pavel

2 1

any plans for PDB files with long CCD codes?
by Marcin Wojdyr 16 May '23

16 May '23

Hello Cctbx and Phenix developers, as you know, extended CCD codes arrive this year. I'm looking into examples https://github.com/wwPDB/extended-wwPDB-identifier-examples and thinking what's the best way to convert such files to PDB. Do you plan anything in this matter, like it was done with https://cci.lbl.gov/hybrid_36/ in the past? Tentatively, I'm thinking about mapping residue names in the HETNAM record. For example, BB87Q would be mapped to a "3 letter code" that code would have a special character to make it different from normal 3-letter codes (let's say '%', or maybe '+' or '_' - I don't have a strong preference), and the original CCD code would be given in columns 72-79 of HETNAM that are currently unused: HETNAM BB% PHOSPHOSERINE [...] BB87Q

2 2

pull request - fable
by Nic Gould 16 May '23

16 May '23

Hi All, Sorry for the intrusion, the Github page suggests contacting the devs on the mailing list before making a pull request. I ran into some minor issues building fable and was wondering if it's worth submitting a pull request. - invalid pre-processor directive (fixed upstream at boost https://github.com/boostorg/thread/issues/364) - Invalid python API call ( fixed upstream at boost here https://github.com/boostorg/python/pull/344) - Need to install python.h : This one seems tricky as it varies between platforms and it's probably not a 'you problem'. Maybe add a line to the setup documentation. Interestingly it needs to be installed on the host machine, not in the conda venv. Kind regards Nic Gould

2 1

Potential website disruptions
by Billy Poon 12 Apr '23

12 Apr '23

Hi everyone, We are doing some maintenance on our websites, https://phenix-online.org and https://cci.lbl.gov, over the next few days. There may be some disruptions. Thanks in advance for your patience. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

github host key
by Robert Oeffner 24 Mar '23

24 Mar '23

Am I the only one being unable to push or pull from the CCTBX repo and instead getting a warning that the host github key has changed or that somebody is eavesdropping on me? Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234

3 3

Python 3.11
by Luc Bourhis 15 Dec '22

15 Dec '22

Hi,may I enquire about the time table for Python 3.11 compatibility? We are planning to move to it as soon as possible at Bruker, hence my question.Best wishes,Luc Bourhis

3 8

Conda dropping Python 3.7 real soon now
by Winter, Graeme (DLSLtd,RAL,LSCI) 16 Sep '22

16 Sep '22

Hi All, Possibly of relevance herein - https://conda-forge.org/docs/user/announcements.html#dropping-python-3-7 "Conda-Forge has been providing support for Python 3.7 for 4 years now. Increasingly projects are moving off it (particularly in the PyData community). With Python 3.11’s release coming around the corner (October 3rd), conda-forge plans to drop Python 3.7 support when Python 3.11 comes out. This will lighten the load on conda-forge infrastructure and make room for the new versions the community would like to support.” ISTR PHENIX is pinned on 3.7 right now because of some non-3.8 compatible code? That could become more of a problem Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Next conda version?
by Luc Bourhis 14 Sep '22

14 Sep '22

Hi, I have just fixed a bug introduced a few years ago that broke some of our code at Bruker. We use conda to add the cctbx to our environment, hence my question: when can the next version be expected on conda-force including that fix? No pressure! I would like to know to plan ahead. Thanks! Best wishes, Luc https://github.com/cctbx/cctbx_project/commit/085f6d8cf57b91dce3a695ebbfbcf… <https://github.com/cctbx/cctbx_project/commit/085f6d8cf57b91dce3a695ebbfbcf…>

4 4

boost python exports
by David Waterman 03 Mar '22

03 Mar '22

Hi folks, I'm trying to create a boost python class that works with either vec3 or mat3 arrays (and I only need those two types). I made a templated class and then exported it twice with different names, for the vec3 and mat3 versions. For example, for mat3: class_<ReconstituteDerivatives< mat3<double> > >("ReconstituteDerivativesMat3", no_init) .def("get_data", &ReconstituteDerivatives< mat3<double> >::get_data) .def("get_indices", &ReconstituteDerivatives< std::size_t >::get_indices); I can create one of these objects from Python >>> foo <dials_refinement_helpers_ext.ReconstituteDerivativesMat3 object at 0x7f3ffa421ac0> but the accessors are not exported properly: >>> foo.get_data() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: No to_python (by-value) converter found for C++ type: scitbx::af::shared<scitbx::mat3<double> > I expected a scitbx_array_family_flex_ext.mat3_double here >>> foo.get_indices() Traceback (most recent call last): File "<stdin>", line 1, in <module> Boost.Python.ArgumentError: Python argument types in ReconstituteDerivativesMat3.get_indices(ReconstituteDerivativesMat3) did not match C++ signature: get_indices(dials::refinement::ReconstituteDerivatives<unsigned long> {lvalue}) and I expected a scitbx_array_family_flex_ext.size_t here. Is it likely there is something simple I am missing, like a #include? I do include flex_wrapper.h. Or perhaps I am just doing something wrong with templated classes and boost python. Looking at other code isn't helping to reveal the mysteries to me yet. Cheers -- David

1 2

DYLD_FALLBACK_LIBRARY_PATH
by Winter, Graeme (DLSLtd,RAL,LSCI) 05 Jan '22

05 Jan '22

Hi Folks, By default cctbx dispatchers on macOS clobber DYLD_FALLBACK_LIBRARY_PATH with some stuff which means that from napari import viewer fails with a huge stack trace ending in OSError: dlopen(libobjc.dylib, 6): image not found If I comment out the clobbering in the dispatcher then everything works fine so I am fairly sure this is the problem Some questions - - why do we do this? - is there a way I can make a dispatcher not to this? or maybe leave the system DYLD… as it was and _append_ to rather than prepend? Seems like a thing I am surprised I have not broken a toe on before now Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 8

Re: [cctbxbb] [git/cctbx_project] master: Reduce2: place NH3 only at residue with resseq 1. (765881e87b)
by Pavel Afonine 11 Nov '21

11 Nov '21

What if chain starts with a number different than 1 (people tend to number chain residues in many different ways, even using negative numbers, and they claim they have valid reasons to do so)? iotbx.pdb.hierarchy has a clear definition of the chain. So I'd suggest adding NH3 to the fist residue of the chain, no matter what that residue number is. Pavel On 11/10/21 14:03, CCTBX commit wrote: > Repository : https://github.com/cctbx/cctbx_project/ > On branch : master > Link : > https://github.com/cctbx/cctbx_project/commit/765881e87b5982c3b68e8dcd8c0e0… > > ------------------------------------------------------------------------ > > > commit 765881e87b5982c3b68e8dcd8c0e00c61c5b5600 > Author: dcliebschner <dcliebschner(a)lbl.gov> > Date: Wed Nov 10 14:03:06 2021 -0800 > > Reduce2: place NH3 only at residue with resseq 1. > > > ------------------------------------------------------------------------ > > > 765881e87b5982c3b68e8dcd8c0e00c61c5b5600 > mmtbx/hydrogens/reduce_hydrogen.py | 19 +- > mmtbx/hydrogens/tst_add_hydrogen.py | 196 ++++++++--------- > mmtbx/hydrogens/tst_add_hydrogen_2.py | 383 > ++++++++++++++++------------------ > 3 files changed, 282 insertions(+), 316 deletions(-) > > Diff suppressed because of size. To see it, use: > > git diff-tree --root --patch-with-stat --no-color > --ignore-space-at-eol --textconv --ext-diff --cc > 765881e87b5982c3b68e8dcd8c0e00c61c5b5600 > ------------------------------------------------------------------------ > > To unsubscribe from the CCTBX-COMMIT list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1 > <https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1> >

1 0

accessing attributes of PHIL parameters
by Robert Oeffner 15 Oct '21

15 Oct '21

Hi, Most likely a trivial question but if I have a PHIL object such as one based on the string arrayinfo_phil_str = """ selected_info { span = True .type = bool .short_caption = "Span" minmax_data = True .type = bool .short_caption = "min,max data" } """ I can then get the PHIL definitions span and minmax_data as python values like: from libtbx.phil import parse parse(arrayinfo_phil_str).extract().selected_info.span parse(arrayinfo_phil_str).extract().selected_info.minmax_data But how do I access the values of the short_caption attribute of these definitions? Does that make sense or have not understood correctly how PHIL works? Thanks, Rob -- This email has been checked for viruses by AVG. https://www.avg.com

2 1

Project Scientist and Postdoctoral research positions in XFEL, ML, and material science computational methods
by Aaron Brewster 18 Aug '21

18 Aug '21

Hello, I'm excited to announce that two positions are immediately open in my group in the fields of material science, serial X-ray crystallography, and machine learning. Links to postings: Computational Project Scientist - https://lbl.taleo.net/careersection/2/jobdetail.ftl?job=93507 Computational Postdoctoral Scholar - https://lbl.taleo.net/careersection/2/jobdetail.ftl?job=93506 In these exciting roles, you will work in the Molecular Biophysics and Integrated Bioimaging Division on a project that will research adding advanced spatial and symmetry aware neural network training methods to small-molecule chemical crystallography and material science data processing. This will use AI methods developed as part of a close collaboration between teams at Lawrence Berkeley National Labs (LBNL), University of Connecticut, and MIT. The larger group is researching crystal engineering of hybrid coordination polymers towards achieving rational control over the optoelectronic properties of low-dimensional nanostructures. These positions are focused on the chemical crystallography aspect of the program, but the successful candidates will be expected to participate in all aspects of the program, including data collection at X-ray free electron lasers worldwide, data analysis on supercomputing facilities, structure solution, research on AI driven structure prediction, paper writing, and software management. Both roles are 3 year appointments. This work will be part of the Department of Energy Integrated Computational and Data Infrastructure for Science <https://insidehpc.com/2021/08/doe-funds-15m-for-integrated-computational-an…> Research program. Specific details follow. Berkeley Lab is committed to Inclusion, Diversity, Equity and Accountability (IDEA) <https://diversity.lbl.gov/ideaberkeleylab/> and strives to continue building community with these shared values and commitments. Berkeley Lab is an Equal Opportunity and Affirmative Action Employer. We heartily welcome applications from women, minorities, veterans, and all who would contribute to the Lab’s mission of leading scientific discovery, inclusion, and professionalism. In support of our diverse global community, all qualified applicants will be considered for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, or protected veteran status. *Computational Project Scientist* *What You Will Do:* - Design and train neural networks that allow determination of the unit cell and crystal orientations of small molecule crystals from sparse, randomly oriented diffraction patterns. This will involve curating training sets from simulated datasets and from existing and new experimental data that will be collected. - Research methods to scale training and use of these neural networks on multiple nodes in high performance computing environments. - Lead data collection and processing at X-ray free electron sources. - Maintain a detailed record of work performed. - Document and manage code developed using standard source control methods. - Publish in peer-reviewed journals. - Present research at seminars and conferences. - Assist Principal Investigator in developing grant proposals. - Document and maintain existing and new data processing software, including the cctbx build system. *What is Required:* - 2-5 years of relevant experience beyond the highest degree in Physics, Statistics, Crystallography, Artificial Intelligence, or a related field. - A strong background in Python, C++, or an equivalent platform for scientific computing. - A track record of critical thinking, problem solving, and mathematical intuition, demonstrated through publications and presentations. *Desired Qualifications:* - Familiarity with general Python scientific data packages such as NumPy, SciPy, and pandas. - Versed in modern coding practices for scientific computing, including algorithm design and evaluation, source code management under a revision control system like git, and incorporation of readable documentation and comments. - Expertise in computational approaches in at least one of the areas related to the research is required, including but not limited to small-molecular crystallography, structure solution and refinement, unit cell determination, neural network design, training, and evaluation, high speed computing, image analysis, large dataset analysis, graph theory, error propagation and statistics, signal processing, and GPU acceleration. - Familiarity with small-molecule material design and synthesis. - Strong analytical and organizational skills. - Excellent interpersonal, oral, and written communication skills. - Ability to work independently as well as part of a diverse team. *Computational **Postdoctoral Scholar* *What You Will Do:* - Design and train neural networks to model diffraction from small molecule crystals. This will involve curating training sets from simulated datasets and from existing and new experimental data that will be collected. - Assist in data collection and processing at X-ray free electron sources. - Maintain a detailed record of work performed. - Document and manage code developed using standard source control methods. - Publish in peer-reviewed journals. - Present research at seminars and conferences. * What is Required:* - PhD degree in Physics, Statistics, Crystallography, Artificial Intelligence, or a related field. - Experience in computational approaches in at least one of the areas related to the research, including but not limited to small-molecular crystallography, structure solution and refinement, unit cell determination, neural network design, training, and evaluation, high speed computing, image analysis, large dataset analysis, graph theory, error propagation and statistics, signal processing, and GPU acceleration. - Strong analytical and organizational skills. - Excellent interpersonal, oral, and written communication skills. - Ability to work independently as well as part of a diverse team. * Desired Qualifications:* - Familiarity with Python, C++, and general scientific data packages such as NumPy, SciPy, and pandas. - Familiarity with small-molecule material design and synthesis. Please do not hesitate to reach out to me for more information. Thank you, Aaron S. Brewster, PhD Computational Research Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720

1 0

DIALS releases in PHENIX
by Winter, Graeme (DLSLtd,RAL,LSCI) 15 Jul '21

15 Jul '21

Hi Folks Following from - # select dials-3.5 branch + # select dials-3.2 branch def _add_git(self, module, parameters, destination=None): super(PhenixBuilder, self)._add_git(module, parameters, destination) if (module == 'dials' or module == 'dxtbx' or module == 'xia2') and self.python3: @@ -2339,9 +2339,9 @@ class PhenixBuilder(CCIBuilder): if module == 'dxtbx': self.add_step(self.shell(command=['git', 'remote', 'set-url', 'origin', 'https://github.com/dials/dxtbx.git'], workdir=workdir)) self.add_step(self.shell(command=['git', 'fetch', 'origin'], workdir=workdir)) - self.add_step(self.shell(command=['git', 'checkout', 'dials-2.2'], workdir=workdir)) + self.add_step(self.shell(command=['git', 'checkout', 'dials-3.2'], workdir=workdir)) self.add_step(self.shell( - command=['git', 'branch', '--set-upstream-to=origin/dials-2.2', 'dials-2.2'], + command=['git', 'branch', '--set-upstream-to=origin/dials-3.2', 'dials-3.2'], workdir=workdir)) I was wondering - what are the constraints in PHENIX builds which mean bouncing around between extremely different DIALS releases is necessary? I note the shift from 2.2 to 3.2 would give a significant change in behaviour to the end user (who ultimately we may end up having to help with debugging) If we knew better why these changes were coming through, maybe we could help to keep PHENIX up to date? I would draw comparisons with CCP4 however they have been stuck in 2.2-series forever now as they’re still not able to support Python3 - will be in CCP4 8.x which will be ready any time now. I only mention this as it would be really good to start eliminating obsolete versions from distribution as “please go to dials.github.io and install a non-ancient version” is annoying advice to an end user with a fresh install of PHENIX or CCP4. Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Is there a schedule for the python3 migration or can cctbx.python already be built to be/use a python3 interpreter?
by Jan M. Simons 13 Jul '21

13 Jul '21

Cheers, I'm getting more and more concerned with the state of cctbx and the impending removal of python2 from quite a lot of major linux distributions in the near future. So far I have not managed to find documentation on how to build the cctbx to then use a python3 interpreter for the cctbx.python script, but I would really like to have this, as I'd have to check/port my code that's using cctbx modules (mostly smtbx and xrax) to python3 as well, so that it does not get obsoleted. So is it possible to build cctbx as a python3 module by now and if so how would one be able to do this? Thanks for your support Jan

5 10

WSL2 (was: Windows icc builds)
by Kay Diederichs 27 May '21

27 May '21

Hi Graeme, I have no solution for this problem but I want to bring up the fact that WSL2 is very easy to install on current Windows10, and lets one run the Linux binaries (including graphics if you install MobaXterm which has an Xserver) quite efficiently. I have not run into a single problem when trying WSL2 with XDS and XDSGUI; my writeup is at https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Installation#WS… . I think those who struggle with developing for Windows, including you, the Phenix team and the CCP4 team, might be able to save a lot of time/effort (and those who just use Windows would benefit in the long run by getting used to a Linux environment). Microsoft announced WSLg (the g stands for graphics) which provides an Xserver, but this currently requires Windows-Build 21362+ (Windows Insider Program). Give it a try! Just my 2 cents, Kay > Date: Wed, 26 May 2021 18:57:27 +0000 > From: "Winter, Graeme (DLSLtd,RAL,LSCI)" <Graeme.Winter(a)diamond.ac.uk> > To: cctbx mailing list <cctbxbb(a)phenix-online.org> > Subject: [cctbxbb] Windows icc builds > Message-ID: <6210D39B-96F7-493F-98D4-D1EDFCBE09A5(a)diamond.ac.uk> > Content-Type: text/plain; charset="utf-8" > > HI Folks > > Following from a DIALS issue about it being slow on Windows > > https://github.com/dials/dials/issues/1666 > > I have fallen into a rabbit hole and I am trying to build DIALS (and hence cctbx) on Windows with the intel compilers from oneAPI - however I keep stumbling over linking errors with e.g. undefined symbol __floor > > Should I abandon hope or has someone done this? > > I know this is not a new path to tread as windows_icc was a defined combination on the SConscripts > > I had had to hack some bits around a little presumably to accommodate 2021 versions of icc compilers, but this felt like I was going down a bad path > > Would welcome input from anyone who can help - currently windows DIALS is a pretty poor relation > > It is noteworthy however that parallel operation does now appear to work on Windows with easy mp :-) > > In other news - has anyone considered adding clang support with clang coming from conda? Could make for more consistency in the builds? I had a look at how one may go about doing this but rapidly found myself in SCons hell. > > Thanks in advance for any help & all the best Graeme > -- Kay Diederichs http://strucbio.biologie.uni-konstanz.de email: Kay.Diederichs(a)uni-konstanz.de Tel +49 7531 88 4049 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz This e-mail is digitally signed. If your e-mail client does not have the necessary capabilities, just ignore the attached signature "smime.p7s".

2 1

Windows icc builds
by Winter, Graeme (DLSLtd,RAL,LSCI) 27 May '21

27 May '21

HI Folks Following from a DIALS issue about it being slow on Windows https://github.com/dials/dials/issues/1666 I have fallen into a rabbit hole and I am trying to build DIALS (and hence cctbx) on Windows with the intel compilers from oneAPI - however I keep stumbling over linking errors with e.g. undefined symbol __floor Should I abandon hope or has someone done this? I know this is not a new path to tread as windows_icc was a defined combination on the SConscripts I had had to hack some bits around a little presumably to accommodate 2021 versions of icc compilers, but this felt like I was going down a bad path Would welcome input from anyone who can help - currently windows DIALS is a pretty poor relation It is noteworthy however that parallel operation does now appear to work on Windows with easy mp :-) In other news - has anyone considered adding clang support with clang coming from conda? Could make for more consistency in the builds… I had a look at how one may go about doing this but rapidly found myself in SCons hell. Thanks in advance for any help & all the best Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Building with system compilers
by Winter, Graeme (DLSLtd,RAL,LSCI) 04 May '21

04 May '21

Hi Folks Recent macOS / xcode update broke the version of clang in conda so I can no longer build (SCons decides that the compilers don’t support C++11) I can build just fine, and everything works and all the tests pass However if I try and recompile anything SCons swears at me and dies as it decides that the perfectly valid C++11 compilers I have don’t compile C++11 because … some reason which makes no sense to me. Here is content of config.log Grey-Area build :( $ cat config.log file /Users/graeme/git/dials/modules/cctbx_project/libtbx/SConscript,line 865: Configure(confdir = .sconf_temp) scons: Configure: ".sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.cpp" is up to date. scons: Configure: The original builder output was: |.sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.cpp <- | |#include <iostream> | |int main() { | | std::cout << "{"; | | #ifndef __llvm__ | | #define __llvm__ "" | | #endif | | std::cout << "\"llvm\":" << __llvm__ << ", "; | | #ifndef __clang__ | | #define __clang__ "" | | #endif | | std::cout << "\"clang\":" << __clang__ << ", "; | | #ifndef __clang_major__ | | #define __clang_major__ "" | | #endif | | std::cout << "\"clang_major\":" << __clang_major__ << ", "; | | #ifndef __clang_minor__ | | #define __clang_minor__ "" | | #endif | | std::cout << "\"clang_minor\":" << __clang_minor__ << ", "; | | #ifndef __clang_patchlevel__ | | #define __clang_patchlevel__ "" | | #endif | | std::cout << "\"clang_patchlevel\":" << __clang_patchlevel__ << ", "; | | #ifndef __GNUC__ | | #define __GNUC__ "" | | #endif | | std::cout << "\"GNUC\":" << __GNUC__ << ", "; | | #ifndef __GNUC_MINOR__ | | #define __GNUC_MINOR__ "" | | #endif | | std::cout << "\"GNUC_MINOR\":" << __GNUC_MINOR__ << ", "; | | #ifndef __GNUC_PATCHLEVEL__ | | #define __GNUC_PATCHLEVEL__ "" | | #endif | | std::cout << "\"GNUC_PATCHLEVEL\":" << __GNUC_PATCHLEVEL__ << ", "; | | #ifndef __clang_version__ | | #define __clang_version__ "" | | #endif | | std::cout << "\"clang_version\": \"" << __clang_version__ << "\", "; | | #ifndef __VERSION__ | | #define __VERSION__ "" | | #endif | | std::cout << "\"VERSION\": \"" << __VERSION__ << "\", "; | | std::cout << "}"; | | return 0; | |} | scons: Configure: ".sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.o" is up to date. scons: Configure: The original builder output was: |/usr/bin/c++ -o .sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.o -c .sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.cpp | scons: Configure: ".sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0_79477a053ab4616bb93a6d1a8e51fb1f" is up to date. scons: Configure: The original builder output was: |/usr/bin/c++ -o .sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0_79477a053ab4616bb93a6d1a8e51fb1f .sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0.o | scons: Configure: ".sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0_79477a053ab4616bb93a6d1a8e51fb1f.out" is up to date. scons: Configure: The original builder output was: |.sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0_79477a053ab4616bb93a6d1a8e51fb1f > .sconf_temp/conftest_171626668f451c9275a8effd06a8fea3_0_79477a053ab4616bb93a6d1a8e51fb1f.out | file /Users/graeme/git/dials/modules/cctbx_project/libtbx/SConscript,line 1143: Configure(confdir = .sconf_temp) scons: Configure: ".sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.cpp" is up to date. scons: Configure: The original builder output was: |.sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.cpp <- | |#include <iostream> | |#include <vector> | |#include <array> | |#include <utility> | |std::vector<int> func(std::vector<int> && in) {std::vector<int> r = std::move(in); return r;} | |int main() {std::vector<int> x(9); for(auto & i : x) {i=7; std::cout << i << std::endl;} | | x = func(std::vector<int>(2)); return 0;} | | | scons: Configure: ".sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.o" is up to date. scons: Configure: The original builder output was: |/usr/bin/c++ -o .sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.o -c -std=c++11 -stdlib=libc++ -mmacosx-version-min=10.9 -I/Users/graeme/git/dials/conda_base/include -I/Users/graeme/git/dials/modules -I/Users/graeme/git/dials/modules/cctbx_project -Iannlib_adaptbx/include -I/Users/graeme/git/dials/modules/annlib_adaptbx/include -I/Users/graeme/git/dials/modules/cctbx_project/annlib_adaptbx/include .sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.cpp | /usr/bin/c++ -o .sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0_3b63d4473a8f41665bd62af890bf3763 -w -stdlib=libc++ -mmacosx-version-min=10.9 -std=c++11 -stdlib=libc++ -mmacosx-version-min=10.9 .sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.o -L/Users/graeme/git/dials/conda_base/lib -Llib -L/Users/graeme/git/dials/modules/lib -L/Users/graeme/git/dials/modules/cctbx_project/lib clang: error: no such file or directory: '.sconf_temp/conftest_e73942b1e04111dec1918a18e43ee693_0.o' clang: error: no input files Key point is this: I can blow away the build directory and it all compiles correctly, but if I touch a file and type make I get build failures as so Has anyone a clue about this? Should I add an issue on GitHub? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

Question/a possible typo in sampling backrub and side chain conformations in mmtbx
by Swati Jain 23 Apr '21

23 Apr '21

Hi, Not sure if this message was posted earlier or not, that is why I am reposting it. I had a couple of questions about sampling backrub and sidechains using the building or refinement modules in cctbx_project/mmtbx. 1) In the file: mmtbx.building.alternate_conformations.conformer_generation.py In the function def __call__: the if statements for checking whether the sampled side chain conformation is a rotamer outlier or not, the check in the if statement is for "selfUTLIER". Also in the function *set_up_backrub*(self): the atom "O" for the previous residue is not included in the backrun motion, as a result this atom is not moved when the backrub is implemented. I was just wondering if this is a mistake (possibly all instances of "O" being replaced by "self" which would explain both these things, and some other instances throughout the file), or is this code old and not used anymore? 2) When trying to run fit_residues.py and tst_fit_residues* scripts in mmtbx.refinement.real_space, I encountered errors mainly related to chem_data/rotamer_chi_angles/ not being present. Looks like this data is not distributed with cctbx. Is this correct, or maybe there is some other known issue ? I look forward to hearing from you. Thanks, Swati

5 7

Question/a possible typo in sampling backrub and side chain conformations in mmtbx
by Swati Jain 20 Apr '21

20 Apr '21

Hi, I had a couple of questions about sampling backrub and sidechains using the building or refinement modules in cctbx_project/mmtbx. 1) In the file: mmtbx.building.alternate_conformations.conformer_generation.py In the function def __call__: the if statements for checking whether the sampled side chain conformation is a rotamer outlier or not, the check in the if statement is for "selfUTLIER". Also in the function *set_up_backrub*(self): the atom "O" for the previous residue is not included in the backrun motion, as a result this atom is not moved when the backrub is implemented. I was just wondering if this is a mistake (possibly all instances of "O" being replaced by "self" which would explain both these things, and some other instances throughout the file), or is this code old and not used anymore? 2) When trying to run fit_residues.py and tst_fit_residues* scripts in mmtbx.refinement.real_space, I encountered errors mainly related to chem_data/rotamer_chi_angles/ not being present. Looks like this data is not distributed with cctbx. Is this correct, or maybe there is some other known issue ? I look forward to hearing from you. Thanks, Swati

2 1

MTZ object creation bug or feature?
by Robert Oeffner 30 Mar '21

30 Mar '21

I wonder if anyone has noticed the behaviour that the creation of columns in an mtz object seems to suffer from a problem. The script below creates an mtz object from three miller arrays of intensities, amplitudes and anomalous differences together with their sigmas on 5 reflections. The intention is that the mtz object should also consist of three pairs of columns ( I,SIGI and F,SIGF and DANO,SIGDANO). However the script shows that if the miller array with anomalous differences is added as the last miller array then the resulting mtz object is messed up: It has grouped F,SIGF and DANO,SIGDANO together as 10 reflections instead of just 5. Only if the miller array with anomalous differences is added as the second miller array after the intensities does the resulting mtz object contain the three distinct pairs of columns I,SIGI and F,SIGF and DANO,SIGDANO as intended each with 5 reflections. from cctbx.xray import observation_types from cctbx.array_family import flex from cctbx import miller from cctbx import crystal xs = crystal.symmetry(unit_cell=(50,50,40, 90,90,120), space_group_symbol="P3 1") mi = flex.miller_index([(1,-2,3), (0,0,-3), (1, 2, 3), (0, 1, 2), (1, 0, 2)]) ma1 = miller.array( miller.set(xs, mi), flex.double( [ 11.2, 6.3, 26.1, 14.9, 2.4 ] ), sigmas=flex.double( [ 13.6, 6.3, 24.9, 6.2, 11.1 ] ) ) ma2 = miller.array( miller.set(xs, mi), flex.double( [ 8.2, 26.3, 2.1, 4.9, 32.4 ] ), sigmas=flex.double( [ 3.6, 76.3, 2.9, 61.2, 1.1 ] ) ) ma3 = miller.array( miller.set(xs, mi), flex.double( [ 8.2, 26.3, 2.1, 4.9, 32.4 ] ), sigmas=flex.double( [ 3.6, 76.3, 2.9, 61.2, 1.1 ] ) ) mtz1 = ma1.as_mtz_dataset(column_root_label = "I", column_types = "JQ") mtz1.add_miller_array(ma2, column_root_label = "F", column_types = "FQ") mtz1.add_miller_array(ma3, column_root_label = "DANO", column_types = "DQ") mtz1.mtz_object().as_miller_arrays() # unexpected F,SIGF,DANO,SIGDANO array looks messed up mtz2 = ma1.as_mtz_dataset(column_root_label = "I", column_types = "JQ") mtz2.add_miller_array(ma3, column_root_label = "DANO", column_types = "DQ") mtz2.add_miller_array(ma2, column_root_label = "F", column_types = "FQ") mtz2.mtz_object().as_miller_arrays() # looks correct Is this a bug, a well known feature or am I simply doing something wrong? Many thanks, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

3 3

Generate mtz from pdb
by Academic Research 07 Mar '21

07 Mar '21

Dear All, Is it possible to generate an .mtz FFT file from a .pdb file in cctbx? the equivalent of "sfall XYZIN filename.pdb HKLOUT filename.mtz" in CCP4? Regards, AC

2 1

testing newer versions of boost
by Robert Oeffner 12 Feb '21

12 Feb '21

Hi, I'm testing newer versions of boost than 1.70 currently distributed with CCTBX. I can't remember if I have to do something special other than unpacking the zip file, say boost_1_74_0.zip, that I download from boost.org and put it into the modules directory for my cctbx build. But I get several errors during build like: C:\Users\Oeffner\OeffnerStuff\Work\PhenixVoyager\modules\boost\libs\filesystem\src\codecvt_error_category.cpp(71): error C2491: 'boost::filesystem::codecvt_error_category': definition of dllimport function not allowed I'm have tried with the VS2015, VS2017 and the VS2019 compiler and they all complain. So I'm wondering if there are some other magic invocations that have to be done before building with a new version of boost. Thanks in advance for any suggestions, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

2 2

Find nearest orthonormal matrix
by Robert Oeffner 26 Jan '21

26 Jan '21

Does the CCTBX has a routine for that? I didn't find anything in the scitbx module. But I came across a procedure online here: http://people.csail.mit.edu/bkph/articles/Nearest_Orthonormal_Matrix.pdf If not already in the CCTBX I plan to include a pythonic version of it within the next few days. Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

3 2

(not really cctbx: PHENIX release bundle content: .o files)
by Winter, Graeme (DLSLtd,RAL,LSCI) 25 Jan '21

25 Jan '21

Hi All, Could not think of a better place to ask this question Installed a PHENIX update this morning and noticed it is quite a substantial download - hard to avoid, in this modern day. While unpacking I noticed it contained a lot of object files - totting it up seems to be ~ 450 MB of .o files which are probably not useful for the average end user. Just wondered, are these worth stripping out before making the release bundles? I note the same is true for DIALS so I am not claiming some moral high ground here & also https://github.com/dials/dials/discussions/1563 … looks like my colleagues decided to pick up on this & happened to agree Just thought worth asking Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Space-saving tips
by Tristan Croll 25 Jan '21

25 Jan '21

Hi all, Following on from Graeme's email about .o files, another way to quite dramatically reduce the size of the distribution (with little impact on code complexity or performance) would be to keep all the various text-based data files in .zip format. With modern Python fetching the data from these is little different from working with uncompressed files anyway (1-2 extra lines of code, generally negligible runtime cost). A couple of snippets as an example from ISOLDE (where I keep all MD ligand definition files in a .zip): To get a list of the file contents: def _ligand_db_from_zip(self, ligand_zip): from zipfile import ZipFile import os namelist = [] with ZipFile(ligand_zip) as zf: for fname in zf.namelist(): name, ext = os.path.splitext(fname) if ext.lower() == '.xml': namelist.append(name) return namelist To read a given file from the zip as needed: if ligand_db is not None: zip, namelist = ligand_db if name in namelist: from zipfile import ZipFile logger.info('Loading residue template for {} from internal database'.format(name)) with ZipFile(zip) as zf: with zf.open(name+'.xml') as xf: forcefield.loadFile(xf) Even just doing this for the CaBLAM contour data cuts about 160 MB off the size of the uncompressed distribution. Best regards, Tristan

1 0

Cleaning clutter / pre-commit hooks
by Winter, Graeme (DLSLtd,RAL,LSCI) 25 Jan '21

25 Jan '21

Greetings cctbxbb Was reminded a moment ago about clutter and pre-commit hooks and wondered if there was merit in revisiting this question for cctbx? Not suggesting that pre-commit hooks should be mandatory (though you can git commit -n even if they are) but thinking that it would be a useful way of avoiding doubling up on commits and “failed builds” due to clutter Welcome viewpoints If consensus is not against the use of pre-commit hooks we can try and adapt some from e.g. dials for use here but I would not bother with this if people prefer to manually remove clutter Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

alternative to scipy.linalg.eig
by Robert Oeffner 11 Dec '20

11 Dec '20

scipy.linalg.eig is useful for diagonalising a 3x3 matrix to find eigenvalues and eigenvectors. Does anybody know if there is code in CCTBX which can also do this? Rob -- This email has been checked for viruses by AVG. https://www.avg.com

3 2

Power disruption (12/1 11 pm to 12/2 2 pm)
by Billy Poon 02 Dec '20

02 Dec '20

Hi everyone, Sorry for the short notice, but due to some electrical work in our building, several of the Berkeley servers will be shut down later tonight at 11 pm Pacific and will return tomorrow afternoon at 2 pm Pacific. This will affect: - The Phenix web site (http://phenix-online.org/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

Re: [cctbxbb] [git/cctbx_project] master: de-clutter (e9be98075a)
by Robert Oeffner 07 Oct '20

07 Oct '20

Hi Graeme, It's just a simple call to the libtbx.clean_clutter script. But with the path set explicitly so it only works for this clone of the repo. Rob [hook "precommit"] cmdline = C:\\Users\\Oeffner\\Work\\Phenix\\phenix-installer-dev-3981-intel-windows-x86_64\\build\\bin\\libtbx.clean_clutter.bat wait = true show = true On 07-10-2020 14:33, Winter, Graeme (DLSLtd,RAL,LSCI) wrote: > Hi Rob, > > No worries - the deluge of commits last night with the same fricken > message have made my hyper sensitive and I have had a pet hate of > clean clutter for a long while, and your commit happened to come along > while I had it in my head > > I really think universal pre-commit hooks would be _a good thing_ > > if you wanted to circulate the hook in question you may also help > others… though I seem to recall there are some who dislike things like > pre-commit hooks as somehow impinging on their freedom or similar… > > Cheers Graeme > >> On 7 Oct 2020, at 14:23, Robert Oeffner <rdo20(a)cam.ac.uk >> <mailto:[email protected]>> wrote: >> >> Sorry about the noise in the commit history. I have now put a >> pre-commit hook in my git settings that runs clean_clutter on my >> clone of the cctbx repo. >> >> Rob >> >> >> On 07-10-2020 13:36, Winter, Graeme (DLSLtd,RAL,LSCI) wrote: >>> Hi Folks >>> >>> Would anyone be offended if we had a pre-commit hook which ran the >>> clutter / unused imports tests to stop the history fulling up with >>> clean clutter commits? >>> >>> If clutter is important, probably should not let it in in the first >>> place? >>> >>> I’m not suggesting anything extreme like black, like we use for >>> dxtbx and dials, just running the tools that developers are supposed >>> to run before commits >>> >>> Cheers Graeme >>> >>>> Begin forwarded message: >>>> >>>> *From: *CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM >>>> <mailto:[email protected]>> >>>> *Subject: **[git/cctbx_project] master: de-clutter (e9be98075a)* >>>> *Date: *7 October 2020 at 12:56:50 BST >>>> *To: *CCTBX-COMMIT(a)JISCMAIL.AC.UK <mailto:[email protected]> >>>> *Reply-To: *cctbxbb(a)PHENIX-ONLINE.ORG >>>> <mailto:[email protected]> >>>> >>>> Repository : https://github.com/cctbx/cctbx_project/ >>>> <https://github.com/cctbx/cctbx_project/> >>>> On branch : master >>>> Link : >>>> https://github.com/cctbx/cctbx_project/commit/e9be98075a31f4584f1eca41f9eba… >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> commit e9be98075a31f4584f1eca41f9eba5e408d0cb8f >>>> Author: Robert Oeffner <rdo20(a)cam.ac.uk <mailto:[email protected]>> >>>> Date: Wed Oct 7 12:56:50 2020 +0100 >>>> >>>> de-clutter >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> e9be98075a31f4584f1eca41f9eba5e408d0cb8f >>>> cctbx/miller/__init__.py | 0 >>>> 1 file changed, 0 insertions(+), 0 deletions(-) >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> To unsubscribe from the CCTBX-COMMIT list, click the following link: >>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1 >>>> >>> >>> >>> -- >>> >>> This e-mail and any attachments may contain confidential, copyright >>> and or privileged material, and are for the use of the intended >>> addressee only. If you are not the intended addressee or an >>> authorised recipient of the addressee please notify us of receipt by >>> returning the e-mail and do not use, copy, retain, distribute or >>> disclose the information in or attached to the e-mail. >>> Any opinions expressed within this e-mail are those of the >>> individual and not necessarily of Diamond Light Source Ltd. >>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any >>> attachments are free from viruses and we cannot accept liability for >>> any damage which you may sustain as a result of software viruses >>> which may be transmitted in or with the message. >>> Diamond Light Source Limited (company no. 4375679). Registered in >>> England and Wales with its registered office at Diamond House, >>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 >>> 0DE, United Kingdom >>> >>> >>> _______________________________________________ >>> cctbxbb mailing list >>> cctbxbb(a)phenix-online.org >>> http://phenix-online.org/mailman/listinfo/cctbxbb >> -- >> Robert Oeffner, Ph.D. >> Research Associate, The Read Group >> Department of Haematology, >> Cambridge Institute for Medical Research >> University of Cambridge >> Cambridge Biomedical Campus >> The Keith Peters Building >> Hills Road >> Cambridge CB2 0XY >> www.cimr.cam.ac.uk/investigators/read/index.html >> tel: +44(0)1223 763234 >> >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> >> Virus-free. www.avg.com >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> >> >> > > -- > > This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended > addressee only. If you are not the intended addressee or an authorised > recipient of the addressee please notify us of receipt by returning > the e-mail and do not use, copy, retain, distribute or disclose the > information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for > any damage which you may sustain as a result of software viruses which > may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United > Kingdom > -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

1 0

Fwd: [git/cctbx_project] master: de-clutter (e9be98075a)
by Winter, Graeme (DLSLtd,RAL,LSCI) 07 Oct '20

07 Oct '20

Hi Folks Would anyone be offended if we had a pre-commit hook which ran the clutter / unused imports tests to stop the history fulling up with clean clutter commits? If clutter is important, probably should not let it in in the first place? I’m not suggesting anything extreme like black, like we use for dxtbx and dials, just running the tools that developers are supposed to run before commits Cheers Graeme Begin forwarded message: From: CCTBX commit <diamondlightsource.jenkins(a)GMAIL.COM<mailto:[email protected]>> Subject: [git/cctbx_project] master: de-clutter (e9be98075a) Date: 7 October 2020 at 12:56:50 BST To: CCTBX-COMMIT(a)JISCMAIL.AC.UK<mailto:[email protected]> Reply-To: cctbxbb(a)PHENIX-ONLINE.ORG<mailto:[email protected]> Repository : https://github.com/cctbx/cctbx_project/ On branch : master Link : https://github.com/cctbx/cctbx_project/commit/e9be98075a31f4584f1eca41f9eba… ________________________________ commit e9be98075a31f4584f1eca41f9eba5e408d0cb8f Author: Robert Oeffner <rdo20(a)cam.ac.uk<mailto:[email protected]>> Date: Wed Oct 7 12:56:50 2020 +0100 de-clutter ________________________________ e9be98075a31f4584f1eca41f9eba5e408d0cb8f cctbx/miller/__init__.py | 0 1 file changed, 0 insertions(+), 0 deletions(-) ________________________________ To unsubscribe from the CCTBX-COMMIT list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCTBX-COMMIT&A=1 -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Clutter etc.
by Winter, Graeme (DLSLtd,RAL,LSCI) 03 Oct '20

03 Oct '20

Hi Folks Please could we make less of a big deal about clutter? I have been hacking away for weeks at a PR & errors such as 2020-10-03T09:39:50.9328925Z Generating script. 2020-10-03T09:39:50.9348357Z ========================== Starting Command Output =========================== 2020-10-03T09:39:50.9378363Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/2787f38a-d1cc-4c10-b0bd-c9b041b41631.sh 2020-10-03T09:40:32.0565443Z 2020-10-03T09:40:32.0566065Z mmtbx/model/model.py: unused imports=1 2020-10-03T09:40:32.0566273Z 2020-10-03T09:40:32.0566573Z *** To clean up please use: libtbx.find_unused_imports_crude *** 2020-10-03T09:40:32.0566824Z 2020-10-03T09:40:32.0711947Z 2020-10-03T09:40:32.0785277Z ##[error]Bash exited with code '1'. 2020-10-03T09:40:32.0802183Z ##[section]Finishing: Check for clutter 2020-10-03T09:40:32.0831426Z ##[section]Starting: Checkout cctbx/cctbx_project@refs/pull/533/merge to ./modules/cctbx_project 2020-10-03T09:40:32.0841670Z ============================================================================== 2020-10-03T09:40:32.0842063Z Task : Get sources 2020-10-03T09:40:32.0842444Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories. 2020-10-03T09:40:32.0842850Z Version : 1.0.0 2020-10-03T09:40:32.0843106Z Author : Microsoft 2020-10-03T09:40:32.0843503Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199) 2020-10-03T09:40:32.0843947Z ============================================================================== 2020-10-03T09:40:32.4896538Z Cleaning any cached credential from repository: cctbx/cctbx_project (GitHub) 2020-10-03T09:40:32.4948059Z ##[section]Finishing: Checkout cctbx/cctbx_project@refs/pull/533/merge to ./modules/cctbx_project 2020-10-03T09:40:32.5010291Z ##[section]Starting: Finalize Job 2020-10-03T09:40:32.5043249Z Cleaning up task key 2020-10-03T09:40:32.5044549Z Start cleaning up orphan processes. 2020-10-03T09:40:32.5217555Z ##[section]Finishing: Finalize Job 2020-10-03T09:40:32.5261526Z ##[section]Finishing: syntax python2.7 are making it hard to actually make the tests pass (this change is something I have not touched) Is it really worth breaking builds in PR’s because of some white space / unused imports in unrelated files? Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 2

Simple C++11 async example
by Tristan Croll 02 Oct '20

02 Oct '20

Hi all, For anyone who's interested: following on from my quick demo of the use of std::async to simplify threading of long-running tasks called from Python, I've put some example code at https://github.com/tristanic/async-hello-world/tree/master/src (implemented as a ChimeraX plugin for demo purposes, but the C++ code and bindings are entirely generic). To re-iterate what this is about, and why I think it's important: in brief, for most real-world problems Python-level threading is not particularly useful since (except in specific cases) the Global Interpreter Lock limits the overall performance to no better than - and often much worse than - single-threaded code. While multiprocessing can be an option for some tasks, it's (a) slow to get started, (b) memory-hungry due to the need for each process to have a complete copy of all the data, (c) rather difficult to organise with respect to intercommunication, and (d) not very cross-platform friendly (I'm looking at you, Windows!). The point of this example is to show that with C++11 organising the threading at the C++ level can often end up being both the simplest and fastest approach - it's only a dozen or so extra lines of fairly boilerplate code required to allow the Python programmer to choose between running the exact same function either in the main thread or asynchronously in a new one. Why I think it's important: I've said in the past, and continue to believe, that we should be aiming for (interactive) model building, validation and refinement to merge together into a single integrated process - as much as anything, ISOLDE is meant to be the proof of principle that this is a possibility. But for that to work, high-performance and cross-platform parallelism is a real necessity. The approach described here is how I'm currently getting ChimeraX to maintain 30+ fps graphics performance while simultaneously running interactive molecular dynamics and structure factor calculations. I suspect there are many functions in CCTBX that could potentially benefit from this approach even when not considering GUI applications. Best regards, Tristan

5 6

boost python conversions help needed
by Winter, Graeme (DLSLtd,RAL,LSCI) 01 Oct '20

01 Oct '20

Hi All, In the branch https://github.com/cctbx/cctbx_project/tree/fixed-width-int-types I am working to add a full range of fixed-width integer types to allow explicit reading of binary data without conversion from images (amongst other things) - mostly I’ve got something _working_ but tests fail because stdout / err contains Standard error: /Users/graeme/svn/cctbx/conda_base/python.app/Contents/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: to-Python converter for scitbx::af::shared<unsigned int> already registered; second conversion method ignored. has anyone got any hints where these may be registered? The details of the low level flex implementation seems to be minimally documented, and the error has no useful stack trace, so I am at a bit of a loss as to where the other unsigned int conversion is registered. Some pointers from someone who has delved into these in the past would be welcome - I have spent a couple days trying to find what I am doing wrong and have so far failed - hopefully someone on the list knows their way around in here Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 4

Documentation on caching mechanisms implemented in structure factor calculation
by Jan M. Simons 30 Sep '20

30 Sep '20

Hello fellow researchers and scientists, It's been a while since I actively worked with cctbx, but now I wonder if there is some in depth documentation on how the structure factor calculation in the cctbx xray module is done. I'm especially interested in caching strategies employed to speed up the calculation during refinement runs (especially in the case of small molecules using direct summation). So is there a kind of caching that keeps (partial) structure factor contributions of each individual scatterer cached so that only the partials of shifted scatterers have to be recalculated? If so is there a python interface for accessing those? Also is there an option to simulate/map the f_calc amplitudes onto a 1dimensional 2theta or d* axis to compare those to powder diffraction patterns? Thanks in advance for giving me some guidance on this (especially if there have been some recent additions to these modules). Cheers Jan

3 2

Warnings (by the hundredweight)
by Winter, Graeme (DLSLtd,RAL,LSCI) 25 Sep '20

25 Sep '20

Hi Folks Spent this afternoon looking at https://github.com/cctbx/cctbx_project/issues/532 which involves touching C++ header code deep in scitbx - which is fine - but every recompile generates hundreds of warnings from all over the place so your actual compiler errors are completely obscured Massive amounts of deprecation errors everywhere e.g. In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/maptbx/asymmetric_map.cpp:5: In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/maptbx/asymmetric_map.h:18: In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/asymmetric_unit.h:6: In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/direct_space_asu.h:9: /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/facet_collection.h:72:34: warning: 'pointer' is deprecated [-Wdeprecated-declarations] faces_ = facet_collection::pointer(a.faces_->new_copy()); In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/asymmetric_unit.h:6: In file included from /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/direct_space_asu.h:9: /Users/graeme/svn/cctbx/modules/cctbx_project/cctbx/sgtbx/direct_space_asu/proto/facet_collection.h:78:33: warning: 'pointer' is deprecated [-Wdeprecated-declarations] faces(const facet_collection::pointer &a) : faces_(a->new_copy()) {} ^ /Users/graeme/svn/cctbx/conda_base/bin/../include/c++/v1/memory:2130:28: note: 'auto_ptr<cctbx::sgtbx::asu::facet_collection>' has been explicitly marked deprecated here class _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX11 auto_ptr ^ /Users/graeme/svn/cctbx/conda_base/bin/../include/c++/v1/__config:972:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11' # define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED ^ /Users/graeme/svn/cctbx/conda_base/bin/../include/c++/v1/__config:961:48: note: expanded from macro '_LIBCPP_DEPRECATED' # define _LIBCPP_DEPRECATED __attribute__ ((deprecated)) is there any appetite on the list to reduce the number of compiler warnings to make development easier? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

Re: [cctbxbb] [EXTERNAL] cctbxbb Digest, Vol 138, Issue 2
by Mooers, Blaine H.M. (HSC) 24 Sep '20

24 Sep '20

Hi Billy, I installed cctbx in a new conda env called cctbx37: conda create --name cctbx37 python=3.7 conda activate cctbx37 conda install -c conda-forge cctbx jupyterlab I cannot find libtbx.run_tests_module. I can only find libtbx.run_tests_parallel. Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular Biology College of Medicine University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center (BRC) Rm. 466 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Wednesday, September 23, 2020 2:00 PM To: cctbxbb(a)phenix-online.org Subject: [EXTERNAL] cctbxbb Digest, Vol 138, Issue 2 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. cctbx now available on conda-forge (Billy Poon) ---------------------------------------------------------------------- Message: 1 Date: Wed, 23 Sep 2020 11:22:03 -0700 From: Billy Poon <BKPoon(a)lbl.gov> To: cctbx mailing list <cctbxbb(a)phenix-online.org>, PHENIX user mailing list <phenixbb(a)phenix-online.org> Subject: [cctbxbb] cctbx now available on conda-forge Message-ID: <CAHmYUcZwVDKZ6v1iuA-nP8AJn1cfp3sNWtiqeo=fcWPq9VbpqQ(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi everyone, The conda-forge package for cctbx is now available. There are 2 packages, "cctbx-base" includes the base code and "cctbx" which includes "cctbx-base" and adds "wxPython", "PySide2", and "ipython". The commands for installing cctbx into your conda environment are conda install -c conda-forge cctbx or conda install -c conda-forge cctbx-base If you want to use Jupyter Lab, you can now easily do this with conda install -c conda-forge cctbx-base jupyterlab jupyter lab The existing documentation will be updated to reflect the easier installation procedure with conda. After installing the conda package, you can also run some of the tests (in an empty directory) with libtbx.run_tests_module module=libtbx module=cctbx nproc=<nproc> I do know of a few issues that I want to fix in upcoming releases, but please submit issues to https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… with steps to reproduce the error. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=…

2 1

cctbx now available on conda-forge
by Billy Poon 24 Sep '20

24 Sep '20

Hi everyone, The conda-forge package for cctbx is now available. There are 2 packages, "cctbx-base" includes the base code and "cctbx" which includes "cctbx-base" and adds "wxPython", "PySide2", and "ipython". The commands for installing cctbx into your conda environment are conda install -c conda-forge cctbx or conda install -c conda-forge cctbx-base If you want to use Jupyter Lab, you can now easily do this with conda install -c conda-forge cctbx-base jupyterlab jupyter lab The existing documentation will be updated to reflect the easier installation procedure with conda. After installing the conda package, you can also run some of the tests (in an empty directory) with libtbx.run_tests_module module=libtbx module=cctbx nproc=<nproc> I do know of a few issues that I want to fix in upcoming releases, but please submit issues to https://github.com/cctbx/cctbx_project with steps to reproduce the error. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

E value calculation
by Academic Research 03 Sep '20

03 Sep '20

Hello Everyone, I am trying to find how to perform the E value calculation using CCTBX. I am unable to find the method in the documentation. Can someone assist me, please? Regards, AC Research

3 2

Changes to boost_adaptbx and
by Billy Poon 19 Aug '20

19 Aug '20

Hi everyone, Two pull requests were merged today that make some important changes. 1) The boost imports were moved to boost_adaptbx.boost. This avoids claiming the main "boost" namespace. A common piece of code that will need to be changed is import boost.python ext = boost.python.import_ext("your_boost_python_ext") That can be changed to import boost_adaptbx.boost.python as bp ext = bp.import_ext("your_boost_python_ext") Otherwise, a FutureWarning will be raised. More information can be found at https://github.com/cctbx/cctbx_project/issues/458 https://github.com/cctbx/cctbx_project/pull/470 https://github.com/cctbx/cctbx_project/pull/516 2) More dependency repositories were moved from cci.lbl.gov to GitHub. If you use bootstrap.py to build cctbx, you will want to delete annlib, ccp4io_adaptbx, clipper, eigen, gui_resources, tntbx. Using a newly updated bootstrap.py, the copies from GitHub will be used instead. More information can be found at https://github.com/cctbx/cctbx_project/pull/408 Let us know if you have any questions. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

unmerged data as mtz object
by Robert Oeffner 05 Aug '20

05 Aug '20

Hi, Does anybody know how to convert a miller array of unmerged data to an mtz object? For merged data I do something like this: mtzobj = millarr.as_mtz_dataset(column_root_label="I") But if millarr contains multiple observations with the same hkl index. Then the above code fails with the exception cctbx Error: Duplicate entries in miller_indices array. Any suggestions? Many thanks, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -- This email has been checked for viruses by AVG. https://www.avg.com

4 5

flex.histogram giving ArgumentError when arguments correct
by Winter, Graeme (DLSLtd,RAL,LSCI) 04 Aug '20

04 Aug '20

H = flex.histogram(coords, data_min=0.0, data_max=dim, n_slots=dim) Gives Boost.Python.ArgumentError: Please report this error to dials-support(a)lists.sourceforge.net: Python argument types in histogram.__init__(histogram) did not match C++ signature: __init__(_object*, double, double, double, scitbx::af::shared<long>, unsigned long) __init__(_object*, scitbx::histogram<double, long> other, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> data, double relative_tolerance=0.0001) __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> data, double data_min, double data_max, unsigned long n_slots=1000, double relative_tolerance=0.0001) __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> data, unsigned long n_slots=1000) Despite coords being <scitbx_array_family_flex_ext.double object at 0x7fa8d671b5b0> with shape (31294944,) and dim being 18093576 (i.e. integer) and h = flex.histogram(a, data_min=0, data_max=100, n_slots=100) working fine in another context i.e. the calling convention is correct This has so far cost a few hours, so what gives? How can this be holding it wrong? C++ code suggests that the bin types are long so this should be fine Any hints? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 5

Python development platform
by Winter, Graeme (DLSLtd,RAL,LSCI) 18 Jul '20

18 Jul '20

Hi Folks, Came up in conversation earlier - what version of Python are people using for their day-to-day development? For what it’s worth I’m currently on a 3.6 base, though wondering about moving to e.g. 3.8 Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

How to extract I(+) and I(-) from miller array?
by Mooers, Blaine H.M. (HSC) 10 Jun '20

10 Jun '20

Hi Pavel, Thank you very much for the leads. They were very helpful. I was able to make the desired plot. Best regards, Blaine ________________________________________ From: Pavel Afonine [pafonine(a)lbl.gov] Sent: Tuesday, June 09, 2020 2:07 PM To: Mooers, Blaine H.M. (HSC); cctbx mailing list Subject: [EXTERNAL] Re: [cctbxbb] How to extract I(+) and I(-) from miller array? Hi Blaine, this is correct, this is how anomalous data are read in and stored in an instance of miller.array class. In this case I(+) and I(-) will be stored in one miller_array, anomalous_flag will be set to True, and .indices() will contain 'plus' and 'minus' mates as well as singletons. Now, if you want to get separate I(+) and I(-), for an example look in def anomalous_signal(self, use_binning=False): def anomalous_differences(self, enforce_positive_sigmas=False): def hemisphere_acentrics(self, plus_or_minus): def hemispheres_acentrics(self): def convert_to_non_anomalous_if_ratio_pairs_lone_less_than(self, threshold): in cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit… . Pavel On 6/9/20 07:55, Mooers, Blaine H.M. (HSC) wrote: > Dear CCTBX experts, > > The diffraction data for 3hz7 from RC Researcher's question includes one dataset with column labels of 'I(+)', 'SIGI(+)', 'I(-)', 'SIGI(-)'. > When the diffraction data of 3hz7 data are read from a mtz file into miller arrays, > it would appear that the I(+) and I(-) are stored in a single column of intensities rather than two parallel columns. > > from iotbx.reflection_file_reader import any_reflection_file > > hkl_file = any_reflection_file('3hz7.mtz') > miller_arrays = hkl_file.as_miller_arrays(merge_equivalents=False) > Iobs = miller_arrays[1] > list(Iobs[100:105]) > > [((7, 4, 38), 84.0, 52.20000076293945), > ((-7, -4, -38), 112.80000305175781, 48.20000076293945), > ((8, 4, 38), 119.69999694824219, 57.400001525878906), > ((-8, -4, -38), 152.10000610351562, 65.9000015258789), > ((6, 5, 38), 732.5, 57.5)] > > How would I extract the I(+) and I(-) into lists to plot against each other in a scatter plot in matplotlib? > > > Best regards, > > Blaine > > Blaine Mooers, Ph.D. > Associate Professor > Department of Biochemistry and Molecular Biology > College of Medicine > University of Oklahoma Health Sciences Center > S.L. Young Biomedical Research Center (BRC) Rm. 466 > 975 NE 10th Street, BRC 466 > Oklahoma City, OK 73104-5419 > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai…

2 1

How to extract I(+) and I(-) from miller array?
by Mooers, Blaine H.M. (HSC) 10 Jun '20

10 Jun '20

Dear CCTBX experts, The diffraction data for 3hz7 from RC Researcher's question includes one dataset with column labels of 'I(+)', 'SIGI(+)', 'I(-)', 'SIGI(-)'. When the diffraction data of 3hz7 data are read from a mtz file into miller arrays, it would appear that the I(+) and I(-) are stored in a single column of intensities rather than two parallel columns. from iotbx.reflection_file_reader import any_reflection_file hkl_file = any_reflection_file('3hz7.mtz') miller_arrays = hkl_file.as_miller_arrays(merge_equivalents=False) Iobs = miller_arrays[1] list(Iobs[100:105]) [((7, 4, 38), 84.0, 52.20000076293945), ((-7, -4, -38), 112.80000305175781, 48.20000076293945), ((8, 4, 38), 119.69999694824219, 57.400001525878906), ((-8, -4, -38), 152.10000610351562, 65.9000015258789), ((6, 5, 38), 732.5, 57.5)] How would I extract the I(+) and I(-) into lists to plot against each other in a scatter plot in matplotlib? Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular Biology College of Medicine University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center (BRC) Rm. 466 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419

2 1

F_obs values
by Mooers, Blaine H.M. (HSC) 09 Jun '20

09 Jun '20

Dear AC Researh, To follow Pavel's suggestion, print your miller arrays, get their length, and then loop over the arrays and print the column labels. len(miller_arrays) # which returns 2 for your example. [print("Miller array %s: %s" % (i, miller_array.info().labels)) for i, miller_array in list(enumerate(miller_arrays))[:2]] Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular BiologyUniversity of Oklahoma Health Sciences Center Oklahoma City, OK 73104-5419 ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Monday, June 08, 2020 2:00 PM To: cctbxbb(a)phenix-online.org Subject: [EXTERNAL] cctbxbb Digest, Vol 135, Issue 1 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. F_obs values (Academic Research) 2. Re: F_obs values (Pavel Afonine) ---------------------------------------------------------------------- Message: 1 Date: Sun, 07 Jun 2020 21:34:50 -0000 From: Academic Research <ac.research(a)icloud.com> To: cctbxbb(a)phenix-online.org Subject: [cctbxbb] F_obs values Message-ID: <9b1bc3ab-2421-4faa-b6f4-fd1ecd63415a(a)me.com> Content-Type: text/plain; charset="utf-8"; Format="flowed" Hello Everyone, I am trying to follow the script on the CCTBX website to get the F_obs values for my 3hz7.mtz file. my code is as follows: from iotbx.reflection_file_reader import any_reflection_file hkl_file = any_reflection_file('3hz7.mtz') arrays = hkl_file.as_miller_arrays(merge_equivalents=False) a = arrays[0] f_obs = a.f_sq_as_f().map_to_asu() I get this following error: Traceback (most recent call last): ? File "https://urldefense.proofpoint.com/v2/url?u=http-3A__crystal.py&d=DwICAg&c=V… ", line 143, in <module> ??? if __name__ == '__main__': main() ? File "https://urldefense.proofpoint.com/v2/url?u=http-3A__crystal.py&d=DwICAg&c=V… ", line 140, in main ??? Cls.features('3hz7.mtz') ? File "https://urldefense.proofpoint.com/v2/url?u=http-3A__crystal.py&d=DwICAg&c=V… ", line 63, in features ??? f_obs = a.f_sq_as_f().map_to_asu() ? File "/home/acresearch/Desktop/CCTBX/mc3/envs/tenv/lib/python3.8/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=oAfuCKnHEMReNwNf8K-J-MEXUEuhmfZEy8PYiJVy68I&s=LeX7G1Xgo9XI-pi0Su5OxLY6PfDhKilIfshCkSXXcPU&e= ", line 2374, in f_sq_as_f ??? result = array(self, converter(self.data()).f) Boost.Python.ArgumentError: Python argument types in ??? array_f_sq_as_f_xtal_3_7.__init__(array_f_sq_as_f_xtal_3_7, int) did not match C++ signature: ??? __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>) ??? __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>) ??? __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, double) When I run: print(i_obs.info()) I get: 3hz7.mtz:FREE print(a.observation_type()) gives: None I repeated this script to many structure from the PDB (all solved by x-rays) yet I get the exact same output for all of them. How can I get the F_obs values? Can someone please assist me? Regards, AC Research

1 0

F_obs values
by Academic Research 08 Jun '20

08 Jun '20

Hello Everyone, I am trying to follow the script on the CCTBX website to get the F_obs values for my 3hz7.mtz file. my code is as follows: from iotbx.reflection_file_reader import any_reflection_file hkl_file = any_reflection_file('3hz7.mtz') arrays = hkl_file.as_miller_arrays(merge_equivalents=False) a = arrays[0] f_obs = a.f_sq_as_f().map_to_asu() I get this following error: Traceback (most recent call last): File "crystal.py", line 143, in <module> if __name__ == '__main__': main() File "crystal.py", line 140, in main Cls.features('3hz7.mtz') File "crystal.py", line 63, in features f_obs = a.f_sq_as_f().map_to_asu() File "/home/acresearch/Desktop/CCTBX/mc3/envs/tenv/lib/python3.8/site-packages/cctbx/miller/__init__.py", line 2374, in f_sq_as_f result = array(self, converter(self.data()).f) Boost.Python.ArgumentError: Python argument types in array_f_sq_as_f_xtal_3_7.__init__(array_f_sq_as_f_xtal_3_7, int) did not match C++ signature: __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>) __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>) __init__(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, scitbx::af::const_ref<double, scitbx::af::trivial_accessor>, double) When I run: print(i_obs.info()) I get: 3hz7.mtz:FREE print(a.observation_type()) gives: None I repeated this script to many structure from the PDB (all solved by x-rays) yet I get the exact same output for all of them. How can I get the F_obs values? Can someone please assist me? Regards, AC Research

2 1

Boost 1.70 update for cctbx
by Billy Poon 09 May '20

09 May '20

Hi everyone, There is currently a pull request for cctbx that will update Boost to 1.70 ( https://github.com/cctbx/cctbx_project/pull/409). The update enables support for Python 3.7 and 3.8. The plan is to merge this next Friday, May 8, at 3 pm Pacific. After merging, developers should delete the build/lib/*boost* files and build/boost directory so that any remnants of the old Boost are removed and then rebuild. Let me know if you have any questions. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 1

Question about HHpred parser
by Louis Dumas 25 Apr '20

25 Apr '20

Dear CCTBX developers, I am a postdoc at EPFL working with HHpred for homology modeling of membrane proteins. I have been trying to write my own HHpred alignment parser until I found the python script under “cctbx_fork/iotbx/bioinformatics/__init__.py/” that contains an HHpred parser. My goal is to correctly parse the raw HHpred output file (.hhr), which involves unwrapping every alignment, parsing out a lot of text to finally obtain something like this: >pdb_name query-sequence column score Example: >4U15 VYGFIGGIFGFMSIMTMAMISIDRYNVIGRPMAASKKMSHRRAFIMIIFVWLWS +........+..+..++|+++|++++.++.+.++++ +..+.++.+|+++|++.++...+........ +...|.. Being somewhat new to python, I was wondering whether the people who wrote this script are still around and could help me figure out whether the parser could be implemented in such a way. Thanks for any help you can provide! Best, Louis D

2 1

Find special positions for a spacegroup in sgtbx
by Collins, Steve (DLSLtd,RAL,SCI) 23 Mar '20

23 Mar '20

Hello, I'm trying to find something I can run from Python to get all the special (high-symmetry) positions and usual (e.g. Wyckoff) labels for a particular space group. I can't find this in the sgtbx documentation. Does anyone know if it can be done? Or maybe a different package? Thanks - and stay safe! Steve Collins -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

5 6

New cctbx conda packages for testing
by Billy Poon 21 Mar '20

21 Mar '20

Hi everyone, Sorry for the long delay, the current Phenix release and helping out with an experiment at SACLA earlier in the year, took priority. But I have been able to clean up the cctbx conda packages that I built at the end of last year. The major differences are as follows. 1) Windows packages are available for Python 2.7, 3.6, and 3.7. It's not clear why conda-build is having issues with Python 3.8, so I'm still sorting it out. 2) The full repositories are copied now. As a consequence, you'll be able to run some tests like "libtbx.run_tests_parallel module=cctbx nproc=24" and all the tests will run. 3) The LIBTBX_BUILD environment variable is no longer needed, so it is not set on activation of the conda environment. 4) The libtbx.configure and libtbx.refresh commands are removed from the conda package until the ability to configure/build against the conda installation of cctbx is available. Currently, those commands will break the environment. The linux and macOS packages support Python 2.7, 3.6, 3.7, and 3.8, just like before. The command to create a new environment is, conda create -n test -c cctbx-dev cctbx python=<python version> If "python=<python version>" is left out, the default Python version will be 3.8. To install cctbx into the currently active conda environment, the command is, conda install -c cctbx-dev cctbx The packages are currently on the cctbx-dev channel, but the goal is to get official packages into conda-forge. To get the cctbx recipe into conda-forge (https://github.com/conda-forge/staged-recipes/pull/10021), we'll need an official release. I've opened up an issue on GitHub ( https://github.com/cctbx/cctbx_project/issues/451). Please provide any feedback on the versioning system, release schedule, and anything release related. Let us know if you're scripts are not working with these packages. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

3 4

Python 2 only code
by Winter, Graeme (DLSLtd,RAL,LSCI) 04 Mar '20

04 Mar '20

Morning all, Looking at commits overnight saw https://github.com/cctbx/cctbx_project/commit/69d506ebd0ac212b8166eec98a8d8… which adds Python 2 only code to cctbx - I thought there were things in place which prevent regressions such as this? Error: the following jobs returned non-zero exit codes or suspicious stderr output: libtbx.python "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/tst_hierarchy.py" [FAIL] 1.0s Time: 1.04 Return code: 1 OKs: 0 Standard error: Traceback (most recent call last): File "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/tst_hierarchy.py", line 7082, in <module> exercise(sys.argv[1:]) File "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/tst_hierarchy.py", line 7069, in exercise exercise_other() File "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/tst_hierarchy.py", line 5845, in exercise_other assert pdb_hierarchy.contains_protein() File "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/hierarchy.py", line 1623, in contains_protein classes=['common_amino_acid', 'modified_amino_acid']) File "/Users/graeme/svn/cctbx/modules/cctbx_project/iotbx/pdb/hierarchy.py", line 214, in get_n_residues_of_classes for resname, count in self.resnames.iteritems(): AttributeError: 'dict' object has no attribute ‘iteritems’ Given how much noise you will receive if you leave trailing whitespace in there, I’d like to think real errors like this would be picked up automatically? Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

IUCr2020 Computing School in Nove Hrady - 17th to 22nd August - Registration opened
by Claudia Millán Nebot 24 Feb '20

24 Feb '20

Dear all, We announce the IUCr Computing School in Nove Hrady (Czech Republic) which will take place between 17th and 22nd August 2020 prior to the IUCr Congress in Prague. Registration has already started. Further information about the school can be found on the website: https://www.xray.cz/iucr/workshops/nh/default.htm. Follow also our hashtag on Twitter, #IUCr2020CompSchool. Active developers of crystallographic software, tools and methods are invited to participate at this Computing School. The school is not introductory and it is recommended that participants already have some basic programming skills and crystallographic knowledge. Our aim is to discuss and exchange ideas and to tackle the challenges related to the method development in crystallography. Participants can be early-career scientists (PhD students, Postdocs, etc.) as well as established developers. Our confirmed speakers are: - Simon Billinge (Columbia University, USA) - Luc Bourhis (Bruker, France) - Paul Boyle (Western University, London, Ontario, Canada) - Iracema Caballero (Institut de Biologia Molecular de Barcelona, Spain) - Gemma de la Flor (Karlsruhe Institute of Technology, Germany) - Dorothee Liebschner (Lawrence Berkeley National Laboratory, USA) - Rafiga Masmaliyeva (Institute of Molecular Biology and Biotechnology, Azerbaijan) - Garib Murshudov (Laboratory of Molecular Biology-MRC, UK) - Reinhard Neder (Institute of Condensed Matter Physics, Erlangen, Germany) - Alexandre Ourjoumtsev (Institut de Génétique et de Biologie Moléculaire et Cellulaire, France) - Randy Read (University of Cambridge, UK) - Massimo Sammito (University of Cambridge, UK) - Arkadiy Simonov (Eidgenössische Technische Hochschule Zürich, Switzerland) - Melanie Vollmar (Diamond Light Source Ltd, Harwell Science & Innovation Campus, United Kingdom) - Thomas White (Center for Free-Electron Laser Science, Hamburg, Germany) For any questions regarding the school please contact us. Best wishes, Claudia Millán (cmncri at ibmb.csic.es) Martin Lutz (m.lutz at uu.nl) Jan Rohlicek (rohlicek at fzu.cz) PD. Please find attached a poster about the school that you can forward, together with this email, to people potentially interested in participating in the Computing School.

1 0

map grid sizes
by GEORGE N PHILLIPS 21 Feb '20

21 Feb '20

I am confused about grid sizes of maps. When i specify a grid size for an fft_map but then pull the numpy array out for direct manimpulation, it's always 2 larger in the last dimension. Why is it not the same size? Any help appreciated. George y# Now make initial (smoothed) map using random phases with Fobs f_obs_B = f_obs1.apply_debye_waller_factors( b_iso=biso_init) phases = f_obs_B.random_phases_compatible_with_phase_restrictions(deg=False) phases_data = phases.data() e_map_coeff = f_obs_B.phase_transfer(phase_source = phases_data) e_map = e_map_coeff.fft_map(crystal_gridding = crystal_gridding, symmetry_flags = maptbx.use_space_group_symmetry) print("cgridding", crystal_gridding.n_real()) # pick initial envelope e_map_array = e_map.real_map().as_numpy_array() print("e_map", e_map_array.shape) cgridding (90, 90, 80) e_map (90, 90, 82) George N. Phillips, Jr., Ph.D. Professor Emeritus University of Wisconsin-Madison 433 Babcock Dr. Madison, Wi 53706 Phone 713 348 6951 (Houston)

4 3

a little help for newbie?
by georgep 18 Feb '20

18 Feb '20

I am starting to write some python code using cctbx, but am missing how to use the C++ bindings. I am fine unless a try to access them. Something about my environment or boost of something. Can anyone point me in the right direction? I have import boost.python but am cleary missing some fundamental knowledge. George Traceback (most recent call last): File "/Users/georgep/PycharmProjects/test_env/HIO_Version3.py", line 225, in <module> S0 = percentile[rho_calc_true > percentile(rho_calc_true, solv_frac)] = 1 File "/Users/georgep/PycharmProjects/test_env/HIO_Version3.py", line 216, in percentile order = flex.sort_permutation(x) Boost.Python.ArgumentError: Python argument types in scitbx_array_family_flex_ext.sort_permutation(double) did not match C++ signature: sort_permutation(scitbx::af::const_ref<double, scitbx::af::trivial_accessor> data, bool reverse=False, bool stable=True) sort_permutation(scitbx::af::const_ref<unsigned long, scitbx::af::trivial_accessor> data, bool reverse=False, bool stable=True) sort_permutation(scitbx::af::const_ref<int, scitbx::af::trivial_accessor> data, bool reverse=False, bool stable=True) George N. Phillips, Jr. Ralph and Dorothy Looney Professor BioSciences at Rice Rice University, MS140 6100 Main Street Houston, Texas 77005-1892 Office: +1 713.348.6951 www.phillipslab.org

2 1

CCTBX documentation questionnaire
by Dorothee Liebschner 15 Jan '20

15 Jan '20

Hi everyone, During this year's ECM computing school, it came up that the CCTBX does not have a lot of documentation. This seems to be a major reason why people don't use it, even if they would like to employ the libraries. We would like to find out which parts of the CCTBX are the most interesting for users. We certainly cannot provide extensive documentation for the entire toolbox, but we can see what we can do once we know the main areas of interest. So if you were in the situation of wanting to use the CCTBX but decided to stay away from it because the documentation was non-existent or obscure, please let us know by filling out the questionnaire below (it's anonymous): questionnaire CCTBX documentation <https://forms.gle/9RawJFfHyEvhCVVP7> Best wishes, Dorothee -- Project Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

11 40

conda cctbx on Mac 10.15.2 working with python3.8
by Mooers, Blaine H.M. (HSC) 06 Jan '20

06 Jan '20

Hi Tristan, Thank you very much for the explanation and the suggested solution. I needed both. When I tried your list comprehension in a Jupyter notebook with the orginal problem, it is too fast: ``` IOPub data rate exceeded. The notebook server will temporarily stop sending output to the client in order to avoid crashing it. To change this limit, set the config variable `--NotebookApp.iopub_data_rate_limit`. Current values: NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec) NotebookApp.rate_limit_window=3.0 (secs) ``` When I reduced the a and b edges by an order of magnitude (now 12,118 reflections), your list comprehension is two orders of magnitude faster a for loop (no surprise). It is also slightly faster than this generator expression coupled to a list comprehension, so there is no advantage to using the generator here. hklgen = (hkl for hkl in ms.indices()) print('\n'.join([str(hkl) for hkl in hklgen])) Best regards, Blaine ________________________________________ From: Tristan Croll [tic20(a)cam.ac.uk] Sent: Sunday, January 05, 2020 4:13 AM To: cctbx mailing list Cc: Mooers, Blaine H.M. (HSC) Subject: [EXTERNAL] Re: [cctbxbb] conda cctbx on Mac 10.15.2 working with python3.8 Hi Blaine, The speed issue is probably because this list comprehension: [print(hkl) for hkl in ms.indices()] ... is calling print() (and returning None) a million times over. I think something like the following should be much faster: print('\n'.join([str(hkl) for hkl in ms.indices()])) Best regards, Tristan On 2020-01-04 22:00, Mooers, Blaine H.M. (HSC) wrote: > Hi Billy, > > Gergely's post yesterday stimulated me to check my conda cctbx setup. > I had earlier reported success with conda cctbx for Python3.6, 3.7, and > 3.8. > This morning I found in my path an earlier python3.6 build of cctbx > that may > have given me false positives with my conda installs of cctbx. > After removing the old build of cctbx from my path, I modified my zshrc > file > as Gergely suggested. I can confirm that results that Gergely reported > yesterday. > > I used a Mac OS X 10.15.2 with zsh as my shell and anaconda3 installed > as /opt/anaconda. I did my testing in a conda env named cctbx38. > I added 'export LIBTBX_BUILD=$CONDA_PREFIX' in my .zshrc file. > After sourcing my .zshrc file, 'echo $LIBTBX_BUILD' returns > '/opt/anaconda/envs/cctbx38' as expected. > > The toy script below prints the calculated Miller indices as a tuples > in a terminal window. > It was run using 'python https://urldefense.proofpoint.com/v2/url?u=http-3A__millerSet.py&d=DwICAg&c… '. > > Note that the printing of 1 million indices to a jupyter notebook cell > is disappointingly slow!! > Reduce each unit cell dimension by two orders of magnitude before > trying this at home! > The following code also worked with Python3.6 and Python3.7 in their > corresponding cctbx36 and cctbx37 envs. > > from cctbx import crystal > from cctbx import miller > > ms = miller.build_set( > crystal_symmetry=crystal.symmetry( > space_group_symbol='P4', > unit_cell=(150.8,150.8,250.4,90.0,90.0,90.0)), > anomalous_flag=False, > d_min=1.4) > > [print(hkl) for hkl in ms.indices()] > > I used conda python3.8.0 built on November 22. Python3.8.1 does not > seem to be > available yet from Anaconda. > > Best regards, > > Blaine > > Blaine Mooers, Ph.D. > Associate Professor > Department of Biochemistry and Molecular Biology > College of Medicine > University of Oklahoma Health Sciences Center > S.L. Young Biomedical Research Center (BRC) Rm. 466 > 975 NE 10th Street, BRC 466 > Oklahoma City, OK 73104-5419 > > ________________________________________ > From: cctbxbb-bounces(a)phenix-online.org > [cctbxbb-bounces(a)phenix-online.org] on behalf of > cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] > Sent: Saturday, January 04, 2020 2:00 PM > To: cctbxbb(a)phenix-online.org > Subject: [EXTERNAL] cctbxbb Digest, Vol 130, Issue 2 > > Send cctbxbb mailing list submissions to > cctbxbb(a)phenix-online.org > > To subscribe or unsubscribe via the World Wide Web, visit > > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > or, via email, send a message with subject or body 'help' to > cctbxbb-request(a)phenix-online.org > > You can reach the person managing the list at > cctbxbb-owner(a)phenix-online.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of cctbxbb digest..." > > > Today's Topics: > > 1. Re: some thoughts on cctbx and pip (Billy Poon) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 3 Jan 2020 21:04:25 +0100 > From: Billy Poon <BKPoon(a)lbl.gov> > To: cctbx mailing list <cctbxbb(a)phenix-online.org> > Subject: Re: [cctbxbb] some thoughts on cctbx and pip > Message-ID: > > <CAHmYUcbWS7FXA6EPUx=AC2X6m6gPY=GqH-MwGD+uzpTaOKSyjQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Gergely, > > Happy New Year! > > Can you provide the output to "conda list" for each environment? > > I suspect that the old version of cctbx might still be around. Is there > a > file named PYTHON_VERSION_MAJOR_MINOR in "/home/gergely/anaconda3/lib"? > If > so, delete that file. To make sure you are downloading a new copy of > the > package, clear the cache where conda keeps copies of all the packages. > To > do that, you can run "conda clean -y --all". You can remove the "-y" > flag > if you want to see what will be deleted. The hash for the new linux > package > should be h8b68381_0. > > The last error looks like an incompatible version of boost or gcc > library > was installed. The output from "conda list" should show the boost and > boost-cpp packages coming from conda-forge. I would avoid installing > into > the root or base environment because that usually pulls from the > default > channels. All the dependencies should be pulled from the conda-forge > channel. > > -- > Billy K. Poon > Research Scientist, Molecular Biophysics and Integrated Bioimaging > Lawrence Berkeley National Laboratory > 1 Cyclotron Road, M/S 33R0345 > Berkeley, CA 94720 > Tel: (510) 486-5709 > Fax: (510) 486-5909 > Web: > https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > > On Fri, Jan 3, 2020 at 2:20 PM Gergely Katona <gkatona(a)gmail.com> > wrote: > >> I found some other errors when using the test environment. >> >> >> --------------------------------------------------------------------------- >> RuntimeError Traceback (most recent call >> last) >> <ipython-input-1-fe308b345b3f> in <module> >> 10 import scipy as sp >> 11 from scipy import linalg >> ---> 12 import iotbx.pdb >> 13 import cctbx >> 14 import pandas as pd >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in <module> >> 8 >> 9 import iotbx.pdb.records >> ---> 10 import iotbx.pdb.hierarchy >> 11 from scitbx import matrix >> 12 >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__hierarchy.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=QlonHINbJqyu1vTwlBJWKyChCMCCJxpIWIaT5_rrkLc&e= >> in <module> >> 14 from six.moves import cStringIO as StringIO >> 15 from iotbx.pdb import hy36encode, hy36decode >> ---> 16 import iotbx.cif.model >> 17 from cctbx import crystal >> 18 from libtbx import group_args >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/cif/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in <module> >> 19 >> 20 from cctbx.array_family import flex >> ---> 21 from cctbx import miller >> 22 from iotbx.cif import model, builders, geometry >> 23 from libtbx.containers import OrderedDict >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in <module> >> 11 >> 12 from cctbx import crystal >> ---> 13 from cctbx import maptbx >> 14 from cctbx import sgtbx >> 15 from cctbx.sgtbx import lattice_symmetry >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/cctbx/maptbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in <module> >> 15 from libtbx import adopt_init_args >> 16 from libtbx.utils import Sorry >> ---> 17 import libtbx.load_env >> 18 import math >> 19 import sys, os >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= >> in >> <module> >> 3 import libtbx.env_config >> 4 import os >> ----> 5 libtbx.env = libtbx.env_config.unpickle() >> 6 libtbx.env.set_os_environ_all_dist() >> 7 libtbx.env.dispatcher_name = >> os.environ.get("LIBTBX_DISPATCHER_NAME") >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= >> in unpickle() >> 2736 env = pickle.load(libtbx_env) >> 2737 if (env.python_version_major_minor != sys.version_info[:2]): >> -> 2738 env.raise_python_version_incompatible() >> 2739 if (op.realpath(build_path) != >> op.realpath(abs(env.build_path))): >> 2740 env.build_path.reset(build_path) >> >> ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= >> in raise_python_version_incompatible(self, prev_pvmm) >> 469 if (prev_pvmm is None): >> 470 prev_pvmm = "%d.%d" % self.python_version_major_minor >> --> 471 raise RuntimeError("Python version incompatible with this >> build:\n" >> 472 + " Build directory: %s\n" % >> show_string(abs(self.build_path)) >> 473 + " Python version used initially: %s\n" % prev_pvmm >> >> RuntimeError: Python version incompatible with this build: >> Build directory: "/home/gergely/anaconda3" >> Python version used initially: 3.6 >> Python version in use now: 3.8 >> >> >> Interestingly in the base environment all import worked, but I got an >> error at: >> >> >> ArgumentError Traceback (most recent call >> last) >> <ipython-input-4-5bc71c36e121> in <module> >> 33 atm_odd.set_b(0) >> 34 >> ---> 35 odd.write_pdb_file("bdiff_in_even_000.pdb") #skriver ny pdb >> med dessa inkluderade >> 36 dfWBvals=pd.DataFrame(rows_list) >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in >> write_pdb_file(self, file_name, open_append, crystal_symmetry, >> cryst1_z, write_scale_records, append_end, atom_hetatm, sigatm, >> anisou, siguij) >> 930 siguij=True): >> 931 if (crystal_symmetry is Auto): >> --> 932 crystal_symmetry = self.crystal_symmetry() >> 933 if (cryst1_z is Auto): >> 934 cryst1_z = self.extract_cryst1_z_columns() >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in >> crystal_symmetry(self, crystal_symmetry, weak_symmetry) >> 1218 crystal_symmetry=None, >> 1219 weak_symmetry=False): >> -> 1220 self_symmetry = self.crystal_symmetry_from_cryst1() >> 1221 if (self_symmetry is None): >> 1222 self_symmetry = >> self._crystal_symmetry_from_cns_remark_sg() >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in >> crystal_symmetry_from_cryst1(self) >> 1195 for line in self.crystallographic_section(): >> 1196 if (line.startswith("CRYST1")): >> -> 1197 return >> cryst1_interpretation.crystal_symmetry(cryst1_record=line) >> 1198 return None >> 1199 >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__cryst1-5Finterpretation.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=3OchLb18AMp3YpXOuEb8BDOwhl2XLrV1QAfZByH-l2E&e= >> in crystal_symmetry(cryst1_record) >> 136 space_group_info=None) >> 137 space_group_info = >> categorize(cryst1_record.sgroup).space_group_info( >> --> 138 unit_cell=u) >> 139 return crystal.symmetry(unit_cell=u, >> space_group_info=space_group_info) >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__cryst1-5Finterpretation.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=3OchLb18AMp3YpXOuEb8BDOwhl2XLrV1QAfZByH-l2E&e= >> in space_group_info(self, unit_cell) >> 85 if (self.symbol is None): return None >> 86 if (self.category is None): >> ---> 87 try: return sgtbx.space_group_info(self.symbol) >> 88 except RuntimeError: return None >> 89 if (isinstance(unit_cell, uctbx.ext.unit_cell)): >> >> ~/anaconda3/lib/python3.6/site-packages/cctbx/sgtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= >> in >> __init__(self, symbol, table_id, group, number, space_group_t_den) >> 100 assert group is None >> 101 if (table_id is None): >> --> 102 symbols = space_group_symbols(symbol) >> 103 else: >> 104 if (isinstance(symbol, int)): symbol = str(symbol) >> >> ArgumentError: Python argument types in >> space_group_symbols.__init__(space_group_symbols, str) >> did not match C++ signature: >> __init__(_object*, int space_group_number) >> __init__(_object*, int space_group_number, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > extension='') >> __init__(_object*, int space_group_number, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > extension='', std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > table_id='') >> __init__(_object*, std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > symbol) >> __init__(_object*, std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > symbol, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > table_id='') >> >> >> Gergely >> >> >> >> On Fri, Jan 3, 2020 at 1:44 PM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> > >> > Dear Billy, >> > >> > I did some tests installing in my base conda was not troublefree in >> > python 3.8. Running iotbx.fetch_pdb --all --mtz 1kp8 resulted in boost >> > related error when converting to mtz. Installing into a test >> > environment went fine with iotbx.fetch_pdb --all --mtz 1kp8 running >> > successfully. I could add pymc3, jupyter and seaborn and iotbx still >> > did not break. My LIBTBX_BUILD is not set however in the test >> > environment, which breaks miller when I was trying to import it. >> > Manually setting the LIBTBX_BUILD to CONDA_PREFIX (setenv LIBTBX_BUILD >> > $CONDA_PREFIX) fixes this and one of my script works fine! I will come >> > back if I find some problems with my other scripts, but so far this >> > looks promising! >> > >> > >> > Happy new year! >> > >> > Gergely >> > >> > >> > >> > >> > >> > On Fri, Dec 20, 2019 at 8:16 PM Billy Poon <BKPoon(a)lbl.gov> wrote: >> > > >> > > Hi everyone, >> > > >> > > I updated the conda packages on the cctbx-dev channel and there are >> also now packages for macOS. The old packages have been removed so you >> can >> create a new environment with >> > > >> > > conda create -n test -c cctbx-dev cctbx python=2.7 >> > > conda activate test >> > > >> > > where "test" is the environment name and 2.7 is the python version. >> For the python version, you can also pick 3.6, 3.7, or 3.8. I think >> the >> latest conda will install 3.8 as the default python if the python >> argument >> is not provided. Or install into an existing environment (with python >> already installed) with the same command as before, >> > > >> > > conda install -c cctbx-dev cctbx >> > > >> > > Again, your .condarc file should have conda-forge as the first channel >> so that dependencies can be pulled correctly. >> > > >> > > The dispatchers should work now and $LIBTBX_BUILD will be set on >> activation of an environment (and unset on environment deactivation) >> or >> after installing into an existing environment, so you can do more >> interesting things like >> > > >> > > iotbx.fetch_pdb --all --mtz 1kp8 >> > > >> > > However, this package does not install databases, so >> > > >> > > molprobity.molprobity 1kp8.pdb 1kp8.mtz >> > > >> > > will fail because the database is not available (also probe and reduce >> are not built). >> > > >> > > Some known issues in no particular order, >> > > >> > > 1) You can run "libtbx.run_tests_parallel module=cctbx nproc=Auto" to >> run tests, but not all the test files were copied, so I got 30 >> failures >> here. The final package will probably not have the test files. >> > > 2) The other python dispatchers, like "cctbx.python", were not >> updated, so those will fail. But you can just run "python" in the >> active >> environment. >> > > 3) Parts of mmtbx still need to be updated for Python 3. >> > > 4) The LIBTBX_BUILD variable should be set for sh and csh shells. Let >> me know if it's not working for your shell (i.e. "echo $LIBTBX_BUILD" >> should show the same thing as $CONDA_PREFIX). >> > > 5) One future step is to enable building other CCTBX-like modules with >> this conda package. This will require some bookkeeping updates so that >> modules in the conda environment and the modules being built are >> tracked >> properly. Also, making sure that all the necessary headers exist in >> $PREFIX/include. >> > > >> > > Gergely, let me know if your scripts are working. >> > > >> > > Happy holidays! >> > > >> > > -- >> > > Billy K. Poon >> > > Research Scientist, Molecular Biophysics and Integrated Bioimaging >> > > Lawrence Berkeley National Laboratory >> > > 1 Cyclotron Road, M/S 33R0345 >> > > Berkeley, CA 94720 >> > > Tel: (510) 486-5709 >> > > Fax: (510) 486-5909 >> > > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > > >> > > >> > > On Wed, Dec 18, 2019 at 11:25 AM Billy Poon <BKPoon(a)lbl.gov> wrote: >> > >> >> > >> Hi Gergely, >> > >> >> > >> That's the other information that hasn't been updated yet in the >> conda package. :) >> > >> >> > >> For LIBTBX_BUILD, I can set up environment variables sort of how the >> conda gcc package sets up environment variables. I have to double >> check to >> see if they're set after installation of the package and after >> activation >> of an environment. The variable will just be a copy of $CONDA_PREFIX. >> > >> >> > >> The libtbx_env file is a file that exists in $LIBTBX_BUILD that >> basically stores configuration information. I'll have to modify the >> contents to update the locations of the different modules since the >> conda >> package will not have a "modules" directory. >> > >> >> > >> -- >> > >> Billy K. Poon >> > >> Research Scientist, Molecular Biophysics and Integrated Bioimaging >> > >> Lawrence Berkeley National Laboratory >> > >> 1 Cyclotron Road, M/S 33R0345 >> > >> Berkeley, CA 94720 >> > >> Tel: (510) 486-5709 >> > >> Fax: (510) 486-5909 >> > >> Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >> >> > >> >> > >> On Wed, Dec 18, 2019 at 2:08 AM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> > >>> >> > >>> Hi Billy, >> > >>> >> > >>> Conda install went fine with your instructions after rearranging the >> > >>> channels by putting cctbx last. I removed all environmental variables >> > >>> and previous build of cctbx. >> > >>> >> > >>> import sys >> > >>> print (sys.path, sys.prefix) >> > >>> >> > >>> ['/home/gergely/anaconda3/lib/https://urldefense.proofpoint.com/v2/url?u=http-3A__python36.zip&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=cUHL8qgeloeUO3oYTQcr9d56RegJk01wJ2i-c3Zo6oE&e= ', >> > >>> '/home/gergely/anaconda3/lib/python3.6', >> > >>> '/home/gergely/anaconda3/lib/python3.6/lib-dynload', '', >> > >>> '/home/gergely/anaconda3/lib/python3.6/site-packages', >> > >>> >> '/home/gergely/anaconda3/lib/python3.6/site-packages/IPython/extensions', >> > >>> '/home/gergely/.ipython'] /home/gergely/anaconda3 >> > >>> >> > >>> Many imports went fine including pymc3, but I encountered problems >> > >>> with these three: >> > >>> >> > >>> from cctbx import miller >> > >>> import iotbx.pdb >> > >>> from iotbx import reflection_file_reader, mtz >> > >>> >> > >>> >> > >>> I also got type error when handling space groups. >> > >>> >> > >>> >> > >>> >> > >>> >> --------------------------------------------------------------------------- >> > >>> KeyError Traceback (most recent >> call last) >> > >>> <ipython-input-5-cedbd4f84f3d> in <module> >> > >>> 25 from cctbx import uctbx >> > >>> 26 from cctbx import sgtbx >> > >>> ---> 27 from cctbx import miller >> > >>> 28 #from iotbx import reflection_file_reader, mtz >> > >>> 29 sns.set_context("poster") >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in >> <module> >> > >>> 11 >> > >>> 12 from cctbx import crystal >> > >>> ---> 13 from cctbx import maptbx >> > >>> 14 from cctbx import sgtbx >> > >>> 15 from cctbx.sgtbx import lattice_symmetry >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/maptbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in >> <module> >> > >>> 15 from libtbx import adopt_init_args >> > >>> 16 from libtbx.utils import Sorry >> > >>> ---> 17 import libtbx.load_env >> > >>> 18 import math >> > >>> 19 import sys, os >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= in >> <module> >> > >>> 3 import libtbx.env_config >> > >>> 4 import os >> > >>> ----> 5 libtbx.env = libtbx.env_config.unpickle() >> > >>> 6 libtbx.env.set_os_environ_all_dist() >> > >>> 7 libtbx.env.dispatcher_name = >> os.environ.get("LIBTBX_DISPATCHER_NAME") >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= in >> unpickle() >> > >>> 2603 >> > >>> 2604 def unpickle(): >> > >>> -> 2605 build_path = os.environ["LIBTBX_BUILD"] >> > >>> 2606 >> set_preferred_sys_prefix_and_sys_executable(build_path=build_path) >> > >>> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") >> > >>> >> > >>> ~/anaconda3/lib/python3.6/https://urldefense.proofpoint.com/v2/url?u=http-3A__os.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=hA1zo5wap0c9wK3DrlWIKqYBIqqbfKXca5jQ9aX5pYU&e= in __getitem__(self, key) >> > >>> 667 except KeyError: >> > >>> 668 # raise KeyError with the original key value >> > >>> --> 669 raise KeyError(key) from None >> > >>> 670 return self.decodevalue(value) >> > >>> 671 >> > >>> >> > >>> KeyError: 'LIBTBX_BUILD' >> > >>> >> > >>> >> > >>> >> > >>> >> --------------------------------------------------------------------------- >> > >>> ArgumentError Traceback (most recent >> call last) >> > >>> <ipython-input-7-2acd3a9ce26a> in <module> >> > >>> 26 return >> ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar >> > >>> 27 >> > >>> ---> 28 >> ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar=initializecrystal() >> > >>> >> > >>> <ipython-input-7-2acd3a9ce26a> in initializecrystal() >> > >>> 11 uc = uctbx.unit_cell(unit_cell) >> > >>> 12 wavelength = 1.54980 >> > >>> ---> 13 xtal_symm = crystal.symmetry(unit_cell=unit_cell, >> > >>> space_group_symbol="P 43 21 2") >> > >>> 14 >> > >>> 15 ms = >> > >>> miller.build_set(crystal_symmetry=xtal_symm,anomalous_flag=True, >> > >>> d_min=1.61) >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/crystal/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in >> > >>> __init__(self, unit_cell, space_group_symbol, space_group_info, >> > >>> space_group, correct_rhombohedral_setting_if_necessary, >> > >>> assert_is_compatible_unit_cell, >> raise_sorry_if_incompatible_unit_cell, >> > >>> force_compatible_unit_cell) >> > >>> 74 if (space_group_symbol is not None): >> > >>> 75 self._space_group_info = sgtbx.space_group_info( >> > >>> ---> 76 symbol=space_group_symbol) >> > >>> 77 elif (space_group is not None): >> > >>> 78 if (isinstance(space_group, sgtbx.space_group)): >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/sgtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in >> > >>> __init__(self, symbol, table_id, group, number, space_group_t_den) >> > >>> 100 assert group is None >> > >>> 101 if (table_id is None): >> > >>> --> 102 symbols = space_group_symbols(symbol) >> > >>> 103 else: >> > >>> 104 if (isinstance(symbol, int)): symbol = str(symbol) >> > >>> >> > >>> ArgumentError: Python argument types in >> > >>> space_group_symbols.__init__(space_group_symbols, str) >> > >>> did not match C++ signature: >> > >>> __init__(_object*, int space_group_number) >> > >>> __init__(_object*, int space_group_number, >> > >>> std::__cxx11::basic_string<char, std::char_traits<char>, >> > >>> std::allocator<char> > extension='') >> > >>> __init__(_object*, int space_group_number, >> > >>> std::__cxx11::basic_string<char, std::char_traits<char>, >> > >>> std::allocator<char> > extension='', std::__cxx11::basic_string<char, >> > >>> std::char_traits<char>, std::allocator<char> > table_id='') >> > >>> __init__(_object*, std::__cxx11::basic_string<char, >> > >>> std::char_traits<char>, std::allocator<char> > symbol) >> > >>> __init__(_object*, std::__cxx11::basic_string<char, >> > >>> std::char_traits<char>, std::allocator<char> > symbol, >> > >>> std::__cxx11::basic_string<char, std::char_traits<char>, >> > >>> std::allocator<char> > table_id='') >> > >>> >> > >>> >> > >>> When I tried to set LIBTBX_BUILD to /home/gergely/anaconda3 (this is >> > >>> sys.prefix on my system), I got other problems when importing. Can >> > >>> LIBTBX_BUILD be set in the conda package? >> > >>> >> > >>> >> > >>> >> > >>> >> --------------------------------------------------------------------------- >> > >>> FileNotFoundError Traceback (most recent >> call last) >> > >>> <ipython-input-1-7e2b144826cc> in <module> >> > >>> 26 from cctbx import sgtbx >> > >>> 27 #from cctbx import miller >> > >>> ---> 28 from iotbx import reflection_file_reader, mtz >> > >>> 29 sns.set_context("poster") >> > >>> 30 plt.rcParams.update({'figure.autolayout': True}) >> > >>> >> > >>> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__reflection-5Ffile-5Freader.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=U5BvLmuCgufvN_LRxQw9n78p33jNaRrXVbH1BRSXgxg&e= >> > >>> in <module> >> > >>> 53 >> > >>> 54 from __future__ import absolute_import, division, >> print_function >> > >>> ---> 55 from iotbx import mtz >> > >>> 56 from iotbx.scalepack import merge as scalepack_merge >> > >>> 57 from iotbx.scalepack import no_merge_original_index as >> > >>> scalepack_no_merge >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in >> <module> >> > >>> 9 import iotbx_mtz_ext as ext >> > >>> 10 >> > >>> ---> 11 from iotbx.mtz import extract_from_symmetry_lib >> > >>> 12 from cctbx import xray >> > >>> 13 import cctbx.xray.observation_types >> > >>> >> > >>> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__extract-5Ffrom-5Fsymmetry-5Flib.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=k3WR6VZd4-PDULPXl4O0Vwz9q3LCaocUUhK_ehTn9UE&e= >> > >>> in <module> >> > >>> 1 from __future__ import absolute_import, division, >> print_function >> > >>> 2 from cctbx import sgtbx >> > >>> ----> 3 import libtbx.load_env >> > >>> 4 import os.path as op >> > >>> 5 from six.moves import range >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= in >> <module> >> > >>> 3 import libtbx.env_config >> > >>> 4 import os >> > >>> ----> 5 libtbx.env = libtbx.env_config.unpickle() >> > >>> 6 libtbx.env.set_os_environ_all_dist() >> > >>> 7 libtbx.env.dispatcher_name = >> os.environ.get("LIBTBX_DISPATCHER_NAME") >> > >>> >> > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= in >> unpickle() >> > >>> 2605 build_path = os.environ["LIBTBX_BUILD"] >> > >>> 2606 >> set_preferred_sys_prefix_and_sys_executable(build_path=build_path) >> > >>> -> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") >> > >>> 2608 env = pickle.load(libtbx_env) >> > >>> 2609 if (env.python_version_major_minor != >> sys.version_info[:2]): >> > >>> >> > >>> FileNotFoundError: [Errno 2] No such file or directory: >> > >>> '/home/gergely/anaconda3/libtbx_env' >> > >>> >> > >>> On Wed, Dec 18, 2019 at 9:10 AM Billy Poon <BKPoon(a)lbl.gov> wrote: >> > >>> > >> > >>> > Hi Gergely, >> > >>> > >> > >>> > I've uploaded linux packages to a new channel, cctbx-dev, and you >> can install it with >> > >>> > >> > >>> > conda install -c cctbx-dev cctbx >> > >>> > >> > >>> > in your current environment. You should set your ~/.condarc file >> to pull the other dependencies from the conda-forge channel first, so >> put >> conda-forge above cctbx. Mine looks like >> > >>> > >> > >>> > channels: >> > >>> > - conda-forge >> > >>> > - defaults >> > >>> > - cctbx >> > >>> > >> > >>> > Do you need dxtbx for your scripts? This package does not build >> that part. I think the plan is to build a separate conda package for >> dxtbx >> so that it can be updated more frequently. I can rebuild the packages >> to >> include it for testing, but the one being submitted to conda-forge >> will not >> have it. >> > >>> > >> > >>> > Also, your error message is probably due to version of HDF5 that >> the development build installs. The >> https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… >> script will install >> 1.10.4, but your other dependency is looking for 1.10.5. Installing >> this >> cctbx conda package should install 1.10.5, which should fix the issue. >> Also, I'm updating those environments and 1.10.5 will be the new >> default >> version. >> > >>> > >> > >>> > Lastly, the dispatchers will not work in these packages because >> the old paths during the build process are still in them and many of >> them >> expect some additional information that has not been updated in the >> packages yet. I'm in the process of doing that and will update the >> cctbx-dev channel when that's done. However, by starting python, you >> can >> import cctbx modules. So you can run commands like >> > >>> > >> > >>> > from scitbx.array_family import flex >> > >>> > a = flex.random_double(1000000) >> > >>> > b = flex.min_max_mean_double(a) >> > >>> > b.min >> > >>> > b.max >> > >>> > b.mean >> > >>> > >> > >>> > Thanks! >> > >>> > >> > >>> > -- >> > >>> > Billy K. Poon >> > >>> > Research Scientist, Molecular Biophysics and Integrated Bioimaging >> > >>> > Lawrence Berkeley National Laboratory >> > >>> > 1 Cyclotron Road, M/S 33R0345 >> > >>> > Berkeley, CA 94720 >> > >>> > Tel: (510) 486-5709 >> > >>> > Fax: (510) 486-5909 >> > >>> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >>> > >> > >>> > >> > >>> > On Tue, Dec 17, 2019 at 5:39 AM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> > >>> >> >> > >>> >> Dear Billy, >> > >>> >> >> > >>> >> Thank you for the detailed explanation, I look forward to do >> further >> > >>> >> testing! I aim to pool all modules under the same environment, if >> > >>> >> there are conflicts then I just try to reshuffle the order of >> imports. >> > >>> >> So far this did not cause problems for me even when I was using >> system >> > >>> >> python, but of course it is not the most prudent thing to do. With >> > >>> >> anaconda everything is much more standardized and isolated >> already and >> > >>> >> without being superuser I can have my familiar environment at any >> > >>> >> synchrotron based cluster. It is great that cctbx will be an >> integral >> > >>> >> part of this ecosystem and this was also the last thing holding me >> > >>> >> back from adopting python3. About hdf5, pymc3 requires it and >> > >>> >> importing causes a kernel restart with the following error >> messages, >> > >>> >> curiously if I import h5py first this can be avoided. >> > >>> >> >> > >>> >> Best wishes, >> > >>> >> >> > >>> >> Gergely >> > >>> >> >> > >>> >> import pymc3 as pm >> > >>> >> >> > >>> >> Warning! ***HDF5 library version mismatched error*** >> > >>> >> The HDF5 header files used to compile this application do not >> match >> > >>> >> the version used by the HDF5 library to which this application is >> linked. >> > >>> >> Data corruption or segmentation faults may occur if the >> application continues. >> > >>> >> This can happen when an application was compiled by one version >> of HDF5 but >> > >>> >> linked with a different version of static or shared HDF5 library. >> > >>> >> You should recompile the application or check your shared library >> related >> > >>> >> settings such as 'LD_LIBRARY_PATH'. >> > >>> >> You can, at your own risk, disable this warning by setting the >> environment >> > >>> >> variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'. >> > >>> >> Setting it to 2 or higher will suppress the warning messages >> totally. >> > >>> >> Headers are 1.10.4, library is 1.10.5 >> > >>> >> SUMMARY OF THE HDF5 CONFIGURATION >> > >>> >> ================================= >> > >>> >> >> > >>> >> General Information: >> > >>> >> ------------------- >> > >>> >> HDF5 Version: 1.10.5 >> > >>> >> Configured on: Tue Oct 22 12:02:13 UTC 2019 >> > >>> >> Configured by: conda@16247e67ecd5 >> > >>> >> Host system: x86_64-conda_cos6-linux-gnu >> > >>> >> Uname information: Linux 16247e67ecd5 >> 4.15.0-1059-azure >> > >>> >> #64-Ubuntu SMP Fri Sep 13 17:02:44 UTC 2019 x86_64 x86_64 x86_64 >> > >>> >> GNU/Linux >> > >>> >> Byte sex: little-endian >> > >>> >> Installation point: /home/gergely/anaconda3 >> > >>> >> >> > >>> >> Compiling Options: >> > >>> >> ------------------ >> > >>> >> Build Mode: production >> > >>> >> Debugging Symbols: no >> > >>> >> Asserts: no >> > >>> >> Profiling: no >> > >>> >> Optimization Level: high >> > >>> >> >> > >>> >> Linking Options: >> > >>> >> ---------------- >> > >>> >> Libraries: static, shared >> > >>> >> Statically Linked Executables: >> > >>> >> LDFLAGS: -Wl,-O2 -Wl,--sort-common >> > >>> >> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags >> > >>> >> -Wl,--gc-sections -Wl,-rpath,/home/gergely/anaconda3/lib >> > >>> >> -Wl,-rpath-link,/home/gergely/anaconda3/lib >> > >>> >> -L/home/gergely/anaconda3/lib >> > >>> >> H5_LDFLAGS: >> > >>> >> AM_LDFLAGS: -L/home/gergely/anaconda3/lib >> > >>> >> Extra libraries: -lrt -lpthread -lz -ldl -lm >> > >>> >> Archiver: >> > >>> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar >> > >>> >> AR_FLAGS: cr >> > >>> >> Ranlib: >> > >>> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib >> > >>> >> >> > >>> >> Languages: >> > >>> >> ---------- >> > >>> >> C: yes >> > >>> >> C Compiler: >> > >>> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc >> > >>> >> CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 >> > >>> >> -I/home/gergely/anaconda3/include >> > >>> >> H5_CPPFLAGS: -D_GNU_SOURCE >> > >>> >> -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API >> > >>> >> AM_CPPFLAGS: >> -I/home/gergely/anaconda3/include >> > >>> >> C Flags: -march=nocona -mtune=haswell >> > >>> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 >> > >>> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include >> > >>> >> >> -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 >> > >>> >> >> -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix >> > >>> >> H5 C Flags: -std=c99 -pedantic -Wall >> -Wextra >> > >>> >> -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual >> -Wconversion >> > >>> >> -Wdeclaration-after-statement -Wdisabled-optimization >> -Wfloat-equal >> > >>> >> -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-declarations >> > >>> >> -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs >> > >>> >> -Wold-style-definition -Wpacked -Wpointer-arith -Wredundant-decls >> > >>> >> -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum >> -Wundef >> > >>> >> -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings >> > >>> >> -finline-functions -s -Wno-inline -Wno-aggregate-return >> > >>> >> -Wno-missing-format-attribute -Wno-missing-noreturn -O >> > >>> >> AM C Flags: >> > >>> >> Shared C Library: yes >> > >>> >> Static C Library: yes >> > >>> >> >> > >>> >> >> > >>> >> Fortran: yes >> > >>> >> Fortran Compiler: >> > >>> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-gfortran >> > >>> >> Fortran Flags: >> > >>> >> H5 Fortran Flags: -pedantic -Wall -Wextra >> -Wunderflow >> > >>> >> -Wimplicit-interface -Wsurprising -Wno-c-binding-type -s -O2 >> > >>> >> AM Fortran Flags: >> > >>> >> Shared Fortran Library: yes >> > >>> >> Static Fortran Library: yes >> > >>> >> >> > >>> >> C++: yes >> > >>> >> C++ Compiler: >> > >>> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++ >> > >>> >> C++ Flags: -fvisibility-inlines-hidden >> > >>> >> -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell >> > >>> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 >> > >>> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include >> > >>> >> >> -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 >> > >>> >> >> -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix >> > >>> >> H5 C++ Flags: -pedantic -Wall -W -Wundef >> -Wshadow >> > >>> >> -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings >> -Wconversion >> > >>> >> -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual >> > >>> >> -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor >> > >>> >> -Wctor-dtor-privacy -Wabi -finline-functions -s -O >> > >>> >> AM C++ Flags: >> > >>> >> Shared C++ Library: yes >> > >>> >> Static C++ Library: yes >> > >>> >> >> > >>> >> Java: no >> > >>> >> >> > >>> >> >> > >>> >> Features: >> > >>> >> --------- >> > >>> >> Parallel HDF5: no >> > >>> >> Parallel Filtered Dataset Writes: no >> > >>> >> Large Parallel I/O: no >> > >>> >> High-level library: yes >> > >>> >> Threadsafety: yes >> > >>> >> Default API mapping: v110 >> > >>> >> With deprecated public symbols: yes >> > >>> >> I/O filters (external): deflate(zlib) >> > >>> >> MPE: no >> > >>> >> Direct VFD: no >> > >>> >> dmalloc: no >> > >>> >> Packages w/ extra debug output: none >> > >>> >> API tracing: no >> > >>> >> Using memory checker: yes >> > >>> >> Memory allocation sanity checks: no >> > >>> >> Function stack tracing: no >> > >>> >> Strict file format checks: no >> > >>> >> Optimization instrumentation: no >> > >>> >> >> > >>> >> On Tue, Dec 17, 2019 at 8:35 AM Billy Poon <BKPoon(a)lbl.gov> >> wrote: >> > >>> >> > >> > >>> >> > Hi Gergely, >> > >>> >> > >> > >>> >> > Let me build the test package tomorrow. All the gritty details >> for building with conda is being finalized and the official >> documentation >> will be updated to describe the steps. It would be too confusing to >> keep >> changing the documentation as the process evolves. You do have the >> general >> process, though, which is summarized as follows. >> > >>> >> > >> > >>> >> > 1) Installing dependencies. The cctbx_dependencies metapackage >> was an initial approach for managing the CCTBX dependencies, but after >> contacting the conda-forge folks, they recommended using the >> --only-deps >> flag. So when the CCTBX conda package is available, you'll be able to >> get a >> set of dependencies with, >> > >>> >> > >> > >>> >> > conda install -c conda-forge --only-deps cctbx >> > >>> >> > >> > >>> >> > By default, the https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file will automatically install a >> set of dependencies in the "conda_base" directory (and a conda >> installation >> if one is not found). It just uses standard conda environment files >> located >> in libtbx/auto_build/conda_envs, so you do not need to install >> cctbx_dependencies as a separate step. The environment files avoid >> channel >> issues by explicitly defining the channel to pull the packages from >> and the >> cctbx channel just stores copies of packages from conda-forge. There >> were >> issues earlier where the conda-forge packages would sometimes be moved >> to a >> different label. The --use-conda flag also accepts a path to >> $CONDA_PREFIX >> if you want to use a specific environment for testing. >> > >>> >> > >> > >>> >> > 2) Building. The https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file handles that with SCons. >> > >>> >> > >> > >>> >> > 3) Running. After building, there should be a https://urldefense.proofpoint.com/v2/url?u=http-3A__setpaths.sh&d=DwICAg&c=… (and >> .csh) file that adds build/bin to your path. The build/bin directory >> has >> our dispatchers, which are just scripts that set up the environment >> variables for you. This prevents other programs from loading our >> libraries, >> whose versions may conflict. You should see that there is a "python" >> dispatcher, which is a convenience for developers. Otherwise, you can >> use >> libtbx.python, which will be able to import CCTBX modules. >> > >>> >> > >> > >>> >> > The running part is where the conda package for CCTBX will be >> different than this build. Since our Python files and extensions >> modules >> will be in the "site-packages" directory for Python, the PYTHONPATH >> variable will not be needed (and conda suggests that that variable not >> be >> set). The other CCTBX libraries will be in $CONDA_PREFIX/lib, so >> LD_LIBRARY_PATH is not needed. In an active environment, PATH will >> already >> be modified. And then our LIBTBX_BUILD directory can be set to >> sys.prefix. >> > >>> >> > >> > >>> >> > So with the conda package, you would only need to activate your >> conda environment and CCTBX should integrate with other conda >> packages. >> What is the conflict with hdf5? That's something that should be fixed. >> Thanks! >> > >>> >> > >> > >>> >> > -- >> > >>> >> > Billy K. Poon >> > >>> >> > Research Scientist, Molecular Biophysics and Integrated >> Bioimaging >> > >>> >> > Lawrence Berkeley National Laboratory >> > >>> >> > 1 Cyclotron Road, M/S 33R0345 >> > >>> >> > Berkeley, CA 94720 >> > >>> >> > Tel: (510) 486-5709 >> > >>> >> > Fax: (510) 486-5909 >> > >>> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >>> >> > >> > >>> >> > >> > >>> >> > On Mon, Dec 16, 2019 at 2:53 AM Gergely Katona < >> gkatona(a)gmail.com> wrote: >> > >>> >> >> >> > >>> >> >> Dear Billy, >> > >>> >> >> >> > >>> >> >> Thank you for this update and for your efforts! I found a >> solution and >> > >>> >> >> indeed most things already work in anaconda3. The steps I took >> (even >> > >>> >> >> if these instructions will have short expiry date): >> > >>> >> >> >> > >>> >> >> Modified .condarc with: >> > >>> >> >> channels: >> > >>> >> >> - cctbx >> > >>> >> >> - conda-forge >> > >>> >> >> - defaults >> > >>> >> >> >> > >>> >> >> run >> > >>> >> >> conda conda install cctbx_dependencies python=3.6 >> > >>> >> >> >> > >>> >> >> I expect this will work without python=3.6 in the near future. >> > >>> >> >> >> > >>> >> >> Then compiling cctbx with anaconda3 python went without any >> problem >> > >>> >> >> when using these flags: >> > >>> >> >> python https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… --use-conda --python3 --nproc=12 >> > >>> >> >> >> > >>> >> >> Then I probably did the unorthodox thing and sourced these >> directories: >> > >>> >> >> >> > >>> >> >> setenv LIBTBX_BUILD /home/gergely/cctbx/build >> > >>> >> >> setenv PATH ${PATH}:/home/gergely/cctbx/build/bin >> > >>> >> >> setenv PYTHONPATH >> > >>> >> >> >> /home/gergely/cctbx/modules/cctbx_project:/home/gergely/cctbx/modules:/home/gergely/cctbx/modules/cctbx_project/boost_adaptbx:/home/gergely/cctbx/build/lib:/home/gergely/cctbx/conda_base/lib/python3.6/site-packages:$PYTHONPATH >> > >>> >> >> setenv LD_LIBRARY_PATH >> > >>> >> >> >> /home/gergely/cctbx/conda_base/lib:/home/gergely/cctbx/build/lib:$LD_LIBRARY_PATH >> > >>> >> >> >> > >>> >> >> There is probably a better way to put this into conda >> environment. >> > >>> >> >> With these steps I could run one of my scripts depending on >> cctbx. The >> > >>> >> >> only problem I found is that hdf5 library had conflict with >> another >> > >>> >> >> package in conda, but I does not find this as a showstopper. >> > >>> >> >> >> > >>> >> >> Best wishes, >> > >>> >> >> >> > >>> >> >> Gergely >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> On Fri, Dec 13, 2019 at 7:23 PM Billy Poon <BKPoon(a)lbl.gov> >> wrote: >> > >>> >> >> > >> > >>> >> >> > Hi Gergely, >> > >>> >> >> > >> > >>> >> >> > It's still a work in progress. I'm sorting out some Windows >> issues right now. I can probably build a test package on a separate >> channel >> for people that want to test it (let's say next week?). I'll provide >> instructions, but basically, the test conda package will be in its own >> separate channel and the dependencies will be pulled from the >> conda-forge >> channel. I want most things to be working correctly on Python 2.7, >> 3.6, >> 3.7, and 3.8 on all 3 platforms. >> > >>> >> >> > >> > >>> >> >> > Thanks! >> > >>> >> >> > >> > >>> >> >> > -- >> > >>> >> >> > Billy K. Poon >> > >>> >> >> > Research Scientist, Molecular Biophysics and Integrated >> Bioimaging >> > >>> >> >> > Lawrence Berkeley National Laboratory >> > >>> >> >> > 1 Cyclotron Road, M/S 33R0345 >> > >>> >> >> > Berkeley, CA 94720 >> > >>> >> >> > Tel: (510) 486-5709 >> > >>> >> >> > Fax: (510) 486-5909 >> > >>> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > On Fri, Dec 13, 2019 at 5:44 AM Gergely Katona < >> gkatona(a)gmail.com> wrote: >> > >>> >> >> >> >> > >>> >> >> >> Dear Billy, >> > >>> >> >> >> >> > >>> >> >> >> This sounds very promising and exciting. I am not sure if >> cctbx is >> > >>> >> >> >> already functional as a conda package in anaconda3 (Linux) >> or this is >> > >>> >> >> >> still work in progress. My technical expertise does not >> allow me to >> > >>> >> >> >> tell the difference. What I tried: >> > >>> >> >> >> >> > >>> >> >> >> Fresh install of anaconda3. Adding - cctbx and - >> conda-forge to >> > >>> >> >> >> .condarc . Installing pyside2 with conda. Running conda >> install >> > >>> >> >> >> conda_dependencies . I get a lot package version conflict, >> and I >> > >>> >> >> >> cannot import cctbx or iotbx to anaconda python. Am I >> following the >> > >>> >> >> >> right instructions? Or it is too early to expect that cctbx >> works when >> > >>> >> >> >> installed through conda? >> > >>> >> >> >> >> > >>> >> >> >> Best wishes, >> > >>> >> >> >> >> > >>> >> >> >> Gergely >> > >>> >> >> >> >> > >>> >> >> >> On Wed, Nov 27, 2019 at 3:56 PM Billy Poon <BKPoon(a)lbl.gov> >> wrote: >> > >>> >> >> >> > >> > >>> >> >> >> > Hi all, >> > >>> >> >> >> > >> > >>> >> >> >> > For a brief update, I have submitted a recipe for >> cctbxlite to conda-forge ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_conda-2Dfor… >> ) and support for >> Python 3.7 and 3.8 is being added ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… >> ). With some fixes for >> Windows >> (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… >> ), all platforms >> (macOS, linux, and Windows) can build for Python 2.7, 3.6, 3.7, and >> 3.8. >> Some additional changes will be needed to get Windows to work with >> Python 3 >> and for tests to pass with Boost 1.70.0. That will enable the >> conda-forge >> recipe to build for all platforms and for all supported versions of >> Python. >> > >>> >> >> >> > >> > >>> >> >> >> > Currently, the conda-forge recipe will install into the >> conda python and cctbx imports can be done without sourcing the >> environment >> scripts. It looks like a lot of the environment variables being set in >> the >> dispatchers can be removed since the Python files and C++ extensions >> are in >> the right places. I'll update the libtbx_env file so that commands >> that >> load the environment can work correctly. >> > >>> >> >> >> > >> > >>> >> >> >> > -- >> > >>> >> >> >> > Billy K. Poon >> > >>> >> >> >> > Research Scientist, Molecular Biophysics and Integrated >> Bioimaging >> > >>> >> >> >> > Lawrence Berkeley National Laboratory >> > >>> >> >> >> > 1 Cyclotron Road, M/S 33R0345 >> > >>> >> >> >> > Berkeley, CA 94720 >> > >>> >> >> >> > Tel: (510) 486-5709 >> > >>> >> >> >> > Fax: (510) 486-5909 >> > >>> >> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >>> >> >> >> > >> > >>> >> >> >> > >> > >>> >> >> >> > On Sun, Aug 25, 2019 at 2:33 PM Tristan Croll < >> tic20(a)cam.ac.uk> wrote: >> > >>> >> >> >> >> >> > >>> >> >> >> >> Hi Luc, >> > >>> >> >> >> >> >> > >>> >> >> >> >> That sounds promising. From there, I?d need to work out >> how to make a fully-packaged installer (basically a modified wheel >> file) >> for the ChimeraX ToolShed - the aim is for the end user to not have to >> worry about any of this. That adds a couple of complications - e.g. >> $LIBTBX_BUILD would need to be set dynamically before first import - >> but >> doesn?t seem insurmountable. >> > >>> >> >> >> >> >> > >>> >> >> >> >> Thanks, >> > >>> >> >> >> >> >> > >>> >> >> >> >> Tristan >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> Tristan Croll >> > >>> >> >> >> >> Research Fellow >> > >>> >> >> >> >> Cambridge Institute for Medical Research >> > >>> >> >> >> >> University of Cambridge CB2 0XY >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> > On 25 Aug 2019, at 18:31, Luc Bourhis < >> luc_j_bourhis(a)mac.com> wrote: >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > Hi Tristan, >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > cctbx could be built to use your ChimeraX python, now >> that cctbx is moving to Python 3. The option ?with-python is there for >> that >> with the bootstrap script. The specific environment setup boil down to >> setting two environment variable LIBTBX_BUILD and either >> LD_LIBRARY_PATH on >> Linux, PATH on Win32, or DYLIB_LIBRARY_PATH on MacOS. If you work >> within a >> framework such as ChimeraX, that should not be difficult to ensure >> those >> two variables are set. >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > Best wishes, >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > Luc >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > >> > >>> >> >> >> >> >> On 23 Aug 2019, at 19:02, Tristan Croll < >> tic20(a)cam.ac.uk> wrote: >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> To add my two cents on this: probably the second-most >> common question I've had about ISOLDE's implementation is, "why didn't >> you >> use CCTBX?". The honest answer to that is, "I didn't know how." >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> Still don't, really - although the current >> developments are rather promising. The problem I've faced is that >> CCTBX was >> designed as its own self-contained Python (2.7, until very recently) >> environment, with its own interpreter and a lot of very specific >> environment setup. Meanwhile I'm developing ISOLDE in ChimeraX, which >> is >> *also* its own self-contained Python (3.7) environment. To plug one >> into >> the other in that form... well, I don't think I'm a good enough >> programmer >> to really know where to start. >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> The move to Conda and a more modular CCTBX >> architecture should make a lot more possible in that direction. Pip >> would >> be even better for me personally (ChimeraX can install directly from >> the >> PyPI, but doesn't interact with Conda) - but I understand pretty well >> the >> substantial challenge that would amount to (not least being that the >> PyPI >> imposes a limit - around 100MB from memory? - on the size of an >> individual >> package). >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> Best regards, >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> Tristan >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >>> On 2019-08-23 09:28, Luc Bourhis wrote: >> > >>> >> >> >> >> >>> Hi Graeme, >> > >>> >> >> >> >> >>> Yes, I know. But ?black" is a program doing a very >> particular task >> > >>> >> >> >> >> >>> (code formatting from the top of my head). Requiring >> to use a wrapper >> > >>> >> >> >> >> >>> for python itself is another level. But ok, I think >> I am mellowing to >> > >>> >> >> >> >> >>> the idea after all! Talking with people around me, >> and extrapolating, >> > >>> >> >> >> >> >>> I would bet that, right now, a great majority of >> people interested by >> > >>> >> >> >> >> >>> cctbx in pip have already used the cctbx, so they >> know about the >> > >>> >> >> >> >> >>> Python wrapper, and they would not be too sanguine >> about that. My >> > >>> >> >> >> >> >>> concern is for the future, when pip will be the >> first time some people >> > >>> >> >> >> >> >>> use cctbx. Big fat warning notices on PyPI page and >> a better error >> > >>> >> >> >> >> >>> message when cctbx fails because LIBTBX_BUILD is not >> set would be >> > >>> >> >> >> >> >>> needed but that could be all right. >> > >>> >> >> >> >> >>> If we do a pip installer, we should aim at a minimal >> install: cctbx, >> > >>> >> >> >> >> >>> iotbx and their dependencies, and that?s it. >> > >>> >> >> >> >> >>> Best wishes, >> > >>> >> >> >> >> >>> Luc >> > >>> >> >> >> >> >>>> On 23 Aug 2019, at 07:17, >> Graeme.Winter(a)Diamond.ac.uk <Graeme.Winter(a)diamond.ac.uk> wrote: >> > >>> >> >> >> >> >>>> Without discussing the merits of this or whether we >> _choose_ to make the move to supporting PIP, I am certain it would be >> _possible_ - many other packages make dispatcher scripts when you pip >> install them e.g. >> > >>> >> >> >> >> >>>> Silver-Surfer rescale_f2 :) $ which black; cat >> $(which black) >> > >>> >> >> >> >> >>>> >> /Library/Frameworks/Python.framework/Versions/3.6/bin/black >> > >>> >> >> >> >> >>>> >> #!/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 >> > >>> >> >> >> >> >>>> # -*- coding: utf-8 -*- >> > >>> >> >> >> >> >>>> import re >> > >>> >> >> >> >> >>>> import sys >> > >>> >> >> >> >> >>>> from black import main >> > >>> >> >> >> >> >>>> if __name__ == '__main__': >> > >>> >> >> >> >> >>>> sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', >> '', sys.argv[0]) >> > >>> >> >> >> >> >>>> sys.exit(main()) >> > >>> >> >> >> >> >>>> So we _could_ work around the absence of >> LIBTBX_BUILD etc. in the system. Whether or not we elect to do the >> work is >> a different question, and it seems clear that here are very mixed >> opinions >> on this. >> > >>> >> >> >> >> >>>> Best wishes Graeme >> > >>> >> >> >> >> >>>> On 23 Aug 2019, at 01:21, Luc Bourhis < >> luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: >> > >>> >> >> >> >> >>>> Hi, >> > >>> >> >> >> >> >>>> Even if we managed to ship our the boost dynamic >> libraries with pip, it would still not be pip-like, as we would still >> need >> our python wrappers to set LIBTBX_BUILD and LD_LIBRARY_PATH. Normal >> pip >> packages work with the standard python exe. LD_LIBRARY_PATH, we could >> get >> around that by changing the way we compile, using -Wl,-R, which is the >> runtime equivalent of build time -L. That?s a significant change that >> would >> need to be tested. But there is no way around setting LIBTBX_BUILD >> right >> now. Leaving that to the user is horrible. Perhaps there is a way to >> hack >> libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&… >> so that we can hardwire LIBTBX_BUILD in there when pip >> installs? >> > >>> >> >> >> >> >>>> Best wishes, >> > >>> >> >> >> >> >>>> Luc >> > >>> >> >> >> >> >>>> On 16 Aug 2019, at 22:47, Luc Bourhis < >> luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: >> > >>> >> >> >> >> >>>> Hi, >> > >>> >> >> >> >> >>>> I did look into that many years ago, and even toyed >> with building a pip installer. What stopped me is the exact conclusion >> you >> reached too: the user would not have the pip experience he expects. >> You are >> right that it is a lot of effort but is it worth it? Considering that >> remark, I don?t think so. Now, Conda was created specifically to go >> beyond >> pip pure-python-only support. Since cctbx has garnered support for >> Conda, >> the best avenue imho is to go the extra length to have a package on >> https://urldefense.proofpoint.com/v2/url?u=http-3A__Anaconda.org&d=DwICAg&c… >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__anaconda.org_&d=DwICAg&… >> >, and then to advertise it hard to >> every potential user out there. >> > >>> >> >> >> >> >>>> Best wishes, >> > >>> >> >> >> >> >>>> Luc >> > >>> >> >> >> >> >>>> On 16 Aug 2019, at 21:45, Aaron Brewster < >> asbrewster(a)lbl.gov<mailto:[email protected]>> wrote: >> > >>> >> >> >> >> >>>> Hi, to avoid clouding Dorothee's documentation >> email thread, which I think is a highly useful enterprise, here's some >> thoughts about putting cctbx into pip. Pip doesn't install non-python >> dependencies well. I don't think boost is available as a package on >> pip >> (at least the package version we use). wxPython4 isn't portable >> through >> pip ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.wxpython.org_How-… >> ). >> MPI libraries are system dependent. If cctbx were a pure python >> package, >> pip would be fine, but cctbx is not. >> > >>> >> >> >> >> >>>> All that said, we could build a manylinux1 version >> of cctbx and upload it to PyPi (I'm just learning about this). For a >> pip >> package to be portable (which is a requirement for cctbx), it needs to >> conform to PEP513, the manylinux1 standard ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… >> ). For example, numpy is built >> according to this standard (see >> https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.org_project_numpy… >> , >> where you'll see the manylinux1 wheel). Note, the manylinux1 standard >> is >> built with Centos 5.11 which we no longer support. >> > >>> >> >> >> >> >>>> There is also a manylinux2010 standard, which is >> based on Centos 6 >> (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… >> ). This is >> likely a more attainable target (note though by default C++11 is not >> supported on Centos 6). >> > >>> >> >> >> >> >>>> If we built a manylinuxX version of cctbx and >> uploaded it to PyPi, the user would need all the non-python >> dependencies. >> There's no way to specify these in pip. For example, cctbx requires >> boost >> 1.63 or better. The user will need to have it in a place their python >> can >> find it, or we could package it ourselves and supply it, similar to >> how the >> pip h5py package now comes with an hd5f library, or how the pip numpy >> package includes an openblas library. We'd have to do the same for >> any >> packages we depend on that aren't on pip using the manylinux >> standards, >> such as wxPython4. >> > >>> >> >> >> >> >>>> Further, we need to think about how dials and other >> cctbx-based packages interact. If pip install cctbx is set up, how >> does >> pip install dials work, such that any dials shared libraries can find >> the >> cctbx libraries? Can shared libraries from one pip package link >> against >> libraries in another pip package? Would each package need to supply >> its >> own boost? Possibly this is well understood in the pip field, but not >> by >> me :) >> > >>> >> >> >> >> >>>> Finally, there's the option of providing a source >> pip package. This would require the full compiler toolchain for any >> given >> platform (macOS, linux, windows). These are likely available for >> developers, but not for general users. >> > >>> >> >> >> >> >>>> Anyway, these are some of the obstacles. Not >> saying it isn't possible, it's just a lot of effort. >> > >>> >> >> >> >> >>>> Thanks, >> > >>> >> >> >> >> >>>> -Aaron >> > >>> >> >> >> >> >>>> _______________________________________________ >> > >>> >> >> >> >> >>>> cctbxbb mailing list >> > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >>>> _______________________________________________ >> > >>> >> >> >> >> >>>> cctbxbb mailing list >> > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >>>> _______________________________________________ >> > >>> >> >> >> >> >>>> cctbxbb mailing list >> > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >>>> -- >> > >>> >> >> >> >> >>>> This e-mail and any attachments may contain >> confidential, copyright and or privileged material, and are for the >> use of >> the intended addressee only. If you are not the intended addressee or >> an >> authorised recipient of the addressee please notify us of receipt by >> returning the e-mail and do not use, copy, retain, distribute or >> disclose >> the information in or attached to the e-mail. >> > >>> >> >> >> >> >>>> Any opinions expressed within this e-mail are those >> of the individual and not necessarily of Diamond Light Source Ltd. >> > >>> >> >> >> >> >>>> Diamond Light Source Ltd. cannot guarantee that >> this e-mail or any attachments are free from viruses and we cannot >> accept >> liability for any damage which you may sustain as a result of software >> viruses which may be transmitted in or with the message. >> > >>> >> >> >> >> >>>> Diamond Light Source Limited (company no. 4375679). >> Registered in England and Wales with its registered office at Diamond >> House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, >> OX11 >> 0DE, United Kingdom >> > >>> >> >> >> >> >>>> _______________________________________________ >> > >>> >> >> >> >> >>>> cctbxbb mailing list >> > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org >> > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >>> _______________________________________________ >> > >>> >> >> >> >> >>> cctbxbb mailing list >> > >>> >> >> >> >> >>> cctbxbb(a)phenix-online.org >> > >>> >> >> >> >> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> >> > >>> >> >> >> >> >> _______________________________________________ >> > >>> >> >> >> >> >> cctbxbb mailing list >> > >>> >> >> >> >> >> cctbxbb(a)phenix-online.org >> > >>> >> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > >> > >>> >> >> >> >> > _______________________________________________ >> > >>> >> >> >> >> > cctbxbb mailing list >> > >>> >> >> >> >> > cctbxbb(a)phenix-online.org >> > >>> >> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> >> > >>> >> >> >> >> >> > >>> >> >> >> >> _______________________________________________ >> > >>> >> >> >> >> cctbxbb mailing list >> > >>> >> >> >> >> cctbxbb(a)phenix-online.org >> > >>> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> > >> > >>> >> >> >> > _______________________________________________ >> > >>> >> >> >> > cctbxbb mailing list >> > >>> >> >> >> > cctbxbb(a)phenix-online.org >> > >>> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> >> > >>> >> >> >> >> > >>> >> >> >> >> > >>> >> >> >> -- >> > >>> >> >> >> Gergely Katona, PhD >> > >>> >> >> >> Associate Professor >> > >>> >> >> >> Department of Chemistry and Molecular Biology, University >> of Gothenburg >> > >>> >> >> >> Box 462, 40530 G?teborg, Sweden >> > >>> >> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: >> +46-31-786-3910 >> > >>> >> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> > >>> >> >> >> >> > >>> >> >> >> _______________________________________________ >> > >>> >> >> >> cctbxbb mailing list >> > >>> >> >> >> cctbxbb(a)phenix-online.org >> > >>> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> > >> > >>> >> >> > _______________________________________________ >> > >>> >> >> > cctbxbb mailing list >> > >>> >> >> > cctbxbb(a)phenix-online.org >> > >>> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> -- >> > >>> >> >> Gergely Katona, PhD >> > >>> >> >> Associate Professor >> > >>> >> >> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> > >>> >> >> Box 462, 40530 G?teborg, Sweden >> > >>> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: >> +46-31-786-3910 >> > >>> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> > >>> >> >> >> > >>> >> >> _______________________________________________ >> > >>> >> >> cctbxbb mailing list >> > >>> >> >> cctbxbb(a)phenix-online.org >> > >>> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> > >> > >>> >> > _______________________________________________ >> > >>> >> > cctbxbb mailing list >> > >>> >> > cctbxbb(a)phenix-online.org >> > >>> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> -- >> > >>> >> Gergely Katona, PhD >> > >>> >> Associate Professor >> > >>> >> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> > >>> >> Box 462, 40530 G?teborg, Sweden >> > >>> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> > >>> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> > >>> >> >> > >>> >> _______________________________________________ >> > >>> >> cctbxbb mailing list >> > >>> >> cctbxbb(a)phenix-online.org >> > >>> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> > >> > >>> > _______________________________________________ >> > >>> > cctbxbb mailing list >> > >>> > cctbxbb(a)phenix-online.org >> > >>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Gergely Katona, PhD >> > >>> Associate Professor >> > >>> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> > >>> Box 462, 40530 G?teborg, Sweden >> > >>> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> > >>> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> > >>> >> > >>> _______________________________________________ >> > >>> cctbxbb mailing list >> > >>> cctbxbb(a)phenix-online.org >> > >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > > >> > > _______________________________________________ >> > > cctbxbb mailing list >> > > cctbxbb(a)phenix-online.org >> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >> > >> > >> > -- >> > Gergely Katona, Professor >> > Department of Chemistry and Molecular Biology, University of Gothenburg >> > Box 462, 40530 G?teborg, Sweden >> > Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> > Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> >> >> >> -- >> Gergely Katona, Professor >> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> Box 462, 40530 G?teborg, Sweden >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> Web: >> https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… >> , Email: gergely.katona(a)gu.se >> >> _______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >

1 0

conda cctbx on Mac 10.15.2 working with python3.8
by Mooers, Blaine H.M. (HSC) 05 Jan '20

05 Jan '20

Hi Billy, Gergely's post yesterday stimulated me to check my conda cctbx setup. I had earlier reported success with conda cctbx for Python3.6, 3.7, and 3.8. This morning I found in my path an earlier python3.6 build of cctbx that may have given me false positives with my conda installs of cctbx. After removing the old build of cctbx from my path, I modified my zshrc file as Gergely suggested. I can confirm that results that Gergely reported yesterday. I used a Mac OS X 10.15.2 with zsh as my shell and anaconda3 installed as /opt/anaconda. I did my testing in a conda env named cctbx38. I added 'export LIBTBX_BUILD=$CONDA_PREFIX' in my .zshrc file. After sourcing my .zshrc file, 'echo $LIBTBX_BUILD' returns '/opt/anaconda/envs/cctbx38' as expected. The toy script below prints the calculated Miller indices as a tuples in a terminal window. It was run using 'python millerSet.py'. Note that the printing of 1 million indices to a jupyter notebook cell is disappointingly slow!! Reduce each unit cell dimension by two orders of magnitude before trying this at home! The following code also worked with Python3.6 and Python3.7 in their corresponding cctbx36 and cctbx37 envs. from cctbx import crystal from cctbx import miller ms = miller.build_set( crystal_symmetry=crystal.symmetry( space_group_symbol='P4', unit_cell=(150.8,150.8,250.4,90.0,90.0,90.0)), anomalous_flag=False, d_min=1.4) [print(hkl) for hkl in ms.indices()] I used conda python3.8.0 built on November 22. Python3.8.1 does not seem to be available yet from Anaconda. Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular Biology College of Medicine University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center (BRC) Rm. 466 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Saturday, January 04, 2020 2:00 PM To: cctbxbb(a)phenix-online.org Subject: [EXTERNAL] cctbxbb Digest, Vol 130, Issue 2 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. Re: some thoughts on cctbx and pip (Billy Poon) ---------------------------------------------------------------------- Message: 1 Date: Fri, 3 Jan 2020 21:04:25 +0100 From: Billy Poon <BKPoon(a)lbl.gov> To: cctbx mailing list <cctbxbb(a)phenix-online.org> Subject: Re: [cctbxbb] some thoughts on cctbx and pip Message-ID: <CAHmYUcbWS7FXA6EPUx=AC2X6m6gPY=GqH-MwGD+uzpTaOKSyjQ(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi Gergely, Happy New Year! Can you provide the output to "conda list" for each environment? I suspect that the old version of cctbx might still be around. Is there a file named PYTHON_VERSION_MAJOR_MINOR in "/home/gergely/anaconda3/lib"? If so, delete that file. To make sure you are downloading a new copy of the package, clear the cache where conda keeps copies of all the packages. To do that, you can run "conda clean -y --all". You can remove the "-y" flag if you want to see what will be deleted. The hash for the new linux package should be h8b68381_0. The last error looks like an incompatible version of boost or gcc library was installed. The output from "conda list" should show the boost and boost-cpp packages coming from conda-forge. I would avoid installing into the root or base environment because that usually pulls from the default channels. All the dependencies should be pulled from the conda-forge channel. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… On Fri, Jan 3, 2020 at 2:20 PM Gergely Katona <gkatona(a)gmail.com> wrote: > I found some other errors when using the test environment. > > > --------------------------------------------------------------------------- > RuntimeError Traceback (most recent call last) > <ipython-input-1-fe308b345b3f> in <module> > 10 import scipy as sp > 11 from scipy import linalg > ---> 12 import iotbx.pdb > 13 import cctbx > 14 import pandas as pd > > ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= > in <module> > 8 > 9 import iotbx.pdb.records > ---> 10 import iotbx.pdb.hierarchy > 11 from scitbx import matrix > 12 > > ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__hierarchy.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=QlonHINbJqyu1vTwlBJWKyChCMCCJxpIWIaT5_rrkLc&e= > in <module> > 14 from six.moves import cStringIO as StringIO > 15 from iotbx.pdb import hy36encode, hy36decode > ---> 16 import iotbx.cif.model > 17 from cctbx import crystal > 18 from libtbx import group_args > > ~/anaconda3/envs/test/lib/python3.8/site-packages/iotbx/cif/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= > in <module> > 19 > 20 from cctbx.array_family import flex > ---> 21 from cctbx import miller > 22 from iotbx.cif import model, builders, geometry > 23 from libtbx.containers import OrderedDict > > ~/anaconda3/envs/test/lib/python3.8/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= > in <module> > 11 > 12 from cctbx import crystal > ---> 13 from cctbx import maptbx > 14 from cctbx import sgtbx > 15 from cctbx.sgtbx import lattice_symmetry > > ~/anaconda3/envs/test/lib/python3.8/site-packages/cctbx/maptbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= > in <module> > 15 from libtbx import adopt_init_args > 16 from libtbx.utils import Sorry > ---> 17 import libtbx.load_env > 18 import math > 19 import sys, os > > ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= in > <module> > 3 import libtbx.env_config > 4 import os > ----> 5 libtbx.env = libtbx.env_config.unpickle() > 6 libtbx.env.set_os_environ_all_dist() > 7 libtbx.env.dispatcher_name = > os.environ.get("LIBTBX_DISPATCHER_NAME") > > ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= > in unpickle() > 2736 env = pickle.load(libtbx_env) > 2737 if (env.python_version_major_minor != sys.version_info[:2]): > -> 2738 env.raise_python_version_incompatible() > 2739 if (op.realpath(build_path) != op.realpath(abs(env.build_path))): > 2740 env.build_path.reset(build_path) > > ~/anaconda3/envs/test/lib/python3.8/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= > in raise_python_version_incompatible(self, prev_pvmm) > 469 if (prev_pvmm is None): > 470 prev_pvmm = "%d.%d" % self.python_version_major_minor > --> 471 raise RuntimeError("Python version incompatible with this > build:\n" > 472 + " Build directory: %s\n" % > show_string(abs(self.build_path)) > 473 + " Python version used initially: %s\n" % prev_pvmm > > RuntimeError: Python version incompatible with this build: > Build directory: "/home/gergely/anaconda3" > Python version used initially: 3.6 > Python version in use now: 3.8 > > > Interestingly in the base environment all import worked, but I got an > error at: > > > ArgumentError Traceback (most recent call last) > <ipython-input-4-5bc71c36e121> in <module> > 33 atm_odd.set_b(0) > 34 > ---> 35 odd.write_pdb_file("bdiff_in_even_000.pdb") #skriver ny pdb > med dessa inkluderade > 36 dfWBvals=pd.DataFrame(rows_list) > > ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > write_pdb_file(self, file_name, open_append, crystal_symmetry, > cryst1_z, write_scale_records, append_end, atom_hetatm, sigatm, > anisou, siguij) > 930 siguij=True): > 931 if (crystal_symmetry is Auto): > --> 932 crystal_symmetry = self.crystal_symmetry() > 933 if (cryst1_z is Auto): > 934 cryst1_z = self.extract_cryst1_z_columns() > > ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > crystal_symmetry(self, crystal_symmetry, weak_symmetry) > 1218 crystal_symmetry=None, > 1219 weak_symmetry=False): > -> 1220 self_symmetry = self.crystal_symmetry_from_cryst1() > 1221 if (self_symmetry is None): > 1222 self_symmetry = self._crystal_symmetry_from_cns_remark_sg() > > ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > crystal_symmetry_from_cryst1(self) > 1195 for line in self.crystallographic_section(): > 1196 if (line.startswith("CRYST1")): > -> 1197 return > cryst1_interpretation.crystal_symmetry(cryst1_record=line) > 1198 return None > 1199 > > ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__cryst1-5Finterpretation.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=3OchLb18AMp3YpXOuEb8BDOwhl2XLrV1QAfZByH-l2E&e= > in crystal_symmetry(cryst1_record) > 136 space_group_info=None) > 137 space_group_info = > categorize(cryst1_record.sgroup).space_group_info( > --> 138 unit_cell=u) > 139 return crystal.symmetry(unit_cell=u, > space_group_info=space_group_info) > > ~/anaconda3/lib/python3.6/site-packages/iotbx/pdb/https://urldefense.proofpoint.com/v2/url?u=http-3A__cryst1-5Finterpretation.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=3OchLb18AMp3YpXOuEb8BDOwhl2XLrV1QAfZByH-l2E&e= > in space_group_info(self, unit_cell) > 85 if (self.symbol is None): return None > 86 if (self.category is None): > ---> 87 try: return sgtbx.space_group_info(self.symbol) > 88 except RuntimeError: return None > 89 if (isinstance(unit_cell, uctbx.ext.unit_cell)): > > ~/anaconda3/lib/python3.6/site-packages/cctbx/sgtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > __init__(self, symbol, table_id, group, number, space_group_t_den) > 100 assert group is None > 101 if (table_id is None): > --> 102 symbols = space_group_symbols(symbol) > 103 else: > 104 if (isinstance(symbol, int)): symbol = str(symbol) > > ArgumentError: Python argument types in > space_group_symbols.__init__(space_group_symbols, str) > did not match C++ signature: > __init__(_object*, int space_group_number) > __init__(_object*, int space_group_number, > std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > extension='') > __init__(_object*, int space_group_number, > std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > extension='', std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > table_id='') > __init__(_object*, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > symbol) > __init__(_object*, std::__cxx11::basic_string<char, > std::char_traits<char>, std::allocator<char> > symbol, > std::__cxx11::basic_string<char, std::char_traits<char>, > std::allocator<char> > table_id='') > > > Gergely > > > > On Fri, Jan 3, 2020 at 1:44 PM Gergely Katona <gkatona(a)gmail.com> wrote: > > > > Dear Billy, > > > > I did some tests installing in my base conda was not troublefree in > > python 3.8. Running iotbx.fetch_pdb --all --mtz 1kp8 resulted in boost > > related error when converting to mtz. Installing into a test > > environment went fine with iotbx.fetch_pdb --all --mtz 1kp8 running > > successfully. I could add pymc3, jupyter and seaborn and iotbx still > > did not break. My LIBTBX_BUILD is not set however in the test > > environment, which breaks miller when I was trying to import it. > > Manually setting the LIBTBX_BUILD to CONDA_PREFIX (setenv LIBTBX_BUILD > > $CONDA_PREFIX) fixes this and one of my script works fine! I will come > > back if I find some problems with my other scripts, but so far this > > looks promising! > > > > > > Happy new year! > > > > Gergely > > > > > > > > > > > > On Fri, Dec 20, 2019 at 8:16 PM Billy Poon <BKPoon(a)lbl.gov> wrote: > > > > > > Hi everyone, > > > > > > I updated the conda packages on the cctbx-dev channel and there are > also now packages for macOS. The old packages have been removed so you can > create a new environment with > > > > > > conda create -n test -c cctbx-dev cctbx python=2.7 > > > conda activate test > > > > > > where "test" is the environment name and 2.7 is the python version. > For the python version, you can also pick 3.6, 3.7, or 3.8. I think the > latest conda will install 3.8 as the default python if the python argument > is not provided. Or install into an existing environment (with python > already installed) with the same command as before, > > > > > > conda install -c cctbx-dev cctbx > > > > > > Again, your .condarc file should have conda-forge as the first channel > so that dependencies can be pulled correctly. > > > > > > The dispatchers should work now and $LIBTBX_BUILD will be set on > activation of an environment (and unset on environment deactivation) or > after installing into an existing environment, so you can do more > interesting things like > > > > > > iotbx.fetch_pdb --all --mtz 1kp8 > > > > > > However, this package does not install databases, so > > > > > > molprobity.molprobity 1kp8.pdb 1kp8.mtz > > > > > > will fail because the database is not available (also probe and reduce > are not built). > > > > > > Some known issues in no particular order, > > > > > > 1) You can run "libtbx.run_tests_parallel module=cctbx nproc=Auto" to > run tests, but not all the test files were copied, so I got 30 failures > here. The final package will probably not have the test files. > > > 2) The other python dispatchers, like "cctbx.python", were not > updated, so those will fail. But you can just run "python" in the active > environment. > > > 3) Parts of mmtbx still need to be updated for Python 3. > > > 4) The LIBTBX_BUILD variable should be set for sh and csh shells. Let > me know if it's not working for your shell (i.e. "echo $LIBTBX_BUILD" > should show the same thing as $CONDA_PREFIX). > > > 5) One future step is to enable building other CCTBX-like modules with > this conda package. This will require some bookkeeping updates so that > modules in the conda environment and the modules being built are tracked > properly. Also, making sure that all the necessary headers exist in > $PREFIX/include. > > > > > > Gergely, let me know if your scripts are working. > > > > > > Happy holidays! > > > > > > -- > > > Billy K. Poon > > > Research Scientist, Molecular Biophysics and Integrated Bioimaging > > > Lawrence Berkeley National Laboratory > > > 1 Cyclotron Road, M/S 33R0345 > > > Berkeley, CA 94720 > > > Tel: (510) 486-5709 > > > Fax: (510) 486-5909 > > > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > > > > > > > > On Wed, Dec 18, 2019 at 11:25 AM Billy Poon <BKPoon(a)lbl.gov> wrote: > > >> > > >> Hi Gergely, > > >> > > >> That's the other information that hasn't been updated yet in the > conda package. :) > > >> > > >> For LIBTBX_BUILD, I can set up environment variables sort of how the > conda gcc package sets up environment variables. I have to double check to > see if they're set after installation of the package and after activation > of an environment. The variable will just be a copy of $CONDA_PREFIX. > > >> > > >> The libtbx_env file is a file that exists in $LIBTBX_BUILD that > basically stores configuration information. I'll have to modify the > contents to update the locations of the different modules since the conda > package will not have a "modules" directory. > > >> > > >> -- > > >> Billy K. Poon > > >> Research Scientist, Molecular Biophysics and Integrated Bioimaging > > >> Lawrence Berkeley National Laboratory > > >> 1 Cyclotron Road, M/S 33R0345 > > >> Berkeley, CA 94720 > > >> Tel: (510) 486-5709 > > >> Fax: (510) 486-5909 > > >> Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > >> > > >> > > >> On Wed, Dec 18, 2019 at 2:08 AM Gergely Katona <gkatona(a)gmail.com> > wrote: > > >>> > > >>> Hi Billy, > > >>> > > >>> Conda install went fine with your instructions after rearranging the > > >>> channels by putting cctbx last. I removed all environmental variables > > >>> and previous build of cctbx. > > >>> > > >>> import sys > > >>> print (sys.path, sys.prefix) > > >>> > > >>> ['/home/gergely/anaconda3/lib/https://urldefense.proofpoint.com/v2/url?u=http-3A__python36.zip&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=cUHL8qgeloeUO3oYTQcr9d56RegJk01wJ2i-c3Zo6oE&e= ', > > >>> '/home/gergely/anaconda3/lib/python3.6', > > >>> '/home/gergely/anaconda3/lib/python3.6/lib-dynload', '', > > >>> '/home/gergely/anaconda3/lib/python3.6/site-packages', > > >>> > '/home/gergely/anaconda3/lib/python3.6/site-packages/IPython/extensions', > > >>> '/home/gergely/.ipython'] /home/gergely/anaconda3 > > >>> > > >>> Many imports went fine including pymc3, but I encountered problems > > >>> with these three: > > >>> > > >>> from cctbx import miller > > >>> import iotbx.pdb > > >>> from iotbx import reflection_file_reader, mtz > > >>> > > >>> > > >>> I also got type error when handling space groups. > > >>> > > >>> > > >>> > > >>> > --------------------------------------------------------------------------- > > >>> KeyError Traceback (most recent > call last) > > >>> <ipython-input-5-cedbd4f84f3d> in <module> > > >>> 25 from cctbx import uctbx > > >>> 26 from cctbx import sgtbx > > >>> ---> 27 from cctbx import miller > > >>> 28 #from iotbx import reflection_file_reader, mtz > > >>> 29 sns.set_context("poster") > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > <module> > > >>> 11 > > >>> 12 from cctbx import crystal > > >>> ---> 13 from cctbx import maptbx > > >>> 14 from cctbx import sgtbx > > >>> 15 from cctbx.sgtbx import lattice_symmetry > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/maptbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > <module> > > >>> 15 from libtbx import adopt_init_args > > >>> 16 from libtbx.utils import Sorry > > >>> ---> 17 import libtbx.load_env > > >>> 18 import math > > >>> 19 import sys, os > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= in > <module> > > >>> 3 import libtbx.env_config > > >>> 4 import os > > >>> ----> 5 libtbx.env = libtbx.env_config.unpickle() > > >>> 6 libtbx.env.set_os_environ_all_dist() > > >>> 7 libtbx.env.dispatcher_name = > os.environ.get("LIBTBX_DISPATCHER_NAME") > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= in > unpickle() > > >>> 2603 > > >>> 2604 def unpickle(): > > >>> -> 2605 build_path = os.environ["LIBTBX_BUILD"] > > >>> 2606 > set_preferred_sys_prefix_and_sys_executable(build_path=build_path) > > >>> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") > > >>> > > >>> ~/anaconda3/lib/python3.6/https://urldefense.proofpoint.com/v2/url?u=http-3A__os.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=hA1zo5wap0c9wK3DrlWIKqYBIqqbfKXca5jQ9aX5pYU&e= in __getitem__(self, key) > > >>> 667 except KeyError: > > >>> 668 # raise KeyError with the original key value > > >>> --> 669 raise KeyError(key) from None > > >>> 670 return self.decodevalue(value) > > >>> 671 > > >>> > > >>> KeyError: 'LIBTBX_BUILD' > > >>> > > >>> > > >>> > > >>> > --------------------------------------------------------------------------- > > >>> ArgumentError Traceback (most recent > call last) > > >>> <ipython-input-7-2acd3a9ce26a> in <module> > > >>> 26 return > ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar > > >>> 27 > > >>> ---> 28 > ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar=initializecrystal() > > >>> > > >>> <ipython-input-7-2acd3a9ce26a> in initializecrystal() > > >>> 11 uc = uctbx.unit_cell(unit_cell) > > >>> 12 wavelength = 1.54980 > > >>> ---> 13 xtal_symm = crystal.symmetry(unit_cell=unit_cell, > > >>> space_group_symbol="P 43 21 2") > > >>> 14 > > >>> 15 ms = > > >>> miller.build_set(crystal_symmetry=xtal_symm,anomalous_flag=True, > > >>> d_min=1.61) > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/crystal/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > > >>> __init__(self, unit_cell, space_group_symbol, space_group_info, > > >>> space_group, correct_rhombohedral_setting_if_necessary, > > >>> assert_is_compatible_unit_cell, > raise_sorry_if_incompatible_unit_cell, > > >>> force_compatible_unit_cell) > > >>> 74 if (space_group_symbol is not None): > > >>> 75 self._space_group_info = sgtbx.space_group_info( > > >>> ---> 76 symbol=space_group_symbol) > > >>> 77 elif (space_group is not None): > > >>> 78 if (isinstance(space_group, sgtbx.space_group)): > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/cctbx/sgtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > > >>> __init__(self, symbol, table_id, group, number, space_group_t_den) > > >>> 100 assert group is None > > >>> 101 if (table_id is None): > > >>> --> 102 symbols = space_group_symbols(symbol) > > >>> 103 else: > > >>> 104 if (isinstance(symbol, int)): symbol = str(symbol) > > >>> > > >>> ArgumentError: Python argument types in > > >>> space_group_symbols.__init__(space_group_symbols, str) > > >>> did not match C++ signature: > > >>> __init__(_object*, int space_group_number) > > >>> __init__(_object*, int space_group_number, > > >>> std::__cxx11::basic_string<char, std::char_traits<char>, > > >>> std::allocator<char> > extension='') > > >>> __init__(_object*, int space_group_number, > > >>> std::__cxx11::basic_string<char, std::char_traits<char>, > > >>> std::allocator<char> > extension='', std::__cxx11::basic_string<char, > > >>> std::char_traits<char>, std::allocator<char> > table_id='') > > >>> __init__(_object*, std::__cxx11::basic_string<char, > > >>> std::char_traits<char>, std::allocator<char> > symbol) > > >>> __init__(_object*, std::__cxx11::basic_string<char, > > >>> std::char_traits<char>, std::allocator<char> > symbol, > > >>> std::__cxx11::basic_string<char, std::char_traits<char>, > > >>> std::allocator<char> > table_id='') > > >>> > > >>> > > >>> When I tried to set LIBTBX_BUILD to /home/gergely/anaconda3 (this is > > >>> sys.prefix on my system), I got other problems when importing. Can > > >>> LIBTBX_BUILD be set in the conda package? > > >>> > > >>> > > >>> > > >>> > --------------------------------------------------------------------------- > > >>> FileNotFoundError Traceback (most recent > call last) > > >>> <ipython-input-1-7e2b144826cc> in <module> > > >>> 26 from cctbx import sgtbx > > >>> 27 #from cctbx import miller > > >>> ---> 28 from iotbx import reflection_file_reader, mtz > > >>> 29 sns.set_context("poster") > > >>> 30 plt.rcParams.update({'figure.autolayout': True}) > > >>> > > >>> > ~/anaconda3/lib/python3.6/site-packages/iotbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__reflection-5Ffile-5Freader.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=U5BvLmuCgufvN_LRxQw9n78p33jNaRrXVbH1BRSXgxg&e= > > >>> in <module> > > >>> 53 > > >>> 54 from __future__ import absolute_import, division, > print_function > > >>> ---> 55 from iotbx import mtz > > >>> 56 from iotbx.scalepack import merge as scalepack_merge > > >>> 57 from iotbx.scalepack import no_merge_original_index as > > >>> scalepack_no_merge > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=9NiF4LTTyEpDCh-vdb5bXVIJ4i9E6HmD6eWJg_3SjzE&e= in > <module> > > >>> 9 import iotbx_mtz_ext as ext > > >>> 10 > > >>> ---> 11 from iotbx.mtz import extract_from_symmetry_lib > > >>> 12 from cctbx import xray > > >>> 13 import cctbx.xray.observation_types > > >>> > > >>> > ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__extract-5Ffrom-5Fsymmetry-5Flib.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=k3WR6VZd4-PDULPXl4O0Vwz9q3LCaocUUhK_ehTn9UE&e= > > >>> in <module> > > >>> 1 from __future__ import absolute_import, division, > print_function > > >>> 2 from cctbx import sgtbx > > >>> ----> 3 import libtbx.load_env > > >>> 4 import os.path as op > > >>> 5 from six.moves import range > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=rJbfdPr3ymiz7Ir508_VGIhspAKCOwozKhY5I6OXux0&e= in > <module> > > >>> 3 import libtbx.env_config > > >>> 4 import os > > >>> ----> 5 libtbx.env = libtbx.env_config.unpickle() > > >>> 6 libtbx.env.set_os_environ_all_dist() > > >>> 7 libtbx.env.dispatcher_name = > os.environ.get("LIBTBX_DISPATCHER_NAME") > > >>> > > >>> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=9IjF95nqsrMq_R_HHQ4nsTpGpSpiU2e4ynF8kN3qTHY&s=UrsFhQ0dxNNuzXmOT3D6pnGJ5lXko4mfrKceNgBaQV4&e= in > unpickle() > > >>> 2605 build_path = os.environ["LIBTBX_BUILD"] > > >>> 2606 > set_preferred_sys_prefix_and_sys_executable(build_path=build_path) > > >>> -> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") > > >>> 2608 env = pickle.load(libtbx_env) > > >>> 2609 if (env.python_version_major_minor != > sys.version_info[:2]): > > >>> > > >>> FileNotFoundError: [Errno 2] No such file or directory: > > >>> '/home/gergely/anaconda3/libtbx_env' > > >>> > > >>> On Wed, Dec 18, 2019 at 9:10 AM Billy Poon <BKPoon(a)lbl.gov> wrote: > > >>> > > > >>> > Hi Gergely, > > >>> > > > >>> > I've uploaded linux packages to a new channel, cctbx-dev, and you > can install it with > > >>> > > > >>> > conda install -c cctbx-dev cctbx > > >>> > > > >>> > in your current environment. You should set your ~/.condarc file > to pull the other dependencies from the conda-forge channel first, so put > conda-forge above cctbx. Mine looks like > > >>> > > > >>> > channels: > > >>> > - conda-forge > > >>> > - defaults > > >>> > - cctbx > > >>> > > > >>> > Do you need dxtbx for your scripts? This package does not build > that part. I think the plan is to build a separate conda package for dxtbx > so that it can be updated more frequently. I can rebuild the packages to > include it for testing, but the one being submitted to conda-forge will not > have it. > > >>> > > > >>> > Also, your error message is probably due to version of HDF5 that > the development build installs. The https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… script will install > 1.10.4, but your other dependency is looking for 1.10.5. Installing this > cctbx conda package should install 1.10.5, which should fix the issue. > Also, I'm updating those environments and 1.10.5 will be the new default > version. > > >>> > > > >>> > Lastly, the dispatchers will not work in these packages because > the old paths during the build process are still in them and many of them > expect some additional information that has not been updated in the > packages yet. I'm in the process of doing that and will update the > cctbx-dev channel when that's done. However, by starting python, you can > import cctbx modules. So you can run commands like > > >>> > > > >>> > from scitbx.array_family import flex > > >>> > a = flex.random_double(1000000) > > >>> > b = flex.min_max_mean_double(a) > > >>> > b.min > > >>> > b.max > > >>> > b.mean > > >>> > > > >>> > Thanks! > > >>> > > > >>> > -- > > >>> > Billy K. Poon > > >>> > Research Scientist, Molecular Biophysics and Integrated Bioimaging > > >>> > Lawrence Berkeley National Laboratory > > >>> > 1 Cyclotron Road, M/S 33R0345 > > >>> > Berkeley, CA 94720 > > >>> > Tel: (510) 486-5709 > > >>> > Fax: (510) 486-5909 > > >>> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > >>> > > > >>> > > > >>> > On Tue, Dec 17, 2019 at 5:39 AM Gergely Katona <gkatona(a)gmail.com> > wrote: > > >>> >> > > >>> >> Dear Billy, > > >>> >> > > >>> >> Thank you for the detailed explanation, I look forward to do > further > > >>> >> testing! I aim to pool all modules under the same environment, if > > >>> >> there are conflicts then I just try to reshuffle the order of > imports. > > >>> >> So far this did not cause problems for me even when I was using > system > > >>> >> python, but of course it is not the most prudent thing to do. With > > >>> >> anaconda everything is much more standardized and isolated > already and > > >>> >> without being superuser I can have my familiar environment at any > > >>> >> synchrotron based cluster. It is great that cctbx will be an > integral > > >>> >> part of this ecosystem and this was also the last thing holding me > > >>> >> back from adopting python3. About hdf5, pymc3 requires it and > > >>> >> importing causes a kernel restart with the following error > messages, > > >>> >> curiously if I import h5py first this can be avoided. > > >>> >> > > >>> >> Best wishes, > > >>> >> > > >>> >> Gergely > > >>> >> > > >>> >> import pymc3 as pm > > >>> >> > > >>> >> Warning! ***HDF5 library version mismatched error*** > > >>> >> The HDF5 header files used to compile this application do not > match > > >>> >> the version used by the HDF5 library to which this application is > linked. > > >>> >> Data corruption or segmentation faults may occur if the > application continues. > > >>> >> This can happen when an application was compiled by one version > of HDF5 but > > >>> >> linked with a different version of static or shared HDF5 library. > > >>> >> You should recompile the application or check your shared library > related > > >>> >> settings such as 'LD_LIBRARY_PATH'. > > >>> >> You can, at your own risk, disable this warning by setting the > environment > > >>> >> variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'. > > >>> >> Setting it to 2 or higher will suppress the warning messages > totally. > > >>> >> Headers are 1.10.4, library is 1.10.5 > > >>> >> SUMMARY OF THE HDF5 CONFIGURATION > > >>> >> ================================= > > >>> >> > > >>> >> General Information: > > >>> >> ------------------- > > >>> >> HDF5 Version: 1.10.5 > > >>> >> Configured on: Tue Oct 22 12:02:13 UTC 2019 > > >>> >> Configured by: conda@16247e67ecd5 > > >>> >> Host system: x86_64-conda_cos6-linux-gnu > > >>> >> Uname information: Linux 16247e67ecd5 > 4.15.0-1059-azure > > >>> >> #64-Ubuntu SMP Fri Sep 13 17:02:44 UTC 2019 x86_64 x86_64 x86_64 > > >>> >> GNU/Linux > > >>> >> Byte sex: little-endian > > >>> >> Installation point: /home/gergely/anaconda3 > > >>> >> > > >>> >> Compiling Options: > > >>> >> ------------------ > > >>> >> Build Mode: production > > >>> >> Debugging Symbols: no > > >>> >> Asserts: no > > >>> >> Profiling: no > > >>> >> Optimization Level: high > > >>> >> > > >>> >> Linking Options: > > >>> >> ---------------- > > >>> >> Libraries: static, shared > > >>> >> Statically Linked Executables: > > >>> >> LDFLAGS: -Wl,-O2 -Wl,--sort-common > > >>> >> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags > > >>> >> -Wl,--gc-sections -Wl,-rpath,/home/gergely/anaconda3/lib > > >>> >> -Wl,-rpath-link,/home/gergely/anaconda3/lib > > >>> >> -L/home/gergely/anaconda3/lib > > >>> >> H5_LDFLAGS: > > >>> >> AM_LDFLAGS: -L/home/gergely/anaconda3/lib > > >>> >> Extra libraries: -lrt -lpthread -lz -ldl -lm > > >>> >> Archiver: > > >>> >> > /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar > > >>> >> AR_FLAGS: cr > > >>> >> Ranlib: > > >>> >> > /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib > > >>> >> > > >>> >> Languages: > > >>> >> ---------- > > >>> >> C: yes > > >>> >> C Compiler: > > >>> >> > /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc > > >>> >> CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 > > >>> >> -I/home/gergely/anaconda3/include > > >>> >> H5_CPPFLAGS: -D_GNU_SOURCE > > >>> >> -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API > > >>> >> AM_CPPFLAGS: > -I/home/gergely/anaconda3/include > > >>> >> C Flags: -march=nocona -mtune=haswell > > >>> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 > > >>> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include > > >>> >> > -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 > > >>> >> > -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix > > >>> >> H5 C Flags: -std=c99 -pedantic -Wall > -Wextra > > >>> >> -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual > -Wconversion > > >>> >> -Wdeclaration-after-statement -Wdisabled-optimization > -Wfloat-equal > > >>> >> -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-declarations > > >>> >> -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs > > >>> >> -Wold-style-definition -Wpacked -Wpointer-arith -Wredundant-decls > > >>> >> -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum > -Wundef > > >>> >> -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings > > >>> >> -finline-functions -s -Wno-inline -Wno-aggregate-return > > >>> >> -Wno-missing-format-attribute -Wno-missing-noreturn -O > > >>> >> AM C Flags: > > >>> >> Shared C Library: yes > > >>> >> Static C Library: yes > > >>> >> > > >>> >> > > >>> >> Fortran: yes > > >>> >> Fortran Compiler: > > >>> >> > /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-gfortran > > >>> >> Fortran Flags: > > >>> >> H5 Fortran Flags: -pedantic -Wall -Wextra > -Wunderflow > > >>> >> -Wimplicit-interface -Wsurprising -Wno-c-binding-type -s -O2 > > >>> >> AM Fortran Flags: > > >>> >> Shared Fortran Library: yes > > >>> >> Static Fortran Library: yes > > >>> >> > > >>> >> C++: yes > > >>> >> C++ Compiler: > > >>> >> > /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++ > > >>> >> C++ Flags: -fvisibility-inlines-hidden > > >>> >> -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell > > >>> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 > > >>> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include > > >>> >> > -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 > > >>> >> > -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix > > >>> >> H5 C++ Flags: -pedantic -Wall -W -Wundef > -Wshadow > > >>> >> -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings > -Wconversion > > >>> >> -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual > > >>> >> -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor > > >>> >> -Wctor-dtor-privacy -Wabi -finline-functions -s -O > > >>> >> AM C++ Flags: > > >>> >> Shared C++ Library: yes > > >>> >> Static C++ Library: yes > > >>> >> > > >>> >> Java: no > > >>> >> > > >>> >> > > >>> >> Features: > > >>> >> --------- > > >>> >> Parallel HDF5: no > > >>> >> Parallel Filtered Dataset Writes: no > > >>> >> Large Parallel I/O: no > > >>> >> High-level library: yes > > >>> >> Threadsafety: yes > > >>> >> Default API mapping: v110 > > >>> >> With deprecated public symbols: yes > > >>> >> I/O filters (external): deflate(zlib) > > >>> >> MPE: no > > >>> >> Direct VFD: no > > >>> >> dmalloc: no > > >>> >> Packages w/ extra debug output: none > > >>> >> API tracing: no > > >>> >> Using memory checker: yes > > >>> >> Memory allocation sanity checks: no > > >>> >> Function stack tracing: no > > >>> >> Strict file format checks: no > > >>> >> Optimization instrumentation: no > > >>> >> > > >>> >> On Tue, Dec 17, 2019 at 8:35 AM Billy Poon <BKPoon(a)lbl.gov> > wrote: > > >>> >> > > > >>> >> > Hi Gergely, > > >>> >> > > > >>> >> > Let me build the test package tomorrow. All the gritty details > for building with conda is being finalized and the official documentation > will be updated to describe the steps. It would be too confusing to keep > changing the documentation as the process evolves. You do have the general > process, though, which is summarized as follows. > > >>> >> > > > >>> >> > 1) Installing dependencies. The cctbx_dependencies metapackage > was an initial approach for managing the CCTBX dependencies, but after > contacting the conda-forge folks, they recommended using the --only-deps > flag. So when the CCTBX conda package is available, you'll be able to get a > set of dependencies with, > > >>> >> > > > >>> >> > conda install -c conda-forge --only-deps cctbx > > >>> >> > > > >>> >> > By default, the https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file will automatically install a > set of dependencies in the "conda_base" directory (and a conda installation > if one is not found). It just uses standard conda environment files located > in libtbx/auto_build/conda_envs, so you do not need to install > cctbx_dependencies as a separate step. The environment files avoid channel > issues by explicitly defining the channel to pull the packages from and the > cctbx channel just stores copies of packages from conda-forge. There were > issues earlier where the conda-forge packages would sometimes be moved to a > different label. The --use-conda flag also accepts a path to $CONDA_PREFIX > if you want to use a specific environment for testing. > > >>> >> > > > >>> >> > 2) Building. The https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file handles that with SCons. > > >>> >> > > > >>> >> > 3) Running. After building, there should be a https://urldefense.proofpoint.com/v2/url?u=http-3A__setpaths.sh&d=DwICAg&c=… (and > .csh) file that adds build/bin to your path. The build/bin directory has > our dispatchers, which are just scripts that set up the environment > variables for you. This prevents other programs from loading our libraries, > whose versions may conflict. You should see that there is a "python" > dispatcher, which is a convenience for developers. Otherwise, you can use > libtbx.python, which will be able to import CCTBX modules. > > >>> >> > > > >>> >> > The running part is where the conda package for CCTBX will be > different than this build. Since our Python files and extensions modules > will be in the "site-packages" directory for Python, the PYTHONPATH > variable will not be needed (and conda suggests that that variable not be > set). The other CCTBX libraries will be in $CONDA_PREFIX/lib, so > LD_LIBRARY_PATH is not needed. In an active environment, PATH will already > be modified. And then our LIBTBX_BUILD directory can be set to sys.prefix. > > >>> >> > > > >>> >> > So with the conda package, you would only need to activate your > conda environment and CCTBX should integrate with other conda packages. > What is the conflict with hdf5? That's something that should be fixed. > Thanks! > > >>> >> > > > >>> >> > -- > > >>> >> > Billy K. Poon > > >>> >> > Research Scientist, Molecular Biophysics and Integrated > Bioimaging > > >>> >> > Lawrence Berkeley National Laboratory > > >>> >> > 1 Cyclotron Road, M/S 33R0345 > > >>> >> > Berkeley, CA 94720 > > >>> >> > Tel: (510) 486-5709 > > >>> >> > Fax: (510) 486-5909 > > >>> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > >>> >> > > > >>> >> > > > >>> >> > On Mon, Dec 16, 2019 at 2:53 AM Gergely Katona < > gkatona(a)gmail.com> wrote: > > >>> >> >> > > >>> >> >> Dear Billy, > > >>> >> >> > > >>> >> >> Thank you for this update and for your efforts! I found a > solution and > > >>> >> >> indeed most things already work in anaconda3. The steps I took > (even > > >>> >> >> if these instructions will have short expiry date): > > >>> >> >> > > >>> >> >> Modified .condarc with: > > >>> >> >> channels: > > >>> >> >> - cctbx > > >>> >> >> - conda-forge > > >>> >> >> - defaults > > >>> >> >> > > >>> >> >> run > > >>> >> >> conda conda install cctbx_dependencies python=3.6 > > >>> >> >> > > >>> >> >> I expect this will work without python=3.6 in the near future. > > >>> >> >> > > >>> >> >> Then compiling cctbx with anaconda3 python went without any > problem > > >>> >> >> when using these flags: > > >>> >> >> python https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… --use-conda --python3 --nproc=12 > > >>> >> >> > > >>> >> >> Then I probably did the unorthodox thing and sourced these > directories: > > >>> >> >> > > >>> >> >> setenv LIBTBX_BUILD /home/gergely/cctbx/build > > >>> >> >> setenv PATH ${PATH}:/home/gergely/cctbx/build/bin > > >>> >> >> setenv PYTHONPATH > > >>> >> >> > /home/gergely/cctbx/modules/cctbx_project:/home/gergely/cctbx/modules:/home/gergely/cctbx/modules/cctbx_project/boost_adaptbx:/home/gergely/cctbx/build/lib:/home/gergely/cctbx/conda_base/lib/python3.6/site-packages:$PYTHONPATH > > >>> >> >> setenv LD_LIBRARY_PATH > > >>> >> >> > /home/gergely/cctbx/conda_base/lib:/home/gergely/cctbx/build/lib:$LD_LIBRARY_PATH > > >>> >> >> > > >>> >> >> There is probably a better way to put this into conda > environment. > > >>> >> >> With these steps I could run one of my scripts depending on > cctbx. The > > >>> >> >> only problem I found is that hdf5 library had conflict with > another > > >>> >> >> package in conda, but I does not find this as a showstopper. > > >>> >> >> > > >>> >> >> Best wishes, > > >>> >> >> > > >>> >> >> Gergely > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> On Fri, Dec 13, 2019 at 7:23 PM Billy Poon <BKPoon(a)lbl.gov> > wrote: > > >>> >> >> > > > >>> >> >> > Hi Gergely, > > >>> >> >> > > > >>> >> >> > It's still a work in progress. I'm sorting out some Windows > issues right now. I can probably build a test package on a separate channel > for people that want to test it (let's say next week?). I'll provide > instructions, but basically, the test conda package will be in its own > separate channel and the dependencies will be pulled from the conda-forge > channel. I want most things to be working correctly on Python 2.7, 3.6, > 3.7, and 3.8 on all 3 platforms. > > >>> >> >> > > > >>> >> >> > Thanks! > > >>> >> >> > > > >>> >> >> > -- > > >>> >> >> > Billy K. Poon > > >>> >> >> > Research Scientist, Molecular Biophysics and Integrated > Bioimaging > > >>> >> >> > Lawrence Berkeley National Laboratory > > >>> >> >> > 1 Cyclotron Road, M/S 33R0345 > > >>> >> >> > Berkeley, CA 94720 > > >>> >> >> > Tel: (510) 486-5709 > > >>> >> >> > Fax: (510) 486-5909 > > >>> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > >>> >> >> > > > >>> >> >> > > > >>> >> >> > On Fri, Dec 13, 2019 at 5:44 AM Gergely Katona < > gkatona(a)gmail.com> wrote: > > >>> >> >> >> > > >>> >> >> >> Dear Billy, > > >>> >> >> >> > > >>> >> >> >> This sounds very promising and exciting. I am not sure if > cctbx is > > >>> >> >> >> already functional as a conda package in anaconda3 (Linux) > or this is > > >>> >> >> >> still work in progress. My technical expertise does not > allow me to > > >>> >> >> >> tell the difference. What I tried: > > >>> >> >> >> > > >>> >> >> >> Fresh install of anaconda3. Adding - cctbx and - > conda-forge to > > >>> >> >> >> .condarc . Installing pyside2 with conda. Running conda > install > > >>> >> >> >> conda_dependencies . I get a lot package version conflict, > and I > > >>> >> >> >> cannot import cctbx or iotbx to anaconda python. Am I > following the > > >>> >> >> >> right instructions? Or it is too early to expect that cctbx > works when > > >>> >> >> >> installed through conda? > > >>> >> >> >> > > >>> >> >> >> Best wishes, > > >>> >> >> >> > > >>> >> >> >> Gergely > > >>> >> >> >> > > >>> >> >> >> On Wed, Nov 27, 2019 at 3:56 PM Billy Poon <BKPoon(a)lbl.gov> > wrote: > > >>> >> >> >> > > > >>> >> >> >> > Hi all, > > >>> >> >> >> > > > >>> >> >> >> > For a brief update, I have submitted a recipe for > cctbxlite to conda-forge ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_conda-2Dfor… ) and support for > Python 3.7 and 3.8 is being added ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… ). With some fixes for > Windows (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… ), all platforms > (macOS, linux, and Windows) can build for Python 2.7, 3.6, 3.7, and 3.8. > Some additional changes will be needed to get Windows to work with Python 3 > and for tests to pass with Boost 1.70.0. That will enable the conda-forge > recipe to build for all platforms and for all supported versions of Python. > > >>> >> >> >> > > > >>> >> >> >> > Currently, the conda-forge recipe will install into the > conda python and cctbx imports can be done without sourcing the environment > scripts. It looks like a lot of the environment variables being set in the > dispatchers can be removed since the Python files and C++ extensions are in > the right places. I'll update the libtbx_env file so that commands that > load the environment can work correctly. > > >>> >> >> >> > > > >>> >> >> >> > -- > > >>> >> >> >> > Billy K. Poon > > >>> >> >> >> > Research Scientist, Molecular Biophysics and Integrated > Bioimaging > > >>> >> >> >> > Lawrence Berkeley National Laboratory > > >>> >> >> >> > 1 Cyclotron Road, M/S 33R0345 > > >>> >> >> >> > Berkeley, CA 94720 > > >>> >> >> >> > Tel: (510) 486-5709 > > >>> >> >> >> > Fax: (510) 486-5909 > > >>> >> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > >>> >> >> >> > > > >>> >> >> >> > > > >>> >> >> >> > On Sun, Aug 25, 2019 at 2:33 PM Tristan Croll < > tic20(a)cam.ac.uk> wrote: > > >>> >> >> >> >> > > >>> >> >> >> >> Hi Luc, > > >>> >> >> >> >> > > >>> >> >> >> >> That sounds promising. From there, I?d need to work out > how to make a fully-packaged installer (basically a modified wheel file) > for the ChimeraX ToolShed - the aim is for the end user to not have to > worry about any of this. That adds a couple of complications - e.g. > $LIBTBX_BUILD would need to be set dynamically before first import - but > doesn?t seem insurmountable. > > >>> >> >> >> >> > > >>> >> >> >> >> Thanks, > > >>> >> >> >> >> > > >>> >> >> >> >> Tristan > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> Tristan Croll > > >>> >> >> >> >> Research Fellow > > >>> >> >> >> >> Cambridge Institute for Medical Research > > >>> >> >> >> >> University of Cambridge CB2 0XY > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> > On 25 Aug 2019, at 18:31, Luc Bourhis < > luc_j_bourhis(a)mac.com> wrote: > > >>> >> >> >> >> > > > >>> >> >> >> >> > Hi Tristan, > > >>> >> >> >> >> > > > >>> >> >> >> >> > cctbx could be built to use your ChimeraX python, now > that cctbx is moving to Python 3. The option ?with-python is there for that > with the bootstrap script. The specific environment setup boil down to > setting two environment variable LIBTBX_BUILD and either LD_LIBRARY_PATH on > Linux, PATH on Win32, or DYLIB_LIBRARY_PATH on MacOS. If you work within a > framework such as ChimeraX, that should not be difficult to ensure those > two variables are set. > > >>> >> >> >> >> > > > >>> >> >> >> >> > Best wishes, > > >>> >> >> >> >> > > > >>> >> >> >> >> > Luc > > >>> >> >> >> >> > > > >>> >> >> >> >> > > > >>> >> >> >> >> >> On 23 Aug 2019, at 19:02, Tristan Croll < > tic20(a)cam.ac.uk> wrote: > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> To add my two cents on this: probably the second-most > common question I've had about ISOLDE's implementation is, "why didn't you > use CCTBX?". The honest answer to that is, "I didn't know how." > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> Still don't, really - although the current > developments are rather promising. The problem I've faced is that CCTBX was > designed as its own self-contained Python (2.7, until very recently) > environment, with its own interpreter and a lot of very specific > environment setup. Meanwhile I'm developing ISOLDE in ChimeraX, which is > *also* its own self-contained Python (3.7) environment. To plug one into > the other in that form... well, I don't think I'm a good enough programmer > to really know where to start. > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> The move to Conda and a more modular CCTBX > architecture should make a lot more possible in that direction. Pip would > be even better for me personally (ChimeraX can install directly from the > PyPI, but doesn't interact with Conda) - but I understand pretty well the > substantial challenge that would amount to (not least being that the PyPI > imposes a limit - around 100MB from memory? - on the size of an individual > package). > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> Best regards, > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> Tristan > > >>> >> >> >> >> >> > > >>> >> >> >> >> >>> On 2019-08-23 09:28, Luc Bourhis wrote: > > >>> >> >> >> >> >>> Hi Graeme, > > >>> >> >> >> >> >>> Yes, I know. But ?black" is a program doing a very > particular task > > >>> >> >> >> >> >>> (code formatting from the top of my head). Requiring > to use a wrapper > > >>> >> >> >> >> >>> for python itself is another level. But ok, I think > I am mellowing to > > >>> >> >> >> >> >>> the idea after all! Talking with people around me, > and extrapolating, > > >>> >> >> >> >> >>> I would bet that, right now, a great majority of > people interested by > > >>> >> >> >> >> >>> cctbx in pip have already used the cctbx, so they > know about the > > >>> >> >> >> >> >>> Python wrapper, and they would not be too sanguine > about that. My > > >>> >> >> >> >> >>> concern is for the future, when pip will be the > first time some people > > >>> >> >> >> >> >>> use cctbx. Big fat warning notices on PyPI page and > a better error > > >>> >> >> >> >> >>> message when cctbx fails because LIBTBX_BUILD is not > set would be > > >>> >> >> >> >> >>> needed but that could be all right. > > >>> >> >> >> >> >>> If we do a pip installer, we should aim at a minimal > install: cctbx, > > >>> >> >> >> >> >>> iotbx and their dependencies, and that?s it. > > >>> >> >> >> >> >>> Best wishes, > > >>> >> >> >> >> >>> Luc > > >>> >> >> >> >> >>>> On 23 Aug 2019, at 07:17, > Graeme.Winter(a)Diamond.ac.uk <Graeme.Winter(a)diamond.ac.uk> wrote: > > >>> >> >> >> >> >>>> Without discussing the merits of this or whether we > _choose_ to make the move to supporting PIP, I am certain it would be > _possible_ - many other packages make dispatcher scripts when you pip > install them e.g. > > >>> >> >> >> >> >>>> Silver-Surfer rescale_f2 :) $ which black; cat > $(which black) > > >>> >> >> >> >> >>>> > /Library/Frameworks/Python.framework/Versions/3.6/bin/black > > >>> >> >> >> >> >>>> > #!/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 > > >>> >> >> >> >> >>>> # -*- coding: utf-8 -*- > > >>> >> >> >> >> >>>> import re > > >>> >> >> >> >> >>>> import sys > > >>> >> >> >> >> >>>> from black import main > > >>> >> >> >> >> >>>> if __name__ == '__main__': > > >>> >> >> >> >> >>>> sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', > '', sys.argv[0]) > > >>> >> >> >> >> >>>> sys.exit(main()) > > >>> >> >> >> >> >>>> So we _could_ work around the absence of > LIBTBX_BUILD etc. in the system. Whether or not we elect to do the work is > a different question, and it seems clear that here are very mixed opinions > on this. > > >>> >> >> >> >> >>>> Best wishes Graeme > > >>> >> >> >> >> >>>> On 23 Aug 2019, at 01:21, Luc Bourhis < > luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: > > >>> >> >> >> >> >>>> Hi, > > >>> >> >> >> >> >>>> Even if we managed to ship our the boost dynamic > libraries with pip, it would still not be pip-like, as we would still need > our python wrappers to set LIBTBX_BUILD and LD_LIBRARY_PATH. Normal pip > packages work with the standard python exe. LD_LIBRARY_PATH, we could get > around that by changing the way we compile, using -Wl,-R, which is the > runtime equivalent of build time -L. That?s a significant change that would > need to be tested. But there is no way around setting LIBTBX_BUILD right > now. Leaving that to the user is horrible. Perhaps there is a way to hack > libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&… so that we can hardwire LIBTBX_BUILD in there when pip > installs? > > >>> >> >> >> >> >>>> Best wishes, > > >>> >> >> >> >> >>>> Luc > > >>> >> >> >> >> >>>> On 16 Aug 2019, at 22:47, Luc Bourhis < > luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: > > >>> >> >> >> >> >>>> Hi, > > >>> >> >> >> >> >>>> I did look into that many years ago, and even toyed > with building a pip installer. What stopped me is the exact conclusion you > reached too: the user would not have the pip experience he expects. You are > right that it is a lot of effort but is it worth it? Considering that > remark, I don?t think so. Now, Conda was created specifically to go beyond > pip pure-python-only support. Since cctbx has garnered support for Conda, > the best avenue imho is to go the extra length to have a package on > https://urldefense.proofpoint.com/v2/url?u=http-3A__Anaconda.org&d=DwICAg&c… <https://urldefense.proofpoint.com/v2/url?u=http-3A__anaconda.org_&d=DwICAg&… >, and then to advertise it hard to > every potential user out there. > > >>> >> >> >> >> >>>> Best wishes, > > >>> >> >> >> >> >>>> Luc > > >>> >> >> >> >> >>>> On 16 Aug 2019, at 21:45, Aaron Brewster < > asbrewster(a)lbl.gov<mailto:[email protected]>> wrote: > > >>> >> >> >> >> >>>> Hi, to avoid clouding Dorothee's documentation > email thread, which I think is a highly useful enterprise, here's some > thoughts about putting cctbx into pip. Pip doesn't install non-python > dependencies well. I don't think boost is available as a package on pip > (at least the package version we use). wxPython4 isn't portable through > pip ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.wxpython.org_How-… ). > MPI libraries are system dependent. If cctbx were a pure python package, > pip would be fine, but cctbx is not. > > >>> >> >> >> >> >>>> All that said, we could build a manylinux1 version > of cctbx and upload it to PyPi (I'm just learning about this). For a pip > package to be portable (which is a requirement for cctbx), it needs to > conform to PEP513, the manylinux1 standard ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… ). For example, numpy is built > according to this standard (see https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.org_project_numpy… , > where you'll see the manylinux1 wheel). Note, the manylinux1 standard is > built with Centos 5.11 which we no longer support. > > >>> >> >> >> >> >>>> There is also a manylinux2010 standard, which is > based on Centos 6 (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… ). This is > likely a more attainable target (note though by default C++11 is not > supported on Centos 6). > > >>> >> >> >> >> >>>> If we built a manylinuxX version of cctbx and > uploaded it to PyPi, the user would need all the non-python dependencies. > There's no way to specify these in pip. For example, cctbx requires boost > 1.63 or better. The user will need to have it in a place their python can > find it, or we could package it ourselves and supply it, similar to how the > pip h5py package now comes with an hd5f library, or how the pip numpy > package includes an openblas library. We'd have to do the same for any > packages we depend on that aren't on pip using the manylinux standards, > such as wxPython4. > > >>> >> >> >> >> >>>> Further, we need to think about how dials and other > cctbx-based packages interact. If pip install cctbx is set up, how does > pip install dials work, such that any dials shared libraries can find the > cctbx libraries? Can shared libraries from one pip package link against > libraries in another pip package? Would each package need to supply its > own boost? Possibly this is well understood in the pip field, but not by > me :) > > >>> >> >> >> >> >>>> Finally, there's the option of providing a source > pip package. This would require the full compiler toolchain for any given > platform (macOS, linux, windows). These are likely available for > developers, but not for general users. > > >>> >> >> >> >> >>>> Anyway, these are some of the obstacles. Not > saying it isn't possible, it's just a lot of effort. > > >>> >> >> >> >> >>>> Thanks, > > >>> >> >> >> >> >>>> -Aaron > > >>> >> >> >> >> >>>> _______________________________________________ > > >>> >> >> >> >> >>>> cctbxbb mailing list > > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: > cctbxbb(a)phenix-online.org> > > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> >>>> _______________________________________________ > > >>> >> >> >> >> >>>> cctbxbb mailing list > > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: > cctbxbb(a)phenix-online.org> > > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> >>>> _______________________________________________ > > >>> >> >> >> >> >>>> cctbxbb mailing list > > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: > cctbxbb(a)phenix-online.org> > > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> >>>> -- > > >>> >> >> >> >> >>>> This e-mail and any attachments may contain > confidential, copyright and or privileged material, and are for the use of > the intended addressee only. If you are not the intended addressee or an > authorised recipient of the addressee please notify us of receipt by > returning the e-mail and do not use, copy, retain, distribute or disclose > the information in or attached to the e-mail. > > >>> >> >> >> >> >>>> Any opinions expressed within this e-mail are those > of the individual and not necessarily of Diamond Light Source Ltd. > > >>> >> >> >> >> >>>> Diamond Light Source Ltd. cannot guarantee that > this e-mail or any attachments are free from viruses and we cannot accept > liability for any damage which you may sustain as a result of software > viruses which may be transmitted in or with the message. > > >>> >> >> >> >> >>>> Diamond Light Source Limited (company no. 4375679). > Registered in England and Wales with its registered office at Diamond > House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 > 0DE, United Kingdom > > >>> >> >> >> >> >>>> _______________________________________________ > > >>> >> >> >> >> >>>> cctbxbb mailing list > > >>> >> >> >> >> >>>> cctbxbb(a)phenix-online.org > > >>> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> >>> _______________________________________________ > > >>> >> >> >> >> >>> cctbxbb mailing list > > >>> >> >> >> >> >>> cctbxbb(a)phenix-online.org > > >>> >> >> >> >> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> > > >>> >> >> >> >> >> _______________________________________________ > > >>> >> >> >> >> >> cctbxbb mailing list > > >>> >> >> >> >> >> cctbxbb(a)phenix-online.org > > >>> >> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> > > > >>> >> >> >> >> > > > >>> >> >> >> >> > _______________________________________________ > > >>> >> >> >> >> > cctbxbb mailing list > > >>> >> >> >> >> > cctbxbb(a)phenix-online.org > > >>> >> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> >> > > >>> >> >> >> >> > > >>> >> >> >> >> _______________________________________________ > > >>> >> >> >> >> cctbxbb mailing list > > >>> >> >> >> >> cctbxbb(a)phenix-online.org > > >>> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> > > > >>> >> >> >> > _______________________________________________ > > >>> >> >> >> > cctbxbb mailing list > > >>> >> >> >> > cctbxbb(a)phenix-online.org > > >>> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> > > >>> >> >> >> -- > > >>> >> >> >> Gergely Katona, PhD > > >>> >> >> >> Associate Professor > > >>> >> >> >> Department of Chemistry and Molecular Biology, University > of Gothenburg > > >>> >> >> >> Box 462, 40530 G?teborg, Sweden > > >>> >> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: > +46-31-786-3910 > > >>> >> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > >>> >> >> >> > > >>> >> >> >> _______________________________________________ > > >>> >> >> >> cctbxbb mailing list > > >>> >> >> >> cctbxbb(a)phenix-online.org > > >>> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> > > > >>> >> >> > _______________________________________________ > > >>> >> >> > cctbxbb mailing list > > >>> >> >> > cctbxbb(a)phenix-online.org > > >>> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> -- > > >>> >> >> Gergely Katona, PhD > > >>> >> >> Associate Professor > > >>> >> >> Department of Chemistry and Molecular Biology, University of > Gothenburg > > >>> >> >> Box 462, 40530 G?teborg, Sweden > > >>> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: > +46-31-786-3910 > > >>> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > >>> >> >> > > >>> >> >> _______________________________________________ > > >>> >> >> cctbxbb mailing list > > >>> >> >> cctbxbb(a)phenix-online.org > > >>> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> > > > >>> >> > _______________________________________________ > > >>> >> > cctbxbb mailing list > > >>> >> > cctbxbb(a)phenix-online.org > > >>> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> >> > > >>> >> > > >>> >> > > >>> >> -- > > >>> >> Gergely Katona, PhD > > >>> >> Associate Professor > > >>> >> Department of Chemistry and Molecular Biology, University of > Gothenburg > > >>> >> Box 462, 40530 G?teborg, Sweden > > >>> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 > > >>> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > >>> >> > > >>> >> _______________________________________________ > > >>> >> cctbxbb mailing list > > >>> >> cctbxbb(a)phenix-online.org > > >>> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> > > > >>> > _______________________________________________ > > >>> > cctbxbb mailing list > > >>> > cctbxbb(a)phenix-online.org > > >>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > >>> > > >>> > > >>> > > >>> -- > > >>> Gergely Katona, PhD > > >>> Associate Professor > > >>> Department of Chemistry and Molecular Biology, University of > Gothenburg > > >>> Box 462, 40530 G?teborg, Sweden > > >>> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 > > >>> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > >>> > > >>> _______________________________________________ > > >>> cctbxbb mailing list > > >>> cctbxbb(a)phenix-online.org > > >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > > > > > _______________________________________________ > > > cctbxbb mailing list > > > cctbxbb(a)phenix-online.org > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… > > > > > > > > -- > > Gergely Katona, Professor > > Department of Chemistry and Molecular Biology, University of Gothenburg > > Box 462, 40530 G?teborg, Sweden > > Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 > > Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > > > -- > Gergely Katona, Professor > Department of Chemistry and Molecular Biology, University of Gothenburg > Box 462, 40530 G?teborg, Sweden > Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 > Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >

2 1

Re: [cctbxbb] [EXTERNAL] cctbxbb Digest, Vol 129, Issue 6
by Mooers, Blaine H.M. (HSC) 21 Dec '19

21 Dec '19

By following Billy's directions and using new envs, I was able to install cctbx with conda on Mac OS 15.2 with Python 3.6, 3.7, and 3.8. I was able to run some simple scripts. I ran the test that Billy suggested and also got 30 errors. Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular Biology College of Medicine University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center (BRC) Rm. 466 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Friday, December 20, 2019 2:00 PM To: cctbxbb(a)phenix-online.org Subject: [EXTERNAL] cctbxbb Digest, Vol 129, Issue 6 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. Re: some thoughts on cctbx and pip (Billy Poon) ---------------------------------------------------------------------- Message: 1 Date: Fri, 20 Dec 2019 11:15:35 -0800 From: Billy Poon <BKPoon(a)lbl.gov> To: cctbx mailing list <cctbxbb(a)phenix-online.org> Subject: Re: [cctbxbb] some thoughts on cctbx and pip Message-ID: <CAHmYUcYdeXspPwWW2fOOBg0p734p9XYKqS-LiXXqyU-ydBu4pw(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi everyone, I updated the conda packages on the cctbx-dev channel and there are also now packages for macOS. The old packages have been removed so you can create a new environment with conda create -n test -c cctbx-dev cctbx python=2.7 conda activate test where "test" is the environment name and 2.7 is the python version. For the python version, you can also pick 3.6, 3.7, or 3.8. I think the latest conda will install 3.8 as the default python if the python argument is not provided. Or install into an existing environment (with python already installed) with the same command as before, conda install -c cctbx-dev cctbx Again, your .condarc file should have conda-forge as the first channel so that dependencies can be pulled correctly. The dispatchers should work now and $LIBTBX_BUILD will be set on activation of an environment (and unset on environment deactivation) or after installing into an existing environment, so you can do more interesting things like iotbx.fetch_pdb --all --mtz 1kp8 However, this package does not install databases, so molprobity.molprobity 1kp8.pdb 1kp8.mtz will fail because the database is not available (also probe and reduce are not built). Some known issues in no particular order, 1) You can run "libtbx.run_tests_parallel module=cctbx nproc=Auto" to run tests, but not all the test files were copied, so I got 30 failures here. The final package will probably not have the test files. 2) The other python dispatchers, like "cctbx.python", were not updated, so those will fail. But you can just run "python" in the active environment. 3) Parts of mmtbx still need to be updated for Python 3. 4) The LIBTBX_BUILD variable should be set for sh and csh shells. Let me know if it's not working for your shell (i.e. "echo $LIBTBX_BUILD" should show the same thing as $CONDA_PREFIX). 5) One future step is to enable building other CCTBX-like modules with this conda package. This will require some bookkeeping updates so that modules in the conda environment and the modules being built are tracked properly. Also, making sure that all the necessary headers exist in $PREFIX/include. Gergely, let me know if your scripts are working. Happy holidays! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… On Wed, Dec 18, 2019 at 11:25 AM Billy Poon <BKPoon(a)lbl.gov> wrote: > Hi Gergely, > > That's the other information that hasn't been updated yet in the conda > package. :) > > For LIBTBX_BUILD, I can set up environment variables sort of how the conda > gcc package sets up environment variables. I have to double check to see if > they're set after installation of the package and after activation of an > environment. The variable will just be a copy of $CONDA_PREFIX. > > The libtbx_env file is a file that exists in $LIBTBX_BUILD that basically > stores configuration information. I'll have to modify the contents to > update the locations of the different modules since the conda package will > not have a "modules" directory. > > -- > Billy K. Poon > Research Scientist, Molecular Biophysics and Integrated Bioimaging > Lawrence Berkeley National Laboratory > 1 Cyclotron Road, M/S 33R0345 > Berkeley, CA 94720 > Tel: (510) 486-5709 > Fax: (510) 486-5909 > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… > > > On Wed, Dec 18, 2019 at 2:08 AM Gergely Katona <gkatona(a)gmail.com> wrote: > >> Hi Billy, >> >> Conda install went fine with your instructions after rearranging the >> channels by putting cctbx last. I removed all environmental variables >> and previous build of cctbx. >> >> import sys >> print (sys.path, sys.prefix) >> >> ['/home/gergely/anaconda3/lib/https://urldefense.proofpoint.com/v2/url?u=http-3A__python36.zip&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ST4siK2y7PNpS4anr425eXK4FO6ig7oA-XNHoaHgCpY&e= ', >> '/home/gergely/anaconda3/lib/python3.6', >> '/home/gergely/anaconda3/lib/python3.6/lib-dynload', '', >> '/home/gergely/anaconda3/lib/python3.6/site-packages', >> '/home/gergely/anaconda3/lib/python3.6/site-packages/IPython/extensions', >> '/home/gergely/.ipython'] /home/gergely/anaconda3 >> >> Many imports went fine including pymc3, but I encountered problems >> with these three: >> >> from cctbx import miller >> import iotbx.pdb >> from iotbx import reflection_file_reader, mtz >> >> >> I also got type error when handling space groups. >> >> >> >> >> --------------------------------------------------------------------------- >> KeyError Traceback (most recent call >> last) >> <ipython-input-5-cedbd4f84f3d> in <module> >> 25 from cctbx import uctbx >> 26 from cctbx import sgtbx >> ---> 27 from cctbx import miller >> 28 #from iotbx import reflection_file_reader, mtz >> 29 sns.set_context("poster") >> >> ~/anaconda3/lib/python3.6/site-packages/cctbx/miller/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ou8J8HxatN0TDK1PJ-GUgX9YcOgS5Fayhbs61Wt2f7g&e= in >> <module> >> 11 >> 12 from cctbx import crystal >> ---> 13 from cctbx import maptbx >> 14 from cctbx import sgtbx >> 15 from cctbx.sgtbx import lattice_symmetry >> >> ~/anaconda3/lib/python3.6/site-packages/cctbx/maptbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ou8J8HxatN0TDK1PJ-GUgX9YcOgS5Fayhbs61Wt2f7g&e= in >> <module> >> 15 from libtbx import adopt_init_args >> 16 from libtbx.utils import Sorry >> ---> 17 import libtbx.load_env >> 18 import math >> 19 import sys, os >> >> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=dUMVNgDtesLENoxBTvPr4XX4mczuZIsmZzdOmPlGbtI&e= in <module> >> 3 import libtbx.env_config >> 4 import os >> ----> 5 libtbx.env = libtbx.env_config.unpickle() >> 6 libtbx.env.set_os_environ_all_dist() >> 7 libtbx.env.dispatcher_name = >> os.environ.get("LIBTBX_DISPATCHER_NAME") >> >> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=2BeTV_6Dw8q_ulzvyRBt0YhebeaqhUvm-em6GAOEaI4&e= in unpickle() >> 2603 >> 2604 def unpickle(): >> -> 2605 build_path = os.environ["LIBTBX_BUILD"] >> 2606 >> set_preferred_sys_prefix_and_sys_executable(build_path=build_path) >> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") >> >> ~/anaconda3/lib/python3.6/https://urldefense.proofpoint.com/v2/url?u=http-3A__os.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=y6dOvVLcNHYw8pJU1uwOEPGTH8lLyIp-mgtyv_X2bFQ&e= in __getitem__(self, key) >> 667 except KeyError: >> 668 # raise KeyError with the original key value >> --> 669 raise KeyError(key) from None >> 670 return self.decodevalue(value) >> 671 >> >> KeyError: 'LIBTBX_BUILD' >> >> >> >> >> --------------------------------------------------------------------------- >> ArgumentError Traceback (most recent call >> last) >> <ipython-input-7-2acd3a9ce26a> in <module> >> 26 return >> ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar >> 27 >> ---> 28 >> ms,msnam,mscent,msacent,msnamacent,msnamcent,msnamacent_dstar=initializecrystal() >> >> <ipython-input-7-2acd3a9ce26a> in initializecrystal() >> 11 uc = uctbx.unit_cell(unit_cell) >> 12 wavelength = 1.54980 >> ---> 13 xtal_symm = crystal.symmetry(unit_cell=unit_cell, >> space_group_symbol="P 43 21 2") >> 14 >> 15 ms = >> miller.build_set(crystal_symmetry=xtal_symm,anomalous_flag=True, >> d_min=1.61) >> >> ~/anaconda3/lib/python3.6/site-packages/cctbx/crystal/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ou8J8HxatN0TDK1PJ-GUgX9YcOgS5Fayhbs61Wt2f7g&e= in >> __init__(self, unit_cell, space_group_symbol, space_group_info, >> space_group, correct_rhombohedral_setting_if_necessary, >> assert_is_compatible_unit_cell, raise_sorry_if_incompatible_unit_cell, >> force_compatible_unit_cell) >> 74 if (space_group_symbol is not None): >> 75 self._space_group_info = sgtbx.space_group_info( >> ---> 76 symbol=space_group_symbol) >> 77 elif (space_group is not None): >> 78 if (isinstance(space_group, sgtbx.space_group)): >> >> ~/anaconda3/lib/python3.6/site-packages/cctbx/sgtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ou8J8HxatN0TDK1PJ-GUgX9YcOgS5Fayhbs61Wt2f7g&e= in >> __init__(self, symbol, table_id, group, number, space_group_t_den) >> 100 assert group is None >> 101 if (table_id is None): >> --> 102 symbols = space_group_symbols(symbol) >> 103 else: >> 104 if (isinstance(symbol, int)): symbol = str(symbol) >> >> ArgumentError: Python argument types in >> space_group_symbols.__init__(space_group_symbols, str) >> did not match C++ signature: >> __init__(_object*, int space_group_number) >> __init__(_object*, int space_group_number, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > extension='') >> __init__(_object*, int space_group_number, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > extension='', std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > table_id='') >> __init__(_object*, std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > symbol) >> __init__(_object*, std::__cxx11::basic_string<char, >> std::char_traits<char>, std::allocator<char> > symbol, >> std::__cxx11::basic_string<char, std::char_traits<char>, >> std::allocator<char> > table_id='') >> >> >> When I tried to set LIBTBX_BUILD to /home/gergely/anaconda3 (this is >> sys.prefix on my system), I got other problems when importing. Can >> LIBTBX_BUILD be set in the conda package? >> >> >> >> >> --------------------------------------------------------------------------- >> FileNotFoundError Traceback (most recent call >> last) >> <ipython-input-1-7e2b144826cc> in <module> >> 26 from cctbx import sgtbx >> 27 #from cctbx import miller >> ---> 28 from iotbx import reflection_file_reader, mtz >> 29 sns.set_context("poster") >> 30 plt.rcParams.update({'figure.autolayout': True}) >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__reflection-5Ffile-5Freader.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=yYMogebnDLYk0-5LYB00qDeyEYddmSEzic2HY01p1-4&e= >> in <module> >> 53 >> 54 from __future__ import absolute_import, division, print_function >> ---> 55 from iotbx import mtz >> 56 from iotbx.scalepack import merge as scalepack_merge >> 57 from iotbx.scalepack import no_merge_original_index as >> scalepack_no_merge >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__-5F-5Finit-5F-5F.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=ou8J8HxatN0TDK1PJ-GUgX9YcOgS5Fayhbs61Wt2f7g&e= in <module> >> 9 import iotbx_mtz_ext as ext >> 10 >> ---> 11 from iotbx.mtz import extract_from_symmetry_lib >> 12 from cctbx import xray >> 13 import cctbx.xray.observation_types >> >> >> ~/anaconda3/lib/python3.6/site-packages/iotbx/mtz/https://urldefense.proofpoint.com/v2/url?u=http-3A__extract-5Ffrom-5Fsymmetry-5Flib.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=sdo4JC4HEfj-rC7w_ahWQIcYyaTeKakuavIvEeWWbf8&e= >> in <module> >> 1 from __future__ import absolute_import, division, print_function >> 2 from cctbx import sgtbx >> ----> 3 import libtbx.load_env >> 4 import os.path as op >> 5 from six.moves import range >> >> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__load-5Fenv.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=dUMVNgDtesLENoxBTvPr4XX4mczuZIsmZzdOmPlGbtI&e= in <module> >> 3 import libtbx.env_config >> 4 import os >> ----> 5 libtbx.env = libtbx.env_config.unpickle() >> 6 libtbx.env.set_os_environ_all_dist() >> 7 libtbx.env.dispatcher_name = >> os.environ.get("LIBTBX_DISPATCHER_NAME") >> >> ~/anaconda3/lib/python3.6/site-packages/libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&d=DwICAg&c=VjzId-SM5S6aVB_cCGQ0d3uo9UfKByQ3sI6Audoy6dY&r=k0gMbcsdOcdbPUNV5tW66KQSZfXL0ewVDPVBp7tqbks&m=XYalX3Fy1IGs4p_M1YHMvPXXo6HJximT9mf_zMIckxU&s=2BeTV_6Dw8q_ulzvyRBt0YhebeaqhUvm-em6GAOEaI4&e= in unpickle() >> 2605 build_path = os.environ["LIBTBX_BUILD"] >> 2606 >> set_preferred_sys_prefix_and_sys_executable(build_path=build_path) >> -> 2607 libtbx_env = open(op.join(build_path, "libtbx_env"), "rb") >> 2608 env = pickle.load(libtbx_env) >> 2609 if (env.python_version_major_minor != sys.version_info[:2]): >> >> FileNotFoundError: [Errno 2] No such file or directory: >> '/home/gergely/anaconda3/libtbx_env' >> >> On Wed, Dec 18, 2019 at 9:10 AM Billy Poon <BKPoon(a)lbl.gov> wrote: >> > >> > Hi Gergely, >> > >> > I've uploaded linux packages to a new channel, cctbx-dev, and you can >> install it with >> > >> > conda install -c cctbx-dev cctbx >> > >> > in your current environment. You should set your ~/.condarc file to >> pull the other dependencies from the conda-forge channel first, so put >> conda-forge above cctbx. Mine looks like >> > >> > channels: >> > - conda-forge >> > - defaults >> > - cctbx >> > >> > Do you need dxtbx for your scripts? This package does not build that >> part. I think the plan is to build a separate conda package for dxtbx so >> that it can be updated more frequently. I can rebuild the packages to >> include it for testing, but the one being submitted to conda-forge will not >> have it. >> > >> > Also, your error message is probably due to version of HDF5 that the >> development build installs. The https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… script will install 1.10.4, >> but your other dependency is looking for 1.10.5. Installing this cctbx >> conda package should install 1.10.5, which should fix the issue. Also, I'm >> updating those environments and 1.10.5 will be the new default version. >> > >> > Lastly, the dispatchers will not work in these packages because the old >> paths during the build process are still in them and many of them expect >> some additional information that has not been updated in the packages yet. >> I'm in the process of doing that and will update the cctbx-dev channel when >> that's done. However, by starting python, you can import cctbx modules. So >> you can run commands like >> > >> > from scitbx.array_family import flex >> > a = flex.random_double(1000000) >> > b = flex.min_max_mean_double(a) >> > b.min >> > b.max >> > b.mean >> > >> > Thanks! >> > >> > -- >> > Billy K. Poon >> > Research Scientist, Molecular Biophysics and Integrated Bioimaging >> > Lawrence Berkeley National Laboratory >> > 1 Cyclotron Road, M/S 33R0345 >> > Berkeley, CA 94720 >> > Tel: (510) 486-5709 >> > Fax: (510) 486-5909 >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> > >> > >> > On Tue, Dec 17, 2019 at 5:39 AM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> >> >> >> Dear Billy, >> >> >> >> Thank you for the detailed explanation, I look forward to do further >> >> testing! I aim to pool all modules under the same environment, if >> >> there are conflicts then I just try to reshuffle the order of imports. >> >> So far this did not cause problems for me even when I was using system >> >> python, but of course it is not the most prudent thing to do. With >> >> anaconda everything is much more standardized and isolated already and >> >> without being superuser I can have my familiar environment at any >> >> synchrotron based cluster. It is great that cctbx will be an integral >> >> part of this ecosystem and this was also the last thing holding me >> >> back from adopting python3. About hdf5, pymc3 requires it and >> >> importing causes a kernel restart with the following error messages, >> >> curiously if I import h5py first this can be avoided. >> >> >> >> Best wishes, >> >> >> >> Gergely >> >> >> >> import pymc3 as pm >> >> >> >> Warning! ***HDF5 library version mismatched error*** >> >> The HDF5 header files used to compile this application do not match >> >> the version used by the HDF5 library to which this application is >> linked. >> >> Data corruption or segmentation faults may occur if the application >> continues. >> >> This can happen when an application was compiled by one version of >> HDF5 but >> >> linked with a different version of static or shared HDF5 library. >> >> You should recompile the application or check your shared library >> related >> >> settings such as 'LD_LIBRARY_PATH'. >> >> You can, at your own risk, disable this warning by setting the >> environment >> >> variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'. >> >> Setting it to 2 or higher will suppress the warning messages totally. >> >> Headers are 1.10.4, library is 1.10.5 >> >> SUMMARY OF THE HDF5 CONFIGURATION >> >> ================================= >> >> >> >> General Information: >> >> ------------------- >> >> HDF5 Version: 1.10.5 >> >> Configured on: Tue Oct 22 12:02:13 UTC 2019 >> >> Configured by: conda@16247e67ecd5 >> >> Host system: x86_64-conda_cos6-linux-gnu >> >> Uname information: Linux 16247e67ecd5 4.15.0-1059-azure >> >> #64-Ubuntu SMP Fri Sep 13 17:02:44 UTC 2019 x86_64 x86_64 x86_64 >> >> GNU/Linux >> >> Byte sex: little-endian >> >> Installation point: /home/gergely/anaconda3 >> >> >> >> Compiling Options: >> >> ------------------ >> >> Build Mode: production >> >> Debugging Symbols: no >> >> Asserts: no >> >> Profiling: no >> >> Optimization Level: high >> >> >> >> Linking Options: >> >> ---------------- >> >> Libraries: static, shared >> >> Statically Linked Executables: >> >> LDFLAGS: -Wl,-O2 -Wl,--sort-common >> >> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags >> >> -Wl,--gc-sections -Wl,-rpath,/home/gergely/anaconda3/lib >> >> -Wl,-rpath-link,/home/gergely/anaconda3/lib >> >> -L/home/gergely/anaconda3/lib >> >> H5_LDFLAGS: >> >> AM_LDFLAGS: -L/home/gergely/anaconda3/lib >> >> Extra libraries: -lrt -lpthread -lz -ldl -lm >> >> Archiver: >> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar >> >> AR_FLAGS: cr >> >> Ranlib: >> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib >> >> >> >> Languages: >> >> ---------- >> >> C: yes >> >> C Compiler: >> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc >> >> CPPFLAGS: -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 >> >> -I/home/gergely/anaconda3/include >> >> H5_CPPFLAGS: -D_GNU_SOURCE >> >> -D_POSIX_C_SOURCE=200809L -DNDEBUG -UH5_DEBUG_API >> >> AM_CPPFLAGS: -I/home/gergely/anaconda3/include >> >> C Flags: -march=nocona -mtune=haswell >> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 >> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include >> >> >> -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 >> >> -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix >> >> H5 C Flags: -std=c99 -pedantic -Wall -Wextra >> >> -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual -Wconversion >> >> -Wdeclaration-after-statement -Wdisabled-optimization -Wfloat-equal >> >> -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-declarations >> >> -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs >> >> -Wold-style-definition -Wpacked -Wpointer-arith -Wredundant-decls >> >> -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef >> >> -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings >> >> -finline-functions -s -Wno-inline -Wno-aggregate-return >> >> -Wno-missing-format-attribute -Wno-missing-noreturn -O >> >> AM C Flags: >> >> Shared C Library: yes >> >> Static C Library: yes >> >> >> >> >> >> Fortran: yes >> >> Fortran Compiler: >> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-gfortran >> >> Fortran Flags: >> >> H5 Fortran Flags: -pedantic -Wall -Wextra -Wunderflow >> >> -Wimplicit-interface -Wsurprising -Wno-c-binding-type -s -O2 >> >> AM Fortran Flags: >> >> Shared Fortran Library: yes >> >> Static Fortran Library: yes >> >> >> >> C++: yes >> >> C++ Compiler: >> >> >> /home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++ >> >> C++ Flags: -fvisibility-inlines-hidden >> >> -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell >> >> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 >> >> -ffunction-sections -pipe -I/home/gergely/anaconda3/include >> >> >> -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/hdf5_split_1571745596770/work=/usr/local/src/conda/hdf5_split-1.10.5 >> >> -fdebug-prefix-map=/home/gergely/anaconda3=/usr/local/src/conda-prefix >> >> H5 C++ Flags: -pedantic -Wall -W -Wundef -Wshadow >> >> -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion >> >> -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual >> >> -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor >> >> -Wctor-dtor-privacy -Wabi -finline-functions -s -O >> >> AM C++ Flags: >> >> Shared C++ Library: yes >> >> Static C++ Library: yes >> >> >> >> Java: no >> >> >> >> >> >> Features: >> >> --------- >> >> Parallel HDF5: no >> >> Parallel Filtered Dataset Writes: no >> >> Large Parallel I/O: no >> >> High-level library: yes >> >> Threadsafety: yes >> >> Default API mapping: v110 >> >> With deprecated public symbols: yes >> >> I/O filters (external): deflate(zlib) >> >> MPE: no >> >> Direct VFD: no >> >> dmalloc: no >> >> Packages w/ extra debug output: none >> >> API tracing: no >> >> Using memory checker: yes >> >> Memory allocation sanity checks: no >> >> Function stack tracing: no >> >> Strict file format checks: no >> >> Optimization instrumentation: no >> >> >> >> On Tue, Dec 17, 2019 at 8:35 AM Billy Poon <BKPoon(a)lbl.gov> wrote: >> >> > >> >> > Hi Gergely, >> >> > >> >> > Let me build the test package tomorrow. All the gritty details for >> building with conda is being finalized and the official documentation will >> be updated to describe the steps. It would be too confusing to keep >> changing the documentation as the process evolves. You do have the general >> process, though, which is summarized as follows. >> >> > >> >> > 1) Installing dependencies. The cctbx_dependencies metapackage was >> an initial approach for managing the CCTBX dependencies, but after >> contacting the conda-forge folks, they recommended using the --only-deps >> flag. So when the CCTBX conda package is available, you'll be able to get a >> set of dependencies with, >> >> > >> >> > conda install -c conda-forge --only-deps cctbx >> >> > >> >> > By default, the https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file will automatically install a set >> of dependencies in the "conda_base" directory (and a conda installation if >> one is not found). It just uses standard conda environment files located in >> libtbx/auto_build/conda_envs, so you do not need to install >> cctbx_dependencies as a separate step. The environment files avoid channel >> issues by explicitly defining the channel to pull the packages from and the >> cctbx channel just stores copies of packages from conda-forge. There were >> issues earlier where the conda-forge packages would sometimes be moved to a >> different label. The --use-conda flag also accepts a path to $CONDA_PREFIX >> if you want to use a specific environment for testing. >> >> > >> >> > 2) Building. The https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… file handles that with SCons. >> >> > >> >> > 3) Running. After building, there should be a https://urldefense.proofpoint.com/v2/url?u=http-3A__setpaths.sh&d=DwICAg&c=… (and .csh) >> file that adds build/bin to your path. The build/bin directory has our >> dispatchers, which are just scripts that set up the environment variables >> for you. This prevents other programs from loading our libraries, whose >> versions may conflict. You should see that there is a "python" dispatcher, >> which is a convenience for developers. Otherwise, you can use >> libtbx.python, which will be able to import CCTBX modules. >> >> > >> >> > The running part is where the conda package for CCTBX will be >> different than this build. Since our Python files and extensions modules >> will be in the "site-packages" directory for Python, the PYTHONPATH >> variable will not be needed (and conda suggests that that variable not be >> set). The other CCTBX libraries will be in $CONDA_PREFIX/lib, so >> LD_LIBRARY_PATH is not needed. In an active environment, PATH will already >> be modified. And then our LIBTBX_BUILD directory can be set to sys.prefix. >> >> > >> >> > So with the conda package, you would only need to activate your >> conda environment and CCTBX should integrate with other conda packages. >> What is the conflict with hdf5? That's something that should be fixed. >> Thanks! >> >> > >> >> > -- >> >> > Billy K. Poon >> >> > Research Scientist, Molecular Biophysics and Integrated Bioimaging >> >> > Lawrence Berkeley National Laboratory >> >> > 1 Cyclotron Road, M/S 33R0345 >> >> > Berkeley, CA 94720 >> >> > Tel: (510) 486-5709 >> >> > Fax: (510) 486-5909 >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> >> > >> >> > >> >> > On Mon, Dec 16, 2019 at 2:53 AM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> >> >> >> >> >> Dear Billy, >> >> >> >> >> >> Thank you for this update and for your efforts! I found a solution >> and >> >> >> indeed most things already work in anaconda3. The steps I took (even >> >> >> if these instructions will have short expiry date): >> >> >> >> >> >> Modified .condarc with: >> >> >> channels: >> >> >> - cctbx >> >> >> - conda-forge >> >> >> - defaults >> >> >> >> >> >> run >> >> >> conda conda install cctbx_dependencies python=3.6 >> >> >> >> >> >> I expect this will work without python=3.6 in the near future. >> >> >> >> >> >> Then compiling cctbx with anaconda3 python went without any problem >> >> >> when using these flags: >> >> >> python https://urldefense.proofpoint.com/v2/url?u=http-3A__bootstrap.py&d=DwICAg&c… --use-conda --python3 --nproc=12 >> >> >> >> >> >> Then I probably did the unorthodox thing and sourced these >> directories: >> >> >> >> >> >> setenv LIBTBX_BUILD /home/gergely/cctbx/build >> >> >> setenv PATH ${PATH}:/home/gergely/cctbx/build/bin >> >> >> setenv PYTHONPATH >> >> >> >> /home/gergely/cctbx/modules/cctbx_project:/home/gergely/cctbx/modules:/home/gergely/cctbx/modules/cctbx_project/boost_adaptbx:/home/gergely/cctbx/build/lib:/home/gergely/cctbx/conda_base/lib/python3.6/site-packages:$PYTHONPATH >> >> >> setenv LD_LIBRARY_PATH >> >> >> >> /home/gergely/cctbx/conda_base/lib:/home/gergely/cctbx/build/lib:$LD_LIBRARY_PATH >> >> >> >> >> >> There is probably a better way to put this into conda environment. >> >> >> With these steps I could run one of my scripts depending on cctbx. >> The >> >> >> only problem I found is that hdf5 library had conflict with another >> >> >> package in conda, but I does not find this as a showstopper. >> >> >> >> >> >> Best wishes, >> >> >> >> >> >> Gergely >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Fri, Dec 13, 2019 at 7:23 PM Billy Poon <BKPoon(a)lbl.gov> wrote: >> >> >> > >> >> >> > Hi Gergely, >> >> >> > >> >> >> > It's still a work in progress. I'm sorting out some Windows >> issues right now. I can probably build a test package on a separate channel >> for people that want to test it (let's say next week?). I'll provide >> instructions, but basically, the test conda package will be in its own >> separate channel and the dependencies will be pulled from the conda-forge >> channel. I want most things to be working correctly on Python 2.7, 3.6, >> 3.7, and 3.8 on all 3 platforms. >> >> >> > >> >> >> > Thanks! >> >> >> > >> >> >> > -- >> >> >> > Billy K. Poon >> >> >> > Research Scientist, Molecular Biophysics and Integrated Bioimaging >> >> >> > Lawrence Berkeley National Laboratory >> >> >> > 1 Cyclotron Road, M/S 33R0345 >> >> >> > Berkeley, CA 94720 >> >> >> > Tel: (510) 486-5709 >> >> >> > Fax: (510) 486-5909 >> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> >> >> > >> >> >> > >> >> >> > On Fri, Dec 13, 2019 at 5:44 AM Gergely Katona <gkatona(a)gmail.com> >> wrote: >> >> >> >> >> >> >> >> Dear Billy, >> >> >> >> >> >> >> >> This sounds very promising and exciting. I am not sure if cctbx >> is >> >> >> >> already functional as a conda package in anaconda3 (Linux) or >> this is >> >> >> >> still work in progress. My technical expertise does not allow me >> to >> >> >> >> tell the difference. What I tried: >> >> >> >> >> >> >> >> Fresh install of anaconda3. Adding - cctbx and - conda-forge to >> >> >> >> .condarc . Installing pyside2 with conda. Running conda install >> >> >> >> conda_dependencies . I get a lot package version conflict, and I >> >> >> >> cannot import cctbx or iotbx to anaconda python. Am I following >> the >> >> >> >> right instructions? Or it is too early to expect that cctbx >> works when >> >> >> >> installed through conda? >> >> >> >> >> >> >> >> Best wishes, >> >> >> >> >> >> >> >> Gergely >> >> >> >> >> >> >> >> On Wed, Nov 27, 2019 at 3:56 PM Billy Poon <BKPoon(a)lbl.gov> >> wrote: >> >> >> >> > >> >> >> >> > Hi all, >> >> >> >> > >> >> >> >> > For a brief update, I have submitted a recipe for cctbxlite to >> conda-forge (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_conda-2Dfor… ) >> and support for Python 3.7 and 3.8 is being added ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… ). With some fixes for >> Windows (https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_cctbx_cctbx… ), all platforms >> (macOS, linux, and Windows) can build for Python 2.7, 3.6, 3.7, and 3.8. >> Some additional changes will be needed to get Windows to work with Python 3 >> and for tests to pass with Boost 1.70.0. That will enable the conda-forge >> recipe to build for all platforms and for all supported versions of Python. >> >> >> >> > >> >> >> >> > Currently, the conda-forge recipe will install into the conda >> python and cctbx imports can be done without sourcing the environment >> scripts. It looks like a lot of the environment variables being set in the >> dispatchers can be removed since the Python files and C++ extensions are in >> the right places. I'll update the libtbx_env file so that commands that >> load the environment can work correctly. >> >> >> >> > >> >> >> >> > -- >> >> >> >> > Billy K. Poon >> >> >> >> > Research Scientist, Molecular Biophysics and Integrated >> Bioimaging >> >> >> >> > Lawrence Berkeley National Laboratory >> >> >> >> > 1 Cyclotron Road, M/S 33R0345 >> >> >> >> > Berkeley, CA 94720 >> >> >> >> > Tel: (510) 486-5709 >> >> >> >> > Fax: (510) 486-5909 >> >> >> >> > Web: https://urldefense.proofpoint.com/v2/url?u=https-3A__phenix-2Donline.org&d=… >> >> >> >> > >> >> >> >> > >> >> >> >> > On Sun, Aug 25, 2019 at 2:33 PM Tristan Croll <tic20(a)cam.ac.uk> >> wrote: >> >> >> >> >> >> >> >> >> >> Hi Luc, >> >> >> >> >> >> >> >> >> >> That sounds promising. From there, I?d need to work out how >> to make a fully-packaged installer (basically a modified wheel file) for >> the ChimeraX ToolShed - the aim is for the end user to not have to worry >> about any of this. That adds a couple of complications - e.g. $LIBTBX_BUILD >> would need to be set dynamically before first import - but doesn?t seem >> insurmountable. >> >> >> >> >> >> >> >> >> >> Thanks, >> >> >> >> >> >> >> >> >> >> Tristan >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Tristan Croll >> >> >> >> >> Research Fellow >> >> >> >> >> Cambridge Institute for Medical Research >> >> >> >> >> University of Cambridge CB2 0XY >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > On 25 Aug 2019, at 18:31, Luc Bourhis < >> luc_j_bourhis(a)mac.com> wrote: >> >> >> >> >> > >> >> >> >> >> > Hi Tristan, >> >> >> >> >> > >> >> >> >> >> > cctbx could be built to use your ChimeraX python, now that >> cctbx is moving to Python 3. The option ?with-python is there for that with >> the bootstrap script. The specific environment setup boil down to setting >> two environment variable LIBTBX_BUILD and either LD_LIBRARY_PATH on Linux, >> PATH on Win32, or DYLIB_LIBRARY_PATH on MacOS. If you work within a >> framework such as ChimeraX, that should not be difficult to ensure those >> two variables are set. >> >> >> >> >> > >> >> >> >> >> > Best wishes, >> >> >> >> >> > >> >> >> >> >> > Luc >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> >> On 23 Aug 2019, at 19:02, Tristan Croll <tic20(a)cam.ac.uk> >> wrote: >> >> >> >> >> >> >> >> >> >> >> >> To add my two cents on this: probably the second-most >> common question I've had about ISOLDE's implementation is, "why didn't you >> use CCTBX?". The honest answer to that is, "I didn't know how." >> >> >> >> >> >> >> >> >> >> >> >> Still don't, really - although the current developments >> are rather promising. The problem I've faced is that CCTBX was designed as >> its own self-contained Python (2.7, until very recently) environment, with >> its own interpreter and a lot of very specific environment setup. Meanwhile >> I'm developing ISOLDE in ChimeraX, which is *also* its own self-contained >> Python (3.7) environment. To plug one into the other in that form... well, >> I don't think I'm a good enough programmer to really know where to start. >> >> >> >> >> >> >> >> >> >> >> >> The move to Conda and a more modular CCTBX architecture >> should make a lot more possible in that direction. Pip would be even better >> for me personally (ChimeraX can install directly from the PyPI, but doesn't >> interact with Conda) - but I understand pretty well the substantial >> challenge that would amount to (not least being that the PyPI imposes a >> limit - around 100MB from memory? - on the size of an individual package). >> >> >> >> >> >> >> >> >> >> >> >> Best regards, >> >> >> >> >> >> >> >> >> >> >> >> Tristan >> >> >> >> >> >> >> >> >> >> >> >>> On 2019-08-23 09:28, Luc Bourhis wrote: >> >> >> >> >> >>> Hi Graeme, >> >> >> >> >> >>> Yes, I know. But ?black" is a program doing a very >> particular task >> >> >> >> >> >>> (code formatting from the top of my head). Requiring to >> use a wrapper >> >> >> >> >> >>> for python itself is another level. But ok, I think I am >> mellowing to >> >> >> >> >> >>> the idea after all! Talking with people around me, and >> extrapolating, >> >> >> >> >> >>> I would bet that, right now, a great majority of people >> interested by >> >> >> >> >> >>> cctbx in pip have already used the cctbx, so they know >> about the >> >> >> >> >> >>> Python wrapper, and they would not be too sanguine about >> that. My >> >> >> >> >> >>> concern is for the future, when pip will be the first >> time some people >> >> >> >> >> >>> use cctbx. Big fat warning notices on PyPI page and a >> better error >> >> >> >> >> >>> message when cctbx fails because LIBTBX_BUILD is not set >> would be >> >> >> >> >> >>> needed but that could be all right. >> >> >> >> >> >>> If we do a pip installer, we should aim at a minimal >> install: cctbx, >> >> >> >> >> >>> iotbx and their dependencies, and that?s it. >> >> >> >> >> >>> Best wishes, >> >> >> >> >> >>> Luc >> >> >> >> >> >>>> On 23 Aug 2019, at 07:17, Graeme.Winter(a)Diamond.ac.uk < >> Graeme.Winter(a)diamond.ac.uk> wrote: >> >> >> >> >> >>>> Without discussing the merits of this or whether we >> _choose_ to make the move to supporting PIP, I am certain it would be >> _possible_ - many other packages make dispatcher scripts when you pip >> install them e.g. >> >> >> >> >> >>>> Silver-Surfer rescale_f2 :) $ which black; cat $(which >> black) >> >> >> >> >> >>>> >> /Library/Frameworks/Python.framework/Versions/3.6/bin/black >> >> >> >> >> >>>> >> #!/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 >> >> >> >> >> >>>> # -*- coding: utf-8 -*- >> >> >> >> >> >>>> import re >> >> >> >> >> >>>> import sys >> >> >> >> >> >>>> from black import main >> >> >> >> >> >>>> if __name__ == '__main__': >> >> >> >> >> >>>> sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', >> sys.argv[0]) >> >> >> >> >> >>>> sys.exit(main()) >> >> >> >> >> >>>> So we _could_ work around the absence of LIBTBX_BUILD >> etc. in the system. Whether or not we elect to do the work is a different >> question, and it seems clear that here are very mixed opinions on this. >> >> >> >> >> >>>> Best wishes Graeme >> >> >> >> >> >>>> On 23 Aug 2019, at 01:21, Luc Bourhis < >> luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: >> >> >> >> >> >>>> Hi, >> >> >> >> >> >>>> Even if we managed to ship our the boost dynamic >> libraries with pip, it would still not be pip-like, as we would still need >> our python wrappers to set LIBTBX_BUILD and LD_LIBRARY_PATH. Normal pip >> packages work with the standard python exe. LD_LIBRARY_PATH, we could get >> around that by changing the way we compile, using -Wl,-R, which is the >> runtime equivalent of build time -L. That?s a significant change that would >> need to be tested. But there is no way around setting LIBTBX_BUILD right >> now. Leaving that to the user is horrible. Perhaps there is a way to hack >> libtbx/https://urldefense.proofpoint.com/v2/url?u=http-3A__env-5Fconfig.py&… so that we can hardwire LIBTBX_BUILD in there when pip >> installs? >> >> >> >> >> >>>> Best wishes, >> >> >> >> >> >>>> Luc >> >> >> >> >> >>>> On 16 Aug 2019, at 22:47, Luc Bourhis < >> luc_j_bourhis(a)mac.com<mailto:[email protected]>> wrote: >> >> >> >> >> >>>> Hi, >> >> >> >> >> >>>> I did look into that many years ago, and even toyed with >> building a pip installer. What stopped me is the exact conclusion you >> reached too: the user would not have the pip experience he expects. You are >> right that it is a lot of effort but is it worth it? Considering that >> remark, I don?t think so. Now, Conda was created specifically to go beyond >> pip pure-python-only support. Since cctbx has garnered support for Conda, >> the best avenue imho is to go the extra length to have a package on >> https://urldefense.proofpoint.com/v2/url?u=http-3A__Anaconda.org&d=DwICAg&c… <https://urldefense.proofpoint.com/v2/url?u=http-3A__anaconda.org_&d=DwICAg&… >, and then to advertise it hard to >> every potential user out there. >> >> >> >> >> >>>> Best wishes, >> >> >> >> >> >>>> Luc >> >> >> >> >> >>>> On 16 Aug 2019, at 21:45, Aaron Brewster < >> asbrewster(a)lbl.gov<mailto:[email protected]>> wrote: >> >> >> >> >> >>>> Hi, to avoid clouding Dorothee's documentation email >> thread, which I think is a highly useful enterprise, here's some thoughts >> about putting cctbx into pip. Pip doesn't install non-python dependencies >> well. I don't think boost is available as a package on pip (at least the >> package version we use). wxPython4 isn't portable through pip ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.wxpython.org_How-… ). >> MPI libraries are system dependent. If cctbx were a pure python package, >> pip would be fine, but cctbx is not. >> >> >> >> >> >>>> All that said, we could build a manylinux1 version of >> cctbx and upload it to PyPi (I'm just learning about this). For a pip >> package to be portable (which is a requirement for cctbx), it needs to >> conform to PEP513, the manylinux1 standard ( >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… ). For example, numpy is built >> according to this standard (see https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.org_project_numpy… , >> where you'll see the manylinux1 wheel). Note, the manylinux1 standard is >> built with Centos 5.11 which we no longer support. >> >> >> >> >> >>>> There is also a manylinux2010 standard, which is based >> on Centos 6 (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.python.org_dev_pep… ). This is likely >> a more attainable target (note though by default C++11 is not supported on >> Centos 6). >> >> >> >> >> >>>> If we built a manylinuxX version of cctbx and uploaded >> it to PyPi, the user would need all the non-python dependencies. There's >> no way to specify these in pip. For example, cctbx requires boost 1.63 or >> better. The user will need to have it in a place their python can find it, >> or we could package it ourselves and supply it, similar to how the pip h5py >> package now comes with an hd5f library, or how the pip numpy package >> includes an openblas library. We'd have to do the same for any packages we >> depend on that aren't on pip using the manylinux standards, such as >> wxPython4. >> >> >> >> >> >>>> Further, we need to think about how dials and other >> cctbx-based packages interact. If pip install cctbx is set up, how does >> pip install dials work, such that any dials shared libraries can find the >> cctbx libraries? Can shared libraries from one pip package link against >> libraries in another pip package? Would each package need to supply its >> own boost? Possibly this is well understood in the pip field, but not by >> me :) >> >> >> >> >> >>>> Finally, there's the option of providing a source pip >> package. This would require the full compiler toolchain for any given >> platform (macOS, linux, windows). These are likely available for >> developers, but not for general users. >> >> >> >> >> >>>> Anyway, these are some of the obstacles. Not saying it >> isn't possible, it's just a lot of effort. >> >> >> >> >> >>>> Thanks, >> >> >> >> >> >>>> -Aaron >> >> >> >> >> >>>> _______________________________________________ >> >> >> >> >> >>>> cctbxbb mailing list >> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >>>> _______________________________________________ >> >> >> >> >> >>>> cctbxbb mailing list >> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >>>> _______________________________________________ >> >> >> >> >> >>>> cctbxbb mailing list >> >> >> >> >> >>>> cctbxbb(a)phenix-online.org<mailto: >> cctbxbb(a)phenix-online.org> >> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >>>> -- >> >> >> >> >> >>>> This e-mail and any attachments may contain >> confidential, copyright and or privileged material, and are for the use of >> the intended addressee only. If you are not the intended addressee or an >> authorised recipient of the addressee please notify us of receipt by >> returning the e-mail and do not use, copy, retain, distribute or disclose >> the information in or attached to the e-mail. >> >> >> >> >> >>>> Any opinions expressed within this e-mail are those of >> the individual and not necessarily of Diamond Light Source Ltd. >> >> >> >> >> >>>> Diamond Light Source Ltd. cannot guarantee that this >> e-mail or any attachments are free from viruses and we cannot accept >> liability for any damage which you may sustain as a result of software >> viruses which may be transmitted in or with the message. >> >> >> >> >> >>>> Diamond Light Source Limited (company no. 4375679). >> Registered in England and Wales with its registered office at Diamond >> House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 >> 0DE, United Kingdom >> >> >> >> >> >>>> _______________________________________________ >> >> >> >> >> >>>> cctbxbb mailing list >> >> >> >> >> >>>> cctbxbb(a)phenix-online.org >> >> >> >> >> >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >>> _______________________________________________ >> >> >> >> >> >>> cctbxbb mailing list >> >> >> >> >> >>> cctbxbb(a)phenix-online.org >> >> >> >> >> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> >> >> cctbxbb mailing list >> >> >> >> >> >> cctbxbb(a)phenix-online.org >> >> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > _______________________________________________ >> >> >> >> >> > cctbxbb mailing list >> >> >> >> >> > cctbxbb(a)phenix-online.org >> >> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> >> cctbxbb mailing list >> >> >> >> >> cctbxbb(a)phenix-online.org >> >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> > >> >> >> >> > _______________________________________________ >> >> >> >> > cctbxbb mailing list >> >> >> >> > cctbxbb(a)phenix-online.org >> >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Gergely Katona, PhD >> >> >> >> Associate Professor >> >> >> >> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> >> >> >> Box 462, 40530 G?teborg, Sweden >> >> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> >> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> cctbxbb mailing list >> >> >> >> cctbxbb(a)phenix-online.org >> >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> > >> >> >> > _______________________________________________ >> >> >> > cctbxbb mailing list >> >> >> > cctbxbb(a)phenix-online.org >> >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Gergely Katona, PhD >> >> >> Associate Professor >> >> >> Department of Chemistry and Molecular Biology, University of >> Gothenburg >> >> >> Box 462, 40530 G?teborg, Sweden >> >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> >> >> >> >> >> _______________________________________________ >> >> >> cctbxbb mailing list >> >> >> cctbxbb(a)phenix-online.org >> >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> > >> >> > _______________________________________________ >> >> > cctbxbb mailing list >> >> > cctbxbb(a)phenix-online.org >> >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> >> >> >> >> -- >> >> Gergely Katona, PhD >> >> Associate Professor >> >> Department of Chemistry and Molecular Biology, University of Gothenburg >> >> Box 462, 40530 G?teborg, Sweden >> >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> >> >> >> _______________________________________________ >> >> cctbxbb mailing list >> >> cctbxbb(a)phenix-online.org >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> > >> > _______________________________________________ >> > cctbxbb mailing list >> > cctbxbb(a)phenix-online.org >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >> >> >> -- >> Gergely Katona, PhD >> Associate Professor >> Department of Chemistry and Molecular Biology, University of Gothenburg >> Box 462, 40530 G?teborg, Sweden >> Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >> Web: https://urldefense.proofpoint.com/v2/url?u=http-3A__katonalab.eu&d=DwICAg&c… , Email: gergely.katona(a)gu.se >> >> _______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… >> >

1 0

Emergency power test at Berkeley this weekend (12/13-12/16)
by Billy Poon 17 Dec '19

17 Dec '19

Hi everyone, To mitigate disruptions from future power shut offs due to wildfire concerns, we are performing an emergency power test for our web servers. This will affect the phenix-online.org and cci.lbl.gov servers. The websites, mailing lists, and repositories will remain available, but the servers will be running on emergency power instead. The servers will be switched to emergency power at around 2 pm Pacific time on Friday (12/13) and switched back to main power around 10 am Pacific time on Monday (12/16). Please let me know if you experience any issues. Thanks! -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

3 4

Postdoc position in Berkeley
by Petrus Zwart 09 Nov '19

09 Nov '19

Dear All, We have two postdoc positions open in LBNL's Center for Advanced Mathematics in Energy Research Applications (CAMERA; https://www.camera.lbl.gov/) to work on a diverse set of problems in managing, cleaning and interpreting biophysical data ranging from microscopy to crystallography. https://lbl.taleo.net/careersection/2/jobdetail.ftl?job=88805 https://lbl.taleo.net/careersection/2/jobdetail.ftl?job=88480 Feel free to contact me for more information. Regards Peter Zwart -- ------------------------------------------------------------------------ P.H. Zwart Staff Scientist Molecular Biophysics and Integrated Bioimaging & Center for Advanced Mathematics for Energy Research Applications Lawrence Berkeley National Laboratories 1 Cyclotron Road, Berkeley, CA-94703, USA Cell: 510 289 9246 PHENIX: http://www.phenix-online.org CAMERA: http://camera.lbl.gov/ -------------------------------------------------------------------------

1 0

Fwd: Target platform with conda
by Winter, Graeme (DLSLtd,RAL,LSCI) 07 Nov '19

07 Nov '19

Hello all Looking at the commits relating to conda, what are the target platforms now? Looks like 3.7 is valid? I’m asking as it would be good to know what language features we can adopt without causing incompatible code problems. We’ve clearly nearly got up to date on Python 3 now... Do we have guidelines somewhere? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 3

Potential Berkeley Server Outage (October 25)
by Billy Poon 30 Oct '19

30 Oct '19

Hi everyone, As a result of the increased frequency of wildfires in California the power company is taking steps to reduce risk under certain weather conditions. This entails shutting down electricity to regions that are experiencing dry windy conditions. We have been informed that the weather this weekend meets their criteria for a shutdown in our part of the Bay Area. Therefore several of the Berkeley servers will probably be shut down today and will return next week. This will affect: - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 2

Berkeley Server Outage (Starting October 26th)
by Paul Adams 26 Oct '19

26 Oct '19

Dear Colleagues, yet again, as a result of the increased frequency of wildfires in California the power company is taking steps to reduce risk under certain weather conditions. This entails shutting down electricity to regions that are experiencing dry windy conditions. We have been informed that the weather this weekend meets their criteria for a shutdown in our part of the Bay Area. Therefore several of the Berkeley servers will be shut down this weekend and will return later the following week. This will affect: - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. Cheers, Paul -- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Berkeley Lab (http://biosciences.lbl.gov/divisions/mbib) Division Deputy for Biosciences, Advanced Light Source, Berkeley Lab (https://als.lbl.gov) Principal Investigator, ALS-ENABLE, Advanced Light Source, Berkeley Lab (http://als-enable.lbl.gov) Group Leader, Computational Crystallography Initiative, Berkeley Lab (http://cci.lbl.gov) Vice President for Technology, the Joint BioEnergy Institute (http://www.jbei.org) Laboratory Research Manager, ENIGMA Science Focus Area (http://enigma.lbl.gov) Adjunct Professor, Department of Bioengineering, U.C. Berkeley (http://bioeng.berkeley.edu) Adjunct Professor, Comparative Biochemistry, U.C. Berkeley (http://compbiochem.berkeley.edu) Building 33, Room 347 Building 978, Room 4126 Building 977, Room 180C Tel: 1-510-486-4225 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ LBenvenue(a)lbl.gov ][ 1-510-495-2506 ] --

1 0

sgtbx.space_group and sgtbx.space_group_info query
by Ben Williams 14 Oct '19

14 Oct '19

Dear CCTBX devs, What is a sgtbx.space_group_info object, and why does it do something subtly different to sgtbx.space_group? >>> from cctbx import sgtbx >>> print(sgtbx.space_group('C2').type().universal_hermann_mauguin_symbol()) P 1 2 1 (c,2*a+c,b) >>> print(sgtbx.space_group_info('C2').group().type().universal_hermann_mauguin_symbol()) C 1 2 1 sgtbx.space_group_info is notably lacking in docstring (or indeed comments in the code). sgtbx.space_group isn't too hot on the documentation either... Cheers, Ben -- Ben Williams Post-doctoral research associate Scientific Software Diamond Light Source -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 6

Berkeley Server Outage (Starting October evening 8th)
by Paul Adams 13 Oct '19

13 Oct '19

Dear Colleagues, as a result of the increased frequency of wildfires in California the power company is taking steps to reduce risk under certain weather conditions. This entails shutting down electricity to regions that are experiencing dry windy conditions. We have been informed that the weather this week meets their criteria for a shutdown in our part of the Bay Area. Therefore several of the Berkeley servers will be shut down today and will return later in the week. This will affect: - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. Cheers, Paul -- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Berkeley Lab (http://biosciences.lbl.gov/divisions/mbib) Division Deputy for Biosciences, Advanced Light Source, Berkeley Lab (https://als.lbl.gov) Principal Investigator, ALS-ENABLE, Advanced Light Source, Berkeley Lab (http://als-enable.lbl.gov) Group Leader, Computational Crystallography Initiative, Berkeley Lab (http://cci.lbl.gov) Vice President for Technology, the Joint BioEnergy Institute (http://www.jbei.org) Laboratory Research Manager, ENIGMA Science Focus Area (http://enigma.lbl.gov) Adjunct Professor, Department of Bioengineering, U.C. Berkeley (http://bioeng.berkeley.edu) Adjunct Professor, Comparative Biochemistry, U.C. Berkeley (http://compbiochem.berkeley.edu) Building 33, Room 347 Building 978, Room 4126 Building 977, Room 180C Tel: 1-510-486-4225 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ LBenvenue(a)lbl.gov ][ 1-510-495-2506 ] --

2 1

Potential Berkeley Server Outage (September 23/24)
by Paul Adams 22 Sep '19

22 Sep '19

Dear Colleagues, as a result of the increased frequency of wildfires in California the power company is taking steps to reduce risk under certain weather conditions. This entails shutting down electricity to regions that are experiencing dry windy conditions. They are predicting that Monday or Tuesday this coming week (September 23/24) might meet their criteria for a shutdown in the Bay Area. Hopefully this is just a worst case prediction and nothing will happen. However, there is a possibility that the Berkeley servers will be down for some time during that period (duration unknown). This will affect: - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. Cheers, Paul -- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Berkeley Lab (http://biosciences.lbl.gov/divisions/mbib) Division Deputy for Biosciences, Advanced Light Source, Berkeley Lab (https://als.lbl.gov) Principal Investigator, ALS-ENABLE, Advanced Light Source, Berkeley Lab (http://als-enable.lbl.gov) Group Leader, Computational Crystallography Initiative, Berkeley Lab (http://cci.lbl.gov) Vice President for Technology, the Joint BioEnergy Institute (http://www.jbei.org) Laboratory Research Manager, ENIGMA Science Focus Area (http://enigma.lbl.gov) Adjunct Professor, Department of Bioengineering, U.C. Berkeley (http://bioeng.berkeley.edu) Adjunct Professor, Comparative Biochemistry, U.C. Berkeley (http://compbiochem.berkeley.edu) Building 33, Room 347 Building 978, Room 4126 Building 977, Room 180C Tel: 1-510-486-4225 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ LBenvenue(a)lbl.gov ][ 1-510-495-2506 ] --

1 0

Re: [cctbxbb] cctbx.python readline issue
by Mooers, Blaine H.M. (HSC) 25 Aug '19

25 Aug '19

In response to Wolfram Tempel's trouble with backspace in interactive sessions, I found the same problem with backspace in interactive sessions in cctbx.python version 1723 on Mac OS 10.14.6. I solved the problem by installing readline with pip using the command below (I did not try using conda) and sourcing the again $HOME/local/cctbx-dev-1723/cctbx_env.sh. $HOME/local/cctbx-dev-1723/build/bin/libtbx.pip install --upgrade readline Best regards, Blaine Blaine Mooers, Ph.D. Associate Professor Department of Biochemistry and Molecular Biology College of Medicine University of Oklahoma Health Sciences Center S.L. Young Biomedical Research Center (BRC) Rm. 466 975 NE 10th Street, BRC 466 Oklahoma City, OK 73104-5419 ________________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of cctbxbb-request(a)phenix-online.org [cctbxbb-request(a)phenix-online.org] Sent: Saturday, August 24, 2019 2:00 PM To: cctbxbb(a)phenix-online.org Subject: [EXTERNAL] cctbxbb Digest, Vol 125, Issue 22 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit https://urldefense.proofpoint.com/v2/url?u=http-3A__phenix-2Donline.org_mai… or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. interactive cctbx.python backspace behavior (on linux) (wtempel) 2. Re: some thoughts on cctbx and pip (Billy Poon) 3. Re: interactive cctbx.python backspace behavior (on linux) (Billy Poon) 4. Re: interactive cctbx.python backspace behavior (on linux) (Billy Poon) 5. Re: some thoughts on cctbx and pip (Tristan Croll) ---------------------------------------------------------------------- Message: 1 Date: Fri, 23 Aug 2019 15:54:09 -0400 From: wtempel <wtempel(a)gmail.com> To: cctbxbb(a)phenix-online.org Subject: [cctbxbb] interactive cctbx.python backspace behavior (on linux) Message-ID: <CAGjFzmBj3m_Uez1Siz6HPrjxAdOJqSczO-udnJo7C2W5Od4Q8A(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi, Currently, cctbx.python behaves like described in this stackoverflow post: https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_ques… . I am missing the convenience of backspace delete, which is available on /usr/bin/python interactive sessions. Could that feature be added during compilation of the pre-built cctbx distributions? Regards. Wolfram Tempel

2 1

interactive cctbx.python backspace behavior (on linux)
by wtempel 24 Aug '19

24 Aug '19

Hi, Currently, cctbx.python behaves like described in this stackoverflow post: https://stackoverflow.com/questions/893053/seeing-escape-characters-when-pr…. I am missing the convenience of backspace delete, which is available on /usr/bin/python interactive sessions. Could that feature be added during compilation of the pre-built cctbx distributions? Regards. Wolfram Tempel

2 2

cls parameter
by Graeme.Winter＠Diamond.ac.uk 23 Aug '19

23 Aug '19

Folks, For reading shelxl .res files I need to define a cls parameter - it is not _used_ for anything though - so 0 works fine e.g. from iotbx import shelx import sys s = shelx.cctbx_xray_structure_from(0, filename=sys.argv[1]) will work fine. I based this on the documentation at https://cci.lbl.gov/cctbx_docs/iotbx/iotbx.shelx.html What have I missed? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 20 Aug '19

20 Aug '19

That is the correct directory for the general Phenix install. @Christopher Williams <christopher.sci.williams(a)gmail.com> Does the script run bootstrap.py from the current directory or does it cd elsewhere. I have noticed that accessing the repos at Duke can be a problem but I just did so the servers are fine. I still think the "running the install script with bootstrap.py in the current directory" could still be the problem. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 19, 2019 at 10:49 AM Charitha Belluru Shetty < charitha-belluru.b.shetty(a)gsk.com> wrote: > Hi Nigel, > > > > Can you please let us know which directory? Since the Molprobity directory > is not created after running > > *. install_via_bootstrap.sh 10* > > > > The steps to install ideally would be as below: > > ./install_via_bootstrap.sh 4 > > cd/molprobity > > ./setup.sh > > > > > > But in our case “modules” directory got created. > > > > -bash-4.2$ ls -l > > total 131 > > -rw-r--r-- 1 cbs72396 appmgmt 90799 Aug 14 18:26 bootstrap.py > > -rw-r--r-- 1 cbs72396 appmgmt 82015 Aug 16 13:38 bootstrap.pyc > > -rw-r--r-- 1 cbs72396 appmgmt 1817 Aug 14 14:03 install_via_bootstrap.sh > > drwxr-sr-x 12 cbs72396 appmgmt 22 Aug 16 13:38 modules > > > > -bash-4.2$ ls -l modules > > total 2913 > > drwxr-xr-x 11 cbs72396 appmgmt 17 Nov 27 2012 annlib > > drwxr-sr-x 8 cbs72396 appmgmt 15 Aug 16 13:38 annlib_adaptbx > > -rw-r--r-- 1 cbs72396 appmgmt 519563 Aug 16 05:13 annlib.gz > > drwxr-sr-x 5 cbs72396 appmgmt 6 Aug 16 13:38 boost > > drwxr-sr-x 20 cbs72396 appmgmt 48 Aug 16 13:38 cbflib > > drwxr-xr-x 5 cbs72396 appmgmt 6 Oct 30 2013 ccp4io > > drwxr-xr-x 5 cbs72396 appmgmt 30 May 19 17:56 ccp4io_adaptbx > > -rw-r--r-- 1 cbs72396 appmgmt 208764 Aug 16 05:13 ccp4io_adaptbx.gz > > -rw-r--r-- 1 cbs72396 appmgmt 1417443 Aug 16 05:13 ccp4io.gz > > drwxr-sr-x 38 cbs72396 appmgmt 45 Aug 16 13:38 cctbx_project > > drwxr-sr-x 5 cbs72396 appmgmt 5 Aug 16 13:37 chem_data > > drwxrwxr-x 4 cbs72396 appmgmt 15 Sep 18 2017 scons > > -rw-r--r-- 1 cbs72396 appmgmt 636186 Aug 16 05:13 scons.gz > > drwxr-xr-x 4 cbs72396 appmgmt 11 Aug 19 2009 tntbx > > -rw-r--r-- 1 cbs72396 appmgmt 38909 Aug 16 05:13 tntbx.gz > > > > Best Regards, > > Charitha Shetty > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 19, 2019 1:08 PM > *To:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Cc:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com>; Robert > Nolte <robert.t.nolte(a)gsk.com>; cctbxbb(a)phenix-online.org > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Sorry guy, I was on holidays and the maintenance was a problem. > > > > Try putting the bootstrap.py in the directory that the command is being > run. > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Fri, Aug 16, 2019 at 7:27 PM Christopher Williams < > christopher.sci.williams(a)gmail.com> wrote: > > Hmm. Nigel, I may need some of your expertise here. It looks like > there's a consistent problem with svn checkouts. > > > > "git clone" operations seem to have been successful, so it probably isn't > a matter of GitHub being down. Especially since the svn checkout of geostd > from Sourceforge also failed. > > > > Do you know if your system has problems with svn? Our svn checkouts > generally take this form: " svn co > https://github.com/rlabduke/MolProbity.git/trunk molprobity > --non-interactive --trust-server-cert " Is there perhaps a security > issue with the --trust-server-cert flag? > > > > I believe that the cci machines at Berkeley are under maintenance until > Sunday, so there are other reasons the script won't work until then. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > On Fri, Aug 16, 2019 at 5:45 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Christopher, > > > > I ran the installer script twice and this is what I got… > > > > *When I ran it first time, only the modules directory was created* > > bash-4.2$ *. install_via_bootstrap.sh 10* > > ++++++++++ creating build directories ... > > svn: E000110: Unable to connect to a repository at URL 'svn:// > svn.code.sf.net/p/geostd/code/trunk' > > svn: E000110: Can't read from connection: Connection timed out > > svn: E175002: Unable to connect to a repository at URL ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_rotame… > ' > > svn: E175002: OPTIONS of ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_rotame…': > SSL handshake failed: SSL socket read failed (https://github.com) > > svn: E175002: Unable to connect to a repository at URL ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_ramach… > ' > > svn: E175002: OPTIONS of ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_ramach…': > SSL handshake failed: SSL socket read failed (https://github.com) > > svn: E175002: Unable to connect to a repository at URL ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_cablam… > ' > > svn: E175002: OPTIONS of ' > https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_cablam…': > SSL handshake failed: SSL socket read failed (https://github.com) > > svn: E175002: Unable to connect to a repository at URL ' > https://github.com/cctbx/cctbx_project.git/trunk/libtbx/auto_build/bootstra… > ' > > svn: E175002: OPTIONS of ' > https://github.com/cctbx/cctbx_project.git/trunk/libtbx/auto_build/bootstra…': > SSL handshake failed: SSL socket read failed (https://github.com) > > python: can't open file 'bootstrap.py': [Errno 2] No such file or directory > > -bash: build/setpaths.sh: No such file or directory > > python: can't open file > 'modules/chem_data/cablam_data/rebuild_cablam_cache.py': [Errno 2] No such > file or directory > > ++++++++++ MolProbity configure.sh finished. > > ++++++++++ Run molprobity/setup.sh to complete installation. > > -bash-4.2$ > > > > -bash-4.2$ pwd > > /hpc/apps/current/molprobity/v4.4.app > > -bash-4.2$ ls -l > > total 31 > > *-rw-r--r-- 1 cbs72396 appmgmt 1817 Aug 14 14:03 install_via_bootstrap.sh* > > *drwxr-sr-x 3 cbs72396 appmgmt 3 Aug 15 17:58 modules* > > > > > > *Second time when I ran..bootstap.py and bootstrap.pyc was crated and the > modules directory…* > > > > -bash-4.2$ *. install_via_bootstrap.sh 4* > > ++++++++++ creating build directories ... > > > > > > svn: E000110: Unable to connect to a repository at URL 'svn:// > svn.code.sf.net/p/geostd/code/trunk' > > svn: E000110: Can't read from connection: Connection timed out > > Performing actions: hot update base build > > Installing base packages using: > > python modules/cctbx_project/libtbx/auto_build/install_conda.py > --builder=molprobity --install_conda > > ===== Downloading http://cci.lbl.gov/repositories/annlib.gz: 0.5 MB > > [0%.........20%.........40%.........60%.........80%.........100%] > > ===== Running in modules: extracting files from annlib.gz > > ===== Downloading http://cci.lbl.gov/repositories/scons.gz: 0.6 MB > > [0%.........20%.........40%.........60%.........80%.........100%] > > ===== Running in modules: extracting files from scons.gz > > ===== Downloading http://cci.lbl.gov/repositories/ccp4io.gz: 1.4 MB > > [0%.........20%.........40%.........60%.........80%.........100%] > > ===== Running in modules: extracting files from ccp4io.gz > > ===== Running in modules: git clone --recursive > https://github.com/cctbx/boost.git boost --progress --verbose > > Cloning into 'boost'... > > POST git-upload-pack (942 bytes) > > remote: Enumerating objects: 27664, done. > > remote: Total 27664 (delta 0), reused 0 (delta 0), pack-reused 27664 > > Receiving objects: 100% (27664/27664), 20.47 MiB | 14.72 MiB/s, done. > > Resolving deltas: 100% (15913/15913), done. > > Checking out files: 100% (14245/14245), done. > > setting branch master to rebase > > ===== Running in modules/boost: git rev-parse HEAD > > 72edfa007c73dd3dd01f5530205f7ca14cda7420 > > ===== Running in modules: git clone --recursive > https://github.com/yayahjb/cbflib.git cbflib --progress --verbose > > Cloning into 'cbflib'... > > POST git-upload-pack (gzip 2442 to 1233 bytes) > > remote: Enumerating objects: 7463, done. > > remote: Total 7463 (delta 0), reused 0 (delta 0), pack-reused 7463 > > Receiving objects: 100% (7463/7463), 24.84 MiB | 13.21 MiB/s, done. > > Resolving deltas: 100% (5380/5380), done. > > setting branch master to rebase > > ===== Running in modules/cbflib: git rev-parse HEAD > > 611f21bbef81c5104954aaab9b72e4e55ec3e70d > > ===== Running in modules: git clone --recursive > https://github.com/cctbx/cctbx_project.git cctbx_project --progress > --verbose > > Cloning into 'cctbx_project'... > > POST git-upload-pack (gzip 1842 to 958 bytes) > > remote: Enumerating objects: 61, done. > > remote: Counting objects: 100% (61/61), done. > > remote: Compressing objects: 100% (52/52), done. > > remote: Total 243691 (delta 14), reused 22 (delta 9), pack-reused 243630 > > Receiving objects: 100% (243691/243691), 91.05 MiB | 14.77 MiB/s, done. > > Resolving deltas: 100% (192566/192566), done. > > Checking out files: 100% (6173/6173), done. > > setting branch master to rebase > > ===== Running in modules/cctbx_project: git rev-parse HEAD > > bd3139c85312f5f43009e1eb31baa34f41734fbb > > ===== Downloading http://cci.lbl.gov/repositories/ccp4io_adaptbx.gz: > 203.9 kB > > [0%.........20%.........40%.........60%.........80%.........100%] > > ===== Running in modules: extracting files from ccp4io_adaptbx.gz > > ===== Running in modules: git clone --recursive > https://github.com/cctbx/annlib_adaptbx.git annlib_adaptbx --progress > --verbose > > Cloning into 'annlib_adaptbx'... > > POST git-upload-pack (842 bytes) > > remote: Enumerating objects: 175, done. > > remote: Total 175 (delta 0), reused 0 (delta 0), pack-reused 175 > > Receiving objects: 100% (175/175), 162.09 KiB | 0 bytes/s, done. > > Resolving deltas: 100% (77/77), done. > > setting branch master to rebase > > ===== Running in modules/annlib_adaptbx: git rev-parse HEAD > > 3ca66b2dd77eacd7524f915fce0705ac0764b46f > > ===== Downloading http://cci.lbl.gov/repositories/tntbx.gz: 38.0 kB > > [0%.........20%.........40%.........60%.........80%.........100%] > > ===== Running in modules: extracting files from tntbx.gz > > ===== Running in .: svn co > https://github.com/rlabduke/MolProbity.git/trunk molprobity > --non-interactive --trust-server-cert > > svn: E175002: Unable to connect to a repository at URL ' > https://github.com/rlabduke/MolProbity.git/trunk' > > svn: E175002: OPTIONS of 'https://github.com/rlabduke/MolProbity.git/trunk': > SSL handshake failed: SSL socket read failed (https://github.com) > > Process failed with return code 1 > > -bash: build/setpaths.sh: No such file or directory > > Traceback (most recent call last): > > File "modules/chem_data/cablam_data/rebuild_cablam_cache.py", line 2, in > <module> > > import libtbx.load_env > > ImportError: No module named libtbx.load_env > > ++++++++++ MolProbity configure.sh finished. > > ++++++++++ Run molprobity/setup.sh to complete installation. > > -bash-4.2$ > > -bash-4.2$ > > *-bash-4.2$ ls -l* > > *total 131* > > *-rw-r--r-- 1 cbs72396 appmgmt 90799 Aug 14 18:26 bootstrap.py* > > *-rw-r--r-- 1 cbs72396 appmgmt 82015 Aug 16 13:38 bootstrap.pyc* > > *-rw-r--r-- 1 cbs72396 appmgmt 1817 Aug 14 14:03 > install_via_bootstrap.sh* > > *drwxr-sr-x 12 cbs72396 appmgmt 22 Aug 16 13:38 modules* > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Wednesday, August 14, 2019 2:23 PM > *To:* Robert Nolte <robert.t.nolte(a)gsk.com> > *Cc:* Nigel Moriarty <nwmoriarty(a)lbl.gov>; Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com>; cctbxbb(a)phenix-online.org > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hi All, > > > > Okay looks like the new installer works. It uses the conda mode of the > cctbx bootstrap builder for everything except chem_data, which it builds on > its own. > > > > Get the new installer from our GitHub here: > > svn --quiet --non-interactive --trust-server-cert export > https://github.com/rlabduke/MolProbity.git/trunk/install_via_bootstrap.sh > > > > Place it in the dir where you want to install MolProbity, then run it. > > . install_via_bootstrap.sh > > The script optionally accepts a single integer argument to set the number > of processors that the bootstrap builder will use. > > . install_via_bootstrap.sh 4 > > > > When this is complete, cd to the molprobity dir it created and run > setup.sh, as usual. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > On Wed, Aug 14, 2019 at 1:33 PM Christopher Williams < > christopher.sci.williams(a)gmail.com> wrote: > > Hi Bob, > > > > I am currently doing one more test build to make sure that the new > installer works. It looks promising. > > > > Regards, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > On Wed, Aug 14, 2019 at 11:17 AM Robert Nolte <robert.t.nolte(a)gsk.com> > wrote: > > Folks, > > Any progress on sorting out our issue, or cleaning up the installer? > > Bob > > > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 7:48 PM > *To:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Cc:* Robert Nolte <robert.t.nolte(a)gsk.com>; Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com>; cctbxbb(a)phenix-online.org > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > The dispatcher should be using both phenix.reduce and molprobity.reduce > interchangeably in all cases. Ditto for all the validations that > MolProbity runs like phenix.rotalyze/molprobity.rotalyze. > > > > My current install of MolProbity is successfully accessing Reduce via > phenix.reduce, so I don't *think* this is the issue. > > > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > On Mon, Aug 12, 2019 at 6:07 PM Nigel Moriarty <nwmoriarty(a)lbl.gov> wrote: > > But maybe the installation of cctbx has molprobity.reduce and not > phenix.reduce so if that is the case we can change the script to look for > molprobity.reduce instead of phenix.reduce. > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 2:58 PM Robert Nolte <robert.t.nolte(a)gsk.com> > wrote: > > According to the log file when I just tried to run it, it is phenix.reduce > that it can’t find. > > Bob > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 5:55 PM > *To:* Robert Nolte <robert.t.nolte(a)gsk.com> > *Cc:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com>; > Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > I'm just trying to find out if, at the command line, whether it is > phenix.reduce or molprobity.reduce. Charitha just needs to type one or the > other to see what happens. > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 2:44 PM Robert Nolte <robert.t.nolte(a)gsk.com> > wrote: > > Nigel, > > Currently it is not running. This was working from the scratch > install last week. I am not sure what is going on now. > > Bob > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:52 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Does molprobity.reduce run? > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 1:14 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Nigel, > > > > I hit the send button too soon..Ignore my previous email 😊 > > > > -bash-4.2$ source setpaths.sh > > -bash-4.2$ > > -bash-4.2$ env |grep -i reduce > > -bash-4.2$ > > > > So the “setpath.sh” has the below contents… > > > > $ cat /hpc/apps/current/molprobity/v4.4.app/MolProbity/build/setpaths.sh > > # THIS IS AN AUTOMATICALLY GENERATED FILE. > > # DO NOT EDIT! CHANGES WILL BE LOST. > > ocwd="`pwd`" > > if [ -n "$LIBTBX_BUILD_RELOCATION_HINT" ]; then > > cd "$LIBTBX_BUILD_RELOCATION_HINT" > > LIBTBX_BUILD_RELOCATION_HINT= > > export LIBTBX_BUILD_RELOCATION_HINT > > elif [ -n "$BASH_SOURCE" ]; then > > LIBTBX_BUILD=`dirname "$BASH_SOURCE[0]"` > > cd "$LIBTBX_BUILD" > > else > > cd "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > fi > > LIBTBX_BUILD=`pwd -P` > > export LIBTBX_BUILD > > LIBTBX_OPATH="$PATH" > > export LIBTBX_OPATH > > PATH="$LIBTBX_BUILD/bin:$PATH" > > export PATH > > cd "$ocwd" > > ocwd= > > alias libtbx.setpaths_all=". \"$LIBTBX_BUILD/setpaths_all.sh\"" > > alias libtbx.unsetpaths=". \"$LIBTBX_BUILD/unsetpaths.sh\"" > > if [ -n "$LIBTBX_OPATH" ]; then > > LIBTBX_TMPVAL="$LIBTBX_OPATH" > > else > > LIBTBX_TMPVAL= > > fi > > export LIBTBX_TMPVAL > > PATH=`libtbx.path_utility prepend LIBTBX_TMPVAL "$LIBTBX_BUILD/bin" < > /dev/null` > > export PATH > > if [ "$PATH" = "L_I_B_T_B_X_E_M_P_T_Y" ]; then unset PATH; fi > > LIBTBX_TMPVAL= > > LIBTBX_OPATH= > > LIBTBX_BUILD= > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:08 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Once you source the setpaths.sh file in the build directory do you have > phenix.reduce or molprobity.reduce? > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Christopher/ Nigel, > > > > Thank you for your quick reply. > > > > The build process was completed without any error. Pasting few of the > lines from it… > > > > > > ===== Running in build: rebuild rotarama > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rota8000-cys.data -> rota8000-cys.pickle: converting ... done. > > rota8000-asp.data -> rota8000-asp.pickle: converting ... done. > > rota8000-ser.data -> rota8000-ser.pickle: converting ... done. > > rota8000-gln.data -> rota8000-gln.pickle: converting ... done. > > rota8000-ile.data -> rota8000-ile.pickle: converting ... done. > > rota8000-pro.data -> rota8000-pro.pickle: converting ... done. > > rota8000-lys.data -> rota8000-lys.pickle: converting ... done. > > rota8000-thr.data -> rota8000-thr.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. > > rota8000-glu.data -> rota8000-glu.pickle: converting ... done. > > rota8000-met.data -> rota8000-met.pickle: converting ... done. > > rota8000-his.data -> rota8000-his.pickle: converting ... done. > > rota8000-leu.data -> rota8000-leu.pickle: converting ... done. > > rota8000-arg.data -> rota8000-arg.pickle: converting ... done. > > rota8000-trp.data -> rota8000-trp.pickle: converting ... done. > > rota8000-asn.data -> rota8000-asn.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. > > rota8000-val.data -> rota8000-val.pickle: converting ... done. > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: converting > ... done. > > rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. > > rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. > > rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: > converting ... done. > > rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. > > rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting ... > done. > > u+s,u,s: 10.06 9.83 0.23 > > ===== Running in .: libtbx.refresh > > Python: 2.7.16 > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" > > Compiler: default > > Build mode: release > > Warning level: 0 > > Precompiled Headers: False > > Static libraries: False > > Static exe: False > > Scan Boost headers: False > > Write full flex_fwd.h files: False > > Build Boost.Python extensions: True > > Define BOOST_PYTHON_NO_PY_SIGNATURES: False > > Define BOOST_PYTHON_BOOL_INT_STRICT: True > > Enable OpenMP if possible: False > > Boost threads enabled: True > > Enable CUDA: False > > Use conda: False > > Use opt_resources if available: False > > Use environment flags: False > > Enable C++11: False > > Python3 migration warning policy: None > > command_version_suffix: None > > Relocatable paths anchored at: > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build > > Top-down list of all modules involved: > > mmtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" > > iotbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" > > ccp4io+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" > > smtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" > > ucif > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" > > cctbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" > > scitbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" > > fable > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" > > omptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" > > boost+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" > > tbxx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" > > chiltbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" > > libtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" > > Creating files in build directory: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > dispatcher_include_template.sh > > dispatcher_head.sh > > setpaths.sh > > setpaths.csh > > setpaths_all.sh > > setpaths_all.csh > > setpaths_debug.sh > > unsetpaths.sh > > setpaths_debug.csh > > unsetpaths.csh > > SConstruct > > Makefile > > run_tests.csh > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" > > Generating C++ header files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" > > Using fable to convert scitbx/lbfgs.f > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" > > Generating C++ files in: > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" > > Using fable to convert iotbx/pdb/hybrid_36_f.f > > Writing: iotbx/pdb/hybrid_36_fem.cpp > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" > > Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" > > Regenerating module files in > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, > could not regenerate pango files > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not > present, could not regenerate gdk-pixbuf.loaders > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, > could not rebuild fonts > > generating index.theme file > > Done. > > > > *Bootstrap success: build* > > *++++++++++ MolProbity configure.sh finished.* > > > > > > > > [tm375568@uptus060-3 MolProbity]$ ./setup.sh > > ++++++++++ creating directories ... > > ++++++++++ setting permisions ... > > *Couldn't get a file descriptor referring to the console **à I assume > this can be ignored. * > > [tm375568@uptus060-3 MolProbity]$ > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 3:34 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hmm. It looks like Reduce and/or Probe has failed to install correctly. > Most likely an early call to Reduce has resulted in an empty file, which is > then passed into other programs, resulting in error messages. > > > > The configure script ran a build process. Do you know if that process > finished successfully or if it ended with errors? If the build exited > early, it may have never reached the part where Reduce gets compiled. Some > dependencies have changed, and I have been having trouble with my own > builds lately. > > > > It is also possible that there was just a network hiccup that prevented > something from downloading correctly, and a second attempt will work. > > > > I am working on a new configure/install script for more consistent > handling of dependencies. Hopefully it will be ready in about a day. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 20 Aug '19

20 Aug '19

Sorry guy, I was on holidays and the maintenance was a problem. Try putting the bootstrap.py in the directory that the command is being run. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Fri, Aug 16, 2019 at 7:27 PM Christopher Williams < christopher.sci.williams(a)gmail.com> wrote: > Hmm. Nigel, I may need some of your expertise here. It looks like > there's a consistent problem with svn checkouts. > > "git clone" operations seem to have been successful, so it probably isn't > a matter of GitHub being down. Especially since the svn checkout of geostd > from Sourceforge also failed. > > Do you know if your system has problems with svn? Our svn checkouts > generally take this form: " svn co > https://github.com/rlabduke/MolProbity.git/trunk molprobity > --non-interactive --trust-server-cert " Is there perhaps a security > issue with the --trust-server-cert flag? > > I believe that the cci machines at Berkeley are under maintenance until > Sunday, so there are other reasons the script won't work until then. > > Good luck, > -Christopher Williams > ---Richardson Lab, Duke University > > > On Fri, Aug 16, 2019 at 5:45 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > >> Hi Christopher, >> >> >> >> I ran the installer script twice and this is what I got… >> >> >> >> *When I ran it first time, only the modules directory was created* >> >> bash-4.2$ *. install_via_bootstrap.sh 10* >> >> ++++++++++ creating build directories ... >> >> svn: E000110: Unable to connect to a repository at URL 'svn:// >> svn.code.sf.net/p/geostd/code/trunk' >> >> svn: E000110: Can't read from connection: Connection timed out >> >> svn: E175002: Unable to connect to a repository at URL ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_rotame… >> ' >> >> svn: E175002: OPTIONS of ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_rotame…': >> SSL handshake failed: SSL socket read failed (https://github.com) >> >> svn: E175002: Unable to connect to a repository at URL ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_ramach… >> ' >> >> svn: E175002: OPTIONS of ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_ramach…': >> SSL handshake failed: SSL socket read failed (https://github.com) >> >> svn: E175002: Unable to connect to a repository at URL ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_cablam… >> ' >> >> svn: E175002: OPTIONS of ' >> https://github.com/rlabduke/reference_data.git/trunk/Top8000/Top8000_cablam…': >> SSL handshake failed: SSL socket read failed (https://github.com) >> >> svn: E175002: Unable to connect to a repository at URL ' >> https://github.com/cctbx/cctbx_project.git/trunk/libtbx/auto_build/bootstra… >> ' >> >> svn: E175002: OPTIONS of ' >> https://github.com/cctbx/cctbx_project.git/trunk/libtbx/auto_build/bootstra…': >> SSL handshake failed: SSL socket read failed (https://github.com) >> >> python: can't open file 'bootstrap.py': [Errno 2] No such file or >> directory >> >> -bash: build/setpaths.sh: No such file or directory >> >> python: can't open file >> 'modules/chem_data/cablam_data/rebuild_cablam_cache.py': [Errno 2] No such >> file or directory >> >> ++++++++++ MolProbity configure.sh finished. >> >> ++++++++++ Run molprobity/setup.sh to complete installation. >> >> -bash-4.2$ >> >> >> >> -bash-4.2$ pwd >> >> /hpc/apps/current/molprobity/v4.4.app >> >> -bash-4.2$ ls -l >> >> total 31 >> >> *-rw-r--r-- 1 cbs72396 appmgmt 1817 Aug 14 14:03 >> install_via_bootstrap.sh* >> >> *drwxr-sr-x 3 cbs72396 appmgmt 3 Aug 15 17:58 modules* >> >> >> >> >> >> *Second time when I ran..bootstap.py and bootstrap.pyc was crated and the >> modules directory…* >> >> >> >> -bash-4.2$ *. install_via_bootstrap.sh 4* >> >> ++++++++++ creating build directories ... >> >> >> >> >> >> svn: E000110: Unable to connect to a repository at URL 'svn:// >> svn.code.sf.net/p/geostd/code/trunk' >> >> svn: E000110: Can't read from connection: Connection timed out >> >> Performing actions: hot update base build >> >> Installing base packages using: >> >> python modules/cctbx_project/libtbx/auto_build/install_conda.py >> --builder=molprobity --install_conda >> >> ===== Downloading http://cci.lbl.gov/repositories/annlib.gz: 0.5 MB >> >> [0%.........20%.........40%.........60%.........80%.........100%] >> >> ===== Running in modules: extracting files from annlib.gz >> >> ===== Downloading http://cci.lbl.gov/repositories/scons.gz: 0.6 MB >> >> [0%.........20%.........40%.........60%.........80%.........100%] >> >> ===== Running in modules: extracting files from scons.gz >> >> ===== Downloading http://cci.lbl.gov/repositories/ccp4io.gz: 1.4 MB >> >> [0%.........20%.........40%.........60%.........80%.........100%] >> >> ===== Running in modules: extracting files from ccp4io.gz >> >> ===== Running in modules: git clone --recursive >> https://github.com/cctbx/boost.git boost --progress --verbose >> >> Cloning into 'boost'... >> >> POST git-upload-pack (942 bytes) >> >> remote: Enumerating objects: 27664, done. >> >> remote: Total 27664 (delta 0), reused 0 (delta 0), pack-reused 27664 >> >> Receiving objects: 100% (27664/27664), 20.47 MiB | 14.72 MiB/s, done. >> >> Resolving deltas: 100% (15913/15913), done. >> >> Checking out files: 100% (14245/14245), done. >> >> setting branch master to rebase >> >> ===== Running in modules/boost: git rev-parse HEAD >> >> 72edfa007c73dd3dd01f5530205f7ca14cda7420 >> >> ===== Running in modules: git clone --recursive >> https://github.com/yayahjb/cbflib.git cbflib --progress --verbose >> >> Cloning into 'cbflib'... >> >> POST git-upload-pack (gzip 2442 to 1233 bytes) >> >> remote: Enumerating objects: 7463, done. >> >> remote: Total 7463 (delta 0), reused 0 (delta 0), pack-reused 7463 >> >> Receiving objects: 100% (7463/7463), 24.84 MiB | 13.21 MiB/s, done. >> >> Resolving deltas: 100% (5380/5380), done. >> >> setting branch master to rebase >> >> ===== Running in modules/cbflib: git rev-parse HEAD >> >> 611f21bbef81c5104954aaab9b72e4e55ec3e70d >> >> ===== Running in modules: git clone --recursive >> https://github.com/cctbx/cctbx_project.git cctbx_project --progress >> --verbose >> >> Cloning into 'cctbx_project'... >> >> POST git-upload-pack (gzip 1842 to 958 bytes) >> >> remote: Enumerating objects: 61, done. >> >> remote: Counting objects: 100% (61/61), done. >> >> remote: Compressing objects: 100% (52/52), done. >> >> remote: Total 243691 (delta 14), reused 22 (delta 9), pack-reused 243630 >> >> Receiving objects: 100% (243691/243691), 91.05 MiB | 14.77 MiB/s, done. >> >> Resolving deltas: 100% (192566/192566), done. >> >> Checking out files: 100% (6173/6173), done. >> >> setting branch master to rebase >> >> ===== Running in modules/cctbx_project: git rev-parse HEAD >> >> bd3139c85312f5f43009e1eb31baa34f41734fbb >> >> ===== Downloading http://cci.lbl.gov/repositories/ccp4io_adaptbx.gz: >> 203.9 kB >> >> [0%.........20%.........40%.........60%.........80%.........100%] >> >> ===== Running in modules: extracting files from ccp4io_adaptbx.gz >> >> ===== Running in modules: git clone --recursive >> https://github.com/cctbx/annlib_adaptbx.git annlib_adaptbx --progress >> --verbose >> >> Cloning into 'annlib_adaptbx'... >> >> POST git-upload-pack (842 bytes) >> >> remote: Enumerating objects: 175, done. >> >> remote: Total 175 (delta 0), reused 0 (delta 0), pack-reused 175 >> >> Receiving objects: 100% (175/175), 162.09 KiB | 0 bytes/s, done. >> >> Resolving deltas: 100% (77/77), done. >> >> setting branch master to rebase >> >> ===== Running in modules/annlib_adaptbx: git rev-parse HEAD >> >> 3ca66b2dd77eacd7524f915fce0705ac0764b46f >> >> ===== Downloading http://cci.lbl.gov/repositories/tntbx.gz: 38.0 kB >> >> [0%.........20%.........40%.........60%.........80%.........100%] >> >> ===== Running in modules: extracting files from tntbx.gz >> >> ===== Running in .: svn co >> https://github.com/rlabduke/MolProbity.git/trunk molprobity >> --non-interactive --trust-server-cert >> >> svn: E175002: Unable to connect to a repository at URL ' >> https://github.com/rlabduke/MolProbity.git/trunk' >> >> svn: E175002: OPTIONS of ' >> https://github.com/rlabduke/MolProbity.git/trunk': SSL handshake failed: >> SSL socket read failed (https://github.com) >> >> Process failed with return code 1 >> >> -bash: build/setpaths.sh: No such file or directory >> >> Traceback (most recent call last): >> >> File "modules/chem_data/cablam_data/rebuild_cablam_cache.py", line 2, >> in <module> >> >> import libtbx.load_env >> >> ImportError: No module named libtbx.load_env >> >> ++++++++++ MolProbity configure.sh finished. >> >> ++++++++++ Run molprobity/setup.sh to complete installation. >> >> -bash-4.2$ >> >> -bash-4.2$ >> >> *-bash-4.2$ ls -l* >> >> *total 131* >> >> *-rw-r--r-- 1 cbs72396 appmgmt 90799 Aug 14 18:26 bootstrap.py* >> >> *-rw-r--r-- 1 cbs72396 appmgmt 82015 Aug 16 13:38 bootstrap.pyc* >> >> *-rw-r--r-- 1 cbs72396 appmgmt 1817 Aug 14 14:03 >> install_via_bootstrap.sh* >> >> *drwxr-sr-x 12 cbs72396 appmgmt 22 Aug 16 13:38 modules* >> >> >> >> >> >> Best Regards, >> >> Charitha Shetty >> >> >> >> *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> >> *Sent:* Wednesday, August 14, 2019 2:23 PM >> *To:* Robert Nolte <robert.t.nolte(a)gsk.com> >> *Cc:* Nigel Moriarty <nwmoriarty(a)lbl.gov>; Charitha Belluru Shetty < >> charitha-belluru.b.shetty(a)gsk.com>; cctbxbb(a)phenix-online.org >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> Hi All, >> >> >> >> Okay looks like the new installer works. It uses the conda mode of the >> cctbx bootstrap builder for everything except chem_data, which it builds on >> its own. >> >> >> >> Get the new installer from our GitHub here: >> >> svn --quiet --non-interactive --trust-server-cert export >> https://github.com/rlabduke/MolProbity.git/trunk/install_via_bootstrap.sh >> >> >> >> Place it in the dir where you want to install MolProbity, then run it. >> >> . install_via_bootstrap.sh >> >> The script optionally accepts a single integer argument to set the number >> of processors that the bootstrap builder will use. >> >> . install_via_bootstrap.sh 4 >> >> >> >> When this is complete, cd to the molprobity dir it created and run >> setup.sh, as usual. >> >> >> >> Good luck, >> >> -Christopher Williams >> >> ---Richardson Lab, Duke University >> >> >> >> >> >> On Wed, Aug 14, 2019 at 1:33 PM Christopher Williams < >> christopher.sci.williams(a)gmail.com> wrote: >> >> Hi Bob, >> >> >> >> I am currently doing one more test build to make sure that the new >> installer works. It looks promising. >> >> >> >> Regards, >> >> -Christopher Williams >> >> ---Richardson Lab, Duke University >> >> >> >> On Wed, Aug 14, 2019 at 11:17 AM Robert Nolte <robert.t.nolte(a)gsk.com> >> wrote: >> >> Folks, >> >> Any progress on sorting out our issue, or cleaning up the installer? >> >> Bob >> >> >> >> >> >> *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> >> *Sent:* Monday, August 12, 2019 7:48 PM >> *To:* Nigel Moriarty <nwmoriarty(a)lbl.gov> >> *Cc:* Robert Nolte <robert.t.nolte(a)gsk.com>; Charitha Belluru Shetty < >> charitha-belluru.b.shetty(a)gsk.com>; cctbxbb(a)phenix-online.org >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> The dispatcher should be using both phenix.reduce and molprobity.reduce >> interchangeably in all cases. Ditto for all the validations that >> MolProbity runs like phenix.rotalyze/molprobity.rotalyze. >> >> >> >> My current install of MolProbity is successfully accessing Reduce via >> phenix.reduce, so I don't *think* this is the issue. >> >> >> >> -Christopher Williams >> >> ---Richardson Lab, Duke University >> >> >> >> On Mon, Aug 12, 2019 at 6:07 PM Nigel Moriarty <nwmoriarty(a)lbl.gov> >> wrote: >> >> But maybe the installation of cctbx has molprobity.reduce and not >> phenix.reduce so if that is the case we can change the script to look for >> molprobity.reduce instead of phenix.reduce. >> >> >> Cheers >> >> >> >> Nigel >> >> >> >> --- >> >> Nigel W. Moriarty >> Building 33R0349, Molecular Biophysics and Integrated Bioimaging >> >> Lawrence Berkeley National Laboratory >> Berkeley, CA 94720-8235 >> Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov >> Fax : 510-486-5909 Web : CCI.LBL.gov >> >> >> >> >> >> On Mon, Aug 12, 2019 at 2:58 PM Robert Nolte <robert.t.nolte(a)gsk.com> >> wrote: >> >> According to the log file when I just tried to run it, it is >> phenix.reduce that it can’t find. >> >> Bob >> >> >> >> >> >> *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> >> *Sent:* Monday, August 12, 2019 5:55 PM >> *To:* Robert Nolte <robert.t.nolte(a)gsk.com> >> *Cc:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com>; >> Christopher Williams <christopher.sci.williams(a)gmail.com>; >> cctbxbb(a)phenix-online.org >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> I'm just trying to find out if, at the command line, whether it is >> phenix.reduce or molprobity.reduce. Charitha just needs to type one or the >> other to see what happens. >> >> >> Cheers >> >> >> >> Nigel >> >> >> >> --- >> >> Nigel W. Moriarty >> Building 33R0349, Molecular Biophysics and Integrated Bioimaging >> >> Lawrence Berkeley National Laboratory >> Berkeley, CA 94720-8235 >> Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov >> Fax : 510-486-5909 Web : CCI.LBL.gov >> >> >> >> >> >> On Mon, Aug 12, 2019 at 2:44 PM Robert Nolte <robert.t.nolte(a)gsk.com> >> wrote: >> >> Nigel, >> >> Currently it is not running. This was working from the scratch >> install last week. I am not sure what is going on now. >> >> Bob >> >> >> >> >> >> *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> >> *Sent:* Monday, August 12, 2019 4:52 PM >> *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> >> *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; >> cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> Does molprobity.reduce run? >> >> >> Cheers >> >> >> >> Nigel >> >> >> >> --- >> >> Nigel W. Moriarty >> Building 33R0349, Molecular Biophysics and Integrated Bioimaging >> >> Lawrence Berkeley National Laboratory >> Berkeley, CA 94720-8235 >> Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov >> Fax : 510-486-5909 Web : CCI.LBL.gov >> >> >> >> >> >> On Mon, Aug 12, 2019 at 1:14 PM Charitha Belluru Shetty < >> charitha-belluru.b.shetty(a)gsk.com> wrote: >> >> Hi Nigel, >> >> >> >> I hit the send button too soon..Ignore my previous email 😊 >> >> >> >> -bash-4.2$ source setpaths.sh >> >> -bash-4.2$ >> >> -bash-4.2$ env |grep -i reduce >> >> -bash-4.2$ >> >> >> >> So the “setpath.sh” has the below contents… >> >> >> >> $ cat /hpc/apps/current/molprobity/v4.4.app/MolProbity/build/setpaths.sh >> >> # THIS IS AN AUTOMATICALLY GENERATED FILE. >> >> # DO NOT EDIT! CHANGES WILL BE LOST. >> >> ocwd="`pwd`" >> >> if [ -n "$LIBTBX_BUILD_RELOCATION_HINT" ]; then >> >> cd "$LIBTBX_BUILD_RELOCATION_HINT" >> >> LIBTBX_BUILD_RELOCATION_HINT= >> >> export LIBTBX_BUILD_RELOCATION_HINT >> >> elif [ -n "$BASH_SOURCE" ]; then >> >> LIBTBX_BUILD=`dirname "$BASH_SOURCE[0]"` >> >> cd "$LIBTBX_BUILD" >> >> else >> >> cd "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" >> >> fi >> >> LIBTBX_BUILD=`pwd -P` >> >> export LIBTBX_BUILD >> >> LIBTBX_OPATH="$PATH" >> >> export LIBTBX_OPATH >> >> PATH="$LIBTBX_BUILD/bin:$PATH" >> >> export PATH >> >> cd "$ocwd" >> >> ocwd= >> >> alias libtbx.setpaths_all=". \"$LIBTBX_BUILD/setpaths_all.sh\"" >> >> alias libtbx.unsetpaths=". \"$LIBTBX_BUILD/unsetpaths.sh\"" >> >> if [ -n "$LIBTBX_OPATH" ]; then >> >> LIBTBX_TMPVAL="$LIBTBX_OPATH" >> >> else >> >> LIBTBX_TMPVAL= >> >> fi >> >> export LIBTBX_TMPVAL >> >> PATH=`libtbx.path_utility prepend LIBTBX_TMPVAL "$LIBTBX_BUILD/bin" < >> /dev/null` >> >> export PATH >> >> if [ "$PATH" = "L_I_B_T_B_X_E_M_P_T_Y" ]; then unset PATH; fi >> >> LIBTBX_TMPVAL= >> >> LIBTBX_OPATH= >> >> LIBTBX_BUILD= >> >> >> >> >> >> Best Regards, >> >> Charitha Shetty >> >> >> >> *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> >> *Sent:* Monday, August 12, 2019 4:08 PM >> *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> >> *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; >> cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> Once you source the setpaths.sh file in the build directory do you have >> phenix.reduce or molprobity.reduce? >> >> Cheers >> >> >> >> Nigel >> >> >> >> --- >> >> Nigel W. Moriarty >> Building 33R0349, Molecular Biophysics and Integrated Bioimaging >> >> Lawrence Berkeley National Laboratory >> Berkeley, CA 94720-8235 >> Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov >> Fax : 510-486-5909 Web : CCI.LBL.gov >> >> >> >> >> >> On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < >> charitha-belluru.b.shetty(a)gsk.com> wrote: >> >> Hi Christopher/ Nigel, >> >> >> >> Thank you for your quick reply. >> >> >> >> The build process was completed without any error. Pasting few of the >> lines from it… >> >> >> >> >> >> ===== Running in build: rebuild rotarama >> >> Processing data files in >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": >> >> rota8000-cys.data -> rota8000-cys.pickle: converting ... done. >> >> rota8000-asp.data -> rota8000-asp.pickle: converting ... done. >> >> rota8000-ser.data -> rota8000-ser.pickle: converting ... done. >> >> rota8000-gln.data -> rota8000-gln.pickle: converting ... done. >> >> rota8000-ile.data -> rota8000-ile.pickle: converting ... done. >> >> rota8000-pro.data -> rota8000-pro.pickle: converting ... done. >> >> rota8000-lys.data -> rota8000-lys.pickle: converting ... done. >> >> rota8000-thr.data -> rota8000-thr.pickle: converting ... done. >> >> rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. >> >> rota8000-glu.data -> rota8000-glu.pickle: converting ... done. >> >> rota8000-met.data -> rota8000-met.pickle: converting ... done. >> >> rota8000-his.data -> rota8000-his.pickle: converting ... done. >> >> rota8000-leu.data -> rota8000-leu.pickle: converting ... done. >> >> rota8000-arg.data -> rota8000-arg.pickle: converting ... done. >> >> rota8000-trp.data -> rota8000-trp.pickle: converting ... done. >> >> rota8000-asn.data -> rota8000-asn.pickle: converting ... done. >> >> rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. >> >> rota8000-val.data -> rota8000-val.pickle: converting ... done. >> >> Processing data files in >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": >> >> rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: >> converting ... done. >> >> rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. >> >> rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. >> >> rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: >> converting ... done. >> >> rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. >> >> rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting >> ... done. >> >> u+s,u,s: 10.06 9.83 0.23 >> >> ===== Running in .: libtbx.refresh >> >> Python: 2.7.16 >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" >> >> Compiler: default >> >> Build mode: release >> >> Warning level: 0 >> >> Precompiled Headers: False >> >> Static libraries: False >> >> Static exe: False >> >> Scan Boost headers: False >> >> Write full flex_fwd.h files: False >> >> Build Boost.Python extensions: True >> >> Define BOOST_PYTHON_NO_PY_SIGNATURES: False >> >> Define BOOST_PYTHON_BOOL_INT_STRICT: True >> >> Enable OpenMP if possible: False >> >> Boost threads enabled: True >> >> Enable CUDA: False >> >> Use conda: False >> >> Use opt_resources if available: False >> >> Use environment flags: False >> >> Enable C++11: False >> >> Python3 migration warning policy: None >> >> command_version_suffix: None >> >> Relocatable paths anchored at: >> /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build >> >> Top-down list of all modules involved: >> >> mmtbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" >> >> iotbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" >> >> ccp4io+adaptbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" >> >> smtbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" >> >> ucif >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" >> >> cctbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" >> >> scitbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" >> >> fable >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" >> >> omptbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" >> >> boost+adaptbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" >> >> tbxx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" >> >> chiltbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" >> >> libtbx >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" >> >> Creating files in build directory: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" >> >> dispatcher_include_template.sh >> >> dispatcher_head.sh >> >> setpaths.sh >> >> setpaths.csh >> >> setpaths_all.sh >> >> setpaths_all.csh >> >> setpaths_debug.sh >> >> unsetpaths.sh >> >> setpaths_debug.csh >> >> unsetpaths.csh >> >> SConstruct >> >> Makefile >> >> run_tests.csh >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" >> >> Generating C++ header files in: >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" >> >> Generating C++ files in: >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" >> >> Using fable to convert scitbx/lbfgs.f >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" >> >> Generating C++ files in: >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" >> >> Generating C++ files in: >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" >> >> Generating C++ files in: >> >> >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" >> >> Using fable to convert iotbx/pdb/hybrid_36_f.f >> >> Writing: iotbx/pdb/hybrid_36_fem.cpp >> >> Processing: >> "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" >> >> Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" >> >> Regenerating module files in >> /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base >> >> /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, >> could not regenerate pango files >> >> /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not >> present, could not regenerate gdk-pixbuf.loaders >> >> /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, >> could not rebuild fonts >> >> generating index.theme file >> >> Done. >> >> >> >> *Bootstrap success: build* >> >> *++++++++++ MolProbity configure.sh finished.* >> >> >> >> >> >> >> >> [tm375568@uptus060-3 MolProbity]$ ./setup.sh >> >> ++++++++++ creating directories ... >> >> ++++++++++ setting permisions ... >> >> *Couldn't get a file descriptor referring to the console **à I assume >> this can be ignored. * >> >> [tm375568@uptus060-3 MolProbity]$ >> >> >> >> Best Regards, >> >> Charitha Shetty >> >> >> >> *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> >> *Sent:* Monday, August 12, 2019 3:34 PM >> *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> >> *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; >> cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> >> *Subject:* Re: Molprobity / CCTBX incompatibility >> >> >> >> *EXTERNAL* >> >> Hmm. It looks like Reduce and/or Probe has failed to install correctly. >> Most likely an early call to Reduce has resulted in an empty file, which is >> then passed into other programs, resulting in error messages. >> >> >> >> The configure script ran a build process. Do you know if that process >> finished successfully or if it ended with errors? If the build exited >> early, it may have never reached the part where Reduce gets compiled. Some >> dependencies have changed, and I have been having trouble with my own >> builds lately. >> >> >> >> It is also possible that there was just a network hiccup that prevented >> something from downloading correctly, and a second attempt will work. >> >> >> >> I am working on a new configure/install script for more consistent >> handling of dependencies. Hopefully it will be ready in about a day. >> >> >> >> Good luck, >> >> -Christopher Williams >> >> ---Richardson Lab, Duke University >> >> >> >> >> >> >> >> >> >> >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >> *GSK monitors email communications sent to and from GSK in order to >> protect GSK, our employees, customers, suppliers and business partners, >> from cyber threats and loss of GSK Information. GSK monitoring is conducted >> with appropriate confidentiality controls and in accordance with local laws >> and after appropriate consultation.* >> >

1 0

Berkeley Server Downtime (8/16 - 8/18)
by Billy Poon 19 Aug '19

19 Aug '19

Dear all, On Friday, August 16, our servers will be shut down starting at 4:00 pm Pacific for scheduled maintenance in our building. We expect that the servers will be available by 10:00 am Pacific on Sunday (8/18) at the latest. This will affect, - The Phenix web site (http://phenix-online.org/) - The DIALS West web site (http://dials.lbl.gov/) - The CCI web site (http://cci.lbl.gov/) - The Structural Biology Technology Portal (http://technology.sbkb.org/) Bug reports and emails requesting help should be queued and we will receive them once the servers are back online. We apologize for any inconvenience. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 1

Disruptive and counterproductive comments
by Graeme.Winter＠Diamond.ac.uk 17 Aug '19

17 Aug '19

Deliberately starting a new thread My thinking was that this was feedback on why users find it hard getting started on cctbx. I had assumed, incorrectly as it turns out, that on the developer list a wider discussion was appropriate. This was not meant as criticism, rather direct feedback from talking to potential users at diamond. However I sense that it was not welcome so I’ll keep my thoughts to myself. I’m obviously aware of the work Billy and co are doing to help the situation with conda, but I’m not aware of the longer term intentions for this. Anyhow, sorry for interfering Best wishes Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Re: [cctbxbb] CCTBX documentation questionnaire
by ben.williams＠diamond.ac.uk 16 Aug '19

16 Aug '19

Dear Dorothee, Is there a reason for only allowing a single choice of library in the questionnaire? I've wanted to use several components from across the CCTBX project and would like my response to reflect that. I'm sure there are probably others in the same boat. Many thanks, Ben — Ben Williams Post-doctoral research associate Scientific Software Diamond Light Source On 15 Aug 2019 20:05, Dorothee Liebschner <dcliebschner(a)lbl.gov> wrote: Hi everyone, During this year's ECM computing school, it came up that the CCTBX does not have a lot of documentation. This seems to be a major reason why people don't use it, even if they would like to employ the libraries. We would like to find out which parts of the CCTBX are the most interesting for users. We certainly cannot provide extensive documentation for the entire toolbox, but we can see what we can do once we know the main areas of interest. So if you were in the situation of wanting to use the CCTBX but decided to stay away from it because the documentation was non-existent or obscure, please let us know by filling out the questionnaire below (it's anonymous): questionnaire CCTBX documentation<https://forms.gle/9RawJFfHyEvhCVVP7> Best wishes, Dorothee -- Project Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org<https://phenix-online.org/> -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 1

cctbx azure pipelines skipping tests if clutter
by richard.gildea＠diamond.ac.uk 15 Aug '19

15 Aug '19

Hi, I noticed that the cctbx azure pipelines run libtbx.find_clutter as part of the Python "syntax" jobs, and if they find any "clutter", then the pipeline immediately fails and doesn't run any further tests (e.g. https://dev.azure.com/cctbx/cctbx_project/_build/results?buildId=374). This raises the following questions: 1) Why skip running tests just because "clutter" has been found? 2) Why run libtbx.find_clutter as part of the "syntax" job, as strictly speaking trailing whitespaces, unused imports etc, aren't syntax issues? 3) If finding clutter is so important, why not add an optional pre-commit hook to help prevent "clutter" ever being committed in the first place? Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 4

Booststrap fails: zlib.h missing on Mojave with Xcode 10.3
by Luc Bourhis 14 Aug '19

14 Aug '19

Hi, I am trying to setup a Python 3 cctbx. I did python bootstrap.py --git-ssh --python3 --builder cctbxlite But it fails at the installation of OpenSSL. Clang version is "Apple LLVM version 10.0.1 (clang-1001.0.46.4)”. I attached the OpenSSL install log but I don’t see any error in it. And the advice to run xcode-select is bogus: I have it already setup. Best wishes, Luc Installing OpenSSL... log file is /Users/luc/Developer/cctbx/base_tmp/OpenSSL_install_log getting package openssl-1.0.2r.tar.gz... downloading from https://gitcdn.link/repo/dials/dependencies/master/openssl-1.0.2r.tar.gz : 5223 kB installing openssl-1.0.2r.tar.gz... package openssl took 219.5s to install -------------------------------------------------------------------------------- zlib.h missing -- try running 'xcode-select --install' first Process failed with return code 1

3 6

Clipper updates for C++11
by Billy Poon 13 Aug '19

13 Aug '19

Hi everyone, As part of making CCTBX-based software compile with C++11, it looks like we need to update our checkout of clipper to the latest version. http://www.ysbl.york.ac.uk/~cowtan/clipper/clipper.html As a side note, is this a feature that is widely used? If not, we can disable the downloading of the clipper source files. Also, in addition to this post on the bulletin board, I have started an issue on Github (https://github.com/cctbx/cctbx_project/issues/379). Feel free to comment on any of these forums. -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 13 Aug '19

13 Aug '19

But maybe the installation of cctbx has molprobity.reduce and not phenix.reduce so if that is the case we can change the script to look for molprobity.reduce instead of phenix.reduce. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 12, 2019 at 2:58 PM Robert Nolte <robert.t.nolte(a)gsk.com> wrote: > According to the log file when I just tried to run it, it is phenix.reduce > that it can’t find. > > Bob > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 5:55 PM > *To:* Robert Nolte <robert.t.nolte(a)gsk.com> > *Cc:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com>; > Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > I'm just trying to find out if, at the command line, whether it is > phenix.reduce or molprobity.reduce. Charitha just needs to type one or the > other to see what happens. > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 2:44 PM Robert Nolte <robert.t.nolte(a)gsk.com> > wrote: > > Nigel, > > Currently it is not running. This was working from the scratch > install last week. I am not sure what is going on now. > > Bob > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:52 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Does molprobity.reduce run? > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 1:14 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Nigel, > > > > I hit the send button too soon..Ignore my previous email 😊 > > > > -bash-4.2$ source setpaths.sh > > -bash-4.2$ > > -bash-4.2$ env |grep -i reduce > > -bash-4.2$ > > > > So the “setpath.sh” has the below contents… > > > > $ cat /hpc/apps/current/molprobity/v4.4.app/MolProbity/build/setpaths.sh > > # THIS IS AN AUTOMATICALLY GENERATED FILE. > > # DO NOT EDIT! CHANGES WILL BE LOST. > > ocwd="`pwd`" > > if [ -n "$LIBTBX_BUILD_RELOCATION_HINT" ]; then > > cd "$LIBTBX_BUILD_RELOCATION_HINT" > > LIBTBX_BUILD_RELOCATION_HINT= > > export LIBTBX_BUILD_RELOCATION_HINT > > elif [ -n "$BASH_SOURCE" ]; then > > LIBTBX_BUILD=`dirname "$BASH_SOURCE[0]"` > > cd "$LIBTBX_BUILD" > > else > > cd "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > fi > > LIBTBX_BUILD=`pwd -P` > > export LIBTBX_BUILD > > LIBTBX_OPATH="$PATH" > > export LIBTBX_OPATH > > PATH="$LIBTBX_BUILD/bin:$PATH" > > export PATH > > cd "$ocwd" > > ocwd= > > alias libtbx.setpaths_all=". \"$LIBTBX_BUILD/setpaths_all.sh\"" > > alias libtbx.unsetpaths=". \"$LIBTBX_BUILD/unsetpaths.sh\"" > > if [ -n "$LIBTBX_OPATH" ]; then > > LIBTBX_TMPVAL="$LIBTBX_OPATH" > > else > > LIBTBX_TMPVAL= > > fi > > export LIBTBX_TMPVAL > > PATH=`libtbx.path_utility prepend LIBTBX_TMPVAL "$LIBTBX_BUILD/bin" < > /dev/null` > > export PATH > > if [ "$PATH" = "L_I_B_T_B_X_E_M_P_T_Y" ]; then unset PATH; fi > > LIBTBX_TMPVAL= > > LIBTBX_OPATH= > > LIBTBX_BUILD= > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:08 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Once you source the setpaths.sh file in the build directory do you have > phenix.reduce or molprobity.reduce? > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Christopher/ Nigel, > > > > Thank you for your quick reply. > > > > The build process was completed without any error. Pasting few of the > lines from it… > > > > > > ===== Running in build: rebuild rotarama > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rota8000-cys.data -> rota8000-cys.pickle: converting ... done. > > rota8000-asp.data -> rota8000-asp.pickle: converting ... done. > > rota8000-ser.data -> rota8000-ser.pickle: converting ... done. > > rota8000-gln.data -> rota8000-gln.pickle: converting ... done. > > rota8000-ile.data -> rota8000-ile.pickle: converting ... done. > > rota8000-pro.data -> rota8000-pro.pickle: converting ... done. > > rota8000-lys.data -> rota8000-lys.pickle: converting ... done. > > rota8000-thr.data -> rota8000-thr.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. > > rota8000-glu.data -> rota8000-glu.pickle: converting ... done. > > rota8000-met.data -> rota8000-met.pickle: converting ... done. > > rota8000-his.data -> rota8000-his.pickle: converting ... done. > > rota8000-leu.data -> rota8000-leu.pickle: converting ... done. > > rota8000-arg.data -> rota8000-arg.pickle: converting ... done. > > rota8000-trp.data -> rota8000-trp.pickle: converting ... done. > > rota8000-asn.data -> rota8000-asn.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. > > rota8000-val.data -> rota8000-val.pickle: converting ... done. > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: converting > ... done. > > rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. > > rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. > > rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: > converting ... done. > > rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. > > rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting ... > done. > > u+s,u,s: 10.06 9.83 0.23 > > ===== Running in .: libtbx.refresh > > Python: 2.7.16 > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" > > Compiler: default > > Build mode: release > > Warning level: 0 > > Precompiled Headers: False > > Static libraries: False > > Static exe: False > > Scan Boost headers: False > > Write full flex_fwd.h files: False > > Build Boost.Python extensions: True > > Define BOOST_PYTHON_NO_PY_SIGNATURES: False > > Define BOOST_PYTHON_BOOL_INT_STRICT: True > > Enable OpenMP if possible: False > > Boost threads enabled: True > > Enable CUDA: False > > Use conda: False > > Use opt_resources if available: False > > Use environment flags: False > > Enable C++11: False > > Python3 migration warning policy: None > > command_version_suffix: None > > Relocatable paths anchored at: > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build > > Top-down list of all modules involved: > > mmtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" > > iotbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" > > ccp4io+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" > > smtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" > > ucif > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" > > cctbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" > > scitbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" > > fable > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" > > omptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" > > boost+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" > > tbxx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" > > chiltbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" > > libtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" > > Creating files in build directory: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > dispatcher_include_template.sh > > dispatcher_head.sh > > setpaths.sh > > setpaths.csh > > setpaths_all.sh > > setpaths_all.csh > > setpaths_debug.sh > > unsetpaths.sh > > setpaths_debug.csh > > unsetpaths.csh > > SConstruct > > Makefile > > run_tests.csh > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" > > Generating C++ header files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" > > Using fable to convert scitbx/lbfgs.f > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" > > Generating C++ files in: > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" > > Using fable to convert iotbx/pdb/hybrid_36_f.f > > Writing: iotbx/pdb/hybrid_36_fem.cpp > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" > > Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" > > Regenerating module files in > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, > could not regenerate pango files > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not > present, could not regenerate gdk-pixbuf.loaders > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, > could not rebuild fonts > > generating index.theme file > > Done. > > > > *Bootstrap success: build* > > *++++++++++ MolProbity configure.sh finished.* > > > > > > > > [tm375568@uptus060-3 MolProbity]$ ./setup.sh > > ++++++++++ creating directories ... > > ++++++++++ setting permisions ... > > *Couldn't get a file descriptor referring to the console **à I assume > this can be ignored. * > > [tm375568@uptus060-3 MolProbity]$ > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 3:34 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hmm. It looks like Reduce and/or Probe has failed to install correctly. > Most likely an early call to Reduce has resulted in an empty file, which is > then passed into other programs, resulting in error messages. > > > > The configure script ran a build process. Do you know if that process > finished successfully or if it ended with errors? If the build exited > early, it may have never reached the part where Reduce gets compiled. Some > dependencies have changed, and I have been having trouble with my own > builds lately. > > > > It is also possible that there was just a network hiccup that prevented > something from downloading correctly, and a second attempt will work. > > > > I am working on a new configure/install script for more consistent > handling of dependencies. Hopefully it will be ready in about a day. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 13 Aug '19

13 Aug '19

I'm just trying to find out if, at the command line, whether it is phenix.reduce or molprobity.reduce. Charitha just needs to type one or the other to see what happens. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 12, 2019 at 2:44 PM Robert Nolte <robert.t.nolte(a)gsk.com> wrote: > Nigel, > > Currently it is not running. This was working from the scratch > install last week. I am not sure what is going on now. > > Bob > > > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:52 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Does molprobity.reduce run? > > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 1:14 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Nigel, > > > > I hit the send button too soon..Ignore my previous email 😊 > > > > -bash-4.2$ source setpaths.sh > > -bash-4.2$ > > -bash-4.2$ env |grep -i reduce > > -bash-4.2$ > > > > So the “setpath.sh” has the below contents… > > > > $ cat /hpc/apps/current/molprobity/v4.4.app/MolProbity/build/setpaths.sh > > # THIS IS AN AUTOMATICALLY GENERATED FILE. > > # DO NOT EDIT! CHANGES WILL BE LOST. > > ocwd="`pwd`" > > if [ -n "$LIBTBX_BUILD_RELOCATION_HINT" ]; then > > cd "$LIBTBX_BUILD_RELOCATION_HINT" > > LIBTBX_BUILD_RELOCATION_HINT= > > export LIBTBX_BUILD_RELOCATION_HINT > > elif [ -n "$BASH_SOURCE" ]; then > > LIBTBX_BUILD=`dirname "$BASH_SOURCE[0]"` > > cd "$LIBTBX_BUILD" > > else > > cd "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > fi > > LIBTBX_BUILD=`pwd -P` > > export LIBTBX_BUILD > > LIBTBX_OPATH="$PATH" > > export LIBTBX_OPATH > > PATH="$LIBTBX_BUILD/bin:$PATH" > > export PATH > > cd "$ocwd" > > ocwd= > > alias libtbx.setpaths_all=". \"$LIBTBX_BUILD/setpaths_all.sh\"" > > alias libtbx.unsetpaths=". \"$LIBTBX_BUILD/unsetpaths.sh\"" > > if [ -n "$LIBTBX_OPATH" ]; then > > LIBTBX_TMPVAL="$LIBTBX_OPATH" > > else > > LIBTBX_TMPVAL= > > fi > > export LIBTBX_TMPVAL > > PATH=`libtbx.path_utility prepend LIBTBX_TMPVAL "$LIBTBX_BUILD/bin" < > /dev/null` > > export PATH > > if [ "$PATH" = "L_I_B_T_B_X_E_M_P_T_Y" ]; then unset PATH; fi > > LIBTBX_TMPVAL= > > LIBTBX_OPATH= > > LIBTBX_BUILD= > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:08 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Once you source the setpaths.sh file in the build directory do you have > phenix.reduce or molprobity.reduce? > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Christopher/ Nigel, > > > > Thank you for your quick reply. > > > > The build process was completed without any error. Pasting few of the > lines from it… > > > > > > ===== Running in build: rebuild rotarama > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rota8000-cys.data -> rota8000-cys.pickle: converting ... done. > > rota8000-asp.data -> rota8000-asp.pickle: converting ... done. > > rota8000-ser.data -> rota8000-ser.pickle: converting ... done. > > rota8000-gln.data -> rota8000-gln.pickle: converting ... done. > > rota8000-ile.data -> rota8000-ile.pickle: converting ... done. > > rota8000-pro.data -> rota8000-pro.pickle: converting ... done. > > rota8000-lys.data -> rota8000-lys.pickle: converting ... done. > > rota8000-thr.data -> rota8000-thr.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. > > rota8000-glu.data -> rota8000-glu.pickle: converting ... done. > > rota8000-met.data -> rota8000-met.pickle: converting ... done. > > rota8000-his.data -> rota8000-his.pickle: converting ... done. > > rota8000-leu.data -> rota8000-leu.pickle: converting ... done. > > rota8000-arg.data -> rota8000-arg.pickle: converting ... done. > > rota8000-trp.data -> rota8000-trp.pickle: converting ... done. > > rota8000-asn.data -> rota8000-asn.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. > > rota8000-val.data -> rota8000-val.pickle: converting ... done. > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: converting > ... done. > > rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. > > rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. > > rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: > converting ... done. > > rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. > > rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting ... > done. > > u+s,u,s: 10.06 9.83 0.23 > > ===== Running in .: libtbx.refresh > > Python: 2.7.16 > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" > > Compiler: default > > Build mode: release > > Warning level: 0 > > Precompiled Headers: False > > Static libraries: False > > Static exe: False > > Scan Boost headers: False > > Write full flex_fwd.h files: False > > Build Boost.Python extensions: True > > Define BOOST_PYTHON_NO_PY_SIGNATURES: False > > Define BOOST_PYTHON_BOOL_INT_STRICT: True > > Enable OpenMP if possible: False > > Boost threads enabled: True > > Enable CUDA: False > > Use conda: False > > Use opt_resources if available: False > > Use environment flags: False > > Enable C++11: False > > Python3 migration warning policy: None > > command_version_suffix: None > > Relocatable paths anchored at: > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build > > Top-down list of all modules involved: > > mmtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" > > iotbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" > > ccp4io+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" > > smtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" > > ucif > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" > > cctbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" > > scitbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" > > fable > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" > > omptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" > > boost+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" > > tbxx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" > > chiltbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" > > libtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" > > Creating files in build directory: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > dispatcher_include_template.sh > > dispatcher_head.sh > > setpaths.sh > > setpaths.csh > > setpaths_all.sh > > setpaths_all.csh > > setpaths_debug.sh > > unsetpaths.sh > > setpaths_debug.csh > > unsetpaths.csh > > SConstruct > > Makefile > > run_tests.csh > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" > > Generating C++ header files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" > > Using fable to convert scitbx/lbfgs.f > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" > > Generating C++ files in: > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" > > Using fable to convert iotbx/pdb/hybrid_36_f.f > > Writing: iotbx/pdb/hybrid_36_fem.cpp > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" > > Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" > > Regenerating module files in > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, > could not regenerate pango files > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not > present, could not regenerate gdk-pixbuf.loaders > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, > could not rebuild fonts > > generating index.theme file > > Done. > > > > *Bootstrap success: build* > > *++++++++++ MolProbity configure.sh finished.* > > > > > > > > [tm375568@uptus060-3 MolProbity]$ ./setup.sh > > ++++++++++ creating directories ... > > ++++++++++ setting permisions ... > > *Couldn't get a file descriptor referring to the console **à I assume > this can be ignored. * > > [tm375568@uptus060-3 MolProbity]$ > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 3:34 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hmm. It looks like Reduce and/or Probe has failed to install correctly. > Most likely an early call to Reduce has resulted in an empty file, which is > then passed into other programs, resulting in error messages. > > > > The configure script ran a build process. Do you know if that process > finished successfully or if it ended with errors? If the build exited > early, it may have never reached the part where Reduce gets compiled. Some > dependencies have changed, and I have been having trouble with my own > builds lately. > > > > It is also possible that there was just a network hiccup that prevented > something from downloading correctly, and a second attempt will work. > > > > I am working on a new configure/install script for more consistent > handling of dependencies. Hopefully it will be ready in about a day. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 13 Aug '19

13 Aug '19

Does molprobity.reduce run? Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 12, 2019 at 1:14 PM Charitha Belluru Shetty < charitha-belluru.b.shetty(a)gsk.com> wrote: > Hi Nigel, > > > > I hit the send button too soon..Ignore my previous email 😊 > > > > -bash-4.2$ source setpaths.sh > > -bash-4.2$ > > -bash-4.2$ env |grep -i reduce > > -bash-4.2$ > > > > So the “setpath.sh” has the below contents… > > > > $ cat /hpc/apps/current/molprobity/v4.4.app/MolProbity/build/setpaths.sh > > # THIS IS AN AUTOMATICALLY GENERATED FILE. > > # DO NOT EDIT! CHANGES WILL BE LOST. > > ocwd="`pwd`" > > if [ -n "$LIBTBX_BUILD_RELOCATION_HINT" ]; then > > cd "$LIBTBX_BUILD_RELOCATION_HINT" > > LIBTBX_BUILD_RELOCATION_HINT= > > export LIBTBX_BUILD_RELOCATION_HINT > > elif [ -n "$BASH_SOURCE" ]; then > > LIBTBX_BUILD=`dirname "$BASH_SOURCE[0]"` > > cd "$LIBTBX_BUILD" > > else > > cd "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > fi > > LIBTBX_BUILD=`pwd -P` > > export LIBTBX_BUILD > > LIBTBX_OPATH="$PATH" > > export LIBTBX_OPATH > > PATH="$LIBTBX_BUILD/bin:$PATH" > > export PATH > > cd "$ocwd" > > ocwd= > > alias libtbx.setpaths_all=". \"$LIBTBX_BUILD/setpaths_all.sh\"" > > alias libtbx.unsetpaths=". \"$LIBTBX_BUILD/unsetpaths.sh\"" > > if [ -n "$LIBTBX_OPATH" ]; then > > LIBTBX_TMPVAL="$LIBTBX_OPATH" > > else > > LIBTBX_TMPVAL= > > fi > > export LIBTBX_TMPVAL > > PATH=`libtbx.path_utility prepend LIBTBX_TMPVAL "$LIBTBX_BUILD/bin" < > /dev/null` > > export PATH > > if [ "$PATH" = "L_I_B_T_B_X_E_M_P_T_Y" ]; then unset PATH; fi > > LIBTBX_TMPVAL= > > LIBTBX_OPATH= > > LIBTBX_BUILD= > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Nigel Moriarty <nwmoriarty(a)lbl.gov> > *Sent:* Monday, August 12, 2019 4:08 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Christopher Williams <christopher.sci.williams(a)gmail.com>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Once you source the setpaths.sh file in the build directory do you have > phenix.reduce or molprobity.reduce? > > Cheers > > > > Nigel > > > > --- > > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov > Fax : 510-486-5909 Web : CCI.LBL.gov > > > > > > On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < > charitha-belluru.b.shetty(a)gsk.com> wrote: > > Hi Christopher/ Nigel, > > > > Thank you for your quick reply. > > > > The build process was completed without any error. Pasting few of the > lines from it… > > > > > > ===== Running in build: rebuild rotarama > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rota8000-cys.data -> rota8000-cys.pickle: converting ... done. > > rota8000-asp.data -> rota8000-asp.pickle: converting ... done. > > rota8000-ser.data -> rota8000-ser.pickle: converting ... done. > > rota8000-gln.data -> rota8000-gln.pickle: converting ... done. > > rota8000-ile.data -> rota8000-ile.pickle: converting ... done. > > rota8000-pro.data -> rota8000-pro.pickle: converting ... done. > > rota8000-lys.data -> rota8000-lys.pickle: converting ... done. > > rota8000-thr.data -> rota8000-thr.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. > > rota8000-glu.data -> rota8000-glu.pickle: converting ... done. > > rota8000-met.data -> rota8000-met.pickle: converting ... done. > > rota8000-his.data -> rota8000-his.pickle: converting ... done. > > rota8000-leu.data -> rota8000-leu.pickle: converting ... done. > > rota8000-arg.data -> rota8000-arg.pickle: converting ... done. > > rota8000-trp.data -> rota8000-trp.pickle: converting ... done. > > rota8000-asn.data -> rota8000-asn.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. > > rota8000-val.data -> rota8000-val.pickle: converting ... done. > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: converting > ... done. > > rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. > > rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. > > rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: > converting ... done. > > rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. > > rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting ... > done. > > u+s,u,s: 10.06 9.83 0.23 > > ===== Running in .: libtbx.refresh > > Python: 2.7.16 > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" > > Compiler: default > > Build mode: release > > Warning level: 0 > > Precompiled Headers: False > > Static libraries: False > > Static exe: False > > Scan Boost headers: False > > Write full flex_fwd.h files: False > > Build Boost.Python extensions: True > > Define BOOST_PYTHON_NO_PY_SIGNATURES: False > > Define BOOST_PYTHON_BOOL_INT_STRICT: True > > Enable OpenMP if possible: False > > Boost threads enabled: True > > Enable CUDA: False > > Use conda: False > > Use opt_resources if available: False > > Use environment flags: False > > Enable C++11: False > > Python3 migration warning policy: None > > command_version_suffix: None > > Relocatable paths anchored at: > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build > > Top-down list of all modules involved: > > mmtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" > > iotbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" > > ccp4io+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" > > smtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" > > ucif > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" > > cctbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" > > scitbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" > > fable > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" > > omptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" > > boost+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" > > tbxx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" > > chiltbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" > > libtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" > > Creating files in build directory: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > dispatcher_include_template.sh > > dispatcher_head.sh > > setpaths.sh > > setpaths.csh > > setpaths_all.sh > > setpaths_all.csh > > setpaths_debug.sh > > unsetpaths.sh > > setpaths_debug.csh > > unsetpaths.csh > > SConstruct > > Makefile > > run_tests.csh > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" > > Generating C++ header files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" > > Using fable to convert scitbx/lbfgs.f > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" > > Generating C++ files in: > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" > > Using fable to convert iotbx/pdb/hybrid_36_f.f > > Writing: iotbx/pdb/hybrid_36_fem.cpp > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" > > Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" > > Regenerating module files in > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, > could not regenerate pango files > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not > present, could not regenerate gdk-pixbuf.loaders > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, > could not rebuild fonts > > generating index.theme file > > Done. > > > > *Bootstrap success: build* > > *++++++++++ MolProbity configure.sh finished.* > > > > > > > > [tm375568@uptus060-3 MolProbity]$ ./setup.sh > > ++++++++++ creating directories ... > > ++++++++++ setting permisions ... > > *Couldn't get a file descriptor referring to the console **à I assume > this can be ignored. * > > [tm375568@uptus060-3 MolProbity]$ > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 3:34 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hmm. It looks like Reduce and/or Probe has failed to install correctly. > Most likely an early call to Reduce has resulted in an empty file, which is > then passed into other programs, resulting in error messages. > > > > The configure script ran a build process. Do you know if that process > finished successfully or if it ended with errors? If the build exited > early, it may have never reached the part where Reduce gets compiled. Some > dependencies have changed, and I have been having trouble with my own > builds lately. > > > > It is also possible that there was just a network hiccup that prevented > something from downloading correctly, and a second attempt will work. > > > > I am working on a new configure/install script for more consistent > handling of dependencies. Hopefully it will be ready in about a day. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 13 Aug '19

13 Aug '19

Once you source the setpaths.sh file in the build directory do you have phenix.reduce or molprobity.reduce? Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 12, 2019 at 12:43 PM Charitha Belluru Shetty < charitha-belluru.b.shetty(a)gsk.com> wrote: > Hi Christopher/ Nigel, > > > > Thank you for your quick reply. > > > > The build process was completed without any error. Pasting few of the > lines from it… > > > > > > ===== Running in build: rebuild rotarama > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rota8000-cys.data -> rota8000-cys.pickle: converting ... done. > > rota8000-asp.data -> rota8000-asp.pickle: converting ... done. > > rota8000-ser.data -> rota8000-ser.pickle: converting ... done. > > rota8000-gln.data -> rota8000-gln.pickle: converting ... done. > > rota8000-ile.data -> rota8000-ile.pickle: converting ... done. > > rota8000-pro.data -> rota8000-pro.pickle: converting ... done. > > rota8000-lys.data -> rota8000-lys.pickle: converting ... done. > > rota8000-thr.data -> rota8000-thr.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: converting ... done. > > rota8000-glu.data -> rota8000-glu.pickle: converting ... done. > > rota8000-met.data -> rota8000-met.pickle: converting ... done. > > rota8000-his.data -> rota8000-his.pickle: converting ... done. > > rota8000-leu.data -> rota8000-leu.pickle: converting ... done. > > rota8000-arg.data -> rota8000-arg.pickle: converting ... done. > > rota8000-trp.data -> rota8000-trp.pickle: converting ... done. > > rota8000-asn.data -> rota8000-asn.pickle: converting ... done. > > rota8000-phetyr.data -> rota8000-phetyr.pickle: already up to date. > > rota8000-val.data -> rota8000-val.pickle: converting ... done. > > Processing data files in > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/chem_data/rotarama_data": > > rama8000-ileval-nopreP.data -> rama8000-ileval-nopreP.pickle: converting > ... done. > > rama8000-gly-sym.data -> rama8000-gly-sym.pickle: converting ... done. > > rama8000-cispro.data -> rama8000-cispro.pickle: converting ... done. > > rama8000-general-noGPIVpreP.data -> rama8000-general-noGPIVpreP.pickle: > converting ... done. > > rama8000-transpro.data -> rama8000-transpro.pickle: converting ... done. > > rama8000-prepro-noGP.data -> rama8000-prepro-noGP.pickle: converting ... > done. > > u+s,u,s: 10.06 9.83 0.23 > > ===== Running in .: libtbx.refresh > > Python: 2.7.16 > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/bin/python2.7" > > Compiler: default > > Build mode: release > > Warning level: 0 > > Precompiled Headers: False > > Static libraries: False > > Static exe: False > > Scan Boost headers: False > > Write full flex_fwd.h files: False > > Build Boost.Python extensions: True > > Define BOOST_PYTHON_NO_PY_SIGNATURES: False > > Define BOOST_PYTHON_BOOL_INT_STRICT: True > > Enable OpenMP if possible: False > > Boost threads enabled: True > > Enable CUDA: False > > Use conda: False > > Use opt_resources if available: False > > Use environment flags: False > > Enable C++11: False > > Python3 migration warning policy: None > > command_version_suffix: None > > Relocatable paths anchored at: > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/build > > Top-down list of all modules involved: > > mmtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx" > > iotbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx" > > ccp4io+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/ccp4io_adaptbx" > > smtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx" > > ucif > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/ucif" > > cctbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx" > > scitbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx" > > fable > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable" > > omptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/omptbx" > > boost+adaptbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/boost" > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx" > > tbxx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/tbxx" > > chiltbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/chiltbx" > > libtbx > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx" > > Creating files in build directory: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build" > > dispatcher_include_template.sh > > dispatcher_head.sh > > setpaths.sh > > setpaths.csh > > setpaths_all.sh > > setpaths_all.csh > > setpaths_debug.sh > > unsetpaths.sh > > setpaths_debug.csh > > unsetpaths.csh > > SConstruct > > Makefile > > run_tests.csh > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/libtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/fable/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/command_line" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/boost_adaptbx/libtbx_refresh.py" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/scitbx/libtbx_refresh.py" > > Generating C++ header files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/scitbx/array_family/boost_python" > > Using fable to convert scitbx/lbfgs.f > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/cctbx/libtbx_refresh.py" > > Generating C++ files in: > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/cctbx/eltbx" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/cctbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/smtbx/libtbx_refresh.py" > > Generating C++ files in: > > > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/include/smtbx/boost_python" > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/iotbx/libtbx_refresh.py" > > Using fable to convert iotbx/pdb/hybrid_36_f.f > > Writing: iotbx/pdb/hybrid_36_fem.cpp > > Processing: > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/libtbx_refresh.py" > > Processing: "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/exe_dev" > > Regenerating module files in > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/pango not present, > could not regenerate pango files > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/gtk-2.0 not > present, could not regenerate gdk-pixbuf.loaders > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/base/etc/fonts not present, > could not rebuild fonts > > generating index.theme file > > Done. > > > > *Bootstrap success: build* > > *++++++++++ MolProbity configure.sh finished.* > > > > > > > > [tm375568@uptus060-3 MolProbity]$ ./setup.sh > > ++++++++++ creating directories ... > > ++++++++++ setting permisions ... > > *Couldn't get a file descriptor referring to the console **à** I assume > this can be ignored.* > > [tm375568@uptus060-3 MolProbity]$ > > > > Best Regards, > > Charitha Shetty > > > > *From:* Christopher Williams <christopher.sci.williams(a)gmail.com> > *Sent:* Monday, August 12, 2019 3:34 PM > *To:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Cc:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org; Robert Nolte <robert.t.nolte(a)gsk.com> > *Subject:* Re: Molprobity / CCTBX incompatibility > > > > *EXTERNAL* > > Hmm. It looks like Reduce and/or Probe has failed to install correctly. > Most likely an early call to Reduce has resulted in an empty file, which is > then passed into other programs, resulting in error messages. > > > > The configure script ran a build process. Do you know if that process > finished successfully or if it ended with errors? If the build exited > early, it may have never reached the part where Reduce gets compiled. Some > dependencies have changed, and I have been having trouble with my own > builds lately. > > > > It is also possible that there was just a network hiccup that prevented > something from downloading correctly, and a second attempt will work. > > > > I am working on a new configure/install script for more consistent > handling of dependencies. Hopefully it will be ready in about a day. > > > > Good luck, > > -Christopher Williams > > ---Richardson Lab, Duke University > > > > > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Re: [cctbxbb] Molprobity / CCTBX incompatibility
by Nigel Moriarty 13 Aug '19

13 Aug '19

Not sure about the selection problem but the missing commands are likely molprobity.reduce and molprobity.probe. Chris, you should be able to change that. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov On Mon, Aug 12, 2019 at 11:43 AM Charitha Belluru Shetty < charitha-belluru.b.shetty(a)gsk.com> wrote: > Hi Christopher/Nigel, > > > > We downloaded the molprobity after the patch and ran the configure.sh and > setup scripts. Now things got worst. > > > > Can you please help us on this. > > > > Here is the output we are getting. > > > > > > > > > > > > > > When we press the *“Click here”* button, below log file is generated. > > > > [PLACE YOUR COMMENTS HERE] > > > > ================================================== > > USER / SERVER INFORMATION > > ================================================== > > User IP : 11.27.95.128 > > User browser : Mozilla/5.0 (Windows NT 10.0; Win64; x64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36 > > Session ID : c2h8n399dagb0ggfn8kpla6fu4 > > Session URL : > http://uptus002vm.corpnet2.com/molprobity/index.php?MolProbSID=c2h8n399dagb… > > > > MolProb. ver. : 4.4 > > Server name : uptus002vm.corpnet2.com > > Server time : 2019 Aug 12 18:24:41 > > > > ================================================== > > ERRORS FROM THIS SESSION > > ================================================== > > > > > > sh: phenix.reduce: command not found > > sh: phenix.reduce: command not found > > sh: phenix.reduce: command not found > > sh: phenix.probe: command not found > > sh: phenix.probe: command not found > > sh: phenix.reduce: command not found > > sh: phenix.reduce: command not found > > sh: phenix.probe: command not found > > sh: phenix.probe: command not found > > > > Starting phenix.clashscore > > on Mon Aug 12 13:50:31 2019 by appmgmt > > > =============================================================================== > > > > Processing files: > > > ------------------------------------------------------------------------------- > > > > Files not used by program: > > -------------------------- > > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/public_html/data/c2h8n399dagb0ggfn8kpla6fu4/coordinates/1irkFH.pdb > > > > Processing PHIL parameters: > > > ------------------------------------------------------------------------------- > > > > Adding command-line PHIL: > > ------------------------- > > b_factor_cutoff=40 > > clash_cutoff=-0.4 > > > > Final processed PHIL parameters: > > > ------------------------------------------------------------------------------- > > data_manager { > > default_output_filename = "cctbx_program" > > } > > b_factor_cutoff = 40 > > > > > > Starting job > > > =============================================================================== > > Sorry: 0 model(s) found. Expected at least 1. > > chartClashlist ran for 5 seconds > > do Charts ran for 0 seconds > > sh: phenix.probe: command not found > > do Kinemage ran for 0 seconds > > > > Traceback (most recent call last): > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/../modules/cctbx_project/mmtbx/command_line/mp_geo.py", > line 7, in <module> > > mp_geo.run(sys.argv[1:]) > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/validation/molprobity/mp_geo.py", > line 132, in run > > pdb_hierarchy.models()[0].chains(): > > IndexError: list index out of range > > > > > > > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.probe: command not found > > > > > > > > Starting phenix.clashscore > > > > on Mon Aug 12 13:50:31 2019 by appmgmt > > > > > =============================================================================== > > > > > > > > Processing files: > > > > > ------------------------------------------------------------------------------- > > > > > > > > Files not used by program: > > > > -------------------------- > > > > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/public_html/data/c2h8n399dagb0ggfn8kpla6fu4/coordinates/1irkFH.pdb > > > > > > > > Processing PHIL parameters: > > > > > ------------------------------------------------------------------------------- > > > > > > > > Adding command-line PHIL: > > > > ------------------------- > > > > b_factor_cutoff=40 > > > > clash_cutoff=-0.4 > > > > > > > > Final processed PHIL parameters: > > > > > ------------------------------------------------------------------------------- > > > > data_manager { > > > > default_output_filename = "cctbx_program" > > > > } > > > > b_factor_cutoff = 40 > > > > > > > > > > > > Starting job > > > > > =============================================================================== > > > > Sorry: 0 model(s) found. Expected at least 1. > > > > chartClashlist ran for 5 seconds > > > > do Charts ran for 0 seconds > > > > sh: phenix.probe: command not found > > > > do Kinemage ran for 0 seconds > > > > > > > > Traceback (most recent call last): > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/../modules/cctbx_project/mmtbx/command_line/mp_geo.py", > line 7, in <module> > > > > mp_geo.run(sys.argv[1:]) > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/validation/molprobity/mp_geo.py", > line 132, in run > > > > pdb_hierarchy.models()[0].chains(): > > > > IndexError: list index out of range > > > > > > > > > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.reduce: command not found > > > > sh: phenix.probe: command not found > > > > sh: phenix.probe: command not found > > > > > > > > Starting phenix.clashscore > > > > on Mon Aug 12 13:50:31 2019 by appmgmt > > > > > =============================================================================== > > > > > > > > Processing files: > > > > > ------------------------------------------------------------------------------- > > > > > > > > Files not used by program: > > > > -------------------------- > > > > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/public_html/data/c2h8n399dagb0ggfn8kpla6fu4/coordinates/1irkFH.pdb > > > > > > > > Processing PHIL parameters: > > > > > ------------------------------------------------------------------------------- > > > > > > > > Adding command-line PHIL: > > > > ------------------------- > > > > b_factor_cutoff=40 > > > > clash_cutoff=-0.4 > > > > > > > > Final processed PHIL parameters: > > > > > ------------------------------------------------------------------------------- > > > > data_manager { > > > > default_output_filename = "cctbx_program" > > > > } > > > > b_factor_cutoff = 40 > > > > > > > > > > > > Starting job > > > > > =============================================================================== > > > > Sorry: 0 model(s) found. Expected at least 1. > > > > chartClashlist ran for 5 seconds > > > > do Charts ran for 0 seconds > > > > sh: phenix.probe: command not found > > > > do Kinemage ran for 0 seconds > > > > > > > > Traceback (most recent call last): > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/../modules/cctbx_project/mmtbx/command_line/mp_geo.py", > line 7, in <module> > > > > mp_geo.run(sys.argv[1:]) > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/validation/molprobity/mp_geo.py", > line 132, in run > > > > pdb_hierarchy.models()[0].chains(): > > > > IndexError: list index out of range > > > > > > > > > > > > > > > > > > > > sh: phenix.reduce: command not found > > > > > > > > sh: phenix.reduce: command not found > > > > > > > > sh: phenix.reduce: command not found > > > > > > > > sh: phenix.probe: command not found > > > > > > > > sh: phenix.probe: command not found > > > > > > > > sh: phenix.reduce: command not found > > > > > > > > sh: phenix.reduce: command not found > > > > > > > > sh: phenix.probe: command not found > > > > > > > > sh: phenix.probe: command not found > > > > > > > > > > > > > > > > Starting phenix.clashscore > > > > > > > > on Mon Aug 12 13:50:31 2019 by appmgmt > > > > > > > > > =============================================================================== > > > > > > > > > > > > > > > > Processing files: > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > Files not used by program: > > > > > > > > -------------------------- > > > > > > > > > /hpc/apps/2018/molprobity/v4.4.app/MolProbity/public_html/data/c2h8n399dagb0ggfn8kpla6fu4/coordinates/1irkFH.pdb > > > > > > > > > > > > > > > > Processing PHIL parameters: > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > Adding command-line PHIL: > > > > > > > > ------------------------- > > > > > > > > b_factor_cutoff=40 > > > > > > > > clash_cutoff=-0.4 > > > > > > > > > > > > > > > > Final processed PHIL parameters: > > > > > > > > > ------------------------------------------------------------------------------- > > > > > > > > data_manager { > > > > > > > > default_output_filename = "cctbx_program" > > > > > > > > } > > > > > > > > b_factor_cutoff = 40 > > > > > > > > > > > > > > > > > > > > > > > > Starting job > > > > > > > > > =============================================================================== > > > > > > > > Sorry: 0 model(s) found. Expected at least 1. > > > > > > > > chartClashlist ran for 5 seconds > > > > > > > > do Charts ran for 0 seconds > > > > > > > > sh: phenix.probe: command not found > > > > > > > > do Kinemage ran for 0 seconds > > > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/build/../modules/cctbx_project/mmtbx/command_line/mp_geo.py", > line 7, in <module> > > > > > > > > mp_geo.run(sys.argv[1:]) > > > > > > > > File > "/hpc/apps/2018/molprobity/v4.4.app/MolProbity/modules/cctbx_project/mmtbx/validation/molprobity/mp_geo.py", > line 132, in run > > > > > > > > pdb_hierarchy.models()[0].chains(): > > > > > > > > IndexError: list index out of range > > > > > > Best Regards, > > Charitha Shetty > > > > *From:* Robert Nolte > *Sent:* Sunday, August 11, 2019 12:39 PM > *To:* Nigel Moriarty (nwmoriarty(a)lbl.gov) <nwmoriarty(a)lbl.gov>; > cctbxbb(a)phenix-online.org > *Cc:* Charitha Belluru Shetty <charitha-belluru.b.shetty(a)gsk.com> > *Subject:* Molprobity / CCTBX incompatibility > > > > We are trying to reinstall our internal molprobity server, and have hit an > issue with the github version where it appears to be a version conflict > with the version of cctbx that is currently being downloaded by > configuration script in the master branch of molprobity. We are getting > errors when we run the validation routine such as > > > > Unrecogized PHIL parameter: data_version=8000 > > > > or > > > > Unrecognized PHIL parameter: clash_cutoff=-0.4 > > > > The analysis for clashscore, rotalyze, are failing to run and we get a > number of divide by zero errors for several of the other ‘visualize’ and > ‘eff_resolution’ routines. > > > > Can you help get this working. > > Thanks, > > Bob Nolte > > > > > > > > *GSK monitors email communications sent to and from GSK in order to > protect GSK, our employees, customers, suppliers and business partners, > from cyber threats and loss of GSK Information. GSK monitoring is conducted > with appropriate confidentiality controls and in accordance with local laws > and after appropriate consultation.* >

1 0

Molprobity / CCTBX incompatibility
by Robert Nolte 12 Aug '19

12 Aug '19

We are trying to reinstall our internal molprobity server, and have hit an issue with the github version where it appears to be a version conflict with the version of cctbx that is currently being downloaded by configuration script in the master branch of molprobity. We are getting errors when we run the validation routine such as Unrecogized PHIL parameter: data_version=8000 or Unrecognized PHIL parameter: clash_cutoff=-0.4 The analysis for clashscore, rotalyze, are failing to run and we get a number of divide by zero errors for several of the other 'visualize' and 'eff_resolution' routines. Can you help get this working. Thanks, Bob Nolte GSK monitors email communications sent to and from GSK in order to protect GSK, our employees, customers, suppliers and business partners, from cyber threats and loss of GSK Information. GSK monitoring is conducted with appropriate confidentiality controls and in accordance with local laws and after appropriate consultation.

1 0

Azure pipelines Linux builds failing with segmentation fault
by richard.gildea＠diamond.ac.uk 01 Aug '19

01 Aug '19

Recent Azure builds are failing for the Linux builds with the following error: ##[section]Starting: Test ============================================================================== Task : Command line Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows Version : 2.151.2 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line ============================================================================== Generating script. ========================== Starting Command Output =========================== [command]/bin/bash --noprofile --norc /__w/_temp/823d09b8-e953-4f93-a9a8-0d863a25f1ce.sh /__w/_temp/823d09b8-e953-4f93-a9a8-0d863a25f1ce.sh: line 11: 6307 Segmentation fault (core dumped) libtbx.run_tests_parallel module=boost_adaptbx module=cctbx module=cma_es module=fable module=gltbx module=iotbx module=libtbx module=rstbx module=scitbx module=simtbx module=smtbx module=spotfinder module=annlib_adaptbx module=cbflib_adaptbx nproc=4 ##[error]Bash exited with code '139'. ##[section]Finishing: Test https://dev.azure.com/cctbx/cctbx_project/_build/results?buildId=256 The last successful Linux Python 2.7 master build was https://dev.azure.com/cctbx/cctbx_project/_build/results?buildId=240 The first failing Linux Python 2.7 master build was https://dev.azure.com/cctbx/cctbx_project/_build/results?buildId=245 Mac builds appear to be fine. Does anyone have any idea what is the issue? Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2