<div dir="ltr">Hi Jan,<div><br></div><div>Good to know about the noexec /tmp error. I'll have to add a $TMPDIR environment variable since it looks like mounting /tmp as noexec is becoming more common.</div><div><br></div><div>Another note is that the dependencies are updated about every 6 months. The versions and builds of the conda packages used for dependencies are explicitly tracked with their URLs and hashes to ensure reproducibility of the build environment. You can delete the "conda_base" directory and the bootstrap.py command (run in a shell before sourcing any of the "setpaths" files) will recreate it.</div><div><br></div><div>Let us know if you have any other issues. Thanks!</div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div>--</div><div><span style="font-size:12.8000001907349px">Billy K. Poon</span><br></div></div><div>Research Scientist, Molecular Biophysics and Integrated Bioimaging</div><div>Lawrence Berkeley National Laboratory</div><div>1 Cyclotron Road, M/S 33R0345</div><div>Berkeley, CA 94720</div><div>Fax: (510) 486-5909</div><div>Web: <a href="https://phenix-online.org" target="_blank">https://phenix-online.org</a></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 9, 2021 at 3:00 PM Jan M. Simons <<a href="mailto:marten@ifk.rwth-aachen.de">marten@ifk.rwth-aachen.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Am 09.07.21 um 15:11 schrieb Billy Poon:<br>
> Hi Jan,<br>
> <br>
> Do you need to build cctbx from scratch? The cctbx has been available<br>
> for Python 3 for a while now, but is using conda packages<br>
> (<a href="https://docs.conda.io/en/latest/" rel="noreferrer" target="_blank">https://docs.conda.io/en/latest/</a> <<a href="https://docs.conda.io/en/latest/" rel="noreferrer" target="_blank">https://docs.conda.io/en/latest/</a>>)<br>
> for managing dependencies. Building the dependencies from scratch for<br>
> multiple versions of Python 3 for multiple platforms was getting too<br>
> complicated.<br>
> <br>
> You can install cctbx as a conda package for Python 3.6 through 3.9 with<br>
> support for macOS (Intel and Apple Silicon), linux, and Windows.<br>
> Instructions can be found here<br>
> (<a href="https://github.com/cctbx/cctbx_project#installation" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#installation</a><br>
> <<a href="https://github.com/cctbx/cctbx_project#installation" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#installation</a>>), but essentially<br>
> it is just<br>
> <br>
> conda install -c conda-forge cctbx-base <br>
> <br>
> to get it into an existing environment. The smtbx module will be<br>
> available, but not fast_linalg yet.<br>
> <br>
> Note that with a conda environment, you do not need to run any of the<br>
> "setpaths" scripts to set it up. Just activate the environment and cctbx<br>
> will be available in python.<br>
> <br>
> There are also nightly builds of the conda packages on a separate<br>
> channel (<a href="https://github.com/cctbx/cctbx_project#nightly-builds" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#nightly-builds</a><br>
> <<a href="https://github.com/cctbx/cctbx_project#nightly-builds" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#nightly-builds</a>>). The command<br>
> above becomes<br>
> <br>
> conda install -c cctbx-nightly -c conda-forge cctbx-base <br>
> <br>
> You can also build cctbx with Python 3 using conda packages as<br>
> dependencies<br>
> (<a href="https://github.com/cctbx/cctbx_project#building-a-development-version" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#building-a-development-version</a><br>
> <<a href="https://github.com/cctbx/cctbx_project#building-a-development-version" rel="noreferrer" target="_blank">https://github.com/cctbx/cctbx_project#building-a-development-version</a>>). The<br>
> bootstrap.py command becomes<br>
> <br>
> python bootstrap.py --use-conda --python 38<br>
> <br>
> This will install miniconda if conda is not already available on your<br>
> system.<br>
<br>
As I did not have conda installed I went for this option and at first I<br>
encountered this:<br>
<br>
===== Running in .: base<br>
Location of conda installation not provided<br>
Proceeding with a fresh installation<br>
Downloading<br>
<a href="https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh" rel="noreferrer" target="_blank">https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh</a><br>
Downloaded file to<br>
/home/jan/Arbeit/cctbx-dev/Miniconda3-latest-Linux-x86_64.sh<br>
Installing miniconda to "/home/jan/Arbeit/cctbx-dev/mc3"<br>
/home/jan/Arbeit/cctbx-dev/mc3/conda.exe: error while loading shared<br>
libraries: libz.so.1: failed to map segment from shared object<br>
/home/jan/Arbeit/cctbx-dev/mc3/conda.exe: error while loading shared<br>
libraries: libz.so.1: failed to map segment from shared object<br>
Traceback (most recent call last):<br>
File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line<br>
1093, in <module><br>
sys.exit(run())<br>
File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line<br>
1069, in run<br>
verbose=namespace.verbose)<br>
File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line<br>
326, in __init__<br>
self.conda_base = self.install_miniconda(prefix=self.root_dir)<br>
File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line<br>
543, in install_miniconda<br>
output = check_output(command_list, env=self.env)<br>
File<br>
"/home/jan/Arbeit/cctbx-dev/modules/cctbx_project/libtbx/auto_build/installer_utils.py",<br>
line 68, in check_output<br>
raise RuntimeError("Call to '%s' failed with exit code %d" %<br>
(popenargs, retcode))<br>
RuntimeError: Call to '(['/bin/sh',<br>
'/home/jan/Arbeit/cctbx-dev/Miniconda3-latest-Linux-x86_64.sh', '-b -u<br>
-p "/home/jan/Arbeit/cctbx-dev/mc3"'],)' failed with exit code 1<br>
Process failed with return code 1<br>
<br>
<br>
But after a little research [1] I found the cause to be a noexec /tmp.<br>
<br>
Remounting /tmp exec made the bootstrap succeed and now I'm happy with<br>
<br>
$ /home/jan/Arbeit/cctbx-dev/build/bin/cctbx.python<br>
Python 3.8.6 | packaged by conda-forge | (default, Oct 7 2020, 19:08:05)<br>
[GCC 7.5.0] on linux<br>
<br>
So on to finally moving my codebase to Python3.<br>
<br>
Thank you all for your support and have a nice weekend<br>
<br>
Jan<br>
<br>
<br>
<br>
[1]<br>
<a href="https://stackoverflow.com/questions/60106630/conda-exe-error-while-loading-shared-libraries-libz-so-1" rel="noreferrer" target="_blank">https://stackoverflow.com/questions/60106630/conda-exe-error-while-loading-shared-libraries-libz-so-1</a><br>
_______________________________________________<br>
cctbxbb mailing list<br>
<a href="mailto:cctbxbb@phenix-online.org" target="_blank">cctbxbb@phenix-online.org</a><br>
<a href="http://phenix-online.org/mailman/listinfo/cctbxbb" rel="noreferrer" target="_blank">http://phenix-online.org/mailman/listinfo/cctbxbb</a><br>
</blockquote></div>