Hi Jan,

Good to know about the noexec /tmp error. I'll have to add a $TMPDIR environment variable since it looks like mounting /tmp as noexec is becoming more common.

Another note is that the dependencies are updated about every 6 months. The versions and builds of the conda packages used for dependencies are explicitly tracked with their URLs and hashes to ensure reproducibility of the build environment. You can delete the "conda_base" directory and the bootstrap.py command (run in a shell before sourcing any of the "setpaths" files) will recreate it.

Let us know if you have any other issues. Thanks!

--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Fax: (510) 486-5909


On Fri, Jul 9, 2021 at 3:00 PM Jan M. Simons <marten@ifk.rwth-aachen.de> wrote:
Am 09.07.21 um 15:11 schrieb Billy Poon:
> Hi Jan,
>
> Do you need to build cctbx from scratch? The cctbx has been available
> for Python 3 for a while now, but is using conda packages
> (https://docs.conda.io/en/latest/ <https://docs.conda.io/en/latest/>)
> for managing dependencies. Building the dependencies from scratch for
> multiple versions of Python 3 for multiple platforms was getting too
> complicated.
>
> You can install cctbx as a conda package for Python 3.6 through 3.9 with
> support for macOS (Intel and Apple Silicon), linux, and Windows.
> Instructions can be found here
> (https://github.com/cctbx/cctbx_project#installation
> <https://github.com/cctbx/cctbx_project#installation>), but essentially
> it is just
>
> conda install -c conda-forge cctbx-base 
>
> to get it into an existing environment. The smtbx module will be
> available, but not fast_linalg yet.
>
> Note that with a conda environment, you do not need to run any of the
> "setpaths" scripts to set it up. Just activate the environment and cctbx
> will be available in python.
>
> There are also nightly builds of the conda packages on a separate
> channel (https://github.com/cctbx/cctbx_project#nightly-builds
> <https://github.com/cctbx/cctbx_project#nightly-builds>). The command
> above becomes
>
> conda install -c cctbx-nightly -c conda-forge cctbx-base 
>
> You can also build cctbx with Python 3 using conda packages as
> dependencies
> (https://github.com/cctbx/cctbx_project#building-a-development-version
> <https://github.com/cctbx/cctbx_project#building-a-development-version>). The
> bootstrap.py command becomes
>
> python bootstrap.py --use-conda --python 38
>
> This will install miniconda if conda is not already available on your
> system.

As I did not have conda installed I went for this option and at first I
encountered this:

===== Running in .: base
Location of conda installation not provided
Proceeding with a fresh installation
Downloading
https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
Downloaded file to
/home/jan/Arbeit/cctbx-dev/Miniconda3-latest-Linux-x86_64.sh
Installing miniconda to "/home/jan/Arbeit/cctbx-dev/mc3"
/home/jan/Arbeit/cctbx-dev/mc3/conda.exe: error while loading shared
libraries: libz.so.1: failed to map segment from shared object
/home/jan/Arbeit/cctbx-dev/mc3/conda.exe: error while loading shared
libraries: libz.so.1: failed to map segment from shared object
Traceback (most recent call last):
  File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line
1093, in <module>
    sys.exit(run())
  File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line
1069, in run
    verbose=namespace.verbose)
  File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line
326, in __init__
    self.conda_base = self.install_miniconda(prefix=self.root_dir)
  File "modules/cctbx_project/libtbx/auto_build/install_conda.py", line
543, in install_miniconda
    output = check_output(command_list, env=self.env)
  File
"/home/jan/Arbeit/cctbx-dev/modules/cctbx_project/libtbx/auto_build/installer_utils.py",
line 68, in check_output
    raise RuntimeError("Call to '%s' failed with exit code %d" %
(popenargs, retcode))
RuntimeError: Call to '(['/bin/sh',
'/home/jan/Arbeit/cctbx-dev/Miniconda3-latest-Linux-x86_64.sh', '-b -u
-p "/home/jan/Arbeit/cctbx-dev/mc3"'],)' failed with exit code 1
Process failed with return code 1


But after a little research [1] I found the cause to be a noexec /tmp.

Remounting /tmp exec made the bootstrap succeed and now I'm happy with

$ /home/jan/Arbeit/cctbx-dev/build/bin/cctbx.python
Python 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)
[GCC 7.5.0] on linux

So on to finally moving my codebase to Python3.

Thank you all for your support and have a nice weekend

Jan



[1]
https://stackoverflow.com/questions/60106630/conda-exe-error-while-loading-shared-libraries-libz-so-1
_______________________________________________
cctbxbb mailing list
cctbxbb@phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb