Install with lsstinstall and eups distrib¶
This page guides you through installing the LSST Science Pipelines software using the lsstinstall tool. This installation method is recommended for anyone who uses or develops the Pipelines software.
If you have issues with the installation, here are two ways to get help:
The LSST Science Pipelines are developed and tested primarily on CentOS, but can be compiled and run on macOS, Debian, Ubuntu, and other Linux distributions. See Platform compatibility for information about LSST’s official reference platform and build reports with other platforms, and follow the instructions under System prerequisites to ensure you have installed the prerequisite software for your platform.
2. Make an installation directory¶
Create a directory where you want to install the LSST Science Pipelines into. For example:
mkdir -p lsst_stack
If you are installing the software for multiple users (a shared stack), see Setting unix permissions for shared installations.
3. Run lsstinstall¶
Run lsstinstall to set up the environment you’ll install the LSST Science Pipelines into. For most use cases we recommend downloading and running lsstinstall like this:
curl -OL https://ls.st/lsstinstall
chmod u+x lsstinstall
./lsstinstall -T v26_0_0
-T option specifies the eups tag associated with the release you would like to install.
Then load the LSST software environment into your shell:
To customize the conda environment used, set the
LSST_CONDA_ENV_NAME environment variable to a conda enviroment name when sourcing the file.
For other conda environments installed by LSST tools, this name will be the
rubin-env metapackage version prefixed with
Here are ways to customize the lsstinstall installation for specific needs:
The recommended installation uses precompiled binary tarballs if they’re available for your platform (and falls back to a source build). See About EUPS tarball packages.
For background information about lsstinstall, see:
4. Install Science Pipelines packages¶
The LSST Science Pipelines is distributed as the
lsst_distrib EUPS package.
Install the current version,
eups distrib install -t v26_0_0 lsst_distrib
curl -sSL https://raw.githubusercontent.com/lsst/shebangtron/main/shebangtron | python
If prebuilt binaries are available for your platform (and you did not specify the
-B argument to the lsstinstall command) the installation should take roughly 10 minutes.
Otherwise, the installation falls back to a source build that can take two hours, depending on the top-level package and your machine’s performance.
See How to determine if tarball packages are available for your platform.
The last command, setup, activates the installed packages in your shell environment. You’ll need to run setup in each shell session you’ll use the LSST Science Pipelines in. See Setting up installed LSST Science Pipelines for more information.
v26_0_0is the current version corresponding to this documentation. You can install other tagged versions of the LSST Science Pipelines, though. See Installing other releases (including daily and weekly tags).
If you’re curious about the shebangtron, see its repository at github.com/lsst/shebangtron.
5. Test your installation (optional)¶
Once the LSST Science Pipelines are installed, you can verify that it works by running a demo pipeline.
See Testing the Science Pipelines installation with a demo for instructions.
Now that you have a working LSST Science Pipelines installation, these topics will help you learn and do more:
Advanced installation topics¶
The above steps guided you through LSST’s recommended installation. These topics provide additional information about the installation and ways to customize it:
What lsstinstall does¶
lsstinstall creates a self-contained environment on your machine where you can install, run, and develop the LSST Science Pipelines. You activate this environment in a shell by sourcing the loadLSST.sh script in the installation directory (see Setting up).
Here is how lsstinstall prepares the environment:
Identifies your operating system to determine what EUPS binary packages should be installed (the EUPS package root, see About EUPS tarball packages).
Activates conda, installing it from Mambaforge if needed.
For information about lsstinstall’s arguments, see lsstinstall argument reference.
About the Mambaforge Python installed by lsstinstall¶
lsstinstall by default installs a conda package manager based on Mambaforge, a minimal version of Anaconda preconfigured to use packages from the curated
conda-forge channel along with the
mamba fast dependency solver.
This Python installation isn’t required, but we recommend it.
See How to use your own conda with lsstinstall if you would like to use your own pre-existing conda (but conda is required).
conda can maintain multiple environments, each with its own version of Python and other packages. lsstinstall will create one for each version of the Science Pipelines dependencies (currently managed by the rubin-env metapackage). This includes the compilers and build tools as well as C++ and Python packages needed by the software. You can add install other packages into a Science Pipelines environment, or you can create independent environments composed of other packages or cloned from a Science Pipelines environment.
This Mambaforge installation won’t affect your other Python installations (like the system’s Python, your own Anaconda or Miniconda, or virtual environments).
The LSST Miniconda environment is only active when you source the
loadLSST script installed by lsstinstall (see Setting up installed LSST Science Pipelines).
If you install other Python packages in a shell where the LSST Mambaforge is activated (with pip install or conda install) those packages are installed into the LSST Mambaforge’s
site-packages, not your system’s.
How to use your own conda with lsstinstall¶
lsstinstall installs a new conda based on Mambaforge by default. If desired, you can use your own pre-existing conda installation.
To do so, either have that conda activated when you run lsstinstall, or provide the
-p option to lsstinstall pointing to the conda installation’s prefix.
Having mamba installed in the
base environment is recommended for faster dependency solves.
A Science Pipelines environment will still be created (unless you already have one).
About the rubin-env metapackage¶
The conda environment created by lsstinstall is based on the rubin-env conda metapackage. Each release of the LSST Science Pipelines is built with a particular rubin-env version. A given rubin-env version is typically used to build many releases (daily, weekly, and sometimes major) of the Science Pipelines, and a given release of the Science Pipelines source is often compatible with more than one rubin-env version.
Note that a given rubin-env version does not itself exactly specify all versions of its dependencies. We typically allow dependency versions to “float” to more recent updates in order to allow greater compatibility with user-installed packages and to pick up bug fixes. We only restrict these updates if newer versions cause incompatibilities with the Science Pipelines source code. This means that one user’s installation of a given rubin-env version may be different from another’s. To assist with debugging, you may be asked to list the installed dependency versions with the conda env export command. In production, the dependencies are frozen at the versions that were tested when the docker containers were built.
You choose the version of the dependencies with the lsstinstall arguments
-T (to match the version used to build a particular tag of the Science Pipelines),
-X (to use the exact packages used for that tag’s build, not allowing any to float to more recent updates), or
-v (to specify a particular version manually).
You can update the versions of the rubin-env dependencies to the latest compatible ones for the rubin-env version specified by using lsstinstall with the
About the rubin-env-developer metapackage¶
The rubin-env-developer metapackage adds tools and utilities that are useful for developers working on improving the LSST Science Pipelines but not necessarily for users processing data with them.
It can be installed on top of the rubin-env installation performed by lsstinstall by specifying the
Only rubin-env versions 5.0.0 and greater can use this option.
About EUPS tarball packages¶
EUPS distrib binary (tarball) packages significantly speed up your installation. Rather than compiling the LSST Science Pipelines from source, EUPS tarballs are prebuilt packages made specifically for supported platforms.
Platforms are defined by two factors:
rubin-env metapackage (Science Pipelines dependencies) version.
When you run lsstinstall, it looks at your system to identify your operating system.
See About the rubin-env metapackage for more about the rubin-env metapackage and how to select its version.
The EUPS packages that make up the Science Pipelines are installed within that environment, as they are only binary-compatible with its conda packages and tools.
The URLs used to retrieve EUPS packages are also stored within the environment in
loadLSST.sh will automatically read this file to enable proper use of eups distrib install by setting the
EUPS_PKGROOT environment variable.
Note that activating (or deactivating) conda environments does not automatically set this variable.
How to determine if tarball packages are available for your platform¶
When you run eups distrib install, it will attempt to install prebuilt binary packages first and fall back to compiling the Science Pipelines if binary packages aren’t available for your platform (by default). This fallback is automatic. You’ll know packages are being compiled from source if you see compiler processes (like gcc or clang) load your machine.
The instructions in this section will help you diagnose why eups distrib install is falling back to a source installation.
First, get your EUPS package root URLs:
eups distrib path
If the only URL listed is https://eups.lsst.codes/stack/src, it means that lsstinstall configured your environment to not use binary packages.
Try re-running lsstinstall (see 3. Run lsstinstall) without the
-B argument, and check to make sure that your computing platform is supported for binary packages (currently Linux Intel and macOS Intel only).
If eups distrib path includes an additional URL that doesn’t end with
/src (for example,
https://eups.lsst.codes/stack/osx/10.9/conda-system/miniconda3-py38_4.9.2-0.8.0), it means lsstinstall has configured a binary package root.
The construction of the binary package root URL is based on your OS and rubin-env version (see About EUPS tarball packages).
eups distrib install will only install binary packages if they exist on the binary package root. To check this, open the binary package root URL in a web browser. If the binary package root URL does not load in a browser it means LSST does not publish prebuilt binaries for your platform. Either continue the installation from source or consider using the LSST Docker images.
If the URL does open, though, search for files with a
.list file is created for each release that has binary packages.
The name of the
.list file matches the release tag (for example,
See Installing other releases (including daily and weekly tags) for more information about tags.
For example, if the binary package root is
https://eups.lsst.codes/stack/osx/10.9/conda-system/miniconda3-4.9.2-0.7.0 and you wish to install the
w_2021_33 tag, the file
https://eups.lsst.codes/stack/osx/10.9/conda-system/miniconda3-4.9.2-0.7.0/w_2021_33.list must exist for a binary installation.
.list file does not exist for the tag you want to install, but does exist for other tags in that EUPS package root, it may be due to an issue with the LSST binary package publishing system.
You can either continue with an installation from source, consider switching to a tag that is known to have binary packages, or consider using LSST’s Docker images.
EUPS tarball packages and compiler compatibility¶
EUPS binary tarball packages are prebuilt on LSST’s continuous integration servers for a specific combination of operating system, compilers, Python, and Python dependencies. The compilers and other linked dependencies are provided by conda-forge. Compatibility with other compilers is not guaranteed. Using a non-conda-forge compiler toolchain requires that the binary interface be the same as that used by the conda-forge toolchain.
lsstinstall argument reference¶
usage: lsstinstall [-n]
[-T EUPS_TAG | -X EUPS_TAG | -v RUBINENV_VERSION]
[-e ENV_NAME] [-u] [-d]
[-p CONDA_PATH] [-P] [-C CHANNEL]
No-op. Echo commands instead of running.
- -T <EUPS_TAG>¶
Select the rubin-env version used to build the given EUPS_TAG.
- -X <EUPS_TAG>¶
Select the exact environment used to build the given EUPS_TAG.
- -v <RUBINENV_VERSION>¶
Select a particular rubin-env version (default=latest).
- -e <ENV_NAME>¶
Specify the environment name to use; if it exists, assume that it is compatible and should be used.
Update rubin-env in an existing environment to the latest build.
Add a compatible rubin-env-developer to rubin-env (5.0.0 and later).
- -p <CONDA_PATH>¶
Specify the path to the conda installation. If a conda installation already exists there, it will be used. If it does not exist, it will be created. If a conda is activated, it will be used, ignoring this option.
DO NOT use an existing activated conda; always install a new one.
- -C <CHANNEL>¶
Use the given conda channel before the conda-forge channel. May be repeated; first has highest priority. Useful primarily for testing new rubin-env versions in the
- -E <EUPS_URL>¶
Select a different EUPS distribution server root URL (default=``https://eups.lsst.codes/stack``).
DO NOT use binary “tarball” eups packages.
DO NOT use source eups packages.
Ignored for backward compatibility.
Ignored for backward compatibility.
Ignored for backward compatibility.
Display a help message.