Installation with lsstsw and lsst-build¶
This page guides you through installing the LSST Science Pipelines from source with lsstsw and lsst-build. These are the same tools LSST Data Management uses to build and test the Science Pipelines.
Since lsstsw presents the Science Pipelines as a directory of Git repositories cloned from github.com/lsst, this installation method can be very convenient for developing Science Pipelines code. Other methods of installing LSST Science Pipelines software are newinstall.sh and Docker.
If you have issues using lsstsw, here are two ways to get help:
The LSST Science Pipelines can generally be compiled on CentOS, Debian, and macOS platforms. See Platform compatibility for information about LSST’s official reference platform and build reports with other platforms.
Before you begin:
- Install prerequisites for your platform: CentOS / RedHat, Debian / Ubuntu, or macOS.
- If you intend to use a Git LFS repository, like ci_hsc or afwdata, install and configure Git LFS.
- If you opt not to use the default Python environment provided by
lsstswyou will need to install these Python dependencies.
2. Deploy lsstsw¶
Begin by choosing a working directory, then deploy
lsstsw into it:
git clone https://github.com/lsst/lsstsw.git cd lsstsw ./bin/deploy source bin/setup.sh
For more information about the deploy command, see About the lsstsw deploy script.
3. Build the Science Pipelines packages¶
lsstsw directory, run:
rebuild step finishes, note the build number printed on screen.
It is formatted as “
Tag this build as
current so that EUPS can set it up by default:
eups tags --clone bNNNN current
Finally, set up the packages with EUPS:
See Setting up installed LSST Science Pipelines for more information.
4. Testing Your installation (optional)¶
Once the LSST Science Pipelines are installed, you can verify that it works by running a demo project. This demo processes a small amount of SDSS data.
Sourcing the Pipelines in a new shell¶
In every new shell session you will need to set up the Science Pipelines environment and EUPS package stack.
Run these two steps:
Activate the lsstsw software environment by sourcing the
setup.shscript in lsstsw’s
If you are running in a csh or tcsh, run this set up script instead:
Set up a top-level package:
lsst_distrib, you can set up a different top-level package like
lsst_appsor any individual EUPS package you previously installed. See Top-level packages to install.
Next steps and advanced topics¶
- About the lsstsw deploy script.
- About the lsstsw rebuild command.
- Building from branches.
- lsstsw deploy command reference.
- lsstsw rebuild command reference.
About the lsstsw deploy script¶
deploy script automates several things to prepare an LSST development environment:
- Installs Git.
- Installs Git LFS (you are still responsible for configuring it).
- Installs a Miniconda Python 3 environment specific to this lsstsw workspace.
- Installs EUPS into
- Clones lsst-build, which runs the build process.
- Clones versiondb, a robot-managed Git repository of package dependency information.
- Creates an empty stack installation directory,
This environment, including the EUPS, Miniconda, Git, and Git LFS software, is only activated when you source the
bin/setup.csh scripts in a shell.
Otherwise, lsstsw does not affect the software installed on your computer.
See also: lsstsw deploy command reference.
About the lsstsw rebuild command¶
The rebuild command accomplishes the following:
- Clones all Science Pipelines packages from github.com/lsst. The repos.yaml file in the https://github.com/lsst/repos repository maps package names to GitHub repositories.
- Runs the Scons-based build process to compile C++, make Pybind11 bindings, and ultimately create the
lsstPython package. The stack is built and installed into the
stack/directory inside your
lsstsw clones repositories using HTTPS (see repos.yaml). Our guide to Setting up a Git credential helper will allow you to push new commits up to GitHub without repeatedly entering your GitHub credentials.
See also: lsstsw rebuild command reference.
Building from branches¶
lsstsw’s rebuild command enables you to clone and build development branches.
lsst_distrb, but use the Git branch
my-feature when it’s available in a package’s repository:
rebuild -r my-feature lsst_distrib
Multiple ticket branches across multiple products can be built in order of priority:
rebuild -r feature-1 -r feature-2 lsst_distrib
In this example, a
feature-1 branch will be used in any package’s Git repository.
feature-2 branch will be used secondarily in repositories where
feature-1 doesn’t exist.
lsstsw falls back to using the
master branch for repositories that lack both
lsstsw deploy command reference¶
usage: deploy.sh [-2|-3] [-b] [-h]
Install a Python 2-based Miniconda (not supported for
v16_0of the LSST Science Pipelines and later)
Use bleeding-edge conda packages.
Print the help message.
lsstsw rebuild command reference¶
rebuild [-p] [-n] [-u] [-r <ref> [-r <ref2> [...]]] [-t <eupstag>] [product1 [product2 [...]]]
Do not run git fetch in already-downloaded repositories.
repos.yamlpackage index to the
masterbranch on GitHub of https://github.com/lsst/repos.
Rebuild using the Git ref. A Git ref can be a branch name, tag, or commit SHA. Multiple
-rarguments can be given, in order or priority.