Source Installation with newinstall.sh¶

This page will guide you through installing the LSST Science Pipelines from source with newinstall.sh (internally based on eups distrib).

The LSST Science Pipelines are officially tested against CentOS 7, however developers regularly use a variety of Linux and macOS operating systems.

We also offer Conda binaries and Docker images if you do not wish to install the Science Pipelines from source.

If you have difficulty installing LSST software:

review the known installation issues for your platform.
reach out on the Support forum at community.lsst.org.

Prerequisites¶

This section lists system prerequisites for macOS, Debian/Ubuntu, and RedHat/CentOS platforms. All platforms also need Python package dependencies listed here.

Note

New since 11.0: The minimum gcc version required to compile the Stack is gcc 4.8. If you using our previous factory platform, RedHat/CentOS 6, and you are unable to upgrade to version 7 (which comes with gcc 4.8 as default) consult the section below on upgrading compilers in legacy Linux.

macOS¶

To build LSST software, macOS systems need:

Xcode, or command line tools.
cmake.

Versions prior to OS X 10.9 and earlier have not been tested recently and may not work.

Xcode¶

You will need to install developer tools, which we recommend you obtain with Apple’s Xcode command line tools package. To do this, run from the command line (e.g. Terminal.app or similar):

xcode-select --install

and follow the on-screen instructions. You can verify where the tools are installed by running:

xcode-select -p

cmake¶

cmake can be installed directly, or though a package manager like Homebrew.

Debian / Ubuntu¶

apt-get install bison ca-certificates \
        cmake flex g++ gettext git libbz2-dev \
        libfontconfig1 libglib2.0-dev libncurses5-dev \
        libreadline6-dev libssl-dev libx11-dev libxrender1 \
        libxt-dev m4 openjdk-8-jre \
        perl-modules zlib1g-dev \

Prefix the apt-get command with sudo if necessary.

RedHat / CentOS¶

yum install bison curl blas bzip2-devel bzip2 flex fontconfig \
    freetype-devel gcc-c++ gcc-gfortran git libuuid-devel \
    libXext libXrender libXt-devel make openssl-devel patch perl \
    readline-devel tar zlib-devel ncurses-devel cmake glib2-devel \
    java-1.8.0-openjdk gettext perl-ExtUtils-MakeMaker

Prefix the yum command with sudo if necessary.

Upgrading compilers for legacy RedHat / CentOS 6¶

The minimum gcc version required to compile the Stack is gcc 4.8. This comes as standard in the LSST “factory” platform, Red Hat / CentOS 7.

On our previous factory platform, Red Hat / CentOS 6, you will need to use a more current version of gcc that what is available with your system. If you can go to Red Hat 7, we recommend that you do; if you cannot, we recommend that you use a newer gcc version for the stack by using a Software Collection (SCL) with a different version of devtoolset. This will enable you to safely use a different version of gcc (4.9) for the stack than that used by your operating system (4.4).

First, install devtoolset-3 (after the installing the standard pre-requisites (above)):

sudo yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
sudo yum install -y https://www.softwarecollections.org/en/scls/rhscl/rh-java-common/epel-6-x86_64/download/rhscl-rh-java-common-epel-6-x86_64.noarch.rpm
sudo yum install -y https://www.softwarecollections.org/en/scls/rhscl/devtoolset-3/epel-6-x86_64/download/rhscl-devtoolset-3-epel-6-x86_64.noarch.rpm
sudo yum install -y scl-utils
sudo yum install -y devtoolset-3

Then enable devtoolset-3 by including this line in your ~/.bash_profile:

scl enable devtoolset-3 bash

Python dependencies¶

You can use your own Python 2.7.* install or let newinstall.sh install Miniconda in your local directory.

Optional dependencies¶

Although not required, we recommend you install the matplotlib and scipy Python packages:

pip install -U matplotlib scipy

Note these are included by default in Anaconda, which newinstall.sh can obtain for you.

We also use SAOImage DS9 to display images for debugging.

Installing from Source with newinstall.sh¶

This section will guide you through installing the current release of the LSST Science Pipelines from source given that prerequisites have been installed.

1. Choose an installation directory¶

First, choose where you want to install the LSST Science Pipelines. We’ll use $HOME/lsst_stack in this example. Create and change into that directory:

mkdir -p $HOME/lsst_stack
cd $HOME/lsst_stack

Installation for groups¶

Those in a system administration role, who are installing a writable stack for multiple users, will likely want to establish a separate group (perhaps lsst) with a umask of 002 (all access permissions for the group; allow other users to read+execute). The installation directory must be owned by the group, have the SGID (2000) bit set, and allow group read/write/execute: that is, mode 2775. Individual users who install a personal Stack on their own machine need not worry about this.

2. Unset environment variables¶

If you’ve run the LSST Science Pipelines previously, you may have conflicting environment variables setup. To be safe, run:

unset LSST_HOME EUPS_PATH LSST_DEVEL EUPS_PKGROOT REPOSITORY_PATH

3. Installation set-up¶

Download and run the installation setup script from GitHub, which installs the basic packages required to install other packages:

curl -OL https://raw.githubusercontent.com/lsst/lsst/12.0/scripts/newinstall.sh
bash newinstall.sh

This installs the loadLSST.* scripts, which you should source to ensure that LSST tools (e.g., the eups command) are included in your path.

The install script will check your system to ensure that appropriate versions of critical packages are installed on your system, to enable bootstrapping the Science Pipelines, including git, and python. If these packages are not available, the script will offer to install them for you (using the Anaconda Python distribution for the latter packages).

Allowing the installation of these core packages will not replace or modify any other version of these packages that may be installed on your system. If you do not choose the Anaconda Python install, and subsequent package build steps fail, you can do one of two things:

Report the problem to community.lsst.org. Include your OS, a description of the problem, plus any error messages. Community members will provide assistance.
Consider removing all contents of the install directory and start from scratch, and accepting the Anaconda Python installation option.

Once newinstall.sh has finished, source the LSST environment to continue the installation by running the appropriate command for your shell:

source $LSST_INSTALL_DIR/loadLSST.bash # for bash users
source $LSST_INSTALL_DIR/loadLSST.csh  # for csh users
source $LSST_INSTALL_DIR/loadLSST.ksh  # for ksh users
source $LSST_INSTALL_DIR/loadLSST.zsh  # for zsh users

where $LSST_INSTALL_DIR is expanded to your installation directory.

4. Install packages¶

Finally, build/install any other components of the LSST Science Pipelines that are relevant for your work. A simple way to ensure that you have a fairly complete set of packages for this need is to install lsst_apps. The dependency tree for lsst_apps ensures that many other packages (about 70, including e.g., pipe_tasks) are also installed.

Installing lsst_apps may take a little while (about 1.2 hr on a 2014-era iMac with 32 GB of memory and 8 cores):

eups distrib install -t v12_0 lsst_apps

After this initial setup, it is a good idea to test the installation. See Testing Your Installation.

5. Source the LSST environment in each shell session¶

Whenever you want to run the installed LSST Science Pipelines in a new terminal session, be sure to source the appropriate loadLSST.bash,csh,ksh,zsh script.

Testing Your Installation¶

Once the LSST Science Pipelines are installed, you can verify that it works by running a demo project. This demo processes a small amount of SDSS data.