Testing the Science Pipelines installation with a demo

This demo will allow you to quickly test your LSST Science Pipelines installation, regardless of your installation method.

1. Activate the LSST Science Pipelines

Remember to first load the LSST Science Pipelines into your shell’s environment. The method depends on how the Science Pipelines were installed:

2. Download the demo project

Choose a directory to run the demo in. For example:

mkdir -p demo_data
cd demo_data

Then download the demo’s data (if you aren’t running the current stable release, see the note below):

curl -L https://github.com/lsst/lsst_dm_stack_demo/archive/master.tar.gz | tar xvzf -
cd lsst_dm_stack_demo-master

Caution

The demo’s version should match your LSST Science Pipelines installed software. If you installed from source (with lsstsw) or with a newer tag, you’ll likely need to run the latest version of the demo (master branch):

curl -L https://github.com/lsst/lsst_dm_stack_demo/archive/master.tar.gz | tar xvzf -
cd lsst_dm_stack_demo-master

The demo repository consumes roughly 41 MB, contains input images, reference data, and configuration files. The demo script will process SDSS images from two fields in Stripe 82, as shown in the following table:

run camcol field filters
4192 4 300 (ur)giz
6377 4 399 (gz)uri

Filters in parentheses are not processed if run with the --small option, see below

3. Run the demo

Now setup the processing package and run the demo:

setup obs_sdss
./bin/demo.sh # --small to process a subset of images

For each input image the script performs the following operations:

  • Generates a subset of basic image characterizations (such photometric zero-point estimation, source detection, and measurement).
  • Creates a ./output directory containing subdirectories of configuration files, processing metadata, calibrated images, and FITS tables of detected sources. These raw outputs are readable by other parts of the LSST pipeline.
  • Generates a master comparison catalog in the working directory from the band-specific source catalogs in the output/sci-results/ subdirectories.

4. Check the demo results

The demo will take a minute or two to execute (depending upon your machine), and will generate a large number of status messages. Upon successful completion, the top-level directory will contain an output ASCII table that can be compared to the expected results from a reference run. This table is for convenience only, and would not ordinarily be produced by the production LSST pipelines.

Demo Invocation Demo Output
demo.sh detected-sources.txt
demo.sh --small detected-sources_small.txt

The demo output may not be identical to the reference output due to minor variation in numerical routines between operating systems (see DM-1086 for details). The bin/compare script will check whether the output matches the reference to within expected tolerances:

./bin/compare detected-sources.txt

The script will print “Ok” if the demo ran correctly.

For more information about the processing done by the demo, refer to its README on GitHub.