Getting started tutorial part 4: Using all the data to calibrate¶

In this part of the tutorial series you will use all of the results from the singleFrame pipeline to improve the photometric and astrometric calibrations. When there are overlapping datasets from multiple visits, it’s possible to average out effects from the atmosphere and improve the overall calibration. This is sometime referred to as ubercalibration.

Improving the photometric and astrometric calibrations can lead to better coadds down the line, and correspondingly better measurements of objects.

For photometric calibration the system is the Forward Global Calibration Method (FGCM). We use a reference catalog from Pan-STARRS.

Refined astrometric calibration is provided by an algorithm called gbdes. For gbdes we use an astrometric reference catalog derived from the second data release of the Gaia source catalog.

It is out of scope to go into the details of the algorithms here, but you will learn how to run them. Also worth noting is that, in this instance, we will not see much benefit from global calibration since our dataset is small, but for larger datasets it can be a big benefit.

Set up¶

Pick up your shell session where you left off in part 2. For convenience, start in the top directory of the example git repository.

cd $RC2_SUBSET_DIR

The lsst_distrib package also needs to be set up in your shell environment. See Setting up installed LSST Science Pipelines for details on doing this.

FGCM¶

As in part 2 you will be running pipelines configured to produce the results we need for later steps.

pipetask run --register-dataset-types \
-b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml \
-i u/$USER/single_frame \
-o u/$USER/fgcm \
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2_subset.yaml#fgcm

This should look very similar to the command executed in part 2. There are three differences: 1) the subset to execute changed from singleFrame to fgcm, 2) the input is now single_frame, which contains pointers to the inputs to and outputs from singleFrame, and 3) the output collection is now fgcm.

Note that unlike the singleFrame pipeline, FGCM must be run on only a single core. Setting the -j switch to anything other than 1 will result in an error.

gbdes¶

You can do gbdes in much the same way as you did FGCM. Change the subset name and collection name appropriately. E.g.:

pipetask run --register-dataset-types \
-b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml \
-i u/$USER/single_frame \
-o u/$USER/gbdes \
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2_subset.yaml#GbdesAstrometricFitTask

Note the input collection is the same as you passed to FGCM since gbdes doesn’t depend on any of the outputs of FGCM.

Apply the calibrations¶

Now you will want to apply the calibrations derived by running FGCM and gbdes to the source catalogs using the following (as always, changing the subset name and collection name appropriately):

pipetask run --register-dataset-types \
-b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml \
-i u/$USER/single_frame,u/$USER/fgcm,u/$USER/gbdes \
-o u/$USER/source_calibration \
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2_subset.yaml#source_calibration

Wrap up¶

In this tutorial, you’ve computed the improved photometric and astrometric calibration from multiple visits, and applied the calibration to the source catalogs from those visits. Here are some key takeaways:

FGCM provides improved photometric calibration.
Astrometric calibration improvements are provided by running gbdes.
Calibrations can be applied to the visit-level source catalogs by running the calibrate subset of tasks.
Given a pipeline description, e.g. the .yaml file used here, a subset can be specified, so running multiple steps can be done with very similar command line syntax.

Continue this tutorial in part 5, where you’ll warp single frame images and stack them to make coadds.

Navigation