Release 24.0.0 (2022-12-09)

Source Identifier
Git tag 24.0.0
EUPS distrib v24_0_0

This release is based on the w_2022_28 weekly build with 13 tickets backported.

The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 24.0.0.

A Characterization Metric Report will be released describing the scientific performance of this release in terms of scientific performance metrics with the first minor release.

If you have questions or comments about this release, visit our community forum for advice.

See also:

Major New Features

Generation 3 Middleware

This has now formally replaced the Generation 2 middleware. Some highlights of this release:

  • We have changed the multi-snap visit definition to match operations at the observatory.
  • New butler commands remove-runs and remove-collections have been added. These should be used instead of (now deprecated) prune-collections, which can remove more than people expect when chained collections are pruned.
  • It is now possible for the Python type associated with a dataset type to not match that of the butler registry if a converter function has been registered.
  • The command-line tools now only set the logging level of lsst loggers by default rather than modifying the root logger.
  • Graph building now includes additional diagnostics when an empty quantum graph is reported.
  • New pipetask commands purge and cleanup have been added.
  • A simple pipeline execution class has been added to make it easier to run pipelines in unit tests.
  • The bps command now includes restart and ping functionality.

For more information, see detailed notes on changes to the following middleware packages on this release:

Scarlet Lite

Scarlet Lite is now used by default for measurements on coadds. It is algorithmically identical to Scarlet when processing data from a single camera and takes advantage of analytic gradients to improve runtime and memory usage. Change the configuration parameter version in ScarletDeblendTask to switch.

See lsst.meas.extensions.scarlet for more info.

PIFF Default for PSF Modeling

The data release pipeline now uses Piff as the default PSF modeling algorithm. PSFex and PcaPsf are still available per optional configuration. [RFC-755]

Repeatable Star Selection for PSF Modeling

The default data release production pipeline now includes Tasks IsolatedStarAssociationTask and FinalizeCharacterizationTask, which combine multiple visits to identify a consistent set of isolated stars to use for PSF modeling in ever visit.

Amp-to-amp Offset Matching During ISR

The IsrTask includes the Pan-STARRS “pattern continuity” algorithm (a.k.a amp-to-amp matching) that applies additive offsets to the background levels of individual amplifiers so that the boundaries are continuous.

Use Python 3.10

As approved in [RFC-845], the Science Pipelines uses 3.10 as the baseline python version.

Significant Interface Changes

New Packages Added

Packages Removed

Pending Deprecations

These Tasks or methods will be deprecated in the next major release.

Deprecated Gen 3 code in pipe_drivers

The PipelineTask classes in pipe_drivers, FocalPlaneBackground and SkyCorrectionTask, should no longer be used and the identical code in pipe_tasks should be used instead. Please update any Gen3 pipeline definitions to use the new location. pipe_drivers (along with ctrl_pool and daf_persistence) will be removed following the release of v25.0.0.

Discourage shared_ptr arguments for C++ member functions

Smart pointers (such as shared_ptr) should only be used as arguments if a reference, const reference, or raw pointer cannot be used. In afw and daf_base some member functions were overloaded and the old implementations using shared_ptr were marked deprecated and will be removed following the release of v25.0.0.

For more information, refer to RFC-817.


These packages/functions are deprecated and will not be available in the next major release.

Deprecated Generation 2 Middleware

Generation 2 middleware (Gen2) is no longer being developed and should not be used for new code. Gen2 infrastructure code (within e.g., lsst.pipe.tasks, lsst.pipe.base, lsst.obs.base, lsst.daf.persistence, and obs packages) will be removed for the next release.

Following this release, we will remove Gen2 infrastructure code and the command-line task entry points, runDataRef, to the Tasks.

Deprecated DecamRawIngestTask

DECam no longer requires a specialist Gen3 ingest task. Please use the default RawIngestTask. DecamRawIngestTask will be removed after v24.

Deprecated doApplyUberCal in ForcedPhotCcdTask

The field doApplyUberCal is deprecated. Use doApplyExternalPhotoCalib and doApplyExternalSkyWcs instead. It will be removed after this release. [DM-23352]

Deprecated configurations in PsfexStarSelectorTask

The following configurations are no longer used and will be removed after this release:

  • maxbad
  • maxbadflag


Deprecate PSF evaluation without position argument

PSF models are spatially-varying. Evaluation of these models with methods such as psf.computeImage() without a position argument will return the realization of the PSF at the average position. A position argument will be required after v24. The current behavior can be replicated by explicitly providing the average position like psf.computeImage(psf.getAveragePosition()).

For more information, refer to RFC-796.

Deprecate f-string log messages

f-string log messages are now deprecated to match Python logging convention.

For more information, refer to RFC-789.

Deprecate ap.verify.Dataset.getSupportedDatasets

The concept of ‘supported’ datasets is deprecated. The getSupportedDatasets method will be removed after v24.

Deprecated Task Configurations

The following configurations are no longer used and will be removed after the 24.0.0 release:

In PsfWcsSelectImagesTask, the following configuration parameters and functionality have been moved to ComputeExposureSummaryStatsTask:

  • starSelection
  • starShape
  • psfShape

The configuration parameter doLegacyStarSelectionComputation has been added to provide backwards compatibility for the duration of two releases.

In lsst.pipe.tasks.InsertFakesTask:

  • raColName: Use ra_col instead.
  • decColName: Use dec_col instead.
  • diskHLR, aDisk, bDisk, paDisk: Use disk_semimajor_col, disk_axis_ratio_col, and disk_pa_col to specify disk half-light ellipse.
  • nDisk: Use disk_n instead.
  • bulgeHLR, bulge_pa_col, aBulge, bBulge, paBulge: Use bulge_semimajor_col, bulge_axis_ratio_col, and to specify disk half-light ellipse.
  • nBulge: Use bulge_n instead.
  • magVar: Use mag_col instead.
  • sourceSelectionColName: Use select_col instead.