Release 24.0.0 (2022-12-09)¶

Source	Identifier
Git tag	24.0.0
EUPS distrib	v24_0_0
rubin-env version	4.0.5

This release is based on the w_2022_28 weekly build with 13 tickets backported.

The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 24.0.0.

A Characterization Metric Report will be released describing the scientific performance of this release in terms of scientific performance metrics with the first minor release.

If you have questions or comments about this release, visit our community forum for advice.

Major New Features
Significant Interface Changes
Pending Deprecations
Deprecations

See also:

Major New Features¶

Generation 3 Middleware
Scarlet Lite
PIFF Default for PSF Modeling
Repeatable Star Selection for PSF Modeling
Amp-to-amp Offset Matching During ISR
Use Python 3.10

Generation 3 Middleware¶

This has now formally replaced the Generation 2 middleware. Some highlights of this release:

We have changed the multi-snap visit definition to match operations at the observatory.
New butler commands remove-runs and remove-collections have been added. These should be used instead of (now deprecated) prune-collections, which can remove more than people expect when chained collections are pruned.
It is now possible for the Python type associated with a dataset type to not match that of the butler registry if a converter function has been registered.
The command-line tools now only set the logging level of lsst loggers by default rather than modifying the root logger.
Graph building now includes additional diagnostics when an empty quantum graph is reported.
New pipetask commands purge and cleanup have been added.
A simple pipeline execution class has been added to make it easier to run pipelines in unit tests.
The bps command now includes restart and ping functionality.

For more information, see detailed notes on changes to the following middleware packages on this release:

Scarlet Lite¶

Scarlet Lite is now used by default for measurements on coadds. It is algorithmically identical to Scarlet when processing data from a single camera and takes advantage of analytic gradients to improve runtime and memory usage. Change the configuration parameter version in ScarletDeblendTask to switch.

See lsst.meas.extensions.scarlet for more info.

PIFF Default for PSF Modeling¶

The data release pipeline now uses Piff as the default PSF modeling algorithm. PSFex and PcaPsf are still available per optional configuration. [RFC-755]

Repeatable Star Selection for PSF Modeling¶

The default data release production pipeline now includes Tasks IsolatedStarAssociationTask and FinalizeCharacterizationTask, which combine multiple visits to identify a consistent set of isolated stars to use for PSF modeling in ever visit.

Amp-to-amp Offset Matching During ISR¶

The IsrTask includes the Pan-STARRS “pattern continuity” algorithm (a.k.a amp-to-amp matching) that applies additive offsets to the background levels of individual amplifiers so that the boundaries are continuous.

Use Python 3.10¶

As approved in [RFC-845], the Science Pipelines uses 3.10 as the baseline python version.

New Packages Added¶

analysis_drp provides static plotting code for monitoriing pipeline quality.
atmospec and spectractor provide support for analyzing spetra from LATISS.
lsst_bps_plugins, lsst.ctrl.bps.htcondor, and lsst.ctrl.bps.panda
lsst.drp.pipe provide high-level pipeline specification for data releases.
resources

Packages Removed¶

validate_drp, validate_data_cfht, validate_data_decam were removed. The algorithms implemented in validate_drp were ported as-is to run in lsst.faro. All future development of scientific performance metrics will be carried out in faro.
A number of packages that were gen2-specific, obs_test, obs_test_data, ctrl_orca, ctrl_execute, ctrl_platform_lsstvc, were also removed.

Pending Deprecations¶

These Tasks or methods will be deprecated in the next major release.

Deprecated Gen 3 code in pipe_drivers
Discourage shared_ptr arguments for C++ member functions

Deprecated Gen 3 code in pipe_drivers¶

The PipelineTask classes in pipe_drivers, FocalPlaneBackground and SkyCorrectionTask, should no longer be used and the identical code in pipe_tasks should be used instead. Please update any Gen3 pipeline definitions to use the new location. pipe_drivers (along with ctrl_pool and daf_persistence) will be removed following the release of v25.0.0.

Discourage shared_ptr arguments for C++ member functions¶

Smart pointers (such as shared_ptr) should only be used as arguments if a reference, const reference, or raw pointer cannot be used. In afw and daf_base some member functions were overloaded and the old implementations using shared_ptr were marked deprecated and will be removed following the release of v25.0.0.

For more information, refer to RFC-817.

Deprecations¶

These packages/functions are deprecated and will not be available in the next major release.

Deprecated Generation 2 Middleware
Deprecated DecamRawIngestTask
Deprecated doApplyUberCal in ForcedPhotCcdTask
Deprecated configurations in PsfexStarSelectorTask
Deprecate PSF evaluation without position argument
Deprecate f-string log messages
Deprecate ap.verify.Dataset.getSupportedDatasets
Deprecated Task Configurations

Deprecated Generation 2 Middleware¶

Generation 2 middleware (Gen2) is no longer being developed and should not be used for new code. Gen2 infrastructure code (within e.g., lsst.pipe.tasks, lsst.pipe.base, lsst.obs.base, lsst.daf.persistence, and obs packages) will be removed for the next release.

Following this release, we will remove Gen2 infrastructure code and the command-line task entry points, runDataRef, to the Tasks.

Deprecated DecamRawIngestTask¶

DECam no longer requires a specialist Gen3 ingest task. Please use the default RawIngestTask. DecamRawIngestTask will be removed after v24.

Deprecated doApplyUberCal in ForcedPhotCcdTask¶

The field doApplyUberCal is deprecated. Use doApplyExternalPhotoCalib and doApplyExternalSkyWcs instead. It will be removed after this release. [DM-23352]

Deprecated configurations in PsfexStarSelectorTask¶

The following configurations are no longer used and will be removed after this release:

maxbad
maxbadflag

[DM-5684]

Deprecate PSF evaluation without position argument¶

PSF models are spatially-varying. Evaluation of these models with methods such as psf.computeImage() without a position argument will return the realization of the PSF at the average position. A position argument will be required after v24. The current behavior can be replicated by explicitly providing the average position like psf.computeImage(psf.getAveragePosition()).

For more information, refer to RFC-796.

Deprecate f-string log messages¶

f-string log messages are now deprecated to match Python logging convention.

For more information, refer to RFC-789.

Deprecate ap.verify.Dataset.getSupportedDatasets¶

The concept of ‘supported’ datasets is deprecated. The getSupportedDatasets method will be removed after v24.

Deprecated Task Configurations¶

The following configurations are no longer used and will be removed after the 24.0.0 release:

In PsfWcsSelectImagesTask, the following configuration parameters and functionality have been moved to ComputeExposureSummaryStatsTask:

starSelection
starShape
psfShape

The configuration parameter doLegacyStarSelectionComputation has been added to provide backwards compatibility for the duration of two releases.

In lsst.pipe.tasks.InsertFakesTask:

raColName: Use ra_col instead.
decColName: Use dec_col instead.
diskHLR, aDisk, bDisk, paDisk: Use disk_semimajor_col, disk_axis_ratio_col, and disk_pa_col to specify disk half-light ellipse.
nDisk: Use disk_n instead.
bulgeHLR, bulge_pa_col, aBulge, bBulge, paBulge: Use bulge_semimajor_col, bulge_axis_ratio_col, and to specify disk half-light ellipse.
nBulge: Use bulge_n instead.
magVar: Use mag_col instead.
sourceSelectionColName: Use select_col instead.

Navigation