Release 21.0.0 (2020-12-08)¶

Source	Identifier
Git tag	21.0.0
EUPS distrib	v21_0_0

This release is based on the w_2020_46 weekly build.

The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 21.0.0.

The Characterization Metric Report (DMTR-281) describes the scientific performance of this release in terms of scientific performance metrics, while Changes to Data Products in Release 21.0.0 covers key issues likely to impact scientists consuming Rubin data products.

If you have questions or comments about this release, visit our community forum for advice.

Major New Features
Significant Interface Changes
Pending Deprecations
Deprecations

See also:

Major New Features¶

Beta release of Generation 3 Middleware
Task to mask streak-like artifacts
Inject arbitrary input images
Proper motions and parallaxes applied to reference sources
Addition of ci_cpp

Beta release of Generation 3 Middleware¶

Reaching schema stability in this release, the Gen3 butler can now support Science Pipelines development. See Removal of “Generation 2” Middleware for more information on the planned deprecation of Generation 2 middleware. Interested parties may find more info at daf_butler and pipe_base. A general release of Generation 3 Middleware is expected with the next release v22.0.0.

Task to mask streak-like artifacts¶

This release adds the MaskStreaksTask to pipe_tasks, which finds and masks streaks and other straight lines in image data. It uses the Kernel Hough Transform (KHT) (Fernandes and Oliveira 2007) implemented in the kht, package which was also added to the Science Pipelines. It can be turned on during coaddition with CompareWarpAssembleCoaddTask with the config parameter doFilterMorphological.

Inject arbitrary input images¶

The task to inject simulated sources into data release processing, InsertFakesTask, can now accept arbitrary input images. [DM-25747]

Proper motions and parallaxes applied to reference sources¶

jointcal now can apply proper motions and parallaxes to reference sources during astrometric fitting. [DM-24277]

Addition of ci_cpp¶

The ci_cpp metapackage for the ci_cpp_gen2 and ci_cpp_gen3 packages provides continuous integration tests for calibration production code. This supplements unit tests in cp_pipe as it can do more computationally expensive tests.

For more information, refer to RFC-718.

Rename the cat package to sdm_schemas¶

The package cat was renamed to sdm_schemas, which contains database schemas for output tables.

For more information, refer to RFC-672.

Addition of the ctrl_bps package¶

The ctrl_bps package provides a PipelineTask execution framework for multi-node processing for the LSST Batch Production Service (BPS).

For more information, refer to RFC-736.

Addition of the pipelines_check package¶

The new pipelines_check package replaces lsst_dm_stack_demo, which relied on the now release-v21-0-0-remove-obs_sdss. This new package demonstrates the processing of a single CCD with the Science Pipelines. It contains input files and astrometric reference catalogs needed to process the data from a single HSC detector. Its main purpose is to check that the pipeline infrastructure is working correctly.

Removal of the obs_sdss package¶

The obs_sdss camera package has been removed from the Science Pipelines before the release of version 21.0.0. Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed. The lsst_dm_stack_demo package, which makes use of obs_sdss was replaced with a package providing similar functionality, Addition of the pipelines_check package. [RFC-645, DM-22578].

Removal of the Synpipe package¶

The Synpipe package is no longer included as part of the Science Pipelines. Tasks for inserting simulated sources are now available as part of pipe_tasks; see community.lsst.org for details. [RFC-661, DM-23344]

Removal of the pex_policy package¶

The obsolete pex_policy package has been removed from the Science Pipelines before the release of version 21.0.0. Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed. Calls to lsst.pex.policy.makePolicy should be modernized to become calls to lsst.pex.config.makePropertySet. [RFC-626, DM-23374]

Modernization of the C++ API in `lsst.afw`¶

As announced in v20.0, a number of deprecated C++ APIs in lsst.afw have been removed. Third-party calls to most of these APIs will simply fail to compile. However, calls to afw::math::convolve that previously passed individual bool values for their final argument(s) instead of using ConvolutionControl may inadvertently call the version with ConvolutionControl, leading to subtle bugs or segmentation faults. [RFC-687, DM-17566, DM-22814, DM-23076, DM-24565]

Pending Deprecations¶

These packages/functions will be deprecated in the next major release.

Removal of “Generation 2” Middleware
Removal of MetricTask

Removal of “Generation 2” Middleware¶

The “Generation 3” middleware first included in the 17.0.0 release is ultimately intended to supplant the current (“Generation 2”) Data Butler and command-line task functionality. When the new middleware is stable and has reached feature parity with the old code, deprecation of the Generation 2 system will be announced. At that time, guidance for transitioning to the new system will be made available. This will include moving migrating from lsst.daf.persistence to lsst.daf.butler, and lsst.pipe.base.CmdLineTask will be replaced by lsst.pipe.base.PipelineTask. Progress towards the deprecation of Generation 2 may be tracked on DM-20572.

Removal of MetricTask¶

The MetricTask is replaced by lsst.verify.tasks.MetricTask. It will be removed along with daf_persistence. [DM-21910]

Deprecations¶

These packages/functions are deprecated and will not be available in the next major release.

Removal of the obs_ctio0m9 package
Deprecated configurations in isrTask.py
Deprecated metric configurations in commonMetric.py
Deprecated doApplyUberCal in forcedPhotCcd.py
Deprecated configurations in fgcmFitCycle.py
Deprecated configurations in psfexStarSelector.py

Removal of the obs_ctio0m9 package¶

The obs_ctio0m9 camera package will be removed from the Science Pipelines before the release of version 22.0.0. [RFC-729, DM-26867, DM-26868]

Deprecated configurations in isrTask.py¶

The following configurations will be removed before the 22.0.0 release:

overscanFitType
overscanOrder
overscanNumSigmaClip
overscanIsInt

Please configure overscan via the OverscanCorrectionConfig interface.

Deprecated metric configurations in commonMetric.py¶

The metric field has been replaced by connections.package and connections.metric. It will be removed along with daf_persistence.

Deprecated doApplyUberCal in forcedPhotCcd.py¶

The field doApplyUberCal is deprecated. Use doApplyExternalPhotoCalib and code:doApplyExternalSkyWcs instead. It will be removed before the 22.0.0 release. [DM-23352]

Deprecated configurations in fgcmFitCycle.py¶

The following configurations are no longer used and will be removed before the 22.0.0 release:

fitFlag
requiredFlag
superStarSubCcd
ccdGraySubCcd
expGrayPhotometricCut
expGrayHighCut
expVarGrayPhotometricCut
aperCorrInputSlopes
sedFudgeFactors, use sedSlopeMap instead
sigFgcmMaxEGray, use sigFgcmMaxEGrayDict instead
approxThroughput, use approxThroughputDict instead
colorSplitIndices, use colorSplitBands instead
useRepeatabilityForExpGrayCuts, use useRepeatabilityForExpGrayCutsDict instead

[DM-23699]

Deprecated configurations in psfexStarSelector.py¶