Release 21.0.0 (2020-12-08)¶
Source | Identifier |
---|---|
Git tag | 21.0.0 |
EUPS distrib | v21_0_0 |
This release is based on the w_2020_46
weekly build.
The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 21.0.0.
The Characterization Metric Report (DMTR-281) describes the scientific performance of this release in terms of scientific performance metrics, while Changes to Data Products in Release 21.0.0 covers key issues likely to impact scientists consuming Rubin data products.
If you have questions or comments about this release, visit our community forum for advice.
See also:
Major New Features¶
- Beta release of Generation 3 Middleware
- Task to mask streak-like artifacts
- Inject arbitrary input images
- Proper motions and parallaxes applied to reference sources
- Addition of ci_cpp
Beta release of Generation 3 Middleware¶
Reaching schema stability in this release, the Gen3 butler can now support Science Pipelines development. See Removal of “Generation 2” Middleware for more information on the planned deprecation of Generation 2 middleware. Interested parties may find more info at daf_butler and pipe_base. A general release of Generation 3 Middleware is expected with the next release v22.0.0.
Task to mask streak-like artifacts¶
This release adds the MaskStreaksTask
to pipe_tasks, which finds and masks streaks and other straight lines in image data.
It uses the Kernel Hough Transform (KHT) (Fernandes and Oliveira 2007) implemented in the kht, package which was also added to the Science Pipelines.
It can be turned on during coaddition with CompareWarpAssembleCoaddTask
with the config parameter doFilterMorphological
.
Inject arbitrary input images¶
The task to inject simulated sources into data release processing, InsertFakesTask
, can now accept arbitrary input images.
[DM-25747]
Proper motions and parallaxes applied to reference sources¶
jointcal now can apply proper motions and parallaxes to reference sources during astrometric fitting. [DM-24277]
Addition of ci_cpp¶
The ci_cpp
metapackage for the ci_cpp_gen2 and ci_cpp_gen3 packages provides continuous integration tests for calibration production code. This supplements unit tests in cp_pipe as it can do more computationally expensive tests.
For more information, refer to RFC-718.
Significant Interface Changes¶
- Rename the cat package to sdm_schemas
- Addition of the ctrl_bps package
- Addition of the pipelines_check package
- Removal of the Synpipe package
- Removal of the obs_sdss package
- Removal of the pex_policy package
- Modernization of the C++ API in lsst.afw
Rename the cat package to sdm_schemas¶
The package cat
was renamed to sdm_schemas, which contains database schemas for output tables.
For more information, refer to RFC-672.
Addition of the ctrl_bps package¶
The ctrl_bps package provides a PipelineTask execution framework for multi-node processing for the LSST Batch Production Service (BPS).
For more information, refer to RFC-736.
Addition of the pipelines_check package¶
The new pipelines_check package replaces lsst_dm_stack_demo
, which relied on the now release-v21-0-0-remove-obs_sdss. This new package demonstrates the processing of a single CCD with the Science Pipelines. It contains input files and astrometric reference catalogs needed to process the data from a single HSC detector. Its main purpose is to check that the pipeline infrastructure is working correctly.
Removal of the obs_sdss package¶
The obs_sdss camera package has been removed from the Science Pipelines before the release of version 21.0.0. Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed. The lsst_dm_stack_demo package, which makes use of obs_sdss was replaced with a package providing similar functionality, Addition of the pipelines_check package. [RFC-645, DM-22578].
Removal of the Synpipe package¶
The Synpipe package is no longer included as part of the Science Pipelines. Tasks for inserting simulated sources are now available as part of pipe_tasks; see community.lsst.org for details. [RFC-661, DM-23344]
Removal of the pex_policy package¶
The obsolete pex_policy package has been removed from the Science Pipelines before the release of version 21.0.0.
Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed.
Calls to lsst.pex.policy.makePolicy
should be modernized to become calls to lsst.pex.config.makePropertySet
.
[RFC-626, DM-23374]
Modernization of the C++ API in lsst.afw
¶
As announced in v20.0, a number of deprecated C++ APIs in lsst.afw
have been removed. Third-party calls to most of these APIs will simply
fail to compile. However, calls to
afw::math::convolve
that previously passed
individual bool
values for their final argument(s) instead of using
ConvolutionControl
may inadvertently call the version
with ConvolutionControl
, leading to subtle bugs or
segmentation faults.
[RFC-687, DM-17566, DM-22814, DM-23076, DM-24565]
Pending Deprecations¶
These packages/functions will be deprecated in the next major release.
Removal of “Generation 2” Middleware¶
The “Generation 3” middleware first included in the 17.0.0 release is ultimately intended to supplant the current (“Generation 2”) Data Butler and command-line task functionality.
When the new middleware is stable and has reached feature parity with the old code, deprecation of the Generation 2 system will be announced.
At that time, guidance for transitioning to the new system will be made available.
This will include moving migrating from lsst.daf.persistence
to lsst.daf.butler
, and lsst.pipe.base.CmdLineTask
will be replaced by lsst.pipe.base.PipelineTask
.
Progress towards the deprecation of Generation 2 may be tracked on DM-20572.
Removal of MetricTask¶
The MetricTask
is replaced by lsst.verify.tasks.MetricTask
.
It will be removed along with daf_persistence.
[DM-21910]
Deprecations¶
These packages/functions are deprecated and will not be available in the next major release.
- Removal of the obs_ctio0m9 package
- Deprecated configurations in isrTask.py
- Deprecated metric configurations in commonMetric.py
- Deprecated doApplyUberCal in forcedPhotCcd.py
- Deprecated configurations in fgcmFitCycle.py
- Deprecated configurations in psfexStarSelector.py
Removal of the obs_ctio0m9 package¶
The obs_ctio0m9 camera package will be removed from the Science Pipelines before the release of version 22.0.0. [RFC-729, DM-26867, DM-26868]
Deprecated configurations in isrTask.py¶
The following configurations will be removed before the 22.0.0 release:
- overscanFitType
- overscanOrder
- overscanNumSigmaClip
- overscanIsInt
Please configure overscan via the OverscanCorrectionConfig
interface.
Deprecated metric configurations in commonMetric.py¶
The metric
field has been replaced by connections.package
and
connections.metric
. It will be removed along with daf_persistence.
Deprecated doApplyUberCal in forcedPhotCcd.py¶
The field doApplyUberCal
is deprecated.
Use doApplyExternalPhotoCalib
and code:doApplyExternalSkyWcs
instead.
It will be removed before the 22.0.0 release.
[DM-23352]
Deprecated configurations in fgcmFitCycle.py¶
The following configurations are no longer used and will be removed before the 22.0.0 release:
- fitFlag
- requiredFlag
- superStarSubCcd
- ccdGraySubCcd
- expGrayPhotometricCut
- expGrayHighCut
- expVarGrayPhotometricCut
- aperCorrInputSlopes
- sedFudgeFactors, use sedSlopeMap instead
- sigFgcmMaxEGray, use sigFgcmMaxEGrayDict instead
- approxThroughput, use approxThroughputDict instead
- colorSplitIndices, use colorSplitBands instead
- useRepeatabilityForExpGrayCuts, use useRepeatabilityForExpGrayCutsDict instead
[DM-23699]
Deprecated configurations in psfexStarSelector.py¶
The following configurations are no longer used and will be removed before the 22.0.0 release:
- maxbad
- maxbadflag