.. _release-latest: .. _release-v22-0-0: ################################### Release 22.0.0 (YYYY-MM-DD) ################################### .. toctree:: :hidden: tickets/v22_0_0 +-------------------------------------------+------------+ | Source | Identifier | +===========================================+============+ | Git tag | 22.0.0 | +-------------------------------------------+------------+ | :doc:`EUPS distrib ` | v22\_0\_0 | +-------------------------------------------+------------+ This release is based on the ``w_2021_14`` weekly build. The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see :doc:`tickets/v22_0_0`. The `Characterization Metric Report (DMTR-311) `_ describes the scientific performance of this release in terms of scientific performance metrics. If you have questions or comments about this release, visit our `community forum `_ for advice. - :ref:`release-v22-0-0-functionality` - :ref:`release-v22-0-0-interface` - :ref:`release-v22-0-0-pending-deprecations` - :ref:`release-v22-0-0-deprecations` *See also:* - :doc:`Installation instructions ` - :doc:`Known issues ` - `Doxygen Documentation`__ __ http://doxygen.lsst.codes/stack/doxygen/xlink_master_2021_04_01_08.31.19/ .. _release-v22-0-0-functionality: Major New Features ================== - :ref:`release-v22-0-0-gen3` - :ref:`release-v22-0-0-meas_extensions_piff` - :ref:`release-v22-0-0-scarlet` - :ref:`release-v22-0-0-dustmaps` - :ref:`release-v22-0-0-faro` .. _release-v22-0-0-gen3: Generation 3 Middleware ----------------------- Major improvements have been made to the Generation 3 middleware. Some highlights include: * Usability improvements to ``PipelineTask`` including URI support. * Additional the ``butler`` command-line commands including * ``butler prune-datasets``, * ``butler query-dimension-records``, * and ``butler associate``. * Allow ``butler.get()`` to support dimension record values such as exposure observing day or detector name in the dataID. * ``pipetask run`` can now execute a subset of a graph. This allows a single graph file to be created with an entire workflow and then only part of it to be executed, necessary for large scale workflow execution. At the time of this release, :ref:`Gen 2 middleware is now deprecated `. .. _release-v22-0-0-meas_extensions_piff: Initial integration of Piff --------------------------- `Piff `_ has been added to rubin-env as a third party package, and the integration package `meas_extension_piff`_ has been added to lsst_distrib. `meas_extension_piff`_ integrates Piff with the pipelines. Tasks can be configured to use Piff PSF models instead of the current default PSFex. :jirab:`RFC-755` .. _meas_extension_piff: https://github.com/lsst/meas_extensions_piff .. _release-v22-0-0-scarlet: Coadd measurement now uses the Scarlet deblender by default ----------------------------------------------------------- `Scarlet` is now the default deblender used for measurements on coadds. The configuration for single-frame measurement has not changed. Scarlet produces additional flags for filtering duplicates, as it handles isolated sources differently: :code:`deblend_parentNPeaks`, :code:`deblend_parentNChild`. :code:`detect_isPrimary` is still populated, and is the recommended column. See the `community.lsst.org post `_ for more information. :jirab:`RFC-745` .. _release-v22-0-0-dustmaps: Dust Maps --------- The `dustmaps_cachedata `_ package has been added to the pipelines, and the `dustmaps `_ module is a third-party dependency. :jirab:`RFC-752` .. _release-v22-0-0-faro: :ref:`Faro ` package for scientific performance metrics ----------------------------------------------------------------------- A package for measuring scientific performance metrics, :ref:`faro `, was added. It implements the Gen3 versions of the tasks in :ref:`validate_drp `. See :ref:`release-v22-0-0-deprecate-validate-drp`. This package was used to generate the Characterization Report for this release. :jirab:`RFC-753, DM-28351` .. _release-v22-0-0-interface: Significant Interface Changes ============================= - :ref:`release-v22-0-0-filterLabel` - :ref:`release-v22-0-0-remove-obs_ctio0m9` - :ref:`release-v22-0-0-remove-metric-in-commonMetrics.py` .. _release-v22-0-0-filterLabel: Replace afw.image.Filter with FilterLabel ----------------------------------------- Filter information is now stored as a FilterLabel, replacing the existing `~lsst.afw.image.filter`. This class replaces `~Filter`'s system of names, canonical names, and aliases with just two names: a `~band` (e.g., “g” or “r”) and a `~physical filter` (e.g., “g DECam SDSS c0001 4720.0 1520.0” or “HSC-R2”). Note that not all FilterLabel objects have both a band and a physical filter, especially during the transition period, so please program defensively. See `community.lsst.org `__ for details. :jirab:`RFC-730` .. _release-v22-0-0-remove-obs_ctio0m9: Removal of the obs_ctio0m9 package ---------------------------------- The `~obs_ctio0m9` camera package has been removed. :jirab:`RFC-729, DM-26867, DM-26868` .. _release-v22-0-0-remove-metric-in-commonMetrics.py: Removal of metric configurations in commonMetric.py --------------------------------------------------- The :code:`metric` field has been replaced by :code:`connections.package` and :code:`connections.metric`. .. _release-v22-0-0-pending-deprecations: Pending Deprecations ==================== There are no known pending deprecations. .. _release-v22-0-0-deprecations: Deprecations ============ These packages/functions are deprecated and will not be available in the next major release. - :ref:`release-v22-0-0-deprecate-gen2` - :ref:`release-v22-0-0-deprecate-validate-drp` - :ref:`release-v22-0-0-deprecate-Filter` - :ref:`release-v22-0-0-deprecate-configurations-in-fgcmFitCycle.py` - :ref:`release-v22-0-0-deprecate-configurations-in-psfexStarSelector.py` .. _release-v22-0-0-deprecate-gen2: Deprecated Generation 2 Middleware ---------------------------------- Generation 2 middleware (Gen2) is no longer being developed and should not be used for new code. Gen2 infrastructure code (within e.g., :ref:`pipe_tasks `, :ref:`pipe_base `, :ref:`obs_base `, `daf_persistence `_, and obs packages) will no longer be maintained after January 1, 2022 and may be removed at any point afterwards. The CI package `ci_hsc_gen2` will continue to be run, and the tasks it checks will be maintained until we remove the Gen2 infrastructure code in 2022. Following this release, we will begin to drop gen2 pipelines from our verification packages (e.g., ap_verify), where a functional and validated gen3 pipeline exists. .. _release-v22-0-0-deprecate-validate-drp: Removal of validate_drp ------------------------ The algorithms implemented in :ref:`validate_drp ` were ported as-is to run in :ref:`faro `. All future development of scientific performance metrics will be carried out in faro. :ref:`validate_drp ` will be removed before the release of version 24.0.0. .. _release-v22-0-0-deprecate-Filter: Removal of `afw.lsst.Filter` ---------------------------- `~afw.lsst.Filter` has been replaced with :ref:`filterLabel`, marked as deprecated, and will be removed before the release of version 24.0.0. :jirab:`RFC-730` .. _release-v22-0-0-deprecate-doApplyUberCal-in-forcedPhotCcd.py: Deprecated doApplyUberCal in forcedPhotCcd.py --------------------------------------------- The field :code:`doApplyUberCal` is deprecated. Use :code:`doApplyExternalPhotoCalib` and :code:`doApplyExternalSkyWcs` instead. It will be removed before the 23.0.0 release. :jirab:`DM-23352` .. _release-v22-0-0-deprecate-configurations-in-isrTask.py: Deprecated configurations in isrTask.py --------------------------------------- The following configurations will be removed before the 23.0.0 release: * :code:`overscanFitType` * :code:`overscanOrder` * :code:`overscanNumSigmaClip` * :code:`overscanIsInt` Please configure overscan via the :code:`OverscanCorrectionConfig` interface. :jirab:`DM-23396` .. _release-v22-0-0-deprecate-configurations-in-fgcmFitCycle.py: Deprecated configurations in fgcmFitCycle.py -------------------------------------------- The following configurations are no longer used and will be removed before the 23.0.0 release: * :code:`fitFlag` * :code:`requiredFlag` * :code:`superStarSubCcd` * :code:`ccdGraySubCcd` * :code:`expGrayPhotometricCut` * :code:`expGrayHighCut` * :code:`expVarGrayPhotometricCut` * :code:`aperCorrInputSlopes` * :code:`sedFudgeFactors`, use :code:`sedSlopeMap` instead * :code:`sigFgcmMaxEGray`, use :code:`sigFgcmMaxEGrayDict` instead * :code:`approxThroughput`, use :code:`approxThroughputDict` instead * :code:`colorSplitIndices`, use :code:`colorSplitBands` instead * :code:`useRepeatabilityForExpGrayCuts`, use :code:`useRepeatabilityForExpGrayCutsDict` instead :jirab:`DM-23699` .. _release-v22-0-0-deprecate-configurations-in-psfexStarSelector.py: Deprecated configurations in psfexStarSelector.py ------------------------------------------------- The following configurations are no longer used and will be removed before the 23.0.0 release: * :code:`maxbad` * :code:`maxbadflag`