Release 20.0.0 (2020-06-24)¶
This release is based on the
w_2020_22 weekly build.
The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 20.0.0.
The Characterization Metric Report (DMTR-251) describes the scientific performance of this release in terms of scientific performance metrics, while Changes to Data Products in Release 20.0.0 covers key issues likely to impact scientists consuming Rubin data products.
If you have questions or comments about this release, visit our community forum for advice.
Major New Features¶
Collimated beam projector support¶
This release adds the cbp package to the Science Pipelines. This provides codes for working with the collimated beam projector (CBP). In particular, cbp can compute the telescope and CBP pointing that will produce a desired beam arrangement, and can compute the hole positions for a CBP mask. [RFC-658]
Alert packets are now generated during Alert Production Pipeline execution¶
The Alert Production system can now generate schema-compliant alerts in Apache Avro format. Note that these alerts do not yet provide all contents specified by LSE-163 (the Data Products Definition Document) — in particular, they do not include cut-out images — and are currently written to disk, rather than broadcast over the Alert Distribution system. In support of this effort, the alert_packet package was added to the Science Pipelines. This provides a number of utility routines for manipulating alert packets. [DM-24324]
Improved linearity correction¶
The mechanisms for correcting for detector linearity have been substantially improved. In particular, the new system makes it easy for user-supplied linearity corrections to override the defaults, and a new generic polynomial linearity correction type has been added. Refer to community.lsst.org and RFC-665 for details. [DM-23023]
Sky sources added to single frame processing¶
“Sky sources” — corresponding to measurements made on empty patches of sky — may now be inserted into single-frame processing. These provide an important source of QA data. See community.lsst.org for details. [DM-23078]
Documentation for fgcmcal¶
Documentation for fgcmcal, including reference material on all included tasks and a “cookbook” providing a tutorial-style introduction, is now available. This package runs the Forward Global Calibration Method to perform global photometric survey calibration; refer to Burke, Rykoff, et al. 2018 for details. [DM-22469]
fgcmcal calibration on local background-corrected fluxes¶
The fgcmcal tool can now correct for the local background on aperture fluxes during calibration. This brings faint stars more into line with their bright counterparts, and thereby improves the resultant calibration. [DM-23036]
Conversion of Generation 2 to Generation 3 data repositories¶
Significant Interface Changes¶
Third party packages managed through Conda¶
The Conda package management system, together with the Conda-Forge package collection, is now used to provide all compilers and most third-party dependencies. This substantially simplifies Installation and makes it easier to integrate the Science Pipelines with other tools, while also facilitating quicker updates to third-party code during development.
For more information, refer to community.lsst.org.
All FITS headers are now stored in uppercase¶
The version of CFITSIO used in this release requires that all header keys be written in uppercase to better comply with the FITS standard. Before this release, some Science Pipelines code would store information in FITS headers in lowercase. This has the consequence that:
If user code attempts to write a lowercase header to FITS, it will be automatically converted to uppercase, and a warning will be issued;
If user code attempts to read FITS data which includes lowercase headers (for example, because it was written by an earlier Pipelines release), they will automatically be converted to uppercase, and an informational message may be logged.
Changes to collections, runs, and
pipetask arguments in Generation 3 middleware¶
The way in which collections behave in the Generation 3 middleware has been substantially overhauled. In particular:
a “run” is now a special type of collection, rather than an entity that is associated with a collection;
the only kind of collection we had before is now called a “tagged” collection;
we now also have “chained” collections, which are simply an ordered list of other collections to be searched.
This changes some prominent interfaces, including both the arguments used to construct a
Butler and the command-line arguments to the
Changes to building using lsstsw¶
The initialization scripts used by the lsstsw system, used by some developers to install and update the Science Pipelines, have been renamed.
Where previously you sourced
setup.csh), you should now source
This helps distinguish this script from the setup command provided by EUPS.
Jointcal now defaults to using Gaia and PanSTARRS1 reference catalogs¶
By default, Jointcal will attempt to use reference catalogs from Gaia (
gaia_dr2_20200414) and PanSTARRS1 (
ps1_pv3_3pi_20170110) for astrometry and photometry respectively.
If these catalogs are not available, you must specify alternatives by setting
Removal of the obs_lsstSim package¶
lsst.afw.geom aliases for code relocated to
As announced in v16.0 (Reworked “geom” package, replacing much of “afw.geom”) some primitives have been moved from
Compatibility aliases making them accessible under their old names have now been removed, and all code must use the
geom package directly.
This obsolete method has been unused since release 11.0 (September 2015). It has therefore been removed. [DM-23371]
Removal of astrometry_net_data¶
The obsolete astrometry_net_data package has been removed. [DM-22233]
This configuration field was no longer used, and has therefore been removed. [DM-23380]
These packages/functions will be deprecated in the next major release.
Removal of “Generation 2” Middleware¶
The “Generation 3” middleware included in the 17.0.0 release is ultimately intended to supplant the current (“Generation 2”) Data Butler and command-line task functionality.
When the new middleware is stable and has reached feature parity with the old code, deprecation of the Generation 2 system will be announced.
At that time, guidance for transitioning to the new system will be made available.
This will include moving migrating from
lsst.daf.persistence to to
lsst.daf.butler, and from
lsst.pipe.base.CmdLineTask, to be replaced by
Progress towards the deprecation of Generation 2 may be tracked on DM-20572.
These packages/functions are deprecated and will not be available in the next major release.
Camera geometry-based distortions are now automatically incorporated into the WCS automatically when reading raw data, rendering
makeDistortedTanWcs will be removed before the release of version 21.0.0
[RFC-616, DM-20937, DM-22114]
Modernization of the C++ API in
Removal of the Synpipe package¶
The Synpipe package for inserting fake galaxies and stars during pipeline execution has been rendered obsolete by the tasks for fake source insertion which were added to pipe_tasks in release 18.0.0. It will therefore be removed before the release of version 21.0.0. [RFC-661, DM-23343]
Removal of the obs_sdss package¶
The obs_sdss camera package will be removed from the Science Pipelines before the release of version 21.0.0. Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed. The lsst_dm_stack_demo package, which makes use of obs_sdss, will be replaced with a new package providing similar functionality. [RFC-645, DM-22578].
Removal of the pex_policy package¶
The obsolete pex_policy package will be removed from the Science Pipelines before the release of version 21.0.0.
Note that this package was previously slated for removal before the release of version 20.0.0, but this has been delayed.
lsst.pex.policy.makePolicy should be modernized to become calls to