Release 18.0.0 (2019-07-09)¶
Source | Identifier |
---|---|
Git tag | 18.0.0 |
EUPS distrib | v18_0_0 |
This release is based on the w_2019_23
weekly build.
These release notes highlight significant changes to the Science Pipelines codebase which are likely to be of wide interest. For a complete list of changes made, see Tickets Addressed in Release 18.0.0.
As of this release, we have adopted the “Semantic Versioning” system, version 2.0.0. In this system, releases are described by a three part number representing major (18 in this case), minor (0), and patch (0) version. The major version is incremented on incompatible API changes; the minor version when new functionality is added; and the patch version when bug fixes are made. For further details, refer to DMTN-106.
If you have questions or comments about this release, visit our Community Forum for advice.
See also:
- Installation instructions
- Known issues
- Characterization Metric Report (DMTR-141)
- Doxygen Documentation
Major Functionality Changes¶
- Improved integration with the ngmix package
- New tasks for inserting “fake” sources
- FCGM can now use reference stars
- DIAObjects now include basic lightcurve characterization
Improved integration with the ngmix package¶
meas_extensions_ngmix, which provides an interface between the LSST codebase and Erin Sheldon’s ngmix package, has been substantially rewritten. This new package exposes more ngmix functionality, and may act as the basis for future development of the LSST measurement framework. For more information, refer to DM-16268.
New tasks for inserting “fake” sources¶
New command-line tasks have been added for inserting simulated sources into single epoch (processFakes.py) and coadded (insertFakes.py) data. For more information, refer to this community.lsst.org post.
FCGM can now use reference stars¶
By default, fgcmcal — LSST’s implementation of the Forward Global Calibration Method (Burke et al., 2018) — does not use an external reference catalog, but solves internally to produce a relative calibration. It has now been upgraded to optionally load a set of reference stars, and use them to produce an absolute calibration. For more information, refer to DM-16702.
DIAObjects now include basic lightcurve characterization¶
DIAObjects generated by the Alert Production system now include a basic set of features which extracted from the object’s light-curve (mean fluxes, best-fit slope, etc). While it is hoped that these features are intrinsically useful, they also serve to exercise the machinery which will be used to calculate the full set of features which is currently being developed in conjunction with the relevant science collaborations. For more details about this functionality, refer to DM-18318; for information about the ultimate set of features which will be computed, refer to DM-11962 and LSE-163.
Significant Interface Changes¶
- Python interface to PropertySet and PropertyList changed
- Alert Production Pipeline command-line interface changed
- New configurations for AstrometryTask source and reference selectors
- lsst.afw.image.Calib removed
Python interface to PropertySet and PropertyList changed¶
__getitem__
, update
, and get
methods have been added to the Python interfaces of PropertySet
and PropertyList
.
This means that they can be used in the same way as native Python dictionaries.
It is also possible to store the value None
in a PropertySet
.
For more information, refer to this community.lsst.org post and RFC-596.
Alert Production Pipeline command-line interface changed¶
The ap_pipe.py command will no longer try to create a “prompt products” database when it is executed. This change makes the pipeline better able to run in different environments, including large-scale testing and operations. An appropriate database should now be created in advance, either by using the make_ppdb.py command, or by configuring the pipeline to use an externally-provided database. For more information, refer to this community.lsst.org post and RFC-587.
New configurations for AstrometryTask source and reference selectors¶
The configuration options for single-frame astrometry (as implemented in lsst.meas.astrom.AstrometryTask
) have changed.
This fixes various bugs where (a) selections of the reference catalog were performed only in some modes of operation; and (b) multiple conflicting selections of the source catalog could be performed in some modes of operation.
All obs package defaults have been updated to reflect the new changes; you need only worry about these changes if you have overridden the obs package defaults.
For more information, refer to this community.lsst.org post and RFC-589.
lsst.afw.image.Calib
removed¶
Calib
, which provided only a photometric zeropoint per CCD, has been replaced with PhotoCalib
, which provides a spatially-varying photometric model.
Some Calib
interfaces are supported by PhotoCalib
, but full API compatibility is not possible; using the old-style interfaces is deprecated, and they will be removed following this release.
PhotoCalib
is able to read files persisted with Calib
objects, so backwards compatibility of on-disk data is maintained.
For more information, refer to RFC-289 and RFC-573.
Pending Deprecations¶
These packages/functions will be deprecated in the next major release.
- Upcoming removal of “Generation 2” Middleware
- Upcoming removal of the obs_lsstSim package
- Upcoming removal of lsst.afw.geom classes that have been relocated to lsst.geom
Upcoming removal of “Generation 2” Middleware¶
The “Generation 3” middleware included in the previous release is ultimately intended to supplant the current (“Generation 2”) Data Butler and command-line task functionality. We expect to deliver a final major release supporting the Generation 2 functionality in late calendar year 2019 (likely version 19.0.0, but that remains to be confirmed). Following that release, the “Generation 2” middleware will be removed from the codebase. This will include:
- The daf_persistence package, to be replaced by daf_butler;
lsst.pipe.base.CmdLineTask
, to be replaced bylsst.pipe.base.PipelineTask
;- The pipe_drivers and ctrl_pool packages, for which replacements are still in development.
Upcoming removal of the obs_lsstSim package¶
The obs_lsst package, included in the previous release, obviates the need for the obs_lsstSim package. All LSST code is expected to transition to the new system later in summer 2019. Some work will be required to update old data repositories to the new system. After that, a final release will be made containing obs_lsstSim in late 2019, after which the package will be retired.
Upcoming removal of lsst.afw.geom
classes that have been relocated to lsst.geom
¶
As announced in v16.0 (Reworked “geom” package, replacing much of “afw.geom”) some primitives have been moved from afw.geom
to geom
.
We currently provide aliases for compatibility purposes, but new code should use the geom
package.
These aliases will be removed after the version 19.0.0 will be released.
Deprecations¶
These packages/functions are deprecated and will not be available in the next major release.
- Upcoming removal of lsst.afw.image.Calib compatibility API
- Upcoming removal of the --silent argument to ap_verify.py
- Upcoming removal of ip_isr functions from isrFunctions.py
- Upcoming removal of meas_algorithms functions from defects.py
Upcoming removal of lsst.afw.image.Calib
compatibility API¶
This release includes (partial) backwards compatibility with the now removed (see above) Calib
API.
This will be removed before the next release.
Upcoming removal of the --silent
argument to ap_verify.py¶
The --silent
argument used to disable upload of metrics from ap_verify.py to SQuaSH.
The capability to upload metrics has been removed from ap_verify.py (see DM-16536), but --silent
has been retained as a no-op for compatibility reasons.
It will be removed before the next release.
Upcoming removal of ip_isr
functions from isrFunctions.py
¶
These functions are replaced by functionality in lsst.meas.algorithms.Defects
:
defectListFromFootprintList
replaced byDefects.fromFootPrintList()
transposeDefectList
replaced byDefects.transpose()
maskPixelsFromDefectList
replaced byDefects.maskPixels()
getDefectListFromMask
replaced byDefects.fromMask()
Upcoming removal of meas_algorithms
functions from defects.py
¶
policyToBadRegionList
, policy defect files no longer supported.