.. _release-v18-0-0: ########################### Release 18.0.0 (2019-07-09) ########################### .. toctree:: :hidden: tickets/v18_0_0 +-------------------------------------------+------------+ | Source | Identifier | +===========================================+============+ | Git tag | 18.0.0 | +-------------------------------------------+------------+ | :doc:`EUPS distrib ` | v18\_0\_0 | +-------------------------------------------+------------+ This release is based on the ``w_2019_23`` weekly build. These release notes highlight significant changes to the Science Pipelines codebase which are likely to be of wide interest. For a complete list of changes made, see :doc:`tickets/v18_0_0`. As of this release, we have adopted the “`Semantic Versioning `_” system, version 2.0.0. In this system, releases are described by a three part number representing major (18 in this case), minor (0), and patch (0) version. The major version is incremented on incompatible API changes; the minor version when new functionality is added; and the patch version when bug fixes are made. For further details, refer to :dmtn:`106`. If you have questions or comments about this release, visit our `Community Forum `_ for advice. - :ref:`release-v18-0-0-functionality` - :ref:`release-v18-0-0-interface` - :ref:`release-v18-0-0-pending-deprecations` - :ref:`release-v18-0-0-deprecations` *See also:* - :doc:`Installation instructions <../install/index>` - :doc:`Known issues ` - `Characterization Metric Report (DMTR-141) `_ - `Doxygen Documentation`__ __ http://doxygen.lsst.codes/stack/doxygen/xlink_master_2019_06_08_08.07.58/> .. _release-v18-0-0-functionality: Major Functionality Changes =========================== - :ref:`release-v18-0-0-ngmix` - :ref:`release-v18-0-0-fakes` - :ref:`release-v18-0-0-fgcm` - :ref:`release-v18-0-0-timeseries` .. _release-v18-0-0-ngmix: Improved integration with the ngmix package ------------------------------------------- meas_extensions_ngmix, which provides an interface between the LSST codebase and Erin Sheldon's `ngmix`__ package, has been substantially rewritten. This new package exposes more ngmix functionality, and may act as the basis for future development of the LSST measurement framework. For more information, refer to :jira:`DM-16268`. __ https://github.com/esheldon/ngmix .. _release-v18-0-0-fakes: New tasks for inserting “fake” sources -------------------------------------- New command-line tasks have been added for inserting simulated sources into single epoch (:command:`processFakes.py`) and coadded (:command:`insertFakes.py`) data. For more information, refer to `this community.lsst.org post`__. __ https://community.lsst.org/t/new-tasks-for-fake-source-insertion/3722 .. _release-v18-0-0-fgcm: FCGM can now use reference stars -------------------------------- By default, fgcmcal — LSST's implementation of the Forward Global Calibration Method (`Burke et al., 2018`__) — does not use an external reference catalog, but solves internally to produce a relative calibration. It has now been upgraded to optionally load a set of reference stars, and use them to produce an absolute calibration. For more information, refer to :jira:`DM-16702`. __ http://adsabs.harvard.edu/abs/2018AJ....155...41B .. _release-v18-0-0-timeseries: DIAObjects now include basic lightcurve characterization -------------------------------------------------------- DIAObjects generated by the Alert Production system now include a basic set of features which extracted from the object's light-curve (mean fluxes, best-fit slope, etc). While it is hoped that these features are intrinsically useful, they also serve to exercise the machinery which will be used to calculate the full set of features which is currently being developed in conjunction with the relevant science collaborations. For more details about this functionality, refer to :jira:`DM-18318`; for information about the ultimate set of features which will be computed, refer to :jira:`DM-11962` and :lse:`163`. .. _release-v18-0-0-interface: Significant Interface Changes ============================= - :ref:`release-v18-0-0-propertyset` - :ref:`release-v18-0-0-ap_pipe` - :ref:`release-v18-0-0-selectors` - :ref:`release-v18-0-0-calib` .. _release-v18-0-0-propertyset: Python interface to PropertySet and PropertyList changed -------------------------------------------------------- ``__getitem__``, ``update``, and ``get`` methods have been added to the Python interfaces of `~lsst.daf.base.PropertySet` and `~lsst.daf.base.PropertyList`. This means that they can be used in the same way as native Python dictionaries. It is also possible to store the value `None` in a `~lsst.daf.base.PropertySet`. For more information, refer to `this community.lsst.org post`__ and :jira:`RFC-596`. __ https://community.lsst.org/t/changes-to-propertyset-and-propertylist-python-interface/3715 .. _release-v18-0-0-ap_pipe: Alert Production Pipeline command-line interface changed -------------------------------------------------------- The :command:`ap_pipe.py` command will no longer try to create a “prompt products” database when it is executed. This change makes the pipeline better able to run in different environments, including large-scale testing and operations. An appropriate database should now be created in advance, either by using the :command:`make_ppdb.py` command, or by configuring the pipeline to use an externally-provided database. For more information, refer to `this community.lsst.org post`__ and :jira:`RFC-587`. __ https://community.lsst.org/t/ap-pipeline-ap-pipe-py-command-line-interface-change/3646 .. _release-v18-0-0-selectors: New configurations for AstrometryTask source and reference selectors -------------------------------------------------------------------- The configuration options for single-frame astrometry (as implemented in :lsst-task:`lsst.meas.astrom.AstrometryTask`) have changed. This fixes various bugs where (a) selections of the reference catalog were performed only in some modes of operation; and (b) multiple conflicting selections of the source catalog could be performed in some modes of operation. All obs package defaults have been updated to reflect the new changes; you need only worry about these changes if you have overridden the obs package defaults. For more information, refer to `this community.lsst.org post`__ and :jira:`RFC-589`. __ https://community.lsst.org/t/new-configurations-for-astrometrytask-source-and-reference-selectors/3661 .. _release-v18-0-0-calib: `lsst.afw.image.Calib` removed ------------------------------ `~lsst.afw.image.Calib`, which provided only a photometric zeropoint per CCD, has been replaced with `~lsst.afw.image.PhotoCalib`, which provides a spatially-varying photometric model. Some `~lsst.afw.image.Calib` interfaces are supported by `~lsst.afw.image.PhotoCalib`, but full API compatibility is not possible; using the old-style interfaces is deprecated, and they will be removed following this release. `~lsst.afw.image.PhotoCalib` is able to read files persisted with `~lsst.afw.image.Calib` objects, so backwards compatibility of on-disk data is maintained. For more information, refer to :jira:`RFC-289` and :jira:`RFC-573`. .. _release-v18-0-0-pending-deprecations: Pending Deprecations ==================== These packages/functions will be deprecated in the next major release. - :ref:`release-v18-0-0-deprecate-gen2` - :ref:`release-v18-0-0-deprecate-lsstsim` - :ref:`release-v18-0-0-deprecate-afwGeom` .. _release-v18-0-0-deprecate-gen2: Upcoming removal of “Generation 2” Middleware --------------------------------------------- The “Generation 3” middleware :ref:`included in the previous release ` is ultimately intended to supplant the current (“Generation 2”) Data Butler and command-line task functionality. We expect to deliver a final major release supporting the Generation 2 functionality in late calendar year 2019 (likely version 19.0.0, but that remains to be confirmed). Following that release, the “Generation 2” middleware will be removed from the codebase. This will include: - The daf_persistence package, to be replaced by daf_butler; - `lsst.pipe.base.CmdLineTask`, to be replaced by `lsst.pipe.base.PipelineTask`; - The pipe_drivers and ctrl_pool packages, for which replacements are still in development. .. _release-v18-0-0-deprecate-lsstSim: Upcoming removal of the obs_lsstSim package ------------------------------------------- The obs_lsst package, :ref:`included in the previous release `, obviates the need for the obs_lsstSim package. All LSST code is expected to transition to the new system later in summer 2019. Some work will be required to update old data repositories to the new system. After that, a final release will be made containing obs_lsstSim in late 2019, after which the package will be retired. .. _release-v18-0-0-deprecate-afwGeom: Upcoming removal of `lsst.afw.geom` classes that have been relocated to `lsst.geom` ----------------------------------------------------------------------------------- As announced in v16.0 (:ref:`release-v16-0-new-geom`) some primitives have been moved from `afw.geom` to `geom`. We currently provide aliases for compatibility purposes, but new code should use the `geom` package. These aliases will be removed after the version 19.0.0 will be released. .. _release-v18-0-0-deprecations: Deprecations ============ These packages/functions are deprecated and will not be available in the next major release. - :ref:`release-v18-0-0-deprecate-calib` - :ref:`release-v18-0-0-deprecate-ap-silent` - :ref:`release-v18-0-0-deprecate-isr` - :ref:`release-v18-0-0-deprecate-meas_algorithms` .. _release-v18-0-0-deprecate-calib: Upcoming removal of `lsst.afw.image.Calib` compatibility API ------------------------------------------------------------ This release includes (partial) backwards compatibility with the now removed (:ref:`see above `) `~lsst.afw.image.Calib` API. This will be removed before the next release. .. _release-v18-0-0-deprecate-ap-silent: Upcoming removal of the ``--silent`` argument to :command:`ap_verify.py` ------------------------------------------------------------------------ The ``--silent`` argument used to disable upload of metrics from :command:`ap_verify.py` to `SQuaSH`__. The capability to upload metrics has been removed from :command:`ap_verify.py` (see :jira:`DM-16536`), but ``--silent`` has been retained as a no-op for compatibility reasons. It will be removed before the next release. .. _release-v18-0-0-deprecate-isr: Upcoming removal of `ip_isr` functions from `isrFunctions.py` ------------------------------------------------------------- These functions are replaced by functionality in `lsst.meas.algorithms.Defects`: - ``defectListFromFootprintList`` replaced by ``Defects.fromFootPrintList()`` - ``transposeDefectList`` replaced by ``Defects.transpose()`` - ``maskPixelsFromDefectList`` replaced by ``Defects.maskPixels()`` - ``getDefectListFromMask`` replaced by ``Defects.fromMask()`` .. _release-v18-0-0-deprecate-meas_algorithms: Upcoming removal of `meas_algorithms` functions from `defects.py` ----------------------------------------------------------------- - ``policyToBadRegionList``, policy defect files no longer supported. __ https://squash.lsst.codes