Release 25.0.4 (2024-03-20)

Source

Identifier

Git tag

25.0.4

EUPS distrib

v25_0_4

rubin-env version

5.0.1

This release is based on the w_2022_48 weekly build with 39 tickets backported.

The notes below highlight significant technical changes to the Science Pipelines codebase in this release. For a complete list of changes made, see Tickets Addressed in Release 25.0.4.

The Characterization Metric Report (DMTR-302) describes the scientific performance of this release in terms of scientific performance metrics.

If you have questions or comments about this release, visit our Community forum for advice.

See also:

Major new features

Analysis tools

The new package, analysis_tools, implements a framework for modular, scalable, and reproducible computation of metrics and plots to assess both image and pipeline quality.

GBDES

A significantly modified fork of Dark Energy Survey’s astrometric calibration algorithm has been added to the Science Pipelines. gbdes will be used for global astrometric fitting in the data release pipeline from v26.0.0. See also RFC-853.

BPS Parsl plugin

A Parsl-based plugin ctrl_bps_parsl for the LSST Batch Production Service (BPS) execution framework as added. It is intended to support running LSST PipelineTask jobs on high-performance computing (HPC) clusters.

Hierarchical Progressive Surveys (HiPS) trees

GenerateHipsTask was added to the pipelines. This Task generates Hierarchical Progressive Surveys (HiPS) trees from high-resolution, healpix-projected coadds. HiPS images produced by this Task are viewable in the Portal for Data Preview 0.2.

Significant interface changes

Refactored image difference tasks

ImageDifferenceTask, which performed both image subtraction and source detection, has been split into AlardLuptonSubtractTask and DetectAndMeasureTask. The first performs image subtraction, and the latter detects and measures sources on the image differences.

Scarlet outputs

lsst.meas.extensions.ScarletDeblendTask now outputs a single storage container that allows users to access both the scarlet models and flux re-distributed model while using an order of magnitude less storage space.

New packages added

  • analysis_tools assists in the creation of quality assurance (QA) plots and metrics from the outputs of a data reduction pipeline

  • ctrl_bps_parsl is the Parsl-based plugin for the LSST Batch Production Service (BPS) execution framework.

  • display_astrowidgets is the astrowidgets implementation for afwDisplay based on the Ginga image viewer

  • drp_tasks contains high-level Tasks specific to the data release production pipeline.

  • felis contains DBMS-agnostic schema descriptions

  • gbdes is a fork of the gbernstein/gbdes package, modified for use by the LSST Science Pipelines as a third-party dependency.

Packages removed

Pending deprecations

These Tasks or methods will be deprecated in the next major release.

Deprecate method getInstrument

obs.base.util.getInstrument will be replaced by lsst.pipe.base.Instrument.from_string and will be removed after release of release v27.0.0.

Deprecate interfaces ParquetTable and MultilevelParquetTable

The Gen 2 I/O interfaces ParquetTable and MultilevelParquetTable in pipe_tasks will be removed after the release of v26.0.0.

Deprecate factory construction with makeIdFactory

ID factory construction with lsst.ip.diffim.detectAndMeasure.makeIdFactory should be replaced by using idGenerator config field and will be removed after the release of v27.0.0.

Deprecate method get_bool_array

The get_bool_array method in lsst.afw.table.BaseColumnView and its derived subclasses should not be used as Catalog.__getitem__ now provides better support accessing flag/bool columns and will be removed after the release of v27.0.0.

Pending deprecations in lsst.daf.butler

The methods Butler.put, Butler.get and Butler.getDeferred have been updated and should now replace calls to Butler.putDirect, Butler.getDirect and Butler.getDeferred, which are being removed. Butler.datasetExistsDirect will be replaced by Butler.stored. Butler.datasetExists will be replaced by Butler.exists. The Butler.datastore property will be deprecated. The DimensionPacker class will be deprecated in v26.0.0 in favor of configurable dimension packers. Before mentioned will be removed after the release of v27.0.0.

Deprecate SkyMapDimensionPacker

In lsst.skymap.packers.SkyMapDimensionPacker the methods getIntFromFilter, getFilterNameFromInt and getMaxIntForFilters will be deprecated in v26.0.0 and be removed after the release of v27.0.0.

Deprecated task configurations

Configurations in lsst.pipe.tasks.skyCorrectionu that will be removed after the release of v26.0.0. (see [RFC-898] for further details):

  • bgModel : Use bgModel1 instead

  • doBgModel

  • doBgModel1

The field doPsfMatch in lsst.pipe.tasks.coaddBase is no longer used and will be released after the release of v27.0.0.

In lsst.cp,pipe.ptc.cpExtractPtcTask the following configurations have been moved to cpSolvePtcTask and will be removed after the release of v26.0.0:

  • minMeanSignal

  • maxMeanSignal

Deprecations

These packages/functions are deprecated and will not be available in the next major release.

Deprecated ImageDifferenceTask

ImageDifferenceTask has been deprecated and will be removed after this release. It has been superseded by AlardLuptonSubtractTask and DetectAndMeasureTask.

Deprecated Gen 3 code in pipe_drivers

The PipelineTask classes in pipe_drivers, FocalPlaneBackground and SkyCorrectionTask, are deprecated and the identical code in pipe_tasks should be used instead. Please update any Gen 3 pipeline definitions to use the new location. pipe_drivers (along with ctrl_pool and daf_persistence) will be removed following this release.

Deprecated shared_ptr arguments for C++ member functions

Smart pointers (such as shared_ptr) should only be used as arguments if a reference, const reference, or raw pointer cannot be used. In afw and daf_base some member functions were overloaded and the old implementations using shared_ptr were marked deprecated and will be removed following this release.

For more information, refer to RFC-817.

The field doApplyUberCal is deprecated. Use doApplyExternalPhotoCalib and doApplyExternalSkyWcs instead. It will be removed after this release. [DM-23352]

Deprecated configurations in PsfexStarSelectorTask

The following configurations are no longer used and will be removed after this release:

  • maxbad

  • maxbadflag

[DM-5684]

Deprecate PSF evaluation without position argument

PSF models are spatially-varying. Evaluation of these models with methods such as psf.computeImage without a position argument will return the realization of the PSF at the average position. A position argument will be required after v25. The current behavior can be replicated by explicitly providing the average position like psf.computeImage(psf.getAveragePosition).

For more information, refer to RFC-796.

Deprecated f-string log messages

f-string log messages are now deprecated to match Python logging convention.

For more information, refer to RFC-789.

Deprecated task configurations

In lsst.pipe.tasks.InsertFakesTask:

  • raColName: Use ra_col instead.

  • decColName: Use dec_col instead.

  • diskHLR, aDisk, bDisk, paDisk: Use disk_semimajor_col, disk_axis_ratio_col, and disk_pa_col to specify disk half-light ellipse.

  • nDisk: Use disk_n instead.

  • bulgeHLR, bulge_pa_col, aBulge, bBulge, paBulge: Use bulge_semimajor_col, bulge_axis_ratio_col, and to specify disk half-light ellipse.

  • nBulge: Use bulge_n instead.

  • magVar: Use mag_col instead.

  • sourceSelectionColName: Use select_col instead.