DcrAssembleCoaddTask

class lsst.drp.tasks.dcr_assemble_coadd.DcrAssembleCoaddTask(*args, **kwargs)

Bases: CompareWarpAssembleCoaddTask

Assemble DCR coadded images from a set of warps.

Attributes

bufferSizeint

The number of pixels to grow each subregion by to allow for DCR.

Notes

As with AssembleCoaddTask, we want to assemble a coadded image from a set of Warps (also called coadded temporary exposures), including the effects of Differential Chromatic Refraction (DCR). For full details of the mathematics and algorithm, please see DMTN-037: DCR-matched template generation (https://dmtn-037.lsst.io).

This Task produces a DCR-corrected Coadd, as well as a dcrCoadd for each subfilter used in the iterative calculation. It begins by dividing the bandpass-defining filter into N equal bandwidth “subfilters”, and divides the flux in each pixel from an initial coadd equally into each as a “dcrModel”. Because the airmass and parallactic angle of each individual exposure is known, we can calculate the shift relative to the center of the band in each subfilter due to DCR. For each exposure we apply this shift as a linear transformation to the dcrModels and stack the results to produce a DCR-matched exposure. The matched exposures are subtracted from the input exposures to produce a set of residual images, and these residuals are reverse shifted for each exposures’ subfilters and stacked. The shifted and stacked residuals are added to the dcrModels to produce a new estimate of the flux in each pixel within each subfilter. The dcrModels are solved for iteratively, which continues until the solution from a new iteration improves by less than a set percentage, or a maximum number of iterations is reached. Two forms of regularization are employed to reduce unphysical results. First, the new solution is averaged with the solution from the previous iteration, which mitigates oscillating solutions where the model overshoots with alternating very high and low values. Second, a common degeneracy when the data have a limited range of airmass or parallactic angle values is for one subfilter to be fit with very low or negative values, while another subfilter is fit with very high values. This typically appears in the form of holes next to sources in one subfilter, and corresponding extended wings in another. Because each subfilter has a narrow bandwidth we assume that physical sources that are above the noise level will not vary in flux by more than a factor of frequencyClampFactor between subfilters, and pixels that have flux deviations larger than that factor will have the excess flux distributed evenly among all subfilters. If splitSubfilters is set, then each subfilter will be further sub- divided during the forward modeling step (only). This approximates using a higher number of subfilters that may be necessary for high airmass observations, but does not increase the number of free parameters in the fit. This is needed when there are high airmass observations which would otherwise have significant DCR even within a subfilter. Because calculating the shifted images takes most of the time, splitting the subfilters is turned off by way of the splitThreshold option for low-airmass observations that do not suffer from DCR within a subfilter.

Methods Summary

applyModelWeights(modelImages, refImage, ...)	Smoothly replace model pixel values with those from a reference at locations away from detected sources.
calculateConvergence(dcrModels, ...)	Calculate a quality of fit metric for the matched templates.
calculateGain(convergenceList, gainList)	Calculate the gain to use for the current iteration.
calculateModelWeights(dcrModels, dcrBBox)	Build an array that smoothly tapers to 0 away from detected sources.
calculateNImage(dcrModels, bbox, ...)	Calculate the number of exposures contributing to each subfilter.
calculateSingleConvergence(dcrModels, ...)	Calculate a quality of fit metric for a single matched template.
dcrAssembleSubregion(dcrModels, ...)	Assemble the DCR coadd for a sub-region.
dcrResiduals(residual, visitInfo, wcs, ...)	Prepare a residual image for stacking in each subfilter by applying the reverse DCR shifts.
fillCoadd(dcrModels, skyInfo, warpRefList, ...)	Create a list of coadd exposures from a list of masked images.
loadSubExposures(bbox, statsCtrl, ...)	Pre-load sub-regions of a list of exposures.
measureCoaddPsf(coaddExposure)	Detect sources on the coadd exposure and measure the final PSF.
newModelFromResidual(dcrModels, ...)	Calculate a new DcrModel from a set of image residuals.
prepareDcrInputs(templateCoadd, warpRefList, ...)	Prepare the DCR coadd by iterating through the visitInfo of the input warps.
run(skyInfo, *, warpRefList, ...[, ...])	Assemble the coadd.
runQuantum(butlerQC, inputRefs, outputRefs)	Do butler IO and transform to provide in memory objects for tasks run method.
selectCoaddPsf(templateCoadd, warpRefList)	Compute the PSF of the coadd from the exposures with the best seeing.
stackCoadd(dcrCoadds)	Add a list of sub-band coadds together.

Methods Documentation

applyModelWeights(modelImages, refImage, modelWeights)

Smoothly replace model pixel values with those from a reference at locations away from detected sources.

Parameters

modelImageslist of lsst.afw.image.Image

The new DCR model images from the current iteration. The values will be modified in place.

refImagelsst.afw.image.MaskedImage

A reference image used to supply the default pixel values.

modelWeightsnumpy.ndarray or float

A 2D array of weight values that tapers smoothly to zero away from detected sources. Set to a placeholder value of 1.0 if self.config.useModelWeights is False.

calculateConvergence(dcrModels, subExposures, bbox, warpRefList, weightList, statsCtrl)

Calculate a quality of fit metric for the matched templates.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

subExposuresdict of lsst.afw.image.ExposureF

The pre-loaded exposures for the current subregion.

bboxlsst.geom.box.Box2I

Sub-region to coadd.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

weightListlist of float

The weight to give each input exposure in the coadd.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd.

Returns

convergenceMetricfloat

Quality of fit metric for all input exposures, within the sub-region.

calculateGain(convergenceList, gainList)

Calculate the gain to use for the current iteration.

After calculating a new DcrModel, each value is averaged with the value in the corresponding pixel from the previous iteration. This reduces oscillating solutions that iterative techniques are plagued by, and speeds convergence. By far the biggest changes to the model happen in the first couple iterations, so we can also use a more aggressive gain later when the model is changing slowly.

Parameters

convergenceListlist of float

The quality of fit metric from each previous iteration.

gainListlist of float

The gains used in each previous iteration: appended with the new gain value. Gains are numbers between self.config.baseGain and 1.

Returns

gainfloat

Relative weight to give the new solution when updating the model. A value of 1.0 gives equal weight to both solutions.

Raises

ValueError

If len(convergenceList) != len(gainList)+1.

calculateModelWeights(dcrModels, dcrBBox)

Build an array that smoothly tapers to 0 away from detected sources.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

dcrBBoxlsst.geom.box.Box2I

Sub-region of the coadd which includes a buffer to allow for DCR.

Returns

weightsnumpy.ndarray or float

A 2D array of weight values that tapers smoothly to zero away from detected sources. Set to a placeholder value of 1.0 if self.config.useModelWeights is False.

Raises

ValueError

If useModelWeights is set and modelWeightsWidth is negative.

calculateNImage(dcrModels, bbox, warpRefList, spanSetMaskList, statsCtrl)

Calculate the number of exposures contributing to each subfilter.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

bboxlsst.geom.box.Box2I

Bounding box of the patch to coadd.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

spanSetMaskListlist of dict containing spanSet lists, or None

Each element of the dict contains the new mask plane name (e.g. “CLIPPED and/or “NO_DATA”) as the key, and the list of SpanSets to apply to the mask.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd

Returns

dcrNImageslist of lsst.afw.image.ImageU

List of exposure count images for each subfilter.

dcrWeightslist of lsst.afw.image.ImageF

Per-pixel weights for each subfilter. Equal to 1/(number of unmasked images contributing to each pixel).

calculateSingleConvergence(dcrModels, exposure, significanceImage, statsCtrl)

Calculate a quality of fit metric for a single matched template.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

exposurelsst.afw.image.ExposureF

The input warped exposure to evaluate.

significanceImagenumpy.ndarray

Array of weights for each pixel corresponding to its significance for the convergence calculation.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd.

Returns

convergenceMetricfloat

Quality of fit metric for one exposure, within the sub-region.

dcrAssembleSubregion(dcrModels, subExposures, bbox, dcrBBox, warpRefList, statsCtrl, convergenceMetric, gain, modelWeights, refImage, dcrWeights)

Assemble the DCR coadd for a sub-region.

Build a DCR-matched template for each input exposure, then shift the residuals according to the DCR in each subfilter. Stack the shifted residuals and apply them as a correction to the solution from the previous iteration. Restrict the new model solutions from varying by more than a factor of modelClampFactor from the last solution, and additionally restrict the individual subfilter models from varying by more than a factor of frequencyClampFactor from their average. Finally, mitigate potentially oscillating solutions by averaging the new solution with the solution from the previous iteration, weighted by their convergence metric.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

subExposuresdict of lsst.afw.image.ExposureF

The pre-loaded exposures for the current subregion.

bboxlsst.geom.box.Box2I

Bounding box of the subregion to coadd.

dcrBBoxlsst.geom.box.Box2I

Sub-region of the coadd which includes a buffer to allow for DCR.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd.

convergenceMetricfloat

Quality of fit metric for the matched templates of the input images.

gainfloat, optional

Relative weight to give the new solution when updating the model.

modelWeightsnumpy.ndarray or float

A 2D array of weight values that tapers smoothly to zero away from detected sources. Set to a placeholder value of 1.0 if self.config.useModelWeights is False.

refImagelsst.afw.image.Image

A reference image used to supply the default pixel values.

dcrWeightslist of lsst.afw.image.Image

Per-pixel weights for each subfilter. Equal to 1/(number of unmasked images contributing to each pixel).

dcrResiduals(residual, visitInfo, wcs, effectiveWavelength, bandwidth)

Prepare a residual image for stacking in each subfilter by applying the reverse DCR shifts.

Parameters

residualnumpy.ndarray

The residual masked image for one exposure, after subtracting the matched template.

visitInfolsst.afw.image.VisitInfo

Metadata for the exposure.

wcslsst.afw.geom.SkyWcs

Coordinate system definition (wcs) for the exposure.

Yields

residualImagenumpy.ndarray

The residual image for the next subfilter, shifted for DCR.

fillCoadd(dcrModels, skyInfo, warpRefList, weightList, calibration=None, coaddInputs=None, mask=None, variance=None)

Create a list of coadd exposures from a list of masked images.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

skyInfolsst.pipe.base.Struct

Patch geometry information, from getSkyInfo.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

weightListlist of float

The weight to give each input exposure in the coadd.

calibrationlsst.afw.Image.PhotoCalib, optional

Scale factor to set the photometric calibration of an exposure.

coaddInputslsst.afw.Image.CoaddInputs, optional

A record of the observations that are included in the coadd.

masklsst.afw.image.Mask, optional

Optional mask to override the values in the final coadd.

variancelsst.afw.image.Image, optional

Optional variance plane to override the values in the final coadd.

Returns

dcrCoaddslist of lsst.afw.image.ExposureF

A list of coadd exposures, each exposure containing the model for one subfilter.

loadSubExposures(bbox, statsCtrl, warpRefList, imageScalerList, spanSetMaskList)

Pre-load sub-regions of a list of exposures.

Parameters

bboxlsst.geom.box.Box2I

Sub-region to coadd.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

imageScalerListlist of lsst.pipe.task.ImageScaler

The image scalars correct for the zero point of the exposures. Deprecated and will be removed after v29 in DM-49083.

spanSetMaskListlist of dict containing spanSet lists, or None

Each element is dict with keys = mask plane name to add the spans to.

Returns

subExposuresdict

The dict keys are the visit IDs, and the values are lsst.afw.image.ExposureF The pre-loaded exposures for the current subregion. The variance plane contains weights, and not the variance

measureCoaddPsf(coaddExposure)

Detect sources on the coadd exposure and measure the final PSF.

Parameters

coaddExposurelsst.afw.image.Exposure

The final coadded exposure.

newModelFromResidual(dcrModels, residualGeneratorList, dcrBBox, statsCtrl, gain, modelWeights, refImage, dcrWeights)

Calculate a new DcrModel from a set of image residuals.

Parameters

dcrModelslsst.pipe.tasks.DcrModel

Current model of the true sky after correcting chromatic effects.

residualGeneratorListgenerator of numpy.ndarray

The residual image for the next subfilter, shifted for DCR.

dcrBBoxlsst.geom.box.Box2I

Sub-region of the coadd which includes a buffer to allow for DCR.

statsCtrllsst.afw.math.StatisticsControl

Statistics control object for coadd.

gainfloat

Relative weight to give the new solution when updating the model.

modelWeightsnumpy.ndarray or float

A 2D array of weight values that tapers smoothly to zero away from detected sources. Set to a placeholder value of 1.0 if self.config.useModelWeights is False.

refImagelsst.afw.image.Image

A reference image used to supply the default pixel values.

dcrWeightslist of lsst.afw.image.Image

Per-pixel weights for each subfilter. Equal to 1/(number of unmasked images contributing to each pixel).

Returns

dcrModellsst.pipe.tasks.DcrModel

New model of the true sky after correcting chromatic effects.

prepareDcrInputs(templateCoadd, warpRefList, weightList)

Prepare the DCR coadd by iterating through the visitInfo of the input warps.

Sets the property bufferSize.

Parameters

templateCoaddlsst.afw.image.ExposureF

The initial coadd exposure before accounting for DCR.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

weightListlist of float

The weight to give each input exposure in the coadd. Will be modified in place if doAirmassWeight is set.

Returns

dcrModelslsst.pipe.tasks.DcrModel

Best fit model of the true sky after correcting chromatic effects.

Raises

NotImplementedError

If lambdaMin is missing from the Mapper class of the obs package being used.

run(skyInfo, *, warpRefList, imageScalerList, weightList, supplementaryData=None, **kwargs)

Assemble the coadd.

Requires additional inputs Struct supplementaryData to contain a templateCoadd that serves as the model of the static sky.

Find artifacts and apply them to the warps’ masks creating a list of alternative masks with a new “CLIPPED” plane and updated “NO_DATA” plane then pass these alternative masks to the base class’s assemble method.

Divide the templateCoadd evenly between each subfilter of a DcrModel as the starting best estimate of the true wavelength- dependent sky. Forward model the DcrModel using the known chromatic effects in each subfilter and calculate a convergence metric based on how well the modeled template matches the input warps. If the convergence has not yet reached the desired threshold, then shift and stack the residual images to build a new DcrModel. Apply conditioning to prevent oscillating solutions between iterations or between subfilters.

Once the DcrModel reaches convergence or the maximum number of iterations has been reached, fill the metadata for each subfilter image and make them proper coaddExposures.

Parameters

skyInfolsst.pipe.base.Struct

Patch geometry information, from getSkyInfo

warpRefListlist [lsst.daf.butler.DeferredDatasetHandle]

The data references to the input warped exposures.

imageScalerListlist [lsst.pipe.task.ImageScaler]

The image scalars correct for the zero point of the exposures. Deprecated and will be removed after v29 in DM-49083.

weightListlist [float]

The weight to give each input exposure in the coadd.

supplementaryDatalsst.pipe.base.Struct

Result struct returned by _makeSupplementaryData with attributes:

templateCoadd

Coadded exposure (lsst.afw.image.Exposure).

Returns

resultlsst.pipe.base.Struct

Results as a struct with attributes:

coaddExposure

Coadded exposure (lsst.afw.image.Exposure).

nImage

Exposure count image (lsst.afw.image.ImageU).

dcrCoadds

list of coadded exposures for each subfilter.

dcrNImages

list of exposure count images for each subfilter.

runQuantum(butlerQC, inputRefs, outputRefs)

Do butler IO and transform to provide in memory objects for tasks run method.

Parameters

butlerQCQuantumContext

A butler which is specialized to operate in the context of a lsst.daf.butler.Quantum.

inputRefsInputQuantizedConnection

Datastructure whose attribute names are the names that identify connections defined in corresponding PipelineTaskConnections class. The values of these attributes are the lsst.daf.butler.DatasetRef objects associated with the defined input/prerequisite connections.

outputRefsOutputQuantizedConnection

Datastructure whose attribute names are the names that identify connections defined in corresponding PipelineTaskConnections class. The values of these attributes are the lsst.daf.butler.DatasetRef objects associated with the defined output connections.

Notes

Assemble a coadd from a set of Warps.

selectCoaddPsf(templateCoadd, warpRefList)

Compute the PSF of the coadd from the exposures with the best seeing.

Parameters

templateCoaddlsst.afw.image.ExposureF

The initial coadd exposure before accounting for DCR.

warpRefListlist of lsst.daf.butler.DeferredDatasetHandle

The data references to the input warped exposures.

Returns

psflsst.meas.algorithms.CoaddPsf

The average PSF of the input exposures with the best seeing.

stackCoadd(dcrCoadds)

Add a list of sub-band coadds together.

Parameters

dcrCoaddslist of lsst.afw.image.ExposureF

A list of coadd exposures, each exposure containing the model for one subfilter.

Returns

coaddExposurelsst.afw.image.ExposureF

A single coadd exposure that is the sum of the sub-bands.