ZogyTask

class lsst.ip.diffim.ZogyTask(config=None, name=None, parentTask=None, log=None)

Bases: lsst.pipe.base.Task

Task to perform ZOGY proper image subtraction. See module-level documentation for additional details.

Methods Summary

calculateFourierDiffim(psf1, im1, varPlane1, …) Convolve and subtract two images in Fourier space.
calculateMaskPlane(mask1, mask2[, effPsf1, …]) Calculate the mask plane of the difference image.
checkCentroids(psfArr1, psfArr2) Check whether two PSF array centroids’ distance is within tolerance.
computeCommonShape(*shapes) Calculate the common shape for FFT operations.
computePsfAtCenter(exposure) Computes the PSF image at the bbox center point.
emptyMetadata() Empty (clear) the metadata for this Task and all sub-Tasks.
getAllSchemaCatalogs() Get schema catalogs for all tasks in the hierarchy, combining the results into a single dict.
getCentroid(A) Calculate the centroid coordinates of a 2D array.
getFullMetadata() Get metadata for all tasks.
getFullName() Get the task name as a hierarchical name including parent task names.
getName() Get the name of the task.
getSchemaCatalogs() Get the schemas generated by this task.
getTaskDict() Get a dictionary of all tasks as a shallow copy.
inverseFftAndCropImage(imgArr, origSize[, …]) Inverse FFT and crop padding from image array.
makeDiffimSubExposure(ftDiff) Wrap array results into Exposure objects.
makeField(doc) Make a lsst.pex.config.ConfigurableField for this task.
makeKernelPsfFromArray(A) Create a non spatially varying PSF from a numpy.ndarray.
makeSubtask(name, **keyArgs) Create a subtask as a new instance as the name attribute of this task.
padAndFftImage(imgArr) Prepare and forward FFT an image array.
padCenterOriginArray(A, newShape[, …]) Zero pad an image where the origin is at the center and replace the origin to the corner as required by the periodic input of FFT.
pixelSpaceSquare(D) Square the argument in pixel space.
prepareFullExposure(exposure1, exposure2[, …]) Performs calculations that apply to the full exposures once only in the psf matching.
prepareSubExposure([bbox1, bbox2, psf1, …]) Perform per-sub exposure preparations.
run(exposure1, exposure2[, calculateScore]) Task entry point to perform the zogy subtraction of exposure1-exposure2.
subtractImageMean(image, mask, statsControl) In-place subtraction of sigma-clipped mean of the image.
timer(name[, logLevel]) Context manager to log performance data for an arbitrary block of code.

Methods Documentation

calculateFourierDiffim(psf1, im1, varPlane1, F1, varMean1, psf2, im2, varPlane2, F2, varMean2, calculateScore=True)

Convolve and subtract two images in Fourier space.

Calculate the ZOGY proper difference image, score image and their PSFs. All input and output arrays are in Fourier space.

Parameters:
psf1, psf2, im1, im2, varPlane1, varPlane2 : numpy.ndarray of numpy.complex,
shape ``self.freqSpaceShape``

Psf, image and variance plane arrays respectively. All arrays must be already in Fourier space.

varMean1, varMean2: `numpy.float` > 0.

Average per-pixel noise variance in im1, im2 respectively. Used as weighing of input images. Must be greater than zero.

F1, F2 : numpy.float > 0.

Photometric scaling of the images. See eqs. (5)–(9)

calculateScore : bool, optional

If True (default), calculate and return the detection significance (score) image. Otherwise, these return fields are None.
Returns:
result : pipe.base.Struct

All arrays are in Fourier space and have shape self.freqSpaceShape. - Fd : float

Photometric level of D.

Notes

All array inputs and outputs are Fourier-space images with size of self.freqSpaceShape in this method.

varMean1, varMean2 quantities are part of the noise model and not to be confused with the variance of image frequency components or with varPlane1, varPlane2 that are the Fourier transform of the variance planes.

static calculateMaskPlane(mask1, mask2, effPsf1=None, effPsf2=None)

Calculate the mask plane of the difference image.

Parameters:
mask1, maks2 : lsst.afw.image.Mask

Mask planes of the two exposures.
Returns:
diffmask : lsst.afw.image.Mask

Mask plane for the subtraction result.

Notes

TODO DM-25174 : Specification of effPsf1, effPsf2 are not yet supported.

checkCentroids(psfArr1, psfArr2)

Check whether two PSF array centroids’ distance is within tolerance.

Parameters:
psfArr1, psfArr2 : numpy.ndarray of float

Input PSF arrays to check.
Returns:
None
Raises:
ValueError:

Centroid distance exceeds config.maxPsfCentroidDist pixels.
computeCommonShape(*shapes)

Calculate the common shape for FFT operations.

Set self.freqSpaceShape internally.

Parameters:
shapes : one or more tuple of int

Shapes of the arrays. All must have the same dimensionality. At least one shape must be provided.
Returns:
None

Notes

For each dimension, gets the smallest even number greater than or equal to N1+N2-1 where N1 and N2 are the two largest values. In case of only one shape given, rounds up to even each dimension value.

static computePsfAtCenter(exposure)

Computes the PSF image at the bbox center point.

This may be at a fractional pixel position.

Parameters:
exposure : lsst.afw.image.Exposure

Exposure with psf.
Returns:
psfImg : lsst.afw.image.Image

Calculated psf image.
emptyMetadata()

Empty (clear) the metadata for this Task and all sub-Tasks.

getAllSchemaCatalogs()

Get schema catalogs for all tasks in the hierarchy, combining the results into a single dict.

Returns:
schemacatalogs : dict

Keys are butler dataset type, values are a empty catalog (an instance of the appropriate lsst.afw.table Catalog type) for all tasks in the hierarchy, from the top-level task down through all subtasks.

Notes

This method may be called on any task in the hierarchy; it will return the same answer, regardless.

The default implementation should always suffice. If your subtask uses schemas the override Task.getSchemaCatalogs, not this method.

static getCentroid(A)

Calculate the centroid coordinates of a 2D array.

Parameters:
A : 2D numpy.ndarray of float

The input array. Must not be all exact zero.
Returns:
ycen, xcen : tuple of float

Notes

Calculates the centroid as if the array represented a 2D geometrical shape with weights per cell, allowing for “negative” weights. If sum equals to exact (float) zero, calculates centroid of absolute value array.

The geometrical center is defined as (0,0), independently of the array shape. For an odd dimension, this is the center of the center pixel, for an even dimension, this is between the two center pixels.

getFullMetadata()

Get metadata for all tasks.

Returns:
metadata : lsst.daf.base.PropertySet

The PropertySet keys are the full task name. Values are metadata for the top-level task and all subtasks, sub-subtasks, etc.

Notes

The returned metadata includes timing information (if @timer.timeMethod is used) and any metadata set by the task. The name of each item consists of the full task name with . replaced by :, followed by . and the name of the item, e.g.:

topLevelTaskName:subtaskName:subsubtaskName.itemName

using : in the full task name disambiguates the rare situation that a task has a subtask and a metadata item with the same name.

getFullName()

Get the task name as a hierarchical name including parent task names.

Returns:
fullName : str

The full name consists of the name of the parent task and each subtask separated by periods. For example:

  • The full name of top-level task “top” is simply “top”.
  • The full name of subtask “sub” of top-level task “top” is “top.sub”.
  • The full name of subtask “sub2” of subtask “sub” of top-level task “top” is “top.sub.sub2”.
getName()

Get the name of the task.

Returns:
taskName : str

Name of the task.

See also

getFullName

getSchemaCatalogs()

Get the schemas generated by this task.

Returns:
schemaCatalogs : dict

Keys are butler dataset type, values are an empty catalog (an instance of the appropriate lsst.afw.table Catalog type) for this task.

See also

Task.getAllSchemaCatalogs

Notes

Warning

Subclasses that use schemas must override this method. The default implementation returns an empty dict.

This method may be called at any time after the Task is constructed, which means that all task schemas should be computed at construction time, not when data is actually processed. This reflects the philosophy that the schema should not depend on the data.

Returning catalogs rather than just schemas allows us to save e.g. slots for SourceCatalog as well.

getTaskDict()

Get a dictionary of all tasks as a shallow copy.

Returns:
taskDict : dict

Dictionary containing full task name: task object for the top-level task and all subtasks, sub-subtasks, etc.
inverseFftAndCropImage(imgArr, origSize, filtInf=None, filtNaN=None, dtype=None)

Inverse FFT and crop padding from image array.

Parameters:
imgArr : numpy.ndarray of numpy.complex

Fourier space array representing a real image.

origSize : tuple of int

Original unpadded shape tuple of the image to be cropped to.

filtInf, filtNan : numpy.ndarray of bool or int, optional

If specified, they are used as index arrays for result to set values to numpy.inf and numpy.nan respectively at these positions.

dtype : numpy.dtype, optional

Dtype of result array to cast return values to implicitly. This is to spare one array copy operation at reducing double precision to single. If None result inherits dtype of imgArr.
Returns:
result : numpy.ndarray of dtype
makeDiffimSubExposure(ftDiff)

Wrap array results into Exposure objects.

Parameters:
ftDiff : lsst.pipe.base.Struct

Result struct by calculateFourierDiffim.
Returns:
resultName : lsst.pipe.base.Struct
  • diffSubExp : lsst.afw.image.Exposure

    The difference (sub)exposure. The exposure is calibrated in its pixel values, and has a constant PhotoCalib object of 1.

  • scoreSubExp : lsst.afw.image.Exposure or None

    The score (sub)exposure if it was calculated.
classmethod makeField(doc)

Make a lsst.pex.config.ConfigurableField for this task.

Parameters:
doc : str

Help text for the field.
Returns:
configurableField : lsst.pex.config.ConfigurableField

A ConfigurableField for this task.

Examples

Provides a convenient way to specify this task is a subtask of another task.

Here is an example of use:

class OtherTaskConfig(lsst.pex.config.Config):
    aSubtask = ATaskClass.makeField("brief description of task")
static makeKernelPsfFromArray(A)

Create a non spatially varying PSF from a numpy.ndarray.

Parameters:
A : numpy.ndarray

2D array to use as the new psf image. The pixels are copied.
Returns:
psfNew : lsst.meas.algorithms.KernelPsf

The constructed PSF.
makeSubtask(name, **keyArgs)

Create a subtask as a new instance as the name attribute of this task.

Parameters:
name : str

Brief name of the subtask.

keyArgs

Extra keyword arguments used to construct the task. The following arguments are automatically provided and cannot be overridden:

  • “config”.
  • “parentTask”.

Notes

The subtask must be defined by Task.config.name, an instance of ConfigurableField or RegistryField.

padAndFftImage(imgArr)

Prepare and forward FFT an image array.

Parameters:
imgArr : numpy.ndarray of float

Original array. In-place modified as numpy.nan and numpy.inf are replaced by array mean.
Returns:
result : lsst.pipe.base.Struct

Notes

Save location of non-finite values for restoration, and replace them with image mean values. Re-center and zero pad array by padCenterOriginArray.

static padCenterOriginArray(A, newShape, useInverse=False, dtype=None)

Zero pad an image where the origin is at the center and replace the origin to the corner as required by the periodic input of FFT.

Implement also the inverse operation, crop the padding and re-center data.

Parameters:
A : numpy.ndarray

An array to copy from.

newShape : tuple of int

The dimensions of the resulting array. For padding, the resulting array must be larger than A in each dimension. For the inverse operation this must be the original, before padding size of the array.

useInverse : bool, optional

Selector of forward, add padding, operation (False) or its inverse, crop padding, operation (True).

dtype: `numpy.dtype`, optional

Dtype of output array. Values must be implicitly castable to this type. Use to get expected result type, e.g. single float (nympy.float32). If not specified, dtype is inherited from A.
Returns:
R : numpy.ndarray

The padded or unpadded array with shape of newShape and dtype of dtype.
Raises:
ValueError : newShape dimensions must be greater than or equal to the

dimensions of A for the forward operation and less than or equal to for the inverse operation.

Notes

For odd dimensions, the splitting is rounded to put the center pixel into the new corner origin (0,0). This is to be consistent e.g. for a dirac delta kernel that is originally located at the center pixel.

static pixelSpaceSquare(D)

Square the argument in pixel space.

Parameters:
D : 2D numpy.ndarray of numpy.complex

Fourier transform of a real valued array.
Returns:
R : numpy.ndarray of numpy.complex

Notes

D is to be inverse Fourier transformed, squared and then forward Fourier transformed again, i.e. an autoconvolution in Fourier space. This operation is not distributive over multiplication. pixelSpaceSquare(A*B) != pixelSpaceSquare(A)*pixelSpaceSquare(B)

prepareFullExposure(exposure1, exposure2, correctBackground=False)

Performs calculations that apply to the full exposures once only in the psf matching.

Parameters:
correctBackground : bool, optional

If True, subtracts sigma-clipped mean of exposures. The algorithm assumes zero expectation value at background pixels.
Returns:
None
Raises:
ValueError : If photometric calibrations are not available while

config.scaleByCalibration equals True.

Notes

Set a number of instance fields with pre-calculated values. psfShape, imgShape fields follow the numpy ndarray shape convention i.e. height, width.

prepareSubExposure(bbox1=None, bbox2=None, psf1=None, psf2=None, sig1=None, sig2=None)

Perform per-sub exposure preparations.

Parameters:
sig1, sig2 : float, optional

For debug purposes only, copnsider that the image may already be rescaled by the photometric calibration.

bbox1, bbox2 : lsst.geom.Box2I, optional

If specified, the region of the full exposure to use.

psf1, psf2 : lsst.afw.detection.Psf, optional

If specified, use given psf as the sub exposure psf. For debug purposes.

sig1, sig2 : float, optional

If specified, use value as the sub-exposures’ background noise sigma value.
Returns:
None
Raises:
ValueError: If sub-exposure dimensions do not match.

Notes

TODO DM-23855: Performing ZOGY on a grid is not yet implemented. Set (replace) a number of instance fields with pre-calculated values about the current sub exposure including the FFT of the psfs.

run(exposure1, exposure2, calculateScore=True)

Task entry point to perform the zogy subtraction of exposure1-exposure2.

Parameters:
exposure1, exposure2 : lsst.afw.image.Exposure

Two exposures warped and matched into matching pixel dimensions.

calculateScore : bool, optional

If True (default), calculate the score image and return in scoreExp.
Returns:
resultName : lsst.pipe.base.Struct
  • diffExp : lsst.afw.image.Exposure

    The Zogy difference exposure (exposure1-exposure2).

  • scoreExp : lsst.afw.image.Exposure or None

    The Zogy significance or score (S) exposure if calculateScore==True.

  • ftDiff : lsst.pipe.base.Struct

    Lower level return struct by calculateFourierDiffim with added fields from the task instance. For debug purposes.

Notes

The score image (S) is defined in the ZOGY paper as the detection statistic value at each pixel. In the ZOGY image model, the input images have uniform variance noises and thus S has uniform per pixel variance (though it is not scaled to 1). In Section 3.3 of the paper, there are “corrections” defined to the score image to correct the significance values for some deviations from the image model. The first of these corrections is the calculation of the _variance plane_ of S allowing for different per pixel variance values by following the overall convolution operation on the pixels of the input images. S scaled (divided) by its corrected per pixel noise is referred as Scorr in the paper.

In the current implementation, scoreExp contains S in its image plane and the calculated (non-uniform) variance plane of S in its variance plane. scoreExp can be used directly for source detection as a likelihood image by respecting its variance plane or can be divided by the square root of the variance plane to scale detection significance values into units of sigma.

TODO DM-23855 : Implement further correction tags to the variance of scoreExp. As of DM-25174 it is not determined how important these further correction tags are.

TODO DM-23855 : spatially varying solution on a grid is not yet implemented

static subtractImageMean(image, mask, statsControl)

In-place subtraction of sigma-clipped mean of the image.

Parameters:
image : lsst.afw.image.Image

Image to manipulate. Its sigma clipped mean is in-place subtracted.

mask : lsst.afw.image.Mask

Mask to use for ignoring pixels.

statsControl : lsst.afw.math.StatisticsControl

Config of sigma clipped mean statistics calculation.
Returns:
None
Raises:
ValueError : If image mean is nan.
timer(name, logLevel=10000)

Context manager to log performance data for an arbitrary block of code.

Parameters:
name : str

Name of code being timed; data will be logged using item name: Start and End.

logLevel

A lsst.log level constant.

See also

timer.logInfo

Examples

Creating a timer context:

with self.timer("someCodeToTime"):
    pass  # code to time