PhotonTransferCurveExtractTask

class lsst.cp.pipe.PhotonTransferCurveExtractTask(*, config: PipelineTaskConfig | None = None, log: logging.Logger | LsstLogAdapter | None = None, initInputs: dict[str, Any] | None = None, **kwargs: Any)

Bases: PipelineTask

Task to measure covariances from flat fields.

This task receives as input a list of flat-field images (flats), and sorts these flats in pairs taken at the same time (the task will raise if there is one one flat at a given exposure time, and it will discard extra flats if there are more than two per exposure time). This task measures the mean, variance, and covariances from a region (e.g., an amplifier) of the difference image of the two flats with the same exposure time (alternatively, all input images could have the same exposure time but their flux changed).

The variance is calculated via afwMath, and the covariance via the methods in Astier+19 (appendix A). In theory, var = covariance[0,0]. This should be validated, and in the future, we may decide to just keep one (covariance). At this moment, if the two values differ by more than the value of thresholdDiffAfwVarVsCov00 (default: 1%), a warning will be issued.

The measured covariances at a given exposure time (along with other quantities such as the mean) are stored in a PTC dataset object (PhotonTransferCurveDataset), which gets partially filled at this stage (the remainder of the attributes of the dataset will be filled after running the second task of the PTC-measurement pipeline, PhotonTransferCurveSolveTask).

The number of partially-filled PhotonTransferCurveDataset objects will be less than the number of input exposures because the task combines input flats in pairs. However, it is required at this moment that the number of input dimensions matches bijectively the number of output dimensions. Therefore, a number of “dummy” PTC datasets are inserted in the output list. This output list will then be used as input of the next task in the PTC-measurement pipeline, PhotonTransferCurveSolveTask, which will assemble the multiple PhotonTransferCurveDataset objects into a single one in order to fit the measured covariances as a function of flux to one of three models (see PhotonTransferCurveSolveTask for details).

Reference: Astier+19: “The Shape of the Photon Transfer Curve of CCD sensors”, arXiv:1905.08677.

Attributes Summary

canMultiprocess

Methods Summary

computeGaussianHistogramParameters(im1Area, ...)	Compute KS test for a Gaussian model fit to a histogram of the difference image.
emptyMetadata()	Empty (clear) the metadata for this Task and all sub-Tasks.
getFullMetadata()	Get metadata for all tasks.
getFullName()	Get the task name as a hierarchical name including parent task names.
getGainFromFlatPair(im1Area, im2Area, ...[, ...])	Estimate the gain from a single pair of flats.
getImageAreasMasksStats(exposure1, exposure2)	Get image areas in a region as well as masks and statistic objects.
getName()	Get the name of the task.
getReadNoise(exposureMetadata, taskMetadata, ...)	Gets readout noise for an amp from ISR metadata.
getTaskDict()	Get a dictionary of all tasks as a shallow copy.
makeCovArray(inputTuple, maxRangeFromTuple)	Make covariances array from tuple.
makeField(doc)	Make a lsst.pex.config.ConfigurableField for this task.
makeSubtask(name, **keyArgs)	Create a subtask as a new instance as the name attribute of this task.
measureMeanVarCov(im1Area, im2Area, ...)	Calculate the mean of each of two exposures and the variance and covariance of their difference.
run(inputExp, inputDims, taskMetadata[, ...])	Measure covariances from difference of flat pairs
runQuantum(butlerQC, inputRefs, outputRefs)	Ensure that the input and output dimensions are passed along.
timer(name[, logLevel])	Context manager to log performance data for an arbitrary block of code.

Attributes Documentation

canMultiprocess: ClassVar[bool] = True

Methods Documentation

computeGaussianHistogramParameters(im1Area, im2Area, imStatsCtrl, mu1, mu2)

Compute KS test for a Gaussian model fit to a histogram of the difference image.

Parameters:

im1Arealsst.afw.image.MaskedImageF

Masked image from exposure 1.

im2Arealsst.afw.image.MaskedImageF

Masked image from exposure 2.

imStatsCtrllsst.afw.math.StatisticsControl

Statistics control object.

mu1float

Clipped mean of im1Area (ADU).

mu2float

Clipped mean of im2Area (ADU).

Returns:

varFitfloat

Variance from the Gaussian fit.

chi2Doffloat

Chi-squared per degree of freedom of Gaussian fit.

kspValuefloat

The KS test p-value for the Gaussian fit.

Notes

The algorithm here was originally developed by Aaron Roodman. Tests on the full focal plane of LSSTCam during testing has shown that a KS test p-value cut of 0.01 is a good discriminant for well-behaved flat pairs (p>0.01) and poorly behaved non-Gaussian flat pairs (p<0.01).

emptyMetadata() → None

Empty (clear) the metadata for this Task and all sub-Tasks.

getFullMetadata() → TaskMetadata

Get metadata for all tasks.

Returns:

metadataTaskMetadata

The keys are the full task name. Values are metadata for the top-level task and all subtasks, sub-subtasks, etc.

Notes

The returned metadata includes timing information (if @timer.timeMethod is used) and any metadata set by the task. The name of each item consists of the full task name with . replaced by :, followed by . and the name of the item, e.g.:

topLevelTaskName:subtaskName:subsubtaskName.itemName

using : in the full task name disambiguates the rare situation that a task has a subtask and a metadata item with the same name.

getFullName() → str

Get the task name as a hierarchical name including parent task names.

Returns:

fullNamestr

The full name consists of the name of the parent task and each subtask separated by periods. For example:

• The full name of top-level task “top” is simply “top”.

• The full name of subtask “sub” of top-level task “top” is “top.sub”.

• The full name of subtask “sub2” of subtask “sub” of top-level task “top” is “top.sub.sub2”.

getGainFromFlatPair(im1Area, im2Area, imStatsCtrl, mu1, mu2, correctionType='NONE', readNoise=None)

Estimate the gain from a single pair of flats.

The basic premise is 1/g = <(I1 - I2)^2/(I1 + I2)> = 1/const, where I1 and I2 correspond to flats 1 and 2, respectively. Corrections for the variable QE and the read-noise are then made following the derivation in Robert Lupton’s forthcoming book, which gets

1/g = <(I1 - I2)^2/(I1 + I2)> - 1/mu(sigma^2 - 1/2g^2).

This is a quadratic equation, whose solutions are given by:

g = mu +/- sqrt(2*sigma^2 - 2*const*mu + mu^2)/(2*const*mu*2

• 2*sigma^2)

where ‘mu’ is the average signal level and ‘sigma’ is the amplifier’s readnoise. The positive solution will be used. The way the correction is applied depends on the value supplied for correctionType.

correctionType is one of [‘NONE’, ‘SIMPLE’ or ‘FULL’]

‘NONE’ : uses the 1/g = <(I1 - I2)^2/(I1 + I2)> formula. ‘SIMPLE’ : uses the gain from the ‘NONE’ method for the

1/2g^2 term.

‘FULL’solves the full equation for g, discarding the

non-physical solution to the resulting quadratic.

Parameters:

im1Arealsst.afw.image.maskedImage.MaskedImageF

Masked image from exposure 1.

im2Arealsst.afw.image.maskedImage.MaskedImageF

Masked image from exposure 2.

imStatsCtrllsst.afw.math.StatisticsControl

Statistics control object.

mu1: `float`

Clipped mean of im1Area (ADU).

mu2: `float`

Clipped mean of im2Area (ADU).

correctionTypestr, optional

The correction applied, one of [‘NONE’, ‘SIMPLE’, ‘FULL’]

readNoisefloat, optional

Amplifier readout noise (ADU).

Returns:

gainfloat

Gain, in e/ADU.

Raises:

RuntimeError

Raise if correctionType is not one of ‘NONE’, ‘SIMPLE’, or ‘FULL’.

getImageAreasMasksStats(exposure1, exposure2, region=None)

Get image areas in a region as well as masks and statistic objects.

Parameters:

exposure1lsst.afw.image.ExposureF

First exposure of flat field pair.

exposure2lsst.afw.image.ExposureF

Second exposure of flat field pair.

regionlsst.geom.Box2I, optional

Region of each exposure where to perform the calculations (e.g, an amplifier).

Returns:

im1Arealsst.afw.image.MaskedImageF

Masked image from exposure 1.

im2Arealsst.afw.image.MaskedImageF

Masked image from exposure 2.

imStatsCtrllsst.afw.math.StatisticsControl

Statistics control object.

mu1float

Clipped mean of im1Area (ADU).

mu2float

Clipped mean of im2Area (ADU).

getName() → str

Get the name of the task.

Returns:

taskNamestr

Name of the task.

See also

getFullName

Get the full name of the task.

getReadNoise(exposureMetadata, taskMetadata, ampName)

Gets readout noise for an amp from ISR metadata.

If possible, this attempts to get the now-standard headers added to the exposure itself. If not found there, the ISR TaskMetadata is searched. If neither of these has the value, warn and set the read noise to NaN.

Parameters:

exposureMetadatalsst.daf.base.PropertySet

Metadata to check for read noise first.

taskMetadatalsst.pipe.base.TaskMetadata

List of exposures metadata from ISR for this exposure.

ampNamestr

Amplifier name.

Returns:

readNoisefloat

The read noise for this set of exposure/amplifier.

getTaskDict() → dict[str, weakref.ReferenceType[lsst.pipe.base.task.Task]]

Get a dictionary of all tasks as a shallow copy.

Returns:

taskDictdict

Dictionary containing full task name: task object for the top-level task and all subtasks, sub-subtasks, etc.

makeCovArray(inputTuple, maxRangeFromTuple)

Make covariances array from tuple.

Parameters:

inputTuplenumpy.ndarray

Structured array with rows with at least (mu, afwVar, cov, var, i, j, npix), where: mu : float

0.5*(m1 + m2), where mu1 is the mean value of flat1 and mu2 is the mean value of flat2.

afwVarfloat

Variance of difference flat, calculated with afw.

covfloat

Covariance value at lag(i, j)

varfloat

Variance(covariance value at lag(0, 0))

iint

Lag in dimension “x”.

jint

Lag in dimension “y”.

npixint

Number of pixels used for covariance calculation.

maxRangeFromTupleint

Maximum range to select from tuple.

Returns:

covnumpy.array

Covariance arrays, indexed by mean signal mu.

vCovnumpy.array

Variance of the [co]variance arrays, indexed by mean signal mu.

muValsnumpy.array

List of mean signal values.

classmethod makeField(doc: str) → ConfigurableField

Make a lsst.pex.config.ConfigurableField for this task.

Parameters:

docstr

Help text for the field.

Returns:

configurableFieldlsst.pex.config.ConfigurableField

A ConfigurableField for this task.

Examples

Provides a convenient way to specify this task is a subtask of another task.

Here is an example of use:

class OtherTaskConfig(lsst.pex.config.Config):
    aSubtask = ATaskClass.makeField("brief description of task")

makeSubtask(name: str, **keyArgs: Any) → None

Create a subtask as a new instance as the name attribute of this task.

Parameters:

namestr

Brief name of the subtask.

**keyArgs

Extra keyword arguments used to construct the task. The following arguments are automatically provided and cannot be overridden:

• config.

• parentTask.

Notes

The subtask must be defined by Task.config.name, an instance of ConfigurableField or RegistryField.

measureMeanVarCov(im1Area, im2Area, imStatsCtrl, mu1, mu2)

Calculate the mean of each of two exposures and the variance and covariance of their difference. The variance is calculated via afwMath, and the covariance via the methods in Astier+19 (appendix A). In theory, var = covariance[0,0]. This should be validated, and in the future, we may decide to just keep one (covariance).

Parameters:

im1Arealsst.afw.image.maskedImage.MaskedImageF

Masked image from exposure 1.

im2Arealsst.afw.image.maskedImage.MaskedImageF

Masked image from exposure 2.

imStatsCtrllsst.afw.math.StatisticsControl

Statistics control object.

mu1: `float`

Clipped mean of im1Area (ADU).

mu2: `float`

Clipped mean of im2Area (ADU).

Returns:

mufloat

0.5*(mu1 + mu2), where mu1, and mu2 are the clipped means of the regions in both exposures. If either mu1 or m2 are NaN’s, the returned value is NaN.

varDifffloat

Half of the clipped variance of the difference of the regions inthe two input exposures. If either mu1 or m2 are NaN’s, the returned value is NaN.

covDiffAstierlist or NaN

List with tuples of the form (dx, dy, var, cov, npix), where:

dxint

Lag in x

dyint

Lag in y

varfloat

Variance at (dx, dy).

covfloat

Covariance at (dx, dy).

nPixint

Number of pixel pairs used to evaluate var and cov.

rowMeanVariancefloat

Variance of the means of each row in the difference image. Taken from github.com/lsst-camera-dh/eo_pipe.

If either mu1 or m2 are NaN’s, the returned value is NaN.

run(inputExp, inputDims, taskMetadata, inputPhotodiodeData=None)

Measure covariances from difference of flat pairs

Parameters:

inputExpdict [float, list

[DeferredDatasetRef]]

Dictionary that groups references to flat-field exposures that have the same exposure time (seconds), or that groups them sequentially by their exposure id.

inputDimslist

List of exposure IDs.

taskMetadatalist [lsst.pipe.base.TaskMetadata]

List of exposures metadata from ISR.

inputPhotodiodeDatadict [str, lsst.ip.isr.PhotodiodeCalib]

Photodiode readings data (optional).

Returns:

resultslsst.pipe.base.Struct

The resulting Struct contains:

outputCovariances

A list containing the per-pair PTC measurements (list [lsst.ip.isr.PhotonTransferCurveDataset])

runQuantum(butlerQC, inputRefs, outputRefs)

Ensure that the input and output dimensions are passed along.

Parameters:

butlerQCQuantumContext

Butler to operate on.

inputRefsInputQuantizedConnection

Input data refs to load.

ouptutRefsOutputQuantizedConnection

Output data refs to persist.

timer(name: str, logLevel: int = 10) → Iterator[None]

Context manager to log performance data for an arbitrary block of code.

Parameters:

namestr

Name of code being timed; data will be logged using item name: Start and End.

logLevelint

A logging level constant.

See also

lsst.utils.timer.logInfo

Implementation function.

Examples

Creating a timer context:

with self.timer("someCodeToTime"):
    pass  # code to time