PhotonTransferCurveSolveTask

class lsst.cp.pipe.PhotonTransferCurveSolveTask(*, config: PipelineTaskConfig | None = None, log: logging.Logger | LsstLogAdapter | None = None, initInputs: dict[str, Any] | None = None, **kwargs: Any)

Bases: PipelineTask

Task to fit the PTC from flat covariances.

The first task of the PTC measurement pipeline, PhotonTransferCurveMeasureTask (and assumed to have been run before this task), produced a list of PhotonTransferCurveDataset objects. Each dataset contains the mean signal and covariances of the difference image of the flat-field images taken at the same exposure time. The list also contains dummy datasets (with no measurements), whose purpose is to have the input and output dimensions of PhotonTransferCurveMeasureTask match.

This task, PhotonTransferCurveSolveTask, assembles the list of individual PTC datasets produced by PhotonTransferCurveMeasureTask into one single final PTC dataset, discarding the dummy datset as appropiate. The task fits the measured (co)variances to one of three models: a polynomial model of a given order, or the models described in equations 16 and 20 of Astier+19. These options are referred to as POLYNOMIAL, EXPAPPROXIMATION, and FULLCOVARIANCE in the configuration options of the task, respectively). Parameters of interest such as the gain and noise are derived from the fits. The FULLCOVARIANCE model is fitted to the full covariance data (as oppossed to the other two models, which are fit to the variance vs mean measurements only).

Astier+19: “The Shape of the Photon Transfer Curve of CCD sensors”, arXiv:1905.08677

Attributes Summary

canMultiprocess

Methods Summary

emptyMetadata()	Empty (clear) the metadata for this Task and all sub-Tasks.
evalCovModel(mu, aMatrix, cMatrix, ...[, ...])	Computes full covariances model (Eq.
fillBadAmp(dataset, ptcFitType, ampName)	Fill the dataset with NaNs if there are not enough good points.
fitDataFullCovariance(dataset)	Fit measured flat covariances to the full model in Astier+19 (Eq.
fitMeasurementsToModel(dataset)	Fit the measured covariances vs mean signal to a polynomial or one of the models in Astier+19 (Eq.
fitPtc(dataset)	Fit the photon transfer curve to a polynomial or to the Astier+19 approximation (Eq.
funcFullCovarianceModel(params, x)	Model to fit covariances from flat fields; Equation 20 of Astier+19.
funcFullCovarianceModelNoB(params, x)	Model to fit covariances from flat fields; Equation 20 of Astier+19, with b=0 (equivalent to c=a*b=0 in this code).
getFullMetadata()	Get metadata for all tasks.
getFullName()	Get the task name as a hierarchical name including parent task names.
getName()	Get the name of the task.
getTaskDict()	Get a dictionary of all tasks as a shallow copy.
initialFitFullCovariance(mu, cov, sqrtW)	Performs a crude parabolic fit of the data in order to start the full fit close to the solution, setting b=0 (c=0) in Eq.
makeField(doc)	Make a lsst.pex.config.ConfigurableField for this task.
makeSubtask(name, **keyArgs)	Create a subtask as a new instance as the name attribute of this task.
run(inputCovariances[, camera, detId])	Fit measured covariances to different models.
runQuantum(butlerQC, inputRefs, outputRefs)	Ensure that the input and output dimensions are passed along.
subtractDistantOffset(muAtAmpMasked, ...[, ...])	Subtract distant offset from the covariance matrices.
timer(name[, logLevel])	Context manager to log performance data for an arbitrary block of code.

Attributes Documentation

canMultiprocess: ClassVar[bool] = True

Methods Documentation

emptyMetadata() → None

Empty (clear) the metadata for this Task and all sub-Tasks.

evalCovModel(mu, aMatrix, cMatrix, noiseMatrix, gain, setBtoZero=False)

Computes full covariances model (Eq. 20 of Astier+19).

Parameters:

munumpy.array, (N,)

List of mean signals (units: adu)

aMatrixnumpy.array, (M, M)

“a” parameter per flux in Eq. 20 of Astier+19 (units: 1/electron)

cMatrixnumpy.array, (M, M)

“c”=”ab” parameter per flux in Eq. 20 of Astier+19 (units: 1/electron^2)

noiseMatrixnumpy.array, (M, M)

“noise” parameter per flux in Eq. 20 of Astier+19 (units: electron^2)

gainfloat

Amplifier gain (e/adu)

setBtoZero=Falsebool, optional

Set “b” parameter in full model (see Astier+19) to zero.

Returns:

covModelnumpy.array, (N, M, M)

Covariances model.

Notes

By default, computes the covModel for the mu’s stored(self.mu). Returns cov[Nmu, M, M]. The variance for the PTC is cov[:, 0, 0]. mu and cov are in adus and adus squared. To use electrons for both, the gain should be set to 1. This routine implements the model in Astier+19 (1905.08677). The parameters of the full model for C_ij(mu) (“C_ij” and “mu” in adu^2 and adu, respectively) in Astier+19 (Eq. 20) are:

• “a” coefficients (M by M matrix), units: 1/electron

• “b” coefficients (M by M matrix), units: 1/electron

• noise matrix (M by M matrix), units: electron^2

• gain, units: electron/adu

“b” appears in Eq. 20 only through the “ab” combination, which is defined in this code as “c=ab”.

fillBadAmp(dataset, ptcFitType, ampName)

Fill the dataset with NaNs if there are not enough good points.

Parameters:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

The dataset containing the means, variances and exposure times.

ptcFitType{‘POLYNOMIAL’, ‘EXPAPPROXIMATION’}

Fit a ‘POLYNOMIAL’ (degree: ‘polynomialFitDegree’) or ‘EXPAPPROXIMATION’ (Eq. 16 of Astier+19) to the PTC.

ampNamestr

Amplifier name.

fitDataFullCovariance(dataset)

Fit measured flat covariances to the full model in Astier+19 (Eq. 20).

Parameters:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

The dataset containing information such as the means, (co)variances, and exposure times.

Returns:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

This is the same dataset as the input parameter, however, it has been modified to include information such as the fit vectors and the fit parameters. See the class PhotonTransferCurveDatase.

Notes

The parameters of the full model for C_ij(mu) (“C_ij” and “mu” in adu^2 and adu, respectively) in Astier+19 (Eq. 20) are:

• “a” coefficients (r by r matrix), units: 1/electron

• “b” coefficients (r by r matrix), units: 1/electron

• noise matrix (r by r matrix), units: electron^2

• gain, units: electron/adu

“b” appears in Eq. 20 only through the “ab” combination, which is defined in this code as “c=ab”.

Total number of parameters: #entries(a) + #entries(c) + #entries(noise) + 1. This is equivalent to r^2 + r^2 + r^2 + 1, where “r” is the maximum lag considered for the covariances calculation, and the extra “1” is the gain. If “b” is 0, then “c” is 0, and len(pInit) will have r^2 fewer entries.

fitMeasurementsToModel(dataset)

Fit the measured covariances vs mean signal to a polynomial or one of the models in Astier+19 (Eq. 16 or Eq.20).

Parameters:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

The dataset containing information such as the means, (co)variances, and exposure times.

Returns:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

This is the same dataset as the input parameter, however, it has been modified to include information such as the fit vectors and the fit parameters. See the class PhotonTransferCurveDatase.

fitPtc(dataset)

Fit the photon transfer curve to a polynomial or to the Astier+19 approximation (Eq. 16).

Fit the photon transfer curve with either a polynomial of the order specified in the task config (POLNOMIAL), or using the exponential approximation in Astier+19 (Eq. 16, FULLCOVARIANCE).

Sigma clipping is performed iteratively for the fit, as well as an initial clipping of data points that are more than config.initialNonLinearityExclusionThreshold away from lying on a straight line. This other step is necessary because the photon transfer curve turns over catastrophically at very high flux (because saturation drops the variance to ~0) and these far outliers cause the initial fit to fail, meaning the sigma cannot be calculated to perform the sigma-clipping.

Parameters:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

The dataset containing the means, variances and exposure times.

Returns:

datasetlsst.ip.isr.ptcDataset.PhotonTransferCurveDataset

This is the same dataset as the input parameter, however, it has been modified to include information such as the fit vectors and the fit parameters. See the class PhotonTransferCurveDatase.

Raises:

RuntimeError

Raised if dataset.ptcFitType is None or empty.

funcFullCovarianceModel(params, x)

Model to fit covariances from flat fields; Equation 20 of Astier+19.

Parameters:

paramslist

Parameters of the model: aMatrix, CMatrix, noiseMatrix, gain (e/adu).

xnumpy.array, (N,)

Signal mu (adu)

Returns:

ynumpy.array, (N,)

Covariance matrix.

funcFullCovarianceModelNoB(params, x)

Model to fit covariances from flat fields; Equation 20 of Astier+19, with b=0 (equivalent to c=a*b=0 in this code).

Parameters:

paramslist

Parameters of the model: aMatrix, CMatrix, noiseMatrix, gain (e/adu).

xnumpy.array, (N,)

Signal mu (adu)

Returns:

ynumpy.array, (N,)

Covariance matrix.

getFullMetadata() → TaskMetadata

Get metadata for all tasks.

Returns:

metadataTaskMetadata

The keys are the full task name. Values are metadata for the top-level task and all subtasks, sub-subtasks, etc.

Notes

The returned metadata includes timing information (if @timer.timeMethod is used) and any metadata set by the task. The name of each item consists of the full task name with . replaced by :, followed by . and the name of the item, e.g.:

topLevelTaskName:subtaskName:subsubtaskName.itemName

using : in the full task name disambiguates the rare situation that a task has a subtask and a metadata item with the same name.

getFullName() → str

Get the task name as a hierarchical name including parent task names.

Returns:

fullNamestr

The full name consists of the name of the parent task and each subtask separated by periods. For example:

• The full name of top-level task “top” is simply “top”.

• The full name of subtask “sub” of top-level task “top” is “top.sub”.

• The full name of subtask “sub2” of subtask “sub” of top-level task “top” is “top.sub.sub2”.

getName() → str

Get the name of the task.

Returns:

taskNamestr

Name of the task.

See also

getFullName

Get the full name of the task.

getTaskDict() → dict[str, weakref.ReferenceType[lsst.pipe.base.task.Task]]

Get a dictionary of all tasks as a shallow copy.

Returns:

taskDictdict

Dictionary containing full task name: task object for the top-level task and all subtasks, sub-subtasks, etc.

initialFitFullCovariance(mu, cov, sqrtW)

Performs a crude parabolic fit of the data in order to start the full fit close to the solution, setting b=0 (c=0) in Eq. 20 of Astier+19.

Parameters:

munumpy.array, (N,)

Signal mu (adu)

covnumpy.array, (N, M, M)

Covariance arrays of size (M, M) (with M = config.maximumRangeCovariancesAstier), indexed by mean signal mu.

sqrtWnumpy.array, (N,)

Covariance weights, defined as 1./sqrt(Variances)

Returns:

anumpy.array, (M, M)

“a” parameter per flux in Eq. 20 of Astier+19 (units: 1/electron).

cnumpy.array, (M, M)

“c”=”ab” parameter per flux in Eq. 20 of Astier+19. (units: 1/electron^2).

noiseMatrixnumpy.array, (M, M)

“noise” parameter per flux in Eq. 20 of Astier+19. (units: electron^2)

gainfloat

Amplifier gain (electron/adu)

classmethod makeField(doc: str) → ConfigurableField

Make a lsst.pex.config.ConfigurableField for this task.

Parameters:

docstr

Help text for the field.

Returns:

configurableFieldlsst.pex.config.ConfigurableField

A ConfigurableField for this task.

Examples

Provides a convenient way to specify this task is a subtask of another task.

Here is an example of use:

class OtherTaskConfig(lsst.pex.config.Config):
    aSubtask = ATaskClass.makeField("brief description of task")

makeSubtask(name: str, **keyArgs: Any) → None

Create a subtask as a new instance as the name attribute of this task.

Parameters:

namestr

Brief name of the subtask.

**keyArgs

Extra keyword arguments used to construct the task. The following arguments are automatically provided and cannot be overridden:

• config.

• parentTask.

Notes

The subtask must be defined by Task.config.name, an instance of ConfigurableField or RegistryField.

run(inputCovariances, camera=None, detId=0)

Fit measured covariances to different models.

Parameters:

inputCovarianceslist [lsst.ip.isr.PhotonTransferCurveDataset]

List of lsst.ip.isr.PhotonTransferCurveDataset datasets.

cameralsst.afw.cameraGeom.Camera, optional

Input camera.

detIdint

Detector ID to locate the detector in the camera and populate the lsst.ip.isr.PhotonTransferCurveDataset metadata.

Returns

——-

resultslsst.pipe.base.Struct

The resultins structure contains:

outputPtcDatset

Final PTC dataset, containing information such as the means, variances, and exposure times (lsst.ip.isr.PhotonTransferCurveDataset).

runQuantum(butlerQC, inputRefs, outputRefs)

Ensure that the input and output dimensions are passed along.

Parameters:

butlerQCQuantumContext

Butler to operate on.

inputRefsInputQuantizedConnection

Input data refs to load.

ouptutRefsOutputQuantizedConnection

Output data refs to persist.

subtractDistantOffset(muAtAmpMasked, covAtAmpMasked, covSqrtWeightsAtAmpMasked, start, degree=1)

Subtract distant offset from the covariance matrices.

Parameters:

muAtAmpMaskednumpy.array

Masked mean flux array for a particular amplifier.

covAtAmpMaskednumpy.array

Masked measured covariances for a particular amplifier.

covSqrtWeightsAtAmpMaskednumpy.array

Masked inverse covariance weights for a particular amplifier.

startint, optional

The starting index to eliminate the core for the fit.

degreeint, optional

Degree of the polynomial fit.

Returns:

covAtAmpMaskednumpy.array

Subtracted measured covariances for a particular amplifier.

covSqrtWeightsAtAmpMaskednumpy.array

Masked inverse covariance weights for a particular amplifier.

Notes

Ported from https://gitlab.in2p3.fr/astier/bfptc by P. Astier.

This function subtracts a distant offset from the covariance matrices using polynomial fitting. The core of the matrices is eliminated for the fit.

The function modifies the internal state of the object, updating the covariance matrices and related attributes.

timer(name: str, logLevel: int = 10) → Iterator[None]

Context manager to log performance data for an arbitrary block of code.

Parameters:

namestr

Name of code being timed; data will be logged using item name: Start and End.

logLevelint

A logging level constant.

See also

lsst.utils.timer.logInfo

Implementation function.

Examples

Creating a timer context:

with self.timer("someCodeToTime"):
    pass  # code to time