PhotonTransferCurveDataset

class lsst.ip.isr.PhotonTransferCurveDataset(ampNames=[], ptcFitType=None, covMatrixSide=1, covMatrixSideFullCovFit=None, **kwargs)

Bases: IsrCalib

A simple class to hold the output data from the PTC task.

The dataset is made up of a dictionary for each item, keyed by the amplifiers’ names, which much be supplied at construction time. New items cannot be added to the class to save accidentally saving to the wrong property, and the class can be frozen if desired. inputExpIdPairs records the exposures used to produce the data. When fitPtc() or fitCovariancesAstier() is run, a mask is built up, which is by definition always the same length as inputExpIdPairs, rawExpTimes, rawMeans and rawVars, and is a list of bools, which are incrementally set to False as points are discarded from the fits.

Parameters

ampNameslist

List with the names of the amplifiers of the detector at hand.

ptcFitTypestr, optional

Type of model fitted to the PTC: “EXPAPPROXIMATION”, or “FULLCOVARIANCE” or “FULLCOVARIANCE_NO_B”.

covMatrixSideint, optional

Maximum lag of measured covariances (size of square covariance matrices).

covMatrixSideFullCovFitint, optional

Maximum covariances lag for FULLCOVARIANCE fit. It should be less or equal than covMatrixSide.

kwargsdict, optional

Other keyword arguments to pass to the parent init.

Notes

The stored attributes are:

badAmpslist [str]

List with bad amplifiers names.

inputExpIdPairsdict, [str, list]

Dictionary keyed by amp names containing the input exposures IDs.

inputExpPairMjdStartListdict, [str, np.ndarray]

Dictionary keyed by amp names containing the start mjd from the first exposure in each flat pair.

expIdMaskdict, [str, np.ndarray]

Dictionary keyed by amp names containing the mask produced after outlier rejection. The mask produced by the “FULLCOVARIANCE” option may differ from the one produced in the other two PTC fit types.

expIdRolloffMaskdict, [str, np.ndarray]

Dictionary keyed by amp names containing the mask produced after outlier rejection and extended to fit the PTC rolloff.

rawExpTimesdict, [str, np.ndarray]

Dictionary keyed by amp names containing the unmasked exposure times.

rawMeansdict, [str, np.ndarray]

Dictionary keyed by amp names containing the unmasked average of the means of the exposures in each flat pair (units: adu).

rawVarsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the variance of the difference image of the exposures in each flat pair (units: adu^2).

rawDeltasdict, [str, np.ndarray]

Dictionary keyed by amp names containing the scaled unmasked delta of the means of the exposures in each flat pair (mean2 - mean1) (units: adu).

rowMeanVariancedict, [str, np.ndarray]

Dictionary keyed by amp names containing the variance of the means of the rows of the difference image of the exposures in each flat pair (units: adu^2).

histVarsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the variance of the difference image of the exposures in each flat pair estimated by fitting a Gaussian model.

histChi2Dofsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the chi-squared per degree of freedom fitting the difference image to a Gaussian model.

kspValuesdict, [str, np.ndarray]

Dictionary keyed by amp names containing the KS test p-value from fitting the difference image to a Gaussian model.

gaindict, [str, float]

Dictionary keyed by amp names containing the fitted gains. May be adjusted by amp-offset gain ratios if configured in PTC solver.

gainUnadjusteddict, [str, float]

Dictionary keyed by amp names containing unadjusted (raw) fit gain values. May be the same as gain values if amp-offset adjustment is not turned on.

gainErrdict, [str, float]

Dictionary keyed by amp names containing the errors on the fitted gains.

gainListdict, [str, np.ndarray]

Dictionary keyed by amp names containing the gain estimated from each flat pair.

overscanMedianLevelListdict, [str, np.ndarray]

Dictionary keyed by amp names containing the median overscan level from each input flat pair (units: adu).

overscanMediandict `, [`str, float]

Dictionary keyed by amp names containing the median of overscanMedianLevelList[expIdMask] (units: adu).

overscanMedianSigmadict `, [`str, float]

Dictionary keyed by amp names containing the median absolute deviation of overscanMedianLevelList[expIdMask] (units: adu).

noiseListdict, [str, np.ndarray]

Dictionary keyed by amp names containing the mean overscan standard deviation from each flat pair (units: adu).

noisedict, [str, float]

Dictionary keyed by amp names containing the fitted noise (units: electron).

noiseErrdict, [str, float]

Dictionary keyed by amp names containing the errors on the fitted noise (units: electron).

ampOffsetsdict, [str, float]

Dictionary keyed by amp names containing amp-to-amp offsets (units: adu).

ptcFitParsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the raveled array fitted parameters of the PTC model.

ptcFitParsErrordict, [str, np.ndarray]

Dictionary keyed by amp names containing the raveled array errors on the fitted parameters of the PTC model for ptcFitType.

ptcFitChiSqdict, [str, float]

Dictionary keyed by amp names containing the reduced chi squared of the fit.

ptcTurnoffdict [str, `float]

Flux value (in adu) where the variance of the PTC curve starts decreasing consistently.

ptcTurnoffSamplingErrordict [str, float]

Sampling error on the ptcTurnoff, based on the flux sampling of the input PTC (units: adu).

ptcRolloffdict [str, `float]

Flux value (in adu) where the variance of the PTC curve begins starts to move away from the PTC model by some threshold.

ptcRolloffErrordict [str, float]

Covariance error from the fit to the PTC rolloff (units: adu).

ptcRolloffdict [str, `float]

Curvature parameter of the PTC rolloff.

ptcRolloffErrordict [str, float]

Fitting error of the curvature parameter of the PTC rolloff.

nPixelCovariancesdict, [str, int]

Dictionary keyed by amp names containing the number of pixels that were used to measure the covariances.

covariancesdict, [str, np.ndarray]

Dictionary keyed by amp names containing a list of measured covariances per mean flux (units: adu^2).

covariancesModeldict, [str, np.ndarray]

Dictionary keyed by amp names containinging covariances model (Eq. 20 of Astier+19) per mean flux (units: adu^2).

covariancesSqrtWeightsdict, [str, np.ndarray]

Dictionary keyed by amp names containinging sqrt. of covariances weights (units: 1/adu).

aMatrixdict, [str, np.ndarray]

Dictionary keyed by amp names containing the “a” parameters from the model in Eq. 20 of Astier+19 (units: 1/electron).

bMatrixdict, [str, np.ndarray]

Dictionary keyed by amp names containing the “b” parameters from the model in Eq. 20 of Astier+19 (units: 1/electron).

noiseMatrixdict, [str, np.ndarray]

Dictionary keyed by amp names containing the “noise” parameters from the model in Eq. 20 of Astier+19 (units: electron^2).

covariancesModelNoBdict, [str, np.ndarray]

Dictionary keyed by amp names containing covariances model (with ‘b’=0 in Eq. 20 of Astier+19) per mean flux (units: adu^2). Will be removed after v29.

aMatrixNoBdict, [str, np.ndarray]

Dictionary keyed by amp names containing the “a” parameters from the model in Eq. 20 of Astier+19 (and ‘b’ = 0) (units: 1/electron). Will be removed after v29.

noiseMatrixNoBdict, [str, np.ndarray]

Dictionary keyed by amp names containing the “noise” parameters from the model in Eq. 20 of Astier+19, with ‘b’ = 0 (units: electron^2). Will be removed after v29.

finalVarsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the masked variance of the difference image of each flat pair. If needed, each array will be right-padded with np.nan to match the length of rawExpTimes.

finalModelVarsdict, [str, np.ndarray]

Dictionary keyed by amp names containing the masked modeled variance of the difference image of each flat pair. If needed, each array will be right-padded with np.nan to match the length of rawExpTimes.

finalMeansdict, [str, np.ndarray]

Dictionary keyed by amp names containing the masked average of the means of the exposures in each flat pair. If needed, each array will be right-padded with np.nan to match the length of rawExpTimes.

photoChargesdict, [str, np.ndarray]

Dictionary keyed by amp names containing the integrated photocharge for linearity calibration.

photoChargeDeltasdict, [str, np.ndarray]

Dictionary keyed by amp names containing the delta for the integrated photocharge (photocharge2 - photocharge1).

auxValuesdict, [str, np.ndarray]

Dictionary of per-detector auxiliary header values that can be used for PTC, linearity computation.

Version 1.1 adds the ptcTurnoff attribute. Version 1.2 adds the histVars, histChi2Dofs, and kspValues

attributes.

Version 1.3 adds the noiseMatrix and noiseMatrixNoB attributes. Version 1.4 adds the auxValues attribute. Version 1.5 adds the covMatrixSideFullCovFit attribute. Version 1.6 adds the rowMeanVariance attribute. Version 1.7 adds the noiseList attribute. Version 1.8 adds the ptcTurnoffSamplingError attribute. Version 1.9 standardizes PTC noise units to electron. Version 2.0 adds the ampOffsets, gainUnadjusted, and

gainList attributes.

Version 2.1 deprecates the covariancesModelNoB, aMatrixNoB, and

noiseMatrixNoB attributes.

Version 2.2 adds the overscanMedianLevelList and

inputExpPairMjdStartList attributes.

Version 2.3 adds the overscanMedian and

overscanMedianSigma attrbutes.

Version 2.4 adds the nPixelCovariances attribute. Version 2.5 adds the rawDeltas and photoChargeDeltas attributes. Version 2.6 adds the expIdRolloffMask, ptcRolloff, ptcRolloffError,

ptcRolloffTau, and ptcRolloffTauError attributes. Also deprecates the POLYNOMIAL fit type.

Methods Summary

appendPartialPtc(partialPtc)	Append a partial PTC dataset to this dataset.
evalPtcModel(mu)	Computes the covariance model at specific signal levels.
fromDetector(detector)	Read metadata parameters from a detector.
fromDict(dictionary)	Construct a calibration from a dictionary of properties.
fromTable(tableList)	Construct calibration from a list of tables.
getExpIdsUsed(ampName)	Get the exposures used, i.e. not discarded, for a given amp.
getGoodAmps()	Get the good amps from this PTC.
getGoodPoints(ampName)	Get the good points used for a given amp in the PTC.
setAmpValuesPartialDataset(ampName[, ...])	Set the amp values for a partial PTC Dataset (from cpExtractPtcTask).
setAuxValuesPartialDataset(auxDict)	Set a dictionary of auxiliary values for a partial dataset.
sort(sortIndex)	Sort the components of the PTC by a given sort index.
toDict()	Return a dictionary containing the calibration properties.
toTable()	Construct a list of tables containing the information in this calibration.
updateMetadata(**kwargs)	Update calibration metadata.
validateGainNoiseTurnoffValues(ampName[, doWarn])	Ensure the gain, read noise, and PTC turnoff have sensible values.

Methods Documentation

appendPartialPtc(partialPtc)

Append a partial PTC dataset to this dataset.

Parameters

partialPtclsst.ip.isr.PhotonTransferCurveDataset

Partial PTC to append. Should only have one element.

evalPtcModel(mu)

Computes the covariance model at specific signal levels.

Parameters

munumpy.array, (N,)

List of mean signals in ADU.

Raises

RuntimeError

Raised if ptcFitType is invalid.

Returns

covModelnumpy.array, (N, M, M)

Covariances model at mu (in ADU^2).

Notes

Computes the covModel for all mu, and it returns cov[N, M, M], where the variance model is cov[:,0,0]. Both mu and cov are in ADUs and ADUs squared. This routine evaluates the approximation in Eq. 16 of Astier+19 (1905.08677) if self.ptcFitType == EXPAPPROXIMATION, and Eq. 20 of Astier+19 if self.ptcFitType == FULLCOVARIANCE(_NO_B).

The EXPAPPROXIMATION model (Eq. 16 of Astier+19) is only an approximation for the variance (cov[0,0]), so the function returns covModel of shape (N,), representing an array of [C_{00}] if self.ptcFitType == EXPAPPROXIMATION.

fromDetector(detector)

Read metadata parameters from a detector.

Parameters

detectorlsst.afw.cameraGeom.detector

Input detector with parameters to use.

Returns

caliblsst.ip.isr.PhotonTransferCurveDataset

The calibration constructed from the detector.

classmethod fromDict(dictionary)

Construct a calibration from a dictionary of properties. Must be implemented by the specific calibration subclasses.

Parameters

dictionarydict

Dictionary of properties.

Returns

caliblsst.ip.isr.PhotonTransferCurveDataset

Constructed calibration.

Raises

RuntimeError

Raised if the supplied dictionary is for a different calibration.

classmethod fromTable(tableList)

Construct calibration from a list of tables. This method uses the fromDict method to create the calibration, after constructing an appropriate dictionary from the input tables.

Parameters

tableListlist [lsst.afw.table.Table]

List of tables to use to construct the datasetPtc.

Returns

caliblsst.ip.isr.PhotonTransferCurveDataset

The calibration defined in the tables.

getExpIdsUsed(ampName)

Get the exposures used, i.e. not discarded, for a given amp. If no mask has been created yet, all exposures are returned.

Parameters

ampName : str

Returns

expIdsUsedlist [tuple]

List of pairs of exposure ids used in PTC.

getGoodAmps()

Get the good amps from this PTC.

getGoodPoints(ampName)

Get the good points used for a given amp in the PTC.

Parameters

ampNamestr

Amplifier’s name.

Returns

goodPointsnp.ndarray

Boolean array of good points used in PTC.

setAmpValuesPartialDataset(ampName, inputExpIdPair=(-1, -1), inputExpPairMjdStart=nan, rawExpTime=nan, rawMean=nan, rawVar=nan, rawDelta=nan, rowMeanVariance=nan, photoCharge=nan, photoChargeDelta=nan, ampOffset=nan, expIdMask=False, expIdRolloffMask=False, nPixelCovariance=-1, covariance=None, covSqrtWeights=None, gain=nan, noise=nan, overscanMedianLevel=nan, histVar=nan, histChi2Dof=nan, kspValue=0.0)

Set the amp values for a partial PTC Dataset (from cpExtractPtcTask).

Parameters

ampNamestr

Name of the amp to set the values.

inputExpIdPairtuple [int]

Exposure IDs of input pair.

inputExpPairMjdStartfloat, optional

The start MJD of first exposure in the flat pair.

rawExpTimefloat, optional

Exposure time for this exposure pair (units: sec).

rawMeanfloat, optional

Average of the means of the exposures in this pair (units: adu).

rawVarfloat, optional

Variance of the difference of the exposures in this pair (units: adu^2).

rawDeltafloat, optional

Delta of the means of the exposure in this pair (units: adu).

rowMeanVariancefloat, optional

Variance of the means of the rows in the difference image of the exposures in this pair (units: adu^2).

photoChargefloat, optional

Integrated photocharge for flat pair for linearity calibration (arbitrary units).

photoChargeDeltafloat, optional

Delta between integrated photocharge for the flat pair (arbitrary units).

ampOffsetfloat, optional

Amp offset for this amplifier.

expIdMaskbool, optional

Flag setting if this exposure pair should be used (True) or not used (False).

expIdRolloffMaskbool, optional

Flag setting if this exposure pair should be used (True) or not used for rolloff search (False).

nPixelCovarianceint, optional

Number of pixels that went into the covariance measurement.

covariancenp.ndarray or None, optional

Measured covariance for this exposure pair (units: adu^2).

covSqrtWeightsnp.ndarray or None, optional

Measured sqrt of covariance weights in this exposure pair (units: 1/adu).

gainfloat, optional

Estimated gain for this exposure pair (units: electron/adu).

noisefloat, optional

Estimated read noise for this exposure pair (units: electron).

overscanMedianLevelfloat, optional

Average of the median overscan levels for this exposure pair. (units: adu)

histVarfloat, optional

Variance estimated from fitting a histogram with a Gaussian model (units: adu).

histChi2Doffloat, optional

Chi-squared per degree of freedom from Gaussian histogram fit.

kspValuefloat, optional

KS test p-value from the Gaussian histogram fit.

setAuxValuesPartialDataset(auxDict)

Set a dictionary of auxiliary values for a partial dataset.

Parameters

auxDictdict [str, float]

Dictionary of float values.

sort(sortIndex)

Sort the components of the PTC by a given sort index.

The PTC is sorted in-place.

Parameters

sortIndexlist or np.ndarray

The sorting index, which must be the same length as the number of elements of the PTC.

toDict()

Return a dictionary containing the calibration properties. The dictionary should be able to be round-tripped through fromDict.

Returns

dictionarydict

Dictionary of properties.

toTable()

Construct a list of tables containing the information in this calibration.

The list of tables should create an identical calibration after being passed to this class’s fromTable method.

Returns

tableListlist [astropy.table.Table]

List of tables containing the linearity calibration information.

updateMetadata(**kwargs)

Update calibration metadata. This calls the base class’s method after ensuring the required calibration keywords will be saved.

Parameters

setDatebool, optional

Update the CALIBDATE fields in the metadata to the current time. Defaults to False.

kwargs :

Other keyword parameters to set in the metadata.

validateGainNoiseTurnoffValues(ampName, doWarn=False)

Ensure the gain, read noise, and PTC turnoff have sensible values.

Parameters

ampNamestr

Amplifier’s name.