DiaPipelineConnections

class lsst.ap.association.DiaPipelineConnections(*, config: PipelineTaskConfig | None = None)

Bases: PipelineTaskConnections

Butler connections for DiaPipelineTask.

Attributes Summary

allConnections	Mapping holding all connection attributes.
apdbMarker	Connection for output dataset.
associatedDiaSources	Connection for output dataset.
associatedSsSources	Connection for output dataset.
defaultTemplates
deprecatedTemplates
diaForcedSources	Connection for output dataset.
diaObjects	Connection for output dataset.
diaSourceTable	Class used for declaring PipelineTask input connections.
diffIm	Class used for declaring PipelineTask input connections.
dimensions	Set of dimension names that define the unit of work for this task.
exposure	Class used for declaring PipelineTask input connections.
initInputs	Set with the names of all InitInput connection attributes.
initOutputs	Set with the names of all InitOutput connection attributes.
inputs	Set with the names of all connectionTypes.Input connection attributes.
marginalDiaSources	Connection for output dataset.
newDiaSources	Connection for output dataset.
outputs	Set with the names of all Output connection attributes.
preloadedDiaForcedSources	Class used for declaring PipelineTask input connections.
preloadedDiaObjects	Class used for declaring PipelineTask input connections.
preloadedDiaSources	Class used for declaring PipelineTask input connections.
prerequisiteInputs	Set with the names of all PrerequisiteInput connection attributes.
solarSystemObjectTable	Class used for declaring PipelineTask input connections.
template	Class used for declaring PipelineTask input connections.
unassociatedSsObjects	Connection for output dataset.

Methods Summary

adjustQuantum(inputs, outputs, label, dataId)

Override to make adjustments to lsst.daf.butler.DatasetRef objects in the lsst.daf.butler.core.Quantum during the graph generation stage of the activator.

Attributes Documentation

allConnections: Mapping[str, BaseConnection] = {'apdbMarker': Output(name='apdb_marker', storageClass='Config', doc='Marker dataset storing the configuration of the Apdb for each visit/detector. Used to signal the completion of the pipeline.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'associatedDiaSources': Output(name='{fakesType}{coaddName}Diff_assocDiaSrc', storageClass='ArrowAstropy', doc='Optional output storing the DiaSource catalog after matching, calibration, and standardization for insertion into the Apdb.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'associatedSsSources': Output(name='{fakesType}{coaddName}Diff_associatedSsSources', storageClass='ArrowAstropy', doc='Optional output storing ssSource data computed during association.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'diaForcedSources': Output(name='{fakesType}{coaddName}Diff_diaForcedSrc', storageClass='ArrowAstropy', doc='Optional output storing the forced sources computed at the diaObject positions.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'diaObjects': Output(name='{fakesType}{coaddName}Diff_diaObject', storageClass='ArrowAstropy', doc='Optional output storing the updated diaObjects associated to these sources.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'diaSourceTable': Input(name='{fakesType}{coaddName}Diff_diaSrcTable', storageClass='DataFrame', doc='Catalog of calibrated DiaSources.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'diffIm': Input(name='{fakesType}{coaddName}Diff_differenceExp', storageClass='ExposureF', doc='Difference image on which the DiaSources were detected.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'exposure': Input(name='{fakesType}calexp', storageClass='ExposureF', doc='Calibrated exposure differenced with a template image during image differencing.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'marginalDiaSources': Output(name='{fakesType}marginal_new_dia_source', storageClass='ArrowAstropy', doc='Low SNR diaSources not associated with an existing diaObject that were rejected instead of creating a new diaObject', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'newDiaSources': Output(name='{fakesType}new_dia_source', storageClass='ArrowAstropy', doc='New diaSources not associated with an existing diaObject that were used to create a new diaObject', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False), 'preloadedDiaForcedSources': Input(name='preloaded_diaForcedSources', storageClass='DataFrame', doc='DiaForcedSources preloaded from the APDB.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'group', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'preloadedDiaObjects': Input(name='preloaded_diaObjects', storageClass='DataFrame', doc='DiaObjects preloaded from the APDB.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'group', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'preloadedDiaSources': Input(name='preloaded_diaSources', storageClass='DataFrame', doc='DiaSources preloaded from the APDB.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'group', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'solarSystemObjectTable': Input(name='preloaded_SsObjects', storageClass='ArrowAstropy', doc='Catalog of SolarSolarSystem objects expected to be observable in this detectorVisit.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'group', 'detector'), isCalibration=False, deferLoad=False, minimum=0, deferGraphConstraint=False, deferBinding=False), 'template': Input(name='{fakesType}{coaddName}Diff_templateExp', storageClass='ExposureF', doc='Warped template used to create `subtractedExposure`. Not PSF matched.', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False, deferLoad=False, minimum=1, deferGraphConstraint=False, deferBinding=False), 'unassociatedSsObjects': Output(name='ssUnassociatedObjects', storageClass='ArrowAstropy', doc='Expected locations of an ssObject with no source', multiple=False, deprecated=None, _deprecation_context='', dimensions=('instrument', 'visit', 'detector'), isCalibration=False)}

Mapping holding all connection attributes.

This is a read-only view that is automatically updated when connection attributes are added, removed, or replaced in __init__. It is also updated after __init__ completes to reflect changes in inputs, prerequisiteInputs, outputs, initInputs, and initOutputs.

apdbMarker

Connection for output dataset.

associatedDiaSources

Connection for output dataset.

associatedSsSources

Connection for output dataset.

defaultTemplates = {'coaddName': 'deep', 'fakesType': ''}

deprecatedTemplates = {}

diaForcedSources

Connection for output dataset.

diaObjects

Connection for output dataset.

diaSourceTable

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

diffIm

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

dimensions: set[str] = {'detector', 'instrument', 'visit'}

Set of dimension names that define the unit of work for this task.

Required and implied dependencies will automatically be expanded later and need not be provided.

This may be replaced or modified in __init__ to change the dimensions of the task. After __init__ it will be a frozenset and may not be replaced.

exposure

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

initInputs: set[str] = frozenset({})

Set with the names of all InitInput connection attributes.

See inputs for additional information.

initOutputs: set[str] = frozenset({})

Set with the names of all InitOutput connection attributes.

See inputs for additional information.

inputs: set[str] = frozenset({'diaSourceTable', 'diffIm', 'exposure', 'preloadedDiaForcedSources', 'preloadedDiaObjects', 'preloadedDiaSources', 'solarSystemObjectTable', 'template'})

Set with the names of all connectionTypes.Input connection attributes.

This is updated automatically as class attributes are added, removed, or replaced in __init__. Removing entries from this set will cause those connections to be removed after __init__ completes, but this is supported only for backwards compatibility; new code should instead just delete the collection attributed directly. After __init__ this will be a frozenset and may not be replaced.

marginalDiaSources

Connection for output dataset.

newDiaSources

Connection for output dataset.

outputs: set[str] = frozenset({'apdbMarker', 'associatedDiaSources', 'associatedSsSources', 'diaForcedSources', 'diaObjects', 'marginalDiaSources', 'newDiaSources', 'unassociatedSsObjects'})

Set with the names of all Output connection attributes.

See inputs for additional information.

preloadedDiaForcedSources

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

preloadedDiaObjects

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

preloadedDiaSources

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

prerequisiteInputs: set[str] = frozenset({})

Set with the names of all PrerequisiteInput connection attributes.

See inputs for additional information.

solarSystemObjectTable

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

template

Class used for declaring PipelineTask input connections.

Raises

TypeError

Raised if minimum is greater than one but multiple=False.

NotImplementedError

Raised if minimum is zero for a regular Input connection; this is not currently supported by our QuantumGraph generation algorithm.

unassociatedSsObjects

Connection for output dataset.

Methods Documentation

adjustQuantum(inputs, outputs, label, dataId)

Override to make adjustments to lsst.daf.butler.DatasetRef objects in the lsst.daf.butler.core.Quantum during the graph generation stage of the activator.

This implementation checks to make sure that the filters in the dataset are compatible with AP processing as set by the Apdb/DPDD schema.

Parameters

inputsdict

Dictionary whose keys are an input (regular or prerequisite) connection name and whose values are a tuple of the connection instance and a collection of associated DatasetRef objects. The exact type of the nested collections is unspecified; it can be assumed to be multi-pass iterable and support len and in, but it should not be mutated in place. In contrast, the outer dictionaries are guaranteed to be temporary copies that are true dict instances, and hence may be modified and even returned; this is especially useful for delegating to super (see notes below).

outputsdict

Dict of output datasets, with the same structure as inputs.

labelstr

Label for this task in the pipeline (should be used in all diagnostic messages).

data_idlsst.daf.butler.DataCoordinate

Data ID for this quantum in the pipeline (should be used in all diagnostic messages).

Returns

adjusted_inputsdict

Dict of the same form as inputs with updated containers of input DatasetRef objects. Connections that are not changed should not be returned at all. Datasets may only be removed, not added. Nested collections may be of any multi-pass iterable type, and the order of iteration will set the order of iteration within PipelineTask.runQuantum.

adjusted_outputsdict

Dict of updated output datasets, with the same structure and interpretation as adjusted_inputs.

Raises

ScalarError

Raised if any Input or PrerequisiteInput connection has multiple set to False, but multiple datasets.

NoWorkFound

Raised to indicate that this quantum should not be run; not enough datasets were found for a regular Input connection, and the quantum should be pruned or skipped.

FileNotFoundError

Raised to cause QuantumGraph generation to fail (with the message included in this exception); not enough datasets were found for a PrerequisiteInput connection.