TractMatchedPreparationConnections

class lsst.faro.preparation.TractMatchedPreparationConnections(*, config=None)

Bases: lsst.faro.base.MatchedBaseConnections

Attributes Summary

allConnections
astromCalibs
defaultTemplates
dimensions
externalPhotoCalibGlobalCatalog
externalPhotoCalibTractCatalog
externalSkyWcsGlobalCatalog
externalSkyWcsTractCatalog
initInputs
initOutputs
inputs
outputCatalog
outputs
photoCalibs
prerequisiteInputs
skyMap
sourceCatalogs

Methods Summary

adjustQuantum(inputs, …) Override to make adjustments to lsst.daf.butler.DatasetRef objects in the lsst.daf.butler.core.Quantum during the graph generation stage of the activator.
buildDatasetRefs(quantum) Builds QuantizedConnections corresponding to input Quantum

Attributes Documentation

allConnections = {'astromCalibs': Input(name='{wcsName}', storageClass='Wcs', doc='WCS for the catalog.', multiple=True, dimensions=('instrument', 'visit', 'detector', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'externalPhotoCalibGlobalCatalog': Input(name='{externalPhotoCalibName}PhotoCalibCatalog', storageClass='ExposureCatalog', doc='Per-visit photometric calibrations computed globally (with no tract information). These catalogs use the detector id for the catalog id, sorted on id for fast lookup.', multiple=True, dimensions=('instrument', 'visit', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'externalPhotoCalibTractCatalog': Input(name='{externalPhotoCalibName}PhotoCalibCatalog', storageClass='ExposureCatalog', doc='Per-tract, per-visit photometric calibrations. These catalogs use the detector id for the catalog id, sorted on id for fast lookup.', multiple=True, dimensions=('instrument', 'visit', 'tract', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'externalSkyWcsGlobalCatalog': Input(name='{externalWcsName}SkyWcsCatalog', storageClass='ExposureCatalog', doc='Per-visit wcs calibrations computed globally (with no tract information). These catalogs use the detector id for the catalog id, sorted on id for fast lookup.', multiple=True, dimensions=('instrument', 'visit', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'externalSkyWcsTractCatalog': Input(name='{externalWcsName}SkyWcsCatalog', storageClass='ExposureCatalog', doc='Per-tract, per-visit wcs calibrations. These catalogs use the detector id for the catalog id, sorted on id for fast lookup.', multiple=True, dimensions=('instrument', 'visit', 'tract', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'outputCatalog': Output(name='matchedCatalogTract', storageClass='SimpleCatalog', doc='Resulting matched catalog.', multiple=False, dimensions=('tract', 'instrument', 'band'), isCalibration=False), 'photoCalibs': Input(name='{photoCalibName}', storageClass='PhotoCalib', doc='Photometric calibration object.', multiple=True, dimensions=('instrument', 'visit', 'detector', 'band'), isCalibration=False, deferLoad=False, minimum=1), 'skyMap': Input(name='skyMap', storageClass='SkyMap', doc='Input definition of geometry/bbox and projection/wcs for warped exposures', multiple=False, dimensions=('skymap',), isCalibration=False, deferLoad=False, minimum=1), 'sourceCatalogs': Input(name='src', storageClass='SourceCatalog', doc='Source catalogs to match up.', multiple=True, dimensions=('instrument', 'visit', 'detector', 'band'), isCalibration=False, deferLoad=False, minimum=1)}
astromCalibs
defaultTemplates = {'coaddName': 'deep', 'externalPhotoCalibName': 'fgcm', 'externalWcsName': 'jointcal', 'photoCalibName': 'calexp.photoCalib', 'wcsName': 'calexp.wcs'}
dimensions = {'band', 'instrument', 'skymap', 'tract'}
externalPhotoCalibGlobalCatalog
externalPhotoCalibTractCatalog
externalSkyWcsGlobalCatalog
externalSkyWcsTractCatalog
initInputs = frozenset()
initOutputs = frozenset()
inputs = frozenset({'astromCalibs', 'externalPhotoCalibTractCatalog', 'sourceCatalogs', 'skyMap', 'externalSkyWcsGlobalCatalog', 'externalPhotoCalibGlobalCatalog', 'externalSkyWcsTractCatalog', 'photoCalibs'})
outputCatalog
outputs = frozenset({'outputCatalog'})
photoCalibs
prerequisiteInputs = frozenset()
skyMap
sourceCatalogs

Methods Documentation

adjustQuantum(inputs: typing.Dict[str, typing.Tuple[BaseInput, typing.Collection[DatasetRef]]], outputs: typing.Dict[str, typing.Tuple[Output, typing.Collection[DatasetRef]]], label: str, data_id: DataCoordinate) → tuple.Tuple[typing.Mapping[str, typing.Tuple[BaseInput, typing.Collection[DatasetRef]]], typing.Mapping[str, typing.Tuple[Output, typing.Collection[DatasetRef]]]]

Override to make adjustments to lsst.daf.butler.DatasetRef objects in the lsst.daf.butler.core.Quantum during the graph generation stage of the activator.

Parameters:
inputs : dict

Dictionary whose keys are an input (regular or prerequisite) connection name and whose values are a tuple of the connection instance and a collection of associated DatasetRef objects. The exact type of the nested collections is unspecified; it can be assumed to be multi-pass iterable and support len and in, but it should not be mutated in place. In contrast, the outer dictionaries are guaranteed to be temporary copies that are true dict instances, and hence may be modified and even returned; this is especially useful for delegating to super (see notes below).

outputs : Mapping

Mapping of output datasets, with the same structure as inputs.

label : str

Label for this task in the pipeline (should be used in all diagnostic messages).

data_id : lsst.daf.butler.DataCoordinate

Data ID for this quantum in the pipeline (should be used in all diagnostic messages).

Returns:
adjusted_inputs : Mapping

Mapping of the same form as inputs with updated containers of input DatasetRef objects. Connections that are not changed should not be returned at all. Datasets may only be removed, not added. Nested collections may be of any multi-pass iterable type, and the order of iteration will set the order of iteration within PipelineTask.runQuantum.

adjusted_outputs : Mapping

Mapping of updated output datasets, with the same structure and interpretation as adjusted_inputs.

Raises:
ScalarError

Raised if any Input or PrerequisiteInput connection has multiple set to False, but multiple datasets.

NoWorkFound

Raised to indicate that this quantum should not be run; not enough datasets were found for a regular Input connection, and the quantum should be pruned or skipped.

FileNotFoundError

Raised to cause QuantumGraph generation to fail (with the message included in this exception); not enough datasets were found for a PrerequisiteInput connection.

Notes

The base class implementation performs important checks. It always returns an empty mapping (i.e. makes no adjustments). It should always called be via super by custom implementations, ideally at the end of the custom implementation with already-adjusted mappings when any datasets are actually dropped, e.g.:

def adjustQuantum(self, inputs, outputs, label, data_id):
    # Filter out some dataset refs for one connection.
    connection, old_refs = inputs["my_input"]
    new_refs = [ref for ref in old_refs if ...]
    adjusted_inputs = {"my_input", (connection, new_refs)}
    # Update the original inputs so we can pass them to super.
    inputs.update(adjusted_inputs)
    # Can ignore outputs from super because they are guaranteed
    # to be empty.
    super().adjustQuantum(inputs, outputs, label_data_id)
    # Return only the connections we modified.
    return adjusted_inputs, {}

Removing outputs here is guaranteed to affect what is actually passed to PipelineTask.runQuantum, but its effect on the larger graph may be deferred to execution, depending on the context in which adjustQuantum is being run: if one quantum removes an output that is needed by a second quantum as input, the second quantum may not be adjusted (and hence pruned or skipped) until that output is actually found to be missing at execution time.

Tasks that desire zip-iteration consistency between any combinations of connections that have the same data ID should generally implement adjustQuantum to achieve this, even if they could also run that logic during execution; this allows the system to see outputs that will not be produced because the corresponding input is missing as early as possible.

buildDatasetRefs(quantum: lsst.daf.butler.core.quantum.Quantum) → Tuple[lsst.pipe.base.connections.InputQuantizedConnection, lsst.pipe.base.connections.OutputQuantizedConnection]

Builds QuantizedConnections corresponding to input Quantum

Parameters:
quantum : lsst.daf.butler.Quantum

Quantum object which defines the inputs and outputs for a given unit of processing

Returns:
retVal : tuple of (InputQuantizedConnection,

OutputQuantizedConnection) Namespaces mapping attribute names (identifiers of connections) to butler references defined in the input lsst.daf.butler.Quantum