PrerequisiteInput

class lsst.pipe.base.connectionTypes.PrerequisiteInput(name: str, storageClass: str, doc: str = '', multiple: bool = False, _deprecation_context: str = '', dimensions: Iterable[str] = (), isCalibration: bool = False, deferLoad: bool = False, minimum: int = 1, lookupFunction: Callable[[DatasetType, Registry, DataCoordinate, Sequence[str]], Iterable[DatasetRef]] | None = None, *, deprecated: str | None = None)

Bases: BaseInput

Class used for declaring PipelineTask prerequisite connections.

Raises:

TypeError

Raised if minimum is greater than one but multiple=False.

Notes

Prerequisite inputs are used for datasets that must exist in the data repository before a pipeline including this is run; they cannot be produced by another task in the same pipeline.

In exchange for this limitation, they have a number of advantages relative to regular Input connections:

• The query used to find them then during QuantumGraph generation can be fully customized by providing a lookupFunction.

• Failed searches for prerequisites during QuantumGraph generation will usually generate more helpful diagnostics than those for regular Input connections.

• The default query for prerequisite inputs relates the quantum dimensions directly to the dimensions of its dataset type, without being constrained by any of the other dimensions in the pipeline. This allows them to be used for temporal calibration lookups (which regular Input connections cannot do at present) and to work around QuantumGraph generation limitations involving cases where naive spatial overlap relationships between dimensions are not desired (e.g. a task that wants all detectors in each visit for which the visit overlaps a tract, not just those where that detector+visit combination overlaps the tract).

• Prerequisite inputs may be optional (regular inputs are never optional).

Attributes Summary

deferLoad	Whether this dataset type will be loaded as a lsst.daf.butler.DeferredDatasetHandle.
deprecated	A description of why this connection is deprecated, including the version after which it may be removed.
dimensions	The keys of the butler data coordinates for this dataset type.
doc	Documentation for this connection.
isCalibration	True if this dataset type may be included in CALIBRATION collections to associate it with a validity range, False (default) otherwise.
lookupFunction	An optional callable function that will look up PrerequisiteInputs using the DatasetType, registry, quantum dataId, and input collections passed to it.
minimum	Minimum number of datasets required for this connection, per quantum.
multiple	Indicates if this connection should expect to contain multiple objects of the given dataset type.

Attributes Documentation

deferLoad: bool = False

Whether this dataset type will be loaded as a lsst.daf.butler.DeferredDatasetHandle. PipelineTasks can use this object to load the object at a later time.

deprecated: str | None = None

A description of why this connection is deprecated, including the version after which it may be removed.

If not None, the string is appended to the docstring for this connection and the corresponding config Field.

dimensions: Iterable[str] = ()

The keys of the butler data coordinates for this dataset type.

doc: str = ''

Documentation for this connection.

isCalibration: bool = False

True if this dataset type may be included in CALIBRATION collections to associate it with a validity range, False (default) otherwise.

lookupFunction: Callable[[DatasetType, Registry, DataCoordinate, Sequence[str]], Iterable[DatasetRef]] | None = None

An optional callable function that will look up PrerequisiteInputs using the DatasetType, registry, quantum dataId, and input collections passed to it.

If no function is specified, the default temporal/spatial lookup will be used.

minimum: int = 1

Minimum number of datasets required for this connection, per quantum.

This is checked in the base implementation of PipelineTaskConnections.adjustQuantum, which raises NoWorkFound if the minimum is not met for Input connections (causing the quantum to be pruned, skipped, or never created, depending on the context), and FileNotFoundError for PrerequisiteInput connections (causing QuantumGraph generation to fail). PipelineTask implementations may provide custom adjustQuantum implementations for more fine-grained or configuration-driven constraints, as long as they are compatible with this minimum.

multiple: bool = False

Indicates if this connection should expect to contain multiple objects of the given dataset type.

Tasks with more than one connection with multiple=True with the same dimensions may want to implement PipelineTaskConnections.adjustQuantum to ensure those datasets are consistent (i.e. zip-iterable) in PipelineTask.runQuantum() and notify the execution system as early as possible of outputs that will not be produced because the corresponding input is missing.