PrerequisiteInput¶
- class lsst.pipe.base.connectionTypes.PrerequisiteInput(name: str, storageClass: str, doc: str = '', multiple: bool = False, _deprecation_context: str = '', dimensions: Iterable[str] = (), isCalibration: bool = False, deferLoad: bool = False, minimum: int = 1, lookupFunction: Callable[[DatasetType, Registry, DataCoordinate, Sequence[str]], Iterable[DatasetRef]] | None = None, *, deprecated: str | None = None)¶
Bases:
BaseInput
Class used for declaring PipelineTask prerequisite connections.
- Raises:
- TypeError
Raised if
minimum
is greater than one butmultiple=False
.
Notes
Prerequisite inputs are used for datasets that must exist in the data repository before a pipeline including this is run; they cannot be produced by another task in the same pipeline.
In exchange for this limitation, they have a number of advantages relative to regular
Input
connections:The query used to find them then during
QuantumGraph
generation can be fully customized by providing alookupFunction
.Failed searches for prerequisites during
QuantumGraph
generation will usually generate more helpful diagnostics than those for regularInput
connections.The default query for prerequisite inputs relates the quantum dimensions directly to the dimensions of its dataset type, without being constrained by any of the other dimensions in the pipeline. This allows them to be used for temporal calibration lookups (which regular
Input
connections cannot do at present) and to work aroundQuantumGraph
generation limitations involving cases where naive spatial overlap relationships between dimensions are not desired (e.g. a task that wants all detectors in each visit for which the visit overlaps a tract, not just those where that detector+visit combination overlaps the tract).Prerequisite inputs may be optional (regular inputs are never optional).
- Attributes:
- name
str
The default name used to identify the dataset type.
- storageClass
str
The storage class used when (un)/persisting the dataset type.
- multiple
bool
Indicates if this connection should expect to contain multiple objects of the given dataset type. Tasks with more than one connection with
multiple=True
with the same dimensions may want to implementPipelineTaskConnections.adjustQuantum
to ensure those datasets are consistent (i.e. zip-iterable) inPipelineTask.runQuantum
and notify the execution system as early as possible of outputs that will not be produced because the corresponding input is missing.- dimensionsiterable of
str
The
lsst.daf.butler.Butler
lsst.daf.butler.Registry
dimensions used to identify the dataset type identified by the specified name.- minimum
bool
Minimum number of datasets required for this connection, per quantum. This is checked in the base implementation of
PipelineTaskConnections.adjustQuantum
, which raisesFileNotFoundError
(causing QuantumGraph generation to fail).PipelineTask
implementations may provide customadjustQuantum
implementations for more fine-grained or configuration-driven constraints, as long as they are compatible with this minium.- lookupFunction
typing.Callable
, optional An optional callable function that will look up PrerequisiteInputs using the DatasetType, registry, quantum dataId, and input collections passed to it. If no function is specified, the default temporal spatial lookup will be used.
- name
Attributes Summary
Methods Summary
makeDatasetType
(universe[, parentStorageClass])Construct a true
DatasetType
instance with normalized dimensions.Attributes Documentation
- lookupFunction: Callable[[DatasetType, Registry, DataCoordinate, Sequence[str]], Iterable[DatasetRef]] | None = None¶
Methods Documentation
- makeDatasetType(universe: DimensionUniverse, parentStorageClass: StorageClass | str | None = None) DatasetType ¶
Construct a true
DatasetType
instance with normalized dimensions.- Parameters:
- universe
lsst.daf.butler.DimensionUniverse
Set of all known dimensions to be used to normalize the dimension names specified in config.
- parentStorageClass
lsst.daf.butler.StorageClass
orstr
, optional Parent storage class for component datasets;
None
otherwise.
- universe
- Returns:
- datasetType
DatasetType
The
DatasetType
defined by this connection.
Deprecated since version 27.0: Deprecated in favor of PipelineGraph, and will be removed after v27.
- datasetType