PipelineDatasetTypes

class lsst.pipe.base.PipelineDatasetTypes(initInputs: NamedValueSet[DatasetType], initOutputs: NamedValueSet[DatasetType], initIntermediates: NamedValueSet[DatasetType], inputs: NamedValueSet[DatasetType], prerequisites: NamedValueSet[DatasetType], intermediates: NamedValueSet[DatasetType], outputs: NamedValueSet[DatasetType], byTask: Mapping[str, TaskDatasetTypes])

Bases: object

An immutable struct that classifies the dataset types used in a Pipeline.

Attributes Summary

packagesDatasetName

Name of a dataset type used to save package versions.

Methods Summary

fromPipeline(pipeline, *, registry[, ...])	Extract and classify the dataset types from all tasks in a Pipeline.
initOutputNames(pipeline, *[, ...])	Return the names of dataset types ot task initOutputs, Configs, and package versions for a pipeline.

Attributes Documentation

packagesDatasetName: ClassVar[str] = 'packages'

Name of a dataset type used to save package versions.

Methods Documentation

classmethod fromPipeline(pipeline: Union[Pipeline, Iterable[TaskDef]], *, registry: Registry, include_configs: bool = True, include_packages: bool = True) → PipelineDatasetTypes

Extract and classify the dataset types from all tasks in a Pipeline.

Parameters:

pipeline: `Pipeline` or `Iterable` [ `TaskDef` ]

A collection of tasks that can be run together.

registry: `Registry`

Registry used to construct normalized DatasetType objects and retrieve those that are incomplete.

include_configsbool, optional

If True (default) include config dataset types as initOutputs.

include_packagesbool, optional

If True (default) include the dataset type for software package versions in initOutputs.

Returns:

types: PipelineDatasetTypes

The dataset types used by this Pipeline.

Raises:

ValueError

Raised if Tasks are inconsistent about which datasets are marked prerequisite. This indicates that the Tasks cannot be run as part of the same Pipeline.

classmethod initOutputNames(pipeline: Union[Pipeline, Iterable[TaskDef]], *, include_configs: bool = True, include_packages: bool = True) → Iterator[str]

Return the names of dataset types ot task initOutputs, Configs, and package versions for a pipeline.

Parameters:

pipeline: `Pipeline` or `Iterable` [ `TaskDef` ]

A Pipeline instance or collection of TaskDef instances.

include_configsbool, optional

If True (default) include config dataset types.

include_packagesbool, optional

If True (default) include the dataset type for package versions.

Yields:

datasetTypeNamestr

Name of the dataset type.