StepDefinitions

class lsst.pipe.base.pipeline_graph.StepDefinitions(universe: DimensionUniverse | None = None, dimensions_by_label: dict[str, frozenset[str]] | None = None, verified: bool = False)

Bases: object

A collection of the ‘steps’ defined in a pipeline graph.

Steps are special task subsets that must be executed separately. They may also be associated with “sharding dimensions”, which are the dimensions of data IDs that are independent within the step: splitting up a quantum graph along a step’s sharding dimensions produces groups that can be safely executed independently.

Parameters:

universelsst.daf.butler.DimensionUniverse, optional

Definitions for data dimensions.

dimensions_by_labeldict [ str, frozenset [ str ] ], optional

Sharding dimensions for step subsets, as dictionary with task labels as keys and sets of dimension names as values.

verifiedbool, optional

Whether the step definitions have been checked since the last time they or some other relevant aspect of the pipeline graph was changed.

Notes

This class only models collections.abc.Collection (it is iterable, sized, and can be used with in tests on label names), but it also supports append, remove, and reset for modifications.

Attributes Summary

verified

Whether the step definitions have been checked since the last time they or some other relevant aspect of the pipeline graph was changed.

Methods Summary

append(label[, dimensions])	Append a new step.
assign(labels)	Set all step definitions to the given labels.
clear()	Remove all step definitions.
copy()	Create a new instance that does not share any mutable state with this one.
get_dimensions(label)	Return the dimensions that can be used to split up a step's quanta into independent groups.
remove(label)	Remove a named step.
set_dimensions(label, dimensions)	Set the dimensions that can be used to split up a step's quanta into independent groups.

Attributes Documentation

verified

Whether the step definitions have been checked since the last time they or some other relevant aspect of the pipeline graph was changed.

This is always True if there are no step definitions.

Methods Documentation

append(label: str, dimensions: Iterable[str] | DimensionGroup = ()) → None

Append a new step.

Parameters:

labelstr

Task subset label for the new step.

dimensionsIterable [ str ] or DimensionGroup, optional

Dimensions that can be used to split up the step’s quanta into independent groups.

assign(labels: Iterable[str]) → None

Set all step definitions to the given labels.

Parameters:

labelsIterable [ str ]

Subset labels to use as the new steps.

Notes

Sharding dimensions are preserved for any label that was previously a step. If labels is a StepDefinitions` instance, sharding dimensions from that instance will be used.

clear() → None

Remove all step definitions.

copy() → StepDefinitions

Create a new instance that does not share any mutable state with this one.

get_dimensions(label: str) → DimensionGroup

Return the dimensions that can be used to split up a step’s quanta into independent groups.

Parameters:

labelstr

Label for the step.

Returns:

dimensionslsst.daf.butler.DimensionGroup

Dimensions that can be used to split up this step’s quanta.

remove(label: str) → None

Remove a named step.

Parameters:

labelstr

Task subset label to remove from the list of steps.

Notes

This does not remove the task subset itself; it just “demotes” it to a non-step subset.

set_dimensions(label: str, dimensions: Iterable[str] | DimensionGroup) → None

Set the dimensions that can be used to split up a step’s quanta into independent groups.

Parameters:

labelstr

Label for the step.

dimensionslsst.daf.butler.DimensionGroup

Dimensions that can be used to split up this step’s quanta.