GraphBuilder

class lsst.pipe.base.GraphBuilder(registry: Registry, skipExistingIn: Any = None, clobberOutputs: bool = True, datastore: Datastore | None = None)

Bases: object

GraphBuilder class is responsible for building task execution graph from a Pipeline.

Parameters:

registryRegistry

Data butler instance.

skipExistingInAny

Expressions representing the collections to search for existing output datasets that should be skipped. See Ordered collection searches.

clobberOutputsbool, optional

If True (default), allow quanta to created even if partial outputs exist; this requires the same behavior behavior to be enabled when executing.

datastoreDatastore, optional

If not None then fill datastore records in each generated Quantum.

Methods Summary

makeGraph(pipeline, collections, run, userQuery)

Create execution graph for a pipeline.

Methods Documentation

makeGraph(pipeline: ~lsst.pipe.base.pipeline.Pipeline | ~collections.abc.Iterable[~lsst.pipe.base.pipeline.TaskDef], collections: ~typing.Any, run: str, userQuery: str | None, datasetQueryConstraint: ~lsst.pipe.base._datasetQueryConstraints.DatasetQueryConstraintVariant = <class 'lsst.pipe.base._datasetQueryConstraints._ALL'>, metadata: ~collections.abc.Mapping[str, ~typing.Any] | None = None, bind: ~collections.abc.Mapping[str, ~typing.Any] | None = None, dataId: ~lsst.daf.butler.dimensions._coordinate.DataCoordinate | None = None) → QuantumGraph

Create execution graph for a pipeline.

Parameters:

pipelinePipeline or Iterable [ TaskDef ]

Pipeline definition, task names/classes and their configs.

collectionsAny

Expressions representing the collections to search for input datasets. See Ordered collection searches.

runstr

Name of the RUN collection for output datasets. Collection does not have to exist and it will be created when graph is executed.

userQuerystr

String which defines user-defined selection for registry, should be empty or None if there is no restrictions on data selection.

datasetQueryConstraintDatasetQueryConstraintVariant, optional

The query constraint variant that should be used to constraint the query based on dataset existance, defaults to DatasetQueryConstraintVariant.ALL.

metadataOptional Mapping of str to primitives

This is an optional parameter of extra data to carry with the graph. Entries in this mapping should be able to be serialized in JSON.

bindMapping, optional

Mapping containing literal values that should be injected into the userQuery expression, keyed by the identifiers they replace.

dataIdlsst.daf.butler.DataCoordinate, optional

Data ID that should also be included in the query constraint. Ignored if pipeline is a Pipeline instance (which has its own data ID).

Returns:

graphQuantumGraph

The constructed graph.

Raises:

UserExpressionError

Raised when user expression cannot be parsed.

OutputExistsError

Raised when output datasets already exist.

Exception

Other exceptions types may be raised by underlying registry classes.