GraphBuilder

class lsst.pipe.base.GraphBuilder(registry: Registry, skipExistingIn: Optional[Any] = None, clobberOutputs: bool = True, datastore: Optional[Datastore] = None)

Bases: object

GraphBuilder class is responsible for building task execution graph from a Pipeline.

Parameters:

registryRegistry

Data butler instance.

skipExistingIn

Expressions representing the collections to search for existing output datasets that should be skipped. May be any of the types accepted by lsst.daf.butler.CollectionSearch.fromExpression.

clobberOutputsbool, optional

If True (default), allow quanta to created even if partial outputs exist; this requires the same behavior behavior to be enabled when executing.

datastoreDatastore, optional

If not None then fill datastore records in each generated Quantum.

Methods Summary

makeGraph(pipeline, collections, run, userQuery)

Create execution graph for a pipeline.

Methods Documentation

makeGraph(pipeline: ~typing.Union[~lsst.pipe.base.pipeline.Pipeline, ~typing.Iterable[~lsst.pipe.base.pipeline.TaskDef]], collections: ~typing.Any, run: ~typing.Optional[str], userQuery: ~typing.Optional[str], datasetQueryConstraint: ~lsst.pipe.base._datasetQueryConstraints.DatasetQueryConstraintVariant = <class 'lsst.pipe.base._datasetQueryConstraints._ALL'>, metadata: ~typing.Optional[~typing.Mapping[str, ~typing.Any]] = None) → QuantumGraph

Create execution graph for a pipeline.

Parameters:

pipelinePipeline or Iterable [ TaskDef ]

Pipeline definition, task names/classes and their configs.

collections

Expressions representing the collections to search for input datasets. May be any of the types accepted by lsst.daf.butler.CollectionSearch.fromExpression.

runstr, optional

Name of the RUN collection for output datasets, if it already exists.

userQuerystr

String which defines user-defined selection for registry, should be empty or None if there is no restrictions on data selection.

datasetQueryConstraintDatasetQueryConstraintVariant, optional

The query constraint variant that should be used to constraint the query based on dataset existance, defaults to DatasetQueryConstraintVariant.ALL.

metadataOptional Mapping of str to primitives

This is an optional parameter of extra data to carry with the graph. Entries in this mapping should be able to be serialized in JSON.

Returns:

graphQuantumGraph

Raises:

UserExpressionError

Raised when user expression cannot be parsed.

OutputExistsError

Raised when output datasets already exist.

Exception

Other exceptions types may be raised by underlying registry classes.