GraphBuilder

class lsst.pipe.base.GraphBuilder(registry: lsst.daf.butler.registry._registry.Registry, skipExistingIn: Optional[Any, None] = None, clobberOutputs: bool = True, datastore: Optional[lsst.daf.butler.core.datastore.Datastore, None] = None)

Bases: object

GraphBuilder class is responsible for building task execution graph from a Pipeline.

Parameters:

registry : Registry

Data butler instance.

skipExistingIn

Expressions representing the collections to search for existing output datasets that should be skipped. See Ordered collection searches.

clobberOutputs : bool, optional

If True (default), allow quanta to created even if partial outputs exist; this requires the same behavior behavior to be enabled when executing.

datastore : Datastore, optional

If not None then fill datastore records in each generated Quantum.

Methods Summary

makeGraph(pipeline, …)

Create execution graph for a pipeline.

Methods Documentation

makeGraph(pipeline: Union[lsst.pipe.base.pipeline.Pipeline, Iterable[lsst.pipe.base.pipeline.TaskDef]], collections: Any, run: Optional[str, None], userQuery: Optional[str, None], datasetQueryConstraint: lsst.pipe.base._datasetQueryConstraints.DatasetQueryConstraintVariant = <class 'lsst.pipe.base._datasetQueryConstraints._ALL'>, metadata: Optional[Mapping[str, Any], None] = None, resolveRefs: bool = False, bind: Optional[Mapping[str, Any], None] = None) → lsst.pipe.base.graph.graph.QuantumGraph

Create execution graph for a pipeline.

Parameters:

pipeline : Pipeline or Iterable [ TaskDef ]

Pipeline definition, task names/classes and their configs.

collections

Expressions representing the collections to search for input datasets. See Ordered collection searches.

run : str, optional

Name of the RUN collection for output datasets. Collection does not have to exist and it will be created when graph is executed.

userQuery : str

String which defines user-defined selection for registry, should be empty or None if there is no restrictions on data selection.

datasetQueryConstraint : DatasetQueryConstraintVariant, optional

The query constraint variant that should be used to constraint the query based on dataset existance, defaults to DatasetQueryConstraintVariant.ALL.

metadata : Optional Mapping of str to primitives

This is an optional parameter of extra data to carry with the graph. Entries in this mapping should be able to be serialized in JSON.

resolveRefs : bool, optional

If True then resolve all input references and generate random dataset IDs for all output and intermediate datasets. True value requires run collection to be specified.

bind : Mapping, optional

Mapping containing literal values that should be injected into the userQuery expression, keyed by the identifiers they replace.

Returns:

graph : QuantumGraph

Raises:

UserExpressionError

Raised when user expression cannot be parsed.

OutputExistsError

Raised when output datasets already exist.

Exception

Other exceptions types may be raised by underlying registry classes.