PreExecInit¶
-
class
lsst.ctrl.mpexec.
PreExecInit
(butler: Butler, taskFactory: TaskFactory, extendRun: bool = False, mock: bool = False)¶ Bases:
lsst.ctrl.mpexec.preExecInit.PreExecInitBase
Initialization of registry for QuantumGraph execution.
This class encapsulates all necessary operations that have to be performed on butler and registry to prepare them for QuantumGraph execution.
Parameters: - butler :
Butler
Data butler instance.
- taskFactory :
TaskFactory
Task factory.
- extendRun :
bool
, optional If
True
then do not try to overwrite any datasets that might exist inbutler.run
; instead compare them when appropriate/possible. IfFalse
, then any existing conflicting dataset will cause a butler exception to be raised.- mock :
bool
, optional If
True
then also do initialization needed for pipeline mocking.
Methods Summary
find_init_input_refs
(taskDef, graph)Return the list of resolved dataset references for task init inputs. find_init_output
(taskDef, dataset_type_name, …)Find task init output for given dataset type. find_packages
(graph)Find packages information. initialize
(graph, saveInitOutputs, …)Perform all initialization steps. initializeDatasetTypes
(graph, …)Save or check DatasetTypes output by the tasks in a graph. saveConfigs
(graph)Write configurations for pipeline tasks to butler or check that existing configurations are equal to the new ones. saveInitOutputs
(graph)Write any datasets produced by initializing tasks in a graph. savePackageVersions
(graph)Write versions of software packages to butler. transaction
()Context manager for transaction. Methods Documentation
-
find_init_input_refs
(taskDef: TaskDef, graph: QuantumGraph) → Iterable[DatasetRef]¶ Return the list of resolved dataset references for task init inputs.
Parameters: - taskDef :
TaskDef
Pipeline task definition.
- graph :
QuantumGraph
Quantum graph.
Returns: - refs :
Iterable
[DatasetRef
] Resolved dataset references.
- taskDef :
-
find_init_output
(taskDef: TaskDef, dataset_type_name: str, graph: QuantumGraph) → tuple[Any | None, DatasetRef]¶ Find task init output for given dataset type.
Parameters: - taskDef :
TaskDef
Pipeline task definition.
- dataset_type :
str
Dataset type name.
- graph :
QuantumGraph
Quantum graph.
Returns: - data
Existing init output object retrieved from butler,
None
if butler has no existing object.- ref :
DatasetRef
Resolved reference for init output to be stored in butler.
Raises: - MissingReferenceError
Raised if reference cannot be found or generated.
- taskDef :
-
find_packages
(graph: QuantumGraph) → tuple[Packages | None, DatasetRef]¶ Find packages information.
Parameters: - graph :
QuantumGraph
Quantum graph.
Returns: - packages :
lsst.utils.packages.Packages
orNone
Existing packages data retrieved from butler, or
None
.- ref :
DatasetRef
Resolved reference for packages to be stored in butler.
Raises: - MissingReferenceError
Raised if reference cannot be found or generated.
- graph :
-
initialize
(graph: QuantumGraph, saveInitOutputs: bool = True, registerDatasetTypes: bool = False, saveVersions: bool = True) → None¶ Perform all initialization steps.
Convenience method to execute all initialization steps. Instead of calling this method and providing all options it is also possible to call methods individually.
Parameters: - graph :
QuantumGraph
Execution graph.
- saveInitOutputs :
bool
, optional If
True
(default) then save “init outputs”, configurations, and package versions to butler.- registerDatasetTypes :
bool
, optional If
True
then register dataset types in registry, otherwise they must be already registered.- saveVersions :
bool
, optional If
False
then do not save package versions even ifsaveInitOutputs
is set toTrue
.
- graph :
-
initializeDatasetTypes
(graph: QuantumGraph, registerDatasetTypes: bool = False) → None¶ Save or check DatasetTypes output by the tasks in a graph.
Iterates over all DatasetTypes for all tasks in a graph and either tries to add them to registry or compares them to existing ones.
Parameters: - graph :
QuantumGraph
Execution graph.
- registerDatasetTypes :
bool
, optional If
True
then register dataset types in registry, otherwise they must be already registered.
Raises: - ValueError
Raised if existing DatasetType is different from DatasetType in a graph.
- KeyError
Raised if
registerDatasetTypes
isFalse
and DatasetType does not exist in registry.
- graph :
-
saveConfigs
(graph: QuantumGraph) → None¶ Write configurations for pipeline tasks to butler or check that existing configurations are equal to the new ones.
Parameters: - graph :
QuantumGraph
Execution graph.
Raises: - TypeError
Raised if existing object in butler is different from new data.
- Exception
Raised if
extendRun
isFalse
and datasets already exists. Content of a butler collection should not be changed if exception is raised.
- graph :
-
saveInitOutputs
(graph: QuantumGraph) → None¶ Write any datasets produced by initializing tasks in a graph.
Parameters: - graph :
QuantumGraph
Execution graph.
Raises: - TypeError
Raised if the type of existing object in butler is different from new data.
- graph :
-
savePackageVersions
(graph: QuantumGraph) → None¶ Write versions of software packages to butler.
Parameters: - graph :
QuantumGraph
Execution graph.
Raises: - TypeError
Raised if existing object in butler is incompatible with new data.
- graph :
-
transaction
() → collections.abc.Iterator[None]¶ Context manager for transaction.
Default implementation has no transaction support.
- butler :