SingleQuantumExecutor¶
- class lsst.ctrl.mpexec.SingleQuantumExecutor(butler: Butler | None, taskFactory: TaskFactory, skipExistingIn: Any = None, clobberOutputs: bool = False, enableLsstDebug: bool = False, exitOnKnownError: bool = False, limited_butler_factory: Callable[[Quantum], LimitedButler] | None = None, resources: ExecutionResources | None = None, skipExisting: bool = False)¶
Bases:
QuantumExecutor
Executor class which runs one Quantum at a time.
- Parameters:
- butler
Butler
orNone
Data butler,
None
means that Quantum-backed butler should be used instead.- taskFactory
TaskFactory
Instance of a task factory.
- skipExistingIn
Any
Expressions representing the collections to search for existing output datasets. See Ordered collection searches for allowed types. This class only checks for the presence of butler output run in the list of collections. If the output run is present in the list then the quanta whose complete outputs exist in the output run will be skipped.
None
or empty string/sequence disables skipping.- clobberOutputs
bool
, optional If
True
, then outputs from a quantum that exist in output run collection will be removed prior to executing a quantum. IfskipExistingIn
contains output run, then only partial outputs from a quantum will be removed. Only used whenbutler
is notNone
.- enableLsstDebug
bool
, optional Enable debugging with
lsstDebug
facility for a task.- exitOnKnownError
bool
, optional If
True
, callsys.exit
with the appropriate exit code for special known exceptions, after printing a traceback, instead of letting the exception propagate up to calling. This is always the behavior for InvalidQuantumError.- limited_butler_factory
Callable
, optional A method that creates a
LimitedButler
instance for a given Quantum. This parameter must be defined ifbutler
isNone
. Ifbutler
is notNone
then this parameter is ignored.- resources
ExecutionResources
, optional The resources available to this quantum when executing.
- skipExisting
bool
, optional If
True
, skip quanta whose metadata datasets are already stored. UnlikeskipExistingIn
, this works with limited butlers as well as full butlers. Always set toTrue
ifskipExistingIn
matchesbutler.run
.
- butler
Methods Summary
checkExistingOutputs
(quantum, task_node, /, ...)Decide whether this quantum needs to be executed.
execute
(task_node, /, quantum)Execute single quantum.
Return execution report from last call to
execute
.initGlobals
(quantum)Initialize global state needed for task execution.
runQuantum
(task, quantum, task_node, /, ...)Execute task on a single quantum.
updatedQuantumInputs
(quantum, task_node, /, ...)Update quantum with extra information, returns a new updated Quantum.
writeMetadata
(quantum, metadata, task_node, ...)Methods Documentation
- checkExistingOutputs(quantum: Quantum, task_node: TaskDef | TaskNode, /, limited_butler: LimitedButler) bool ¶
Decide whether this quantum needs to be executed.
If only partial outputs exist then they are removed if
clobberOutputs
is True, otherwise an exception is raised.The
LimitedButler
is used for everything, and should be set toself.butler
if no separateLimitedButler
is available.- Parameters:
- quantum
Quantum
Quantum to check for existing outputs.
- task_node
TaskDef
orTaskNode
Task definition structure.
TaskDef
support is deprecated and will be removed after v27.- limited_butler
LimitedButler
Butler to use for querying and clobbering.
- quantum
- Returns:
- Raises:
- RuntimeError
Raised if some outputs exist and some not.
- execute(task_node: TaskDef | TaskNode, /, quantum: Quantum) Quantum ¶
Execute single quantum.
- Parameters:
- Returns:
- quantum
Quantum
The quantum actually executed.
- quantum
Notes
Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.
- getReport() QuantumReport | None ¶
Return execution report from last call to
execute
.- Returns:
- report
QuantumReport
Structure describing the status of the execution of a quantum.
None
is returned if implementation does not support this feature.
- report
- Raises:
- RuntimeError
Raised if this method is called before
execute
.
- initGlobals(quantum: Quantum) None ¶
Initialize global state needed for task execution.
- Parameters:
- quantum
Quantum
Single Quantum instance.
- quantum
Notes
There is an issue with initializing filters singleton which is done by instrument, to avoid requiring tasks to do it in runQuantum() we do it here when any dataId has an instrument dimension. Also for now we only allow single instrument, verify that all instrument names in all dataIds are identical.
This will need revision when filter singleton disappears.
- runQuantum(task: PipelineTask, quantum: Quantum, task_node: TaskDef | TaskNode, /, limited_butler: LimitedButler) None ¶
Execute task on a single quantum.
- Parameters:
- task
PipelineTask
Task object.
- quantum
Quantum
Single Quantum instance.
- task_node
TaskDef
orTaskNode
Task definition structure.
TaskDef
support is deprecated and will be removed after v27.- limited_butler
LimitedButler
Butler to use for dataset I/O.
- task
- updatedQuantumInputs(quantum: Quantum, task_node: TaskDef | TaskNode, /, limited_butler: LimitedButler) Quantum ¶
Update quantum with extra information, returns a new updated Quantum.
Some methods may require input DatasetRefs to have non-None
dataset_id
, but in case of intermediate dataset it may not be filled during QuantumGraph construction. This method will retrieve missing info from registry.