SingleQuantumExecutor

class lsst.ctrl.mpexec.SingleQuantumExecutor(butler: Butler | None, taskFactory: TaskFactory, skipExistingIn: Any = None, clobberOutputs: bool = False, enableLsstDebug: bool = False, exitOnKnownError: bool = False, limited_butler_factory: Callable[[Quantum], LimitedButler] | None = None, resources: ExecutionResources | None = None)

Bases: QuantumExecutor

Executor class which runs one Quantum at a time.

Parameters:

butlerButler or None

Data butler, None means that Quantum-backed butler should be used instead.

taskFactoryTaskFactory

Instance of a task factory.

skipExistingInAny

Expressions representing the collections to search for existing output datasets. See Ordered collection searches for allowed types. This class only checks for the presence of butler output run in the list of collections. If the output run is present in the list then the quanta whose complete outputs exist in the output run will be skipped. None or empty string/sequence disables skipping.

clobberOutputsbool, optional

If True, then outputs from a quantum that exist in output run collection will be removed prior to executing a quantum. If skipExistingIn contains output run, then only partial outputs from a quantum will be removed. Only used when butler is not None.

enableLsstDebugbool, optional

Enable debugging with lsstDebug facility for a task.

exitOnKnownErrorbool, optional

If True, call sys.exit with the appropriate exit code for special known exceptions, after printing a traceback, instead of letting the exception propagate up to calling. This is always the behavior for InvalidQuantumError.

limited_butler_factoryCallable, optional

A method that creates a LimitedButler instance for a given Quantum. This parameter must be defined if butler is None. If butler is not None then this parameter is ignored.

resourcesExecutionResources, optional

The resources available to this quantum when executing.

Methods Summary

checkExistingOutputs(quantum, taskDef, ...)	Decide whether this quantum needs to be executed.
execute(taskDef, quantum)	Execute single quantum.
getReport()	Return execution report from last call to execute.
initGlobals(quantum)	Initialize global state needed for task execution.
runQuantum(task, quantum, taskDef, ...)	Execute task on a single quantum.
updatedQuantumInputs(quantum, taskDef, ...)	Update quantum with extra information, returns a new updated Quantum.
writeMetadata(quantum, metadata, taskDef, ...)

Methods Documentation

checkExistingOutputs(quantum: Quantum, taskDef: TaskDef, limited_butler: LimitedButler) → bool

Decide whether this quantum needs to be executed.

If only partial outputs exist then they are removed if clobberOutputs is True, otherwise an exception is raised.

The LimitedButler is used for everything, and should be set to self.butler if no separate LimitedButler is available.

Parameters:

quantumQuantum

Quantum to check for existing outputs.

taskDefTaskDef

Task definition structure.

limited_butlerLimitedButler

Butler to use for querying.

Returns:

existbool

True if self.skipExisting is defined, and a previous execution of this quanta appears to have completed successfully (either because metadata was written or all datasets were written). False otherwise.

Raises:

RuntimeError

Raised if some outputs exist and some not.

execute(taskDef: TaskDef, quantum: Quantum) → Quantum

Execute single quantum.

Parameters:

taskDefTaskDef

Task definition structure.

quantumQuantum

Quantum for this execution.

Returns:

quantumQuantum

The quantum actually executed.

Notes

Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.

getReport() → QuantumReport | None

Return execution report from last call to execute.

Returns:

reportQuantumReport

Structure describing the status of the execution of a quantum. None is returned if implementation does not support this feature.

Raises:

RuntimeError

Raised if this method is called before execute.

initGlobals(quantum: Quantum) → None

Initialize global state needed for task execution.

Parameters:

quantumQuantum

Single Quantum instance.

Notes

There is an issue with initializing filters singleton which is done by instrument, to avoid requiring tasks to do it in runQuantum() we do it here when any dataId has an instrument dimension. Also for now we only allow single instrument, verify that all instrument names in all dataIds are identical.

This will need revision when filter singleton disappears.

runQuantum(task: PipelineTask, quantum: Quantum, taskDef: TaskDef, limited_butler: LimitedButler) → None

Execute task on a single quantum.

Parameters:

taskPipelineTask

Task object.

quantumQuantum

Single Quantum instance.

taskDefTaskDef

Task definition structure.

limited_butlerLimitedButler

Butler to use for dataset I/O.

updatedQuantumInputs(quantum: Quantum, taskDef: TaskDef, limited_butler: LimitedButler) → Quantum

Update quantum with extra information, returns a new updated Quantum.

Some methods may require input DatasetRefs to have non-None dataset_id, but in case of intermediate dataset it may not be filled during QuantumGraph construction. This method will retrieve missing info from registry.

Parameters:

quantumQuantum

Single Quantum instance.

taskDefTaskDef

Task definition structure.

limited_butlerLimitedButler

Butler to use for querying.

Returns:

updateQuantum

Updated Quantum instance.

writeMetadata(quantum: Quantum, metadata: Any, taskDef: TaskDef, limited_butler: LimitedButler) → None