SingleQuantumExecutor

class lsst.ctrl.mpexec.SingleQuantumExecutor(taskFactory, skipExisting=False, clobberPartialOutputs=False, enableLsstDebug=False)

Bases: lsst.ctrl.mpexec.QuantumExecutor

Executor class which runs one Quantum at a time.

Parameters

butlerButler

Data butler.

taskFactoryTaskFactory

Instance of a task factory.

skipExistingbool, optional

If True then quanta with all existing outputs are not executed.

clobberPartialOutputsbool, optional

If True then delete any partial outputs from quantum execution. If complete outputs exists then exception is raise if skipExisting is False.

enableLsstDebugbool, optional

Enable debugging with lsstDebug facility for a task.

Methods Summary

checkExistingOutputs(quantum, butler, taskDef)	Decide whether this quantum needs to be executed.
execute(taskDef, quantum, butler)	Execute single quantum.
initGlobals(quantum, butler)	Initialize global state needed for task execution.
makeTask(taskClass, config, butler)	Make new task instance.
runQuantum(task, quantum, taskDef, butler)	Execute task on a single quantum.
setupLogging(taskDef, quantum)	Configure logging system for execution of this task.
updatedQuantumInputs(quantum, butler)	Update quantum with extra information, returns a new updated Quantum.

Methods Documentation

checkExistingOutputs(quantum, butler, taskDef)

Decide whether this quantum needs to be executed.

If only partial outputs exist then they are removed if clobberPartialOutputs is True, otherwise an exception is raised.

Parameters

quantumQuantum

Quantum to check for existing outputs

butlerButler

Data butler.

taskDefTaskDef

Task definition structure.

Returns

existbool

True if all quantum’s outputs exist in a collection and skipExisting is True, False otherwise.

Raises

RuntimeError

Raised if some outputs exist and some not.

execute(taskDef, quantum, butler)

Execute single quantum.

Parameters

taskDefTaskDef

Task definition structure.

quantumQuantum

Quantum for this execution.

butlerButler

Data butler instance

Notes

Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.

initGlobals(quantum, butler)

Initialize global state needed for task execution.

Parameters

quantumQuantum

Single Quantum instance.

butlerButler

Data butler.

Notes

There is an issue with initializing filters singleton which is done by instrument, to avoid requiring tasks to do it in runQuantum() we do it here when any dataId has an instrument dimension. Also for now we only allow single instrument, verify that all instrument names in all dataIds are identical.

This will need revision when filter singleton disappears.

makeTask(taskClass, config, butler)

Make new task instance.

Parameters

taskClasstype

Sub-class of PipelineTask.

configPipelineTaskConfig

Configuration object for this task

Returns

taskPipelineTask

Instance of taskClass type.

butlerButler

Data butler.

runQuantum(task, quantum, taskDef, butler)

Execute task on a single quantum.

Parameters

taskPipelineTask

Task object.

quantumQuantum

Single Quantum instance.

taskDefTaskDef

Task definition structure.

butlerButler

Data butler.

setupLogging(taskDef, quantum)

Configure logging system for execution of this task.

Ths method can setup logging to attach task- or quantum-specific information to log messages. Potentially this can take into account some info from task configuration as well.

Parameters

taskDeflsst.pipe.base.TaskDef

The task definition.

quantumQuantum

Single Quantum instance.

updatedQuantumInputs(quantum, butler)

Update quantum with extra information, returns a new updated Quantum.

Some methods may require input DatasetRefs to have non-None dataset_id, but in case of intermediate dataset it may not be filled during QuantumGraph construction. This method will retrieve missing info from registry.

Parameters

quantumQuantum

Single Quantum instance.

butlerButler

Data butler.

Returns

updateQuantum

Updated Quantum instance