SingleQuantumExecutor

class lsst.ctrl.mpexec.SingleQuantumExecutor(taskFactory, skipExisting=False, clobberPartialOutputs=False, enableLsstDebug=False)

Bases: lsst.ctrl.mpexec.QuantumExecutor

Executor class which runs one Quantum at a time.

Parameters:

butler : Butler

Data butler.

taskFactory : TaskFactory

Instance of a task factory.

skipExisting : bool, optional

If True then quanta with all existing outputs are not executed.

clobberPartialOutputs : bool, optional

If True then delete any partial outputs from quantum execution. If complete outputs exists then exception is raise if skipExisting is False.

enableLsstDebug : bool, optional

Enable debugging with lsstDebug facility for a task.

Methods Summary

checkExistingOutputs(quantum, butler, taskDef)	Decide whether this quantum needs to be executed.
execute(taskDef, quantum, butler)	Execute single quantum.
initGlobals(quantum, butler)	Initialize global state needed for task execution.
makeTask(taskClass, config, butler)	Make new task instance.
runQuantum(task, quantum, taskDef, butler)	Execute task on a single quantum.
setupLogging(taskDef, quantum)	Configure logging system for execution of this task.
updatedQuantumInputs(quantum, butler)	Update quantum with extra information, returns a new updated Quantum.

Methods Documentation

checkExistingOutputs(quantum, butler, taskDef)

Decide whether this quantum needs to be executed.

If only partial outputs exist then they are removed if clobberPartialOutputs is True, otherwise an exception is raised.

Parameters:

quantum : Quantum

Quantum to check for existing outputs

butler : Butler

Data butler.

taskDef : TaskDef

Task definition structure.

Returns:

exist : bool

True if all quantum’s outputs exist in a collection and skipExisting is True, False otherwise.

Raises:

RuntimeError

Raised if some outputs exist and some not.

execute(taskDef, quantum, butler)

Execute single quantum.

Parameters:

taskDef : TaskDef

Task definition structure.

quantum : Quantum

Quantum for this execution.

butler : Butler

Data butler instance

Notes

Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.

initGlobals(quantum, butler)

Initialize global state needed for task execution.

Parameters:

quantum : Quantum

Single Quantum instance.

butler : Butler

Data butler.

Notes

There is an issue with initializing filters singleton which is done by instrument, to avoid requiring tasks to do it in runQuantum() we do it here when any dataId has an instrument dimension. Also for now we only allow single instrument, verify that all instrument names in all dataIds are identical.

This will need revision when filter singleton disappears.

makeTask(taskClass, config, butler)

Make new task instance.

Parameters:

taskClass : type

Sub-class of PipelineTask.

config : PipelineTaskConfig

Configuration object for this task

Returns:

task : PipelineTask

Instance of taskClass type.

butler : Butler

Data butler.

runQuantum(task, quantum, taskDef, butler)

Execute task on a single quantum.

Parameters:

task : PipelineTask

Task object.

quantum : Quantum

Single Quantum instance.

taskDef : TaskDef

Task definition structure.

butler : Butler

Data butler.

setupLogging(taskDef, quantum)

Configure logging system for execution of this task.

Ths method can setup logging to attach task- or quantum-specific information to log messages. Potentially this can take into account some info from task configuration as well.

Parameters:

taskDef : lsst.pipe.base.TaskDef

The task definition.

quantum : Quantum

Single Quantum instance.

updatedQuantumInputs(quantum, butler)

Update quantum with extra information, returns a new updated Quantum.

Some methods may require input DatasetRefs to have non-None dataset_id, but in case of intermediate dataset it may not be filled during QuantumGraph construction. This method will retrieve missing info from registry.

Parameters:

quantum : Quantum

Single Quantum instance.

butler : Butler

Data butler.

Returns:

update : Quantum

Updated Quantum instance