SingleQuantumExecutor

class lsst.ctrl.mpexec.SingleQuantumExecutor(taskFactory, skipExisting=False, clobberOutputs=False, enableLsstDebug=False, exitOnKnownError=False)

Bases: lsst.ctrl.mpexec.QuantumExecutor

Executor class which runs one Quantum at a time.

Parameters:

butler : Butler

Data butler.

taskFactory : TaskFactory

Instance of a task factory.

skipExisting : bool, optional

If True, then quanta that succeeded will not be rerun.

clobberOutputs : bool, optional

If True, then existing outputs will be overwritten. If skipExisting is also True, only outputs from failed quanta will be overwritten.

enableLsstDebug : bool, optional

Enable debugging with lsstDebug facility for a task.

exitOnKnownError : bool, optional

If True, call sys.exit with the appropriate exit code for special known exceptions, after printing a traceback, instead of letting the exception propagate up to calling. This is always the behavior for InvalidQuantumError.

Methods Summary

checkExistingOutputs(quantum, butler, taskDef)	Decide whether this quantum needs to be executed.
execute(taskDef, quantum, butler)	Execute single quantum.
initGlobals(quantum, butler)	Initialize global state needed for task execution.
makeTask(taskClass, name, config, butler)	Make new task instance.
runQuantum(task, quantum, taskDef, butler)	Execute task on a single quantum.
setupLogging(taskDef, quantum)	Configure logging system for execution of this task.
updatedQuantumInputs(quantum, butler, taskDef)	Update quantum with extra information, returns a new updated Quantum.
writeMetadata(quantum, metadata, taskDef, butler)

Methods Documentation

checkExistingOutputs(quantum, butler, taskDef)

Decide whether this quantum needs to be executed.

If only partial outputs exist then they are removed if clobberOutputs is True, otherwise an exception is raised.

Parameters:

quantum : Quantum

Quantum to check for existing outputs

butler : Butler

Data butler.

taskDef : TaskDef

Task definition structure.

Returns:

exist : bool

True if self.skipExisting is True, and a previous execution of this quanta appears to have completed successfully (either because metadata was written or all datasets were written). False otherwise.

Raises:

RuntimeError

Raised if some outputs exist and some not.

execute(taskDef, quantum, butler)

Execute single quantum.

Parameters:

taskDef : TaskDef

Task definition structure.

quantum : Quantum

Quantum for this execution.

butler : Butler

Data butler instance

Notes

Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.

initGlobals(quantum, butler)

Initialize global state needed for task execution.

Parameters:

quantum : Quantum

Single Quantum instance.

butler : Butler

Data butler.

Notes

There is an issue with initializing filters singleton which is done by instrument, to avoid requiring tasks to do it in runQuantum() we do it here when any dataId has an instrument dimension. Also for now we only allow single instrument, verify that all instrument names in all dataIds are identical.

This will need revision when filter singleton disappears.

makeTask(taskClass, name, config, butler)

Make new task instance.

Parameters:

taskClass : type

Sub-class of PipelineTask.

name : str

Name for this task.

config : PipelineTaskConfig

Configuration object for this task

Returns:

task : PipelineTask

Instance of taskClass type.

butler : Butler

Data butler.

runQuantum(task, quantum, taskDef, butler)

Execute task on a single quantum.

Parameters:

task : PipelineTask

Task object.

quantum : Quantum

Single Quantum instance.

taskDef : TaskDef

Task definition structure.

butler : Butler

Data butler.

setupLogging(taskDef, quantum)

Configure logging system for execution of this task.

Ths method can setup logging to attach task- or quantum-specific information to log messages. Potentially this can take into account some info from task configuration as well.

Parameters:

taskDef : lsst.pipe.base.TaskDef

The task definition.

quantum : Quantum

Single Quantum instance.

updatedQuantumInputs(quantum, butler, taskDef)

Update quantum with extra information, returns a new updated Quantum.

Some methods may require input DatasetRefs to have non-None dataset_id, but in case of intermediate dataset it may not be filled during QuantumGraph construction. This method will retrieve missing info from registry.

Parameters:

quantum : Quantum

Single Quantum instance.

butler : Butler

Data butler.

taskDef : TaskDef

Task definition structure.

Returns:

update : Quantum

Updated Quantum instance

writeMetadata(quantum, metadata, taskDef, butler)