SingleQuantumExecutor

class lsst.pipe.base.single_quantum_executor.SingleQuantumExecutor(*, butler: Butler | None = None, task_factory: TaskFactory | None = None, skip_existing_in: Any = None, clobber_outputs: bool = False, enable_lsst_debug: bool = False, limited_butler_factory: Callable[[Quantum], LimitedButler] | None = None, resources: ExecutionResources | None = None, skip_existing: bool = False, assume_no_existing_outputs: bool = False, raise_on_partial_outputs: bool = True, job_metadata: Mapping[str, int | str | float] | None = None)

Bases: QuantumExecutor

Executor class which runs one Quantum at a time.

Parameters:

butlerButler or None, optional

Data butler, None means that a limited butler should be used instead.

task_factoryTaskFactory, optional

Instance of a task factory. Defaults to a new instance of lsst.pipe.base.TaskFactory.

skip_existing_instr or Iterable [ str ]

A collection name or list of collections to search for the existing outputs of quanta, which indicates that those quanta should be skipped. This class only checks for the presence of butler output run in the list of collections. If the output run is present in the list then the quanta whose complete outputs exist in the output run will be skipped. None or empty string/sequence disables skipping.

clobber_outputsbool, optional

If True, then outputs from a quantum that exist in output run collection will be removed prior to executing a quantum. If skip_existing_in contains output run, then only partial outputs from a quantum will be removed. Only used when butler is not None.

enable_lsst_debugbool, optional

Enable debugging with lsstDebug facility for a task.

limited_butler_factoryCallable, optional

A method that creates a LimitedButler instance for a given Quantum. This parameter must be defined if butler is None. If butler is not None then this parameter is ignored.

resourcesExecutionResources, optional

The resources available to this quantum when executing.

skip_existingbool, optional

If True, skip quanta whose metadata datasets are already stored. Unlike skip_existing_in, this works with limited butlers as well as full butlers. Always set to True if skip_existing_in matches butler.run.

assume_no_existing_outputsbool, optional

If True, assume preexisting outputs are impossible (e.g. because this is known by higher-level code to be a new RUN collection), and do not look for them. This causes the skip_existing and clobber_outputs options to be ignored, but unlike just setting both of those to False, it also avoids all dataset existence checks.

raise_on_partial_outputsbool, optional

If True raise exceptions chained by AnnotatedPartialOutputsError immediately, instead of considering the partial result a success and continuing to run downstream tasks.

job_metadataMapping

Mapping with extra metadata to embed within the quantum metadata under the “job” key. This is intended to correspond to information common to all quanta being executed in a single process, such as the time taken to load the quantum graph in a BPS job.

Methods Summary

execute(task_node, /, quantum[, quantum_id, ...])

Execute single quantum.

Methods Documentation

execute(task_node: TaskNode, /, quantum: Quantum, quantum_id: UUID | None = None, *, log_records: ButlerLogRecords | None = None) → QuantumExecutionResult

Execute single quantum.

Parameters:

task_nodeTaskNode

Task definition structure.

quantumQuantum

Quantum for this execution.

quantum_iduuid.UUID or None, optional

The ID of the quantum to be executed.

log_recordslsst.daf.butler.ButlerLogRecords, optional

Container that should be used to store logs in memory before writing them to the butler. This disables streaming log (since we’d have to store them in memory anyway), but it permits the caller to prepend logs to be stored in the butler and allows task logs to be inspected by the caller after execution is complete.

Returns:

resultQuantumExecutionResult

Result struct. May also be unpacked as a 2-tuple (see type documentation).

Notes

Any exception raised by the task or code that wraps task execution is propagated to the caller of this method.