MPGraphExecutor

class lsst.pipe.base.mp_graph_executor.MPGraphExecutor(*, num_proc: int, timeout: float, quantum_executor: QuantumExecutor, start_method: Literal['spawn'] | Literal['forkserver'] | None = None, fail_fast: bool = False, pdb: str | None = None, execution_graph_fixup: ExecutionGraphFixup | None = None)

Bases: QuantumGraphExecutor

Implementation of QuantumGraphExecutor using same-host multiprocess execution of Quanta.

Parameters

num_procint

Number of processes to use for executing tasks.

timeoutfloat

Time in seconds to wait for tasks to finish.

quantum_executorquantum_graph_executor.QuantumExecutor

Executor for single quantum. For multiprocess-style execution when num_proc is greater than one this instance must support pickle.

start_methodstr, optional

Start method from multiprocessing module, None selects the best one for current platform.

fail_fastbool, optional

If set to True then stop processing on first error from any task.

pdbstr, optional

Debugger to import and use (via the post_mortem function) in the event of an exception.

execution_graph_fixupexecution_graph_fixup.ExecutionGraphFixup, optional

Instance used for modification of execution graph.

Methods Summary

execute(graph)	Execute whole graph.
getReport()	Return execution report from last call to execute.

Methods Documentation

execute(graph: QuantumGraph | PredictedQuantumGraph) → None

Execute whole graph.

Implementation of this method depends on particular execution model and it has to be provided by a subclass. Execution model determines what happens here; it can be either actual running of the task or, for example, generation of the scripts for delayed batch execution.

Parameters

graphQuantumGraph

Execution graph.

getReport() → Report

Return execution report from last call to execute.

Returns

reportReport, optional

Structure describing the status of the execution of a quantum graph. None is returned if implementation does not support this feature.

Raises

RuntimeError

Raised if this method is called before execute.