MPGraphExecutor

class lsst.ctrl.mpexec.MPGraphExecutor(numProc: int, timeout: float, quantumExecutor: lsst.ctrl.mpexec.quantumGraphExecutor.QuantumExecutor, *, startMethod: Union[Literal[spawn], Literal[fork], Literal[forkserver], None] = None, failFast: bool = False, pdb: Optional[str, None] = None, executionGraphFixup: Optional[lsst.ctrl.mpexec.executionGraphFixup.ExecutionGraphFixup, None] = None)

Bases: lsst.ctrl.mpexec.QuantumGraphExecutor

Implementation of QuantumGraphExecutor using same-host multiprocess execution of Quanta.

Parameters:

numProc : int

Number of processes to use for executing tasks.

timeout : float

Time in seconds to wait for tasks to finish.

quantumExecutor : QuantumExecutor

Executor for single quantum. For multiprocess-style execution when numProc is greater than one this instance must support pickle.

startMethod : str, optional

Start method from multiprocessing module, None selects the best one for current platform.

failFast : bool, optional

If set to True then stop processing on first error from any task.

pdb : str, optional

Debugger to import and use (via the post_mortem function) in the event of an exception.

executionGraphFixup : ExecutionGraphFixup, optional

Instance used for modification of execution graph.

Methods Summary

execute(graph)	Execute whole graph.
getReport()	Return execution report from last call to execute.

Methods Documentation

execute(graph: lsst.pipe.base.graph.graph.QuantumGraph) → None

Execute whole graph.

Implementation of this method depends on particular execution model and it has to be provided by a subclass. Execution model determines what happens here; it can be either actual running of the task or, for example, generation of the scripts for delayed batch execution.

Parameters:

graph : QuantumGraph

Execution graph.

getReport() → Optional[lsst.ctrl.mpexec.reports.Report, None]

Return execution report from last call to execute.

Returns:

report : Report, optional

Structure describing the status of the execution of a quantum graph. None is returned if implementation does not support this feature.

Raises:

RuntimeError

Raised if this method is called before execute.