MPGraphExecutor¶
- class lsst.pipe.base.mp_graph_executor.MPGraphExecutor(*, num_proc: int, timeout: float, quantum_executor: QuantumExecutor, start_method: Literal['spawn'] | Literal['forkserver'] | None = None, fail_fast: bool = False, pdb: str | None = None, execution_graph_fixup: ExecutionGraphFixup | None = None)¶
Bases:
QuantumGraphExecutor
Implementation of QuantumGraphExecutor using same-host multiprocess execution of Quanta.
- Parameters:
- num_proc
int
Number of processes to use for executing tasks.
- timeout
float
Time in seconds to wait for tasks to finish.
- quantum_executor
quantum_graph_executor.QuantumExecutor
Executor for single quantum. For multiprocess-style execution when
num_proc
is greater than one this instance must support pickle.- start_method
str
, optional Start method from
multiprocessing
module,None
selects the best one for current platform.- fail_fast
bool
, optional If set to
True
then stop processing on first error from any task.- pdb
str
, optional Debugger to import and use (via the
post_mortem
function) in the event of an exception.- execution_graph_fixup
execution_graph_fixup.ExecutionGraphFixup
, optional Instance used for modification of execution graph.
- num_proc
Methods Summary
execute
(graph)Execute whole graph.
Return execution report from last call to
execute
.Methods Documentation
- execute(graph: QuantumGraph) None ¶
Execute whole graph.
Implementation of this method depends on particular execution model and it has to be provided by a subclass. Execution model determines what happens here; it can be either actual running of the task or, for example, generation of the scripts for delayed batch execution.
- Parameters:
- graph
QuantumGraph
Execution graph.
- graph