TripleSlurm

class lsst.ctrl.bps.parsl.sites.TripleSlurm(*args, **kwargs)

Bases: Slurm

Configuration for running jobs on a Slurm cluster with three levels.

Parameters

*argsAny

Parameters forwarded to base class constructor.

**kwargsAny

Keyword arguments passed to base class constructor.

Notes

The three levels are useful for having workers with different amount of available memory (and this is how executors are selected, by default), though other uses are possible.

The following BPS configuration parameters are recognised, overriding the defaults:

• nodes (int): number of nodes for each Slurm job.

• cores_per_node (int): number of cores per node for each Slurm job; by default we use all cores on the node.

• walltime (str): time limit for each Slurm job; setting this would override each of the small_walltime, medium_walltime and large_walltime values.

• mem_per_node (float): memory per node for each Slurm job; by default we use whatever Slurm gives us.

• qos (str): quality of service to request for each Slurm job; by default we use whatever Slurm gives us.

• small_memory (float): memory per worker (GB) for each ‘small’ Slurm job.

• medium_memory (float): memory per worker (GB) for each ‘medium’ Slurm job.

• large_memory (float): memory per worker (GB) for each ‘large’ Slurm job.

• small_walltime (str): time limit for each ‘small’ Slurm job.

• medium_walltime (str): time limit for each ‘medium’ Slurm job.

• large_walltime (str): time limit for each ‘large’ Slurm job.

Methods Summary

get_executors([small_options, ...])	Get a list of executors to be used in processing.
select_executor(job)	Get the label of the executor to use to execute a job.

Methods Documentation

get_executors(small_options: dict[str, Any] | None = None, medium_options: dict[str, Any] | None = None, large_options: dict[str, Any] | None = None, **common_options) → list[ParslExecutor]

Get a list of executors to be used in processing.

We create three executors, with different walltime and memory per worker.

Parameters

small_optionskwargs

Options for make_executor for small executor.

medium_optionskwargs

Options for make_executor for medium executor.

large_optionskwargs

Options for make_executor for large executor.

**common_options

Common options for make_executor for each of the executors.

select_executor(job: ParslJob) → str

Get the label of the executor to use to execute a job.

This implementation only looks at the requested memory.

Parameters

jobParslJob

Job to be executed.

Returns

labelstr

Label of executor to use to execute job.