Torque

class lsst.ctrl.bps.parsl.sites.Torque(*args, **kwargs)

Bases: SiteConfig

Configuration for generic Torque cluster.

This can be used directly as the site configuration for a Torque cluster by setting the BPS config, e.g.:

computeSite: torque
site:
  torque:
    class: lsst.ctrl.bps.parsl.sites.Torque
    nodes: 4
    tasks_per_node: 20
    walltime: "00:59:00"  # Note: always quote walltime in YAML

Alternatively, it can be used as a base class for Torque cluster configurations.

The following BPS configuration parameters are recognised (and required unless there is a default mentioned here, or provided by a subclass):

• queue (int): Queue for the Torque job.

• nodes (int): number of nodes for each Torque job.

• tasks_per_node (int): number of cores per node for each Torque job; by default we use all cores on the node.

• walltime (str): time limit for each Torque job.

• scheduler_options (str): text to prepend to the Torque submission script (each line usually starting with #PBS).

Methods Summary

get_executors()	Get a list of executors to be used in processing.
make_executor(label, *[, queue, nodes, ...])	Return an executor for running on a Torque cluster.
select_executor(job)	Get the label of the executor to use to execute a job.

Methods Documentation

get_executors() → list[ParslExecutor]

Get a list of executors to be used in processing.

Each executor should have a unique label.

make_executor(label: str, *, queue: str | None = None, nodes: int | None = None, tasks_per_node: int | None = None, walltime: str | None = None, mem_per_worker: float | None = None, scheduler_options: str | None = None, worker_init: str | None = None, provider_options: dict[str, Any] | None = None, executor_options: dict[str, Any] | None = None) → ParslExecutor

Return an executor for running on a Torque cluster.

Parameters

labelstr

Label for executor.

queuestr, optional

Queue for the Torque job.

nodesint, optional

Default number of nodes for each Torque job.

tasks_per_nodeint, optional

Default number of cores per node for each Torque job.

walltimestr, optional

Default time limit for each Torque job.

mem_per_workerfloat, optional

Minimum memory per worker (GB), limited by the executor.

worker_initstr, optional

Environment initiation command

scheduler_optionsstr, optional

#SBATCH directives to prepend to the Torque submission script.

provider_optionsdict, optional

Additional arguments for TorqueProvider constructor.

executor_optionsdict, optional

Additional arguments for HighThroughputExecutor constructor.

Returns

executorHighThroughputExecutor

Executor for Torque jobs.

select_executor(job: ParslJob) → str

Get the label of the executor to use to execute a job.

Parameters

jobParslJob

Job to be executed.

Returns

labelstr

Label of executor to use to execute job.