Ccin2p3#

class lsst.ctrl.bps.parsl.sites.Ccin2p3(*args, **kwargs)#

Bases: SiteConfig

Configuration for executing Parsl jobs in CC-IN2P3 Slurm batch farm.

This class provides four job slot sizes each with its specific requirements, in particular in terms of memory. Those slot sizes are named “small”, “medium”, “large” and “xlarge”.

Sensible default values for those requirements are provided for each job slot but you can overwrite those defaults either in the the BPS submission file or in a site configuration file that you include in your BPS submission file.

If you don’t need to modify the default requirements for the job slot sizes, use the site specification below in your BPS configuration file:

wmsServiceClass: lsst.ctrl.bps.parsl.ParslService
computeSite: ccin2p3

site:
  ccin2p3:
    class: lsst.ctrl.bps.parsl.sites.ccin2p3.Ccin2p3

If you do need to modify those defaults, you can overwrite them for all job slots or for specific each job slots. Requirements specified for a job slot take priority over those specified for all job slots at the level of entry ‘.site.ccin2p3:’.

This is an example of how to overwrite selected requirements in your BPS submission file:

wmsServiceClass: lsst.ctrl.bps.parsl.ParslService
computeSite: ccin2p3

site:
  ccin2p3:
    class: lsst.ctrl.bps.parsl.sites.ccin2p3.Ccin2p3
    walltime: "72:00:00"
    scheduler_options:
      - "--licenses=sps"
      - "--qos=normal"
    small:
      memory: 6
      partition: "flash"
    medium:
      memory: 10
      partition: "lsst,htc"
    large:
      memory: 80
    xlarge:
      memory: 180
      partition: "lsst"
      scheduler_options:
        - "--constraint=el7"
        - "--licenses=my_product"
        - "--reservation=my_reservation"

At the level of entry ‘site.ccin2p3:’ in the BPS submission file, the following configuration parameters are accepted, which apply to all slot sizes:

partition (str): name of the one or more configured partitions. If
more than one, separate them with comma (‘,’). (Default: “lsst,htc”)
walltime (str): walltime to require for the job (Default: “72:00:00”)
scheduler_options (list [str] ): scheduler options to send to Slurm
for scheduling purposes. (Default: “–licenses=sps”)

In addition, as shown in the previous example, for each job slot (i.e. “small”, “medium”, etc.) you can specify the requirements above as well as the following:

max_blocks (int): maximum number of Slurm jobs that your workflow can
simultaneously use.
memory (int): required amount of memory for each job, in Gigabytes.
(Defaults: 4 for “small”, 10 for “medium”, 50 fo “large” and 150 for “xlarge”).

Parameters#

*argsoptional: Arguments to initialize the super-class.
**kwargsoptional: Keyword arguments to initialize the super-class.

Returns#

Ccin2p3SiteConfig: Concrete instance of a SiteConfig specific for the CC-IN2P3 Slurm farm.

Attributes Summary

`DEFAULT_ACCOUNT`
`DEFAULT_SCHEDULER_OPTIONS`
`DEFAULT_WALLTIME`

Methods Summary

`get_executors`()	Get a list of Parsl executors that can be used for processing a workflow.
`get_parsl_config`()	Get Parsl configuration for using CC-IN2P3 Slurm farm as a Parsl execution site.
`select_executor`(job)	Get the `label` of the executor to use to execute `job`.

Attributes Documentation

DEFAULT_ACCOUNT: str = 'lsst'#

DEFAULT_SCHEDULER_OPTIONS: list[str] = ['--licenses=sps']#

DEFAULT_WALLTIME: str = '72:00:00'#

Methods Documentation

get_executors() → list[ParslExecutor]#

Get a list of Parsl executors that can be used for processing a workflow.

Each executor must have a unique label.

get_parsl_config() → Config#

Get Parsl configuration for using CC-IN2P3 Slurm farm as a Parsl execution site.

Returns#

configparsl.config.Config: The configuration to be used to initialize Parsl for this site.

select_executor(job: ParslJob) → str#

Get the label of the executor to use to execute job.

Parameters#

jobParslJob: Job to be executed.

Returns#

labelstr: Label of executor to use to execute job.