Overview

LSST Batch Processing Service (BPS) allow large-scale workflows to execute in well-managed fashion, potentially in multiple environments. The service is provided by ctrl_bps package. ctrl_bps_panda is a plugin allowing ctrl_bps to execute workflows on computational resources managed by PanDA.

Note

Documentation describing how to submit workflows for execution using PanDA deployment in Interim Data Facility (aka Google cloud) can can be found at https://panda.lsst.io.

Specifying the plugin

The class providing PanDA support for ctrl_bps is

lsst.ctrl.bps.htcondor.PanDAService

Inform ctrl_bps about its location using one of the methods described in its documentation.

Defining a submission

BPS configuration files are YAML files with some reserved keywords and some special features. See BPS configuration file for details.

The memory autoscaling is not supported supported by the ctrl_bps_panda, i.e., configuration settings like:

  • requestMemoryMax,

  • executeMachinesPattern,

  • memoryMultiplier,

  • memoryLimit,

will have not effect on workflows submitted with this plugin.

Authenticating

See https://panda.lsst.io for details.

Submitting a run

See bps submit and https://panda.lsst.io for details.

Checking status

bps report is not supported, use the WMS commands/tools directly.

Canceling submitted jobs

bps cancel is not supported, use the WMS commands/tools directly.

Restarting a failed run

bps restart is not supported, use the WMS commands/tools directly.