PanDAService

class lsst.ctrl.bps.panda.PanDAService(config)

Bases: BaseWmsService

PanDA version of WMS service.

Attributes Summary

defaults	Service default settings (lsst.daf.butler.Config).
defaults_uri	URI to WMS default settings (lsst.resources.ResourcePath).

Methods Summary

cancel(wms_id[, pass_thru])	Cancel submitted workflows/jobs.
list_submitted_jobs([wms_id, user, ...])	Query WMS for list of submitted WMS workflows/jobs.
ping([pass_thru])	Check whether WMS services are up, reachable, and can authenticate if authentication is required.
prepare(config, generic_workflow[, out_prefix])	Create submission for a generic workflow for a specific WMS.
report([wms_workflow_id, user, hist, ...])	Query WMS for status of submitted WMS workflows.
restart(wms_workflow_id)	Restart a workflow from the point of failure.
run_submission_checks()	Check to run at start if running WMS specific submission steps.
submit(workflow, **kwargs)	Submit a single WMS workflow.

Attributes Documentation

defaults

Service default settings (lsst.daf.butler.Config).

Notes

This property is currently being used in BpsConfig.__init__(). As long as that’s the case it cannot be changed to return a BpsConfig instance.

defaults_uri

URI to WMS default settings (lsst.resources.ResourcePath).

Methods Documentation

cancel(wms_id, pass_thru=None)

Cancel submitted workflows/jobs.

Parameters:

wms_idstr

ID or path of job that should be canceled.

pass_thrustr, optional

Information to pass through to WMS.

Returns:

deletedbool

Whether successful deletion or not. Currently, if any doubt or any individual jobs not deleted, return False.

messagestr

Any message from WMS (e.g., error details).

list_submitted_jobs(wms_id=None, user=None, require_bps=True, pass_thru=None, is_global=False)

Query WMS for list of submitted WMS workflows/jobs.

This should be a quick lookup function to create list of jobs for other functions.

Parameters:

wms_idint or str, optional

Id or path that can be used by WMS service to look up job.

userstr, optional

User whose submitted jobs should be listed.

require_bpsbool, optional

Whether to require jobs returned in list to be bps-submitted jobs.

pass_thrustr, optional

Information to pass through to WMS.

is_globalbool, optional

If set, all available job queues will be queried for job information. Defaults to False which means that only a local job queue will be queried for information.

Only applicable in the context of a WMS using distributed job queues (e.g., HTCondor). A WMS with a centralized job queue (e.g. PanDA) can safely ignore it.

Returns:

job_idslist [Any]

Only job ids to be used by cancel and other functions. Typically this means top-level jobs (i.e., not children jobs).

ping(pass_thru=None)

Check whether WMS services are up, reachable, and can authenticate if authentication is required.

The services to be checked are those needed for submit, report, cancel, restart, but ping cannot guarantee whether jobs would actually run successfully.

Parameters:

pass_thrustr, optional

Information to pass through to WMS.

Returns:

statusint

0 for success, non-zero for failure.

messagestr

Any message from WMS (e.g., error details).

prepare(config, generic_workflow, out_prefix=None)

Create submission for a generic workflow for a specific WMS.

Parameters:

configlsst.ctrl.bps.BpsConfig

BPS configuration.

generic_workflowlsst.ctrl.bps.GenericWorkflow

Generic representation of a single workflow.

out_prefixstr

Prefix for all WMS output files.

Returns:

wms_workflowlsst.ctrl.bps.BaseWmsWorkflow

Prepared WMS Workflow to submit for execution.

report(wms_workflow_id=None, user=None, hist=0, pass_thru=None, is_global=False, return_exit_codes=False)

Query WMS for status of submitted WMS workflows.

Parameters:

wms_workflow_idint or str, optional

Id that can be used by WMS service to look up status.

userstr, optional

Limit report to submissions by this particular user.

histint, optional

Number of days to expand report to include finished WMS workflows.

pass_thrustr, optional

Additional arguments to pass through to the specific WMS service.

is_globalbool, optional

If set, all available job queues will be queried for job information. Defaults to False which means that only a local job queue will be queried for information.

Only applicable in the context of a WMS using distributed job queues (e.g., HTCondor). A WMS with a centralized job queue (e.g. PanDA) can safely ignore it.

return_exit_codesbool, optional

If set, return exit codes related to jobs with a non-success status. Defaults to False, which means that only the summary state is returned.

Only applicable in the context of a WMS with associated handlers to return exit codes from jobs.

Returns:

run_reportslist [lsst.ctrl.bps.WmsRunReport]

Status information for submitted WMS workflows.

messagestr

Message to user on how to find more status information specific to this particular WMS.

restart(wms_workflow_id)

Restart a workflow from the point of failure.

Parameters:

wms_workflow_idstr

Id that can be used by WMS service to identify workflow that need to be restarted.

Returns:

wms_idstr

Id of the restarted workflow. If restart failed, it will be set to None.

run_namestr

Name of the restarted workflow. If restart failed, it will be set to None.

messagestr

A message describing any issues encountered during the restart. If there were no issue, an empty string is returned.

run_submission_checks()

Check to run at start if running WMS specific submission steps.

Any exception other than NotImplementedError will halt submission. Submit directory may not yet exist when this is called.

submit(workflow, **kwargs)

Submit a single WMS workflow.

Parameters:

workflowlsst.ctrl.bps.BaseWmsWorkflow

Prepared WMS Workflow to submit for execution.

**kwargsAny

Additional modifiers to the configuration.