ParslJob

class lsst.ctrl.bps.parsl.ParslJob(generic: GenericWorkflowJob, config: BpsConfig, file_paths: dict[str, str])

Bases: object

Job to execute with parsl.

Parameters:

genericGenericWorkflowJob

BPS job information.

configBpsConfig

BPS configuration.

file_pathsdict mapping str to str

File paths for job, indexed by symbolic name.

Methods Summary

evaluate_command_line(command)	Evaluate the bash command-line.
get_command_line([allow_stage])	Get the bash command-line to run to execute this job.
get_future(app, inputs[, command_prefix, ...])	Get the parsl app future for the job.
get_resources()	Return what resources are required for executing this job.
run_local()	Run the command locally.

Methods Documentation

evaluate_command_line(command: str) → str

Evaluate the bash command-line.

BPS provides a command-line with symbolic names for BPS variables, environment variables and files. Here, we replace those symbolic names with the actual values, to provide a concrete command that can be executed.

In replacing file paths, we are implicitly assuming that we are working on a shared file system, i.e., that workers can see the butler directory, and that files do not need to be staged to the worker.

Parameters:

commandstr

Command-line to execute, from BPS.

Returns:

commandstr

Command ready for execution on a worker.

get_command_line(allow_stage=True) → str

Get the bash command-line to run to execute this job.

Parameters:

allow_stagebool

Allow staging of execution butler? This is not appropriate for the initial or final jobs that run on the local nodes.

Returns:

commandstr

Command-line to execute for job.

get_future(app: BashApp, inputs: list[concurrent.futures._base.Future], command_prefix: str | None = None, add_resources: bool = False) → Future | None

Get the parsl app future for the job.

This effectively queues the job for execution by a worker, subject to dependencies.

Parameters:

appcallable

A parsl bash_app decorator to use.

inputslist of Future

Dependencies to be satisfied before executing this job.

command_prefixstr, optional

Bash commands to execute before the job command, e.g., for setting the environment.

add_resourcesbool

Add resource specification when submitting the job? This is only appropriate for the WorkQueue executor; other executors will raise an exception.

Returns:

futureFuture or None

A Future object linked to the execution of the job, or None if the job has already been done (e.g., by run_local).

get_resources() → dict[str, Any]

Return what resources are required for executing this job.

run_local()

Run the command locally.

This is intended to support jobs that should not be done by a worker.