Installation

Parsl

This plugin requires Parsl, which may be installed into the LSST stack:

mamba install --no-update-deps parsl

Parsl with monitoring support

The parsl-visualize executable is a web app for monitoring the progress of the Parsl workflow. It has extra dependencies that are (as of the time of writing), formally (but not practically) incompatible with the LSST butler (which requires sqlalchemy>=1.4). Attempting to install the parsl-with-monitoring conda package with --no-update-deps results in a failure, and without -no-update-deps results in a broken butler. The solution is a little complicated.

First, install the necessary dependencies:

mamba install --only-deps parsl-with-monitoring

This will downgrade sqlalchemy (e.g., to 1.3.24). We need to undo this in order for the LSST butler to work:

mamba upgrade sqlalchemy=1.4.36

Now we need to uninstall parsl (installed as a dependency of parsl-with-monitoring), and install its dependencies:

mamba uninstall parsl
mamba install --only-deps parsl

Now we have all the necessary dependencies but not parsl itself. To avoid conda or pip dependency problems in parsl, we install it directly from source:

git clone https://github.com/parsl/parsl.git
cd parsl
python setup.py install --user

Hopefully the next parsl release won’t have this problem.

Plugin

Clone this package from GitHub, set it up and build:

git clone https://github.com/lsst/ctrl_bps_parsl
cd ctrl_bps_parsl
setup -kr .
scons

To use this plugin, you’ll need to set it up in your environment, as you do for the rest of the LSST pipeline code:

setup -kr /path/to/ctrl_bps_parsl