lsst-ctrl-bps-htcondor v30.0.12 (2026-10-27)#

New Features#

  • Migrated htcondor usage to htcondor2, the next-generation Python package for HTCondor.

    Added support for HTCondor 24.x and 25.x, which only implements htcondor2. Existing support for HTCondor 24.0.x implements htcondor2 as a feature preview. Support for a future HTCondor 26.x is not declared at this time. (DM-53098)

  • Added capabilities needed to run submit processes as batch jobs on same cluster with shared filesystems. This included handling of lazy subdags. (DM-53494)

API Changes#

  • When reading JobCompletion events, htcondor2 does not parse a “ToE” (Ticket of Execution) tag from the event. The associated handler for these events no longer differentiates between events with and without a ToE tag and operates in a single mode (no ToE tag).

    Because the SecMan class is deprecated and removed in htcondor2, the legacy 24.0.x client does not have access to a ping method in the htcondor2 package. In this one case, the older htcondor package is loaded so the SecMan.ping method can be used.

    In all other cases, behaviors have been migrated and/or refactored to htcondor2 standards. (DM-53098)

Bug Fixes#

  • Fixed bug when setting an environment variable in the HTCondor submit description’s environment when the value included an environment variable. (DM-55998)

  • Fixed a bug in prepare_utils._translate_command_line that caused submitcmd to throw an exception when given 0 job arguments. (DM-56065)

Other Changes and Additions#

  • Updated the local submissions to handle the case when the given executable name is not an absolute path. It uses the submit-side location of the executable or prints a warning that the job may be held. This was triggered by ctrl_bps default yaml definitions no longer using *_DIR environment variables; instead they now rely on the PATH. (DM-51000)

  • Library versions for development and packaging builds are now managed using uv.lock instead of requirements.txt.

    Development environments are now managed by uv, with uv sync sufficient to create a virtual environment for local development. Dependency specifications have been consolidated in the pyproject.toml file. LSST dependencies are installed from VCS sources according to the override-dependencies section.

    Github Actions configurations have been updated to use uv tooling for test setup and execution. Where possible, Github Actions toolings have been pinned to commit SHAs.

    The redundant and misconfigured docstyle.yaml Github Actions workflow has been removed.

    Dependabot configuration has been extended to include the uv package ecosystem. (DM-53098)

  • Added the capability to condor_search to skip checking history if found information in the queue. The skipping is enabled by a new optional parameter. (DM-56633)

lsst-ctrl-bps-htcondor v30.0.10 (2026-07-20)#

Bug Fixes#

    • Fixed bug causing bps report to emit confusing error due to mixing information specific to a condor_dagman job with the information related to its jobs.

    • Fixed the issue with bps report providing incorrect job counts after restarts. (DM-54540)

lsst-ctrl-bps-htcondor v30.0.9 (2026-07-15)#

Bug Fixes#

  • Preserved nodeset value when creating wms_check_status jobs. (DM-54548)

  • Added signal handling and resilience to default provisioningScript. (DM-55434)

Other Changes and Additions#

  • Added dagman.conf value to DAGMAN_MAX_JOBS_IDLE workaround. (DM-55194)

lsst-ctrl-bps-htcondor v30.0.6 (2026-04-07)#

New Features#

  • Implemented wmsAttemptNum when specified in the command arguments or environment values. (DM-53292)

Bug Fixes#

  • Modified the HTCondor ClassAd expression managing job’s memory requests between restarts so the request never exceeds the memory limit defined by the memoryLimit setting. (DM-54079)

lsst-ctrl-bps-htcondor v30.0.4 (2026-02-23)#

New Features#

  • Added nodeset keyword in submit yaml to specify the nodeset name for the jobs to match specific glideins. When submit yaml requests to do automatic provisioning, automatically set nodeset name to the run’s timestamp thus restricting the run’s jobs to the glideins started by the automatic provisioning. (DM-53960)

Bug Fixes#

  • Fixed a bug causing the plugin to report some idle jobs as running. (DM-53782)

  • Fixed a bug breaking use of global or finalJob-specific priority. Bug was introduced on DM-53753. (DM-53927)

  • Fixed passing of expert-level HTCondor job settings from submit yaml’s profile.condor section. (DM-53960)

  • Fixed formatting in glidein section instructions. (DM-54104)

lsst-ctrl-bps-htcondor v30.0.1 (2026-02-03)#

Dropped support for Python 3.11. Tested with Python 3.14.

Other Changes and Additions#

    • Switched docs action to use sphinxutils.

    • Bumped minimum Python version to 3.12.

    • Dropped black and isort sections from pyproject.toml. (DM-54005)

lsst-ctrl-bps-htcondor v30.0.0 (2026-01-16)#

New Features#

  • Added ability to bring the output files produced during job’s execution back to the submit site. (DM-48479)

  • Added ability to save job stdout/stderr across job retries by setting overwriteJobFiles to false in the submit yaml. (DM-51905)

  • Added a mechanism that lets the user configure DAGMan via submit YAML. (DM-52778)

  • Added using the given priority as the DAG node priority in addition to the HTCondor job priority. (DM-53753)

Bug Fixes#

  • Fixed bug causing bps status --id to always report MISFIT status. Bug was introduced in DM-51868. (DM-53014)

  • Fixed bug causing finalJob’s memory to not be increased on retries like on regular payload jobs. (DM-53053)

Other Changes and Additions#

  • Redirected payload’s stderr and stdout to the same file (*.out) to reduce number of files the plugin creates during submission. (DM-31879)

  • Added documentation on condor_tail and condor_ssh_to_job. (DM-50900)

  • Fixed issue with some failed jobs being reported as deleted when using bps report with HTCondor plugin. (DM-51868)

  • Divided htcondor_service.py and test_htcondor_service.py into smaller files. (DM-52552)

lsst-ctrl-bps-htcondor v29.1.0 (2025-06-13)#

New Features#

  • Added capability for NOOP and EXTERNAL SUBDAG DAG nodes. (DM-46294)

  • Added ability to add job release expression. (DM-50614)

  • Added get_status method to HTCondorService class for quick checking of run status. (DM-50619)

Other Changes and Additions#

  • Explicitly define MaxIdle to workaround bug where HTCondor overrides config and environment variables when it is responsible for making DAGMan submit file (affects at least certain 24.0.x versions). (DM-50212)

lsst-ctrl-bps-htcondor v29.0.0 (2025-03-25)#

New Features#

  • Added GenericWorkflowJob’s environment values to HTCondor job submit files. (DM-48245)

Bug Fixes#

  • Included non-Python data artifacts when packaging for PyPi distribution, including etc/*yaml and final_post.sh. (DM-48032)

Other Changes and Additions#

  • Modified the function responsible for replacing placeholders in command line arguments so it gracefully handles the case when there are no placeholders to replace. (DM-46307)

  • Improved reporting of provisioning job in atypical situations. (DM-48752)

lsst-ctrl-bps-htcondor v28.0.0 (2024-11-21)#

New Features#

  • Implemented basic ping method for HTCondor plugin that checks Schedd and Collector are running and user can authenticate to them. It does not check that there are compute resources that can run the user’s jobs. (DM-35145)

  • Added ability for the plugin to call allocateNodes.py during workflow execution in order to manage required computational resources automatically. (DM-42579)

  • Updated plugin to use retryUnlessExit values so WMS won’t rerun some failures that will just fail every time. (DM-44668)

Bug Fixes#

  • Fixed status when job held and released. (DM-44107)

  • Fixed report listing auto-memory retry as failed when actually successful. (DM-44668)

Other Changes and Additions#

  • Reported better error message when failed submission from /tmp. (DM-43932)

  • Provided a default value for the memoryLimit parameter so it will be set automatically for the users if this plugin is used. (DM-44110)

  • Fixed held and deleted state_counts for reporting. (DM-44457)

  • Updated plugin to allow spaces in job submit file path. (DM-45654)

  • Updated bps restart to work with relative path as id. Updated bps report --id <relpath> to display absolute path. (DM-46046)

  • Added a section describing how to release held jobs to the package documentation. (DM-38538)

lsst-ctrl-bps-htcondor v27.0.0 (2024-06-04)#

New Features#

  • Updated the open-source license to allow for the code to be distributed with either GPLv3 or BSD 3-clause license. (DM-37231)

  • Made the plugin properly handle new node status FUTILE that represents a node that will never run due to the failure of a node that the FUTILE node depends on either directly or indirectly through a chain of PARENT / CHILD relationships. (DM-38627)

  • Made bps restart accept other types of run IDs beside the submit directory. (DM-41561)

  • Added plugin support for reporting error exit codes with bps report. (DM-42127)

Bug Fixes#

  • Fixed bug preventing bps cancel from working. (DM-40906)

  • Fixed bug preventing bps report from showing error codes/counts correctly when called with the submit directory as the run id. (DM-43381)

  • Fixed compute_site keyword error in submit introduced by DM-38138. (DM-43721)

Other Changes and Additions#

  • Handle changes between different version of HTCondor Python API gracefully so deprecation warnings don’t pop up when using bps report. (DM-37020)

  • Replaced function/methods that are being deprecated by the HTCondor team with their preferred equivalents to remove deprecation warnings during executions of BPS commands. (DM-42759)

lsst-ctrl-bps-htcondor v26.0.0 (2023-09-25)#

No significant changes. This release includes minor code cleanups and reformatting. It has been verified to work with Python 3.11.

lsst-ctrl-bps-htcondor v25.0.0 (2023-03-02)#

Other Changes and Additions#

  • Made the plugin always report on the latest run even if the old run id was provided to bps report. (DM-35533)

lsst-ctrl-bps-htcondor v24.0.0 (2022-08-29)#

New Features#

  • This package has been extracted from lsst_ctrl_bps into a standalone package to make it easier to manage development of the HTCondor plugin. (DM-33521)

  • Add support for a new command, bps restart, that allows one to restart the failed workflow from the point of its failure. It restarts the workflow as it is just retrying failed jobs, no configuration changes are possible at the moment. (DM-29575)

  • Add support for a new option of bps cancel, --global, which allows the user to interact (cancel or get the report on) with jobs in any HTCondor job queue. (DM-29614)

  • Add a configurable memory threshold to the memory scaling mechanism. (DM-32047)

Bug Fixes#

  • HTCondor plugin now correctly passes attributes defined in site’s ‘profile’ section to the HTCondor submission files. (DM-33887)

Other Changes and Additions#

  • Make HTCondor treat all jobs exiting with a signal as if they ran out of memory. (DM-32968)

  • Make HTCondor plugin pass a group and user attribute to any batch systems that require such attributes for accounting purposes. (DM-33887)

lsst-ctrl-bps-htcondor v23.0.0 (2021-12-10)#

New Features#

  • Added BPS htcondor job setting that should put jobs that get the signal 7 when exceeding memory on hold. Held message will say: “Job raised a signal 7. Usually means job has gone over memory limit.” Until bps has the automatic memory exceeded retries, you can restart these the same way as with jobs that htcondor held for exceeding memory limits (condor_qedit and condor_release).

    • Add numberOfRetries option which specifies the maximum number of retries allowed for a job.

    • Add memoryMultiplier option to allow for increasing the memory requirements automatically between retries for jobs which exceeded memory during their execution. At the moment this option is only supported by HTCondor plugin. (DM-29756)

  • Change HTCondor bps plugin to use HTCondor curl plugin for local job transfers. (DM-32074)

Bug Fixes#

    • Fix bug in HTCondor plugin for reporting final job status when --id <path>. (DM-31887)

  • Fix execution butler with HTCondor plugin bug when output collection has period. (DM-32201)

  • Disable HTCondor auto detection of files to copy back from jobs. (DM-32220)

    • Fixed bug when not using lazy commands but using execution butler.

    • Fixed bug in htcondor_service.py that overwrote message in bps report. (DM-32241)

    • Fixed bug when a pipetask process killed by a signal on the edge node did not expose the failing status. (DM-32435)