htc_backup_files¶
- lsst.ctrl.bps.htcondor.htc_backup_files(wms_path, subdir=None, limit=100)¶
Backup select HTCondor files in the submit directory.
Files will be saved in separate subdirectories which will be created in the submit directory where the files are located. These subdirectories will be consecutive, zero-padded integers. Their values will correspond to the number of HTCondor rescue DAGs in the submit directory.
Hence, with the default settings, copies after the initial failed run will be placed in ‘001’ subdirectory, ‘002’ after the first restart, and so on until the limit of backups is reached. If there’s no rescue DAG yet, files will be copied to ‘000’ subdirectory.
- Parameters:
- wms_path
strorpathlib.Path Path to the submit directory either absolute or relative.
- subdir
strorpathlib.Path, optional A path, relative to the submit directory, where all subdirectories with backup files will be kept. Defaults to None which means that the backup subdirectories will be placed directly in the submit directory.
- limit
int, optional Maximal number of backups. If the number of backups reaches the limit, the last backup files will be overwritten. The default value is 100 to match the default value of HTCondor’s DAGMAN_MAX_RESCUE_NUM in version 8.8+.
- wms_path
- Raises:
- FileNotFoundError
If the submit directory or the file that needs to be backed up does not exist.
- OSError
If the submit directory cannot be accessed or backing up a file failed either due to permission or filesystem related issues.
Notes
This is not a generic function for making backups. It is intended to be used once, just before a restart, to make snapshots of files which will be overwritten by HTCondor after during the next run.