FileTransferSource

class lsst.daf.butler.datastore.FileTransferSource(*args, **kwargs)

Bases: Protocol

Protocol for an object that can return information about files that need to be transferred to copy datasets from one Butler repository to another.

Methods Summary

get_file_info_for_transfer(dataset_ids)

Given a list of dataset IDs, return all file information associated with the datasets that can be determined without searching the filesystem.

locate_missing_files_for_transfer(refs, ...)

Given a list of DatasetRef, search the filesystem to locate artifacts associated with the dataset.

Methods Documentation

get_file_info_for_transfer(dataset_ids: Iterable[UUID]) dict[uuid.UUID, list[lsst.daf.butler.datastore._transfer.FileTransferRecord]]

Given a list of dataset IDs, return all file information associated with the datasets that can be determined without searching the filesystem.

Parameters:
dataset_idsIterable [ DatasetId ]

List of dataset IDs for which we will retrieve file information.

locate_missing_files_for_transfer(refs: Iterable[DatasetRef], artifact_existence: dict[lsst.resources._resourcePath.ResourcePath, bool]) dict[uuid.UUID, list[lsst.daf.butler.datastore._transfer.FileTransferRecord]]

Given a list of DatasetRef, search the filesystem to locate artifacts associated with the dataset.

Parameters:
refsiterable of DatasetRef

The datasets to be checked.

artifact_existencedict [lsst.resources.ResourcePath, bool]

Optional mapping of datastore artifact to existence. Updated by this method with details of all artifacts tested.