FileTransferSource

class lsst.daf.butler.datastore.FileTransferSource(*args, **kwargs)

Bases: Protocol

Protocol for an object that can return information about files that need to be transferred to copy datasets from one Butler repository to another.

Methods Summary

get_file_info_for_transfer(dataset_ids)	Given a list of dataset IDs, return all file information associated with the datasets that can be determined without searching the filesystem.
locate_missing_files_for_transfer(refs, ...)	Given a list of DatasetRef, search the filesystem to locate artifacts associated with the dataset.

Methods Documentation

get_file_info_for_transfer(dataset_ids: Iterable[UUID]) → dict[uuid.UUID, list[lsst.daf.butler.datastore._transfer.FileTransferRecord]]

Given a list of dataset IDs, return all file information associated with the datasets that can be determined without searching the filesystem.

Parameters:

dataset_idsIterable [ DatasetId ]

List of dataset IDs for which we will retrieve file information.

Returns:

transfer_mapFileTransferMap

Dictionary from DatasetId to a list of files found for that dataset. If information about any given dataset IDs could not be found, the missing IDs are omitted from the dictionary.

locate_missing_files_for_transfer(refs: Iterable[DatasetRef], artifact_existence: dict[lsst.resources._resourcePath.ResourcePath, bool]) → dict[uuid.UUID, list[lsst.daf.butler.datastore._transfer.FileTransferRecord]]

Given a list of DatasetRef, search the filesystem to locate artifacts associated with the dataset.

Parameters:

refsiterable of DatasetRef

The datasets to be checked.

artifact_existencedict [lsst.resources.ResourcePath, bool]

Optional mapping of datastore artifact to existence. Updated by this method with details of all artifacts tested.

Returns:

transfer_mapFileTransferMap

Dictionary from DatasetId to a list of files found for that dataset. If information about any given dataset IDs could not be found, the missing IDs are omitted from the dictionary.