DatastoreRegistryBridge

class lsst.daf.butler.registry.interfaces.DatastoreRegistryBridge(datastoreName: str)

Bases: abc.ABC

An abstract base class that defines the interface that a Datastore uses to communicate with a Registry.

Parameters:
datastoreName : str

Name of the Datastore as it should appear in Registry tables referencing it.

Methods Summary

check(refs, …) Check which refs are listed for this datastore.
emptyTrash(records_table, record_class, …) Retrieve all the dataset ref IDs that are in the trash associated for this datastore, and then remove them if the context exists without an exception being raised.
forget(refs, …) Remove dataset location information without any attempt to put it in the trash while waiting for external deletes.
insert(refs, …) Record that a datastore holds the given datasets.
moveToTrash(refs, transaction) Move dataset location information to trash.

Methods Documentation

check(refs: Iterable[Union[lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.registry.interfaces._bridge.FakeDatasetRef]]) → Iterable[Union[lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.registry.interfaces._bridge.FakeDatasetRef]]

Check which refs are listed for this datastore.

Parameters:
refs : Iterable of DatasetIdRef

References to the datasets.

Returns:
present : Iterable [ DatasetIdRef ]

Datasets from refs that are recorded as being in this datastore.

Raises:
AmbiguousDatasetError

Raised if any(ref.id is None for ref in refs).

emptyTrash(records_table: Optional[OpaqueTableStorage] = None, record_class: Optional[Type[StoredDatastoreItemInfo]] = None, record_column: Optional[str] = None) → ContextManager[Tuple[Iterable[Tuple[DatasetIdRef, Optional[StoredDatastoreItemInfo]]], Optional[Set[str]]]]

Retrieve all the dataset ref IDs that are in the trash associated for this datastore, and then remove them if the context exists without an exception being raised.

Parameters:
records_table : OpaqueTableStorage, optional

Table of records to query with the trash records.

record_class : type of StoredDatastoreItemInfo, optional

Class to use when reading records from records_table.

record_column : str, optional

Name of the column in records_table that refers to the artifact.

Yields:
matches : iterable of (DatasetIdRef, StoredDatastoreItemInfo)

The IDs of datasets that can be safely removed from this datastore and the corresponding information from the records table. Can be empty.

artifacts_to_keep : set of str, optional

Any external artifacts that are known to the table but which should not be deleted. If None, the caller should check themselves.

Notes

The object yielded by the context manager may be a single-pass iterator. If multiple passes are required, it should be converted to a list or other container.

Datastores should never raise (except perhaps in testing) when an artifact cannot be removed only because it is already gone - this condition is an unavoidable outcome of concurrent delete operations, and must not be considered and error for those to be safe.

If a table record is provided the trashed records will be deleted when the context manager completes.

Examples

Typical usage by a Datastore is something like:

with self.bridge.emptyTrash() as trashed:
    iter, to_keep = trashed
    for ref, info in iter:
        # Remove artifacts associated with id,
        # raise an exception if something goes wrong.
forget(refs: Iterable[Union[lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.registry.interfaces._bridge.FakeDatasetRef]]) → None

Remove dataset location information without any attempt to put it in the trash while waiting for external deletes.

This should be used only to implement Datastore.forget, or in cases where deleting the actual datastore artifacts cannot fail.

Parameters:
refs : Iterable of DatasetIdRef

References to the datasets.

Raises:
AmbiguousDatasetError

Raised if any(ref.id is None for ref in refs).

insert(refs: Iterable[Union[lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.registry.interfaces._bridge.FakeDatasetRef]]) → None

Record that a datastore holds the given datasets.

Parameters:
refs : Iterable of DatasetIdRef

References to the datasets.

Raises:
AmbiguousDatasetError

Raised if any(ref.id is None for ref in refs).

moveToTrash(refs: Iterable[DatasetIdRef], transaction: Optional[DatastoreTransaction]) → None

Move dataset location information to trash.

Parameters:
refs : Iterable of DatasetIdRef

References to the datasets.

transaction : DatastoreTransaction or None

Transaction object. Can be None in some bridges or if no rollback is required.

Raises:
AmbiguousDatasetError

Raised if any(ref.id is None for ref in refs).