AbstractDatastoreCacheManager

class lsst.daf.butler.AbstractDatastoreCacheManager(config: Union[str, DatastoreCacheManagerConfig], universe: DimensionUniverse)

Bases: abc.ABC

An abstract base class for managing caching in a Datastore.

Parameters:

config : str or DatastoreCacheManagerConfig

Configuration to control caching.

universe : DimensionUniverse

Set of all known dimensions, used to expand and validate any used in lookup keys.

Attributes Summary

cache_size	Size of the cache in bytes.
file_count	Return number of cached files tracked by registry.

Methods Summary

find_in_cache(ref, extension)	Look for a dataset in the cache and return its location.
move_to_cache(uri, ref)	Move a file to the cache.
remove_from_cache(ref, …)	Remove the specified datasets from the cache.
should_be_cached(entity, DatasetType, …)	Indicate whether the entity should be added to the cache.

Attributes Documentation

cache_size

Size of the cache in bytes.

file_count

Return number of cached files tracked by registry.

Methods Documentation

find_in_cache(ref: lsst.daf.butler.core.datasets.ref.DatasetRef, extension: str) → Iterator[Optional[lsst.resources._resourcePath.ResourcePath]]

Look for a dataset in the cache and return its location.

Parameters:

ref : DatasetRef

Dataset to locate in the cache.

extension : str

File extension expected. Should include the leading “.”.

Yields:

uri : lsst.resources.ResourcePath or None

The URI to the cached file, or None if the file has not been cached.

Notes

Should be used as a context manager in order to prevent this file from being removed from the cache for that context.

move_to_cache(uri: lsst.resources._resourcePath.ResourcePath, ref: lsst.daf.butler.core.datasets.ref.DatasetRef) → Optional[lsst.resources._resourcePath.ResourcePath]

Move a file to the cache.

Move the given file into the cache, using the supplied DatasetRef for naming. A call is made to should_be_cached() and if the DatasetRef should not be accepted None will be returned.

Cache expiry can occur during this.

Parameters:

uri : lsst.resources.ResourcePath

Location of the file to be relocated to the cache. Will be moved.

ref : DatasetRef

Ref associated with this file. Will be used to determine the name of the file within the cache.

Returns:

new : lsst.resources.ResourcePath or None

URI to the file within the cache, or None if the dataset was not accepted by the cache.

remove_from_cache(ref: Union[lsst.daf.butler.core.datasets.ref.DatasetRef, Iterable[lsst.daf.butler.core.datasets.ref.DatasetRef]]) → None

Remove the specified datasets from the cache.

It is not an error for these datasets to be missing from the cache.

Parameters:

ref : DatasetRef or iterable of DatasetRef

The datasets to remove from the cache.

should_be_cached(entity: Union[DatasetRef, DatasetType, StorageClass]) → bool

Indicate whether the entity should be added to the cache.

This is relevant when reading or writing.

Parameters:

entity : StorageClass or DatasetType or DatasetRef

Thing to test against the configuration. The name property is used to determine a match. A DatasetType will first check its name, before checking its StorageClass. If there are no matches the default will be returned.

Returns:

should_cache : bool

Returns True if the dataset should be cached; False otherwise.