AbstractDatastoreCacheManager¶
- class lsst.daf.butler.AbstractDatastoreCacheManager(config: str | DatastoreCacheManagerConfig, universe: DimensionUniverse)¶
Bases:
ABC
An abstract base class for managing caching in a Datastore.
- Parameters:
- config
str
orDatastoreCacheManagerConfig
Configuration to control caching.
- universe
DimensionUniverse
Set of all known dimensions, used to expand and validate any used in lookup keys.
- config
Attributes Summary
Size of the cache in bytes.
Return number of cached files tracked by registry.
Methods Summary
find_in_cache
(ref, extension)Look for a dataset in the cache and return its location.
known_to_cache
(ref[, extension])Report if the dataset is known to the cache.
move_to_cache
(uri, ref)Move a file to the cache.
remove_from_cache
(ref)Remove the specified datasets from the cache.
should_be_cached
(entity)Indicate whether the entity should be added to the cache.
Attributes Documentation
- cache_size¶
Size of the cache in bytes.
- file_count¶
Return number of cached files tracked by registry.
Methods Documentation
- abstract find_in_cache(ref: DatasetRef, extension: str) Iterator[ResourcePath | None] ¶
Look for a dataset in the cache and return its location.
- Parameters:
- ref
DatasetRef
Dataset to locate in the cache.
- extension
str
File extension expected. Should include the leading “
.
”.
- ref
- Yields:
- uri
lsst.resources.ResourcePath
orNone
The URI to the cached file, or
None
if the file has not been cached.
- uri
Notes
Should be used as a context manager in order to prevent this file from being removed from the cache for that context.
- abstract known_to_cache(ref: DatasetRef, extension: str | None = None) bool ¶
Report if the dataset is known to the cache.
- Parameters:
- ref
DatasetRef
Dataset to check for in the cache.
- extension
str
, optional File extension expected. Should include the leading “
.
”. IfNone
the extension is ignored and the dataset ID alone is used to check in the cache. The extension must be defined if a specific component is being checked.
- ref
- Returns:
Notes
This method can only report if the dataset is known to the cache in this specific instant and does not indicate whether the file can be read from the cache later.
find_in_cache()
should be called if the cached file is to be used.
- abstract move_to_cache(uri: ResourcePath, ref: DatasetRef) ResourcePath | None ¶
Move a file to the cache.
Move the given file into the cache, using the supplied DatasetRef for naming. A call is made to
should_be_cached()
and if the DatasetRef should not be acceptedNone
will be returned.Cache expiry can occur during this.
- Parameters:
- uri
lsst.resources.ResourcePath
Location of the file to be relocated to the cache. Will be moved.
- ref
DatasetRef
Ref associated with this file. Will be used to determine the name of the file within the cache.
- uri
- Returns:
- new
lsst.resources.ResourcePath
orNone
URI to the file within the cache, or
None
if the dataset was not accepted by the cache.
- new
- abstract remove_from_cache(ref: DatasetRef | Iterable[DatasetRef]) None ¶
Remove the specified datasets from the cache.
It is not an error for these datasets to be missing from the cache.
- Parameters:
- ref
DatasetRef
or iterable ofDatasetRef
The datasets to remove from the cache.
- ref
- abstract should_be_cached(entity: DatasetRef | DatasetType | StorageClass) bool ¶
Indicate whether the entity should be added to the cache.
This is relevant when reading or writing.
- Parameters:
- entity
StorageClass
orDatasetType
orDatasetRef
Thing to test against the configuration. The
name
property is used to determine a match. ADatasetType
will first check its name, before checking itsStorageClass
. If there are no matches the default will be returned.
- entity
- Returns: