InMemoryDatastore

class lsst.daf.butler.datastores.inMemoryDatastore.InMemoryDatastore(config, registry=None)

Bases: lsst.daf.butler.Datastore

Basic Datastore for writing to an in memory cache.

This datastore is ephemeral in that the contents of the datastore disappear when the Python process completes. This also means that other processes can not access this datastore.

Parameters:

config : DatastoreConfig or str

Configuration.

Attributes:

config : DatastoreConfig

Configuration used to create Datastore.

storageClassFactory : StorageClassFactory

Factory for creating storage class instances from name.

name : str

Label associated with this Datastore.

Attributes Summary

defaultConfigFile	Path to configuration defaults.
isEphemeral	A new datastore is created every time and datasets disappear when the process shuts down.

Methods Summary

addStoredItemInfo(ref, info)	Record internal storage information associated with this DatasetRef.
exists(ref)	Check if the dataset exists in the datastore.
get(ref[, parameters])	Load an InMemoryDataset from the store.
getStoredItemInfo(ref)	Retrieve information associated with object stored in this Datastore.
getUri(ref[, predict])	URI to the Dataset.
put(inMemoryDataset, ref)	Write a InMemoryDataset with a given DatasetRef to the store.
remove(ref)	Indicate to the Datastore that a Dataset can be removed.
removeStoredItemInfo(ref)	Remove information about the object associated with this dataset.
setConfigRoot(root, config, full)	Set any filesystem-dependent config options for this Datastore to be appropriate for a new empty repository with the given root.
transfer(inputDatastore, ref)	Retrieve a Dataset from an input Datastore, and store the result in this Datastore.

Attributes Documentation

defaultConfigFile = 'datastores/inMemoryDatastore.yaml'

Path to configuration defaults. Relative to $DAF_BUTLER_DIR/config or absolute path. Can be None if no defaults specified.

isEphemeral = True

A new datastore is created every time and datasets disappear when the process shuts down.

Methods Documentation

addStoredItemInfo(ref, info)

Record internal storage information associated with this DatasetRef.

Parameters:

ref : DatasetRef

The Dataset that has been stored.

info : StoredItemInfo

Metadata associated with the stored Dataset.

Raises:

KeyError

An entry with this DatasetRef already exists.

exists(ref)

Check if the dataset exists in the datastore.

Parameters:

ref : DatasetRef

Reference to the required dataset.

Returns:

exists : bool

True if the entity exists in the Datastore.

get(ref, parameters=None)

Load an InMemoryDataset from the store.

Parameters:

ref : DatasetRef

Reference to the required Dataset.

parameters : dict

StorageClass-specific parameters that specify, for example, a slice of the Dataset to be loaded.

Returns:

inMemoryDataset : object

Requested Dataset or slice thereof as an InMemoryDataset.

Raises:

FileNotFoundError

Requested dataset can not be retrieved.

TypeError

Return value from formatter has unexpected type.

ValueError

Formatter failed to process the dataset.

getStoredItemInfo(ref)

Retrieve information associated with object stored in this Datastore.

Parameters:

ref : DatasetRef

The Dataset that is to be queried.

Returns:

info : StoredItemInfo

Stored information about the internal location of this file and its formatter.

Raises:

KeyError

Dataset with that id can not be found.

getUri(ref, predict=False)

URI to the Dataset.

Always uses “mem://” URI prefix.

Parameters:

ref : DatasetRef

Reference to the required Dataset.

predict : bool

If True, allow URIs to be returned of datasets that have not been written.

Returns:

uri : str

URI string pointing to the Dataset within the datastore. If the Dataset does not exist in the datastore, and if predict is True, the URI will be a prediction and will include a URI fragment “#predicted”. If the datastore does not have entities that relate well to the concept of a URI the returned URI string will be descriptive. The returned URI is not guaranteed to be obtainable.

Raises:

FileNotFoundError

A URI has been requested for a dataset that does not exist and guessing is not allowed.

put(inMemoryDataset, ref)

Write a InMemoryDataset with a given DatasetRef to the store.

Parameters:

inMemoryDataset : object

The Dataset to store.

ref : DatasetRef

Reference to the associated Dataset.

Raises:

TypeError

Supplied object and storage class are inconsistent.

remove(ref)

Indicate to the Datastore that a Dataset can be removed.

Parameters:

ref : DatasetRef

Reference to the required Dataset.

Raises:

FileNotFoundError

Attempt to remove a dataset that does not exist.

removeStoredItemInfo(ref)

Remove information about the object associated with this dataset.

Parameters:

ref : DatasetRef

The Dataset that has been removed.

classmethod setConfigRoot(root, config, full)

Set any filesystem-dependent config options for this Datastore to be appropriate for a new empty repository with the given root.

Does nothing in this implementation.

Parameters:

root : str

Filesystem path to the root of the data repository.

config : Config

A Config to update. Only the subset understood by this component will be updated. Will not expand defaults.

full : Config

A complete config with all defaults expanded that can be converted to a DatastoreConfig. Read-only and will not be modified by this method. Repository-specific options that should not be obtained from defaults when Butler instances are constructed should be copied from full to Config.

transfer(inputDatastore, ref)

Retrieve a Dataset from an input Datastore, and store the result in this Datastore.

Parameters:

inputDatastore : Datastore

The external Datastore from which to retreive the Dataset.

ref : DatasetRef

Reference to the required Dataset in the input data store.