DatasetRecordStorageManager

class lsst.daf.butler.registry.interfaces.DatasetRecordStorageManager

Bases: lsst.daf.butler.registry.interfaces.VersionedExtension

An interface that manages the tables that describe datasets.

DatasetRecordStorageManager primarily serves as a container and factory for DatasetRecordStorage instances, which each provide access to the records for a different DatasetType.

Methods Summary

addDatasetForeignKey(tableSpec, *[, name, …])	Add a foreign key (field and constraint) referencing the dataset table.
currentVersion()	Return extension version as defined by current implementation.
extensionName()	Return full name of the extension.
find(name)	Return an object that provides access to the records associated with the given DatasetType name, if one exists.
getCollectionSummary(collection)	Return a summary for the given collection.
getDatasetRef(id)	Return a DatasetRef for the given dataset primary key value.
getIdColumnType()	Return type used for columns storing dataset IDs.
initialize(db, context, *, collections, …)	Construct an instance of the manager.
refresh()	Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.
register(datasetType)	Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.
remove(name)	Remove the dataset type.
schemaDigest()	Return digest for schema piece managed by this extension.

Methods Documentation

abstract classmethod addDatasetForeignKey(tableSpec: lsst.daf.butler.core.ddl.TableSpec, *, name: str = 'dataset', constraint: bool = True, onDelete: Optional[str] = None, **kwargs: Any) → lsst.daf.butler.core.ddl.FieldSpec

Add a foreign key (field and constraint) referencing the dataset table.

Parameters

tableSpecddl.TableSpec

Specification for the table that should reference the dataset table. Will be modified in place.

name: `str`, optional

A name to use for the prefix of the new field; the full name is {name}_id.

onDelete: `str`, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

constraint: `bool`, optional

If False (True is default), add a field that can be joined to the dataset primary key, but do not add a foreign key constraint.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).

Returns

idSpecddl.FieldSpec

Specification for the ID field.

abstract classmethod currentVersion() → Optional[lsst.daf.butler.registry.interfaces.VersionTuple]

Return extension version as defined by current implementation.

This method can return None if an extension does not require its version to be saved or checked.

Returns

versionVersionTuple

Current extension version or None.

classmethod extensionName() → str

Return full name of the extension.

This name should match the name defined in registry configuration. It is also stored in registry attributes. Default implementation returns full class name.

Returns

namestr

Full extension name.

abstract find(name: str) → Optional[lsst.daf.butler.registry.interfaces.DatasetRecordStorage]

Return an object that provides access to the records associated with the given DatasetType name, if one exists.

Parameters

namestr

Name of the dataset type.

Returns

recordsDatasetRecordStorage or None

The object representing the records for the given dataset type, or None if there are no records for that dataset type.

Notes

Dataset types registered by another client of the same repository since the last call to initialize or refresh may not be found.

abstract getCollectionSummary(collection: CollectionRecord) → CollectionSummary

Return a summary for the given collection.

Parameters

collectionCollectionRecord

Record describing the collection for which a summary is to be retrieved.

Returns

summaryCollectionSummary

Summary of the dataset types and governor dimension values in this collection.

abstract getDatasetRef(id: Union[int, uuid.UUID]) → Optional[lsst.daf.butler.DatasetRef]

Return a DatasetRef for the given dataset primary key value.

Parameters

idDatasetId

Primary key value for the dataset.

Returns

refDatasetRef or None

Object representing the dataset, or None if no dataset with the given primary key values exists in this layer.

abstract classmethod getIdColumnType() → type

Return type used for columns storing dataset IDs.

This type is used for columns storing DatasetRef.id values, usually a type subclass provided by SQLAlchemy.

Returns

dtypetype

Type used for dataset identification in database.

abstract classmethod initialize(db: Database, context: StaticTablesContext, *, collections: CollectionManager, dimensions: DimensionRecordStorageManager) → DatasetRecordStorageManager

Construct an instance of the manager.

Parameters

dbDatabase

Interface to the underlying database engine and namespace.

contextStaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present.

collections: `CollectionManager`

Manager object for the collections in this Registry.

dimensionsDimensionRecordStorageManager

Manager object for the dimensions in this Registry.

Returns

managerDatasetRecordStorageManager

An instance of a concrete DatasetRecordStorageManager subclass.

abstract refresh() → None

Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.

abstract register(datasetType: lsst.daf.butler.DatasetType) → Tuple[lsst.daf.butler.registry.interfaces.DatasetRecordStorage, bool]

Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.

Parameters

datasetTypeDatasetType

Dataset type for which a table should created (as necessary) and an associated DatasetRecordStorage returned.

Returns

recordsDatasetRecordStorage

The object representing the records for the given dataset type.

insertedbool

True if the dataset type did not exist in the registry before.

Notes

This operation may not be invoked within a Database.transaction context.

abstract remove(name: str) → None

Remove the dataset type.

Parameters

namestr

Name of the dataset type.

abstract schemaDigest() → Optional[str]

Return digest for schema piece managed by this extension.

Returns

digeststr or None

String representation of the digest of the schema, None should be returned if schema digest is not to be saved or checked. The length of the returned string cannot exceed the length of the “value” column of butler attributes table, currently 65535 characters.

Notes

There is no exact definition of digest format, any string should work. The only requirement for string contents is that it has to remain stable over time if schema does not change but it should produce different string for any change in the schema. In many cases default implementation in _defaultSchemaDigest can be used as a reasonable choice.