DatasetRecordStorageManager

class lsst.daf.butler.registry.interfaces.DatasetRecordStorageManager

Bases: VersionedExtension

An interface that manages the tables that describe datasets.

DatasetRecordStorageManager primarily serves as a container and factory for DatasetRecordStorage instances, which each provide access to the records for a different DatasetType.

Methods Summary

addDatasetForeignKey(tableSpec, *[, name, ...])	Add a foreign key (field and constraint) referencing the dataset table.
currentVersion()	Return extension version as defined by current implementation.
extensionName()	Return full name of the extension.
find(name)	Return an object that provides access to the records associated with the given DatasetType name, if one exists.
getCollectionSummary(collection)	Return a summary for the given collection.
getDatasetRef(id)	Return a DatasetRef for the given dataset primary key value.
getIdColumnType()	Return type used for columns storing dataset IDs.
initialize(db, context, *, collections, ...)	Construct an instance of the manager.
refresh()	Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.
register(datasetType)	Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.
remove(name)	Remove the dataset type.
resolve_wildcard(expression[, components, ...])	Resolve a dataset type wildcard expression.
schemaDigest()	Return digest for schema piece managed by this extension.
supportsIdGenerationMode(mode)	Test whether the given dataset ID generation mode is supported by insert.

Methods Documentation

abstract classmethod addDatasetForeignKey(tableSpec: TableSpec, *, name: str = 'dataset', constraint: bool = True, onDelete: str | None = None, **kwargs: Any) → FieldSpec

Add a foreign key (field and constraint) referencing the dataset table.

Parameters:

tableSpecddl.TableSpec

Specification for the table that should reference the dataset table. Will be modified in place.

name: `str`, optional

A name to use for the prefix of the new field; the full name is {name}_id.

onDelete: `str`, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

constraint: `bool`, optional

If False (True is default), add a field that can be joined to the dataset primary key, but do not add a foreign key constraint.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).

Returns:

idSpecddl.FieldSpec

Specification for the ID field.

abstract classmethod currentVersion() → VersionTuple | None

Return extension version as defined by current implementation.

This method can return None if an extension does not require its version to be saved or checked.

Returns:

versionVersionTuple

Current extension version or None.

classmethod extensionName() → str

Return full name of the extension.

This name should match the name defined in registry configuration. It is also stored in registry attributes. Default implementation returns full class name.

Returns:

namestr

Full extension name.

abstract find(name: str) → DatasetRecordStorage | None

Return an object that provides access to the records associated with the given DatasetType name, if one exists.

Parameters:

namestr

Name of the dataset type.

Returns:

recordsDatasetRecordStorage or None

The object representing the records for the given dataset type, or None if there are no records for that dataset type.

Notes

Dataset types registered by another client of the same repository since the last call to initialize or refresh may not be found.

abstract getCollectionSummary(collection: CollectionRecord) → CollectionSummary

Return a summary for the given collection.

Parameters:

collectionCollectionRecord

Record describing the collection for which a summary is to be retrieved.

Returns:

summaryCollectionSummary

Summary of the dataset types and governor dimension values in this collection.

abstract getDatasetRef(id: int | UUID) → DatasetRef | None

Return a DatasetRef for the given dataset primary key value.

Parameters:

idDatasetId

Primary key value for the dataset.

Returns:

refDatasetRef or None

Object representing the dataset, or None if no dataset with the given primary key values exists in this layer.

abstract classmethod getIdColumnType() → type

Return type used for columns storing dataset IDs.

This type is used for columns storing DatasetRef.id values, usually a type subclass provided by SQLAlchemy.

Returns:

dtypetype

Type used for dataset identification in database.

abstract classmethod initialize(db: Database, context: StaticTablesContext, *, collections: CollectionManager, dimensions: DimensionRecordStorageManager) → DatasetRecordStorageManager

Construct an instance of the manager.

Parameters:

dbDatabase

Interface to the underlying database engine and namespace.

contextStaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present.

collections: `CollectionManager`

Manager object for the collections in this Registry.

dimensionsDimensionRecordStorageManager

Manager object for the dimensions in this Registry.

Returns:

managerDatasetRecordStorageManager

An instance of a concrete DatasetRecordStorageManager subclass.

abstract refresh() → None

Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.

abstract register(datasetType: DatasetType) → tuple[lsst.daf.butler.registry.interfaces._datasets.DatasetRecordStorage, bool]

Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.

Parameters:

datasetTypeDatasetType

Dataset type for which a table should created (as necessary) and an associated DatasetRecordStorage returned.

Returns:

recordsDatasetRecordStorage

The object representing the records for the given dataset type.

insertedbool

True if the dataset type did not exist in the registry before.

Notes

This operation may not be invoked within a Database.transaction context.

abstract remove(name: str) → None

Remove the dataset type.

Parameters:

namestr

Name of the dataset type.

abstract resolve_wildcard(expression: Any, components: bool | None = None, missing: list[str] | None = None, explicit_only: bool = False, components_deprecated: bool = True) → dict[lsst.daf.butler.core.datasets.type.DatasetType, list[str | None]]

Resolve a dataset type wildcard expression.

Parameters:

expression

Expression to resolve. Will be passed to DatasetTypeWildcard.from_expression.

componentsbool, optional

If True, apply all expression patterns to component dataset type names as well. If False, never apply patterns to components. If None (default), apply patterns to components only if their parent datasets were not matched by the expression. Fully-specified component datasets (str or DatasetType instances) are always included.

missinglist of str, optional

String dataset type names that were explicitly given (i.e. not regular expression patterns) but not found will be appended to this list, if it is provided.

explicit_onlybool, optional

If True, require explicit DatasetType instances or str names, with re.Pattern instances deprecated and ... prohibited.

components_deprecatedbool, optional

If True, this is a context in which component dataset support is deprecated. This will result in a deprecation warning when components=True or components=None and a component dataset is matched. In the future this will become an error.

Returns:

dataset_typesdict [ DatasetType, list [ None, str ] ]

A mapping with resolved dataset types as keys and lists of matched component names as values, where None indicates the parent composite dataset type was matched.

abstract schemaDigest() → str | None

Return digest for schema piece managed by this extension.

Returns:

digeststr or None

String representation of the digest of the schema, None should be returned if schema digest is not to be saved or checked. The length of the returned string cannot exceed the length of the “value” column of butler attributes table, currently 65535 characters.

Notes

There is no exact definition of digest format, any string should work. The only requirement for string contents is that it has to remain stable over time if schema does not change but it should produce different string for any change in the schema. In many cases default implementation in _defaultSchemaDigest can be used as a reasonable choice.

abstract classmethod supportsIdGenerationMode(mode: DatasetIdGenEnum) → bool

Test whether the given dataset ID generation mode is supported by insert.

Parameters:

modeDatasetIdGenEnum

Enum value for the mode to test.

Returns:

supportedbool

Whether the given mode is supported.