DatasetRecordStorageManager

class lsst.daf.butler.registry.interfaces.DatasetRecordStorageManager

Bases: lsst.daf.butler.registry.interfaces.VersionedExtension

An interface that manages the tables that describe datasets.

DatasetRecordStorageManager primarily serves as a container and factory for DatasetRecordStorage instances, which each provide access to the records for a different DatasetType.

Methods Summary

addDatasetForeignKey(tableSpec, *, name, …) Add a foreign key (field and constraint) referencing the dataset table.
currentVersion() Return extension version as defined by current implementation.
extensionName() Return full name of the extension.
find(name) Return an object that provides access to the records associated with the given DatasetType name, if one exists.
getCollectionSummary(collection) Return a summary for the given collection.
getDatasetRef(id, uuid.UUID]) Return a DatasetRef for the given dataset primary key value.
getIdColumnType() Return type used for columns storing dataset IDs.
initialize(db, context, *, collections, …) Construct an instance of the manager.
refresh() Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.
register(datasetType) Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.
remove(name) Remove the dataset type.
resolve_wildcard(expression, components, …) Resolve a dataset type wildcard expression.
schemaDigest() Return digest for schema piece managed by this extension.
supportsIdGenerationMode(mode) Test whether the given dataset ID generation mode is supported by insert.

Methods Documentation

classmethod addDatasetForeignKey(tableSpec: lsst.daf.butler.core.ddl.TableSpec, *, name: str = 'dataset', constraint: bool = True, onDelete: Optional[str, None] = None, **kwargs) → lsst.daf.butler.core.ddl.FieldSpec

Add a foreign key (field and constraint) referencing the dataset table.

Parameters:
tableSpec : ddl.TableSpec

Specification for the table that should reference the dataset table. Will be modified in place.

name: `str`, optional

A name to use for the prefix of the new field; the full name is {name}_id.

onDelete: `str`, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

constraint: `bool`, optional

If False (True is default), add a field that can be joined to the dataset primary key, but do not add a foreign key constraint.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).
Returns:
idSpec : ddl.FieldSpec

Specification for the ID field.
classmethod currentVersion() → Optional[lsst.daf.butler.registry.interfaces._versioning.VersionTuple, None]

Return extension version as defined by current implementation.

This method can return None if an extension does not require its version to be saved or checked.

Returns:
version : VersionTuple

Current extension version or None.
classmethod extensionName() → str

Return full name of the extension.

This name should match the name defined in registry configuration. It is also stored in registry attributes. Default implementation returns full class name.

Returns:
name : str

Full extension name.
find(name: str) → lsst.daf.butler.registry.interfaces._datasets.DatasetRecordStorage | None[lsst.daf.butler.registry.interfaces._datasets.DatasetRecordStorage, None]

Return an object that provides access to the records associated with the given DatasetType name, if one exists.

Parameters:
name : str

Name of the dataset type.
Returns:
records : DatasetRecordStorage or None

The object representing the records for the given dataset type, or None if there are no records for that dataset type.

Notes

Dataset types registered by another client of the same repository since the last call to initialize or refresh may not be found.

getCollectionSummary(collection: CollectionRecord) → CollectionSummary

Return a summary for the given collection.

Parameters:
collection : CollectionRecord

Record describing the collection for which a summary is to be retrieved.
Returns:
summary : CollectionSummary

Summary of the dataset types and governor dimension values in this collection.
getDatasetRef(id: Union[int, uuid.UUID]) → lsst.daf.butler.core.datasets.ref.DatasetRef | None[lsst.daf.butler.core.datasets.ref.DatasetRef, None]

Return a DatasetRef for the given dataset primary key value.

Parameters:
id : DatasetId

Primary key value for the dataset.
Returns:
ref : DatasetRef or None

Object representing the dataset, or None if no dataset with the given primary key values exists in this layer.
classmethod getIdColumnType() → type

Return type used for columns storing dataset IDs.

This type is used for columns storing DatasetRef.id values, usually a type subclass provided by SQLAlchemy.

Returns:
dtype : type

Type used for dataset identification in database.
classmethod initialize(db: Database, context: StaticTablesContext, *, collections: CollectionManager, dimensions: DimensionRecordStorageManager) → DatasetRecordStorageManager

Construct an instance of the manager.

Parameters:
db : Database

Interface to the underlying database engine and namespace.

context : StaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present.

collections: `CollectionManager`

Manager object for the collections in this Registry.

dimensions : DimensionRecordStorageManager

Manager object for the dimensions in this Registry.
Returns:
manager : DatasetRecordStorageManager

An instance of a concrete DatasetRecordStorageManager subclass.
refresh() → None

Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.

register(datasetType: lsst.daf.butler.core.datasets.type.DatasetType) → tuple

Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.

Parameters:
datasetType : DatasetType

Dataset type for which a table should created (as necessary) and an associated DatasetRecordStorage returned.
Returns:
records : DatasetRecordStorage

The object representing the records for the given dataset type.

inserted : bool

True if the dataset type did not exist in the registry before.

Notes

This operation may not be invoked within a Database.transaction context.

remove(name: str) → None

Remove the dataset type.

Parameters:
name : str

Name of the dataset type.
resolve_wildcard(expression: Any, components: Optional[bool, None] = None, missing: Optional[list, None] = None, explicit_only: bool = False, components_deprecated: bool = True) → dict

Resolve a dataset type wildcard expression.

Parameters:
expression

Expression to resolve. Will be passed to DatasetTypeWildcard.from_expression.

components : bool, optional

If True, apply all expression patterns to component dataset type names as well. If False, never apply patterns to components. If None (default), apply patterns to components only if their parent datasets were not matched by the expression. Fully-specified component datasets (str or DatasetType instances) are always included.

missing : list of str, optional

String dataset type names that were explicitly given (i.e. not regular expression patterns) but not found will be appended to this list, if it is provided.

explicit_only : bool, optional

If True, require explicit DatasetType instances or str names, with re.Pattern instances deprecated and ... prohibited.

components_deprecated : bool, optional

If True, this is a context in which component dataset support is deprecated. This will result in a deprecation warning when components=True or components=None and a component dataset is matched. In the future this will become an error.
Returns:
dataset_types : dict [ DatasetType, list [ None, str ] ]

A mapping with resolved dataset types as keys and lists of matched component names as values, where None indicates the parent composite dataset type was matched.
schemaDigest() → Optional[str, None]

Return digest for schema piece managed by this extension.

Returns:
digest : str or None

String representation of the digest of the schema, None should be returned if schema digest is not to be saved or checked. The length of the returned string cannot exceed the length of the “value” column of butler attributes table, currently 65535 characters.

Notes

There is no exact definition of digest format, any string should work. The only requirement for string contents is that it has to remain stable over time if schema does not change but it should produce different string for any change in the schema. In many cases default implementation in _defaultSchemaDigest can be used as a reasonable choice.

classmethod supportsIdGenerationMode(mode: lsst.daf.butler.registry.interfaces._datasets.DatasetIdGenEnum) → bool

Test whether the given dataset ID generation mode is supported by insert.

Parameters:
mode : DatasetIdGenEnum

Enum value for the mode to test.
Returns:
supported : bool

Whether the given mode is supported.