DatasetRecordStorageManager

class lsst.daf.butler.registry.interfaces.DatasetRecordStorageManager(*, registry_schema_version: VersionTuple | None = None)

Bases: VersionedExtension

An interface that manages the tables that describe datasets.

DatasetRecordStorageManager primarily serves as a container and factory for DatasetRecordStorage instances, which each provide access to the records for a different DatasetType.

Parameters:

registry_schema_versionVersionTuple or None, optional

Version of registry schema.

Methods Summary

addDatasetForeignKey(tableSpec, *[, name, ...])	Add a foreign key (field and constraint) referencing the dataset table.
clone(*, db, collections, dimensions, ...)	Make an independent copy of this manager instance bound to new instances of Database and other managers.
fetch_summaries(collections[, dataset_types])	Fetch collection summaries given their names and dataset types.
find(name)	Return an object that provides access to the records associated with the given DatasetType name, if one exists.
getCollectionSummary(collection)	Return a summary for the given collection.
getDatasetRef(id)	Return a DatasetRef for the given dataset primary key value.
getIdColumnType()	Return type used for columns storing dataset IDs.
ingest_date_dtype()	Return type of the ingest_date column.
initialize(db, context, *, collections, ...)	Construct an instance of the manager.
refresh()	Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.
register(datasetType)	Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.
remove(name)	Remove the dataset type.
resolve_wildcard(expression[, missing, ...])	Resolve a dataset type wildcard expression.
supportsIdGenerationMode(mode)	Test whether the given dataset ID generation mode is supported by insert.

Methods Documentation

abstract classmethod addDatasetForeignKey(tableSpec: TableSpec, *, name: str = 'dataset', constraint: bool = True, onDelete: str | None = None, **kwargs: Any) → FieldSpec

Add a foreign key (field and constraint) referencing the dataset table.

Parameters:

tableSpecddl.TableSpec

Specification for the table that should reference the dataset table. Will be modified in place.

namestr, optional

A name to use for the prefix of the new field; the full name is {name}_id.

constraintbool, optional

If False (True is default), add a field that can be joined to the dataset primary key, but do not add a foreign key constraint.

onDeletestr, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).

Returns:

idSpecddl.FieldSpec

Specification for the ID field.

abstract clone(*, db: Database, collections: CollectionManager, dimensions: DimensionRecordStorageManager, caching_context: CachingContext) → DatasetRecordStorageManager

Make an independent copy of this manager instance bound to new instances of Database and other managers.

Parameters:

dbDatabase

New Database object to use when instantiating the manager.

collectionsCollectionManager

New CollectionManager object to use when instantiating the manager.

dimensionsDimensionRecordStorageManager

New DimensionRecordStorageManager object to use when instantiating the manager.

caching_contextCachingContext

New CachingContext object to use when instantiating the manager.

Returns:

instanceDatasetRecordStorageManager

New manager instance with the same configuration as this instance, but bound to a new Database object.

abstract fetch_summaries(collections: Iterable[CollectionRecord], dataset_types: Iterable[DatasetType] | Iterable[str] | None = None) → Mapping[Any, CollectionSummary]

Fetch collection summaries given their names and dataset types.

Parameters:

collectionsIterable [CollectionRecord]

Collection records to query.

dataset_typesIterable [DatasetType] or None

Dataset types to include into returned summaries. If None then all dataset types will be included.

Returns:

summariesMapping [Any, CollectionSummary]

Collection summaries indexed by collection record key. This mapping will also contain all nested non-chained collections of the chained collections.

abstract find(name: str) → DatasetRecordStorage | None

Return an object that provides access to the records associated with the given DatasetType name, if one exists.

Parameters:

namestr

Name of the dataset type.

Returns:

recordsDatasetRecordStorage or None

The object representing the records for the given dataset type, or None if there are no records for that dataset type.

Notes

Dataset types registered by another client of the same repository since the last call to initialize or refresh may not be found.

abstract getCollectionSummary(collection: CollectionRecord) → CollectionSummary

Return a summary for the given collection.

Parameters:

collectionCollectionRecord

Record describing the collection for which a summary is to be retrieved.

Returns:

summaryCollectionSummary

Summary of the dataset types and governor dimension values in this collection.

abstract getDatasetRef(id: UUID) → DatasetRef | None

Return a DatasetRef for the given dataset primary key value.

Parameters:

idDatasetId

Primary key value for the dataset.

Returns:

refDatasetRef or None

Object representing the dataset, or None if no dataset with the given primary key values exists in this layer.

abstract classmethod getIdColumnType() → type

Return type used for columns storing dataset IDs.

This type is used for columns storing DatasetRef.id values, usually a type subclass provided by SQLAlchemy.

Returns:

dtypetype

Type used for dataset identification in database.

abstract ingest_date_dtype() → type

Return type of the ingest_date column.

abstract classmethod initialize(db: Database, context: StaticTablesContext, *, collections: CollectionManager, dimensions: DimensionRecordStorageManager, caching_context: CachingContext, registry_schema_version: VersionTuple | None = None) → DatasetRecordStorageManager

Construct an instance of the manager.

Parameters:

dbDatabase

Interface to the underlying database engine and namespace.

contextStaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present.

collectionsCollectionManager

Manager object for the collections in this Registry.

dimensionsDimensionRecordStorageManager

Manager object for the dimensions in this Registry.

caching_contextCachingContext

Object controlling caching of information returned by managers.

registry_schema_versionVersionTuple or None

Schema version of this extension as defined in registry.

Returns:

managerDatasetRecordStorageManager

An instance of a concrete DatasetRecordStorageManager subclass.

abstract refresh() → None

Ensure all other operations on this manager are aware of any dataset types that may have been registered by other clients since it was initialized or last refreshed.

abstract register(datasetType: DatasetType) → bool

Ensure that this Registry can hold records for the given DatasetType, creating new tables as necessary.

Parameters:

datasetTypeDatasetType

Dataset type for which a table should created (as necessary) and an associated DatasetRecordStorage returned.

Returns:

insertedbool

True if the dataset type did not exist in the registry before.

Notes

This operation may not be invoked within a Database.transaction context.

abstract remove(name: str) → None

Remove the dataset type.

Parameters:

namestr

Name of the dataset type.

abstract resolve_wildcard(expression: Any, missing: list[str] | None = None, explicit_only: bool = False) → list[lsst.daf.butler._dataset_type.DatasetType]

Resolve a dataset type wildcard expression.

Parameters:

expressionAny

Expression to resolve. Will be passed to DatasetTypeWildcard.from_expression.

missinglist of str, optional

String dataset type names that were explicitly given (i.e. not regular expression patterns) but not found will be appended to this list, if it is provided.

explicit_onlybool, optional

If True, require explicit DatasetType instances or str names, with re.Pattern instances deprecated and ... prohibited.

Returns:

dataset_typeslist [ DatasetType ]

A list of resolved dataset types.

abstract classmethod supportsIdGenerationMode(mode: DatasetIdGenEnum) → bool

Test whether the given dataset ID generation mode is supported by insert.

Parameters:

modeDatasetIdGenEnum

Enum value for the mode to test.

Returns:

supportedbool

Whether the given mode is supported.