DatabaseDimensionRecordStorage

class lsst.daf.butler.registry.interfaces.DatabaseDimensionRecordStorage

Bases: DimensionRecordStorage

Intermediate interface for DimensionRecordStorage objects that provide storage for DatabaseDimensionElement instances.

Attributes Summary

element

The element whose records this instance managers (DimensionElement).

Methods Summary

clearCaches()	Clear any in-memory caches held by the storage instance.
connect(overlaps)	Inform this record storage object of the object that will manage the overlaps between this element and another element.
digestTables()	Return tables used for schema digest.
fetch_one(data_id, context)	Retrieve a single record from storage.
get_record_cache(context)	Return a local cache of all DimensionRecord objects for this element, fetching it if necessary.
initialize(db, element, *[, context, ...])	Construct an instance of this class using a standardized interface.
insert(*records[, replace, skip_existing])	Insert one or more records into storage.
join(target, join, context)	Join this dimension element's records to a relation.
make_relation(context)	Return a relation that represents this dimension element's table.
make_spatial_join_relation(other, context, ...)	Return a lsst.daf.relation.Relation that represents the spatial overlap join between two dimension elements.
sync(record[, update])	Synchronize a record with the database, inserting it only if it does not exist and comparing values if it does.

Attributes Documentation

element

Methods Documentation

abstract clearCaches() → None

Clear any in-memory caches held by the storage instance.

This is called by Registry when transactions are rolled back, to avoid in-memory caches from ever containing records that are not present in persistent storage.

connect(overlaps: DatabaseDimensionOverlapStorage) → None

Inform this record storage object of the object that will manage the overlaps between this element and another element.

This will only be called if self.element.spatial is not None, and will be called immediately after construction (before any other methods). In the future, implementations will be required to call a method on any connected overlap storage objects any time new records for the element are inserted.

Parameters:

overlapsDatabaseDimensionRecordStorage

Object managing overlaps between this element and another database-backed element.

abstract digestTables() → list[sqlalchemy.sql.schema.Table]

Return tables used for schema digest.

Returns:

tableslist [ sqlalchemy.schema.Table ]

Possibly empty list of tables for schema digest calculations.

abstract fetch_one(data_id: DataCoordinate, context: queries.SqlQueryContext) → DimensionRecord | None

Retrieve a single record from storage.

Parameters:

data_idDataCoordinate

Data ID of the record to fetch. Implied dimensions do not need to be present.

contextqueries.SqlQueryContext

Context to be used to execute queries when no cached result is available.

Returns:

recordDimensionRecord or None

Fetched record, or possibly None if there was no match for the given data ID.

get_record_cache(context: queries.SqlQueryContext) → Mapping[DataCoordinate, DimensionRecord] | None

Return a local cache of all DimensionRecord objects for this element, fetching it if necessary.

Implementations that never cache records should return None.

Parameters:

contextqueries.SqlQueryContext

Context to be used to execute queries when no cached result is available.

Returns:

cacheMapping [ DataCoordinate, DimensionRecord ] or None

Mapping from data ID to dimension record, or None.

abstract classmethod initialize(db: Database, element: DatabaseDimensionElement, *, context: StaticTablesContext | None = None, config: Mapping[str, Any], governors: NamedKeyMapping[GovernorDimension, GovernorDimensionRecordStorage], view_target: DatabaseDimensionRecordStorage | None = None) → DatabaseDimensionRecordStorage

Construct an instance of this class using a standardized interface.

Parameters:

dbDatabase

Interface to the underlying database engine and namespace.

elementDatabaseDimensionElement

Dimension element the new instance will manage records for.

contextStaticTablesContext, optional

If provided, an object to use to create any new tables. If not provided, db.ensureTableExists should be used instead.

configMapping

Extra configuration options specific to the implementation.

governorsNamedKeyMapping

Mapping containing all governor dimension storage implementations.

view_targetDatabaseDimensionRecordStorage, optional

Storage object for the element this target’s storage is a view of (i.e. when viewOf is not None).

Returns:

storageDatabaseDimensionRecordStorage

A new DatabaseDimensionRecordStorage subclass instance.

abstract insert(*records: DimensionRecord, replace: bool = False, skip_existing: bool = False) → None

Insert one or more records into storage.

Parameters:

*records

One or more instances of the DimensionRecord subclass for the element this storage is associated with.

replacebool, optional

If True (False is default), replace existing records in the database if there is a conflict.

skip_existingbool, optional

If True (False is default), skip insertion if a record with the same primary key values already exists.

Raises:

TypeError

Raised if the element does not support record insertion.

sqlalchemy.exc.IntegrityError

Raised if one or more records violate database integrity constraints.

Notes

As insert is expected to be called only by a Registry, we rely on Registry to provide transactionality, both by using a SQLALchemy connection shared with the Registry and by relying on it to call clearCaches when rolling back transactions.

join(target: Relation, join: Join, context: queries.SqlQueryContext) → Relation

Join this dimension element’s records to a relation.

Parameters:

targetRelation

Existing relation to join to. Implementations may require that this relation already include dimension key columns for this dimension element and assume that dataset or spatial join relations that might provide these will be included in the relation tree first.

joinJoin

Join operation to use when the implementation is an actual join. When a true join is being simulated by other relation operations, this objects min_columns and max_columns should still be respected.

contextqueries.SqlQueryContext

Object that manages relation engines and database-side state (e.g. temporary tables) for the query.

Returns:

joinedRelation

New relation that includes this relation’s dimension key and record columns, as well as all columns in target, with rows constrained to those for which this element’s dimension key values exist in the registry and rows already exist in target.

abstract make_relation(context: queries.SqlQueryContext) → Relation

Return a relation that represents this dimension element’s table.

This is used to provide an implementation for DimensionRecordStorage.join, and is also callable in its own right.

Parameters:

contextqueries.SqlQueryContext

Object that manages relation engines and database-side state (e.g. temporary tables) for the query.

Returns:

relationRelation

New relation that includes this relation’s dimension key and record columns, with rows constrained to those for which the dimension key values exist in the registry.

make_spatial_join_relation(other: DimensionElement, context: queries.SqlQueryContext, governor_constraints: Mapping[str, Set[str]]) → Relation | None

Return a lsst.daf.relation.Relation that represents the spatial overlap join between two dimension elements.

High-level code should generally call DimensionRecordStorageManager.make_spatial_join_relation (which delegates to this) instead of calling this method directly.

Parameters:

otherDimensionElement

Element to compute overlaps with. Guaranteed by caller to be spatial (as is self), with a different topological family. May be a DatabaseDimensionElement or a SkyPixDimension.

contextqueries.SqlQueryContext

Object that manages relation engines and database-side state (e.g. temporary tables) for the query.

governor_constraintsMapping [ str, Set ], optional

Constraints imposed by other aspects of the query on governor dimensions.

Returns:

relationlsst.daf.relation.Relation or None

Join relation. Should be None when no direct overlaps for this combination are stored; higher-level code is responsible for working out alternative approaches involving multiple joins.

abstract sync(record: DimensionRecord, update: bool = False) → bool | dict[str, Any]

Synchronize a record with the database, inserting it only if it does not exist and comparing values if it does.

Parameters:

recordDimensionRecord

An instance of the DimensionRecord subclass for the element this storage is associated with.

updatebool, optional

If True (False is default), update the existing record in the database if there is a conflict.

Returns:

inserted_or_updatedbool or dict

True if a new row was inserted, False if no changes were needed, or a dict mapping updated column names to their old values if an update was performed (only possible if update=True).

Raises:

DatabaseConflictError

Raised if the record exists in the database (according to primary key lookup) but is inconsistent with the given one.

TypeError

Raised if the element does not support record synchronization.

sqlalchemy.exc.IntegrityError

Raised if one or more records violate database integrity constraints.