ObsCoreTableManager

class lsst.daf.butler.registry.interfaces.ObsCoreTableManager(*, registry_schema_version: VersionTuple | None = None)

Bases: VersionedExtension

An interface for populating ObsCore tables(s).

Methods Summary

add_datasets(refs, context)	Possibly add datasets to the obscore table.
associate(refs, collection, context)	Possibly add datasets to the obscore table.
checkCompatibility(registry_schema_version, ...)	Check that schema version defined in registry is compatible with current implementation.
checkNewSchemaVersion(schema_version)	Verify that requested schema version can be created by an extension.
clsNewSchemaVersion(schema_version)	Class method which returns schema version to use for newly created registry database.
config_json()	Dump configuration in JSON format.
currentVersions()	Return schema version(s) supported by this extension class.
disassociate(refs, collection)	Possibly remove datasets from the obscore table.
extensionName()	Return full name of the extension.
initialize(db, context, *, universe, config, ...)	Construct an instance of the manager.
newSchemaVersion()	Return schema version for newly created registry.
query([columns])	Run a SELECT query against obscore table and return result rows.
update_exposure_regions(instrument, region_data)	Update existing exposure records with spatial region data.

Methods Documentation

abstract add_datasets(refs: Iterable[DatasetRef], context: SqlQueryContext) → int

Possibly add datasets to the obscore table.

This method should be called when new datasets are added to a RUN collection.

Parameters:

refsiterable [ DatasetRef ]

Dataset refs to add. Dataset refs have to be completely expanded. If a record with the same dataset ID is already in obscore table, the dataset is ignored.

contextSqlQueryContext

Context used to execute queries for additional dimension metadata.

Returns:

countint

Actual number of records inserted into obscore table.

Notes

Dataset data types and collection names are checked against configured list of collections and dataset types, non-matching datasets are ignored and not added to the obscore table.

When configuration parameter collection_type is not “RUN”, this method should return immediately.

Note that there is no matching method to remove datasets from obscore table, we assume that removal happens via foreign key constraint to dataset table with “ON DELETE CASCADE” option.

abstract associate(refs: Iterable[DatasetRef], collection: CollectionRecord, context: SqlQueryContext) → int

Possibly add datasets to the obscore table.

This method should be called when existing datasets are associated with a TAGGED collection.

Parameters:

refsiterable [ DatasetRef ]

Dataset refs to add. Dataset refs have to be completely expanded. If a record with the same dataset ID is already in obscore table, the dataset is ignored.

collectionCollectionRecord

Collection record for a TAGGED collection.

contextSqlQueryContext

Context used to execute queries for additional dimension metadata.

Returns:

countint

Actual number of records inserted into obscore table.

Notes

Dataset data types and collection names are checked against configured list of collections and dataset types, non-matching datasets are ignored and not added to the obscore table.

When configuration parameter collection_type is not “TAGGED”, this method should return immediately.

classmethod checkCompatibility(registry_schema_version: VersionTuple, update: bool) → None

Check that schema version defined in registry is compatible with current implementation.

Parameters:

registry_schema_versionVersionTuple

Schema version that exists in registry or defined in a configuration for a registry to be created.

updatebool

If True then read-write access is expected.

Raises:

IncompatibleVersionError

Raised if schema version is not supported by implementation.

Notes

Default implementation uses VersionTuple.checkCompatibility on the versions returned from currentVersions method. Subclasses that support different compatibility model will overwrite this method.

classmethod checkNewSchemaVersion(schema_version: VersionTuple) → None

Verify that requested schema version can be created by an extension.

Parameters:

schema_versionVersionTuple

Schema version that this extension is asked to create.

Notes

This method may be used only occasionally when a specific schema version is given in a regisitry config file. This can be used with an extension that supports multiple schem versions to make it create new schema with a non-default version number. Default implementation compares requested version with one of the version returned from currentVersions.

classmethod clsNewSchemaVersion(schema_version: VersionTuple | None) → VersionTuple | None

Class method which returns schema version to use for newly created registry database.

Parameters:

schema_versionVersionTuple or None

Configured schema version or None if default schema version should be created. If not None then it is guaranteed to be compatible with currentVersions.

Returns:

versionVersionTuple or None

Schema version created by this extension. None is returned if an extension does not require its version to be saved or checked.

Notes

Default implementation of this method can work in simple cases. If the extension only supports single schema version than that version is returned. If the extension supports multiple schema versions and schema_version is not None then schema_version is returned. If the extension supports multiple schema versions, but schema_version is None it calls _newDefaultSchemaVersion method which needs to be reimplemented in a subsclass.

abstract config_json() → str

Dump configuration in JSON format.

Returns:

jsonstr

Configuration serialized in JSON format.

abstract classmethod currentVersions() → list[lsst.daf.butler.registry.interfaces._versioning.VersionTuple]

Return schema version(s) supported by this extension class.

Returns:

versionlist [VersionTuple]

Schema versions for this extension. Empty list is returned if an extension does not require its version to be saved or checked.

abstract disassociate(refs: Iterable[DatasetRef], collection: CollectionRecord) → int

Possibly remove datasets from the obscore table.

This method should be called when datasets are disassociated from a TAGGED collection.

Parameters:

refsiterable [ DatasetRef ]

Dataset refs to remove. Dataset refs have to be resolved.

collectionCollectionRecord

Collection record for a TAGGED collection.

Returns:

countint

Actual number of records removed from obscore table.

Notes

Dataset data types and collection names are checked against configured list of collections and dataset types, non-matching datasets are ignored and not added to the obscore table.

When configuration parameter collection_type is not “TAGGED”, this method should return immediately.

classmethod extensionName() → str

Return full name of the extension.

This name should match the name defined in registry configuration. It is also stored in registry attributes. Default implementation returns full class name.

Returns:

namestr

Full extension name.

abstract classmethod initialize(db: Database, context: StaticTablesContext, *, universe: DimensionUniverse, config: Mapping, datasets: type[DatasetRecordStorageManager], dimensions: DimensionRecordStorageManager, registry_schema_version: VersionTuple | None = None) → ObsCoreTableManager

Construct an instance of the manager.

Parameters:

dbDatabase

Interface to the underlying database engine and namespace.

contextStaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present in a layer implemented with this manager.

universeDimensionUniverse

All dimensions known to the registry.

configdict [ str, Any ]

Configuration of the obscore manager.

datasetstype

Type of dataset manager.

dimensions: `DimensionRecordStorageManager`

Manager for Registry dimensions.

registry_schema_versionVersionTuple or None

Schema version of this extension as defined in registry.

Returns:

managerObsCoreTableManager

An instance of a concrete ObsCoreTableManager subclass.

newSchemaVersion() → VersionTuple | None

Return schema version for newly created registry.

Returns:

versionVersionTuple or None

Schema version created by this extension. None is returned if an extension does not require its version to be saved or checked.

Notes

Extension classes that support multiple schema versions need to override _newDefaultSchemaVersion method.

abstract query(columns: Iterable[str | ColumnElement] | None = None, /, **kwargs: Any) → Iterator[CursorResult]

Run a SELECT query against obscore table and return result rows.

Parameters:

columnsIterable [str]

Columns to return from query. It is a sequence which can include column names or any other column elements (e.g. sqlalchemy.sql.functions.count function).

**kwargs

Restriction on values of individual obscore columns. Key is the column name, value is the required value of the column. Multiple restrictions are ANDed together.

Returns:

result_contextsqlalchemy.engine.CursorResult

Context manager that returns the query result object when entered. These results are invalidated when the context is exited.

abstract update_exposure_regions(instrument: str, region_data: Iterable[tuple[int, int, Region]]) → int

Update existing exposure records with spatial region data.

Parameters:

instrumentstr

Instrument name.

region_dataIterable [tuple [int, int, Region ]]

Sequence of tuples, each tuple contains three values - exposure ID, detector ID, and corresponding region.

Returns:

countint

Actual number of records updated.

Notes

This method is needed to update obscore records for raw exposures which are ingested before their corresponding visits are defined. Exposure records added when visit is already defined will get their regions from their matching visits automatically.