CollectionManager

class lsst.daf.butler.registry.interfaces.CollectionManager

Bases: lsst.daf.butler.registry.interfaces.VersionedExtension

An interface for managing the collections (including runs) in a Registry.

Notes

Each layer in a multi-layer Registry has its own record for any collection for which it has datasets (or quanta). Different layers may use different IDs for the same collection, so any usage of the IDs obtained through the CollectionManager APIs are strictly for internal (to Registry) use.

Methods Summary

addCollectionForeignKey(tableSpec, *, …) Add a foreign key (field and constraint) referencing the collection table.
addRunForeignKey(tableSpec, *, prefix, …) Add a foreign key (field and constraint) referencing the run table.
currentVersion() Return extension version as defined by current implementation.
extensionName() Return full name of the extension.
find(name) Return the collection record associated with the given name.
getCollectionForeignKeyName(prefix) Return the name of the field added by addCollectionForeignKey if called with the same prefix.
getDocumentation(key) Retrieve the documentation string for a collection.
getParentChains(key) Find all CHAINED collections that directly contain the given collection.
getRunForeignKeyName(prefix) Return the name of the field added by addRunForeignKey if called with the same prefix.
initialize(db, context, *, dimensions) Construct an instance of the manager.
refresh() Ensure all other operations on this manager are aware of any collections that may have been registered by other clients since it was initialized or last refreshed.
register(name, type, doc, None] = None) Ensure that a collection of the given name and type are present in the layer this manager is associated with.
remove(name) Completely remove a collection.
schemaDigest() Return digest for schema piece managed by this extension.
setDocumentation(key, doc, None]) Set the documentation string for a collection.

Methods Documentation

classmethod addCollectionForeignKey(tableSpec: lsst.daf.butler.core.ddl.TableSpec, *, prefix: str = 'collection', onDelete: Optional[str, None] = None, constraint: bool = True, **kwargs) → lsst.daf.butler.core.ddl.FieldSpec

Add a foreign key (field and constraint) referencing the collection table.

Parameters:
tableSpec : ddl.TableSpec

Specification for the table that should reference the collection table. Will be modified in place.

prefix: `str`, optional

A name to use for the prefix of the new field; the full name may have a suffix (and is given in the returned ddl.FieldSpec).

onDelete: `str`, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

constraint: `bool`, optional

If False (True is default), add a field that can be joined to the collection primary key, but do not add a foreign key constraint.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).
Returns:
fieldSpec : ddl.FieldSpec

Specification for the field being added.
classmethod addRunForeignKey(tableSpec: lsst.daf.butler.core.ddl.TableSpec, *, prefix: str = 'run', onDelete: Optional[str, None] = None, constraint: bool = True, **kwargs) → lsst.daf.butler.core.ddl.FieldSpec

Add a foreign key (field and constraint) referencing the run table.

Parameters:
tableSpec : ddl.TableSpec

Specification for the table that should reference the run table. Will be modified in place.

prefix: `str`, optional

A name to use for the prefix of the new field; the full name may have a suffix (and is given in the returned ddl.FieldSpec).

onDelete: `str`, optional

One of “CASCADE” or “SET NULL”, indicating what should happen to the referencing row if the collection row is deleted. None indicates that this should be an integrity error.

constraint: `bool`, optional

If False (True is default), add a field that can be joined to the run primary key, but do not add a foreign key constraint.

**kwargs

Additional keyword arguments are forwarded to the ddl.FieldSpec constructor (only the name and dtype arguments are otherwise provided).
Returns:
fieldSpec : ddl.FieldSpec

Specification for the field being added.
classmethod currentVersion() → Optional[lsst.daf.butler.registry.interfaces._versioning.VersionTuple, None]

Return extension version as defined by current implementation.

This method can return None if an extension does not require its version to be saved or checked.

Returns:
version : VersionTuple

Current extension version or None.
classmethod extensionName() → str

Return full name of the extension.

This name should match the name defined in registry configuration. It is also stored in registry attributes. Default implementation returns full class name.

Returns:
name : str

Full extension name.
find(name: str) → lsst.daf.butler.registry.interfaces._collections.CollectionRecord

Return the collection record associated with the given name.

Parameters:
name : str

Name of the collection.
Returns:
record : CollectionRecord

Object representing the collection, including its type and ID. If record.type is CollectionType.RUN, this will be a RunRecord instance. If record.type is CollectionType.CHAIN, this will be a ChainedCollectionRecord instance.
Raises:
MissingCollectionError

Raised if the given collection does not exist.

Notes

Collections registered by another client of the same layer since the last call to initialize or refresh may not be found.

classmethod getCollectionForeignKeyName(prefix: str = 'collection') → str

Return the name of the field added by addCollectionForeignKey if called with the same prefix.

Parameters:
prefix : str

A name to use for the prefix of the new field; the full name may have a suffix.
Returns:
name : str

The field name.
getDocumentation(key: Any) → Optional[str, None]

Retrieve the documentation string for a collection.

Parameters:
key

Internal primary key value for the collection.
Returns:
docs : str or None

Docstring for the collection with the given key.
getParentChains(key: Any) → Iterator[lsst.daf.butler.registry.interfaces._collections.ChainedCollectionRecord]

Find all CHAINED collections that directly contain the given collection.

Parameters:
key

Internal primary key value for the collection.
classmethod getRunForeignKeyName(prefix: str = 'run') → str

Return the name of the field added by addRunForeignKey if called with the same prefix.

Parameters:
prefix : str

A name to use for the prefix of the new field; the full name may have a suffix.
Returns:
name : str

The field name.
classmethod initialize(db: Database, context: StaticTablesContext, *, dimensions: DimensionRecordStorageManager) → CollectionManager

Construct an instance of the manager.

Parameters:
db : Database

Interface to the underlying database engine and namespace.

context : StaticTablesContext

Context object obtained from Database.declareStaticTables; used to declare any tables that should always be present in a layer implemented with this manager.

dimensions : DimensionRecordStorageManager

Manager object for the dimensions in this Registry.
Returns:
manager : CollectionManager

An instance of a concrete CollectionManager subclass.
refresh() → None

Ensure all other operations on this manager are aware of any collections that may have been registered by other clients since it was initialized or last refreshed.

register(name: str, type: lsst.daf.butler.registry._collectionType.CollectionType, doc: Optional[str, None] = None) → Tuple[lsst.daf.butler.registry.interfaces._collections.CollectionRecord, bool]

Ensure that a collection of the given name and type are present in the layer this manager is associated with.

Parameters:
name : str

Name of the collection.

type : CollectionType

Enumeration value indicating the type of collection.

doc : str, optional

Documentation string for the collection. Ignored if the collection already exists.
Returns:
record : CollectionRecord

Object representing the collection, including its type and ID. If type is CollectionType.RUN, this will be a RunRecord instance. If type is CollectionType.CHAIN, this will be a ChainedCollectionRecord instance.

registered : bool

True if the collection was registered, False if it already existed.
Raises:
TransactionInterruption

Raised if this operation is invoked within a Database.transaction context.

DatabaseConflictError

Raised if a collection with this name but a different type already exists.

Notes

Concurrent registrations of the same collection should be safe; nothing should happen if the types are consistent, and integrity errors due to inconsistent types should happen before any database changes are made.

remove(name: str) → None

Completely remove a collection.

Any existing CollectionRecord objects that correspond to the removed collection are considered invalidated.

Parameters:
name : str

Name of the collection to remove.

Notes

If this collection is referenced by foreign keys in tables managed by other objects, the ON DELETE clauses of those tables will be invoked. That will frequently delete many dependent rows automatically (via “CASCADE”, but it may also cause this operation to fail (with rollback) unless dependent rows that do not have an ON DELETE clause are removed first.

schemaDigest() → Optional[str, None]

Return digest for schema piece managed by this extension.

Returns:
digest : str or None

String representation of the digest of the schema, None should be returned if schema digest is not to be saved or checked. The length of the returned string cannot exceed the length of the “value” column of butler attributes table, currently 65535 characters.

Notes

There is no exact definition of digest format, any string should work. The only requirement for string contents is that it has to remain stable over time if schema does not change but it should produce different string for any change in the schema. In many cases default implementation in _defaultSchemaDigest can be used as a reasonable choice.

setDocumentation(key: Any, doc: Optional[str, None]) → None

Set the documentation string for a collection.

Parameters:
key

Internal primary key value for the collection.

docs : str, optional

Docstring for the collection with the given key.