SqlQueryBackend

class lsst.daf.butler.registry.queries.SqlQueryBackend(db: Database, managers: RegistryManagerInstances, dimension_record_cache: DimensionRecordCache)

Bases: QueryBackend[SqlQueryContext]

An implementation of QueryBackend for SqlRegistry.

Parameters:

dbDatabase

Object that abstracts the database engine.

managersRegistryManagerInstances

Struct containing the manager objects that back a SqlRegistry.

dimension_record_cacheDimensionRecordCache

Cache of all records for dimension elements with is_cached True.

Attributes Summary

universe

Definition of all dimensions and dimension elements for this registry (DimensionUniverse).

Methods Summary

caching_context()	Enable caching of collection records and summaries for the duration of the returned context manager.
context()	Return a context manager that can be used to execute queries with this backend.
filter_dataset_collections(dataset_types, ...)	Filter a sequence of collections to those for which a dataset query might succeed.
get_collection_name(key)	Return the collection name associated with a collection primary key value.
get_dimension_record_cache(element_name)	Return a local cache of all DimensionRecord objects for a dimension element, fetching it if necessary.
make_dimension_relation(dimensions, columns, ...)	Construct a relation that provides columns and constraints from dimension records.
resolve_collection_wildcard(expression, *[, ...])	Return the collection records that match a wildcard expression.
resolve_dataset_type_wildcard(expression[, ...])	Return the dataset types that match a wildcard expression.
resolve_governor_constraints(dimensions, ...)	Resolve governor dimension constraints provided by user input to a query against the content in the Registry.

Attributes Documentation

universe

Methods Documentation

caching_context() → AbstractContextManager[None]

Enable caching of collection records and summaries for the duration of the returned context manager.

context() → SqlQueryContext

Return a context manager that can be used to execute queries with this backend.

Returns:

contextQueryContext

Context manager that manages state and connections needed to execute queries.

filter_dataset_collections(dataset_types: Iterable[DatasetType], collections: Sequence[CollectionRecord], *, governor_constraints: Mapping[str, Set[str]], rejections: list[str] | None = None) → dict[lsst.daf.butler._dataset_type.DatasetType, list[lsst.daf.butler.registry.interfaces._collections.CollectionRecord]]

Filter a sequence of collections to those for which a dataset query might succeed.

Parameters:

dataset_typesIterable [ DatasetType ]

Dataset types that are being queried. Must include only parent or standalone dataset types, not components.

collectionsSequence [ CollectionRecord ]

Sequence of collections that will be searched.

governor_constraintsMapping [ str, Set [ str ] ], optional

Constraints imposed by other aspects of the query on governor dimensions; collections inconsistent with these constraints will be skipped.

rejectionslist [ str ], optional

If not None, a list that diagnostic messages will be appended to, for any collection that matches collections that is not returned. At least one message is guaranteed whenever the result is empty.

Returns:

dataset_collectionsdict [ DatasetType, list [ CollectionRecord ] ]

The collections to search for each dataset. The dictionary’s keys are always exactly dataset_types (in the same order), and each nested list of collections is ordered consistently with the given collections.

Notes

This method accepts multiple dataset types and multiple collections at once to enable implementations to batch up the fetching of summary information needed to relate them.

get_collection_name(key: Any) → str

Return the collection name associated with a collection primary key value.

Parameters:

keyAny

Collection primary key value.

Returns:

namestr

Collection name.

get_dimension_record_cache(element_name: str) → DimensionRecordSet | None

Return a local cache of all DimensionRecord objects for a dimension element, fetching it if necessary.

Parameters:

element_namestr

Name of the dimension element.

Returns:

cacheMapping [ DataCoordinate, DimensionRecord ] or None

Mapping from data ID to dimension record, or None if this element’s records are never cached.

make_dimension_relation(dimensions: DimensionGroup, columns: Set[ColumnTag], context: SqlQueryContext, *, initial_relation: Relation | None = None, initial_join_max_columns: frozenset[lsst.daf.relation._columns._tag.ColumnTag] | None = None, initial_dimension_relationships: Set[frozenset[str]] | None = None, spatial_joins: Iterable[tuple[str, str]] = (), governor_constraints: Mapping[str, Set[str]]) → Relation

Construct a relation that provides columns and constraints from dimension records.

Parameters:

dimensionsDimensionGroup

Dimensions to include. The key columns for all dimensions (both required and implied) will be included in the returned relation.

columnsSet [ ColumnTag ]

Dimension record columns to include. This set may include key column tags as well, though these may be ignored; the set of key columns to include is determined by the dimensions argument instead.

contextQueryContext

Context that manages per-query state.

initial_relationRelation, optional

Initial relation to join to the dimension relations. If this relation provides record columns, key columns, and relationships between key columns (see initial_dimension_relationships below) that would otherwise have been added by joining in a dimension element’s relation, that relation may not be joined in at all.

initial_join_max_columnsfrozenset [ ColumnTag ], optional

Maximum superset of common columns for joins to initial_relation (i.e. columns in the ON expression of SQL JOIN clauses). If provided, this is a subset of the dimension key columns in initial_relation, which are otherwise all considered as potential common columns for joins. Ignored if initial_relation is not provided.

initial_dimension_relationshipsSet [ frozenset [ str ] ], optional

A set of sets of dimension names representing relationships between dimensions encoded in the rows of initial_relation. If not provided (and initial_relation is), extract_dimension_relationships will be called on initial_relation.

spatial_joinscollections.abc.Iterable [ tuple [ str, str ] ]

Iterable of dimension element name pairs that should be spatially joined.

governor_constraintsMapping [ str [ Set [ str ] ] ], optional

Constraints on governor dimensions that are provided by other parts of the query that either have been included in initial_relation or are guaranteed to be added in the future. This is a mapping from governor dimension name to sets of values that dimension may take.

Returns:

relationlsst.daf.relation.Relation

Relation containing the given dimension columns and constraints.

resolve_collection_wildcard(expression: Any, *, collection_types: Set[CollectionType] = frozenset({CollectionType.RUN, CollectionType.TAGGED, CollectionType.CHAINED, CollectionType.CALIBRATION}), done: set[str] | None = None, flatten_chains: bool = True, include_chains: bool | None = None) → list[lsst.daf.butler.registry.interfaces._collections.CollectionRecord]

Return the collection records that match a wildcard expression.

Parameters:

expressionAny

Names and/or patterns for collections; will be passed to CollectionWildcard.from_expression.

collection_typescollections.abc.Set [ CollectionType ], optional

If provided, only yield collections of these types.

doneset [ str ], optional

A set of collection names that should be skipped, updated to include all processed collection names on return.

flatten_chainsbool, optional

If True (default) recursively yield the child collections of CHAINED collections.

include_chainsbool, optional

If False, return records for CHAINED collections themselves. The default is the opposite of flattenChains: either return records for CHAINED collections or their children, but not both.

Returns:

recordslist [ CollectionRecord ]

Matching collection records.

resolve_dataset_type_wildcard(expression: Any, missing: list[str] | None = None, explicit_only: bool = False) → list[lsst.daf.butler._dataset_type.DatasetType]

Return the dataset types that match a wildcard expression.

Parameters:

expressionAny

Names and/or patterns for dataset types; will be passed to DatasetTypeWildcard.from_expression.

missinglist of str, optional

String dataset type names that were explicitly given (i.e. not regular expression patterns) but not found will be appended to this list, if it is provided.

explicit_onlybool, optional

If True, require explicit DatasetType instances or str names, with re.Pattern instances deprecated and ... prohibited.

Returns:

dataset_typeslist [ DatasetType ]

A list of resolved dataset types.

resolve_governor_constraints(dimensions: DimensionGroup, constraints: Mapping[str, Set[str]]) → Mapping[str, Set[str]]

Resolve governor dimension constraints provided by user input to a query against the content in the Registry.

Parameters:

dimensionsDimensionGroup

Dimensions that bound the governor dimensions to consider (via dimensions.governors, more specifically).

constraintsMapping [ str, Set [ str ] ]

Constraints from user input to the query (e.g. from data IDs and string expression predicates).

Returns:

resolvedMapping [ str, Set [ str ] ]

A shallow copy of constraints with keys equal to dimensions.governors.names and value sets constrained by the Registry content if they were not already in constraints.

Raises:

DataIdValueError

Raised if constraints includes governor dimension values that are not present in the Registry.