CollectionSearch

class lsst.daf.butler.registry.CollectionSearch(items: List[Tuple[str, lsst.daf.butler.registry.wildcards.DatasetTypeRestriction]])

Bases: object

An ordered search path of collections and dataset type restrictions.

The fromExpression method should almost always be used to construct instances, as the regular constructor performs no checking of inputs (and that can lead to confusing error messages downstream).

Parameters:

items : list [ tuple [ str, DatasetTypeRestriction ] ]

Tuples that relate a collection name to the restriction on dataset types to search for within it. This is not a mapping because the same collection name may appear multiple times with different restrictions.

Notes

A CollectionSearch is used to find a single dataset according to its dataset type and data ID, giving preference to collections in which the order they are specified. A CollectionQuery can be constructed from a broader range of expressions but does not order the collections to be searched.

CollectionSearch is iterable, yielding two-element tuples of str (collection name) and DatasetTypeRestriction.

A CollectionSearch instance constructed properly (e.g. via fromExpression) is a unique representation of a particular search path; it is exactly the same internally and compares as equal to any CollectionSearch constructed from an equivalent expression, regardless of how different the original expressions appear.

Methods Summary

fromExpression(expression)	Process a general expression to construct a CollectionSearch instance.
iter(manager, *, datasetType, …)	Iterate over collection records that match this instance and the given criteria, in order.
iterPairs(manager, *, datasetType, …)	Like iter, but yield pairs of CollectionRecord, DatasetTypeRestriction instead of just the former.

Methods Documentation

classmethod fromExpression(expression: Any) → lsst.daf.butler.registry.wildcards.CollectionSearch

Process a general expression to construct a CollectionSearch instance.

Parameters:

expression

May be:

• a str collection name;
• a two-element tuple containing a str and any expression accepted by DatasetTypeRestriction.fromExpression;
• any non-mapping iterable containing either of the above;
• a mapping from str to any expression accepted by DatasetTypeRestriction.
• another CollectionSearch instance (passed through unchanged).

Multiple consecutive entries for the same collection with different restrictions will be merged. Non-consecutive entries will not, because that actually represents a different search path.

Returns:

collections : CollectionSearch

A CollectionSearch instance.

iter(manager: CollectionManager, *, datasetType: Optional[DatasetType] = None, collectionTypes: AbstractSet[CollectionType] = frozenset({<CollectionType.RUN: 1>, <CollectionType.TAGGED: 2>, <CollectionType.CHAINED: 3>, <CollectionType.CALIBRATION: 4>}), done: Optional[Set[str]] = None, flattenChains: bool = True, includeChains: Optional[bool] = None) → Iterator[CollectionRecord]

Iterate over collection records that match this instance and the given criteria, in order.

This method is primarily intended for internal use by Registry; other callers should generally prefer Registry.findDatasets or other Registry query methods.

Parameters:

manager : CollectionManager

Object responsible for managing the collection tables in a Registry.

datasetType : DatasetType, optional

If given, only yield collections whose dataset type restrictions include this dataset type.

collectionTypes : AbstractSet [ CollectionType ], optional

If provided, only yield collections of these types.

done : set, optional

A set containing the names of all collections already yielded; any collections whose names are already present in this set will not be yielded again, and those yielded will be added to it while iterating. If not provided, an empty set will be created and used internally to avoid duplicates.

flattenChains : bool, optional

If True (default) recursively yield the child collections of CHAINED collections.

includeChains : bool, optional

If False, return records for CHAINED collections themselves. The default is the opposite of flattenChains: either return records for CHAINED collections or their children, but not both.

Yields:

record : CollectionRecord

Matching collection records.

iterPairs(manager: CollectionManager, *, datasetType: Optional[DatasetType] = None, collectionTypes: AbstractSet[CollectionType] = frozenset({<CollectionType.RUN: 1>, <CollectionType.TAGGED: 2>, <CollectionType.CHAINED: 3>, <CollectionType.CALIBRATION: 4>}), done: Optional[Set[str]] = None, flattenChains: bool = True, includeChains: Optional[bool] = None) → Iterator[Tuple[CollectionRecord, DatasetTypeRestriction]]

Like iter, but yield pairs of CollectionRecord, DatasetTypeRestriction instead of just the former.

See iter for all parameter descriptions.

Yields:

record : CollectionRecord

Matching collection records.

restriction : DatasetTypeRestriction

The given dataset type restriction.