ButlerCollections

class lsst.daf.butler.ButlerCollections

Bases: ABC, Sequence

Methods for working with collections stored in the Butler.

Attributes Summary

defaults

Collection defaults associated with this butler.

Methods Summary

extend_chain(parent_collection_name, ...)	Add children to the end of a CHAINED collection.
get_info(name[, include_parents, ...])	Obtain information for a specific collection.
prepend_chain(parent_collection_name, ...)	Add children to the beginning of a CHAINED collection.
query(expression[, collection_types, ...])	Query the butler for collections matching an expression.
query_info(expression[, collection_types, ...])	Query the butler for collections matching an expression and return detailed information about those collections.
redefine_chain(parent_collection_name, ...)	Replace the contents of a CHAINED collection with new children.
register(name[, type, doc])	Add a new collection if one with the given name does not exist.
remove_from_chain(parent_collection_name, ...)	Remove children from a CHAINED collection.
x_remove(name)	Remove the given collection from the registry.

Attributes Documentation

defaults

Collection defaults associated with this butler.

Methods Documentation

abstract extend_chain(parent_collection_name: str, child_collection_names: str | Iterable[str]) → None

Add children to the end of a CHAINED collection.

If any of the children already existed in the chain, they will be moved to the new position at the end of the chain.

Parameters:

parent_collection_namestr

The name of a CHAINED collection to which we will add new children.

child_collection_namesIterable [ str ] | str

A child collection name or list of child collection names to be added to the parent.

Raises:

MissingCollectionError

If any of the specified collections do not exist.

CollectionTypeError

If the parent collection is not a CHAINED collection.

CollectionCycleError

If this operation would create a collection cycle.

Notes

If this function is called within a call to Butler.transaction, it will hold a lock that prevents other processes from modifying the parent collection until the end of the transaction. Keep these transactions short.

abstract get_info(name: str, include_parents: bool = False, include_summary: bool = False) → CollectionInfo

Obtain information for a specific collection.

Parameters:

namestr

The name of the collection of interest.

include_parentsbool, optional

If True any parents of this collection will be included.

include_summarybool, optional

If True dataset type names and governor dimensions of datasets stored in this collection will be included in the result.

Returns:

infoCollectionInfo

Information on the requested collection.

abstract prepend_chain(parent_collection_name: str, child_collection_names: str | Iterable[str]) → None

Add children to the beginning of a CHAINED collection.

If any of the children already existed in the chain, they will be moved to the new position at the beginning of the chain.

Parameters:

parent_collection_namestr

The name of a CHAINED collection to which we will add new children.

child_collection_namesIterable [ str ] | str

A child collection name or list of child collection names to be added to the parent.

Raises:

MissingCollectionError

If any of the specified collections do not exist.

CollectionTypeError

If the parent collection is not a CHAINED collection.

CollectionCycleError

If this operation would create a collection cycle.

Notes

If this function is called within a call to Butler.transaction, it will hold a lock that prevents other processes from modifying the parent collection until the end of the transaction. Keep these transactions short.

query(expression: str | Iterable[str], collection_types: Set[CollectionType] | CollectionType | None = None, flatten_chains: bool = False, include_chains: bool | None = None) → Sequence[str]

Query the butler for collections matching an expression.

Parameters:

expressionstr or Iterable [ str ]

One or more collection names or globs to include in the search.

collection_typesset [CollectionType], CollectionType or None

Restrict the types of collections to be searched. If None all collection types are searched.

flatten_chainsbool, optional

If True (False is default), recursively yield the child collections of matching CHAINED collections.

include_chainsbool or None, optional

If True, yield records for matching CHAINED collections. Default is the opposite of flatten_chains: include either CHAINED collections or their children, but not both.

Returns:

collectionsSequence [ str ]

The names of collections that match expression.

Notes

The order in which collections are returned is unspecified, except that the children of a CHAINED collection are guaranteed to be in the order in which they are searched. When multiple parent CHAINED collections match the same criteria, the order in which the two lists appear is unspecified, and the lists of children may be incomplete if a child has multiple parents.

The default implementation is a wrapper around x_query_info.

abstract query_info(expression: str | Iterable[str], collection_types: Set[CollectionType] | CollectionType | None = None, flatten_chains: bool = False, include_chains: bool | None = None, include_parents: bool = False, include_summary: bool = False, include_doc: bool = False, summary_datasets: Iterable[DatasetType] | Iterable[str] | None = None) → Sequence[CollectionInfo]

Query the butler for collections matching an expression and return detailed information about those collections.

Parameters:

expressionstr or Iterable [ str ]

One or more collection names or globs to include in the search.

collection_typesset [CollectionType], CollectionType or None

Restrict the types of collections to be searched. If None all collection types are searched.

flatten_chainsbool, optional

If True (False is default), recursively yield the child collections of matching CHAINED collections.

include_chainsbool or None, optional

If True, yield records for matching CHAINED collections. Default is the opposite of flatten_chains: include either CHAINED collections or their children, but not both.

include_parentsbool, optional

Whether the returned information includes parents.

include_summarybool, optional

Whether the returned information includes dataset type and governor information for the collections.

include_docbool, optional

Whether the returned information includes collection documentation string.

summary_datasetsIterable [ DatasetType ] or Iterable [ str ], optional

Dataset types to include in returned summaries. Only used if include_summary is True. If not specified then all dataset types will be included.

Returns:

collectionsSequence [ CollectionInfo ]

The names of collections that match expression.

Notes

The order in which collections are returned is unspecified, except that the children of a CHAINED collection are guaranteed to be in the order in which they are searched. When multiple parent CHAINED collections match the same criteria, the order in which the two lists appear is unspecified, and the lists of children may be incomplete if a child has multiple parents.

abstract redefine_chain(parent_collection_name: str, child_collection_names: str | Iterable[str]) → None

Replace the contents of a CHAINED collection with new children.

Parameters:

parent_collection_namestr

The name of a CHAINED collection to which we will assign new children.

child_collection_namesIterable [ str ] | str

A child collection name or list of child collection names to be added to the parent.

Raises:

MissingCollectionError

If any of the specified collections do not exist.

CollectionTypeError

If the parent collection is not a CHAINED collection.

CollectionCycleError

If this operation would create a collection cycle.

Notes

If this function is called within a call to Butler.transaction, it will hold a lock that prevents other processes from modifying the parent collection until the end of the transaction. Keep these transactions short.

abstract register(name: str, type: CollectionType = CollectionType.RUN, doc: str | None = None) → bool

Add a new collection if one with the given name does not exist.

Parameters:

namestr

The name of the collection to create.

typeCollectionType, optional

Enum value indicating the type of collection to create. Default is to create a RUN collection.

docstr, optional

Documentation string for the collection.

Returns:

registeredbool

Boolean indicating whether the collection was already registered or was created by this call.

Notes

Avoid calling this method multiple times within a Butler.transaction. If concurrent processes register the same collection names, they may block each other until the end of the transaction and in some cases the database will be required to abort one of the transactions to prevent deadlock.

abstract remove_from_chain(parent_collection_name: str, child_collection_names: str | Iterable[str]) → None

Remove children from a CHAINED collection.

Parameters:

parent_collection_namestr

The name of a CHAINED collection from which we will remove children.

child_collection_namesIterable [ str ] | str

A child collection name or list of child collection names to be removed from the parent.

Raises:

MissingCollectionError

If any of the specified collections do not exist.

CollectionTypeError

If the parent collection is not a CHAINED collection.

Notes

If this function is called within a call to Butler.transaction, it will hold a lock that prevents other processes from modifying the parent collection until the end of the transaction. Keep these transactions short.

abstract x_remove(name: str) → None

Remove the given collection from the registry.

This is an experimental interface that can change at any time.

Parameters:

namestr

The name of the collection to remove.

Raises:

lsst.daf.butler.registry.MissingCollectionError

Raised if no collection with the given name exists.

lsst.daf.butler.registry.OrphanedRecordError

Raised if the database rows associated with the collection are still referenced by some other table, such as a dataset in a datastore (for RUN collections only) or a CHAINED collection of which this collection is a child.

Notes

If this is a RUN collection, all datasets and quanta in it will removed from the Registry database. This requires that those datasets be removed (or at least trashed) from any datastores that hold them first.

A collection may not be deleted as long as it is referenced by a CHAINED collection; the CHAINED collection must be deleted or redefined first.