LimitedButler¶

class lsst.daf.butler.LimitedButler¶

Bases: ABC

A minimal butler interface that is sufficient to back PipelineTask execution.

Attributes Summary

`GENERATION`	This is a Generation 3 Butler.
`dimensions`	Structure managing all dimensions recognized by this data repository (`DimensionUniverse`).

Methods Summary

`datasetExistsDirect`(ref)	Return `True` if a dataset is actually present in the Datastore.
`get`(ref, /, *[, parameters, storageClass])	Retrieve a stored dataset.
`getDeferred`(ref, /, *[, parameters, ...])	Create a `DeferredDatasetHandle` which can later retrieve a dataset, after an immediate registry lookup.
`getDirect`(ref, *[, parameters, storageClass])	Retrieve a stored dataset.
`getDirectDeferred`(ref, *[, parameters, ...])	Create a `DeferredDatasetHandle` which can later retrieve a dataset,
`isWriteable`()	Return `True` if this `Butler` supports write operations.
`markInputUnused`(ref)	Indicate that a predicted input was not actually used when processing a `Quantum`.
`pruneDatasets`(refs, *[, disassociate, ...])	Remove one or more datasets from a collection and/or storage.
`put`(obj, ref, /)	Store a dataset that already has a UUID and `RUN` collection.
`putDirect`(obj, ref, /)	Store a dataset that already has a UUID and `RUN` collection.

Attributes Documentation

GENERATION: ClassVar[int] = 3¶

This is a Generation 3 Butler.

This attribute may be removed in the future, once the Generation 2 Butler interface has been fully retired; it should only be used in transitional code.

dimensions¶: Structure managing all dimensions recognized by this data repository (DimensionUniverse).

Methods Documentation

datasetExistsDirect(ref: DatasetRef) → bool¶

Return True if a dataset is actually present in the Datastore.

Parameters:

refDatasetRef: Resolved reference to a dataset.

Returns:

existsbool: Whether the dataset exists in the Datastore.

get(ref: DatasetRef, /, *, parameters: dict[str, Any] | None = None, storageClass: StorageClass | str | None = None) → Any¶

Retrieve a stored dataset.

Parameters:

ref: `DatasetRef`: A resolved DatasetRef directly associated with a dataset.
parametersdict: Additional StorageClass-defined options to control reading, typically used to efficiently read only a subset of the dataset.
storageClassStorageClass or str, optional: The storage class to be used to override the Python type returned by this method. By default the returned type matches the dataset type definition for this dataset. Specifying a read StorageClass can force a different type to be returned. This type must be compatible with the original type.

Returns:

objobject: The dataset.

Raises:

AmbiguousDatasetError: Raised if the supplied DatasetRef is unresolved.

Notes

In a LimitedButler the only allowable way to specify a dataset is to use a resolved DatasetRef. Subclasses can support more options.

getDeferred(ref: DatasetRef, /, *, parameters: dict[str, Any] | None = None, storageClass: StorageClass | str | None = None) → DeferredDatasetHandle¶

Create a DeferredDatasetHandle which can later retrieve a dataset, after an immediate registry lookup.

Parameters:

refDatasetRef: For the default implementation of a LimitedButler, the only acceptable parameter is a resolved DatasetRef.
parametersdict: Additional StorageClass-defined options to control reading, typically used to efficiently read only a subset of the dataset.
storageClassStorageClass or str, optional: The storage class to be used to override the Python type returned by this method. By default the returned type matches the dataset type definition for this dataset. Specifying a read StorageClass can force a different type to be returned. This type must be compatible with the original type.

Returns:

objDeferredDatasetHandle: A handle which can be used to retrieve a dataset at a later time.

Raises:

AmbiguousDatasetError: Raised if an unresolved DatasetRef is passed as an input.

Notes

In a LimitedButler the only allowable way to specify a dataset is to use a resolved DatasetRef. Subclasses can support more options.

getDirect(ref: DatasetRef, *, parameters: Dict[str, Any] | None = None, storageClass: StorageClass | str | None = None) → Any¶

Retrieve a stored dataset.

Parameters:

refDatasetRef: Resolved reference to an already stored dataset.
parametersdict: Additional StorageClass-defined options to control reading, typically used to efficiently read only a subset of the dataset.
storageClassStorageClass or str, optional: The storage class to be used to override the Python type returned by this method. By default the returned type matches the dataset type definition for this dataset. Specifying a read StorageClass can force a different type to be returned. This type must be compatible with the original type.

Returns:

objobject: The dataset.

Raises:

AmbiguousDatasetError: Raised if ref.id is None, i.e. the reference is unresolved.

Deprecated since version v26.0: Butler.get() now behaves like Butler.getDirect() when given a DatasetRef. Please use Butler.get(). Will be removed after v27.0.

getDirectDeferred(ref: DatasetRef, *, parameters: dict | None = None, storageClass: StorageClass | str | None = None) → DeferredDatasetHandle¶

Create a DeferredDatasetHandle which can later retrieve a dataset,: from a resolved DatasetRef.

Parameters:

refDatasetRef: Resolved reference to an already stored dataset.
parametersdict: Additional StorageClass-defined options to control reading, typically used to efficiently read only a subset of the dataset.
storageClassStorageClass or str, optional: The storage class to be used to override the Python type returned by this method. By default the returned type matches the dataset type definition for this dataset. Specifying a read StorageClass can force a different type to be returned. This type must be compatible with the original type.

Returns:

objDeferredDatasetHandle: A handle which can be used to retrieve a dataset at a later time.

Raises:

AmbiguousDatasetError: Raised if ref.id is None, i.e. the reference is unresolved.

Deprecated since version v26.0: Butler.getDeferred() now behaves like getDirectDeferred() when given a DatasetRef. Please use Butler.getDeferred(). Will be removed after v27.0.

abstract isWriteable() → bool¶: Return True if this Butler supports write operations.

markInputUnused(ref: DatasetRef) → None¶

Indicate that a predicted input was not actually used when processing a Quantum.

Parameters:

refDatasetRef: Reference to the unused dataset.

Notes

By default, a dataset is considered “actually used” if it is accessed via getDirect or a handle to it is obtained via getDirectDeferred (even if the handle is not used). This method must be called after one of those in order to remove the dataset from the actual input list.

This method does nothing for butlers that do not store provenance information (which is the default implementation provided by the base class).

abstract pruneDatasets(refs: Iterable[DatasetRef], *, disassociate: bool = True, unstore: bool = False, tags: Iterable[str] = (), purge: bool = False) → None¶

Remove one or more datasets from a collection and/or storage.

Parameters:

refsIterable of DatasetRef

Datasets to prune. These must be “resolved” references (not just a DatasetType and data ID).

disassociatebool, optional

Disassociate pruned datasets from tags, or from all collections if purge=True.

unstorebool, optional

If True (False is default) remove these datasets from all datastores known to this butler. Note that this will make it impossible to retrieve these datasets even via other collections. Datasets that are already not stored are ignored by this option.

tagsIterable [ str ], optional

TAGGED collections to disassociate the datasets from. Ignored if disassociate is False or purge is True.

purgebool, optional

If True (False is default), completely remove the dataset from the Registry. To prevent accidental deletions, purge may only be True if all of the following conditions are met:

disassociate is True;

unstore is True.

This mode may remove provenance information from datasets other than those provided, and should be used with extreme care.

Raises:

TypeError: Raised if the butler is read-only, if no collection was provided, or the conditions for purge=True were not met.

abstract put(obj: Any, ref: DatasetRef, /) → DatasetRef¶

Store a dataset that already has a UUID and RUN collection.

Parameters:

objobject: The dataset.
refDatasetRef: Resolved reference for a not-yet-stored dataset.

Returns:

refDatasetRef: The same as the given, for convenience and symmetry with Butler.put.

Raises:

TypeError: Raised if the butler is read-only.
AmbiguousDatasetError: Raised if ref.id is None, i.e. the reference is unresolved.

Notes

Whether this method inserts the given dataset into a Registry is implementation defined (some LimitedButler subclasses do not have a Registry), but it always adds the dataset to a Datastore, and the given ref.id and ref.run are always preserved.

putDirect(obj: Any, ref: DatasetRef, /) → DatasetRef¶

Store a dataset that already has a UUID and RUN collection.

Parameters:

objobject: The dataset.
refDatasetRef: Resolved reference for a not-yet-stored dataset.

Returns:

refDatasetRef: The same as the given, for convenience and symmetry with Butler.put.

Raises:

TypeError: Raised if the butler is read-only.
AmbiguousDatasetError: Raised if ref.id is None, i.e. the reference is unresolved.

Deprecated since version v26.0: Butler.put() now behaves like Butler.putDirect() when given a DatasetRef. Please use Butler.put(). Will be removed after v27.0.

Navigation

LimitedButler¶