ButlerQuantumContext¶

class lsst.pipe.base.ButlerQuantumContext(*, limited: LimitedButler, quantum: Quantum, butler: Butler | None = None)¶

Bases: object

A Butler-like class specialized for a single quantum.

A ButlerQuantumContext class wraps a standard butler interface and specializes it to the context of a given quantum. What this means in practice is that the only gets and puts that this class allows are DatasetRefs that are contained in the quantum.

In the future this class will also be used to record provenance on what was actually get and put. This is in contrast to what the preflight expects to be get and put by looking at the graph before execution.

Do not use constructor directly, instead use from_full or from_limited factory methods.

Notes

ButlerQuantumContext instances are backed by either lsst.daf.butler.Butler or lsst.daf.butler.LimitedButler. When a limited butler is used then quantum has to contain dataset references that are completely resolved (usually when graph is constructed by GraphBuilder).

When instances are backed by full butler, the quantum graph does not have to resolve output or intermediate references, but input references of each quantum have to be resolved before they can be used by this class. When executing such graphs, intermediate references used as input to some Quantum are resolved by lsst.ctrl.mpexec.SingleQuantumExecutor. If output references of a quanta are resolved, they will be unresolved when full butler is used.

Attributes Summary

dimensions

Structure managing all dimensions recognized by this data repository (DimensionUniverse).

Methods Summary

`from_full`(butler, quantum)	Make ButlerQuantumContext backed by `lsst.daf.butler.Butler`.
`from_limited`(butler, quantum)	Make ButlerQuantumContext backed by `lsst.daf.butler.LimitedButler`.
`get`(dataset)	Fetches data from the butler
`put`(values, dataset)	Puts data into the butler

Attributes Documentation

dimensions¶: Structure managing all dimensions recognized by this data repository (DimensionUniverse).

Methods Documentation

classmethod from_full(butler: Butler, quantum: Quantum) → ButlerQuantumContext¶

Make ButlerQuantumContext backed by lsst.daf.butler.Butler.

Parameters:

butlerlsst.daf.butler.Butler: Butler object from/to which datasets will be get/put.
quantumlsst.daf.butler.core.Quantum: Quantum object that describes the datasets which will be get/put by a single execution of this node in the pipeline graph. All input dataset references must be resolved in this Quantum. Output references can be resolved, but they will be unresolved.

Returns:

butlerQCButlerQuantumContext: Instance of butler wrapper.

classmethod from_limited(butler: LimitedButler, quantum: Quantum) → ButlerQuantumContext¶

Make ButlerQuantumContext backed by lsst.daf.butler.LimitedButler.

Parameters:

butlerlsst.daf.butler.LimitedButler: Butler object from/to which datasets will be get/put.
quantumlsst.daf.butler.core.Quantum: Quantum object that describes the datasets which will be get/put by a single execution of this node in the pipeline graph. Both input and output dataset references must be resolved in this Quantum.

Returns:

butlerQCButlerQuantumContext: Instance of butler wrapper.

Fetches data from the butler

Parameters:

dataset: This argument may either be an InputQuantizedConnection which describes all the inputs of a quantum, a list of DatasetRef, or a single DatasetRef. The function will get and return the corresponding datasets from the butler. If None is passed in place of a DatasetRef then the corresponding returned object will be None.

Returns:

returnobject: This function returns arbitrary objects fetched from the bulter. The structure these objects are returned in depends on the type of the input argument. If the input dataset argument is a InputQuantizedConnection, then the return type will be a dictionary with keys corresponding to the attributes of the InputQuantizedConnection (which in turn are the attribute identifiers of the connections). If the input argument is of type list of DatasetRef then the return type will be a list of objects. If the input argument is a single DatasetRef then a single object will be returned.

Raises:

ValueError: Raised if a DatasetRef is passed to get that is not defined in the quantum object

put(values: Struct | List[Any] | Any, dataset: OutputQuantizedConnection | List[DatasetRef] | DatasetRef) → None¶

Puts data into the butler

Parameters:

valuesStruct or list of object or object: The data that should be put with the butler. If the type of the dataset is OutputQuantizedConnection then this argument should be a Struct with corresponding attribute names. Each attribute should then correspond to either a list of object or a single object depending of the type of the corresponding attribute on dataset. I.e. if dataset.calexp is [datasetRef1, datasetRef2] then values.calexp should be [calexp1, calexp2]. Like wise if there is a single ref, then only a single object need be passed. The same restriction applies if dataset is directly a list of DatasetRef or a single DatasetRef.
dataset: This argument may either be an InputQuantizedConnection which describes all the inputs of a quantum, a list of lsst.daf.butler.DatasetRef, or a single lsst.daf.butler.DatasetRef. The function will get and return the corresponding datasets from the butler.

Raises:

ValueError: Raised if a DatasetRef is passed to put that is not defined in the quantum object, or the type of values does not match what is expected from the type of dataset.

Navigation

ButlerQuantumContext¶