ButlerQuantumContext

class lsst.pipe.base.ButlerQuantumContext(*, limited: lsst.daf.butler._limited_butler.LimitedButler, quantum: lsst.daf.butler.core.quantum.Quantum, butler: Optional[lsst.daf.butler._butler.Butler, None] = None)

Bases: object

A Butler-like class specialized for a single quantum.

A ButlerQuantumContext class wraps a standard butler interface and specializes it to the context of a given quantum. What this means in practice is that the only gets and puts that this class allows are DatasetRefs that are contained in the quantum.

In the future this class will also be used to record provenance on what was actually get and put. This is in contrast to what the preflight expects to be get and put by looking at the graph before execution.

Do not use constructor directly, instead use from_full or from_limited factory methods.

Notes

ButlerQuantumContext instances are backed by either lsst.daf.butler.Butler or lsst.daf.butler.LimitedButler. When a limited butler is used then quantum has to contain dataset references that are completely resolved (usually when graph is constructed by GraphBuilder).

When instances are backed by full butler, the quantum graph does not have to resolve output or intermediate references, but input references of each quantum have to be resolved before they can be used by this class. When executing such graphs, intermediate references used as input to some Quantum are resolved by lsst.ctrl.mpexec.SingleQuantumExecutor. If output references of a quanta are resolved, they will be unresolved when full butler is used.

Attributes Summary

dimensions

Structure managing all dimensions recognized by this data repository (DimensionUniverse).

Methods Summary

from_full(butler, quantum)	Make ButlerQuantumContext backed by lsst.daf.butler.Butler.
from_limited(butler, quantum)	Make ButlerQuantumContext backed by lsst.daf.butler.LimitedButler.
get(dataset, …)	Fetches data from the butler
put(values, List[Any], Any], dataset, …)	Puts data into the butler

Attributes Documentation

dimensions

Structure managing all dimensions recognized by this data repository (DimensionUniverse).

Methods Documentation

classmethod from_full(butler: lsst.daf.butler._butler.Butler, quantum: lsst.daf.butler.core.quantum.Quantum) → lsst.pipe.base.butlerQuantumContext.ButlerQuantumContext

Make ButlerQuantumContext backed by lsst.daf.butler.Butler.

Parameters:

butler : lsst.daf.butler.Butler

Butler object from/to which datasets will be get/put.

quantum : lsst.daf.butler.core.Quantum

Quantum object that describes the datasets which will be get/put by a single execution of this node in the pipeline graph. All input dataset references must be resolved in this Quantum. Output references can be resolved, but they will be unresolved.

Returns:

butlerQC : ButlerQuantumContext

Instance of butler wrapper.

classmethod from_limited(butler: lsst.daf.butler._limited_butler.LimitedButler, quantum: lsst.daf.butler.core.quantum.Quantum) → lsst.pipe.base.butlerQuantumContext.ButlerQuantumContext

Make ButlerQuantumContext backed by lsst.daf.butler.LimitedButler.

Parameters:

butler : lsst.daf.butler.LimitedButler

Butler object from/to which datasets will be get/put.

quantum : lsst.daf.butler.core.Quantum

Quantum object that describes the datasets which will be get/put by a single execution of this node in the pipeline graph. Both input and output dataset references must be resolved in this Quantum.

Returns:

butlerQC : ButlerQuantumContext

Instance of butler wrapper.

get(dataset: Union[lsst.pipe.base.connections.InputQuantizedConnection, List[Optional[lsst.daf.butler.core.datasets.ref.DatasetRef, None]], List[Optional[lsst.pipe.base.connections.DeferredDatasetRef, None]], lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.pipe.base.connections.DeferredDatasetRef, None]) → Any

Fetches data from the butler

Parameters:

dataset

This argument may either be an InputQuantizedConnection which describes all the inputs of a quantum, a list of DatasetRef, or a single DatasetRef. The function will get and return the corresponding datasets from the butler. If None is passed in place of a DatasetRef then the corresponding returned object will be None.

Returns:

return : object

This function returns arbitrary objects fetched from the bulter. The structure these objects are returned in depends on the type of the input argument. If the input dataset argument is a InputQuantizedConnection, then the return type will be a dictionary with keys corresponding to the attributes of the InputQuantizedConnection (which in turn are the attribute identifiers of the connections). If the input argument is of type list of DatasetRef then the return type will be a list of objects. If the input argument is a single DatasetRef then a single object will be returned.

Raises:

ValueError

Raised if a DatasetRef is passed to get that is not defined in the quantum object

put(values: Union[lsst.pipe.base.struct.Struct, List[Any], Any], dataset: Union[lsst.pipe.base.connections.OutputQuantizedConnection, List[lsst.daf.butler.core.datasets.ref.DatasetRef], lsst.daf.butler.core.datasets.ref.DatasetRef]) → None

Puts data into the butler

Parameters:

values : Struct or list of object or object

The data that should be put with the butler. If the type of the dataset is OutputQuantizedConnection then this argument should be a Struct with corresponding attribute names. Each attribute should then correspond to either a list of object or a single object depending of the type of the corresponding attribute on dataset. I.e. if dataset.calexp is [datasetRef1, datasetRef2] then values.calexp should be [calexp1, calexp2]. Like wise if there is a single ref, then only a single object need be passed. The same restriction applies if dataset is directly a list of DatasetRef or a single DatasetRef.

dataset

This argument may either be an InputQuantizedConnection which describes all the inputs of a quantum, a list of lsst.daf.butler.DatasetRef, or a single lsst.daf.butler.DatasetRef. The function will get and return the corresponding datasets from the butler.

Raises:

ValueError

Raised if a DatasetRef is passed to put that is not defined in the quantum object, or the type of values does not match what is expected from the type of dataset.