PersistenceContextVars¶
- class lsst.daf.butler.persistence_context.PersistenceContextVars¶
Bases:
object
Helper class for deserializing butler data structures.
When serializing various butler data structures nested dataset types get serialized independently. This means what were multiple references to the same object in memory are all duplicated in the serialization process.
Upon deserialization multiple independent data structures are created to represent the same logical bit of data.
This class can be used to remove this duplication by caching objects as they are created and returning a reference to that object. This is done in concert with
direct
andfrom_simple
methods on the various butler dataset structures.This class utilizes class level variables as a form of global state. Each of the various data structures can look to see if these global caches has been initialized as a cache (a dictionary) or is in the default None state.
Users of this class are intended to create an instance, and then call the
run
method, supplying a callable function, and passing any required arguments. Therun
method then creates a specific execution context, initializing the caches, and then runs the supplied function. Upon completion of the function call, the caches are cleared and returned to the default state.This process is thread safe.
Notes
Caches of
SerializedDatasetRef
s are intentionally left out. It was discovered that these caused excessive python memory allocations which though cleaned up upon completion, left the process using more memory than it otherwise needed as python does not return allocated memory to the OS until process completion. It was determined the runtime cost of recreating theSerializedDatasetRef
s was worth the memory savings.Attributes Summary
A cache of
DataCoordinate
s.A cache of
DatastoreRecordData
objects.A cache of
DatasetRef
s.A cache of
DimensionRecord
s.A cache of
DatasetType
s.A cache of
SerializedDataCoordinate
s.A cache of
SerializedDatasetType
s.A cache of
SerializedDimensionRecord
s.Methods Summary
run
(function, *args, **kwargs)Execute the supplied function inside context specific caches.
Attributes Documentation
- dataCoordinates: ContextVar[dict[tuple[frozenset, bool], DataCoordinate] | None] = <ContextVar name='dataCoordinates' default=None>¶
A cache of
DataCoordinate
s.
- dataStoreRecords: ContextVar[dict[frozenset[str | uuid.UUID], DatastoreRecordData] | None] = <ContextVar name='dataStoreRecords' default=None>¶
A cache of
DatastoreRecordData
objects.
- datasetRefs: ContextVar[dict[int, DatasetRef] | None] = <ContextVar name='datasetRefs' default=None>¶
A cache of
DatasetRef
s.Keys are UUID converted to int, but only refs of parent dataset types are cached AND THE STORAGE CLASS IS UNSPECIFIED; consumers of this cache must call overrideStorageClass on the result.
- dimensionRecords: ContextVar[dict[Hashable, DimensionRecord] | None] = <ContextVar name='dimensionRecords' default=None>¶
A cache of
DimensionRecord
s.
- loadedTypes: ContextVar[dict[tuple[str, str], DatasetType] | None] = <ContextVar name='loadedTypes' default=None>¶
A cache of
DatasetType
s.
- serializedDataCoordinateMapping: ContextVar[dict[tuple[frozenset, bool], SerializedDataCoordinate] | None] = <ContextVar name='serializedDataCoordinateMapping' default=None>¶
A cache of
SerializedDataCoordinate
s.
- serializedDatasetTypeMapping: ContextVar[dict[tuple[str, str], SerializedDatasetType] | None] = <ContextVar name='serializedDatasetTypeMapping' default=None>¶
A cache of
SerializedDatasetType
s.
- serializedDimensionRecordMapping: ContextVar[dict[tuple[str, frozenset] | tuple[int, DataCoordinate], SerializedDimensionRecord] | None] = <ContextVar name='serializedDimensionRecordMapping' default=None>¶
A cache of
SerializedDimensionRecord
s.
Methods Documentation
- run(function: ~collections.abc.Callable[[~_Q], ~lsst.daf.butler.persistence_context._T], *args: ~typing.~_Q, **kwargs: ~typing.~_Q) _T ¶
Execute the supplied function inside context specific caches.
- Parameters:
- function
Callable
A callable which is to be executed inside a specific context.
- *argstuple
Positional arguments which are to be passed to the
Callable
.- **kwargsdict, optional
Extra key word arguments which are to be passed to the
Callable
.
- function
- Returns:
- result
Any
The result returned by executing the supplied
Callable
.
- result