PersistenceContextVars¶
- class lsst.daf.butler.PersistenceContextVars¶
Bases:
objectHelper class for deserializing butler data structures.
When serializing various butler data structures nested dataset types get serialized independently. This means what were multiple references to the same object in memory are all duplicated in the serialization process.
Upon deserialization multiple independent data structures are created to represent the same logical bit of data.
This class can be used to remove this duplication by caching objects as they are created and returning a reference to that object. This is done in concert with
directandfrom_simplemethods on the various butler dataset structures.This class utilizes class level variables as a form of global state. Each of the various data structures can look to see if these global caches has been initialized as a cache (a dictionary) or is in the default None state.
Users of this class are intended to create an instance, and then call the
runmethod, supplying a callable function, and passing any required arguments. Therunmethod then creates a specific execution context, initializing the caches, and then runs the supplied function. Upon completion of the function call, the caches are cleared and returned to the default state.This process is thread safe.
Notes
Caches of
SerializedDatasetRefs are intentionally left out. It was discovered that these caused excessive python memory allocations which though cleaned up upon completion, left the process using more memory than it otherwise needed as python does not return allocated memory to the OS until process completion. It was determined the runtime cost of recreating theSerializedDatasetRefs was worth the memory savings.Attributes Summary
A cache of
DataCoordinates.A cache of
DatastoreRecordDataobjects.A cache of
DatasetRefs.A cache of
DimensionRecords.A cache of
DatasetTypes.A cache of
SerializedDataCoordinates.A cache of
SerializedDatasetTypes.A cache of
SerializedDimensionRecords.Methods Summary
run(function, *args, **kwargs)Execute the supplied function inside context specific caches.
Attributes Documentation
- dataCoordinates: ContextVar[dict[tuple[frozenset, bool], DataCoordinate] | None] = <ContextVar name='dataCoordinates' default=None>¶
A cache of
DataCoordinates.
- dataStoreRecords: ContextVar[dict[frozenset[str | uuid.UUID], DatastoreRecordData] | None] = <ContextVar name='dataStoreRecords' default=None>¶
A cache of
DatastoreRecordDataobjects.
- datasetRefs: ContextVar[dict[tuple[int, str], DatasetRef] | None] = <ContextVar name='datasetRefs' default=None>¶
A cache of
DatasetRefs.
- dimensionRecords: ContextVar[dict[Hashable, DimensionRecord] | None] = <ContextVar name='dimensionRecords' default=None>¶
A cache of
DimensionRecords.
- loadedTypes: ContextVar[dict[tuple[str, str], DatasetType] | None] = <ContextVar name='loadedTypes' default=None>¶
A cache of
DatasetTypes.
- serializedDataCoordinateMapping: ContextVar[dict[tuple[frozenset, bool], SerializedDataCoordinate] | None] = <ContextVar name='serializedDataCoordinateMapping' default=None>¶
A cache of
SerializedDataCoordinates.
- serializedDatasetTypeMapping: ContextVar[dict[tuple[str, str], SerializedDatasetType] | None] = <ContextVar name='serializedDatasetTypeMapping' default=None>¶
A cache of
SerializedDatasetTypes.
- serializedDimensionRecordMapping: ContextVar[dict[tuple[str, frozenset], SerializedDimensionRecord] | None] = <ContextVar name='serializedDimensionRecordMapping' default=None>¶
A cache of
SerializedDimensionRecords.
Methods Documentation
- run(function: ~collections.abc.Callable[[~_Q], ~lsst.daf.butler.core.persistenceContext._T], *args: ~typing.~_Q, **kwargs: ~typing.~_Q) _T¶
Execute the supplied function inside context specific caches.
- Parameters:
- function
Callable A callable which is to be executed inside a specific context.
- *argstuple
Positional arguments which are to be passed to the
Callable- **kwargs: dict, optional
Extra key word arguments which are to be passed to the
Callable
- function
- Returns:
- result
Any The result returned by executing the supplied
Callable
- result