DatasetType¶
- class lsst.daf.butler.DatasetType(name: str, dimensions: DimensionGroup | DimensionGraph | Iterable[Dimension | str], storageClass: StorageClass | str, parentStorageClass: StorageClass | str | None = None, *, universe: DimensionUniverse | None = None, isCalibration: bool = False)¶
Bases:
object
A named category of Datasets.
Defines how they are organized, related, and stored.
A concrete, final class whose instances represent
DatasetType
s.DatasetType
instances may be constructed without aRegistry
, but they must be registered viaRegistry.registerDatasetType()
before corresponding Datasets may be added.DatasetType
instances are immutable.- Parameters:
- name
str
A string name for the Dataset; must correspond to the same
DatasetType
across all Registries. Names must start with an upper or lowercase letter, and may contain only letters, numbers, and underscores. Component dataset types should contain a single period separating the base dataset type name from the component name (and may be recursive).- dimensions
DimensionGroup
,DimensionGraph
, or Dimensions used to label and relate instances of this
DatasetType
. If not aDimensionGraph
orDimensionGroup
,universe
must be provided as well.- storageClass
StorageClass
orstr
Instance of a
StorageClass
or name ofStorageClass
that defines how thisDatasetType
is persisted.- parentStorageClass
StorageClass
orstr
, optional Instance of a
StorageClass
or name ofStorageClass
that defines how the composite parent is persisted. Must beNone
if this is not a component.- universe
DimensionUniverse
, optional Set of all known dimensions, used to normalize
dimensions
if it is not already aDimensionGraph
.- isCalibration
bool
, optional If
True
, this dataset type may be included inCALIBRATION
collections.
- name
Notes
See also Organizing and identifying datasets.
Attributes Summary
Return the dimensions of this dataset type (
DimensionGraph
).Return a string name for the Dataset.
Return the storage class of the composite containing this component.
Return
StorageClass
instance associated with this dataset type.Return the storage class name.
Methods Summary
Return the component name (if defined).
componentTypeName
(component)Derive a component dataset type from a composite.
from_json
(json_str[, universe, registry])Convert from JSON to a pydantic model.
from_simple
(simple[, universe, registry])Construct a new object from the simplified form.
Return if datasets of this type can be in calibration collections.
Return whether this
DatasetType
refers to a component.Return whether this
DatasetType
is a composite.is_compatible_with
(other)Determine if the given
DatasetType
is compatible with this one.Return all component dataset types for this composite.
makeComponentDatasetType
(component)Return a component dataset type from a composite.
Return a composite dataset type from the component.
Return the root name of this dataset type and any component.
nameWithComponent
(datasetTypeName, componentName)Form a valid DatasetTypeName from a parent and component.
overrideStorageClass
(storageClass)Create a new
DatasetType
from this one but with an updatedStorageClass
.splitDatasetTypeName
(datasetTypeName)Return the root name and the component from a composite name.
to_json
([minimal])Convert this class to JSON assuming that the
to_simple()
returns a pydantic model.to_simple
([minimal])Convert this class to a simple python type.
Attributes Documentation
- VALID_NAME_REGEX = re.compile('^[a-zA-Z_][a-zA-Z0-9_]*(\\.[a-zA-Z_][a-zA-Z0-9_]*)*$')¶
- dimensions¶
Return the dimensions of this dataset type (
DimensionGraph
).The dimensions of a define the keys of its datasets’ data IDs..
- name¶
Return a string name for the Dataset.
Must correspond to the same
DatasetType
across all Registries.
- parentStorageClass¶
Return the storage class of the composite containing this component.
Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property. Can be
None
if this is not a component of a composite. Must be defined if this is a component.
- storageClass¶
Return
StorageClass
instance associated with this dataset type.The
StorageClass
defines how thisDatasetType
is persisted. Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property.
- storageClass_name¶
Return the storage class name.
This will never force the storage class to be imported.
Methods Documentation
- component() str | None ¶
Return the component name (if defined).
- Returns:
- comp
str
Name of component part of DatasetType name.
None
if thisDatasetType
is not associated with a component.
- comp
- componentTypeName(component: str) str ¶
Derive a component dataset type from a composite.
- Parameters:
- component
str
Name of component.
- component
- Returns:
- derived
str
Compound name of this
DatasetType
and the component.
- derived
- Raises:
- KeyError
Requested component is not supported by this
DatasetType
.
- classmethod from_json(json_str: str, universe: DimensionUniverse | None = None, registry: Registry | None = None) SupportsSimple ¶
Convert from JSON to a pydantic model.
- Parameters:
- cls_
type
ofSupportsSimple
The Python type being created.
- json_str
str
The JSON string representing this object.
- universe
DimensionUniverse
orNone
, optional The universe required to instantiate some models. Required if
registry
isNone
.- registry
Registry
orNone
, optional Registry from which to obtain the dimension universe if an explicit universe has not been given.
- cls_
- Returns:
- model
SupportsSimple
Pydantic model constructed from JSON and validated.
- model
- classmethod from_simple(simple: SerializedDatasetType, universe: DimensionUniverse | None = None, registry: Registry | None = None) DatasetType ¶
Construct a new object from the simplified form.
This is usually data returned from the
to_simple
method.- Parameters:
- simple
SerializedDatasetType
The value returned by
to_simple()
.- universe
DimensionUniverse
The special graph of all known dimensions of which this graph will be a subset. Can be
None
if a registry is provided.- registry
lsst.daf.butler.Registry
, optional Registry to use to convert simple name of a DatasetType to a full
DatasetType
. Can beNone
if a full description of the type is provided along with a universe.
- simple
- Returns:
- datasetType
DatasetType
Newly-constructed object.
- datasetType
- isComponent() bool ¶
Return whether this
DatasetType
refers to a component.- Returns:
- isComponent
bool
True
if thisDatasetType
is a component,False
otherwise.
- isComponent
- isComposite() bool ¶
Return whether this
DatasetType
is a composite.- Returns:
- isComposite
bool
True
if thisDatasetType
is a composite type,False
otherwise.
- isComposite
- is_compatible_with(other: DatasetType) bool ¶
Determine if the given
DatasetType
is compatible with this one.Compatibility requires a matching name and dimensions and a storage class for this dataset type that can convert the python type associated with the other storage class to this python type.
- Parameters:
- other
DatasetType
Dataset type to check.
- other
- Returns:
- makeAllComponentDatasetTypes() list[lsst.daf.butler._dataset_type.DatasetType] ¶
Return all component dataset types for this composite.
- Returns:
- all
list
ofDatasetType
All the component dataset types. If this is not a composite then returns an empty list.
- all
- makeComponentDatasetType(component: str) DatasetType ¶
Return a component dataset type from a composite.
Assumes the same dimensions as the parent.
- Parameters:
- component
str
Name of component.
- component
- Returns:
- datasetType
DatasetType
A new DatasetType instance.
- datasetType
- makeCompositeDatasetType() DatasetType ¶
Return a composite dataset type from the component.
- Returns:
- composite
DatasetType
The composite dataset type.
- composite
- Raises:
- RuntimeError
Raised if this dataset type is not a component dataset type.
- nameAndComponent() tuple[str, str | None] ¶
Return the root name of this dataset type and any component.
- Returns:
- rootName
str
Root name for this
DatasetType
without any components.- componentName
str
The component if it has been specified, else
None
.
- rootName
- static nameWithComponent(datasetTypeName: str, componentName: str) str ¶
Form a valid DatasetTypeName from a parent and component.
No validation is performed.
- overrideStorageClass(storageClass: str | StorageClass) DatasetType ¶
Create a new
DatasetType
from this one but with an updatedStorageClass
.- Parameters:
- storageClass
str
orStorageClass
The new storage class.
- storageClass
- Returns:
- modified
DatasetType
A dataset type that is the same as the current one but with a different storage class. Will be
self
if the given storage class is the current one.
- modified
Notes
If this is a component dataset type, the parent storage class will be retained.
- static splitDatasetTypeName(datasetTypeName: str) tuple[str, str | None] ¶
Return the root name and the component from a composite name.
- Parameters:
- datasetTypeName
str
The name of the dataset type, can include a component using a “.”-separator.
- datasetTypeName
- Returns:
Notes
If the dataset type name is
a.b.c
this method will return a root name ofa
and a component name ofb.c
.
- to_json(minimal: bool = False) str ¶
Convert this class to JSON assuming that the
to_simple()
returns a pydantic model.- Parameters:
- minimal
bool
Return minimal possible representation.
- minimal
- to_simple(minimal: bool = False) SerializedDatasetType ¶
Convert this class to a simple python type.
This makes it suitable for serialization.
- Parameters:
- minimal
bool
, optional Use minimal serialization. Requires Registry to convert back to a full type.
- minimal
- Returns:
- simple
SerializedDatasetType
The object converted to a class suitable for serialization.
- simple