DatasetType¶

Bases: object

A named category of Datasets.

Defines how they are organized, related, and stored.

A concrete, final class whose instances represent DatasetTypes. DatasetType instances may be constructed without a Registry, but they must be registered via Registry.registerDatasetType() before corresponding Datasets may be added. DatasetType instances are immutable.

Parameters:

namestr: A string name for the Dataset; must correspond to the same DatasetType across all Registries. Names must start with an upper or lowercase letter, and may contain only letters, numbers, and underscores. Component dataset types should contain a single period separating the base dataset type name from the component name (and may be recursive).
dimensionsDimensionGraph or iterable of Dimension or str: Dimensions used to label and relate instances of this DatasetType. If not a DimensionGraph, universe must be provided as well.
storageClassStorageClass or str: Instance of a StorageClass or name of StorageClass that defines how this DatasetType is persisted.
parentStorageClassStorageClass or str, optional: Instance of a StorageClass or name of StorageClass that defines how the composite parent is persisted. Must be None if this is not a component.
universeDimensionUniverse, optional: Set of all known dimensions, used to normalize dimensions if it is not already a DimensionGraph.
isCalibrationbool, optional: If True, this dataset type may be included in CALIBRATION collections.

See also

Organizing and identifying datasets

Attributes Summary

`VALID_NAME_REGEX`
`dimensions`	Return the `Dimension`s fir this dataset type.
`name`	Return a string name for the Dataset.
`parentStorageClass`	Return the storage class of the composite containing this component.
`storageClass`	Return `StorageClass` instance associated with this dataset type.
`storageClass_name`	Return the storage class name.

Methods Summary

`component`()	Return the component name (if defined).
`componentTypeName`(component)	Derive a component dataset type from a composite.
`from_json`(json_str[, universe, registry])	Convert from JSON to a pydantic model.
`from_simple`(simple[, universe, registry])	Construct a new object from the simplified form.
`isCalibration`()	Return if datasets of this type can be in calibration collections.
`isComponent`()	Return whether this `DatasetType` refers to a component.
`isComposite`()	Return whether this `DatasetType` is a composite.
`is_compatible_with`(other)	Determine if the given `DatasetType` is compatible with this one.
`makeAllComponentDatasetTypes`()	Return all component dataset types for this composite.
`makeComponentDatasetType`(component)	Return a component dataset type from a composite.
`makeCompositeDatasetType`()	Return a composite dataset type from the component.
`nameAndComponent`()	Return the root name of this dataset type and any component.
`nameWithComponent`(datasetTypeName, componentName)	Form a valid DatasetTypeName from a parent and component.
`overrideStorageClass`(storageClass)	Create a new `DatasetType` from this one but with an updated `StorageClass`.
`splitDatasetTypeName`(datasetTypeName)	Return the root name and the component from a composite name.
`to_json`([minimal])	Convert this class to JSON assuming that the `to_simple()` returns a pydantic model.
`to_simple`([minimal])	Convert this class to a simple python type.

Attributes Documentation

VALID_NAME_REGEX = re.compile('^[a-zA-Z_][a-zA-Z0-9_]*(\\.[a-zA-Z_][a-zA-Z0-9_]*)*$')¶

dimensions¶

Return the Dimensions fir this dataset type.

The dimensions label and relate instances of this DatasetType (DimensionGraph).

name¶

Return a string name for the Dataset.

Must correspond to the same DatasetType across all Registries.

parentStorageClass¶

Return the storage class of the composite containing this component.

Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property. Can be None if this is not a component of a composite. Must be defined if this is a component.

storageClass¶

Return StorageClass instance associated with this dataset type.

The StorageClass defines how this DatasetType is persisted. Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property.

storageClass_name¶

Return the storage class name.

This will never force the storage class to be imported.

Methods Documentation

component() → str | None¶

Return the component name (if defined).

Returns:

compstr: Name of component part of DatasetType name. None if this DatasetType is not associated with a component.

componentTypeName(component: str) → str¶

Derive a component dataset type from a composite.

Parameters:

componentstr: Name of component

Returns:

derivedstr: Compound name of this DatasetType and the component.

Raises:

KeyError: Requested component is not supported by this DatasetType.

classmethod from_json(json_str: str, universe: DimensionUniverse | None = None, registry: Registry | None = None) → SupportsSimple¶: Convert from JSON to a pydantic model.

classmethod from_simple(simple: SerializedDatasetType, universe: DimensionUniverse | None = None, registry: Registry | None = None) → DatasetType¶

Construct a new object from the simplified form.

This is usually data returned from the to_simple method.

Parameters:

simpleSerializedDatasetType: The value returned by to_simple().
universeDimensionUniverse: The special graph of all known dimensions of which this graph will be a subset. Can be None if a registry is provided.
registrylsst.daf.butler.Registry, optional: Registry to use to convert simple name of a DatasetType to a full DatasetType. Can be None if a full description of the type is provided along with a universe.

Returns:

datasetTypeDatasetType: Newly-constructed object.

isCalibration() → bool¶

Return if datasets of this type can be in calibration collections.

Returns:

flagbool: True if datasets of this type may be included in calibration collections.

isComponent() → bool¶

Return whether this DatasetType refers to a component.

Returns:

isComponentbool: True if this DatasetType is a component, False otherwise.

isComposite() → bool¶

Return whether this DatasetType is a composite.

Returns:

isCompositebool: True if this DatasetType is a composite type, False otherwise.

is_compatible_with(other: DatasetType) → bool¶

Determine if the given DatasetType is compatible with this one.

Compatibility requires a matching name and dimensions and a storage class for this dataset type that can convert the python type associated with the other storage class to this python type.

Parameters:

otherDatasetType: Dataset type to check.

Returns:

is_compatiblebool: Returns True if the other dataset type is either the same as this or the storage class associated with the other can be converted to this.

makeAllComponentDatasetTypes() → List[DatasetType]¶

Return all component dataset types for this composite.

Returns:

alllist of DatasetType: All the component dataset types. If this is not a composite then returns an empty list.

makeComponentDatasetType(component: str) → DatasetType¶

Return a component dataset type from a composite.

Assumes the same dimensions as the parent.

Parameters:

componentstr: Name of component

Returns:

datasetTypeDatasetType: A new DatasetType instance.

makeCompositeDatasetType() → DatasetType¶

Return a composite dataset type from the component.

Returns:

compositeDatasetType: The composite dataset type.

Raises:

RuntimeError: Raised if this dataset type is not a component dataset type.

nameAndComponent() → Tuple[str, str | None]¶

Return the root name of this dataset type and any component.

Returns:

rootNamestr: Root name for this DatasetType without any components.
componentNamestr: The component if it has been specified, else None.

static nameWithComponent(datasetTypeName: str, componentName: str) → str¶

Form a valid DatasetTypeName from a parent and component.

No validation is performed.

Parameters:

datasetTypeNamestr: Base type name.
componentNamestr: Name of component.

Returns:

compTypeNamestr: Name to use for component DatasetType.

overrideStorageClass(storageClass: str | StorageClass) → DatasetType¶

Create a new DatasetType from this one but with an updated StorageClass.

Parameters:

storageClassstr or StorageClass: The new storage class.

Returns:

modifiedDatasetType: A new dataset type that is the same as the current one but with a different storage class.

Notes

If this is a component dataset type, the parent storage class will be retained.

static splitDatasetTypeName(datasetTypeName: str) → Tuple[str, str | None]¶

Return the root name and the component from a composite name.

Parameters:

datasetTypeNamestr: The name of the dataset type, can include a component using a “.”-separator.

Returns:

rootNamestr: Root name without any components.
componentNamestr: The component if it has been specified, else None.

Notes

If the dataset type name is a.b.c this method will return a root name of a and a component name of b.c.

to_json(minimal: bool = False) → str¶: Convert this class to JSON assuming that the to_simple() returns a pydantic model.

to_simple(minimal: bool = False) → SerializedDatasetType¶

Convert this class to a simple python type.

This makes it suitable for serialization.

Parameters:

minimalbool, optional: Use minimal serialization. Requires Registry to convert back to a full type.

Returns:

simpleSerializedDatasetType: The object converted to a class suitable for serialization.

Navigation

DatasetType¶