DatasetType¶
-
class
lsst.daf.butler.
DatasetType
(name: str, dimensions: Union[DimensionGraph, Iterable[Dimension]], storageClass: Union[StorageClass, str], parentStorageClass: Optional[Union[StorageClass, str]] = None, *, universe: Optional[DimensionUniverse] = None, isCalibration: bool = False)¶ Bases:
object
A named category of Datasets that defines how they are organized, related, and stored.
A concrete, final class whose instances represent
DatasetType
s.DatasetType
instances may be constructed without aRegistry
, but they must be registered viaRegistry.registerDatasetType()
before corresponding Datasets may be added.DatasetType
instances are immutable.Parameters: - name :
str
A string name for the Dataset; must correspond to the same
DatasetType
across all Registries. Names must start with an upper or lowercase letter, and may contain only letters, numbers, and underscores. Component dataset types should contain a single period separating the base dataset type name from the component name (and may be recursive).- dimensions :
DimensionGraph
or iterable ofDimension
Dimensions used to label and relate instances of this
DatasetType
. If not aDimensionGraph
,universe
must be provided as well.- storageClass :
StorageClass
orstr
Instance of a
StorageClass
or name ofStorageClass
that defines how thisDatasetType
is persisted.- parentStorageClass :
StorageClass
orstr
, optional Instance of a
StorageClass
or name ofStorageClass
that defines how the composite parent is persisted. Must beNone
if this is not a component. Mandatory if it is a component but can be the special temporary placeholder (DatasetType.PlaceholderParentStorageClass
) to allow construction with an intent to finalize later.- universe :
DimensionUniverse
, optional Set of all known dimensions, used to normalize
dimensions
if it is not already aDimensionGraph
.- isCalibration :
bool
, optional If
True
, this dataset type may be included inCALIBRATION
collections.
Attributes Summary
PlaceholderParentStorageClass
Placeholder StorageClass that can be used temporarily for a component. VALID_NAME_REGEX
dimensions
The Dimension
s that label and relate instances of thisDatasetType
(DimensionGraph
).name
A string name for the Dataset; must correspond to the same DatasetType
across all Registries.parentStorageClass
StorageClass
instance that defines how the composite associated with thisDatasetType
is persisted.storageClass
StorageClass
instance that defines how thisDatasetType
is persisted.Methods Summary
component
()Component name (if defined) componentTypeName
(component)Given a component name, derive the datasetTypeName of that component finalizeParentStorageClass
(newParent)Replace the current placeholder parent storage class with the real parent. isCalibration
()Return whether datasets of this type may be included in calibration collections. isComponent
()Boolean indicating whether this DatasetType
refers to a component of a composite.isComposite
()Boolean indicating whether this DatasetType
is a composite type.makeAllComponentDatasetTypes
()Return all the component dataset types assocaited with this dataset type. makeComponentDatasetType
(component)Return a DatasetType suitable for the given component, assuming the same dimensions as the parent. nameAndComponent
()Return the root name of this dataset type and the component name (if defined). nameWithComponent
(datasetTypeName, componentName)Form a valid DatasetTypeName from a parent and component. splitDatasetTypeName
(datasetTypeName)Given a dataset type name, return the root name and the component name. Attributes Documentation
-
PlaceholderParentStorageClass
= StorageClass('PlaceHolder')¶ Placeholder StorageClass that can be used temporarily for a component.
This can be useful in pipeline construction where we are creating dataset types without a registry.
-
VALID_NAME_REGEX
= re.compile('^[a-zA-Z][a-zA-Z0-9_]*(\\.[a-zA-Z][a-zA-Z0-9_]*)*$')¶
-
dimensions
¶ The
Dimension
s that label and relate instances of thisDatasetType
(DimensionGraph
).
-
name
¶ A string name for the Dataset; must correspond to the same
DatasetType
across all Registries.
-
parentStorageClass
¶ StorageClass
instance that defines how the composite associated with thisDatasetType
is persisted.Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property. Can be
None
if this is not a component of a composite. Must be defined if this is a component.
-
storageClass
¶ StorageClass
instance that defines how thisDatasetType
is persisted. Note that if DatasetType was constructed with a name of a StorageClass then Butler has to be initialized before using this property.
Methods Documentation
-
component
() → Optional[str]¶ Component name (if defined)
Returns: - comp :
str
Name of component part of DatasetType name.
None
if thisDatasetType
is not associated with a component.
- comp :
-
componentTypeName
(component: str) → str¶ Given a component name, derive the datasetTypeName of that component
Parameters: - component :
str
Name of component
Returns: - derived :
str
Compound name of this
DatasetType
and the component.
Raises: - KeyError
Requested component is not supported by this
DatasetType
.
- component :
-
finalizeParentStorageClass
(newParent: lsst.daf.butler.core.storageClass.StorageClass) → None¶ Replace the current placeholder parent storage class with the real parent.
Parameters: - newParent :
StorageClass
The new parent to be associated with this composite dataset type. This replaces the temporary placeholder parent that was specified during construction.
Raises: - ValueError
Raised if this dataset type is not a component of a composite. Raised if a StorageClass is not given. Raised if the parent currently associated with the dataset type is not a placeholder.
- newParent :
-
isCalibration
() → bool¶ Return whether datasets of this type may be included in calibration collections.
Returns:
-
isComponent
() → bool¶ Boolean indicating whether this
DatasetType
refers to a component of a composite.Returns: - isComponent :
bool
True
if thisDatasetType
is a component,False
otherwise.
- isComponent :
-
isComposite
() → bool¶ Boolean indicating whether this
DatasetType
is a composite type.Returns: - isComposite :
bool
True
if thisDatasetType
is a composite type,False
otherwise.
- isComposite :
-
makeAllComponentDatasetTypes
() → List[lsst.daf.butler.core.datasets.type.DatasetType]¶ Return all the component dataset types assocaited with this dataset type.
Returns: - all :
list
ofDatasetType
All the component dataset types. If this is not a composite then returns an empty list.
- all :
-
makeComponentDatasetType
(component: str) → lsst.daf.butler.core.datasets.type.DatasetType¶ Return a DatasetType suitable for the given component, assuming the same dimensions as the parent.
Parameters: - component :
str
Name of component
Returns: - datasetType :
DatasetType
A new DatasetType instance.
- component :
-
nameAndComponent
() → Tuple[str, Optional[str]]¶ Return the root name of this dataset type and the component name (if defined).
Returns: - rootName :
str
Root name for this
DatasetType
without any components.- componentName :
str
The component if it has been specified, else
None
.
- rootName :
-
static
nameWithComponent
(datasetTypeName: str, componentName: str) → str¶ Form a valid DatasetTypeName from a parent and component.
No validation is performed.
Parameters: Returns: - compTypeName :
str
Name to use for component DatasetType.
- compTypeName :
-
static
splitDatasetTypeName
(datasetTypeName: str) → Tuple[str, Optional[str]]¶ Given a dataset type name, return the root name and the component name.
Parameters: - datasetTypeName :
str
The name of the dataset type, can include a component using a “.”-separator.
Returns: Notes
If the dataset type name is
a.b.c
this method will return a root name ofa
and a component name ofb.c
.- datasetTypeName :
- name :