DatasetRef¶

class lsst.daf.butler.DatasetRef(datasetType: DatasetType, dataId: DataCoordinate, run: str, *, id: UUID | None = None, conform: bool = True, id_generation_mode: DatasetIdGenEnum = DatasetIdGenEnum.UNIQUE)¶

Bases: object

Reference to a Dataset in a Registry.

A DatasetRef may point to a Dataset that currently does not yet exist (e.g., because it is a predicted input for provenance).

Parameters:

datasetTypeDatasetType: The DatasetType for this Dataset.
dataIdDataCoordinate: A mapping of dimensions that labels the Dataset within a Collection.
runstr: The name of the run this dataset was associated with when it was created.
idDatasetId, optional: The unique identifier assigned when the dataset is created. If id is not specified, a new unique ID will be created.
conformbool, optional: If True (default), call DataCoordinate.standardize to ensure that the data ID’s dimensions are consistent with the dataset type’s. DatasetRef instances for which those dimensions are not equal should not be created in new code, but are still supported for backwards compatibility. New code should only pass False if it can guarantee that the dimensions are already consistent.
id_generation_modeDatasetIdGenEnum: ID generation option. UNIQUE makes a random UUID4-type ID. DATAID_TYPE makes a deterministic UUID5-type ID based on a dataset type name and dataId. DATAID_TYPE_RUN makes a deterministic UUID5-type ID based on a dataset type name, run collection name, and dataId.

Raises:

ValueError: Raised if run is provided but id is not, or if id is provided but run is not.

See also

Organizing and identifying datasets

Attributes Summary

`dataId`	A mapping of `Dimension` primary key values that labels the dataset within a Collection (`DataCoordinate`).
`datasetType`	The definition of this dataset (`DatasetType`).
`dimensions`	Dimensions associated with the underlying `DatasetType`.
`id`	Primary key of the dataset (`DatasetId`).
`run`	The name of the run that produced the dataset.

Methods Summary

`expanded`(dataId)	Return a new `DatasetRef` with the given expanded data ID.
`from_json`(json_str[, universe, registry])	Convert from JSON to a pydantic model.
`from_simple`(simple[, universe, registry, ...])	Construct a new object from simplified form.
`groupByType`(refs)	Group an iterable of `DatasetRef` by `DatasetType`.
`isComponent`()	Indicate whether this `DatasetRef` refers to a component.
`isComposite`()	Boolean indicating whether this `DatasetRef` is a composite type.
`makeComponentRef`(name)	Create a `DatasetRef` that corresponds to a component.
`makeCompositeRef`()	Create a `DatasetRef` of the composite from a component ref.
`overrideStorageClass`(storageClass)	Create a new `DatasetRef` from this one, but with a modified `DatasetType` that has a different `StorageClass`.
`to_json`([minimal])	Convert this class to JSON assuming that the `to_simple()` returns a pydantic model.
`to_simple`([minimal])	Convert this class to a simple python type.

Attributes Documentation

dataId: DataCoordinate¶

A mapping of Dimension primary key values that labels the dataset within a Collection (DataCoordinate).

Cannot be changed after a DatasetRef is constructed.

datasetType: DatasetType¶

The definition of this dataset (DatasetType).

Cannot be changed after a DatasetRef is constructed.

dimensions¶: Dimensions associated with the underlying DatasetType.

id: DatasetId¶

Primary key of the dataset (DatasetId).

Cannot be changed after a DatasetRef is constructed.

run: str¶

The name of the run that produced the dataset.

Cannot be changed after a DatasetRef is constructed.

Methods Documentation

expanded(dataId: DataCoordinate) → DatasetRef¶

Return a new DatasetRef with the given expanded data ID.

Parameters:

dataIdDataCoordinate: Data ID for the new DatasetRef. Must compare equal to the original data ID.

Returns:

refDatasetRef: A new DatasetRef with the given data ID.

classmethod from_json(json_str: str, universe: DimensionUniverse | None = None, registry: Registry | None = None) → SupportsSimple¶: Convert from JSON to a pydantic model.

classmethod from_simple(simple: SerializedDatasetRef, universe: DimensionUniverse | None = None, registry: Registry | None = None, datasetType: DatasetType | None = None) → DatasetRef¶

Construct a new object from simplified form.

Generally this is data returned from the to_simple method.

Parameters:

simpledict of [str, Any]: The value returned by to_simple().
universeDimensionUniverse: The special graph of all known dimensions. Can be None if a registry is provided.
registrylsst.daf.butler.Registry, optional: Registry to use to convert simple form of a DatasetRef to a full DatasetRef. Can be None if a full description of the type is provided along with a universe.
datasetTypeDatasetType, optional: If datasetType is supplied, this will be used as the datasetType object in the resulting DatasetRef instead of being read from the SerializedDatasetRef. This is useful when many refs share the same type as memory can be saved. Defaults to None.

Returns:

refDatasetRef: Newly-constructed object.

static groupByType(refs: Iterable[DatasetRef]) → NamedKeyDict[DatasetType, List[DatasetRef]]¶

Group an iterable of DatasetRef by DatasetType.

Parameters:

refsIterable [ DatasetRef ]: DatasetRef instances to group.

Returns:

groupedNamedKeyDict [ DatasetType, list [ DatasetRef ] ]: Grouped DatasetRef instances.

isComponent() → bool¶

Indicate whether this DatasetRef refers to a component.

Returns:

isComponentbool: True if this DatasetRef is a component, False otherwise.

isComposite() → bool¶

Boolean indicating whether this DatasetRef is a composite type.

Returns:

isCompositebool: True if this DatasetRef is a composite type, False otherwise.

makeComponentRef(name: str) → DatasetRef¶

Create a DatasetRef that corresponds to a component.

Parameters:

namestr: Name of the component.

Returns:

refDatasetRef: A DatasetRef with a dataset type that corresponds to the given component, and the same ID and run (which may be None, if they are None in self).

makeCompositeRef() → DatasetRef¶

Create a DatasetRef of the composite from a component ref.

Requires that this DatasetRef is a component.

Returns:

refDatasetRef: A DatasetRef with a dataset type that corresponds to the composite parent of this component, and the same ID and run (which may be None, if they are None in self).

overrideStorageClass(storageClass: str | StorageClass) → DatasetRef¶

Create a new DatasetRef from this one, but with a modified DatasetType that has a different StorageClass.

Parameters:

storageClassstr or StorageClass: The new storage class.

Returns:

modifiedDatasetRef: A new dataset reference that is the same as the current one but with a different storage class in the DatasetType.

to_json(minimal: bool = False) → str¶: Convert this class to JSON assuming that the to_simple() returns a pydantic model.

to_simple(minimal: bool = False) → SerializedDatasetRef¶

Convert this class to a simple python type.

This makes it suitable for serialization.

Parameters:

minimalbool, optional: Use minimal serialization. Requires Registry to convert back to a full type.

Returns:

simpledict or int: The object converted to a dictionary.

Navigation

DatasetRef¶