DatasetRef¶

class lsst.daf.butler.DatasetRef(datasetType: lsst.daf.butler.core.datasets.type.DatasetType, dataId: lsst.daf.butler.core.dimensions._coordinate.DataCoordinate, *, id: Union[int, uuid.UUID, None] = None, run: Optional[str, None] = None, conform: bool = True)¶

Bases: object

Reference to a Dataset in a Registry.

A DatasetRef may point to a Dataset that currently does not yet exist (e.g., because it is a predicted input for provenance).

Parameters:

datasetType : DatasetType: The DatasetType for this Dataset.
dataId : DataCoordinate: A mapping of dimensions that labels the Dataset within a Collection.
id : DatasetId, optional: The unique identifier assigned when the dataset is created.
run : str, optional: The name of the run this dataset was associated with when it was created. Must be provided if id is.
conform : bool, optional: If True (default), call DataCoordinate.standardize to ensure that the data ID’s dimensions are consistent with the dataset type’s. DatasetRef instances for which those dimensions are not equal should not be created in new code, but are still supported for backwards compatibility. New code should only pass False if it can guarantee that the dimensions are already consistent.

Raises:

ValueError: Raised if run is provided but id is not, or if id is provided but run is not.

See also

Organizing and identifying datasets

Attributes Summary

`dataId`	A mapping of `Dimension` primary key values that labels the dataset within a Collection (`DataCoordinate`).
`datasetType`	The definition of this dataset (`DatasetType`).
`dimensions`	Dimensions associated with the underlying `DatasetType`.
`id`	Primary key of the dataset (`DatasetId` or `None`).
`run`	The name of the run that produced the dataset.

Methods Summary

`expanded`(dataId)	Return a new `DatasetRef` with the given expanded data ID.
`from_json`(json_str, universe, registry)	Convert from JSON to a pydantic model.
`from_simple`(simple, universe, registry, …)	Construct a new object from simplified form.
`getCheckedId`()	Return `self.id`, or raise if it is `None`.
`groupByType`(refs)	Group an iterable of `DatasetRef` by `DatasetType`.
`isComponent`()	Indicate whether this `DatasetRef` refers to a component.
`isComposite`()	Boolean indicating whether this `DatasetRef` is a composite type.
`makeComponentRef`(name)	Create a `DatasetRef` that corresponds to a component.
`makeCompositeRef`()	Create a `DatasetRef` of the composite from a component ref.
`overrideStorageClass`(storageClass)	Create a new `DatasetRef` from this one, but with a modified `DatasetType` that has a different `StorageClass`.
`resolved`(id, uuid.UUID], run)	Return resolved `DatasetRef`.
`to_json`(minimal)	Convert this class to JSON assuming that the `to_simple()` returns a pydantic model.
`to_simple`(minimal)	Convert this class to a simple python type.
`unresolved`()	Return unresolved `DatasetRef`.

Attributes Documentation

dataId¶

A mapping of Dimension primary key values that labels the dataset within a Collection (DataCoordinate).

Cannot be changed after a DatasetRef is constructed.

datasetType¶

The definition of this dataset (DatasetType).

Cannot be changed after a DatasetRef is constructed.

dimensions¶: Dimensions associated with the underlying DatasetType.

id¶

Primary key of the dataset (DatasetId or None).

Cannot be changed after a DatasetRef is constructed; use resolved or unresolved to add or remove this information when creating a new DatasetRef.

run¶

The name of the run that produced the dataset.

Cannot be changed after a DatasetRef is constructed; use resolved or unresolved to add or remove this information when creating a new DatasetRef.

Methods Documentation

expanded(dataId: lsst.daf.butler.core.dimensions._coordinate.DataCoordinate) → lsst.daf.butler.core.datasets.ref.DatasetRef¶

Return a new DatasetRef with the given expanded data ID.

Parameters:	dataId : `DataCoordinate` Data ID for the new `DatasetRef`. Must compare equal to the original data ID.
Returns:	ref : `DatasetRef` A new `DatasetRef` with the given data ID.

classmethod from_json(json_str: str, universe: Optional[DimensionUniverse] = None, registry: Optional[Registry] = None) → SupportsSimple¶: Convert from JSON to a pydantic model.

classmethod from_simple(simple: SerializedDatasetRef, universe: Optional[DimensionUniverse] = None, registry: Optional[Registry] = None, datasetType: Optional[DatasetType] = None) → DatasetRef¶

Construct a new object from simplified form.

Generally this is data returned from the to_simple method.

Parameters:

simple : dict of [str, Any]: The value returned by to_simple().
universe : DimensionUniverse: The special graph of all known dimensions. Can be None if a registry is provided.
registry : lsst.daf.butler.Registry, optional: Registry to use to convert simple form of a DatasetRef to a full DatasetRef. Can be None if a full description of the type is provided along with a universe.
datasetType : DatasetType, optional: If datasetType is supplied, this will be used as the datasetType object in the resulting DatasetRef instead of being read from the SerializedDatasetRef. This is useful when many refs share the same type as memory can be saved. Defaults to None.

Returns:

ref : DatasetRef: Newly-constructed object.

getCheckedId() → Union[int, uuid.UUID]¶

Return self.id, or raise if it is None.

This trivial method exists to allow operations that would otherwise be natural list comprehensions to check that the ID is not None as well.

Returns:	id : `DatasetId` `self.id` if it is not `None`.
Raises:	AmbiguousDatasetError Raised if `ref.id` is `None`.

static groupByType(refs: Iterable[lsst.daf.butler.core.datasets.ref.DatasetRef]) → lsst.daf.butler.core.named.NamedKeyDict[lsst.daf.butler.core.datasets.type.DatasetType, List[lsst.daf.butler.core.datasets.ref.DatasetRef]]¶

Group an iterable of DatasetRef by DatasetType.

Parameters:	refs : `Iterable` [ `DatasetRef` ] `DatasetRef` instances to group.
Returns:	grouped : `NamedKeyDict` [ `DatasetType`, `list` [ `DatasetRef` ] ] Grouped `DatasetRef` instances.

isComponent() → bool¶

Indicate whether this DatasetRef refers to a component.

Returns:	isComponent : `bool` `True` if this `DatasetRef` is a component, `False` otherwise.

isComposite() → bool¶

Boolean indicating whether this DatasetRef is a composite type.

Returns:	isComposite : `bool` `True` if this `DatasetRef` is a composite type, `False` otherwise.

makeComponentRef(name: str) → lsst.daf.butler.core.datasets.ref.DatasetRef¶

Create a DatasetRef that corresponds to a component.

Parameters:	name : `str` Name of the component.
Returns:	ref : `DatasetRef` A `DatasetRef` with a dataset type that corresponds to the given component, and the same ID and run (which may be `None`, if they are `None` in `self`).

makeCompositeRef() → lsst.daf.butler.core.datasets.ref.DatasetRef¶

Create a DatasetRef of the composite from a component ref.

Requires that this DatasetRef is a component.

Returns:	ref : `DatasetRef` A `DatasetRef` with a dataset type that corresponds to the composite parent of this component, and the same ID and run (which may be `None`, if they are `None` in `self`).

overrideStorageClass(storageClass: str | StorageClass) → DatasetRef¶

Create a new DatasetRef from this one, but with a modified DatasetType that has a different StorageClass.

Parameters:	storageClass : `str` or `StorageClass` The new storage class.
Returns:	modified : `DatasetRef` A new dataset reference that is the same as the current one but with a different storage class in the `DatasetType`.

resolved(id: Union[int, uuid.UUID], run: str) → lsst.daf.butler.core.datasets.ref.DatasetRef¶

Return resolved DatasetRef.

This is a new DatasetRef with the same data ID and dataset type and the given ID and run.

Parameters:	id : `DatasetId` The unique identifier assigned when the dataset is created. run : `str` The run this dataset was associated with when it was created.
Returns:	ref : `DatasetRef` A new `DatasetRef`.

to_json(minimal: bool = False) → str¶: Convert this class to JSON assuming that the to_simple() returns a pydantic model.

to_simple(minimal: bool = False) → lsst.daf.butler.core.datasets.ref.SerializedDatasetRef¶

Convert this class to a simple python type.

This makes it suitable for serialization.

Parameters:	minimal : `bool`, optional Use minimal serialization. Requires Registry to convert back to a full type.
Returns:	simple : `dict` or `int` The object converted to a dictionary.

unresolved() → lsst.daf.butler.core.datasets.ref.DatasetRef¶

Return unresolved DatasetRef.

This is a new DatasetRef with the same data ID and dataset type, but no ID or run.

Returns:	ref : `DatasetRef` A new `DatasetRef`.

Notes

This can be used to compare only the data ID and dataset type of a pair of DatasetRef instances, regardless of whether either is resolved:

if ref1.unresolved() == ref2.unresolved():
    ...

Navigation

DatasetRef¶