Formatter

class lsst.daf.butler.Formatter(fileDescriptor: FileDescriptor, dataId: Optional[DataCoordinate] = None, writeParameters: Optional[Dict[str, Any]] = None, writeRecipes: Optional[Dict[str, Any]] = None)

Bases: object

Interface for reading and writing Datasets with a particular StorageClass.

Parameters:
fileDescriptor : FileDescriptor, optional

Identifies the file to read or write, and the associated storage classes and parameter information. Its value can be None if the caller will never call Formatter.read or Formatter.write.

dataId : DataCoordinate, optional

Data ID associated with this formatter.

writeParameters : dict, optional

Any parameters to be hard-coded into this instance to control how the dataset is serialized.

Attributes Summary

dataId DataId associated with this formatter (DataCoordinate)
fileDescriptor FileDescriptor associated with this formatter (FileDescriptor, read-only)
supportedExtensions Set of all extensions supported by this formatter.
supportedWriteParameters Parameters understood by this formatter that can be used to control how a dataset is serialized.
unsupportedParameters Set of read parameters not understood by this Formatter.
writeParameters Parameters to use when writing out datasets.
writeRecipes Detailed write Recipes indexed by recipe name.

Methods Summary

can_read_bytes() Indicate if this formatter can format from bytes.
fromBytes(serializedDataset, component) Reads serialized data into a Dataset or its component.
makeUpdatedLocation(location) Return a new Location instance updated with this formatter’s extension.
name() Returns the fully qualified name of the formatter.
predictPath() Return the path that would be returned by write, without actually writing.
read(component) Read a Dataset.
segregateParameters(parameters, Any]] = None) Segregate the supplied parameters into those understood by the formatter and those not understood by the formatter.
toBytes(inMemoryDataset) Serialize the Dataset to bytes based on formatter.
validateExtension(location) Check that the provided location refers to a file extension that is understood by this formatter.
validateWriteRecipes(recipes, Any]]) Validate supplied recipes for this formatter.
write(inMemoryDataset) Write a Dataset.

Attributes Documentation

dataId

DataId associated with this formatter (DataCoordinate)

fileDescriptor

FileDescriptor associated with this formatter (FileDescriptor, read-only)

supportedExtensions = frozenset()

Set of all extensions supported by this formatter.

Only expected to be populated by Formatters that write files. Any extension assigned to the extension property will be automatically included in the list of supported extensions.

supportedWriteParameters = None

Parameters understood by this formatter that can be used to control how a dataset is serialized. None indicates that no parameters are supported.

unsupportedParameters = frozenset()

Set of read parameters not understood by this Formatter. An empty set means all parameters are supported. None indicates that no parameters are supported. These param (frozenset).

writeParameters

Parameters to use when writing out datasets.

writeRecipes

Detailed write Recipes indexed by recipe name.

Methods Documentation

classmethod can_read_bytes() → bool

Indicate if this formatter can format from bytes.

Returns:
can : bool

True if the fromBytes method is implemented.

fromBytes(serializedDataset: bytes, component: Optional[str] = None) → object

Reads serialized data into a Dataset or its component.

Parameters:
serializedDataset : bytes

Bytes object to unserialize.

component : str, optional

Component to read from the Dataset. Only used if the StorageClass for reading differed from the StorageClass used to write the file.

Returns:
inMemoryDataset : object

The requested data as a Python object. The type of object is controlled by the specific formatter.

makeUpdatedLocation(location: lsst.daf.butler.core.location.Location) → lsst.daf.butler.core.location.Location

Return a new Location instance updated with this formatter’s extension.

Parameters:
location : Location

The location to update.

Returns:
updated : Location

A new Location with a new file extension applied.

Raises:
NotImplementedError

Raised if there is no extension attribute associated with this formatter.

Notes

This method is available to all Formatters but might not be implemented by all formatters. It requires that a formatter set an extension attribute containing the file extension used when writing files. If extension is None the supplied file will not be updated. Not all formatters write files so this is not defined in the base class.

classmethod name() → str

Returns the fully qualified name of the formatter.

Returns:
name : str

Fully-qualified name of formatter class.

predictPath() → str

Return the path that would be returned by write, without actually writing.

Uses the FileDescriptor associated with the instance.

Returns:
path : str

Path within datastore that would be associated with the location stored in this Formatter.

read(component: Optional[str] = None) → Any

Read a Dataset.

Parameters:
component : str, optional

Component to read from the file. Only used if the StorageClass for reading differed from the StorageClass used to write the file.

Returns:
inMemoryDataset : object

The requested Dataset.

segregateParameters(parameters: Optional[Dict[str, Any]] = None) → Tuple[Dict[KT, VT], Dict[KT, VT]]

Segregate the supplied parameters into those understood by the formatter and those not understood by the formatter.

Any unsupported parameters are assumed to be usable by associated assemblers.

Parameters:
parameters : dict, optional

Parameters with values that have been supplied by the caller and which might be relevant for the formatter. If None parameters will be read from the registered FileDescriptor.

Returns:
supported : dict

Those parameters supported by this formatter.

unsupported : dict

Those parameters not supported by this formatter.

toBytes(inMemoryDataset: Any) → bytes

Serialize the Dataset to bytes based on formatter.

Parameters:
inMemoryDataset : object

The Python object to serialize.

Returns:
serializedDataset : bytes

Bytes representing the serialized dataset.

classmethod validateExtension(location: lsst.daf.butler.core.location.Location) → None

Check that the provided location refers to a file extension that is understood by this formatter.

Parameters:
location : Location

Location from which to extract a file extension.

Raises:
NotImplementedError

Raised if file extensions are a concept not understood by this formatter.

ValueError

Raised if the formatter does not understand this extension.

Notes

This method is available to all Formatters but might not be implemented by all formatters. It requires that a formatter set an extension attribute containing the file extension used when writing files. If extension is None only the set of supported extensions will be examined.

classmethod validateWriteRecipes(recipes: Optional[Mapping[str, Any]]) → Optional[Mapping[str, Any]]

Validate supplied recipes for this formatter.

The recipes are supplemented with default values where appropriate.

Parameters:
recipes : dict

Recipes to validate.

Returns:
validated : dict

Validated recipes.

Raises:
RuntimeError

Raised if validation fails. The default implementation raises if any recipes are given.

write(inMemoryDataset: Any) → None

Write a Dataset.

Parameters:
inMemoryDataset : object

The Dataset to store.