FileFormatter¶

class lsst.daf.butler.formatters.file.FileFormatter(fileDescriptor: FileDescriptor, dataId: Optional[DataCoordinate] = None, writeParameters: Optional[Dict[str, Any]] = None, writeRecipes: Optional[Dict[str, Any]] = None)¶

Bases: lsst.daf.butler.Formatter

Interface for reading and writing files on a POSIX file system.

Attributes Summary

`dataId`	DataId associated with this formatter (`DataCoordinate`)
`extension`	Default file extension to use for writing files.
`fileDescriptor`	FileDescriptor associated with this formatter (`FileDescriptor`, read-only)
`supportedExtensions`
`supportedWriteParameters`
`unsupportedParameters`
`writeParameters`	Parameters to use when writing out datasets.
`writeRecipes`	Detailed write Recipes indexed by recipe name.

Methods Summary

`fromBytes`(serializedDataset, component)	Reads serialized data into a Dataset or its component.
`makeUpdatedLocation`(location)	Return a new `Location` instance updated with this formatter’s extension.
`name`()	Returns the fully qualified name of the formatter.
`predictPath`()	Return the path that would be returned by write, without actually writing.
`read`(component)	Read data from a file.
`segregateParameters`(parameters, Any]] = None)	Segregate the supplied parameters into those understood by the formatter and those not understood by the formatter.
`toBytes`(inMemoryDataset)	Serialize the Dataset to bytes based on formatter.
`validateExtension`(location)	Check that the provided location refers to a file extension that is understood by this formatter.
`validateWriteRecipes`(recipes, Any]])	Validate supplied recipes for this formatter.
`write`(inMemoryDataset)	Write a Python object to a file.

Attributes Documentation

dataId¶: DataId associated with this formatter (DataCoordinate)

extension = None¶: Default file extension to use for writing files. None means that no modifications will be made to the supplied file extension. (str)

fileDescriptor¶: FileDescriptor associated with this formatter (FileDescriptor, read-only)

supportedExtensions = frozenset()¶

supportedWriteParameters = None¶

unsupportedParameters = frozenset()¶

writeParameters¶: Parameters to use when writing out datasets.

writeRecipes¶: Detailed write Recipes indexed by recipe name.

Methods Documentation

fromBytes(serializedDataset: bytes, component: Optional[str] = None) → Any¶

Reads serialized data into a Dataset or its component.

Parameters:	serializedDataset : `bytes` Bytes object to unserialize. component : `str`, optional Component to read from the Dataset. Only used if the `StorageClass` for reading differed from the `StorageClass` used to write the file.
Returns:	inMemoryDataset : `object` The requested data as a Python object. The type of object is controlled by the specific formatter.
Raises:	NotImplementedError Formatter does not support reading from bytes.

makeUpdatedLocation(location: lsst.daf.butler.core.location.Location) → lsst.daf.butler.core.location.Location¶

Return a new Location instance updated with this formatter’s extension.

Parameters:	location : `Location` The location to update.
Returns:	updated : `Location` A new `Location` with a new file extension applied.
Raises:	NotImplementedError Raised if there is no `extension` attribute associated with this formatter.

Notes

This method is available to all Formatters but might not be implemented by all formatters. It requires that a formatter set an extension attribute containing the file extension used when writing files. If extension is None the supplied file will not be updated. Not all formatters write files so this is not defined in the base class.

classmethod name() → str¶

Returns the fully qualified name of the formatter.

Returns:	name : `str` Fully-qualified name of formatter class.

predictPath() → str¶

Return the path that would be returned by write, without actually writing.

Uses the FileDescriptor associated with the instance.

Returns:	path : `str` Path within datastore that would be associated with the location stored in this `Formatter`.

read(component: Optional[str] = None) → Any¶

Read data from a file.

Parameters:	fileDescriptor : `FileDescriptor` Identifies the file to read, type to read it into and parameters to be used for reading. component : `str`, optional Component to read from the file. Only used if the `StorageClass` for reading differed from the `StorageClass` used to write the file.
Returns:	inMemoryDataset : `object` The requested data as a Python object. The type of object is controlled by the specific formatter.
Raises:	ValueError Component requested but this file does not seem to be a concrete composite. NotImplementedError Formatter does not implement a method to read from files.

segregateParameters(parameters: Optional[Dict[str, Any]] = None) → Tuple[Dict[KT, VT], Dict[KT, VT]]¶

Segregate the supplied parameters into those understood by the formatter and those not understood by the formatter.

Any unsupported parameters are assumed to be usable by associated assemblers.

Parameters:	parameters : `dict`, optional Parameters with values that have been supplied by the caller and which might be relevant for the formatter. If `None` parameters will be read from the registered `FileDescriptor`.
Returns:	supported : `dict` Those parameters supported by this formatter. unsupported : `dict` Those parameters not supported by this formatter.

toBytes(inMemoryDataset: Any) → bytes¶

Serialize the Dataset to bytes based on formatter.

Parameters:	inMemoryDataset : `object` Object to serialize.
Returns:	serializedDataset : `bytes` Bytes representing the serialized dataset.
Raises:	NotImplementedError Formatter does not support reading from bytes.

classmethod validateExtension(location: lsst.daf.butler.core.location.Location) → None¶

Check that the provided location refers to a file extension that is understood by this formatter.

Parameters:	location : `Location` Location from which to extract a file extension.
Raises:	NotImplementedError Raised if file extensions are a concept not understood by this formatter. ValueError Raised if the formatter does not understand this extension.

Notes

This method is available to all Formatters but might not be implemented by all formatters. It requires that a formatter set an extension attribute containing the file extension used when writing files. If extension is None only the set of supported extensions will be examined.

classmethod validateWriteRecipes(recipes: Optional[Mapping[str, Any]]) → Optional[Mapping[str, Any]]¶

Validate supplied recipes for this formatter.

The recipes are supplemented with default values where appropriate.

Parameters:	recipes : `dict` Recipes to validate.
Returns:	validated : `dict` Validated recipes.
Raises:	RuntimeError Raised if validation fails. The default implementation raises if any recipes are given.

write(inMemoryDataset: Any) → str¶

Write a Python object to a file.

Parameters:	inMemoryDataset : `object` The Python object to store.
Returns:	path : `str` The path where the primary file is stored within the datastore.

Navigation

FileFormatter¶