TypelessFormatter

class lsst.daf.butler.formatters.typeless.TypelessFormatter(file_descriptor: FileDescriptor, *, ref: DatasetRef, write_parameters: Mapping[str, Any] | None = None, write_recipes: Mapping[str, Any] | None = None, **kwargs: Any)

Bases: FormatterV2

Formatter V2 base class that attempts to coerce generic objects read in subclasses into the correct Python type.

Notes

This class provides a read() method that will run FormatterV2.read and coerce the return type using a variety of techniques. Use the standard FormatterV2 methods for reading bytes/files and writing bytes/files.

Methods Summary

read([component, expected_size, cache_manager])

Read a Dataset.

Methods Documentation

read(component: str | None = None, expected_size: int = -1, cache_manager: AbstractDatastoreCacheManager | None = None) → Any

Read a Dataset.

Parameters:
componentstr, optional

Component to read from the file. Only used if the StorageClass for reading differed from the StorageClass used to write the file.

expected_sizeint, optional

If known, the expected size of the resource to read. This can be used for verification or to decide whether to do a direct read or a file download. -1 indicates the file size is not known.

cache_managerAbstractDatastoreCacheManager

A cache manager to use to allow a formatter to cache a remote file locally or read a cached file that is already local.

Returns:
in_memory_datasetobject

The requested Dataset.

Raises:
lsst.daf.butler.FormatterNotImplementedError

Raised if no implementations were found that could read this resource.

Notes

This method should not be subclassed. Instead formatter subclasses should re-implement the specific read_from_* methods as appropriate. Each of these methods has a corresponding class property that must be True for the method to be called.

The priority for reading is:

  • read_from_uri

  • read_from_stream

  • read_from_local_file

  • read_from_uri (but with a local file)

Any of these methods can return NotImplemented if there is a desire to skip to the next one in the list. If a dataset is being requested with no component, no parameters, and it should also be added to the local cache, the first two calls will be skipped (unless read_from_stream is the only implemented read method) such that a local file will be used.

A Formatter can also read a file from within a Zip file if the URI associated with the FileDescriptor corresponds to a file with a zip extension and a URI fragment of the form zip-path={path_in_zip}. When reading a file from within a Zip file the priority for reading is:

  • read_from_stream

  • read_from_local_file

  • read_from_uri

There are multiple cases that must be handled for reading:

For a single file:

  • No component requested, read the whole file.

  • Component requested, optionally read the component efficiently, else read the whole file and extract the component.

  • Derived component requested, read whole file or read relevant component and derive.

Disassembled Composite:

  • The file to read here is the component itself. Formatter only knows about this one component file. Should be no component specified in the read call but the FileDescriptor will know which component this is.

  • A derived component. The file to read is a component but not the specified component. The caching needs the component from which it’s derived.