FormatterFactory

class lsst.daf.butler.FormatterFactory

Bases: object

Factory for Formatter instances.

Attributes Summary

defaultKey Configuration key associated with default write parameter settings.
writeRecipesKey Configuration key associated with write recipes.

Methods Summary

getFormatter(entity, …) Get a new formatter instance.
getFormatterClass(entity, …) Get the matching formatter class.
getFormatterClassWithMatch(entity, …) Get the matching formatter class along with the registry key.
getFormatterWithMatch(entity, …) Get a new formatter instance along with the matching registry key.
getLookupKeys() Retrieve the look up keys for all the registry entries.
registerFormatter(type_, str, …) Register a Formatter.
registerFormatters(config, *, universe) Bulk register formatters from a config.

Attributes Documentation

defaultKey = LookupKey(name='default',)

Configuration key associated with default write parameter settings.

writeRecipesKey = LookupKey(name='write_recipes',)

Configuration key associated with write recipes.

Methods Documentation

getFormatter(entity: Union[lsst.daf.butler.core.datasets.type.DatasetType, lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.core.storageClass.StorageClass, str], *args, **kwargs) → lsst.daf.butler.core.formatter.Formatter

Get a new formatter instance.

Parameters:
entity : DatasetRef, DatasetType, StorageClass, or str

Entity to use to determine the formatter to return. StorageClass will be used as a last resort if DatasetRef or DatasetType instance is provided. Supports instrument override if a DatasetRef is provided configured with an instrument value for the data ID.

args : tuple

Positional arguments to use pass to the object constructor.

kwargs : dict

Keyword arguments to pass to object constructor.

Returns:
formatter : Formatter

An instance of the registered formatter.

getFormatterClass(entity: Union[lsst.daf.butler.core.datasets.type.DatasetType, lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.core.storageClass.StorageClass, str]) → Type[CT_co]

Get the matching formatter class.

Parameters:
entity : DatasetRef, DatasetType, StorageClass, or str

Entity to use to determine the formatter to return. StorageClass will be used as a last resort if DatasetRef or DatasetType instance is provided. Supports instrument override if a DatasetRef is provided configured with an instrument value for the data ID.

Returns:
formatter : type

The class of the registered formatter.

getFormatterClassWithMatch(entity: Union[lsst.daf.butler.core.datasets.type.DatasetType, lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.core.storageClass.StorageClass, str]) → Tuple[lsst.daf.butler.core.configSupport.LookupKey, Type[lsst.daf.butler.core.formatter.Formatter], Dict[str, Any]]

Get the matching formatter class along with the registry key.

Parameters:
entity : DatasetRef, DatasetType, StorageClass, or str

Entity to use to determine the formatter to return. StorageClass will be used as a last resort if DatasetRef or DatasetType instance is provided. Supports instrument override if a DatasetRef is provided configured with an instrument value for the data ID.

Returns:
matchKey : LookupKey

The key that resulted in the successful match.

formatter : type

The class of the registered formatter.

formatter_kwargs : dict

Keyword arguments that are associated with this formatter entry.

getFormatterWithMatch(entity: Union[lsst.daf.butler.core.datasets.type.DatasetType, lsst.daf.butler.core.datasets.ref.DatasetRef, lsst.daf.butler.core.storageClass.StorageClass, str], *args, **kwargs) → Tuple[lsst.daf.butler.core.configSupport.LookupKey, lsst.daf.butler.core.formatter.Formatter]

Get a new formatter instance along with the matching registry key.

Parameters:
entity : DatasetRef, DatasetType, StorageClass, or str

Entity to use to determine the formatter to return. StorageClass will be used as a last resort if DatasetRef or DatasetType instance is provided. Supports instrument override if a DatasetRef is provided configured with an instrument value for the data ID.

args : tuple

Positional arguments to use pass to the object constructor.

kwargs : dict

Keyword arguments to pass to object constructor.

Returns:
matchKey : LookupKey

The key that resulted in the successful match.

formatter : Formatter

An instance of the registered formatter.

getLookupKeys() → Set[lsst.daf.butler.core.configSupport.LookupKey]

Retrieve the look up keys for all the registry entries.

Returns:
keys : set of LookupKey

The keys available for matching in the registry.

registerFormatter(type_: Union[lsst.daf.butler.core.configSupport.LookupKey, str, lsst.daf.butler.core.storageClass.StorageClass, lsst.daf.butler.core.datasets.type.DatasetType], formatter: str, *, overwrite: bool = False, **kwargs) → None

Register a Formatter.

Parameters:
type_ : LookupKey, str, StorageClass or DatasetType

Type for which this formatter is to be used. If a LookupKey is not provided, one will be constructed from the supplied string or by using the name property of the supplied entity.

formatter : str or class of type Formatter

Identifies a Formatter subclass to use for reading and writing Datasets of this type. Can be a Formatter class.

overwrite : bool, optional

If True an existing entry will be replaced by the new value. Default is False.

kwargs : dict

Keyword arguments to always pass to object constructor when retrieved.

Raises:
ValueError

Raised if the formatter does not name a valid formatter type and overwrite is False.

registerFormatters(config: lsst.daf.butler.core.config.Config, *, universe: lsst.daf.butler.core.dimensions._universe.DimensionUniverse) → None

Bulk register formatters from a config.

Parameters:
config : Config

formatters section of a configuration.

universe : DimensionUniverse, optional

Set of all known dimensions, used to expand and validate any used in lookup keys.

Notes

The configuration can include one level of hierarchy where an instrument-specific section can be defined to override more general template specifications. This is represented in YAML using a key of form instrument<name> which can then define templates that will be returned if a DatasetRef contains a matching instrument name in the data ID.

The config is parsed using the function processLookupConfigs.

The values for formatter entries can be either a simple string referring to a python type or a dict representing the formatter and parameters to be hard-coded into the formatter constructor. For the dict case the following keys are supported:

  • formatter: The python type to be used as the formatter class.
  • parameters: A further dict to be passed directly to the
    writeParameters Formatter constructor to seed it. These parameters are validated at instance creation and not at configuration.

Additionally, a special default section can be defined that uses the formatter type (class) name as the keys and specifies default write parameters that should be used whenever an instance of that class is constructed.

formatters:
  default:
    lsst.daf.butler.formatters.example.ExampleFormatter:
      max: 10
      min: 2
      comment: Default comment
  calexp: lsst.daf.butler.formatters.example.ExampleFormatter
  coadd:
    formatter: lsst.daf.butler.formatters.example.ExampleFormatter
    parameters:
      max: 5

Any time an ExampleFormatter is constructed it will use those parameters. If an explicit entry later in the configuration specifies a different set of parameters, the two will be merged with the later entry taking priority. In the example above calexp will use the default parameters but coadd will override the value for max.

Formatter configuration can also include a special section describing collections of write parameters that can be accessed through a simple label. This allows common collections of options to be specified in one place in the configuration and reused later. The write_recipes section is indexed by Formatter class name and each key is the label to associate with the parameters.

formatters:
  write_recipes:
    lsst.obs.base.formatters.fitsExposure.FixExposureFormatter:
      lossless:
        ...
      noCompression:
        ...

By convention a formatter that uses write recipes will support a recipe write parameter that will refer to a recipe name in the write_recipes component. The Formatter will be constructed in the FormatterFactory with all the relevant recipes and will not attempt to filter by looking at writeParameters in advance. See the specific formatter documentation for details on acceptable recipe options.