Config¶

class lsst.daf.butler.Config(other=None)¶

Bases: collections.abc.MutableMapping

Implements a datatype that is used by Butler for configuration parameters.

It is essentially a dict with key/value pairs, including nested dicts (as values). In fact, it can be initialized with a dict. This is explained next:

Config extends the dict api so that hierarchical values may be accessed with delimited notation or as a tuple. If a string is given the delimiter is picked up from the first character in that string. For example, foo.getValue(".a.b.c"), foo["a"]["b"]["c"], foo["a", "b", "c"], foo[".a.b.c"], and foo["/a/b/c"] all achieve the same outcome. If the first character is alphanumeric, no delimiter will be used. foo["a.b.c"] will be a single key a.b.c as will foo[":a.b.c"]. Unicode characters can be used as the delimiter for distinctiveness if required.

If a key in the hierarchy starts with a non-alphanumeric character care should be used to ensure that either the tuple interface is used or a distinct delimiter is always given in string form.

Finally, the delimiter can be escaped if it is part of a key and also has to be used as a delimiter. For example, foo[r".a.b\.c"] results in a two element hierarchy of a and b.c. For hard-coded strings it is always better to use a different delimiter in these cases.

Note that adding a multi-level key implicitly creates any nesting levels that do not exist, but removing multi-level keys does not automatically remove empty nesting levels. As a result:

>>> c = Config()
>>> c[".a.b"] = 1
>>> del c[".a.b"]
>>> c["a"]
Config({'a': {}})

Storage formats supported:

yaml: read and write is supported.

Parameters:	other : `str` or `Config` or `dict` Other source of configuration, can be: (`str`) Treated as a path to a config file on disk. Must end with “.yaml”. (`Config`) Copies the other Config’s values into this one. (`dict`) Copies the values from the dict into this Config. If `None` is provided an empty `Config` will be created.

Attributes Summary

includeKey Key used to indicate that another config should be included at this part of the hierarchy.

Methods Summary

`asArray`(name)	Get a value as an array.
`clear`()
`copy`()
`dump`(output)	Writes the config to a yaml stream.
`dumpToFile`(path)	Writes the config to a file.
`dumpToS3File`(bucket, key)	Writes the config to a file in S3 Bucket.
`dumpToUri`(uri[, updateFile, defaultFileName])	Writes the config to location pointed to by given URI.
`get`(k[,d])
`items`()
`keys`()
`merge`(other)	Like Config.update, but will add keys & values from other that DO NOT EXIST in self.
`nameTuples`([topLevelOnly])	Get tuples representing the name hierarchies of all keys.
`names`([topLevelOnly, delimiter])	Get a delimited name of all the keys in the hierarchy.
`pop`(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
`popitem`()	as a 2-tuple; but raise KeyError if D is empty.
`ppprint`()	helper function for debugging, prints a config out in a readable way in the debugger.
`setdefault`(k[,d])
`update`(other)	Like dict.update, but will add or modify keys in nested dicts, instead of overwriting the nested dict entirely.
`updateParameters`(configType, config, full[, …])	Generic helper function for updating specific config parameters.
`values`()

Attributes Documentation

includeKey = 'includeConfigs'¶: Key used to indicate that another config should be included at this part of the hierarchy.

Methods Documentation

asArray(name)¶

Get a value as an array.

May contain one or more elements.

Parameters:	name : `str` Key to use to retrieve value.
Returns:	array : `collections.abc.Sequence` The value corresponding to name, but guaranteed to be returned as a list with at least one element. If the value is a `Sequence` (and not a `str`) the value itself will be returned, else the value will be the first element.

clear() → None. Remove all items from D.¶

copy()¶

dump(output)¶

Writes the config to a yaml stream.

Parameters:	output The YAML stream to use for output.

dumpToFile(path)¶

Writes the config to a file.

Parameters:	path : `str` Path to the file to use for output.

Notes

The name of the config file is stored in the Config object.

dumpToS3File(bucket, key)¶

Writes the config to a file in S3 Bucket.

Parameters:	bucketname: `str` Name of the Bucket into which config will be written. key : `str` Path to the file to use for output, relative to the bucket.

dumpToUri(uri, updateFile=True, defaultFileName='butler.yaml')¶

Writes the config to location pointed to by given URI.

Currently supports ‘s3’ and ‘file’ URI schemes.

Parameters:

uri: `str` or `ButlerURI`: URI of location where the Config will be written.
updateFile : bool, optional: If True and uri does not end on a filename with extension, will append defaultFileName to the target uri. True by default.
defaultFileName : bool, optional: The file name that will be appended to target uri if updateFile is True and uri does not end on a file with an extension.

get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶

items() → a set-like object providing a view on D's items¶

keys() → a set-like object providing a view on D's keys¶

merge(other)¶

Like Config.update, but will add keys & values from other that DO NOT EXIST in self.

Keys and values that already exist in self will NOT be overwritten.

Parameters:	other : `dict` or `Config` Source of configuration:

nameTuples(topLevelOnly=False)¶

Get tuples representing the name hierarchies of all keys.

The tuples returned from this method are guaranteed to be usable to access items in the configuration object.

Parameters:	topLevelOnly : `bool`, optional If False, the default, a full hierarchy of names is returned. If True, only the top level are returned.
Returns:	names : `list` of `tuple` of `str` List of all names present in the `Config` where each element in the list is a `tuple` of strings representing the hierarchy.

names(topLevelOnly=False, delimiter=None)¶

Get a delimited name of all the keys in the hierarchy.

The values returned from this method are guaranteed to be usable to access items in the configuration object.

Parameters:	topLevelOnly : `bool`, optional If False, the default, a full hierarchy of names is returned. If True, only the top level are returned. delimiter : `str`, optional Delimiter to use when forming the keys. If the delimiter is present in any of the keys, it will be escaped in the returned names. If `None` given a delimiter will be automatically provided. The delimiter can not be alphanumeric.
Returns:	names : `list` of `str` List of all names present in the `Config`.
Raises:	ValueError: The supplied delimiter is alphanumeric.

Notes

This is different than the built-in method dict.keys, which will return only the first level keys.

pop(k[, d]) → v, remove specified key and return the corresponding value.¶: If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair¶: as a 2-tuple; but raise KeyError if D is empty.

ppprint()¶

helper function for debugging, prints a config out in a readable way in the debugger.

use: pdb> print(myConfigObject.ppprint())

Returns:	s : `str` A prettyprint formatted string representing the config

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶

update(other)¶

Like dict.update, but will add or modify keys in nested dicts, instead of overwriting the nested dict entirely.

For example, for the given code: foo = {“a”: {“b”: 1}} foo.update({“a”: {“c”: 2}})

Parameters:	other : `dict` or `Config` Source of configuration: If foo is a dict, then after the update foo == {“a”: {“c”: 2}} But if foo is a Config, then after the update foo == {“a”: {“b”: 1, “c”: 2}}

static updateParameters(configType, config, full, toUpdate=None, toCopy=None, overwrite=True)¶

Generic helper function for updating specific config parameters.

Allows for named parameters to be set to new values in bulk, and for other values to be set by copying from a reference config.

Assumes that the supplied config is compatible with configType and will attach the updated values to the supplied config by looking for the related component key. It is assumed that config and full are from the same part of the configuration hierarchy.

Parameters:

configType : ConfigSubset

Config type to use to extract relevant items from config.

config : Config

A Config to update. Only the subset understood by the supplied ConfigSubset will be modified. Default values will not be inserted and the content will not be validated since mandatory keys are allowed to be missing until populated later by merging.

full : Config

A complete config with all defaults expanded that can be converted to a configType. Read-only and will not be modified by this method. Values are read from here if toCopy is defined.

Repository-specific options that should not be obtained from defaults when Butler instances are constructed should be copied from full to config.

toUpdate : dict, optional

A dict defining the keys to update and the new value to use. The keys and values can be any supported by Config assignment.

toCopy : tuple, optional

tuple of keys whose values should be copied from full into config.

overwrite : bool, optional

If False, do not modify a value in config if the key already exists. Default is always to overwrite.

Raises:

ValueError: Neither toUpdate not toCopy were defined.

values() → an object providing a view on D's values¶

Navigation

Config¶