Packages

class lsst.utils.packages.Packages

Bases: dict

A table of packages and their versions.

There are a few different types of packages, and their versions are collected in different ways:

  1. Installed Conda packages are obtained via the Conda API. Conda is not required.
  2. Python modules (e.g., afw, numpy; galsim is also in this group even though we only use it through the library, because no version information is currently provided through the library): we get their version from the __version__ module variable. Note that this means that we’re only aware of modules that have already been imported.
  3. Other packages provide no run-time accessible version information (e.g., astrometry_net): we get their version from interrogating the environment. Currently, that means EUPS; if EUPS is replaced or dropped then we’ll need to consider an alternative means of getting this version information.
  4. Local versions of packages (a non-installed EUPS package, selected with setup -r /path/to/package): we identify these through the environment (EUPS again) and use as a version the path supplemented with the git SHA and, if the git repo isn’t clean, an MD5 of the diff.

These package versions are collected and stored in a Packages object, which provides useful comparison and persistence features.

Example usage:

from lsst.utils.packages import Packages
pkgs = Packages.fromSystem()
print("Current packages:", pkgs)
old = Packages.read("/path/to/packages.pickle")
print("Old packages:", old)
print("Missing packages compared to before:", pkgs.missing(old))
print("Extra packages compared to before:", pkgs.extra(old))
print("Different packages: ", pkgs.difference(old))
old.update(pkgs)  # Include any new packages in the old
old.write("/path/to/packages.pickle")
Parameters:
packages : dict

A mapping {package: version} where both keys and values are type str.

Notes

This is a wrapper around a dict with some convenience methods.

Attributes Summary

formats

Methods Summary

clear()
copy()
difference(other) Get packages in symmetric difference of self and another Packages object.
extra(other) Get packages in self but not in another Packages object.
fromBytes(data, format) Construct the object from a byte representation.
fromSystem() Construct a Packages by examining the system.
fromkeys(iterable[, value]) Create a new dictionary with keys from iterable and values set to value.
get(key[, default]) Return the value for key if key is in the dictionary, else default.
items()
keys()
missing(other) Get packages in another Packages object but missing from self.
pop(key[, default]) If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem(/) Remove and return a (key, value) pair as a 2-tuple.
read(filename) Read packages from filename.
setdefault(key[, default]) Insert key with a value of default if key is not in the dictionary.
toBytes(format) Convert the object to a serialized bytes form using the specified format.
update([E, ]**F) If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values()
write(filename) Write to file.

Attributes Documentation

formats = {'.json': 'json', '.pickle': 'pickle', '.pkl': 'pickle', '.yaml': 'yaml'}

Methods Documentation

clear() → None. Remove all items from D.
copy() → a shallow copy of D
difference(other: collections.abc.Mapping) → Dict[str, Tuple[str, str]]

Get packages in symmetric difference of self and another Packages object.

Parameters:
other : Packages

Other packages to compare against.

Returns:
difference : dict [str, tuple [str, str]]

Packages in symmetric difference. Keys (type str) are package names; values (type tuple`[`str, str]) are their versions.

extra(other: collections.abc.Mapping) → Dict[str, str]

Get packages in self but not in another Packages object.

Parameters:
other : Packages or Mapping

Other packages to compare against.

Returns:
extra : dict

Extra packages. Keys (type str) are package names; values (type str) are their versions.

classmethod fromBytes(data: bytes, format: str) → lsst.utils.packages.Packages

Construct the object from a byte representation.

Parameters:
data : bytes

The serialized form of this object in bytes.

format : str

The format of those bytes. Can be yaml, json, or pickle.

Returns:
packages : Packages

The package information read from the input data.

classmethod fromSystem() → lsst.utils.packages.Packages

Construct a Packages by examining the system.

Determine packages by examining python’s sys.modules, conda libraries and EUPS. EUPS packages take precedence over conda and general python packages.

Returns:
packages : Packages

All version package information that could be obtained.

fromkeys(iterable, value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D's items
keys() → a set-like object providing a view on D's keys
missing(other: collections.abc.Mapping) → Dict[str, str]

Get packages in another Packages object but missing from self.

Parameters:
other : Packages

Other packages to compare against.

Returns:
missing : dict [str, str]

Missing packages. Keys (type str) are package names; values (type str) are their versions.

pop(key, default=<unrepresentable>, /)

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem(/)

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

classmethod read(filename: str) → lsst.utils.packages.Packages

Read packages from filename.

Parameters:
filename : str

Filename from which to read. The format is determined from the file extension. Currently support .pickle, .pkl, .json, and .yaml.

Returns:
packages : Packages

The packages information read from the file.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

toBytes(format: str) → bytes

Convert the object to a serialized bytes form using the specified format.

Parameters:
format : str

Format to use when serializing. Can be yaml, json, or pickle.

Returns:
data : bytes

Byte string representing the serialized object.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values
write(filename: str) → None

Write to file.

Parameters:
filename : str

Filename to which to write. The format of the data file is determined from the file extension. Currently supports .pickle, .json, and .yaml