Packages#

class lsst.utils.packages.Packages#

Bases: dict

A table of packages and their versions.

There are a few different types of packages, and their versions are collected in different ways:

Installed Conda packages are obtained via the Conda API. Conda is not required.
Python modules (e.g., afw, numpy; galsim is also in this group even though we only use it through the library, because no version information is currently provided through the library): we get their version from the __version__ module variable. Note that this means that we’re only aware of modules that have already been imported.
Other packages provide no run-time accessible version information (e.g., astrometry_net): we get their version from interrogating the environment. Currently, that means EUPS; if EUPS is replaced or dropped then we’ll need to consider an alternative means of getting this version information.
Local versions of packages (a non-installed EUPS package, selected with setup -r /path/to/package): we identify these through the environment (EUPS again) and use as a version the path supplemented with the git SHA and, if the git repo isn’t clean, an MD5 of the diff.

These package versions are collected and stored in a Packages object, which provides useful comparison and persistence features.

Example usage:

from lsst.utils.packages import Packages
pkgs = Packages.fromSystem()
print("Current packages:", pkgs)
old = Packages.read("/path/to/packages.pickle")
print("Old packages:", old)
print("Missing packages compared to before:", pkgs.missing(old))
print("Extra packages compared to before:", pkgs.extra(old))
print("Different packages: ", pkgs.difference(old))
old.update(pkgs)  # Include any new packages in the old
old.write("/path/to/packages.pickle").

Notes#

This is a wrapper around a dict with some convenience methods.

Attributes Summary

formats

Methods Summary

`difference`(other)	Get packages in symmetric difference of self and another `Packages` object.
`extra`(other)	Get packages in self but not in another `Packages` object.
`fromBytes`(data, format)	Construct the object from a byte representation.
`fromSystem`([include_all])	Construct a `Packages` by examining the system.
`missing`(other)	Get packages in another `Packages` object but missing from self.
`read`(filename)	Read packages from filename.
`toBytes`(format)	Convert the object to a serialized bytes form using the specified format.
`write`(filename)	Write to file.

Attributes Documentation

formats: ClassVar[dict[str, str]] = {'.json': 'json', '.pickle': 'pickle', '.pkl': 'pickle', '.yaml': 'yaml'}#

Methods Documentation

difference(other: Mapping) → dict[str, tuple[str, str]]#

Get packages in symmetric difference of self and another Packages object.

Parameters#

otherPackages: Other packages to compare against.

Returns#

differencedict [str, tuple [ str, str ]]: Packages in symmetric difference. Keys (type str) are package names; values (type tuple [ str, str ]) are their versions.

extra(other: Mapping) → dict[str, str]#

Get packages in self but not in another Packages object.

Parameters#

otherPackages or Mapping: Other packages to compare against.

Returns#

extradict: Extra packages. Keys (type str) are package names; values (type str) are their versions.

classmethod fromBytes(data: bytes, format: str) → Packages#

Construct the object from a byte representation.

Parameters#

databytes: The serialized form of this object in bytes.
formatstr: The format of those bytes. Can be yaml, json, or pickle.

Returns#

packagesPackages: The package information read from the input data.

classmethod fromSystem(include_all: bool = False) → Packages#

Construct a Packages by examining the system.

Determine packages by examining python’s installed packages (by default filtered by sys.modules) or distributions, conda libraries and EUPS. EUPS packages take precedence over conda and general python packages.

Parameters#

include_allbool, optional: If False, will only include imported Python packages, installed Conda packages and locally-setup EUPS packages. If True all installed Python distributions and conda packages will be reported as well as all EUPS packages that are set up.

Returns#

packagesPackages: All version package information that could be obtained.

Note#

The names of Python distributions can differ from the names of the Python packages installed by those distributions. Since include_all set to True uses Python distributions and False uses Python packages do not expect that the answers are directly comparable.

missing(other: Mapping) → dict[str, str]#

Get packages in another Packages object but missing from self.

Parameters#

otherPackages: Other packages to compare against.

Returns#

missingdict [str, str]: Missing packages. Keys (type str) are package names; values (type str) are their versions.

classmethod read(filename: str) → Packages#

Read packages from filename.

Parameters#

filenamestr: Filename from which to read. The format is determined from the file extension. Currently support .pickle, .pkl, .json, and .yaml.

Returns#

packagesPackages: The packages information read from the file.

toBytes(format: str) → bytes#

Convert the object to a serialized bytes form using the specified format.

Parameters#

formatstr: Format to use when serializing. Can be yaml, json, or pickle.

Returns#

databytes: Byte string representing the serialized object.

write(filename: str) → None#

Write to file.

Parameters#

filenamestr: Filename to which to write. The format of the data file is determined from the file extension. Currently supports .pickle, .json, and .yaml.