lsst.daf.butler¶
This module provides an abstracted data access interface.
Python API reference¶
lsst.daf.butler Package¶
Data Access Butler
Functions¶
disableWhenLimited(func) |
Decorator that indicates that a method should raise NotImplementedError on Registries whose limited attribute is True. |
makeBoxWcsRegion(box, wcs, margin) |
Construct a spherical ConvexPolygon from a WCS and a bounding box. |
toNameSet(container) |
Transform an iterable of DimensionElement or names thereof into a DimensionNameSet. |
Classes¶
Butler([config, collection, run]) |
Main entry point for the data access system. |
ButlerConfig([other]) |
Contains the configuration for a Butler |
CompositeAssembler(storageClass) |
Class for providing assembler and disassembler support for composites. |
CompositesConfig([other, validate, …]) |
|
CompositesMap(config) |
Determine whether a specific datasetType or StorageClass should be disassembled. |
Config([other]) |
Implements a datatype that is used by Butler for configuration parameters. |
ConfigSubset([other, validate, …]) |
Config representing a subset of a more general configuration. |
DataId([dataId, dimensions, dimension, …]) |
A dict-like identifier for data usable across multiple collections and DatasetTypes. |
DataIdPacker |
An abstract base class for a bidirectional mappings between a DataId and a packed integer or bytes blob. |
DataIdPackerDimensions(given, required) |
A helper class that holds the three categories of dimensions associated with DataIdPacker objects. |
DatabaseDict(config, types, key, value) |
An abstract base class for dict-like objects with a specific key type and namedtuple values, backed by a database. |
DatasetComponent(name, storageClass, component) |
Component of a dataset and associated information. |
DatasetOriginInfo |
Interface for classes providing collection information for pre-flight. |
DatasetOriginInfoDef(defaultInputs, …[, …]) |
Default implementation of the DatasetOriginInfo. |
DatasetRef(datasetType, dataId[, id, run, hash]) |
Reference to a Dataset in a Registry. |
DatasetType(name, dimensions, storageClass) |
A named category of Datasets that defines how they are organized, related, and stored. |
DatasetTypeNotSupportedError |
A DatasetType is not handled by this routine. |
Datastore(config, registry) |
Datastore interface. |
DatastoreConfig([other, validate, …]) |
|
Dimension(universe, name, config) |
A discrete dimension of data used to organize Datasets and associate them with metadata. |
DimensionConfig([other, validate, …]) |
|
DimensionElement(universe, name, hasRegion, …) |
Base class for elements in the dimension schema. |
DimensionGraph(universe[, dimensions, …]) |
An automatically-expanded collection of both Dimensions and DimensionJoins. |
DimensionJoin(universe, name, config) |
A join that relates two or more Dimensions. |
DimensionKeyDict([other, universe, keys, …]) |
An immutable mapping that uses DimensionElement instances as keys. |
DimensionNameSet(names) |
An incomplete, name-only stand-in for DimensionSet or DimensionGraph. |
DimensionSet(universe, elements[, expand, …]) |
A custom set/dict hybrid class for collections of DimensionElements. |
Execution([startTime, endTime, host, id]) |
Any step in a production. |
FileDescriptor(location, storageClass[, …]) |
Describes a particular file. |
FileTemplate(template) |
Format a path template into a fully expanded path. |
FileTemplates(config[, default]) |
Collection of FileTemplate templates. |
FileTemplatesConfig([other]) |
Configuration information for FileTemplates |
Formatter |
Interface for reading and writing Datasets with a particular StorageClass. |
FormatterFactory() |
Factory for Formatter instances. |
Location(datastoreRoot, uri) |
Identifies a location in the Datastore. |
LocationFactory(datastoreRoot) |
Factory for Location instances. |
MappingFactory(refType) |
Register the mapping of some key to a python type and retrieve instances. |
PreFlightDimensionsRow(dataId, datasetRefs) |
Simple data class holding combination of Dimension values for one row returned by pre-flight solver. |
Quantum(task, run, *args, **kwargs) |
A discrete unit of work that may depend on one or more datasets and produces one or more datasets. |
Registry(registryConfig[, schemaConfig, …]) |
Registry interface. |
RegistryConfig([other, validate, …]) |
|
Run(collection[, environment, pipeline]) |
Represent a processing run. |
Schema([config, limited]) |
The SQL schema for a Butler Registry. |
SchemaBuilder(config[, limited]) |
Builds a Schema step-by-step. |
SchemaConfig([other, validate, …]) |
|
StorageClass([name, pytype, components, …]) |
Class describing how a label maps to a particular Python type. |
StorageClassConfig([other, validate, …]) |
|
StorageClassFactory([config]) |
Factory for StorageClass instances. |
StoredFileInfo(formatter, path, storageClass) |
Information associated with a stored file in a Datastore. |
Class Inheritance Diagram¶
Example Datastores¶
lsst.daf.butler.datastores.posixDatastore Module¶
POSIX datastore.
Classes¶
PosixDatastore(config, registry) |
Basic POSIX filesystem backed Datastore. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.inMemoryDatastore Module¶
In-memory datastore.
Classes¶
StoredItemInfo(timestamp, storageClass[, …]) |
Internal Metadata associated with a DatasetRef. |
InMemoryDatastore(config[, registry]) |
Basic Datastore for writing to an in memory cache. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.chainedDatastore Module¶
Chained datastore.
Classes¶
ChainedDatastore(config[, registry]) |
Chained Datastores to allow read and writes from multiple datastores. |
Class Inheritance Diagram¶
Example Registries¶
lsst.daf.butler.registries.sqlRegistry Module¶
Classes¶
SqlRegistryConfig([other, validate, …]) |
|
SqlRegistry(registryConfig, schemaConfig, …) |
Registry backed by a SQL database. |
Class Inheritance Diagram¶
Example Formatters¶
lsst.daf.butler.formatters.fileFormatter Module¶
Support for reading and writing files to a POSIX file system.
Classes¶
FileFormatter |
Interface for reading and writing files on a POSIX file system. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.jsonFormatter Module¶
Classes¶
JsonFormatter |
Interface for reading and writing Python objects to and from JSON files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.yamlFormatter Module¶
Classes¶
YamlFormatter |
Interface for reading and writing Python objects to and from YAML files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.pickleFormatter Module¶
Formatter associated with Python pickled objects.
Classes¶
PickleFormatter |
Interface for reading and writing Python objects to and from pickle files. |
Class Inheritance Diagram¶
Support API¶
lsst.daf.butler.core.safeFileIo Module¶
Utilities for safe file IO
Functions¶
safeMakeDir(directory) |
Make a directory in a manner avoiding race conditions |
setFileMode(filename) |
Set a file mode according to the user’s umask |
FileForWriteOnceCompareSame(name) |
Context manager to get a file that can be written only once and all other writes will succeed only if they match the initial write. |
SafeFile(name) |
Context manager to create a file in a manner avoiding race conditions |
SafeFilename(name) |
Context manager for creating a file in a manner avoiding race conditions. |
SafeLockedFileForRead(name) |
Context manager for reading a file that may be locked with an exclusive lock via SafeLockedFileForWrite. |
Classes¶
DoNotWrite |
|
FileForWriteOnceCompareSameFailure |
|
SafeLockedFileForWrite(name) |
File-like object that is used to create a file if needed, lock it with an exclusive lock, and contain file descriptors to readable and writable versions of the file. |
Class Inheritance Diagram¶
lsst.daf.butler.core.utils Module¶
Functions¶
iterable(a) |
Make input iterable. |
allSlots(self) |
Return combined __slots__ for all classes in objects mro. |
slotValuesAreEqual(self, other) |
Test for equality by the contents of all slots, including those of its parents. |
slotValuesToHash(self) |
Generate a hash from slot values. |
getFullTypeName(cls) |
Return full type name of the supplied entity. |
getInstanceOf(typeOrName) |
Given the type name or a type, instantiate an object of that type. |
transactional(func) |
Decorator that wraps a method and makes it transactional. |
getObjectSize(obj[, seen]) |
Recursively finds size of objects. |
stripIfNotNone(s) |
Strip leading and trailing whitespace if the given object is not None. |
Classes¶
Singleton |
Metaclass to convert a class to a Singleton. |
PrivateConstructorMeta |
A metaclass that disables regular construction syntax. |