lsst.daf.butler¶
This module provides an abstracted data access interface.
Python API reference¶
lsst.daf.butler Package¶
Data Access Butler
Functions¶
conformSet (container) |
Transform an iterable of DimensionElement or names thereof into an object with a set-like .names attribute. |
disableWhenLimited (func) |
Decorator that indicates that a method should raise NotImplementedError on Registries whose limited attribute is True . |
makeBoxWcsRegion (box, wcs, margin) |
Construct a spherical ConvexPolygon from a WCS and a bounding box. |
Classes¶
AmbiguousDatasetError |
Exception raised when a DatasetRef has no ID and a Registry operation requires one. |
Butler ([config, collection, run]) |
Main entry point for the data access system. |
ButlerConfig ([other]) |
Contains the configuration for a Butler |
CompositeAssembler (storageClass) |
Class for providing assembler and disassembler support for composites. |
CompositesConfig ([other, validate, …]) |
|
CompositesMap (config) |
Determine whether a specific datasetType or StorageClass should be disassembled. |
Config ([other]) |
Implements a datatype that is used by Butler for configuration parameters. |
ConfigSubset ([other, validate, …]) |
Config representing a subset of a more general configuration. |
ConflictingDefinitionError |
Exception raised when trying to insert a database record when a conflicting record already exists. |
DataId ([dataId, dimensions, dimension, …]) |
A dict-like identifier for data usable across multiple collections and DatasetType s. |
DataIdPacker |
An abstract base class for a bidirectional mappings between a DataId and a packed integer or bytes blob. |
DataIdPackerDimensions (given, required) |
A helper class that holds the three categories of dimensions associated with DataIdPacker objects. |
DatabaseDict (config, types, key, value) |
An abstract base class for dict-like objects with a specific key type and namedtuple values, backed by a database. |
DatasetComponent (name, storageClass, component) |
Component of a dataset and associated information. |
DatasetOriginInfo |
Interface for classes providing collection information for pre-flight. |
DatasetOriginInfoDef (defaultInputs, …[, …]) |
Default implementation of the DatasetOriginInfo. |
DatasetRef (datasetType, dataId[, id, run, …]) |
Reference to a Dataset in a Registry . |
DatasetType (name, dimensions, storageClass) |
A named category of Datasets that defines how they are organized, related, and stored. |
DatasetTypeNotSupportedError |
A DatasetType is not handled by this routine. |
Datastore (config, registry) |
Datastore interface. |
DatastoreConfig ([other, validate, …]) |
|
Dimension (universe, name, config) |
A discrete dimension of data used to organize Datasets and associate them with metadata. |
DimensionConfig ([other, validate, …]) |
|
DimensionElement (universe, name, hasRegion, …) |
Base class for elements in the dimension schema. |
DimensionGraph (universe, elements, …) |
An automatically-expanded collection of both Dimension s and DimensionJoin s. |
DimensionJoin (universe, name, config) |
A join that relates two or more Dimension s. |
DimensionKeyDict ([other, universe, keys, …]) |
An immutable mapping that uses DimensionElement instances as keys. |
DimensionNameSet (names) |
An incomplete, name-only stand-in for DimensionSet or DimensionGraph . |
DimensionSet (universe, elements[, expand, …]) |
A custom set/dict hybrid class for collections of DimensionElement s. |
DimensionUniverse () |
Specialization for a DimensionGraph that defines a complete system of dimensions. |
Execution ([startTime, endTime, host, id]) |
Any step in a production. |
FileDescriptor (location, storageClass[, …]) |
Describes a particular file. |
FileTemplate (template) |
Format a path template into a fully expanded path. |
FileTemplates (config[, default]) |
Collection of FileTemplate templates. |
FileTemplatesConfig ([other]) |
Configuration information for FileTemplates |
Formatter |
Interface for reading and writing Datasets with a particular StorageClass . |
FormatterFactory () |
Factory for Formatter instances. |
Location (datastoreRoot, uri) |
Identifies a location in the Datastore . |
LocationFactory (datastoreRoot) |
Factory for Location instances. |
MappingFactory (refType) |
Register the mapping of some key to a python type and retrieve instances. |
PreFlightDimensionsRow (dataId, datasetRefs) |
Simple data class holding combination of Dimension values for one row returned by pre-flight solver. |
Quantum (task, run, *args, **kwargs) |
A discrete unit of work that may depend on one or more datasets and produces one or more datasets. |
Registry (registryConfig[, schemaConfig, …]) |
Registry interface. |
RegistryConfig ([other, validate, …]) |
|
Run (collection[, environment, pipeline]) |
Represent a processing run. |
Schema ([config, limited]) |
The SQL schema for a Butler Registry. |
SchemaBuilder (config[, limited]) |
Builds a Schema step-by-step. |
SchemaConfig ([other, validate, …]) |
|
StorageClass ([name, pytype, components, …]) |
Class describing how a label maps to a particular Python type. |
StorageClassConfig ([other, validate, …]) |
|
StorageClassFactory ([config]) |
Factory for StorageClass instances. |
StoredFileInfo (formatter, path, storageClass) |
Information associated with a stored file in a Datastore. |
Class Inheritance Diagram¶
Example Datastores¶
lsst.daf.butler.datastores.posixDatastore Module¶
POSIX datastore.
Classes¶
PosixDatastore (config, registry) |
Basic POSIX filesystem backed Datastore. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.inMemoryDatastore Module¶
In-memory datastore.
Classes¶
StoredItemInfo (timestamp, storageClass[, …]) |
Internal Metadata associated with a DatasetRef. |
InMemoryDatastore (config[, registry]) |
Basic Datastore for writing to an in memory cache. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.chainedDatastore Module¶
Chained datastore.
Classes¶
ChainedDatastore (config[, registry]) |
Chained Datastores to allow read and writes from multiple datastores. |
Class Inheritance Diagram¶
Example Registries¶
lsst.daf.butler.registries.sqlRegistry Module¶
Classes¶
SqlRegistryConfig ([other, validate, …]) |
|
SqlRegistry (registryConfig, schemaConfig, …) |
Registry backed by a SQL database. |
Class Inheritance Diagram¶
Example Formatters¶
lsst.daf.butler.formatters.fileFormatter Module¶
Support for reading and writing files to a POSIX file system.
Classes¶
FileFormatter |
Interface for reading and writing files on a POSIX file system. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.jsonFormatter Module¶
Classes¶
JsonFormatter |
Interface for reading and writing Python objects to and from JSON files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.yamlFormatter Module¶
Classes¶
YamlFormatter |
Interface for reading and writing Python objects to and from YAML files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.pickleFormatter Module¶
Formatter associated with Python pickled objects.
Classes¶
PickleFormatter |
Interface for reading and writing Python objects to and from pickle files. |
Class Inheritance Diagram¶
Support API¶
lsst.daf.butler.core.safeFileIo Module¶
Utilities for safe file IO
Functions¶
safeMakeDir (directory) |
Make a directory in a manner avoiding race conditions |
setFileMode (filename) |
Set a file mode according to the user’s umask |
FileForWriteOnceCompareSame (name) |
Context manager to get a file that can be written only once and all other writes will succeed only if they match the initial write. |
SafeFile (name) |
Context manager to create a file in a manner avoiding race conditions |
SafeFilename (name) |
Context manager for creating a file in a manner avoiding race conditions. |
SafeLockedFileForRead (name) |
Context manager for reading a file that may be locked with an exclusive lock via SafeLockedFileForWrite. |
Classes¶
DoNotWrite |
|
FileForWriteOnceCompareSameFailure |
|
SafeLockedFileForWrite (name) |
File-like object that is used to create a file if needed, lock it with an exclusive lock, and contain file descriptors to readable and writable versions of the file. |
Class Inheritance Diagram¶
lsst.daf.butler.core.utils Module¶
Functions¶
iterable (a) |
Make input iterable. |
allSlots (self) |
Return combined __slots__ for all classes in objects mro. |
slotValuesAreEqual (self, other) |
Test for equality by the contents of all slots, including those of its parents. |
slotValuesToHash (self) |
Generate a hash from slot values. |
getFullTypeName (cls) |
Return full type name of the supplied entity. |
getInstanceOf (typeOrName) |
Given the type name or a type, instantiate an object of that type. |
transactional (func) |
Decorator that wraps a method and makes it transactional. |
getObjectSize (obj[, seen]) |
Recursively finds size of objects. |
stripIfNotNone (s) |
Strip leading and trailing whitespace if the given object is not None. |
Classes¶
Singleton |
Metaclass to convert a class to a Singleton. |
PrivateConstructorMeta |
A metaclass that disables regular construction syntax. |