lsst.daf.butler

This module provides an abstracted data access interface.

Python API reference

lsst.daf.butler Package

Data Access Butler

Functions

makeBoxWcsRegion(box, wcs, margin) Construct a spherical ConvexPolygon from a WCS and a bounding box.

Classes

Butler([config, collection, run]) Main entry point for the data access system.
ButlerConfig([other]) Contains the configuration for a Butler
CompositeAssembler(storageClass) Class for providing assembler and disassembler support for composites.
CompositesConfig([other, validate, …])
CompositesMap(config) Determine whether a specific datasetType or StorageClass should be disassembled.
Config([other]) Implements a datatype that is used by Butler for configuration parameters.
ConfigSubset([other, validate, …]) Config representing a subset of a more general configuration.
DataUnit(name, requiredDependencies, …[, …]) A discrete abstract unit of data that can be associated with metadata and used to label datasets.
DataUnitRegistry() Instances of this class keep track of DataUnit relations.
DatabaseDict(config, types, key, value) An abstract base class for dict-like objects with a specific key type and namedtuple values, backed by a database.
DatasetComponent(name, storageClass, component) Component of a dataset and associated information.
DatasetOriginInfo Interface for classes providing collection information for pre-flight.
DatasetOriginInfoDef(defaultInputs, …[, …]) Default implementation of the DatasetOriginInfo.
DatasetRef(datasetType, dataId[, id, run]) Reference to a Dataset in a Registry.
DatasetType(name, dataUnits, storageClass) A named category of Datasets that defines how they are organized, related, and stored.
DatasetTypeNotSupportedError A DatasetType is not handled by this routine.
Datastore(config, registry) Datastore interface.
DatastoreConfig([other, validate, …])
FileDescriptor(location, storageClass[, …]) Describes a particular file.
FileTemplate(template) Format a path template into a fully expanded path.
FileTemplates(config[, default]) Collection of FileTemplate templates.
FileTemplatesConfig([other]) Configuration information for FileTemplates
Formatter Interface for reading and writing Datasets with a particular StorageClass.
FormatterFactory() Factory for Formatter instances.
Location(datastoreRoot, uri) Identifies a location in the Datastore.
LocationFactory(datastoreRoot) Factory for Location instances.
MappingFactory(refType) Register the mapping of some key to a python type and retrieve instances.
PreFlightUnitsRow(dataId, datasetRefs) Simple data class holding combination of DataUnit values for one row returned by pre-flight solver.
Quantum(task, run, *args, **kwargs) A discrete unit of work that may depend on one or more datasets and produces one or more datasets.
Registry(registryConfig[, schemaConfig, create]) Registry interface.
RegistryConfig([other, validate, …])
Run(collection[, environment, pipeline]) Represent a processing run.
Schema([config]) The SQL schema for a Butler Registry.
SchemaBuilder() Builds a Schema step-by-step.
SchemaConfig([other, validate, …])
StorageClass([name, pytype, components, …]) Class describing how a label maps to a particular Python type.
StorageClassConfig([other, validate, …])
StorageClassFactory([config]) Factory for StorageClass instances.
StoredFileInfo(formatter, path, storageClass) Information associated with a stored file in a Datastore.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.butler.Butler, lsst.daf.butler.core.butlerConfig.ButlerConfig, lsst.daf.butler.core.assembler.CompositeAssembler, lsst.daf.butler.core.composites.CompositesConfig, lsst.daf.butler.core.composites.CompositesMap, lsst.daf.butler.core.config.Config, lsst.daf.butler.core.config.ConfigSubset, lsst.daf.butler.core.dataUnit.DataUnit, lsst.daf.butler.core.dataUnit.DataUnitRegistry, lsst.daf.butler.core.databaseDict.DatabaseDict, lsst.daf.butler.core.assembler.DatasetComponent, lsst.daf.butler.core.preFlight.DatasetOriginInfo, lsst.daf.butler.core.preFlight.DatasetOriginInfoDef, lsst.daf.butler.core.datasets.DatasetRef, lsst.daf.butler.core.datasets.DatasetType, lsst.daf.butler.core.exceptions.DatasetTypeNotSupportedError, lsst.daf.butler.core.datastore.Datastore, lsst.daf.butler.core.datastore.DatastoreConfig, lsst.daf.butler.core.fileDescriptor.FileDescriptor, lsst.daf.butler.core.fileTemplates.FileTemplate, lsst.daf.butler.core.fileTemplates.FileTemplates, lsst.daf.butler.core.fileTemplates.FileTemplatesConfig, lsst.daf.butler.core.formatter.Formatter, lsst.daf.butler.core.formatter.FormatterFactory, lsst.daf.butler.core.location.Location, lsst.daf.butler.core.location.LocationFactory, lsst.daf.butler.core.mappingFactory.MappingFactory, lsst.daf.butler.core.preFlight.PreFlightUnitsRow, lsst.daf.butler.core.quantum.Quantum, lsst.daf.butler.core.registry.Registry, lsst.daf.butler.core.registry.RegistryConfig, lsst.daf.butler.core.run.Run, lsst.daf.butler.core.schema.Schema, lsst.daf.butler.core.schema.SchemaBuilder, lsst.daf.butler.core.schema.SchemaConfig, lsst.daf.butler.core.storageClass.StorageClass, lsst.daf.butler.core.storageClass.StorageClassConfig, lsst.daf.butler.core.storageClass.StorageClassFactory, lsst.daf.butler.core.storedFileInfo.StoredFileInfo

Example Datastores

lsst.daf.butler.datastores.posixDatastore Module

POSIX datastore.

Classes

PosixDatastore(config, registry) Basic POSIX filesystem backed Datastore.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.datastores.posixDatastore.PosixDatastore

lsst.daf.butler.datastores.inMemoryDatastore Module

In-memory datastore.

Classes

StoredItemInfo(timestamp, storageClass[, …]) Internal Metadata associated with a DatasetRef.
InMemoryDatastore(config[, registry]) Basic Datastore for writing to an in memory cache.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.datastores.inMemoryDatastore.StoredItemInfo, lsst.daf.butler.datastores.inMemoryDatastore.InMemoryDatastore

lsst.daf.butler.datastores.chainedDatastore Module

Chained datastore.

Classes

ChainedDatastore(config[, registry]) Chained Datastores to allow read and writes from multiple datastores.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.datastores.chainedDatastore.ChainedDatastore

Example Registries

lsst.daf.butler.registries.sqlRegistry Module

Classes

SqlRegistryConfig([other, validate, …])
SqlRegistry(registryConfig, schemaConfig[, …]) Registry backed by a SQL database.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.registries.sqlRegistry.SqlRegistryConfig, lsst.daf.butler.registries.sqlRegistry.SqlRegistry

Example Formatters

lsst.daf.butler.formatters.fileFormatter Module

Support for reading and writing files to a POSIX file system.

Classes

FileFormatter Interface for reading and writing files on a POSIX file system.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.formatters.fileFormatter.FileFormatter

lsst.daf.butler.formatters.jsonFormatter Module

Classes

JsonFormatter Interface for reading and writing Python objects to and from JSON files.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.formatters.jsonFormatter.JsonFormatter

lsst.daf.butler.formatters.yamlFormatter Module

Classes

YamlFormatter Interface for reading and writing Python objects to and from YAML files.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.formatters.yamlFormatter.YamlFormatter

lsst.daf.butler.formatters.pickleFormatter Module

Formatter associated with Python pickled objects.

Classes

PickleFormatter Interface for reading and writing Python objects to and from pickle files.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.formatters.pickleFormatter.PickleFormatter

Support API

lsst.daf.butler.core.safeFileIo Module

Utilities for safe file IO

Functions

safeMakeDir(directory) Make a directory in a manner avoiding race conditions
setFileMode(filename) Set a file mode according to the user’s umask
FileForWriteOnceCompareSame(name) Context manager to get a file that can be written only once and all other writes will succeed only if they match the initial write.
SafeFile(name) Context manager to create a file in a manner avoiding race conditions
SafeFilename(name) Context manager for creating a file in a manner avoiding race conditions.
SafeLockedFileForRead(name) Context manager for reading a file that may be locked with an exclusive lock via SafeLockedFileForWrite.

Classes

DoNotWrite
FileForWriteOnceCompareSameFailure
SafeLockedFileForWrite(name) File-like object that is used to create a file if needed, lock it with an exclusive lock, and contain file descriptors to readable and writable versions of the file.

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.core.safeFileIo.DoNotWrite, lsst.daf.butler.core.safeFileIo.FileForWriteOnceCompareSameFailure, lsst.daf.butler.core.safeFileIo.SafeLockedFileForWrite

lsst.daf.butler.core.utils Module

Functions

iterable(a) Make input iterable.
allSlots(self) Return combined __slots__ for all classes in objects mro.
slotValuesAreEqual(self, other) Test for equality by the contents of all slots, including those of its parents.
slotValuesToHash(self) Generate a hash from slot values.
getFullTypeName(cls) Return full type name of the supplied entity.
doImport(pythonType) Import a python object given an importable string and return the type object
getInstanceOf(typeOrName) Given the type name or a type, instantiate an object of that type.
transactional(func) Decorator that wraps a method and makes it transactional.
getObjectSize(obj[, seen]) Recursively finds size of objects.

Classes

Singleton Metaclass to convert a class to a Singleton.
TopologicalSet(elements) A collection that behaves like a builtin set, but where elements can be interconnected (like a graph).
TopologicalSetNode(element, sourceElements)

Class Inheritance Diagram

Inheritance diagram of lsst.daf.butler.core.utils.Singleton, lsst.daf.butler.core.utils.TopologicalSet, lsst.daf.butler.core.utils.TopologicalSetNode