lsst.daf.butler¶
This module provides an abstracted data access interface, known as the Butler. It can be used to read and write data without having to know the details of file formats or locations.
Contributing¶
lsst.daf.butler
is developed at https://github.com/lsst/daf_butler.
You can find Jira issues for this module under the daf_butler component.
Using the Butler¶
The Dimensions System¶
Python API reference¶
lsst.daf.butler Package¶
Functions¶
|
Construct a spherical ConvexPolygon from a WCS and a bounding box. |
Classes¶
Exception raised when a |
|
|
Main entry point for the data access system. |
|
Contains the configuration for a |
|
Convenience wrapper around URI parsers. |
There is a problem with the Butler configuration. |
|
|
Class for providing assembler and disassembler support for composites. |
|
|
|
Determine whether a specific datasetType or StorageClass should be disassembled. |
|
Implements a datatype that is used by |
|
Config representing a subset of a more general configuration. |
Exception raised when trying to insert a database record when a conflicting record already exists. |
|
|
Determine whether a |
|
Configuration information for |
Exception thrown when a constraints list has mutually exclusive definitions. |
|
An immutable data ID dictionary that guarantees that its key-value pairs identify all required dimensions in a |
|
|
Component of a dataset and associated information. |
|
Reference to a Dataset in a |
|
A named category of Datasets that defines how they are organized, related, and stored. |
A |
|
|
Datastore interface. |
|
|
There is a problem with the Datastore configuration. |
|
|
Retrieves authentication information for database connections. |
A problem has occurred retrieving database authentication information. |
|
Credentials file has incorrect permissions. |
|
|
This is a class to support deferred loading of a dataset from a butler. |
|
A named data-organization concept that can be used as a key in a data ID. |
|
Configuration that defines a |
|
A named data-organization concept that defines a label and/or metadata in the dimensions system. |
An immutable, dependency-complete collection of dimensions. |
|
|
An abstract base class for bidirectional mappings between a |
|
Base class for the Python representation of database records for a |
A special |
|
|
Any step in a production. |
A data ID that has been expanded to include all relevant metadata. |
|
|
A struct-like class used to define a column in a logical Registry table. |
|
A struct that represents a dataset exported to a file. |
|
Describes a particular file. |
|
Format a path template into a fully expanded path. |
Exception thrown when a file template is not consistent with the associated |
|
|
Collection of |
|
Configuration information for |
|
A struct-like class used to define a foreign key constraint in a logical Registry table. |
|
Interface for reading and writing Datasets with a particular |
Factory for |
|
|
Identifies a location within the |
|
Factory for |
|
Representation of key that can be used to lookup information based on dataset type name, storage class name, dimensions. |
|
Register the mapping of some key to a python type and retrieve instances. |
Exception raised when trying to remove or modify a database record that is still being used in some other table. |
|
|
A discrete unit of work that may depend on one or more datasets and produces one or more datasets. |
|
Registry interface. |
|
Public interface for exporting a subset of a data repository. |
An abstract interface for data repository export implementations. |
|
An abstract interface for data repository import implementations. |
|
|
The section of butler configuration that associates repo import/export backends with file formats. |
|
Represent a processing run. |
|
The SQL schema for a Butler Registry. |
|
|
|
A special |
|
Class describing how a label maps to a particular Python type. |
|
|
|
Factory for |
Internal information associated with a stored dataset in a |
|
|
Datastore-private metadata associated with a file stored in a Datastore. |
|
A struct-like class used to define a logical Registry table (which may also be implemented in the database as a view). |
Some sort of validation error has occurred. |
|
|
A repository export implementation that saves to a YAML file. |
|
A repository import implementation that reads from a YAML file. |
Class Inheritance Diagram¶
Example datastores¶
lsst.daf.butler.datastores.posixDatastore Module¶
Classes¶
|
Basic POSIX filesystem backed Datastore. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.inMemoryDatastore Module¶
Classes¶
|
Internal InMemoryDatastore Metadata associated with a stored DatasetRef. |
|
Basic Datastore for writing to an in memory cache. |
Class Inheritance Diagram¶
lsst.daf.butler.datastores.chainedDatastore Module¶
Classes¶
|
Chained Datastores to allow read and writes from multiple datastores. |
Class Inheritance Diagram¶
Example registries¶
lsst.daf.butler.registries.sqlRegistry Module¶
Classes¶
|
|
|
Registry backed by a SQL database. |
Class Inheritance Diagram¶
Example formatters¶
lsst.daf.butler.formatters.fileFormatter Module¶
Classes¶
|
Interface for reading and writing files on a POSIX file system. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.jsonFormatter Module¶
Classes¶
|
Interface for reading and writing Python objects to and from JSON files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.yamlFormatter Module¶
Classes¶
|
Interface for reading and writing Python objects to and from YAML files. |
Class Inheritance Diagram¶
lsst.daf.butler.formatters.pickleFormatter Module¶
Classes¶
|
Interface for reading and writing Python objects to and from pickle files. |
Class Inheritance Diagram¶
Support API¶
lsst.daf.butler.core.safeFileIo Module¶
Functions¶
|
Make a directory in a manner avoiding race conditions |
|
Set a file mode according to the user’s umask |
Context manager to get a file that can be written only once and all other writes will succeed only if they match the initial write. |
|
|
Context manager to create a file in a manner avoiding race conditions |
|
Context manager for creating a file in a manner avoiding race conditions. |
|
Context manager for reading a file that may be locked with an exclusive lock via SafeLockedFileForWrite. |
Classes¶
|
File-like object that is used to create a file if needed, lock it with an exclusive lock, and contain file descriptors to readable and writable versions of the file. |
Class Inheritance Diagram¶
lsst.daf.butler.core.utils Module¶
Functions¶
|
Return combined |
|
Given the type name or a type, return the python type. |
|
Return full type name of the supplied entity. |
|
Given the type name or a type, instantiate an object of that type. |
|
Recursively finds size of objects. |
|
A class decorator that simulates a simple form of immutability for the decorated class. |
|
Make input iterable. |
|
Test for equality by the contents of all slots, including those of its parents. |
|
Generate a hash from slot values. |
Strip leading and trailing whitespace if the given object is not None. |
|
|
Decorator that wraps a method and makes it transactional. |
Classes¶
An immutable mapping that combines a tuple of values with a (possibly shared) mapping from key to tuple index. |
|
|
A dictionary wrapper that require keys to have a |
|
A custom mutable set class that requires elements to have a |
A metaclass that disables regular construction syntax. |
|
Metaclass to convert a class to a Singleton. |
Class Inheritance Diagram¶
lsst.daf.butler.core.repoRelocation Module¶
Functions¶
|
Update a configuration root with the butler root location. |
Variables¶
The special string to be used in configuration files to indicate that the butler root location should be used. |