lsst.obs.base

The lsst.obs.base module provides the framework and common API for telescope/camera specific “obs” packages. New cameras will derive from the classes defined here. lsst.obs.base.tests provides the tests that all “obs” packages should pass.

Contributing

lsst.obs.base is developed at https://github.com/lsst/obs_base. You can find Jira issues for this module under the obs_base component.

Command Line Interface

daf_butler implements a command line interface command called butler. The following subcommands are implemented by this package and available to the butler command when this package is setup.

butler

butler [OPTIONS] COMMAND [ARGS]...

convert

Convert one or more Butler gen 2 repositories into a gen 3 repository.

REPO is the URI or path to the gen3 repository. Will be created if it does not already exist

This is a highly simplified interface that should only be used to convert suites of gen 2 repositories that contain at most one calibration repo and has no chained reruns. Custom scripts that call ConvertRepoTask should be used on more complex suites of repositories.

butler convert [OPTIONS] REPO

Options

--gen2root <gen2root>

Required Root path of the gen 2 repo to be converted.

--skymap-name <skymap_name>

Name of the new gen3 skymap (e.g. ‘discrete/ci_hsc’).

--skymap-config <skymap_config>

Path to skymap config file defining the new gen3 skymap.

--calibs <calibs>

Path to the gen 2 calibration repo. It can be absolute or relative to gen2root.

--reruns <TEXT ...>

List of rerun paths to convert. Output collection names will be guessed, which can fail if the Gen2 repository paths do not follow a recognized convention. In this case, the command-line interface cannot be used.

-t, --transfer <transfer>

Mode to use to transfer files into the new repository.

Options:auto | link | symlink | hardlink | copy | move | relsymlink | direct
-j, --processes <processes>

Number of processes to use.

-C, --config-file <config_file>

Path to a ConvertRepoConfig override to be included after the Instrument config overrides are applied.

-@, --options-file <options_file>

URI to YAML file containing overrides of command line options. The YAML should be organized as a hierarchy with subcommand names at the top level options for that subcommand below.

Arguments

REPO

Required argument

See ‘butler –help’ for more options.

define-visits

Define visits from exposures in the butler registry.

REPO is the URI or path to the gen3 repository. Will be created if it does not already exist

The name or fully-qualified class name of an instrument.

The calibration collection containing the camera geometry can not be specified.

butler define-visits [OPTIONS] REPO INSTRUMENT

Options

-C, --config-file <config_file>

Path to a pex_config override to be included after the Instrument config overrides are applied.

--collections <TEXT ...>

The collections to be searched (in order) when reading datasets.

--where <where>

A string expression similar to a SQL WHERE clause.

-@, --options-file <options_file>

URI to YAML file containing overrides of command line options. The YAML should be organized as a hierarchy with subcommand names at the top level options for that subcommand below.

Arguments

REPO

Required argument

INSTRUMENT

Required argument

See ‘butler –help’ for more options.

ingest-raws

Ingest raw frames into from a directory into the butler registry

REPO is the URI or path to the gen3 repository. Will be created if it does not already exist

LOCATIONS specifies files to ingest and/or locations to search for files.

butler ingest-raws [OPTIONS] REPO LOCATIONS ...

Options

--regex <regex>

Regex string used to find files in directories listed in LOCATIONS. Searches for fits files by default.

-c, --config <TEXT=TEXT>

Config override, as a key-value pair.

-C, --config-file <config_file>

Path to a pex config override to be included after the Instrument config overrides are applied.

--output-run <output_run>

The name of the run datasets should be output to.

-t, --transfer <transfer>

The external data transfer mode.

Options:auto | link | symlink | hardlink | copy | move | relsymlink | direct
-j, --processes <processes>

Number of processes to use.

--ingest-task <ingest_task>

The fully qualified class name of the ingest task to use.

--track-file-attrs, --no-track-file-attrs

Indicate to the datastore whether file attributes such as file size or checksum should be tracked or not. Whether this parameter is honored depends on the specific datastore implentation.

--fail-fast

Stop ingest as soon as any problem is encountered with any file. Otherwise problem files will be skipped and logged and a report issued at completion.

-@, --options-file <options_file>

URI to YAML file containing overrides of command line options. The YAML should be organized as a hierarchy with subcommand names at the top level options for that subcommand below.

Arguments

REPO

Required argument

LOCATIONS

Required argument(s)

See ‘butler –help’ for more options.

write-curated-calibrations

Add an instrument’s curated calibrations to the data repository.

REPO is the URI or path to the gen3 repository. Will be created if it does not already exist

The name or fully-qualified class name of an instrument.

butler write-curated-calibrations [OPTIONS] REPO INSTRUMENT

Options

--collection <collection>

Name of the calibration collection that associates datasets with validity ranges.

--label <labels>

Extra strings to include (with automatic delimiters) in all RUN collection names, as well as the calibration collection name if it is not provided via –collection.

-@, --options-file <options_file>

URI to YAML file containing overrides of command line options. The YAML should be organized as a hierarchy with subcommand names at the top level options for that subcommand below.

Arguments

REPO

Required argument

INSTRUMENT

Required argument

See ‘butler –help’ for more options.

Python API reference

lsst.obs.base Package

Functions

bboxFromIraf(irafBBoxStr) Return a Box2I corresponding to an IRAF-style BBOX
createInitialSkyWcs(visitInfo, detector[, flipX]) Create a SkyWcs from the visit information and detector geometry.
createInitialSkyWcsFromBoresight(boresight, …) Create a SkyWcs from the telescope boresight and detector geometry.
exposureFromImage(image[, dataId, mapper, …]) Generate an Exposure from an image-like object
loadCamera(butler, dataId, Mapping[str, …) Attempt to load versioned camera geometry from a butler, but fall back to obtaining a nominal camera from the Instrument class if that fails.
makeExposureRecordFromObsInfo(obsInfo, …) Construct an exposure DimensionRecord from astro_metadata_translator.ObservationInfo.
makeTransferChoiceField(doc, default) Create a Config field with options for transferring data between repos.

Classes

CalibrationMapping(datasetType, policy, …) CalibrationMapping is a Mapping subclass for calibration-type products.
CameraMapper(policy, repositoryDir[, root, …]) CameraMapper is a base class for mappers that handle images from a
DatasetMapping(datasetType, policy, …) DatasetMapping is a Mapping subclass for non-Exposure datasets that can be retrieved by the standard daf_persistence mechanism.
DefineVisitsConfig
DefineVisitsTask(config, *, butler, **kwargs) Driver Task for defining visits (and their spatial regions) in Gen3 Butler repositories.
ExposureIdInfo([expId, expBits, maxBits]) Struct representing an exposure ID and the number of bits it uses.
ExposureMapping(datasetType, policy, …) ExposureMapping is a Mapping subclass for normal exposures.
FilterDefinition(physical_filter, lambdaEff, …) The definition of an instrument’s filter bandpass.
FilterDefinitionCollection(*filters) An order-preserving collection of multiple FilterDefinition.
FitsRawFormatterBase(*args, **kwargs) Abstract base class for reading and writing raw data to and from FITS files.
GroupExposuresConfig
GroupExposuresTask(config, **kwargs) Abstract base class for the subtask of DefineVisitsTask that is responsible for grouping exposures into visits.
ImageMapping(datasetType, policy, registry, …) ImageMapping is a Mapping subclass for non-camera images.
InitialSkyWcsError For handling failures when creating a SkyWcs from a camera geometry and boresight.
Instrument(collection_prefix) Rubin-specified base for instrument-specific logic for the Gen3 Butler.
MakeRawVisitInfo([log, doStripHeader]) Base class functor to make a VisitInfo from the FITS header of a raw image.
MakeRawVisitInfoViaObsInfo([log, doStripHeader]) Base class functor to make a VisitInfo from the FITS header of a raw image using ObservationInfo translators.
Mapping(datasetType, policy, registry, …) Mapping is a base class for all mappings.
RawIngestConfig Configuration class for RawIngestTask.
RawIngestTask(config, *, butler, on_success, …) Driver Task for ingesting raw data into Gen3 Butler repositories.
VisitDefinitionData(instrument, id, name, …) Struct representing a group of exposures that will be used to define a visit.

Class Inheritance Diagram

Inheritance diagram of lsst.obs.base.mapping.CalibrationMapping, lsst.obs.base.cameraMapper.CameraMapper, lsst.obs.base.mapping.DatasetMapping, lsst.obs.base.defineVisits.DefineVisitsConfig, lsst.obs.base.defineVisits.DefineVisitsTask, lsst.obs.base.exposureIdInfo.ExposureIdInfo, lsst.obs.base.mapping.ExposureMapping, lsst.obs.base.filters.FilterDefinition, lsst.obs.base.filters.FilterDefinitionCollection, lsst.obs.base._fitsRawFormatterBase.FitsRawFormatterBase, lsst.obs.base.defineVisits.GroupExposuresConfig, lsst.obs.base.defineVisits.GroupExposuresTask, lsst.obs.base.mapping.ImageMapping, lsst.obs.base.utils.InitialSkyWcsError, lsst.obs.base._instrument.Instrument, lsst.obs.base.makeRawVisitInfo.MakeRawVisitInfo, lsst.obs.base.makeRawVisitInfoViaObsInfo.MakeRawVisitInfoViaObsInfo, lsst.obs.base.mapping.Mapping, lsst.obs.base.ingest.RawIngestConfig, lsst.obs.base.ingest.RawIngestTask, lsst.obs.base.defineVisits.VisitDefinitionData

lsst.obs.base.tests Module

Functions

make_ramp_array(bbox, pedestal) Make a 2-d ramp array.
make_ramp_exposure_trimmed(detector[, dtype]) Create a trimmed, assembled exposure with different ramps for each amplifier region.
make_ramp_exposure_untrimmed(detector[, dtype]) Create an untrimmed, assembled exposure with different ramps for each sub-amplifier region.

Classes

ObsTests Aggregator class for all of the obs_* test classes.

Class Inheritance Diagram

Inheritance diagram of lsst.obs.base.tests.ObsTests

lsst.obs.base.formatters.fitsExposure Module

Functions

standardizeAmplifierParameters(parameters, …) Preprocess the Exposure storage class’s “amp” and “detector” parameters

Classes

FitsExposureFormatter(fileDescriptor, …) Concrete formatter for reading/writing Exposure from/to FITS.
FitsImageFormatter(fileDescriptor, dataId, …) Concrete formatter for reading/writing Image from/to FITS.
FitsMaskFormatter(fileDescriptor, dataId, …) Concrete formatter for reading/writing Mask from/to FITS.
FitsMaskedImageFormatter(fileDescriptor, …) Concrete formatter for reading/writing MaskedImage from/to FITS.

Class Inheritance Diagram

Inheritance diagram of lsst.obs.base.formatters.fitsExposure.FitsExposureFormatter, lsst.obs.base.formatters.fitsExposure.FitsImageFormatter, lsst.obs.base.formatters.fitsExposure.FitsMaskFormatter, lsst.obs.base.formatters.fitsExposure.FitsMaskedImageFormatter

lsst.obs.base.formatters.fitsGeneric Module

Classes

FitsGenericFormatter(fileDescriptor, dataId, …) Interface for reading and writing objects that support the standard afw I/O readFits/writeFits methods.

Class Inheritance Diagram

Inheritance diagram of lsst.obs.base.formatters.fitsGeneric.FitsGenericFormatter