Instrument¶

class lsst.pipe.base.Instrument(collection_prefix: str | None = None)¶

Bases: object

Base class for instrument-specific logic for the Gen3 Butler.

Parameters:

collection_prefixstr, optional: Prefix for collection names to use instead of the instrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.

Notes

Concrete instrument subclasses must have the same construction signature as the base class.

Attributes Summary

`configPaths`	Paths to config files to read for specific Tasks.
`policyName`	Instrument specific name to use when locating a policy or configuration file in the file system.
`raw_definition`	Dataset type definition to use for "raw" datasets.

Methods Summary

`applyConfigOverrides`(name, config)	Apply instrument-specific overrides for a task config.
`formatCollectionTimestamp`(timestamp)	Format a timestamp for use in a collection name.
`fromName`(name, registry[, collection_prefix])	Given an instrument name and a butler registry, retrieve a corresponding instantiated instrument object.
`from_data_id`(data_id[, collection_prefix])	Instantiate an `Instrument` object from a fully-expanded data ID.
`from_string`(name[, registry, collection_prefix])	Return an instance from the short name or class name.
`getName`()	Return the short (dimension) name for this instrument.
`getRawFormatter`(dataId)	Return the Formatter class that should be used to read a particular raw file.
`importAll`(registry)	Import all the instruments known to this registry.
`makeCalibrationCollectionName`(*labels)	Make a CALIBRATION collection name appropriate for associating calibration datasets with validity ranges.
`makeCollectionName`(*labels)	Get the instrument-specific collection string to use as derived from the supplied labels.
`makeCollectionTimestamp`()	Create a timestamp string for use in a collection name from the current time.
`makeCuratedCalibrationRunName`(calibDate, *labels)	Make a RUN collection name appropriate for inserting curated calibration datasets with the given `CALIBDATE` metadata value.
`makeDefaultRawIngestRunName`()	Make the default instrument-specific run collection string for raw data ingest.
`makeRefCatCollectionName`(*labels)	Return a global (not instrument-specific) name for a collection that holds reference catalogs.
`makeUmbrellaCollectionName`()	Return the name of the umbrella `CHAINED` collection for this instrument that combines all standard recommended input collections.
`makeUnboundedCalibrationRunName`(*labels)	Make a RUN collection name appropriate for inserting calibration datasets whose validity ranges are unbounded.
`make_default_dimension_packer`(data_id[, ...])	Return the default dimension packer for the given data ID.
`make_dimension_packer_config_field`([doc])	Make an `lsst.pex.config.Field` that can be used to configure how data IDs for this instrument are packed.
`register`(registry, *[, update])	Insert instrument, and other relevant records into `Registry`.

Attributes Documentation

configPaths: Sequence[str] = ()¶

Paths to config files to read for specific Tasks.

The paths in this list should contain files of the form task.py, for each of the Tasks that requires special configuration.

policyName: str | None = None¶: Instrument specific name to use when locating a policy or configuration file in the file system.

raw_definition: tuple[str, tuple[str, ...], str] | None = None¶: Dataset type definition to use for “raw” datasets. This is a tuple of the dataset type name, a tuple of dimension names, and the storage class name. If None the ingest system will use its default definition.

Methods Documentation

applyConfigOverrides(name: str, config: Config) → None¶

Apply instrument-specific overrides for a task config.

Parameters:

namestr: Name of the object being configured; typically the _DefaultName of a Task.
configlsst.pex.config.Config: Config instance to which overrides should be applied.

static formatCollectionTimestamp(timestamp: str | datetime) → str¶

Format a timestamp for use in a collection name.

Parameters:

timestampstr or datetime.datetime: Timestamp to format. May be a date or datetime string in extended ISO format (assumed UTC), with or without a timezone specifier, a datetime string in basic ISO format with a timezone specifier, a naive datetime.datetime instance (assumed UTC) or a timezone-aware datetime.datetime instance (converted to UTC). This is intended to cover all forms that string CALIBDATE metadata values have taken in the past, as well as the format this method itself writes out (to enable round-tripping).

Returns:

formattedstr: Standardized string form for the timestamp.

classmethod fromName(name: str, registry: Registry, collection_prefix: str | None = None) → Self¶

Given an instrument name and a butler registry, retrieve a corresponding instantiated instrument object.

Parameters:

namestr: Name of the instrument (must match the return value of getName).
registrylsst.daf.butler.Registry: Butler registry to query to find the information.
collection_prefixstr, optional: Prefix for collection names to use instead of the instrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.

Returns:

instrumentInstrument: An instance of the relevant Instrument.

Raises:

LookupError: Raised if the instrument is not known to the supplied registry.
ModuleNotFoundError: Raised if the class could not be imported. This could mean that the relevant obs package has not been setup.
TypeError: Raised if the class name retrieved is not a string or the imported symbol is not an Instrument subclass.

Notes

The instrument must be registered in the corresponding butler.

classmethod from_data_id(data_id: DataCoordinate, collection_prefix: str | None = None) → Self¶

Instantiate an Instrument object from a fully-expanded data ID.

Parameters:

data_idDataCoordinate: Expanded data ID that includes the instrument dimension.
collection_prefixstr, optional: Prefix for collection names to use instead of the instrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.

Returns:

instrumentInstrument: An instance of the relevant Instrument.

Raises:

TypeError: Raised if the class name retrieved is not a string or the imported symbol is not an Instrument subclass.

classmethod from_string(name: str, registry: Registry | None = None, collection_prefix: str | None = None) → Self¶

Return an instance from the short name or class name.

If the instrument name is not qualified (does not contain a ‘.’) and a butler registry is provided, this will attempt to load the instrument using Instrument.fromName(). Otherwise the instrument will be imported and instantiated.

Parameters:

namestr: The name or fully-qualified class name of an instrument.
registrylsst.daf.butler.Registry, optional: Butler registry to query to find information about the instrument, by default None.
collection_prefixstr, optional: Prefix for collection names to use instead of the instrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.

Returns:

instrumentInstrument: The instantiated instrument.

Raises:

RuntimeError: Raised if the instrument can not be imported, instantiated, or obtained from the registry.
TypeError: Raised if the instrument is not a subclass of Instrument.

See also

Instrument.fromName: Constructing Instrument from a name.

abstract classmethod getName() → str¶

Return the short (dimension) name for this instrument.

This is not (in general) the same as the class name - it’s what is used as the value of the “instrument” field in data IDs, and is usually an abbreviation of the full name.

abstract getRawFormatter(dataId: DataCoordinate | Mapping[str, Any]) → type[lsst.daf.butler._formatter.Formatter]¶

Return the Formatter class that should be used to read a particular raw file.

Parameters:

dataIdDataId: Dimension-based ID for the raw file or files being ingested.

Returns:

formatterlsst.daf.butler.Formatter class: Class to be used that reads the file into the correct Python object for the raw data.

static importAll(registry: Registry) → None¶

Import all the instruments known to this registry.

This will ensure that all metadata translators have been registered.

Parameters:

registrylsst.daf.butler.Registry: Butler registry to query to find the information.

Notes

It is allowed for a particular instrument class to fail on import. This might simply indicate that a particular obs package has not been setup.

makeCalibrationCollectionName(*labels: str) → str¶

Make a CALIBRATION collection name appropriate for associating calibration datasets with validity ranges.

Parameters:

*labelsstr: Strings to be appended to the base name, using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.

Returns:

namestr: Calibration collection name.

makeCollectionName(*labels: str) → str¶

Get the instrument-specific collection string to use as derived from the supplied labels.

Parameters:

*labelsstr: Strings to be combined with the instrument name to form a collection name.

Returns:

namestr: Collection name to use that includes the instrument’s recommended prefix.

static makeCollectionTimestamp() → str¶

Create a timestamp string for use in a collection name from the current time.

Returns:

formattedstr: Standardized string form of the current time.

makeCuratedCalibrationRunName(calibDate: str, *labels: str) → str¶

Make a RUN collection name appropriate for inserting curated calibration datasets with the given CALIBDATE metadata value.

Parameters:

calibDatestr: The CALIBDATE metadata value.
*labelsstr: Strings to be included in the collection name (before calibDate, but after all other terms), using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.

Returns:

namestr: Run collection name.

makeDefaultRawIngestRunName() → str¶

Make the default instrument-specific run collection string for raw data ingest.

Returns:

collstr: Run collection name to be used as the default for ingestion of raws.

static makeRefCatCollectionName(*labels: str) → str¶

Return a global (not instrument-specific) name for a collection that holds reference catalogs.

With no arguments, this returns the name of the collection that holds all reference catalogs (usually a CHAINED collection, at least in long-lived repos that may contain more than one reference catalog).

Parameters:

*labelsstr: Strings to be added to the global collection name, in order to define a collection name for one or more reference catalogs being ingested at the same time.

Returns:

namestr: Collection name.

Notes

This is a staticmethod, not a classmethod, because it should be the same for all instruments.

makeUmbrellaCollectionName() → str¶

Return the name of the umbrella CHAINED collection for this instrument that combines all standard recommended input collections.

This method should almost never be overridden by derived classes.

Returns:

namestr: Name for the umbrella collection.

makeUnboundedCalibrationRunName(*labels: str) → str¶

Make a RUN collection name appropriate for inserting calibration datasets whose validity ranges are unbounded.

Parameters:

*labelsstr: Extra strings to be included in the base name, using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.

Returns:

namestr: Run collection name.

final static make_default_dimension_packer(data_id: DataCoordinate, is_exposure: bool | None = None) → DimensionPacker¶

Return the default dimension packer for the given data ID.

Parameters:

data_idlsst.daf.butler.DataCoordinate: Data ID that identifies at least the instrument dimension. Must have dimension records attached.
is_exposurebool, optional: If False, construct a packer for visit+detector data IDs. If True, construct a packer for exposure+detector data IDs. If None, this is determined based on whether visit or exposure is present in data_id, with visit checked first and hence used if both are present.

Returns:

packerlsst.daf.butler.DimensionPacker: Object that packs {visit, detector} or {exposure, detector} data IDs into integers.

Notes

When using a dimension packer in task code, using make_dimension_packer_config_field to make the packing algorithm configurable is preferred over this method.

When obtaining a dimension packer to unpack IDs that were packed by task code, it is similarly preferable to load the configuration for that task and the existing packer configuration field there, to ensure any config overrides are respected. That is sometimes quite difficult, however, and since config overrides for dimension packers are expected to be exceedingly rare, using this simpler method will almost always work.

static make_dimension_packer_config_field(doc: str = 'How to pack visit+detector or exposure+detector data IDs into integers. The default (None) is to delegate to the Instrument class for which registered implementation to use (but still use the nested configuration for that implementation).') → RegistryField¶

Make an lsst.pex.config.Field that can be used to configure how data IDs for this instrument are packed.

Parameters:

docstr, optional: Documentation for the config field.

Returns:

fieldlsst.pex.config.RegistryField: A config field for which calling apply on the instance attribute constructs an lsst.daf.butler.DimensionPacker that defaults to the appropriate one for this instrument.

Notes

This method is expected to be used whenever code requires a single integer that represents the combination of a detector and either a visit or exposure, but in most cases the lsst.meas.base.IdGenerator class and its helper configs provide a simpler high-level interface that should be used instead of calling this method directly.

This system is designed to work best when the configuration for the ID packer is not overridden at all, allowing the appropriate instrument class to determine the behavior for each data ID encountered. When the configuration does need to be modified (most often when the scheme for packing an instrument’s data IDs is undergoing an upgrade), it is important to ensure the overrides are only applied to data IDs with the desired instrument value.

Unit tests of code that use a field produced by this method will often want to explicitly set the packer to “observation” and manually set its n_detectors and n_observations fields; this will make it unnecessary for tests to provide expanded data IDs.

abstract register(registry: Registry, *, update: bool = False) → None¶

Insert instrument, and other relevant records into Registry.

Parameters:

registrylsst.daf.butler.Registry: Registry client for the data repository to modify.
updatebool, optional: If True (False is default), update existing records if they differ from the new ones.

Raises:

lsst.daf.butler.registry.ConflictingDefinitionError: Raised if any existing record has the same key but a different definition as one being registered.

Notes

New records can always be added by calling this method multiple times, as long as no existing records have changed (if existing records have changed, update=True must be used). Old records can never be removed by this method.

Implementations should guarantee that registration is atomic (the registry should not be modified if any error occurs) and idempotent at the level of individual dimension entries; new detectors and filters should be added, but changes to any existing record should not be. This can generally be achieved via a block like

with registry.transaction():
    registry.syncDimensionData("instrument", ...)
    registry.syncDimensionData("detector", ...)
    self.registerFilters(registry)

Navigation

Instrument¶