Instrument¶

class lsst.obs.base.Instrument(collection_prefix: Optional[str] = None)¶

Bases: object

Base class for instrument-specific logic for the Gen3 Butler.

Parameters:	collection_prefix : `str`, optional Prefix for collection names to use instead of the intrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.

Notes

Concrete instrument subclasses must have the same construction signature as the base class.

Attributes Summary

`additionalCuratedDatasetTypes`	Curated dataset types specific to this particular instrument that do not follow the standard organization found in obs data packages.
`configPaths`	Paths to config files to read for specific Tasks.
`filterDefinitions`	`FilterDefinitionCollection`, defining the filters for this instrument.
`obsDataPackage`	Name of the package containing the text curated calibration files.
`policyName`	Instrument specific name to use when locating a policy or configuration file in the file system.
`standardCuratedDatasetTypes`	The dataset types expected to be obtained from the obsDataPackage.

Methods Summary

`applyConfigOverrides`(name, config)	Apply instrument-specific overrides for a task config.
`formatCollectionTimestamp`(timestamp, …)	Format a timestamp for use in a collection name.
`fromName`(name, registry, collection_prefix)	Given an instrument name and a butler, retrieve a corresponding instantiated instrument object.
`getCamera`()	Retrieve the cameraGeom representation of this instrument.
`getCuratedCalibrationNames`()	Return the names of all the curated calibration dataset types.
`getName`()	Return the short (dimension) name for this instrument.
`getObsDataPackageDir`()	The root of the obs data package that provides specializations for this instrument.
`getRawFormatter`(dataId)	Return the Formatter class that should be used to read a particular raw file.
`importAll`(registry)	Import all the instruments known to this registry.
`makeCalibrationCollectionName`(*labels)	Make a CALIBRATION collection name appropriate for associating calibration datasets with validity ranges.
`makeCollectionName`(*labels)	Get the instrument-specific collection string to use as derived from the supplied labels.
`makeCollectionTimestamp`()	Create a timestamp string for use in a collection name from the current time.
`makeCuratedCalibrationRunName`(calibDate, *labels)	Make a RUN collection name appropriate for inserting curated calibration datasets with the given `CALIBDATE` metadata value.
`makeDataIdTranslatorFactory`()	Return a factory for creating Gen2->Gen3 data ID translators, specialized for this instrument.
`makeDefaultRawIngestRunName`()	Make the default instrument-specific run collection string for raw data ingest.
`makeRefCatCollectionName`(*labels)	Return a global (not instrument-specific) name for a collection that holds reference catalogs.
`makeUmbrellaCollectionName`()	Return the name of the umbrella `CHAINED` collection for this instrument that combines all standard recommended input collections.
`makeUnboundedCalibrationRunName`(*labels)	Make a RUN collection name appropriate for inserting calibration datasets whose validity ranges are unbounded.
`register`(registry, *[, update])	Insert instrument, physical_filter, and detector entries into a `Registry`.
`writeAdditionalCuratedCalibrations`(butler, …)	Write additional curated calibrations that might be instrument specific and are not part of the standard set.
`writeCameraGeom`(butler, collection, labels)	Write the default camera geometry to the butler repository and associate it with the appropriate validity range in a calibration collection.
`writeCuratedCalibrations`(butler, collection, …)	Write human-curated calibration Datasets to the given Butler with the appropriate validity ranges.
`writeStandardTextCuratedCalibrations`(butler, …)	Write the set of standardized curated text calibrations to the repository.

Attributes Documentation

additionalCuratedDatasetTypes = frozenset()¶

Curated dataset types specific to this particular instrument that do not follow the standard organization found in obs data packages.

These are the instrument-specific dataset types written by writeAdditionalCuratedCalibrations in addition to the calibrations found in obs data packages that follow the standard scheme. (set of str)

configPaths = ()¶

Paths to config files to read for specific Tasks.

The paths in this list should contain files of the form task.py, for each of the Tasks that requires special configuration.

filterDefinitions¶: FilterDefinitionCollection, defining the filters for this instrument.

obsDataPackage = None¶: Name of the package containing the text curated calibration files. Usually a obs _data package. If None no curated calibration files will be read. (str)

policyName = None¶: Instrument specific name to use when locating a policy or configuration file in the file system.

standardCuratedDatasetTypes = frozenset({'defects', 'linearizer', 'qe_curve', 'crosstalk', 'bfk'})¶

The dataset types expected to be obtained from the obsDataPackage.

These dataset types are all required to have standard definitions and must be known to the base class. Clearing this list will prevent any of these calibrations from being stored. If a dataset type is not known to a specific instrument it can still be included in this list since the data package is the source of truth. (set of str)

Methods Documentation

applyConfigOverrides(name, config)¶

Apply instrument-specific overrides for a task config.

Parameters:	name : `str` Name of the object being configured; typically the _DefaultName of a Task. config : `lsst.pex.config.Config` Config instance to which overrides should be applied.

static formatCollectionTimestamp(timestamp: Union[str, datetime.datetime]) → str¶

Format a timestamp for use in a collection name.

Parameters:

timestamp : str or datetime.datetime: Timestamp to format. May be a date or datetime string in extended ISO format (assumed UTC), with or without a timezone specifier, a datetime string in basic ISO format with a timezone specifier, a naive datetime.datetime instance (assumed UTC) or a timezone-aware datetime.datetime instance (converted to UTC). This is intended to cover all forms that string CALIBDATE metadata values have taken in the past, as well as the format this method itself writes out (to enable round-tripping).

Returns:

formatted : str: Standardized string form for the timestamp.

static fromName(name: str, registry: Registry, collection_prefix: Optional[str] = None) → Instrument¶

Given an instrument name and a butler, retrieve a corresponding instantiated instrument object.

Parameters:	name : `str` Name of the instrument (must match the return value of `getName`). registry : `lsst.daf.butler.Registry` Butler registry to query to find the information. collection_prefix : `str`, optional Prefix for collection names to use instead of the intrument’s own name. This is primarily for use in simulated-data repositories, where the instrument name may not be necessary and/or sufficient to distinguish between collections.
Returns:	instrument : `Instrument` An instance of the relevant `Instrument`.
Raises:	LookupError Raised if the instrument is not known to the supplied registry. ModuleNotFoundError Raised if the class could not be imported. This could mean that the relevant obs package has not been setup. TypeError Raised if the class name retrieved is not a string.

Notes

The instrument must be registered in the corresponding butler.

getCamera()¶

Retrieve the cameraGeom representation of this instrument.

This is a temporary API that should go away once obs packages have a standardized approach to writing versioned cameras to a Gen3 repo.

classmethod getCuratedCalibrationNames() → Set[str]¶

Return the names of all the curated calibration dataset types.

Returns:	names : `set` of `str` The dataset type names of all curated calibrations. This will include the standard curated calibrations even if the particular instrument does not support them.

Notes

The returned list does not indicate whether a particular dataset is present in the Butler repository, simply that these are the dataset types that are handled by writeCuratedCalibrations.

classmethod getName()¶

Return the short (dimension) name for this instrument.

This is not (in general) the same as the class name - it’s what is used as the value of the “instrument” field in data IDs, and is usually an abbreviation of the full name.

classmethod getObsDataPackageDir()¶

The root of the obs data package that provides specializations for this instrument.

Returns:	dir : `str` The root of the relevat obs data package.

getRawFormatter(dataId)¶

Return the Formatter class that should be used to read a particular raw file.

Parameters:	dataId : `DataCoordinate` Dimension-based ID for the raw file or files being ingested.
Returns:	formatter : `Formatter` class Class to be used that reads the file into an `lsst.afw.image.Exposure` instance.

static importAll(registry: Registry) → None¶

Import all the instruments known to this registry.

This will ensure that all metadata translators have been registered.

Parameters:	registry : `lsst.daf.butler.Registry` Butler registry to query to find the information.

Notes

It is allowed for a particular instrument class to fail on import. This might simply indicate that a particular obs package has not been setup.

makeCalibrationCollectionName(*labels) → str¶

Make a CALIBRATION collection name appropriate for associating calibration datasets with validity ranges.

Parameters:	*labels : `str` Strings to be appended to the base name, using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.
Returns:	name : `str` Calibration collection name.

makeCollectionName(*labels) → str¶

Get the instrument-specific collection string to use as derived from the supplied labels.

Parameters:	*labels : `str` Strings to be combined with the instrument name to form a collection name.
Returns:	name : `str` Collection name to use that includes the instrument’s recommended prefix.

static makeCollectionTimestamp() → str¶

Create a timestamp string for use in a collection name from the current time.

Returns:	formatted : `str` Standardized string form of the current time.

makeCuratedCalibrationRunName(calibDate: str, *labels) → str¶

Make a RUN collection name appropriate for inserting curated calibration datasets with the given CALIBDATE metadata value.

Parameters:	calibDate : `str` The `CALIBDATE` metadata value. *labels : `str` Strings to be included in the collection name (before `calibDate`, but after all other terms), using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.
Returns:	name : `str` Run collection name.

makeDataIdTranslatorFactory() → TranslatorFactory¶

Return a factory for creating Gen2->Gen3 data ID translators, specialized for this instrument.

Derived class implementations should generally call TranslatorFactory.addGenericInstrumentRules with appropriate arguments, but are not required to (and may not be able to if their Gen2 raw data IDs are sufficiently different from the HSC/DECam/CFHT norm).

Returns:	factory : `TranslatorFactory`. Factory for `Translator` objects.

makeDefaultRawIngestRunName() → str¶

Make the default instrument-specific run collection string for raw data ingest.

Returns:	coll : `str` Run collection name to be used as the default for ingestion of raws.

static makeRefCatCollectionName(*labels) → str¶

Return a global (not instrument-specific) name for a collection that holds reference catalogs.

With no arguments, this returns the name of the collection that holds all reference catalogs (usually a CHAINED collection, at least in long-lived repos that may contain more than one reference catalog).

Parameters:	*labels : `str` Strings to be added to the global collection name, in order to define a collection name for one or more reference catalogs being ingested at the same time.
Returns:	name : `str` Collection name.

Notes

This is a staticmethod, not a classmethod, because it should be the same for all instruments.

makeUmbrellaCollectionName() → str¶

Return the name of the umbrella CHAINED collection for this instrument that combines all standard recommended input collections.

This method should almost never be overridden by derived classes.

Returns:	name : `str` Name for the umbrella collection.

makeUnboundedCalibrationRunName(*labels) → str¶

Make a RUN collection name appropriate for inserting calibration datasets whose validity ranges are unbounded.

Parameters:	*labels : `str` Extra strings to be included in the base name, using the default delimiter for collection names. Usually this is the name of the ticket on which the calibration collection is being created.
Returns:	name : `str` Run collection name.

register(registry, *, update=False)¶

Insert instrument, physical_filter, and detector entries into a Registry.

Parameters:	registry : `lsst.daf.butler.Registry` Registry client for the data repository to modify. update : `bool`, optional If `True` (`False` is default), update existing records if they differ from the new ones.
Raises:	lsst.daf.butler.registry.ConflictingDefinitionError Raised if any existing record has the same key but a different definition as one being registered.

Notes

New detectors and physical filters can always be added by calling this method multiple times, as long as no existing records have changed (if existing records have changed, update=True must be used). Old records can never be removed by this method.

Implementations should guarantee that registration is atomic (the registry should not be modified if any error occurs) and idempotent at the level of individual dimension entries; new detectors and filters should be added, but changes to any existing record should not be. This can generally be achieved via a block like:

with registry.transaction():
    registry.syncDimensionData("instrument", ...)
    registry.syncDimensionData("detector", ...)
    self.registerFilters(registry)

writeAdditionalCuratedCalibrations(butler: lsst.daf.butler._butler.Butler, collection: Optional[str] = None, labels: Sequence[str] = ()) → None¶

Write additional curated calibrations that might be instrument specific and are not part of the standard set.

Default implementation does nothing.

Parameters:

butler : lsst.daf.butler.Butler: Butler to use to store these calibrations.
collection : str, optional: Name to use for the calibration collection that associates all datasets with a validity range. If this collection already exists, it must be a CALIBRATION collection, and it must not have any datasets that would conflict with those inserted by this method. If None, a collection name is worked out automatically from the instrument name and other metadata by calling makeCalibrationCollectionName, but this default name may not work well for long-lived repositories unless labels is also provided (and changed every time curated calibrations are ingested).
labels : Sequence [ str ], optional: Extra strings to include in collection names, after concatenating them with the standard collection name delimeter. If provided, these are inserted into the names of the RUN collections that datasets are inserted directly into, as well the CALIBRATION collection if it is generated automatically (i.e. if collection is None). Usually this is just the name of the ticket on which the calibration collection is being created.

writeCameraGeom(butler: lsst.daf.butler._butler.Butler, collection: Optional[str] = None, labels: Sequence[str] = ()) → None¶

Write the default camera geometry to the butler repository and associate it with the appropriate validity range in a calibration collection.

Parameters:

butler : lsst.daf.butler.Butler: Butler to use to store these calibrations.
collection : str, optional: Name to use for the calibration collection that associates all datasets with a validity range. If this collection already exists, it must be a CALIBRATION collection, and it must not have any datasets that would conflict with those inserted by this method. If None, a collection name is worked out automatically from the instrument name and other metadata by calling makeCalibrationCollectionName, but this default name may not work well for long-lived repositories unless labels is also provided (and changed every time curated calibrations are ingested).
labels : Sequence [ str ], optional: Extra strings to include in collection names, after concatenating them with the standard collection name delimeter. If provided, these are inserted into the names of the RUN collections that datasets are inserted directly into, as well the CALIBRATION collection if it is generated automatically (i.e. if collection is None). Usually this is just the name of the ticket on which the calibration collection is being created.

writeCuratedCalibrations(butler: lsst.daf.butler._butler.Butler, collection: Optional[str] = None, labels: Sequence[str] = ()) → None¶

Write human-curated calibration Datasets to the given Butler with the appropriate validity ranges.

Parameters:

butler : lsst.daf.butler.Butler: Butler to use to store these calibrations.
collection : str, optional: Name to use for the calibration collection that associates all datasets with a validity range. If this collection already exists, it must be a CALIBRATION collection, and it must not have any datasets that would conflict with those inserted by this method. If None, a collection name is worked out automatically from the instrument name and other metadata by calling makeCalibrationCollectionName, but this default name may not work well for long-lived repositories unless labels is also provided (and changed every time curated calibrations are ingested).
labels : Sequence [ str ], optional: Extra strings to include in collection names, after concatenating them with the standard collection name delimeter. If provided, these are inserted into the names of the RUN collections that datasets are inserted directly into, as well the CALIBRATION collection if it is generated automatically (i.e. if collection is None). Usually this is just the name of the ticket on which the calibration collection is being created.

Notes

Expected to be called from subclasses. The base method calls writeCameraGeom, writeStandardTextCuratedCalibrations, and writeAdditionalCuratdCalibrations.

writeStandardTextCuratedCalibrations(butler: lsst.daf.butler._butler.Butler, collection: Optional[str] = None, labels: Sequence[str] = ()) → None¶

Write the set of standardized curated text calibrations to the repository.

Parameters:

butler : lsst.daf.butler.Butler: Butler to receive these calibration datasets.
collection : str, optional: Name to use for the calibration collection that associates all datasets with a validity range. If this collection already exists, it must be a CALIBRATION collection, and it must not have any datasets that would conflict with those inserted by this method. If None, a collection name is worked out automatically from the instrument name and other metadata by calling makeCalibrationCollectionName, but this default name may not work well for long-lived repositories unless labels is also provided (and changed every time curated calibrations are ingested).
labels : Sequence [ str ], optional: Extra strings to include in collection names, after concatenating them with the standard collection name delimeter. If provided, these are inserted into the names of the RUN collections that datasets are inserted directly into, as well the CALIBRATION collection if it is generated automatically (i.e. if collection is None). Usually this is just the name of the ticket on which the calibration collection is being created.

Navigation

Instrument¶