IdGenerator¶

class lsst.meas.base.IdGenerator¶

Bases: object

A helper class for packing some combination of a data ID, a per-data-ID counter, and a release ID into a single 64-bit integer.

As an object frequently passed into code that otherwise has no knowledge of its own data ID, IdGenerator also implements __str__ to provide a human-readable representation of the data ID for use in logs and exception messages, with a suitable fallback when no data ID was provided to it.

See also

Generating source and object IDs

Notes

Instances of this class are expected to usually be created via configuration, which will return a derived instance. This pattern starts with one of DetectorExposureIdGeneratorConfig, DetectorVisitIdGeneratorConfig, and SkyMapIdGeneratorConfig (which have the same interface), and looks something this:

from lsst.meas.base import DetectorVisitIdGeneratorConfig from lsst.pex.config import Config from lsst.pipe.base import PipelineTask

class SomeTaskConfig(PipelineTaskConfig, …):
id_generator = DetectorVisitIdGeneratorConfig.make_field()

class SomeTask(PipelineTaskTask):

ConfigClass = SomeTaskConfig

…

def runQuantum(self, …, data_id: DataCoordinate):
id_generator = self.config.apply(data_id) catalog = id_generator.make_source_catalog(self.schema) …

There is no requirement that IdGenerator instances be constructed in PipelineTask.runQuantum methods and passed to the run method, but this is the most common approach.

Code that wishes to instead unpack these record IDs to obtain the release ID, data ID and counter value should use the same config (often loaded from the Butler) and pass a fully-expanded data ID identifying only a particular skymap or instrument to unpacker_from_config:

config = butler.get("some_task_config")
catalog = butler.get("some_output_catalog", given_data_id)
unpacker = IdGenerator.unpacker_from_config(
    config.id_generator, butler.registry.expandDataId(skymap="HSC"),
)
release_id, embedded_data_id, counter = unpacker(catalog[0]["id"])
assert embedded_data_id == given_data_id

This example is a bit contrived, as the ability to reconstruct the data ID is really only useful when you don’t have it already, such as when the record ID is obtained from some further-processed version of the original table (such as a SQL database), and in that context the right config to load will not be obvious unless it has been carefully documented.

Simple instances of the base class that do not include a data ID may also be constructed by calling the constructor directly:

id_generator = IdGenerator()

These IDs may not be unpacked, but they also don’t need to be, because they’re just the per-catalog “counter” integer already.

Attributes Summary

catalog_id

The integer identifier for the full catalog with this data ID, not just one of its rows (int).

Methods Summary

`arange`(args, *kwargs)	Generate an array of integer IDs for this catalog.
`make_source_catalog`(schema)	Construct a empty catalog object with an ID factory.
`make_table_id_factory`()	Construct a new `lsst.afw.table.IdFactory` for this catalog.
`unpacker_from_config`(config, fixed)	Return a callable that unpacks the IDs generated by this class, from a config field.
`unpacker_from_dimension_packer`(dimension_packer)	Return a callable that unpacks the IDs generated by this class, from a `lsst.daf.butler.DimensionPacker` instance.

Attributes Documentation

catalog_id¶

The integer identifier for the full catalog with this data ID, not just one of its rows (int).

This combines the packed data ID and release ID, but not the counter.

Methods Documentation

arange(*args, **kwargs) → ndarray¶

Generate an array of integer IDs for this catalog.

All parameters are forwarded to numpy.arange to generate an array of per-catalog counter integers. These are then combined with the catalog_id` to form the returned array.

The IDs generated by arange will be equivalent to those generated by make_table_id_factory (and by extension, make_source_catalog) only if the counter integers start with 1, not 0, because that’s what IdFactory does.

make_source_catalog(schema: Schema) → SourceCatalog¶

Construct a empty catalog object with an ID factory.

This is a convenience function for the common pattern of calling make_table_id_factory, constructing a SourceTable from that, and then constructing an (empty) SourceCatalog from that.

make_table_id_factory() → IdFactory¶: Construct a new lsst.afw.table.IdFactory for this catalog.

classmethod unpacker_from_config(config: BaseIdGeneratorConfig, fixed: DataCoordinate) → Callable[[int], tuple[lsst.daf.butler.core.dimensions._coordinate.DataCoordinate, int]]¶

Return a callable that unpacks the IDs generated by this class, from a config field.

Parameters:

configBaseIdGeneratorConfig: Configuration for an ID generator.
fixedDataCoordinate: Data ID identifying the dimensions that are considered fixed by the IdGenerator that produced the IDs: usually just instrument or skymap, depending on the configuration. For most configurations this will need to be a fully-expanded data ID.

Returns:

unpacker

Callable that takes a single int argument (an ID generated by an identically-configured IdGenerator) and returns a tuple of:

release_id: the integer that identifies a data release or similar (int);
data_id : the data ID used to initialize the original ID generator (DataCoordinate);
counter : the counter part of the original ID (int).

Notes

This method cannot be used on IDs generated without a data ID.

classmethod unpacker_from_dimension_packer(dimension_packer: DimensionPacker, n_releases: int = 1) → Callable[[int], tuple[int, lsst.daf.butler.core.dimensions._coordinate.DataCoordinate, int]]¶

Return a callable that unpacks the IDs generated by this class, from a lsst.daf.butler.DimensionPacker instance.

Parameters:

dimension_packerlsst.daf.butler.DimensionPacker: Dimension packer used to construct the original DimensionPackerIdGenerator.
n_releasesint, optional: Number of (contiguous, starting from zero) release_id values to reserve space for. One (not zero) is used to reserve no space.

Returns:

unpacker

Callable that takes a single int argument (an ID generated by an identically-constructed DimensionPackerIdGenerator) and returns a tuple of:

release_id: the integer that identifies a data release or similar (int);
data_id : the data ID used to initialize the original ID generator (DataCoordinate);
counter : the counter part of the original ID (int).

Notes

This method cannot be used on IDs generated with no data ID.

Navigation

IdGenerator¶