IdGenerator¶
- class lsst.meas.base.IdGenerator¶
Bases:
object
A helper class for packing some combination of a data ID, a per-data-ID counter, and a release ID into a single 64-bit integer.
As an object frequently passed into code that otherwise has no knowledge of its own data ID,
IdGenerator
also implements__str__
to provide a human-readable representation of the data ID for use in logs and exception messages, with a suitable fallback when no data ID was provided to it.See also
Notes
Instances of this class are expected to usually be created via configuration, which will return a derived instance. This pattern starts with one of
DetectorExposureIdGeneratorConfig
,DetectorVisitIdGeneratorConfig
, andSkyMapIdGeneratorConfig
(which have the same interface), and looks something this:from lsst.meas.base import DetectorVisitIdGeneratorConfig from lsst.pex.config import Config from lsst.pipe.base import PipelineTask
- class SomeTaskConfig(PipelineTaskConfig, …):
id_generator = DetectorVisitIdGeneratorConfig.make_field()
class SomeTask(PipelineTaskTask):
ConfigClass = SomeTaskConfig
…
- def runQuantum(self, …, data_id: DataCoordinate):
id_generator = self.config.apply(data_id) catalog = id_generator.make_source_catalog(self.schema) …
There is no requirement that
IdGenerator
instances be constructed inPipelineTask.runQuantum
methods and passed to therun
method, but this is the most common approach.Code that wishes to instead unpack these record IDs to obtain the release ID, data ID and counter value should use the same config (often loaded from the
Butler
) and pass a fully-expanded data ID identifying only a particularskymap
orinstrument
tounpacker_from_config
:config = butler.get("some_task_config") catalog = butler.get("some_output_catalog", given_data_id) unpacker = IdGenerator.unpacker_from_config( config.id_generator, butler.registry.expandDataId(skymap="HSC"), ) release_id, embedded_data_id, counter = unpacker(catalog[0]["id"]) assert embedded_data_id == given_data_id
This example is a bit contrived, as the ability to reconstruct the data ID is really only useful when you don’t have it already, such as when the record ID is obtained from some further-processed version of the original table (such as a SQL database), and in that context the right config to load will not be obvious unless it has been carefully documented.
Simple instances of the base class that do not include a data ID may also be constructed by calling the constructor directly:
id_generator = IdGenerator()
These IDs may not be unpacked, but they also don’t need to be, because they’re just the per-catalog “counter” integer already.
Attributes Summary
The integer identifier for the full catalog with this data ID, not just one of its rows (
int
).Methods Summary
arange
(*args, **kwargs)Generate an array of integer IDs for this catalog.
make_source_catalog
(schema)Construct a empty catalog object with an ID factory.
Construct a new
lsst.afw.table.IdFactory
for this catalog.unpacker_from_config
(config, fixed)Return a callable that unpacks the IDs generated by this class, from a config field.
unpacker_from_dimension_packer
(dimension_packer)Return a callable that unpacks the IDs generated by this class, from a
lsst.daf.butler.DimensionPacker
instance.Attributes Documentation
- catalog_id¶
The integer identifier for the full catalog with this data ID, not just one of its rows (
int
).This combines the packed data ID and release ID, but not the counter.
Methods Documentation
- arange(*args, **kwargs) ndarray ¶
Generate an array of integer IDs for this catalog.
All parameters are forwarded to
numpy.arange
to generate an array of per-catalog counter integers. These are then combined with thecatalog_id`
to form the returned array.The IDs generated by
arange
will be equivalent to those generated bymake_table_id_factory
(and by extension,make_source_catalog
) only if the counter integers start with1
, not0
, because that’s whatIdFactory
does.
- make_source_catalog(schema: Schema) SourceCatalog ¶
Construct a empty catalog object with an ID factory.
This is a convenience function for the common pattern of calling
make_table_id_factory
, constructing aSourceTable
from that, and then constructing an (empty)SourceCatalog
from that.
- make_table_id_factory() IdFactory ¶
Construct a new
lsst.afw.table.IdFactory
for this catalog.
- classmethod unpacker_from_config(config: BaseIdGeneratorConfig, fixed: DataCoordinate) Callable[[int], tuple[lsst.daf.butler.dimensions._coordinate.DataCoordinate, int]] ¶
Return a callable that unpacks the IDs generated by this class, from a config field.
- Parameters:
- config
BaseIdGeneratorConfig
Configuration for an ID generator.
- fixed
DataCoordinate
Data ID identifying the dimensions that are considered fixed by the
IdGenerator
that produced the IDs: usually justinstrument
orskymap
, depending on the configuration. For most configurations this will need to be a fully-expanded data ID.
- config
- Returns:
- unpacker
Callable that takes a single
int
argument (an ID generated by an identically-configuredIdGenerator
) and returns a tuple of:
Notes
This method cannot be used on IDs generated without a data ID.
- classmethod unpacker_from_dimension_packer(dimension_packer: DimensionPacker, n_releases: int = 1) Callable[[int], tuple[int, lsst.daf.butler.dimensions._coordinate.DataCoordinate, int]] ¶
Return a callable that unpacks the IDs generated by this class, from a
lsst.daf.butler.DimensionPacker
instance.- Parameters:
- dimension_packer
lsst.daf.butler.DimensionPacker
Dimension packer used to construct the original
DimensionPackerIdGenerator
.- n_releases
int
, optional Number of (contiguous, starting from zero)
release_id
values to reserve space for. One (not zero) is used to reserve no space.
- dimension_packer
- Returns:
- unpacker
Callable that takes a single
int
argument (an ID generated by an identically-constructedDimensionPackerIdGenerator
) and returns a tuple of:
Notes
This method cannot be used on IDs generated with no data ID.