MetricsControllerTask¶

MetricsControllerTask runs collections of lsst.verify.gen2tasks.MetricTask, and stores the resulting Measurement objects using the Job persistence framework. It is a stand-in for functionality provided by the Gen 3 Tasks framework. The datasets that MetricsControllerTask consumes depend on the MetricTasks to be run, and are handled automatically.

MetricsControllerTask is not a command-line task, but may be called from within both task- and non-task pipelines.

Processing summary¶

Unlike most tasks, MetricsControllerTask has a runDataRefs method that takes a list of data references. MetricsControllerTask calls every MetricTask in measurers on every data reference, loading any datasets necessary. It produces one Job object for each input data reference, and writes them to disk.

Python API summary¶

from lsst.verify.gen2tasks.metricsControllerTask import MetricsControllerTask

classMetricsControllerTask(config=None, **kwargs): A Task for executing a collection of `lsst.verify.gen2tasks.MetricTask` objects...

attributeconfig: Access configuration fields and retargetable subtasks.

Retargetable subtasks¶

measurers¶

Default

None

Field type

Multi-selection RegistryField

Choices

'numSciSources': lsst.ip.diffim.metrics.NumberSciSourcesMetricTask
'fracDiaSourcesToSciSources': lsst.ip.diffim.metrics.FractionDiaSourcesToSciSourcesMetricTask
'timing': lsst.verify.gen2tasks.metricRegistry._MultiConfigFactory
'numNewDiaObjects': lsst.ap.association.metrics.NumberNewDiaObjectsMetricTask
'numUnassociatedDiaObjects': lsst.ap.association.metrics.NumberUnassociatedDiaObjectsMetricTask
'fracUpdatedDiaObjects': lsst.ap.association.metrics.FractionUpdatedDiaObjectsMetricTask
'totalUnassociatedDiaObjects': lsst.ap.association.metrics.TotalUnassociatedDiaObjectsMetricTask

MetricTasks to call and their configuration. Each MetricTask must be identified by the name passed to its register or registerMultiple decorator.

metadataAdder¶

Default: lsst.verify.gen2tasks.metadataTask.SquashMetadataTask
Field type: ConfigurableField

Task for adding metadata needed by measurement clients. Its run method must take a Job as its first parameter, and should accept unknown keyword arguments. It must return a Struct with the field job pointing to the modified job.

Configuration fields¶

jobFileTemplate¶

Default: 'metrics{id}.{dataId}.verify.json'
Field type: str Field

A template for the path to which the measurements are written. {id} is replaced with a unique index (recommended), while {dataId} is replaced with the data ID.

In Depth¶

Because MetricsControllerTask applies every MetricTask to every input data reference indiscriminately, it may not give good results with metrics or data references having a mixture of granularities (e.g., CCD-level, visit-level, dataset-level). The recommended way around this limitation is to create multiple MetricsControllerTask objects, and configure each one for metrics of a single granularity.

Each MetricTask in a MetricsControllerTask must measure a different metric, or they will overwrite each others’ values.

Examples¶

Typically, a user of MetricsControllerTask will configure it with tasks that have register decorator:

from lsst.verify.gen2tasks import register, \
    MetricTask, MetricsControllerTask


@register("ultimate")
class UltimateMetric(MetricTask):
    ...


@register("second")
class SecondaryMetric(MetricTask):
    ...


config = MetricsControllerTask.ConfigClass()
config.measurers = ["ultimate", "second"]
config.measurers["ultimate"].answer = 42
task = MetricsControllerTask(config)

# CCD-level metrics need CCD-level datarefs
# Exact dataset type doesn't matter
datarefs = [butler.subset("calexp", visit=42, ccd=ccd)
            for ccd in range(1, 101)]
struct = task.runDataRefs(datarefs)
assert len(struct.jobs) == len(datarefs)

A MetricTask must have the registerMultiple decorator to be used multiple times:

from lsst.verify.gen2tasks import registerMultiple, MetricTask, MetricsControllerTask

@registerMultiple("common")
class CommonMetric(MetricTask):
    ...

config = MetricsControllerTask.ConfigClass()
config.measurers = ["common"]
config.measurers["common"].configs["case1"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case1"].metric = "misc_tasks.Case1Metric"
config.measurers["common"].configs["case2"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case2"].metric = "misc_tasks.Case2Metric"
task = MetricsControllerTask(config)

MetricsControllerTask will create and run two different CommonMetric objects, one configured with metric = "misc_tasks.Case1Metric" and one with metric = "misc_tasks.Case2Metric". The names "case1" and "case2" can be anything, so long as they are unique.

Navigation