MetricsControllerTask

MetricsControllerTask runs collections of lsst.verify.tasks.MetricTask, and stores the resulting Measurement objects using the Job persistence framework. It is a stand-in for functionality provided by the Gen 3 Tasks framework. The datasets that MetricsControllerTask consumes depend on the MetricTasks to be run, and are handled automatically.

MetricsControllerTask is not a command-line task, but may be called from within both task- and non-task pipelines.

Processing summary

Unlike most tasks, MetricsControllerTask has a runDataRefs method that takes a list of data references. MetricsControllerTask calls every MetricTask in measurers on every data reference, loading any datasets necessary. It produces one Job object for each input data reference, and writes them to disk.

Python API summary

from lsst.verify.gen2tasks.metricsControllerTask import MetricsControllerTask
classMetricsControllerTask(config=None, **kwargs)

A Task for executing a collection of `lsst.verify.tasks.MetricTask` objects...

attributeconfig

Access configuration fields and retargetable subtasks.

See also

See the MetricsControllerTask API reference for complete details.

Configuration fields

jobFileTemplate

Default
'metrics{id}.{dataId}.verify.json'
Field type

str Field

A template for the path to which the measurements are written. {id} is replaced with a unique index (recommended), while {dataId} is replaced with the data ID.

In Depth

Because MetricsControllerTask applies every MetricTask to every input data reference indiscriminately, it may not give good results with metrics or data references having a mixture of granularities (e.g., CCD-level, visit-level, dataset-level). The recommended way around this limitation is to create multiple MetricsControllerTask objects, and configure each one for metrics of a single granularity.

Each MetricTask in a MetricsControllerTask must measure a different metric, or they will overwrite each others’ values.

Examples

Typically, a user of MetricsControllerTask will configure it with tasks that have register decorator:

from lsst.verify.gen2tasks import register, \
    MetricsControllerTask
from lsst.verify.tasks import MetricTask


@register("ultimate")
class UltimateMetric(MetricTask):
    ...


@register("second")
class SecondaryMetric(MetricTask):
    ...


config = MetricsControllerTask.ConfigClass()
config.measurers = ["ultimate", "second"]
config.measurers["ultimate"].answer = 42
task = MetricsControllerTask(config)

# CCD-level metrics need CCD-level datarefs
# Exact dataset type doesn't matter
datarefs = [butler.subset("calexp", visit=42, ccd=ccd)
            for ccd in range(1, 101)]
struct = task.runDataRefs(datarefs)
assert len(struct.jobs) == len(datarefs)

A MetricTask must have the registerMultiple decorator to be used multiple times:

from lsst.verify.gen2tasks import registerMultiple, MetricsControllerTask
from lsst.verify.tasks import MetricTask

@registerMultiple("common")
class CommonMetric(MetricTask):
    ...

config = MetricsControllerTask.ConfigClass()
config.measurers = ["common"]
config.measurers["common"].configs["case1"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case1"].metric = "misc_tasks.Case1Metric"
config.measurers["common"].configs["case2"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case2"].metric = "misc_tasks.Case2Metric"
task = MetricsControllerTask(config)

MetricsControllerTask will create and run two different CommonMetric objects, one configured with metric = "misc_tasks.Case1Metric" and one with metric = "misc_tasks.Case2Metric". The names "case1" and "case2" can be anything, so long as they are unique.