MetricsControllerTask

MetricsControllerTask runs collections of lsst.verify.gen2tasks.MetricTask, and stores the resulting Measurement objects using the Job persistence framework. It is a stand-in for functionality provided by the Gen 3 Tasks framework. The datasets that MetricsControllerTask consumes depend on the MetricTasks to be run, and are handled automatically.

MetricsControllerTask is not a command-line task, but may be called from within both task- and non-task pipelines.

Processing summary

Unlike most tasks, MetricsControllerTask has a runDataRefs method that takes a list of data references. MetricsControllerTask calls every MetricTask in measurers on every data reference, loading any datasets necessary. It produces one Job object for each input data reference, and writes them to disk.

Python API summary

from lsst.verify.gen2tasks.metricsControllerTask import MetricsControllerTask
classMetricsControllerTask(config=None, **kwargs)

A Task for executing a collection of `lsst.verify.gen2tasks.MetricTask` objects...

attributeconfig

Access configuration fields and retargetable subtasks.

See also

See the MetricsControllerTask API reference for complete details.

Retargetable subtasks

measurers

Default
None
Field type
Multi-selection RegistryField
Choices
'timing'
lsst.verify.gen2tasks.metricRegistry._MultiConfigFactory
MetricTasks to call and their configuration. Each MetricTask must be identified by the name passed to its register or registerMultiple decorator.

metadataAdder

Default
lsst.verify.gen2tasks.metadataTask.SquashMetadataTask
Field type
ConfigurableField
Task for adding metadata needed by measurement clients. Its run method must take a Job as its first parameter, and should accept unknown keyword arguments. It must return a Struct with the field job pointing to the modified job.

Configuration fields

jobFileTemplate

Default
'metrics{id}.{dataId}.verify.json'
Field type
str Field
A template for the path to which the measurements are written. {id} is replaced with a unique index (recommended), while {dataId} is replaced with the data ID.

In Depth

Because MetricsControllerTask applies every MetricTask to every input data reference indiscriminately, it may not give good results with metrics or data references having a mixture of granularities (e.g., CCD-level, visit-level, dataset-level). The recommended way around this limitation is to create multiple MetricsControllerTask objects, and configure each one for metrics of a single granularity.

Each MetricTask in a MetricsControllerTask must measure a different metric, or they will overwrite each others’ values.

Examples

Typically, a user of MetricsControllerTask will configure it with tasks that have register decorator:

from lsst.verify.gen2tasks import register, \
    MetricTask, MetricsControllerTask


@register("ultimate")
class UltimateMetric(MetricTask):
    ...


@register("second")
class SecondaryMetric(MetricTask):
    ...


config = MetricsControllerTask.ConfigClass()
config.measurers = ["ultimate", "second"]
config.measurers["ultimate"].answer = 42
task = MetricsControllerTask(config)

# CCD-level metrics need CCD-level datarefs
# Exact dataset type doesn't matter
datarefs = [butler.subset("calexp", visit=42, ccd=ccd)
            for ccd in range(1, 101)]
struct = task.runDataRefs(datarefs)
assert len(struct.jobs) == len(datarefs)

A MetricTask must have the registerMultiple decorator to be used multiple times:

from lsst.verify.gen2tasks import registerMultiple, MetricTask, MetricsControllerTask

@registerMultiple("common")
class CommonMetric(MetricTask):
    ...

config = MetricsControllerTask.ConfigClass()
config.measurers = ["common"]
config.measurers["common"].configs["case1"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case1"].metric = "misc_tasks.Case1Metric"
config.measurers["common"].configs["case2"] = CommonMetric.ConfigClass()
config.measurers["common"].configs["case2"].metric = "misc_tasks.Case2Metric"
task = MetricsControllerTask(config)

MetricsControllerTask will create and run two different CommonMetric objects, one configured with metric = "misc_tasks.Case1Metric" and one with metric = "misc_tasks.Case2Metric". The names "case1" and "case2" can be anything, so long as they are unique.