MetricTask (lsst.verify.gen2tasks)¶
MetricTask is a base class for generating lsst.verify.Measurement given input data.
Each MetricTask class accepts specific type(s) of datasets and produces measurements for a specific metric or family of metrics.
While its API mirrors that of PipelineTask, this version of MetricTask is designed to be used with the Gen 2 framework.
Python API summary¶
from lsst.verify.gen2tasks.metricTask import MetricTask
-
class
MetricTask A base class for tasks that compute one metric from input datasets
...
-
attribute
config Access configuration fields and retargetable subtasks.
See also
See the MetricTask API reference for complete details.
Retargetable subtasks¶
No subtasks.
Configuration fields¶
No configuration fields.
In Depth¶
Subclassing¶
MetricTask is primarily customized using the run or adaptArgsAndRun methods.
Each subclass must also implement the getOutputMetricName method.
The task config should use lsst.pipe.base.InputDatasetConfig to identify input datasets as if it were a PipelineTask.
Only the name field is used in a Gen 2 context, but use of InputDatasetConfig is expected to simplify the transition to Gen 3.
Error Handling¶
In general, a MetricTask may run in three cases:
- the task can compute the metric without incident.
- the task does not have the datasets required to compute the metric. This often happens if the user runs generic metric configurations on arbitrary pipelines, or if they make changes to the pipeline configuration that enable or disable processing steps. More rarely, it can happen when trying to compute diagnostic metrics on incomplete (i.e., failed) pipeline runs.
- the task has the data it needs, but cannot compute the metric. This could be because the data are corrupted, because the selected algorithm fails, or because the metric is ill-defined given the data.
A MetricTask must distinguish between these cases so that MetricsControllerTask and future calling frameworks can handle them appropriately.
A task for a metric that does not apply to a particular pipeline run (case 2) must return None in place of a Measurement.
A task that cannot give a valid result (case 3) must raise MetricComputationError.
In grey areas, developers should choose a MetricTask’s behavior based on whether the root cause is closer to case 2 or case 3.
For example, TimingMetricTask accepts top-level task metadata as input, but returns None if it can’t find metadata for the subtask it is supposed to time.
While the input dataset is available, the subtask metadata are most likely missing because the subtask was never run, making the situation equivalent to case 2.
On the other hand, metadata with nonsense values falls squarely under case 3.
Registration¶
The most common way to run MetricTask is as plugins to MetricsControllerTask.
Most MetricTask classes should use the register decorator to assign a plugin name.
Because of implementation limitations, each registered name may appear at most once in MetricsControllerConfig.
If you expect to need multiple instances of the same MetricTask class (typically when the same class can compute multiple metrics), it must have the registerMultiple decorator instead.