verify_metrics

verify_metrics provides centralized metric and specification definitions for the LSST Science Pipelines Verification Framework. The lsst.verify module uses these metrics and specifications by default.

For background, see the SQR-017 (design) and SQR-019 (demo) technotes.

Project info

Repository
https://github.com/lsst/verify_metrics
JIRA component
Verification.

Contributing

Basics

Registering metrics

Metrics are measureable things. They can be astronomy concepts (an astrometric RMS) or computational (a task’s timing or memory usage).

Metrics that are measured by packages are registered here in verify_metrics by adding or extending a metric definition YAML file in the metrics/ directory. See metrics/README.rst for more information.

Registering specifications

Specifications are tests of metric measurements. A specification might say that a metric measurement should be less than a certain value if that measurement was made under a certain set of conditions (provenance). Whenever metrics are measured, DM’s SQUASH harness tests those measurements against specifications. Depending on the pass/fail outcome, subscribers can receive notifications (alerts).

New specifications are registered here in verify_metrics by adding or extending a specification definition YAML file in the specs/ directory. See specs/README.rst for more information.