Job

class lsst.verify.Job(measurements=None, metrics=None, specs=None, meta=None)

Bases: lsst.verify.jsonmixin.JsonSerializationMixin

Container for Measurements, Blob s, and Metadata associated with a pipeline run.

Parameters:
measurements : MeasurementSet or list of Measurements, optional

Measurements to report in the Job.

metrics : list of Metrics or a MetricSet, optional

Optional list of Metrics, or a MetricSet.

specs : SpecificationSet or list of Specifications, optional

Optional specification information.

meta : dict, optional

Optional dictionary of metadata key-value entries.

Attributes Summary

json Job data as a JSON-serialiable dict.
measurements Measurements associated with the pipeline verification job (MeasurementSet).
meta Metadata mapping (Metadata).
metrics Metrics associated with the pipeline verification job (MetricSet).
specs Specifications associated with the pipeline verifification job (SpecificationSet).

Methods Summary

deserialize([measurements, blobs, metrics, …]) Deserialize a Verification Framework Job from a JSON serialization.
dispatch([api_user, api_password, api_url]) POST the job to SQUASH, LSST Data Management’s metric dashboard.
load_metrics_package([package_name_or_path, …]) Create a Job with metrics and specifications pre-loaded from a Verification Framework metrics package, such as verify_metrics.
reload_metrics_package([…]) Load a metrics package and add metric and specification definitions to the Job, as well as the collected measurements.
report([name, spec_tags, metric_tags]) Create a verification report that lists the pass/fail status of measurements against specifications in this job.
write(filename) Write a JSON serialization to the filesystem.

Attributes Documentation

json

Job data as a JSON-serialiable dict.

measurements

Measurements associated with the pipeline verification job (MeasurementSet).

meta

Metadata mapping (Metadata).

metrics

Metrics associated with the pipeline verification job (MetricSet).

specs

Specifications associated with the pipeline verifification job (SpecificationSet).

Methods Documentation

classmethod deserialize(measurements=None, blobs=None, metrics=None, specs=None, meta=None)

Deserialize a Verification Framework Job from a JSON serialization.

Parameters:
measurements : list, optional

List of serialized Measurement objects.

blobs : list, optional

List of serialized Blob objects.

metrics : list, optional

List of serialized Metric objects.

specs : list, optional

List of serialized specification objects.

meta : dict, optional

Dictionary of key-value metadata entries.

Returns:
job : Job

Job instance built from serialized data.

Examples

Together, Job.json and Job.deserialize allow a verification job to be serialized and later re-instantiated.

>>> import json
>>> job = Job()
>>> json_str = json.dumps(job.json)
>>> json_obj = json.loads(json_str)
>>> new_job = Job.deserialize(**json_obj)
dispatch(api_user=None, api_password=None, api_url='https://squash-restful-api.lsst.codes', **kwargs)

POST the job to SQUASH, LSST Data Management’s metric dashboard.

Parameters:
api_url : str, optional

Root URL of the SQUASH API server.

api_user : str, optional

API username.

api_password : str, optional

API password.

**kwargs : optional

Additional keyword arguments passed to lsst.verify.squash.post.

classmethod load_metrics_package(package_name_or_path='verify_metrics', subset=None, measurements=None, meta=None)

Create a Job with metrics and specifications pre-loaded from a Verification Framework metrics package, such as verify_metrics.

Parameters:
package_name_or_path : str, optional

Name of an EUPS package that hosts metric and specification definition YAML files or the file path to a metrics package. 'verify_metrics' is the default package, and is where metrics and specifications are defined for most LSST Science Pipelines packages.

subset : str, optional

If set, only metrics and specification for this package are loaded. For example, if subset='validate_drp', only validate_drp metrics are loaded. This argument is equivalent to the MetricSet.subset method. Default is None.

measurements : MeasurementSet or list of Measurements, optional

Measurements to report in the Job.

meta : dict, optional

Optional dictionary of metadata key-value entries to include in the Job.

Returns:
job : Job

Job instance.

reload_metrics_package(package_name_or_path='verify_metrics', subset=None)

Load a metrics package and add metric and specification definitions to the Job, as well as the collected measurements.

Parameters:
package_name_or_path : str, optional

Name of an EUPS package that hosts metric and specification definition YAML files or the file path to a metrics package. 'verify_metrics' is the default package, and is where metrics and specifications are defined for most packages.

subset : str, optional

If set, only metrics and specification for this package are loaded. For example, if subset='validate_drp', only validate_drp metrics are included in the MetricSet. This argument is equivalent to the MetricSet.subset method. Default is None.

Notes

This method is useful for loading metric and specification definitions into a job that was created without this information. In addition to being added to Job.metrics, metrics are also attached to Job.measurements items. This ensures that measurement values are normalized into the units of the metric definition when a Job is serialized.

report(name=None, spec_tags=None, metric_tags=None)

Create a verification report that lists the pass/fail status of measurements against specifications in this job.

In a Jupyter notebook, this report can be shown as an inline table.

Parameters:
name : str or lsst.verify.Name, optional

A package or metric name to subset specifications by. When set, only measurement and specification combinations belonging to that package or metric are included in the report.

spec_tags : sequence of str, optional

A set of specification tag strings. when given, only specifications that have all the given tags are included in the report. For example, spec_tags=['LPM-17', 'minimum'].

metric_tags : sequence of str, optional

A set of metric tag strings. When given, only specifications belonging to metrics that posess all given tags are included in the report. For example, metric_tags=['LPM-17', 'photometry'] selects sepifications that have both the 'LPM-17' and 'photometry' tags.

Returns:
report : lsst.verify.Report

Report instance. In a Jupyter notebook, you can view the report by calling Report.show.

Notes

This method uses the lsst.verify.SpecificationSet.report API to create the lsst.verify.Report, automatically inserting the Job’s measurements and metadata for filtering specifiation tests.

In a Jupyter notebook environment, use the lsst.verify.Report.show method to view an interactive HTML table.

job = lsst.verify.Job()
# ...
report = job.report()
report.show()
write(filename)

Write a JSON serialization to the filesystem.

Parameters:
filename : str

Name of the JSON file (including directories). This name should be unique among all task executions in a pipeline. The recommended extension is '.verify.json'. This convention is used by post-processing tools to discover verification framework outputs.