Job¶
-
class
lsst.verify.
Job
(measurements=None, metrics=None, specs=None, meta=None)¶ Bases:
lsst.verify.jsonmixin.JsonSerializationMixin
Container for
Measurement
s,Blob
s, andMetadata
associated with a pipeline run.Parameters: - measurements :
MeasurementSet
orlist
ofMeasurement
s, optional Measurement
s to report in the Job.- metrics :
list
ofMetric
s or aMetricSet
, optional - specs :
SpecificationSet
orlist
ofSpecification
s, optional Optional specification information.
- meta :
dict
, optional Optional dictionary of metadata key-value entries.
Attributes Summary
json
Job
data as a JSON-serialiabledict
.measurements
Measurements associated with the pipeline verification job ( MeasurementSet
).meta
Metadata mapping ( Metadata
).metrics
Metrics associated with the pipeline verification job ( MetricSet
).specs
Specifications associated with the pipeline verifification job ( SpecificationSet
).Methods Summary
deserialize
([measurements, blobs, metrics, …])Deserialize a Verification Framework Job from a JSON serialization. dispatch
([api_user, api_password, api_url])POST the job to SQUASH, LSST Data Management’s metric dashboard. load_metrics_package
([package_name_or_path, …])Create a Job with metrics and specifications pre-loaded from a Verification Framework metrics package, such as verify_metrics. reload_metrics_package
([…])Load a metrics package and add metric and specification definitions to the Job, as well as the collected measurements. report
([name, spec_tags, metric_tags])Create a verification report that lists the pass/fail status of measurements against specifications in this job. write
(filename)Write a JSON serialization to the filesystem. Attributes Documentation
-
measurements
¶ Measurements associated with the pipeline verification job (
MeasurementSet
).
-
specs
¶ Specifications associated with the pipeline verifification job (
SpecificationSet
).
Methods Documentation
-
classmethod
deserialize
(measurements=None, blobs=None, metrics=None, specs=None, meta=None)¶ Deserialize a Verification Framework Job from a JSON serialization.
Parameters: - measurements :
list
, optional List of serialized
Measurement
objects.- blobs :
list
, optional List of serialized
Blob
objects.- metrics :
list
, optional List of serialized
Metric
objects.- specs :
list
, optional List of serialized specification objects.
- meta :
dict
, optional Dictionary of key-value metadata entries.
Returns: Examples
Together,
Job.json
andJob.deserialize
allow a verification job to be serialized and later re-instantiated.>>> import json >>> job = Job() >>> json_str = json.dumps(job.json) >>> json_obj = json.loads(json_str) >>> new_job = Job.deserialize(**json_obj)
- measurements :
-
dispatch
(api_user=None, api_password=None, api_url='https://squash-restful-api.lsst.codes', **kwargs)¶ POST the job to SQUASH, LSST Data Management’s metric dashboard.
Parameters: - api_url :
str
, optional Root URL of the SQUASH API server.
- api_user :
str
, optional API username.
- api_password :
str
, optional API password.
- **kwargs : optional
Additional keyword arguments passed to
lsst.verify.squash.post
.
- api_url :
-
classmethod
load_metrics_package
(package_name_or_path='verify_metrics', subset=None, measurements=None, meta=None)¶ Create a Job with metrics and specifications pre-loaded from a Verification Framework metrics package, such as verify_metrics.
Parameters: - package_name_or_path :
str
, optional Name of an EUPS package that hosts metric and specification definition YAML files or the file path to a metrics package.
'verify_metrics'
is the default package, and is where metrics and specifications are defined for most LSST Science Pipelines packages.- subset :
str
, optional If set, only metrics and specification for this package are loaded. For example, if
subset='validate_drp'
, onlyvalidate_drp
metrics are loaded. This argument is equivalent to theMetricSet.subset
method. Default isNone
.- measurements :
MeasurementSet
orlist
ofMeasurement
s, optional Measurements to report in the Job.
- meta :
dict
, optional Optional dictionary of metadata key-value entries to include in the Job.
Returns: - package_name_or_path :
-
reload_metrics_package
(package_name_or_path='verify_metrics', subset=None)¶ Load a metrics package and add metric and specification definitions to the Job, as well as the collected measurements.
Parameters: - package_name_or_path :
str
, optional Name of an EUPS package that hosts metric and specification definition YAML files or the file path to a metrics package.
'verify_metrics'
is the default package, and is where metrics and specifications are defined for most packages.- subset :
str
, optional If set, only metrics and specification for this package are loaded. For example, if
subset='validate_drp'
, onlyvalidate_drp
metrics are included in theMetricSet
. This argument is equivalent to theMetricSet.subset
method. Default isNone
.
Notes
This method is useful for loading metric and specification definitions into a job that was created without this information. In addition to being added to
Job.metrics
, metrics are also attached toJob.measurements
items. This ensures that measurement values are normalized into the units of the metric definition when a Job is serialized.- package_name_or_path :
-
report
(name=None, spec_tags=None, metric_tags=None)¶ Create a verification report that lists the pass/fail status of measurements against specifications in this job.
In a Jupyter notebook, this report can be shown as an inline table.
Parameters: - name :
str
orlsst.verify.Name
, optional A package or metric name to subset specifications by. When set, only measurement and specification combinations belonging to that package or metric are included in the report.
- spec_tags : sequence of
str
, optional A set of specification tag strings. when given, only specifications that have all the given tags are included in the report. For example,
spec_tags=['LPM-17', 'minimum']
.- metric_tags : sequence of
str
, optional A set of metric tag strings. When given, only specifications belonging to metrics that posess all given tags are included in the report. For example,
metric_tags=['LPM-17', 'photometry']
selects sepifications that have both the'LPM-17'
and'photometry'
tags.
Returns: - report :
lsst.verify.Report
Report instance. In a Jupyter notebook, you can view the report by calling
Report.show
.
See also
Notes
This method uses the
lsst.verify.SpecificationSet.report
API to create thelsst.verify.Report
, automatically inserting theJob
’s measurements and metadata for filtering specifiation tests.In a Jupyter notebook environment, use the
lsst.verify.Report.show
method to view an interactive HTML table.job = lsst.verify.Job() # ... report = job.report() report.show()
- name :
-
write
(filename)¶ Write a JSON serialization to the filesystem.
Parameters: - filename :
str
Name of the JSON file (including directories). This name should be unique among all task executions in a pipeline. The recommended extension is
'.verify.json'
. This convention is used by post-processing tools to discover verification framework outputs.
- filename :
- measurements :