Characterization Metric Report¶
Starting from Summer 2015, administrative (“cycle”) releases are accompanied by a measurements report characterizing the current performance. Metrics included in these reports are expected to increase in number and sophistication at subsequent releases. This brief report describe measurements of interest that were carried out for the Summer 2015 cycle. In Winter 16 we focussed on moving towards the infrastructure to calculate and monitor these automatically so we can provide more extensive updates; therefore there are no changes from last cycle’s values.
Summary of Photometric Repeatability Measurements¶
Submitted by Jim Bosch
This dataset is a selection of i-band HyperSuprime-Cam engineering data taken in the SDSS Stripe 82 region. This dataset consists of 30s exposures, so it is somewhat similar to projected LSST data in depth. Our current calibration approach has many limitations relative to what we ultimately plan to implement for LSST:
- There’s currently no relative calibration being run at all.
- We have only limited correction for chromatic effects.
- There’s currently no allowance for zeropoint variations smaller than the scale of a CCD.
- We also use a much simpler sample selection than that proposed by the SRD.
A Jupyter notebook to compute the metrics can be found at https://github.com/lsst/afw/blob/tickets/DM-3896/examples/repeatability.ipynb.
| Metric Characterized | Metric Ref | Target | Measured Value | Measurement Method |
|---|---|---|---|---|
| Photometric repeatability (procCalRep) | DLP-307 | \(\leq 13\) mmag | 10.6 mmag | DM-3338 (i band) |
| Photometric repeatability (PA1gri) | DLP-315 | \(\leq 13\) mmag | 10.6 mmag | DM-3338 |
| Photometric repeatability (PA1uzy) | DLP-316 | \(\leq 13\) mmag | 10.6 mmag | DM-3338 (i band) |
Summary of Algorithmic Performance Measurements¶
Submitted by John Swinbank
The i-band HSC engineering data (described above) was used where possible and the same caveats apply. Consult the tickets in the Measurement Method column for more details.
| Metric Characterized | Metric Ref | Target | Measured Value | Measurement |
|---|---|---|---|---|
| Residual PSF Ellipticity Correlations (TE1) | DLP-290 | \(\leq 5\times 10^{-3}\) | \(6\times 10^{-5}\) | DM-3040 |
| Residual PSF Ellipticity Correlations (TE2) | DLP-290 | \(\leq 5\times 10^{-3}\) | \(2\times 10^{-5}\) | DM-3047 |
| Relative Astrometry (AM1) | DLP-310 | \(< 60\) mas | 12.49 mas | DM-3057 |
| Relative Astrometry (AM2) | DLP-311 | \(< 60\) mas | 12.19 mas | DM-3064 |
Summary of Computational Performance Measurements¶
Submitted by John Swinbank and Simon Krughoff
At this point of Construction, the computational performance measurements are a combination of precursor data processing and extrapolation from R&D assumptions.
DECam/HITS data was used for the OTT1 estimate and for the diffim and single-frame measurement of the Alert Production Computational Budget in combination with data from the 3rd Data Challenge.
For the Data Release Production of the computational budget, we used DECam/HITS data for estimating diffim performance, HSC-I for assembling and measuring coadds and for forced measurement, estimates from FDR for multifit, and data from the 3rd Data Challenge for SDQA. Calculations for the DRP computational budget used this iPython notebook.
| Metric Characterized | Metric Ref | Target | Measured Value | Measurement Method |
|---|---|---|---|---|
| OTT1 | DLP-328 | \(\leq 240\) sec | 200-250 sec | DM-3724 |
| AP Computational Budget | DLP-329 | \(\leq 231\) TFLOPS | 34-39 TFLOPS | DM-3267 |
| DRP Computational Budget | DLP-314 | \(\leq 645\) TFLOPS | 318 TFLOPS | DM-3083 |