Tickets Addressed in Summer 2019 Release (v18_0_0)

  • DM-1448: Move code for mock images into afw so it reusable. [ip_isr]
  • DM-2579: Calling AliasMap::get(“”) can return incorrect results [afw]
  • DM-2891: meas.algorithms.utils uses measurement algorithms that are no longer available [meas_algorithms]
  • DM-2944: SourceMeasurementTask still referenced in our stack [ip_diffim, meas_algorithms, meas_deblender, pipe_tasks]
  • DM-3655: LSST_CFG_PATH support broken because of recent sconsUtils change [sconsUtils]
  • DM-10242: Stop using astrometry_net by default [obs_cfht, obs_ctio0m9, obs_decam, obs_lsst, obs_subaru, pipe_tasks]
  • DM-10800: Merge matcherSourceSelector and matcherPessimisticSourceSelector [meas_algorithms, meas_astrom, obs_ctio0m9]
  • DM-10826: fix jointcal collectRefStar handling of position errors [jointcal, obs_decam, obs_lsst, obs_subaru]
  • DM-11397: Remove twinkles1 jointcal testdata and tests [jointcal, testdata_jointcal]
  • DM-12384: exit/raise when data is less than parameters [jointcal]
  • DM-13053: allow refcat selection improve fitting [jointcal]
  • DM-13180: Specifications without corresponding metric in verify_metrics [validate_drp, verify_metrics]
  • DM-13432: Remove all explicit imports of ds9 [afw, ip_diffim, ip_isr, meas_algorithms, meas_astrom, meas_base, meas_extensions_astrometryNet, meas_extensions_photometryKron, meas_extensions_psfex, obs_cfht, obs_subaru, pipe_tasks, synpipe]
  • DM-13567: Add tests to ap_pipe [ap_pipe]
  • DM-13757: enable jointcal config writing [jointcal, obs_base]
  • DM-14459: Add check to (Posix)Datastore that prevents silent overwrite [daf_butler]
  • DM-15004: Upgrading display_ds9 to numpydoc status [display_ds9]
  • DM-15374: Per dataset allow/deny lists for datastores [daf_butler]
  • DM-15450: Update deprecated use of time.clock() [pipe_base]
  • DM-15514: Write insertFakeObjectsTask to add fake sources for QA purposes. [meas_base, obs_base, obs_subaru, pipe_tasks]
  • DM-15683: Add tests for ip_isr/isrTask [ip_isr]
  • DM-16268: Rewrite meas_extensions_ngmix for better functionality and upstream collaboration [obs_base]
  • DM-16292: Calibration products need more metadata [astro_metadata_translator, pipe_drivers]
  • DM-16429: Ensure WCS (and other Exposure components) are retrieved properly when loaded individually [obs_base]
  • DM-16501: Propagate calexp_fakes through skyCorrection and coaddDriver [pipe_drivers]
  • DM-16536: Migrate all metrics from ap.verify.measurements [ap_association, ap_verify, ip_diffim, verify, verify_metrics]
  • DM-16539: Add level of indirection betwen calib identifier and exposure range in schema [ctrl_mpexec, daf_butler, ip_isr, obs_subaru, pipe_base, pipe_tasks]
  • DM-16606: ap_pipe should not create DB automatically [ap_pipe, ap_verify, dax_ppdb]
  • DM-16699: Initial atmospec development work [obs_base, obs_lsst]
  • DM-16702: Add reference stars to fgcmcal fit as an option [fgcm, fgcmcal, obs_base]
  • DM-16736: Let MetricsControllerTask store application-specific metadata [verify]
  • DM-16859: Convert afw.fits to numpydoc [afw]
  • DM-16925: Provide backwards-compatibility with Calib API [afw]
  • DM-17004: JointcalRunner.__call__ not receiving “butler” in kwargs [jointcal]
  • DM-17025: Improve filename template mechanisms in PosixDatastore and Butler [daf_butler]
  • DM-17029: Update LoadReferenceObjectsTask to output fluxes in nanojansky [jointcal, meas_algorithms, meas_extensions_astrometryNet, pipe_tasks, testdata_jointcal]
  • DM-17074: Bad pixel masking log message unneeded when nothing masked [ip_isr]
  • DM-17248: ap_verify should create .json files in the workspace by default [ap_verify]
  • DM-17413: Investigate performance of background tweak during dynamic detection [meas_algorithms]
  • DM-17426: Improve full-visit sky subtraction [obs_subaru, pipe_drivers]
  • DM-17427: Incorporate Goulding bright star masks [pipe_tasks]
  • DM-17428: Allow rotation before interpolating on coadd [obs_subaru, pipe_tasks]
  • DM-17429: PSFs on coadds are narrower than in model [obs_subaru]
  • DM-17431: Peak culling is removing real sources [afw]
  • DM-17521: Add warning when ap_pipe skips association [ap_pipe]
  • DM-17522: Write getting started notes on ip_diffim [ip_diffim, pipe_tasks]
  • DM-17528: Quantify impact of variable seeing on current DCR correction [ip_diffim]
  • DM-17562: Broken links in afw doxygen [afw]
  • DM-17654: PPDB Scaling Test in Google Cloud [dax_ppdb]
  • DM-17663: Make Registry table names lowercase [daf_butler, ip_isr, meas_base, obs_base, obs_subaru, pipe_tasks, skymap]
  • DM-17757: Key error when trying to use pipetask to output pipeline/quantum graph and then read it. [meas_base, pex_config]
  • DM-17830: Investigate color-dependent offsets from ref cat in jointcal vs. meas_mosaic [jointcal]
  • DM-17845: Fix warnings in image coaddition [pipe_tasks]
  • DM-17866: Support BOT data gen 2 ingest once data exists [obs_lsst]
  • DM-17875: DESC config updates for DC2 stars only refererence catalog [obs_lsst]
  • DM-17912: gen3 ip_isr can attempt to remove a non-existant dataset, causing KeyError [ip_isr]
  • DM-17932: Validate object type when reading pipeline or graph from pickle [ctrl_mpexec]
  • DM-17950: Make Schema picklable [afw]
  • DM-17956: Add option to output chi2 files at each outer fit iteration [jointcal]
  • DM-17981: Create heterogenous map [afw]
  • DM-18022: meas_modelfit test_truncatedGaussian.py failure using bleed environment [meas_modelfit]
  • DM-18051: Get defect machinery working for the AuxTel [obs_base, obs_lsst]
  • DM-18065: Bad logic in saturation interpolation config options [ip_isr]
  • DM-18067: Add fluxMag0 PhotoCalib factory function [afw]
  • DM-18068: Write pipe_analysis parquet tables as butler datasets [obs_base]
  • DM-18167: display_firefly needs to handle viewer_ids properly [display_firefly]
  • DM-18174: constructFlat fails when image is identically zero [obs_subaru, pipe_drivers]
  • DM-18181: Provide tool to validate datastore template configurations [daf_butler]
  • DM-18189: Investigate mapping of ip_diffim dipole fields to DPDD [ap_association, dax_ppdb]
  • DM-18196: Fix afw schema missmatch between ap_association and dax_ppdb [ap_association, dax_ppdb]
  • DM-18203: numpy unicode warnings in readTextCatalogTask.py [meas_algorithms]
  • DM-18293: Reduce FitTanSipWcsTask default order to 2 [meas_astrom, obs_ctio0m9, obs_decam, obs_lsst]
  • DM-18314: Namespace confusion prevents use of MatchOptimisticB with singleFrameDriver [meas_astrom]
  • DM-18318: Create initial subset of timeseries features for DIAObject [ap_association]
  • DM-18327: Upgrade third-party coord package to version 1.1 [coord]
  • DM-18329: Add jointcal templates to obs_lsst [obs_lsst]
  • DM-18352: update allocateNodes.py with ability to start HTCondor with partitionable slots [ctrl_execute, ctrl_platform_lsstvc]
  • DM-18354: afw test fails due to assertEqual on float [afw]
  • DM-18356: Create Gen3 Butler Templates for ci_hsc pipeline [daf_butler]
  • DM-18367: Get butler working with Oracle [ctrl_mpexec, daf_butler]
  • DM-18372: w_2019_10 pipelines.lsst.io build failure (matchOptimisticB task refactoring) [meas_astrom]
  • DM-18387: Write a task to make a visit->coadd match table [obs_base]
  • DM-18393: Add some needed features for testing at SLAC [afw, obs_base, obs_lsst]
  • DM-18394: Crash when running ap_pipe on calexp templates [pipe_tasks]
  • DM-18400: Number of matches is smaller than request [meas_astrom]
  • DM-18403: CoaddPsf.getWcs(i) segfaults on i=getComponentCount() [meas_algorithms]
  • DM-18438: Specify string lengths in PosixDatastoreRecords table [daf_butler]
  • DM-18488: Update pyyaml to v5.x [daf_base, daf_persistence, pyyaml]
  • DM-18490: Move TimingMetricTask to verify [ap_verify, verify]
  • DM-18531: convert skyCorrection to gen3 pipelineTask [pipe_drivers]
  • DM-18544: Add python-level deprecation warnings for Calib interface [afw]
  • DM-18551: update ccdExposureId_bits in obs_lsst for imSim and phoSim [obs_lsst]
  • DM-18552: Support for new brighter-fatter kernels reverted [ip_isr]
  • DM-18556: Adapt visit and coadd qa analysis scripts to run on DESC DC2 outputs [obs_lsst]
  • DM-18565: Fix artifacts at patch boundaries in DCR templates [ip_diffim]
  • DM-18575: Update imsim and phosim configs for star+galaxy reference catalogs [obs_lsst]
  • DM-18576: Issue warnings if translator methods are shadowed [astro_metadata_translator]
  • DM-18577: Fix PosixDataStore naming [daf_butler, obs_subaru]
  • DM-18624: speed up Table pickling [afw]
  • DM-18638: Check for at least 2 visits in AMx metric calculation. [validate_drp]
  • DM-18643: Move AstrometryTask source selection from “matcher” into AstrometryTask [meas_astrom, obs_cfht, obs_ctio0m9, obs_lsst, obs_subaru]
  • DM-18678: Update to boost 1.69 [afw, boost, meas_base, meas_modelfit]
  • DM-18703: constructFlat.py –config isr.doCrosstalkBeforeAssemble=False raises a LengthError exception [ip_isr]
  • DM-18708: Investigate ingest problems with BOT data [astro_metadata_translator, obs_lsst]
  • DM-18735: Convert dax_ppdb to use Pandas data frames (rather than afw::table) as an interface [dax_ppdb]
  • DM-18742: Speed up DcrModel convergence [ip_diffim, pipe_tasks]
  • DM-18748: Missing ExpTime in auxTel teststand images cause ingestImages.py to abort [astro_metadata_translator, obs_lsst]
  • DM-18754: macOS Jenkins nodes need to change default matplotlib backend [validate_drp]
  • DM-18838: Fix butlerRoot in OracleRegistry [daf_butler]
  • DM-18839: Remove explicit registry close in the butler [ctrl_mpexec, daf_butler]
  • DM-18855: Pex exceptions TypeError should not inherit from RuntimeError [pex_exceptions]
  • DM-18863: Investigate and fix warnings in daf_butler [daf_butler]
  • DM-18864: Update afw to support undefined values in FITS headers [afw]
  • DM-18885: Log number of MeasuredStars in Associations [jointcal]
  • DM-18895: Use std::ptrdiff_t as index type in jointcal Eigen objects [eigen, jointcal, jointcal_cholmod]
  • DM-18905: Use safe YAML loading in dax_ppdb [dax_ppdb]
  • DM-18906: Use safe YAML loading in verify [verify]
  • DM-18908: Update baseline versions in EUPS stub packages for new conda env [astropy, matplotlib, numpy, python, scipy]
  • DM-18914: Jenkins docs build failure [meas_base]
  • DM-19015: HSC warp making is broken with doApplyUberCal=True [meas_mosaic]
  • DM-19075: jointcal is ignoring writeChi2FilesOuterLoop [jointcal]
  • DM-19188: Add system for reading header corrections from external files [astro_metadata_translator, obs_lsst]
  • DM-19236: Remove errant cout when reading old Calibs [afw]
  • DM-19242: Recent changes to afw compression broke macos builds [afw]
  • DM-19256: Continue cameraGeom work to support BOT@SLAC [obs_lsst]
  • DM-19265: Jacobian lost in meas_mosaic photometric solution [meas_mosaic]
  • DM-19270: Set configs and lsstCamMapper.yaml for skyCorrection [obs_lsst]
  • DM-19271: Add transactions to gen3 skymap registry [skymap]
  • DM-19272: Make script for creating skymaps in gen3 bulter [pipe_tasks, skymap]
  • DM-19277: Update HSC defects again [obs_subaru]
  • DM-19290: Create an lsst.utils.deprecated that works with pybind11 [utils]
  • DM-19293: Clean up photoCalib examples in docstrings [afw]
  • DM-19300: Ignore ZQUANTIZ header keyword [afw]
  • DM-19303: Disable writing postISRCCD [obs_subaru]
  • DM-19304: Astrometric metric regression on CFHT [meas_astrom]
  • DM-19310: Update the raft definitions for TS8 camera test stand [obs_lsst]
  • DM-19311: Please update to pybind11 2.2.4 [pybind11]
  • DM-19320: Restructure obs_subaru HSC configs for Gen3 [obs_subaru]
  • DM-19331: Fix detector name for HSC [astro_metadata_translator]
  • DM-19366: afw unit tests do not run on some platforms [afw]
  • DM-19371: cameraGeom.showCamera applies nQuarter to processed images [afw]
  • DM-19372: Produce warning message when QuantumGraph is empty [ctrl_mpexec]
  • DM-19373: Add defect list support to butler gen 3 [daf_butler, ip_isr, meas_algorithms, obs_cfht, obs_subaru, pipe_tasks]
  • DM-19382: Refactor and reorder ISR steps to support writing pre-interpolated pixels [ip_isr, obs_base, obs_decam, obs_lsst, obs_subaru, pipe_tasks]
  • DM-19383: Fix BOT translator when DARKTIME is present [obs_lsst]
  • DM-19387: Vectorize dimension entry addition to gen3 registry [daf_butler]
  • DM-19388: Investigate timing of Pandas dax_ppdb interface [dax_ppdb]
  • DM-19393: Fix HSC y stray-light lookup In Gen2 [ip_isr, obs_subaru]
  • DM-19404: Make obs_cfht dependency in jointcal optional [jointcal]
  • DM-19409: Add getCenter to Box2I [geom]
  • DM-19412: Bad CCD rotations in visualizeVisit.py [pipe_drivers]
  • DM-19428: Add eups python as a dependency of pybind11 [pybind11]
  • DM-19452: jointcal should not depend on meas_extensions_astrometryNet [jointcal]
  • DM-19456: fix docstring for PhotoCalib.calibrateCatalog [afw]
  • DM-19458: FieldValidationError usage incorrect in many tasks [meas_algorithms, meas_extensions_psfex, pex_config]
  • DM-19461: Investigate and possibly fix afw::table Record allocation performance` [afw]
  • DM-19467: Add C++ iteration to GenericMap [afw]
  • DM-19485: Fix length of instrument name in gen 3 butler schema [daf_butler]
  • DM-19506: Adjust defect FITS files to be compatible with DS9 [meas_algorithms]
  • DM-19517: Calculate the PSF for DCR coadds [pipe_tasks]
  • DM-19531: Add option to include regions when ingesting gen3 images [obs_base]
  • DM-19535: Move MakeBrighterFatterKernelTaskRunner to cp_pipe/pairedVisitTaskRunner [cp_pipe]
  • DM-19568: Backwards-compatibility for maskPixelsFromDefectList is broken [ip_isr]
  • DM-19573: Allow color outputs from compilers under scons [sconsUtils]
  • DM-19575: Add Storable mixin to ExposureInfo components [afw]
  • DM-19583: Investigate butler gen 3 configuration system [daf_butler]
  • DM-19585: Fix length of instrument name in gen 3 butler schema (see also DM-19485) [daf_butler]
  • DM-19598: Remove lsst.ap.verify.measurements automodapi statement [ap_verify]
  • DM-19614: Write transmission curves in writeCuratedCalibrations [daf_butler, obs_subaru]
  • DM-19615: Change raw storage class in isr task [daf_butler, ip_isr]
  • DM-19622: Make PosixDatastore’s internal table lowercase [daf_butler]
  • DM-19623: Change sqlalchemy syntax in addDimensionEntryList [daf_butler]
  • DM-19627: Add text file serialization to meas_algorithms Defects class [meas_algorithms]
  • DM-19638: Create parent task/script for bootstrapping Gen3 repos [daf_butler, log, obs_base, obs_subaru]
  • DM-19641: Use jointcal instead of meas_mosaic in obs_subaru HSC coaddition [obs_subaru]
  • DM-19660: Bug? The PSF is not warped during image differencing [afw, ip_diffim]
  • DM-19664: Set OBSTYPE rather than appending it [pipe_drivers]
  • DM-19670: Writing missing docstrings and docstring updates in ip_diffim [ip_diffim]
  • DM-19671: setConfigRoot sometimes needs to not update the root [daf_butler]
  • DM-19674: Reading fringe exposure can modify a WCS in another exposure [astshim]
  • DM-19677: Disable writing postISRCCDs in ProcessCcdTask [obs_decam, obs_subaru, pipe_tasks]
  • DM-19682: Fix DAYOBS calculation and allow for gen2 header correction for ingest [astro_metadata_translator, obs_lsst]
  • DM-19693: Generate correction files to patch up DAYOBS from 20190308_000139 to 20190329_000022 [obs_lsst]
  • DM-19694: Make Defects presize internal tables. [meas_algorithms]
  • DM-19710: Add astro_metadata_translator corrections for older HSC data [astro_metadata_translator]
  • DM-19717: Add setup_module to ip_isr unit tests. [ip_isr]
  • DM-19720: Change multiband.py to support meas_extensions_scarlet [pipe_drivers, pipe_tasks]
  • DM-19732: remnant ip_isr debug statement left in [ip_isr]
  • DM-19752: monowl string parse in gen2 butler ingest of ts8 data [obs_lsst]
  • DM-19755: Add a check in CameraMapper for duplicate mappings [obs_base]
  • DM-19766: DECam instcals fail to process with invalid DateTime [astro_metadata_translator, obs_subaru]
  • DM-19768: Fix jointcal handling of coordinate errors [jointcal]
  • DM-19769: Fix CmdLineTask –longlog time formatting [ap_pipe, ctrl_mpexec, pipe_base]
  • DM-19797: Error from loadReferenceObjects with a Gen3 HSC-RC2 repo [meas_algorithms]
  • DM-19802: Fix jointcal ra/dec bounding box calculations [jointcal]
  • DM-19808: Avoid duplication in deferred prelight follow-up queries [daf_butler]
  • DM-19818: Fix Gen3 Butler pickling broken on DM-19638 [daf_butler]
  • DM-19871: Fix validity range end in Gen3 calibration bootstrapping [daf_butler, obs_base]
  • DM-19873: Implement PropertySet.getitem and return get() [daf_base]
  • DM-19877: Replace PropertySet.get with getScalar or getArray [obs_lsst, pipe_drivers, pipe_tasks]
  • DM-19884: Add configs to __all__ in isrTask.py [ip_isr]
  • DM-19887: Switch skymap to geom rather than afwGeom [skymap]
  • DM-19891: Fix __eq__ for defects class [meas_algorithms]
  • DM-19902: String representation of Observatory coordinates flips lat/lon [afw]
  • DM-19907: Don’t pad numerator with zeros in smoothArray during SkyCorrection [pipe_drivers]
  • DM-19916: Investigate URI inconsistencies in daf_butler LocationFactory [daf_butler]
  • DM-19958: Missing _mapper in Gen2 Butler repo gives confusing error [pipe_base]
  • DM-19971: Fixup for tickets/DM-19272 [pipe_tasks]
  • DM-20008: AuxTel translator in obs_lsst needs TSTART adjusted [obs_lsst]
  • DM-20019: Fix pickling of String Fields [afw]
  • DM-20024: BackgroundList.readFits doesn’t close fits files [afw]