Tickets Addressed in Release 19.0.0

  • DM-3349: Add test case for ExposureRecord::contains [afw]

  • DM-5081: Add mask plane to indicate no brighter fatter correction around edges [ip_isr]

  • DM-5635: not flagged NaN in sdssCentroid [meas_base]

  • DM-7109: remove install_name_tool hack from boost’s eups.pkg.cfg.sh [boost]

  • DM-9457: test failure due to esutil/numpy problem [esutil]

  • DM-10333: duplicate keys in obs_base/policy/datasets.yaml [obs_base]

  • DM-12594: Remove daf::base::Citizen [afw, daf_base, daf_persistence, ip_diffim, meas_algorithms, meas_astrom, meas_base, pex_policy, utils]

  • DM-13361: Minimal S3 backed Datastore [daf_butler]

  • DM-16179: Change log level to WARN for the footprint skipping [meas_deblender]

  • DM-16297: Add Gen3 Instrument and Formatter classes for obs_lsst [astro_metadata_translator, daf_butler, obs_base, obs_lsst]

  • DM-16490: Update fgcmcal to use makeSkyWcs from camera geometry for sky/focal plane transforms [fgcmcal]

  • DM-16557: Move ap_verify_queries function into Ppdb [ap_association, ap_pipe, dax_ppdb]

  • DM-16704: Add fgcmcal-tract mode to fgcmcal for small-scale processing with a reference catalog [fgcm, fgcmcal, obs_base, obs_subaru]

  • DM-16863: Convert afw.table to numpydoc [afw]

  • DM-17023: Refactor the Dimensions and query system [ctrl_mpexec, daf_butler, meas_base, obs_base, obs_cfht, obs_ctio0m9, obs_decam, obs_lsst, obs_subaru, pipe_base, pipe_tasks, skymap]

  • DM-17205: pytest runs extra tests in lsst.verify [verify]

  • DM-17670: Update scarlet notebooks and docs [scarlet]

  • DM-17847: Update defaults in scarlet and create changelog [scarlet]

  • DM-17982: Reimplement ExposureInfo using GenericMap [afw, meas_extensions_convolved]

  • DM-18036: Convert stack demo refcat to HTM indexed [lsst_dm_stack_demo, meas_algorithms]

  • DM-18120: Remove –silent command-line argument from ap_verify.py [ap_verify]

  • DM-18134: Get atmospec pipeline working with ctio0m9 data [obs_ctio0m9]

  • DM-18151: Make RA, dec plots for fake sources [obs_subaru]

  • DM-18494: Mimic the meas_base plugin system for use in ap_association, DiaObject summary metrics [ap_association]

  • DM-18601: Investigate newer SQLAlchemy versions [sqlalchemy]

  • DM-18610: Add fields, limited mutability, and trim/assembly-state tracking to cameraGeom [afw, cp_pipe, fgcmcal, geom, ip_isr, meas_base, obs_base, obs_cfht, obs_ctio0m9, obs_lsst, obs_lsstSim, obs_sdss, obs_test, pipe_tasks]

  • DM-18666: Add support for authorization headers to firefly_client [firefly_client]

  • DM-18667: Add support to display_firefly for obtaining and passing along an authorization token for Firefly [display_firefly]

  • DM-18736: Convert ap_association to use Pandas data frames (rather than afw::table) as an interface [ap_association, ap_pipe]

  • DM-18739: Work on obs_subaru [daf_butler, obs_base, obs_lsstSim, obs_subaru, obs_test, pipe_tasks, testdata_subaru]

  • DM-18751: Use 1/(n-1) definition of std in AMx calculation [validate_drp]

  • DM-19207: Remove deprecated Calib interfaces after next release [afw, jointcal, meas_modelfit, pipe_tasks]

  • DM-19214: ip_isr crosstalk shouldn’t depend on detector hasCrosstalk( )and getCrosstalk() [ip_isr, obs_subaru]

  • DM-19329: Update obs_lsst to support ComCam [obs_lsst]

  • DM-19443: Extract and visualize the local and the spatial AL kernel solution coefficients [ip_diffim]

  • DM-19556: Use TemplateMeta with GenericMap [afw]

  • DM-19582: Add cross-language GenericMap unit tests [afw]

  • DM-19616: Make IngestIndexReferenceObjectsTask multiprocessing capable [esutil, meas_algorithms, meas_astrom]

  • DM-19672: Use smaller-scale sky subtraction for HSC release [obs_subaru]

  • DM-19709: Write a first version of a defect finding task [cp_pipe, obs_base, obs_lsst, obs_test]

  • DM-19727: Propagate Parallax in refcat Production [meas_algorithms]

  • DM-19728: Convert obs_lsst defects to new form [obs_lsst]

  • DM-19729: Convert obs_cfht defects to new form [obs_cfht]

  • DM-19730: Convert obs_decam defects to new form [obs_decam, testdata_decam]

  • DM-19830: newinstall.sh needs updates for miniconda 4.6 [lsst]

  • DM-19837: Support the eimage of PhoSim in Scientific Pipeline [obs_lsst]

  • DM-19839: Fix bug in recent DcrCoadd PSF calculation [pipe_tasks]

  • DM-19841: Look into unmasked defects on edge of bad amps leaking into coadds [obs_subaru]

  • DM-19857: Update ap_verify to use new DECam defect ingestion [ap_verify]

  • DM-19874: Add support for decam illumcor calibration products [ip_isr, obs_decam, pipe_tasks]

  • DM-19903: Add columnar defect identification to findDefectsTask in cp_pipe please [obs_lsst]

  • DM-19906: Convert diaForcedSourceTask to take pandas DataFrame as input [ap_association, ap_pipe]

  • DM-19961: Add region padding to HSC config for Gen3 raw ingest [obs_subaru]

  • DM-19978: Fix bug in DcrModel coordinates [ip_diffim]

  • DM-19985: forcedPhotCcd_metadata template missing [obs_lsst]

  • DM-19988: fewer reference catalogs for a ccd if a tract is specified in making quantum graph [ctrl_mpexec, daf_butler, ip_isr, pipe_base]

  • DM-19989: Document how to generate a refcat [meas_algorithms]

  • DM-19991: Apply changes from deblending sprint to master [scarlet]

  • DM-20030: Update docs for scarlet v0.5 [scarlet]

  • DM-20046: Cleanup docstrings in detection.py [meas_algorithms]

  • DM-20054: Normalize dimensions in DatasetType and config mappings up-front [ctrl_mpexec, daf_butler, obs_base, obs_subaru, pipe_base, pipe_tasks]

  • DM-20069: Finishing writing the basic version of the PTC task [cp_pipe, obs_base, obs_lsst]

  • DM-20073: Add ability to construct ObservationInfo from kwargs (or allow properties to be changed) [astro_metadata_translator, obs_base]

  • DM-20109: Improve ip_isr log messages to be more explicit [ip_isr]

  • DM-20112: ip_isr is not handling BAD pixels as expected [ip_isr]

  • DM-20128: Create unit tests for scarlet [proxmin, scarlet]

  • DM-20138: Further expand masks on HSC [obs_subaru]

  • DM-20142: Write DMTN on Exposure and persistence work [afw]

  • DM-20143: Improve handling of blank keyword comments in FITS headers [afw]

  • DM-20154: Implement new initial WCS design [ip_isr, obs_base, obs_decam, obs_lsstSim, obs_subaru]

  • DM-20169: Enable header fixups for decam and cfht [astro_metadata_translator, daf_butler, obs_base, obs_decam]

  • DM-20189: Investigate change in processing time when using pandas in ap_association [ap_association]

  • DM-20201: Fix DECam rotator type and angle in metadata translator [astro_metadata_translator, obs_decam]

  • DM-20202: Fix CFHT MegaPrime rotator type and angle in metadata translator [astro_metadata_translator, obs_cfht]

  • DM-20205: Refactor PipelineTask interface [ctrl_mpexec, daf_butler, pex_config, pipe_base]

  • DM-20241: Fix typo in lsst.meas.algorithms.utils [meas_algorithms]

  • DM-20242: Megacam coaddExposureId calculation is not compatible with large sky maps [obs_cfht]

  • DM-20244: Gen3 ISR task broken wanting dataset illum [ip_isr]

  • DM-20246: lsst.meas.algorithms.Defects seems to be causing a segmentation fault in very specific situations. [afw, meas_algorithms]

  • DM-20251: Improve documentation for how to register metrics with ap_verify [ap_verify]

  • DM-20286: Modernize pybind11 wrappers for afw.table and afw.table.io [afw, meas_base]

  • DM-20287: Fix logging in IsrTask [ip_isr]

  • DM-20317: DCR model performance improvements [ip_diffim, pipe_tasks]

  • DM-20325: Add option to get password securely from dispatch_verify.py [verify]

  • DM-20342: Fix typo in eImageIsr [obs_lsstSim]

  • DM-20347: Change name of processFakes to match the task name processCcdWithFakes [obs_base, pipe_tasks]

  • DM-20371: Create new shift/rot only WCS fitter [meas_astrom]

  • DM-20373: Move FitsRawFormatterBase from daf_butler into obs_base [daf_butler, obs_base, obs_subaru]

  • DM-20378: System for deprecating Config fields [pex_config]

  • DM-20384: Clarify usage of make_ppdb.py script [ap_pipe]

  • DM-20431: Re-Implement symmetry in scarlet [scarlet]

  • DM-20440: Update sample footprints notebook in display_firefly [display_firefly]

  • DM-20441: catch-all exceptions in gen3 butler should use raise from [daf_butler]

  • DM-20472: Empty matches in measureCoaddSources results in unnecessary error [pipe_tasks]

  • DM-20478: ISR fails processing QA for detectors with no overscan [ip_isr]

  • DM-20499: Add basic stringification to SkyWcs [afw]

  • DM-20506: Allow PropertySet to handle unsigned 64-bit integers [daf_base]

  • DM-20524: Add deepDiff_diaObject dataset types and correct deepDiff_diaSrc templates [obs_lsst]

  • DM-20540: Remove long deprecated methods from utils package [utils]

  • DM-20546: Cleanup some afw deprecations [afw, obs_sdss, pipe_drivers]

  • DM-20548: Update HSC’s camera geometry to use units of millimeters [fgcmcal, obs_subaru, pipe_drivers]

  • DM-20554: Log or store if ip_diffim running in deconvolution case [ip_diffim]

  • DM-20566: Remove aliased afwGeom geom usage from stack [coadd_utils, daf_butler, display_firefly, fgcmcal, ip_diffim, ip_isr, jointcal, meas_astrom, meas_deblender, meas_extensions_convolved, meas_extensions_photometryKron, meas_extensions_psfex, meas_extensions_shapeHSM, meas_extensions_simpleShape, meas_modelfit, obs_base, obs_cfht, obs_lsstSim, obs_sdss, obs_subaru, obs_test, pipe_drivers, pipe_tasks, shapelet, skymap]

  • DM-20570: Pipeline failure treated as ap_verify success [ap_verify, pipe_base]

  • DM-20574: Remove deprecated ip_isr functions from isrFunctions.py [ip_isr, obs_lsstSim, obs_test]

  • DM-20575: Remove meas_algorithms deprecated functions from defects.py [meas_algorithms]

  • DM-20585: Add meas_extensions_scarlet to lsst_distrib [lsst_distrib]

  • DM-20659: Resolve d_2019_07_17 pipelines.lsst.io build failure [afw]

  • DM-20692: Convert PipelineTasks to new api [ap_association, ap_verify, ip_diffim, ip_isr, meas_algorithms, meas_base, pipe_drivers, pipe_tasks, utils, verify]

  • DM-20702: Create memory usage metric [ap_verify, verify, verify_metrics]

  • DM-20705: Track down Gen3 processing failures in MergeMeasurementsTask [pipe_tasks]

  • DM-20742: Add “raw_header_wcs” datasetType to butler [obs_base, obs_test]

  • DM-20758: Improve error message of the quantum executor [ctrl_mpexec]

  • DM-20760: Drop DiaSources if RA/DEC is NaN and throw warning. [ap_association]

  • DM-20763: Add initial support for Gen3 Butler to obs_decam [daf_butler, jointcal, meas_algorithms, obs_base, obs_decam, obs_subaru]

  • DM-20768: Create script documentation pages for lsst.verify [verify]

  • DM-20771: IsrTaskConfig.numEdgeSuspect field duplicated [ip_isr]

  • DM-20775: Modify compareWarp default configs fix clipping stars (when one visit has bad seeing) [pipe_tasks]

  • DM-20812: Rename TablePersistable storage classes in gen 3 to drop TablePersistable [daf_butler, ip_isr, obs_subaru, pipe_drivers, pipe_tasks]

  • DM-20836: Extend Gaia reference catalog creation to include flux errors [meas_algorithms]

  • DM-20842: Move FileDescriptor to a property in Formatter [daf_butler, obs_base, obs_subaru]

  • DM-20845: Support re-run of pipetask on the same output collection [ctrl_mpexec]

  • DM-20902: FractionUpdatedDiaObjectsMetricTask should expect 0 DIAObjects [ap_association]

  • DM-20915: Add stringification to Formatter [daf_butler, obs_subaru]

  • DM-20924: Release astro_metadata_translator to PyPI [astro_metadata_translator]

  • DM-20929: Fix bug in DcrAssembleCoadd when running with slurm [pipe_tasks]

  • DM-20937: Add deprecation warnings for to-be-removed distorted WCS functions [afw]

  • DM-20941: Remove python_future dependency from lsst.verify [verify]

  • DM-20953: Add support for range selection to gen3 expression parser [daf_butler]

  • DM-20957: Fix checks for lsst.verify’s squash client with responses [verify]

  • DM-20963: inspect_job.py broken [verify]

  • DM-20974: Remove aggregation support/requirement from MetricTask [ap_association, ip_diffim, verify]

  • DM-20978: Stub out base lsstimport and move lsstcppimport import to afw [afw, base, sconsUtils]

  • DM-20992: Add database authentication code to daf_butler [daf_butler]

  • DM-20993: Implement translators for SDSS [astro_metadata_translator]

  • DM-21003: pex_exceptions wrapper exceptions should only inherit from one kind of python exception [pex_exceptions]

  • DM-21008: Config class missing schemeless URI support in some places. [daf_butler]

  • DM-21009: Refactor S3 and Posix datastores to remove code duplication [daf_butler]

  • DM-21012: Fix meas_modelfit’s CModel display [meas_modelfit]

  • DM-21013: Add description for daf_butler expression parser [daf_butler]

  • DM-21015: S3 tests should not require external fake credentials [daf_butler]

  • DM-21024: Fix pipelines.lsst.io build issue from d_2019_08_20 [daf_butler]

  • DM-21043: Qserv log diet: use named context for query ID [log]

  • DM-21048: butler.get(‘raw’,…) raises a lsst::pex::exceptions::NotFoundError for BOT data at lsst-dev [obs_base]

  • DM-21053: Officially deprecate meas_mosaic [lsst_distrib]

  • DM-21055: Do not fill output collection if QuantumGraph is empty. [ctrl_mpexec, pipe_base]

  • DM-21064: Prepare ground for possible change of license in pex_config [pex_config]

  • DM-21065: Replace pexConfig.makePolicy with pexConfig.makePropertySet [ip_diffim, meas_algorithms, pex_config, pipe_tasks]

  • DM-21078: Fix pipelines.lsst.io build issue from d_2019_08_23 [obs_base]

  • DM-21098: Isolate outdated Gen 3 methods in gen2tasks.MetricTask [ap_association, ip_diffim, verify]

  • DM-21103: lsst.verify seems to be broken [utils]

  • DM-21104: Upgrade GSL to v2.6 to see if this fixes intermittent failures to build gsl on macOS [gsl]

  • DM-21129: Improve “unsupported operand types” error for afwImage arithmetic [afw]

  • DM-21141: ip_diffim breaks with numpy 1.17 [ip_diffim]

  • DM-21146: Adding support for PostgreSQL Registry [daf_butler]

  • DM-21152: Remove pex_policy usage when not part of gen2 butler [afw, astrometry_net_data, meas_astrom, meas_deblender, meas_extensions_photometryKron, obs_cfht, obs_decam, obs_lsst, obs_lsstSim, obs_sdss, obs_test]

  • DM-21153: Fix bugs in DcrAssembleCoaddTask from PipelineTask merge [pipe_tasks]

  • DM-21166: Create DiaCalculation plugins that replicate current ap_association behavior. [ap_association]

  • DM-21167: Migrate ap_association fully to new DiaCalculation plugin system for time-series features. [ap_association]

  • DM-21169: Add Corner Rafts with correct positions and rotations to obs_lsst [afw, obs_lsst]

  • DM-21171: Create task to add local PhotoCalib and WCS value to source table rows [meas_base, pipe_tasks]

  • DM-21181: Add getLocalCalibration function to PhotoCalib [afw]

  • DM-21186: Rename auxTel to LATISS [obs_lsst, pipe_tasks]

  • DM-21187: Camera name is wrong in YAMLCamera files [obs_lsst]

  • DM-21198: Remove unsupported obs packages from lsst_obs metapackage [lsst_obs]

  • DM-21201: Research cross-database approach to inserts with custom conflict resolution [daf_butler]

  • DM-21210: On transaction abort, PostgreSQL ignores commands in the entire transaction block. [daf_butler]

  • DM-21222: Connection string needs to support Oracle Wallet [daf_butler]

  • DM-21229: Investigate coadd/diffim platescale issues [pipe_tasks]

  • DM-21240: Fix usage of auxTel in obs_lsst config files [obs_lsst]

  • DM-21241: Fix setting of boresight rotation angle for imsim data [obs_lsst]

  • DM-21248: cameraMapper _standardizeExposure should not try to create a WCS for each amp [obs_base]

  • DM-21286: Enable Astropy download cache [sconsUtils]

  • DM-21287: Verify multi-Gaussian Sersic approximations in meas_modelfit and MultiProFit [shapelet]

  • DM-21296: Do not persist ineffectual deprecated Config fields [pex_config]

  • DM-21301: pipetask –register-dataset-types doesn’t work with Postgres registry on AWS [daf_butler]

  • DM-21314: Investigate GC problems with Storable [afw, utils]

  • DM-21320: Add first metric to colorAnalysis [obs_subaru, verify_metrics]

  • DM-21327: Replace ExposureInfo implementation with homogeneous map [afw]

  • DM-21351: Move pipe_base tests to obs_test to normalize dependencies [obs_test, pipe_base]

  • DM-21355: A NaN value in a PropertySet can cause an abort when reading WCS [afw]

  • DM-21357: Add items() method to PropertySet/PropertyList [daf_base]

  • DM-21361: Need to {{reorderAndPadList}} in Gen2 too in AssembleCoadd [pipe_tasks]

  • DM-21363: Fix semantics of PropertySet.update [daf_base]

  • DM-21365: Fix presumed master breakage from merge of PR#123 to obs_lsst [obs_lsst]

  • DM-21367: Upgrade Jenkins jobs to devtoolset-8 [lsst]

  • DM-21379: Fix validate_drp’s HscQuick test script [validate_drp]

  • DM-21390: Add a show option to examine a qgraph [ctrl_mpexec, pipe_base]

  • DM-21398: Change background handling defaults in imageDifference.py [pipe_tasks]

  • DM-21407: Remove unneeded dependency on astrometry_net in meas_astrom [meas_astrom]

  • DM-21421: Create a system to define pipeline level execution and configuration [ctrl_mpexec, obs_subaru, pipe_base, pipe_tasks]

  • DM-21422: Fix ip_diffim mosaic debug plots broken by Mosaic.makeMosaic signature change and compiler warning [afw, ip_diffim]

  • DM-21429: Stop S3-backed butler tests from attempting import/export [daf_butler]

  • DM-21451: Remove DatabaseDict and vectorize Datastore/Butler ingest APIs [daf_butler, obs_base]

  • DM-21452: Deal with undefined values in ingest [pipe_tasks]

  • DM-21453: Remove extra sdssCentroid plugin from default DipoleTask plugin list [ip_diffim]

  • DM-21454: Foreign key error when running makeButlerRepo.py against Oracle [daf_butler]

  • DM-21488: Using asAstropy() on a BaseCatalog will raise an exception unless lsst.daf.base has been imported [afw]

  • DM-21501: Implement internal aperture corrections for fgcmcal tract mode [fgcm, fgcmcal]

  • DM-21531: Update scarlet to latest version [scarlet]

  • DM-21701: Rebase and update with deblender sprint code [pipe_tasks]

  • DM-21705: Use the correct kernelSize in case of science image convolution in image differencing [ip_diffim]

  • DM-21709: Update _is_on_mountain to reflect actual status [obs_lsst]

  • DM-21724: Unpickling error reading qgraph with DimensionUniverse [ctrl_mpexec]

  • DM-21738: Put Dual License on pex_config so it can be distibuted under bsd clause 3. [pex_config]

  • DM-21746: Add pyarrow to scipipe_conda_env [lsst]

  • DM-21748: oracle ci_hsc_gen3 sqlalchemy.exc.ObjectNotExectuableError [daf_butler]

  • DM-21785: Create HTM index, nDiaSources, and DiaObjectFlags plugin [ap_association]

  • DM-21813: The “filter” should only be set for raw ccds and not for raw amps in obs_lsst [obs_lsst]

  • DM-21814: Propagate calib_psfCandidate flags from SFM to fgcmcal star outputs [fgcm, fgcmcal]

  • DM-21833: afw.table.SourceCatalog.readFits signals reading partially-written fits file [afw]

  • DM-21836: Add OBSTYPE/purpose to Gen3 Registry exposure table [daf_butler, obs_base, obs_subaru]

  • DM-21838: Add some table methods to astshim [astshim]

  • DM-21843: pipetask runner fails at constructing qgraph dot files [ctrl_mpexec, daf_butler]

  • DM-21853: error occurring when ROTANGLE isn’t set [obs_lsst]

  • DM-21855: Move daf.butler.instrument to obs_base [daf_butler, obs_base, obs_decam, obs_lsst, obs_subaru]

  • DM-21859: Multiple PrerequisiteInput quanta are clobbered by a single value. [pipe_base]

  • DM-21889: CmdLineActivator reuses command-line argument options in different sub-commands [ctrl_mpexec]

  • DM-21899: Remove pytest EUPS package [lsst, pytest_flake8, pytest_session2file, python_mccabe, sconsUtils]

  • DM-21981: Inconsistency in LSST amplifier bboxes between Gen2 and Gen3 [obs_lsst]

  • DM-21982: Update fgcmcal round-trip flux test to consistently use the same float length [fgcmcal]

  • DM-21990: latitude and longitude are transposed in LSST_LOCATION [obs_lsst]

  • DM-22035: Update Astropy to version 3.2.3 [lsst]

  • DM-22068: Add ABC, StorageClass, and Formatter for stray-light correction [daf_butler, ip_isr, obs_subaru]

  • DM-22069: Add lazy-product BoundedField class [afw]

  • DM-22070: Add unnormalized (but continuous) version of PixelScaleBoundedField [afw]

  • DM-22079: Linearity input bug in DM-18610 [afw]

  • DM-22108: Gen3 ip_isr raises if ‘defects’ not supplied in inputs [ip_isr]

  • DM-22120: ap_verify scales poorly to large runs [ap_verify, verify]

  • DM-22139: AP association bug with >1000 diaSources [ap_association]

  • DM-22147: Add python-only function to ChebyshevBoundedField to approximate another BoundedField [afw]

  • DM-22148: Fix HSC ingestion: autoguider unset [obs_subaru]

  • DM-22164: Fix pipe_base after Instrument was moved to obs_base [pipe_base]

  • DM-22166: Fix pipetask –show=pipeline option [ctrl_mpexec]

  • DM-22177: ctrl_mpexec calls non-existent Pipeline.addConfigOverrideFile method [ctrl_mpexec]

  • DM-22182: Race condition between tests and doc targets can break scons build [sconsUtils]

  • DM-22191: isr for yaml cameras is broken due to zero value for suspectLevel [ip_isr, obs_base]

  • DM-22198: pipetask qgraph Unexpected pipeline action: new_instrument [ctrl_mpexec]

  • DM-22209: Fix pipelines_lsst_io bug from d_2019_11_06 [afw]

  • DM-22217: Do not over-fix obs_lsst detector bbox [obs_lsst]