Tickets Addressed in the Spring 2020 release (v20_0_0)

  • DM-6075: Document the need for abstract base tasks for tasks [pipe_base]
  • DM-6981: Add column setters for Flag types in catalogs [afw]
  • DM-7584: Fix missing code lines in doxygen page on using masked image locators [afw]
  • DM-7718: psfex tries to build with the wrong cc [psfex]
  • DM-10190: Implement a sparsity constraint with a cutoff [scarlet]
  • DM-11097: Document required entry points for Tasks [pipe_base]
  • DM-11990: Fix occasional 1-pixel shifts in Zogy [ip_diffim]
  • DM-11991: Understand and fix (if necessary) relative calibration for Zogy [ip_diffim]
  • DM-13403: numpy types fail in butler dataIds [daf_butler]
  • DM-13440: Check that amplifier gains are >= 0 and set to 1.0 if they are bad after warning [ip_isr]
  • DM-14909: Add disableCc=True to validate_drp SConstruct [validate_drp]
  • DM-15536: Add level of indirection in defining Visits from Exposures [pipe_tasks, pipe_base, ip_isr, obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-15813: convert log to numpydoc status [log]
  • DM-15890: Switch butler schema to use TAI nanoseconds rather than datetime [obs_subaru, obs_base, daf_butler, obs_lsst]
  • DM-16234: Move qa_explorer/postprocess.py and writeObjectTable to pipe_tasks [pipe_tasks, obs_subaru, obs_base]
  • DM-17169: Crosstalk sources dataset does not cleanly fit Gen3 butler concept [pipe_base]
  • DM-17566: Formally deprecate discouraged C++ afw/geom components [geom, afw]
  • DM-17597: Update testdata_jointcal catalogs to be processed with gaia+ps1 [fgcmcal, jointcal, testdata_jointcal, validate_drp, obs_decam]
  • DM-18529: Add unique parts of aggregated dataids to the job metadata [validate_drp]
  • DM-18683: Review Craig’s PR and merge [cp_pipe, obs_subaru, obs_base, obs_lsst]
  • DM-19903: Add columnar defect identification to findDefectsTask in cp_pipe please [cp_pipe]
  • DM-20039: Change fake source insertion tasks to use generators rather than appending to lists. [pipe_tasks]
  • DM-20074: Remove SubaruMakeCoaddTempExpTask after S19A AND problem fixed upstream [obs_subaru_data, meas_algorithms, obs_subaru]
  • DM-20163: Investigate using WCS Jacobian BoundedField in fgcmcal fit [fgcmcal, fgcm]
  • DM-20551: Add a number of bug-fixes and requested features to display_matplotlib [display_matplotlib]
  • DM-20558: Investigate rescaling the coadd variances for difference imaging templates [pipe_tasks]
  • DM-20565: Remove afwGeom aliases for geom [lsst_dm_stack_demo, meas_algorithms, afw, cp_pipe, jointcal, meas_modelfit, meas_astrom, meas_deblender]
  • DM-20573: Remove obs_lsstSim package from lsst_obs metapackage [testdata_jointcal, lsst_obs, lsst_apps]
  • DM-20823: Interface Spectractor with DM [obs_base, obs_test]
  • DM-20841: Investigate reports of incorrect meas_mosaic WCSs after rotation by 90deg [obs_subaru, afw]
  • DM-21004: Add writeInitialModel option to jointcal config [jointcal]
  • DM-21155: Implement QE curve interface as described in RFC-625 [meas_algorithms, pipe_tasks, obs_lsst, ap_verify, obs_subaru, obs_base, obs_test, obs_decam, obs_lsst_data]
  • DM-21204: Ensure CCS LSST data can be ingested by butler without collision [obs_lsst]
  • DM-21207: Remove deprecated Policy usage from pex_config, ip_diffim, and meas_algorithms [meas_algorithms, pex_config, ip_diffim]
  • DM-21212: Update existing cp_pipe tasks to pipelineTasks [astro_metadata_translator]
  • DM-21221: PTC task should produce a linearity model [cp_pipe]
  • DM-21237: BBox error in some DCR image differencing templates [ip_diffim]
  • DM-21246: Allow deferred passing of run/collection to Butler [pipe_tasks, obs_base, daf_butler]
  • DM-21276: Ensure all prescan/overscan accessors exist [afw]
  • DM-21308: Update pipe_tasks to allow for changing external calibrations [pipe_tasks, obs_subaru, obs_lsst]
  • DM-21380: Add a galaxy photometric repeatability metric to validate_drp [lsst_ci, validate_drp, verify_metrics]
  • DM-21448: Clean up DatasetRef comparisons and immutability [daf_butler, ctrl_mpexec]
  • DM-21688: Create multi plugins for DiaObject summary statistics [ap_association]
  • DM-21711: Remove cameraGeom interfaces deprecated in DM-18610 [afw, ip_isr]
  • DM-21739: Enable fringe and HSC y-band stray light correction in Gen3 IsrTask [obs_decam, obs_subaru, pipe_base, ip_isr]
  • DM-21764: Better encapsulate dataset storage in Registry [pipe_tasks, daf_butler]
  • DM-21768: Vectorize dataset insert API [daf_butler, ctrl_mpexec]
  • DM-21795: Rework Registry provenance objects to match prototype [daf_butler, pipe_base, ctrl_mpexec]
  • DM-21809: Check the variance plane operations and the decorrelation afterburner operations in the science convolution case [pipe_tasks]
  • DM-21849: Make runs a type of collection [pipe_base, ctrl_mpexec, obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-21869: Make IngestTask callable by OODS [pipe_tasks]
  • DM-21875: Add StorageClass and Formatter support necessary to persist lsst.verify.Measurement in Gen3 repos [verify, daf_butler]
  • DM-21877: Create “marker” Butler dataset for PPDB [ap_association, ap_verify, obs_subaru, obs_base, obs_test, verify, obs_decam, ap_pipe]
  • DM-21886: Create PipelineTask driver for ap_pipe tasks that interact with the APDB [ap_association, ap_pipe]
  • DM-21900: More functionality for filename template generation [daf_butler]
  • DM-21910: Move lsst.verify.gen2tasks.MetricTask to lsst.verify.tasks [verify, ip_diffim]
  • DM-21911: Retrofit Gen 3 functionality onto MetricTask [ap_association, pipe_base, ap_verify, verify, ip_diffim]
  • DM-21912: Implement PpdbMetricTask.runQuantum [verify]
  • DM-21917: Convert DCR templates to PipelineTasks [pipe_tasks]
  • DM-21950: Update validate_drp to work with fgcmcal calibrations as an option [validate_drp, obs_base]
  • DM-21965: Investigate failures when running scarlet on fake images [meas_extensions_scarlet]
  • DM-22027: Re-make Gaia refcat to fix epoch [meas_algorithms]
  • DM-22039: Rename dax_ppdb to dax_apdb together with all dependencies [ap_verify, verify, ap_pipe, ap_association]
  • DM-22062: Add parquet support to Gen3 Butler [daf_butler]
  • DM-22073: Add matplotlib (output) support to Gen3 butler [daf_butler]
  • DM-22093: Store begin/end times of ap_pipe in ap_verify [verify]
  • DM-22137: Test proximal-ADAM scarlet branch on HSC [meas_extensions_scarlet, pipe_tasks]
  • DM-22138: validateDrp arguments (including brightSnr) not passed/parsed correctly [lsst_ci, validate_drp]
  • DM-22162: Add metadata writing to PipelineTask execution logic [pipe_base, ctrl_mpexec]
  • DM-22173: Support and document support for dates/times in query expressions [daf_butler]
  • DM-22192: Remove PixelScaleBoundedField [afw]
  • DM-22199: Add decorators for iterating over tests [utils]
  • DM-22205: Surprising error when passing a string to queryDimensions [daf_butler]
  • DM-22207: Update dev guide and deprecate_pybind11 message to reflect the fact that it works on classes, top [utils]
  • DM-22222: Add default processCcd configs to the processCcd pipeline [pipe_tasks]
  • DM-22227: measurePhotonTransferCurve.py fails to find raftName [cp_pipe, obs_base]
  • DM-22233: Remove astrometry_net_data dependency from pipe_tasks [lsst_ci, pipe_tasks]
  • DM-22234: Remove residual python_future usage from stack [obs_cfht, shapelet, display_firefly, ctrl_execute, ap_association, ctrl_orca, meas_modelfit, pipe_drivers, display_matplotlib, ctrl_pool, sphgeom, daf_persistence]
  • DM-22235: Fix deprecated collections imports [meas_base, base]
  • DM-22255: Move plotting IO out of plotting functions in coaddAnalysis [obs_subaru]
  • DM-22256: Remove SkipUnless PyArrow when PyArrow works with macos stack [pipe_tasks]
  • DM-22260: Metadata is not persisted when persisting an ExposureCatalog to fits [afw]
  • DM-22283: Removed unneeded calls to RegistryTask.addVisits [obs_decam, pipe_drivers]
  • DM-22299: Speed up specific diaCalculation plugins using fast pandas functionality [ap_association]
  • DM-22301: Printing Gen3 configs in a manner that is easily diff’d [pex_config, ctrl_mpexec]
  • DM-22305: Remove eups dependencies from data packages [validation_data_cfht, validation_data_decam, lsst_ci, afwdata]
  • DM-22363: Add ability for defefredDatasetHandles to retrieve dataset components [daf_butler]
  • DM-22364: Gen 3 Butler cannot be created using daf.butler.Config [daf_butler, ctrl_mpexec]
  • DM-22371: Add post-processing tasks to ci_hsc_gen2 [obs_base]
  • DM-22372: Fix docstring formatting [pipe_base]
  • dm-22377: Slight numeric differences in demo using ubuntu 19 (release candidate v19.0.0.rc1 [lsst_dm_stack_demo]
  • DM-22386: Jointcal fails unit tests with Astropy 4 [jointcal]
  • DM-22389: verify fails with Astropy 4 [verify]
  • DM-22405: Rewrite MeasurePhotonTransferCurveTask data types [cp_pipe, obs_base, obs_lsst]
  • DM-22409: Include meas_extensions_scarlet in pipelines.lsst.io [meas_extensions_scarlet]
  • DM-22417: Remove python future from obs_base [obs_base]
  • DM-22427: latitude and longitude are transposed in AUXTEL_LOCATION in obs_lsst [obs_lsst]
  • DM-22453: Update load to safe_load for yaml reading in validate_drp [validate_drp]
  • DM-22469: Add fgcmcal documentation tree [fgcmcal]
  • DM-22470: Remove all uses of future from fgcmcal and fgcm [fgcm]
  • DM-22478: Lightly restructure ap_association tasks and create new DiaObject/DiaSource pre-loading task. [ap_association, ap_pipe]
  • DM-22479: SQuaSH being spammed with timestamps [verify]
  • DM-22481: Do not delete datastore directory on error [daf_butler]
  • DM-22485: Fix docstring heading to make example appear [daf_butler]
  • DM-22495: validate_drp reportPerformance.py fails on new metrics [validate_drp]
  • DM-22504: Support for lsstDebug functionality in Gen3 middleware [ctrl_mpexec]
  • DM-22505: Create run method in ImageDifferenceTask and move all functionality from runDataRef except butler usage [pipe_tasks, ip_diffim]
  • DM-22508: Migrate Database interface from prototype to master and add implementations [daf_butler]
  • DM-22519: The GroupView iterator is broken [afw]
  • DM-22541: Add gen3 support classes to ImageDifferenceTask [pipe_tasks, ip_diffim]
  • DM-22550: Handle latiss diffuser filter in obs_lsst [obs_base, obs_lsst]
  • DM-22576: Add programmatic header fixups to astro_metadata_translator [astro_metadata_translator]
  • DM-22599: Develop PipelineTask unit test framework [pipe_base, daf_butler]
  • DM-22605: Add measured QE and read noise to lsstcam [obs_lsst_data, obs_lsst]
  • DM-22609: Start using Database in Registry [obs_base, daf_butler]
  • DM-22641: ap_verify CI broken [ap_verify]
  • DM-22643: convert visualizeVisit to gen3 [pipe_tasks]
  • DM-22647: Bug in isrMock.getCamera’s use of CameraWrapper [ip_isr]
  • DM-22648: Add DcrModel to Gen3 butler [daf_butler]
  • DM-22655: Genericize gen2to3.py to be useable with any gen2 repo [pipe_base, obs_lsst, ip_isr, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-22659: Fix bitrot with how brighter-fatter kernels are handled, add option to use new gain values [validation_data_cfht, jointcal, ip_isr, testdata_jointcal, obs_subaru]
  • DM-22661: isrTask failed to find master flat due to filter difference [ip_isr]
  • DM-22677: Modernize python scripts in daf_butler [daf_butler]
  • DM-22680: linearizer attempts to run in gen3 when LinearizerType=None [ip_isr]
  • DM-22704: filter ‘diffuser’ cannot be read [obs_base, obs_lsst]
  • DM-22708: Fix decam gen3 ingest [obs_decam, obs_lsst, obs_subaru, obs_base, afw, testdata_decam]
  • DM-22718: Typos in obs_lsst Mapper for difference image data products [obs_lsst]
  • DM-22727: Add numpy warnings catch to DiaCalculationPlugins [ap_association]
  • DM-22728: Handle immutable amplifiers in ip_isr test suite [ip_isr]
  • DM-22741: Remove ap_association subtasks from ap_pipe. Change to single DiaPipelineTask subtask. [ap_verify, ap_pipe, ap_association]
  • DM-22770: Remove duplication of BaseMapper [pipe_tasks, obs_base]
  • DM-22771: Resurrect HSC RC2 Gen3 repo bootstrap [obs_subaru, obs_base, daf_butler]
  • DM-22776: Update Gen3 isrTask for BF corrections [daf_butler, ip_isr]
  • DM-22777: Prune diaForcedSources in DiaForcedSourceTask [ap_association]
  • DM-22788: Responses tests fail with modern responses and old requests [verify]
  • DM-22790: pex_config FutureWarning reports wrong line number [pex_config]
  • DM-22794: obs_base tests should not use daf_butler test configs [obs_base]
  • DM-22796: pipe_tasks installs 200MB of temporary test output [pipe_tasks]
  • DM-22797: Various FITS header fixes [pipe_tasks, afw]
  • DM-22798: Remove (unused) ability for fgcmcal to run on a full repo without specifying any ids [fgcmcal]
  • DM-22800: Fix missing format % in pex_config [pex_config]
  • DM-22803: validate_drp and numpy1.17/astropy4/matplotlib3.1 fails [validate_drp]
  • DM-22817: Update conda base environment [eigen]
  • DM-22818: Matplotlib 3.1 bug triggered by fgcmcal on macOS [fgcmcal]
  • DM-22952: Add support of dcr coadds to ImageDifferenceTask in gen3 mode [pipe_tasks, ip_diffim]
  • DM-22955: Add spatially varying HSC NB filters to obs_subaru [obs_subaru]
  • DM-22958: CModel has unflagged flux=0 fluxSigma=NAN for 1% of sources [meas_modelfit]
  • DM-22962: Add a --walltime arg to ctrl_pool’s BatchArgumentParser [ctrl_pool]
  • DM-23008: Add DCR model subfilters to Gen3 registry [pipe_tasks]
  • DM-23023: Simplify linearity corrections [obs_decam, obs_subaru, ip_isr]
  • DM-23024: Support multi-dataset single file ingest in daf_butler [obs_subaru, obs_base, daf_butler]
  • DM-23026: Make sure calibration columns are propagating properly in processCcdWithFakes [pipe_tasks]
  • DM-23030: Modify Photometry SDM Functor to use stored calibration value. [pipe_tasks]
  • DM-23031: Create dipole separation functor [pipe_tasks]
  • DM-23033: Updates to reference catalog creation documentation [meas_algorithms]
  • DM-23036: Add ability for fgcmcal to do calibrations on local background-corrected fluxes [fgcmcal]
  • DM-23044: PTC task should persist usable linearity models [pipe_tasks, cp_pipe, obs_lsst, ip_isr, obs_base]
  • DM-23045: Validate linearity correction [cp_pipe, ip_isr]
  • DM-23048: validate_drp uses deprecated matplotlib call [validate_drp]
  • DM-23054: Resolving formatting issues in lsst.utils.tests documentation [utils]
  • DM-23056: Suppress FutureWarnings from LSST code [meas_algorithms, utils, afw, pipe_tasks]
  • DM-23062: Preserve input headers [obs_base]
  • DM-23063: Request for ability to turn off checksumming in Gen3’s POSIX datastore [daf_butler]
  • DM-23065: Test fgcmcal on NB0387 HSC data [fgcmcal, fgcm]
  • DM-23067: Delete commented code in tests/test_matchBackgrounds.py [pipe_tasks]
  • DM-23071: Update default aperture correction configuration in hsc to select bright stars [obs_subaru]
  • DM-23073: makeBrighterFatterKernel uses wrong PTC fit type [cp_pipe]
  • DM-23074: Make the schema of the output Object parquet files input-independent [pipe_tasks, obs_subaru]
  • DM-23077: Update default interpolation for Curve classes [meas_algorithms]
  • DM-23078: Add sky objects to the single frame processing step [lsst_dm_stack_demo, pipe_tasks]
  • DM-23079: Move opaque table Registry code into helper classes [daf_butler]
  • DM-23080: Move dimension Registry code into helper classes [daf_butler]
  • DM-23083: Update large masks for BF convolution issues [obs_subaru, ip_isr]
  • DM-23085: Deprecate SourceDetectionTask.makeSourceCatalog [synpipe, meas_algorithms, pipe_tasks, testdata_deblender, ip_diffim]
  • DM-23090: Update LATISS filters in obs_lsst to match commissioning filters [obs_lsst]
  • DM-23091: HSC-Y failed on w_2020_02 [obs_subaru]
  • DM-23102: Update python types for matchVisits and objectTable [obs_base]
  • DM-23103: Update functor unittests to no longer rely on test_multilevel_parq.csv.gz [pipe_tasks]
  • DM-23104: Augment ObjectTable to be useable for QA [pipe_tasks, obs_subaru]
  • DM-23112: Fix cgUtils.overlayCcdBoxes for rename of getAmpInfoCatalog [afw]
  • DM-23129: Update obs_base ingest RawFileData for multi-dataId files [obs_base]
  • DM-23131: Fix “unordered” map documentation in DetectorCollection getters [afw]
  • DM-23149: Fix LATISS data ingest for on sky data [obs_lsst]
  • DM-23166: Add __all__ to lsst.utils.deprecated module. [utils]
  • DM-23171: Add exposure group to metadata translator [astro_metadata_translator, obs_lsst]
  • DM-23172: Fix flake8 violations in astshim [astshim]
  • DM-23173: Implement RFC-650: update packages to prefer binary operators at the beginning of continued lines [display_ds9, skymap, pex_config, obs_test, ip_diffim, base, obs_decam, meas_extensions_simpleShape, meas_extensions_psfex, meas_base, daf_persistence, meas_algorithms, ap_association, geom, meas_extensions_scarlet, meas_astrom, meas_deblender, coadd_utils, obs_sdss, pex_exceptions, pipe_base, daf_base, sphgeom, pipe_tasks, afw, ap_pipe]
  • DM-23174: Consolidate daf_butler test code [daf_butler]
  • DM-23178: Convert some of afw to use f strings [afw]
  • DM-23179: revised color terms for HSC [obs_subaru]
  • DM-23192: Use correct weight maps in scarlet [scarlet, meas_extensions_scarlet, proxmin]
  • DM-23195: DM-21221 broke cp_pipe due to lack of tests [cp_pipe]
  • DM-23196: Add calibration storage to multiBand.py [pipe_tasks, meas_base]
  • DM-23203: Add tabular output to translate_header [astro_metadata_translator]
  • DM-23206: validate_drp crashes when trying to apply external skyWcs [validate_drp]
  • DM-23208: Add exposure group to gen3 registry [obs_base, daf_butler]
  • DM-23212: pipetask run with multiple “-i” command line arguments fails [ctrl_mpexec]
  • DM-23213: It’s necessary to restart the butler to see newly ingested files [pipe_tasks]
  • DM-23222: Fix OBJECT ENGTEST date and RADEC [obs_lsst]
  • DM-23223: Allow translate_header to dump the fixed header [astro_metadata_translator]
  • DM-23224: Cross-check the schema column names in the Object table [obs_subaru]
  • DM-23231: Sort out visit vs exposure ID in gen 2 butler [obs_lsst]
  • DM-23237: Strange image types ingested for LATISS images [obs_lsst]
  • DM-23238: Add per row overscan removal [ip_isr]
  • DM-23249: New decam ingest tests need skipif for testdata_decam [obs_decam]
  • DM-23258: matchedVisitMetricsTask will crash if any visits have a missing source catalog [validate_drp]
  • DM-23275: Restore outfile option for butler makeRepo [daf_butler]
  • DM-23277: Update ImageDifferenceTask to remove v20 deprecation warnings [pipe_tasks, ip_diffim]
  • DM-23278: Fix RADEC for LATISS data on 27th Jan [obs_lsst]
  • DM-23281: FILTER and GRATING not concatenated in LATISS data [obs_lsst]
  • DM-23282: LATISS mapper doesn’t find defects [obs_lsst]
  • DM-23310: Some ComCam images do not have LSST_NUM header [obs_lsst]
  • DM-23313: Fix dominant failure mode in deblending sources with 2-components [meas_extensions_scarlet]
  • DM-23329: Fix controller code in phosim and imsim data [obs_lsst]
  • DM-23331: Add default fgcmcal configuration files for HSC processing [fgcmcal, fgcm, obs_subaru]
  • DM-23342: Fix ingestDriver [pipe_drivers, pipe_tasks]
  • DM-23343: Mark Synpipe as deprecated [synpipe]
  • DM-23352: Add jointcal, skyCorr to forcedPhotCcd [pipe_tasks, pex_config, obs_subaru, obs_base, daf_butler, meas_base]
  • DM-23359: Allow pex_config configs to use __file__ [pex_config]
  • DM-23362: Old LATISS data has missing OBSID/DAYOBS [obs_lsst]
  • DM-23364: Flag edge sources and shift them in meas_extensions_scarlet [meas_extensions_scarlet]
  • DM-23369: Remove lsst.ip.isr.addDistortionModel [obs_cfht, ip_isr]
  • DM-23371: Remove lsst.obs.base.CameraMapper._extractAmpId [obs_subaru, obs_base]
  • DM-23380: Remove lsst.fgcmcal.FgcmFitCycleConfig.pixelScale [fgcmcal]
  • DM-23396: Function “overscanCorrection” in “isrFunctions.py” needs refactoring [obs_decam, obs_subaru, obs_lsst, ip_isr]
  • DM-23397: Remove refCols entries in obs_lsst calibrations [obs_lsst]
  • DM-23414: lsst_ci fails with astropy 4 and numpy >=1.17 [validate_drp]
  • DM-23420: ap_association does not work with numpy 1.18 and pandas 1.0 [ap_association]
  • DM-23426: Use PipelineTask test framework for MetricTask [ap_association, verify, ip_diffim]
  • DM-23436: apdb creation instructions outdated [ap_association, ap_pipe, ip_diffim]
  • DM-23448: Sort out defect generation and ingest for LATISS (and others) [cp_pipe, pipe_tasks, obs_lsst]
  • DM-23449: Butler can’t get bias for LATISS using expId [obs_lsst]
  • DM-23477: pipe_base ScalarError can’t be pickled [pipe_base]
  • DM-23490: Correct plate scale and boresight position for LATISS [obs_lsst]
  • DM-23498: Explore using PSF-matched warps for DCR coadds [pipe_tasks]
  • DM-23503: Butler gen3 datastore templates should handle slashes in data Ids [daf_butler]
  • DM-23509: obs_lsst failing LATISS plate scale test [obs_lsst]
  • DM-23526: Fix fgcmcal issues exposed during PDR2 run [fgcmcal, fgcm, obs_subaru]
  • DM-23558: DCR coadds are missing PhotoCalib [pipe_tasks, ip_diffim]
  • DM-23566: Provide a method to get dataset types from a Gen 2 butler [daf_persistence]
  • DM-23584: expId has wrong number of digits in comCamMapper.yaml template [obs_lsst]
  • DM-23589: Update LATISS filters in obs_lsst [obs_lsst]
  • DM-23600: Gen2 butler can’t get bias for ComCam using expId [obs_lsst]
  • DM-23616: Run converted ap_verify testdata through gen3 pipeline [obs_decam, pipe_tasks, pipe_base, ap_pipe_testdata]
  • DM-23623: Measure crosstalk coefficients for AuxTel chip and add for use. [obs_lsst, obs_base, ip_isr]
  • DM-23627: Missing psfMatched_nImage definition [obs_base]
  • DM-23630: fgcmcal failure Ubuntu [fgcmcal]
  • DM-23635: Confusing help text for pipetask [ctrl_mpexec]
  • DM-23638: Add filename attribute to ParquetTable [pipe_tasks]
  • DM-23651: ap_pipe calls some deprecated things [validation_data_cfht, meas_algorithms, cp_pipe, ap_association, obs_lsst, obs_subaru, ip_diffim, afw, obs_decam, meas_base]
  • DM-23652: Add documentation for –show history to pipe_base [pipe_base]
  • DM-23663: Make insertFakes work for HSC artificial star tests [pipe_tasks, obs_base]
  • DM-23671: Improve dataset deletion in Gen3 butler [daf_butler]
  • DM-23678: Bug in s3Datastore when using temporary file [daf_butler]
  • DM-23681: Check, in the unit tests, that fitPtcAndNonLinearity returns what it is supposed to return. [cp_pipe]
  • DM-23699: Update fgcmcal default config format to remove possibility of index errors [fgcmcal, fgcm, obs_subaru]
  • DM-23701: pipetask-produced DOT for pipelines should show prerequisite inputs [ctrl_mpexec]
  • DM-23702: IsrTask shoud use regular Input for raw data [ip_isr]
  • DM-23703: ap_association fails if a new visit has 0 new diaObjects [ap_association]
  • DM-23711: Allow butler configs to use environment variables to find other configs [daf_butler]
  • DM-23718: Replace dots in gen3 file names [daf_butler]
  • DM-23722: Validate dataset type definitions in pipeline task connections [daf_butler, pipe_base, ip_isr]
  • DM-23728: Cleanup ci_hsc_gen2 to use new convert script instead of custom one [obs_base]
  • DM-23729: Generalize dataset to formatter mapping in obs packages [obs_decam, obs_base, daf_butler]
  • DM-23762: Check bad amps in LSSTCam are being tracked in the defects file [cp_pipe]
  • DM-23778: Write config and tests for obs_lsst gen2-gen3 convert [pipe_tasks, obs_lsst, obs_base, daf_butler, obs_lsst_data]
  • DM-23798: Fix scons install of bin [sconsUtils]
  • DM-23827: wrong python type for matchVisits_config [obs_base]
  • DM-23828: Create small test decam gen2 repo with calibs for test_convert2to3.py [testdata_decam, obs_decam]
  • DM-23830: Add cbp package to lsst_distrib [lsst_distrib]
  • DM-23835: Update environment with conda-forge third parties and conda compilers [obs_cfht, display_ds9, meas_modelfit, validate_drp, pex_config, skymap, obs_test, ip_diffim, base, meas_extensions_simpleShape, meas_extensions_psfex, proxmin, meas_base, daf_persistence, log, scarlet, sconsUtils, pex_policy, astro_metadata_translator, meas_extensions_photometryKron, meas_algorithms, ap_association, jointcal, ip_isr, obs_lsst, ap_verify, geom, meas_extensions_scarlet, meas_astrom, display_matplotlib, meas_deblender, meas_extensions_shapeHSM, coadd_utils, shapelet, psfex, utils, obs_sdss, pex_exceptions, pipe_base, daf_base, verify, obs_base, daf_butler, astshim, sphgeom, synpipe, lsst_dm_stack_demo, fgcmcal, pipe_tasks, jointcal_cholmod, fgcm, obs_subaru, firefly_client, obs_ctio0m9, afw, ctrl_pool]
  • DM-23836: DCR templates have incorrect variance [ip_diffim]
  • DM-23846: YAML files with python/object/apply fail in pyyaml>5.2.1 [obs_lsst]
  • DM-23873: Typo in cp_pipe makeBrighterFatterKernel.py [cp_pipe]
  • DM-23928: PropertySet not robust against duplicate and empty FITS cards [afw]
  • DM-23931: Allow butler.makeRepo to complain if a config already exists [daf_butler]
  • DM-23946: Change localWcs plugins to use a localGnomonicWcs transform. [pipe_tasks, meas_base]
  • DM-23952: Remove scarlet and proxmin from meas_deblender [pipe_tasks, meas_deblender]
  • DM-23959: CameraMapper._standardizeExposure should patch header [obs_base]
  • DM-23971: Undo TE1 “regression” [validate_drp]
  • DM-23975: Subfilter type inconsistency [pipe_tasks, daf_butler]
  • DM-23976: Move gen3 generic curated calibrations ingest code to obs_base [obs_decam, obs_subaru, obs_base, obs_lsst]
  • DM-23980: Standardize Gen3 instrument class names and location [obs_subaru, obs_lsst]
  • DM-24013: Fix bug in –output-run handling introduced in DM-21849 [ctrl_mpexec]
  • DM-24018: Failure to flatten or filter chained collections in queries [daf_butler]
  • DM-24027: CalibDate misinterpreted in curated calibration ingest [pipe_tasks]
  • DM-24055: Speed up table parsing in Defects [meas_algorithms]
  • DM-24060: Improve table creation efficiency in Defects [meas_algorithms]
  • DM-24062: Write Source Table transform tasks and Source.yaml for obs_subaru [pipe_drivers, pipe_tasks, obs_subaru, obs_base]
  • DM-24244: w_2020_13 makeButlerRepo.py missing left parens error with Oracle [obs_base, daf_butler]
  • DM-24250: Improve startup speed of fgcmcal and add checkpointing for restarts [fgcmcal]
  • DM-24259: Create “stub“ Gen2 HSC dataset for CI testing [ap_verify, obs_subaru, ap_pipe]
  • DM-24260: Create Gen3 versions of ap_verify datasets [obs_base]
  • DM-24273: pex_config test_history embeds too much tester knowledge [pex_config]
  • DM-24285: fitsExposureFormatter fails to read “Exposure” entries correctly [daf_butler]
  • DM-24288: Prototype datastore changes to support dataset disassembly/assembly [daf_butler]
  • DM-24290: Enable running ci_hsc_gen3 contents on AWS [pipe_tasks, daf_butler]
  • DM-24314: Support following links in Butler operations [daf_butler]
  • DM-24324: Generate alert packets in the alert production pipeline [ap_verify, ap_association]
  • DM-24330: add ability to run an obs_base command via the butler command [obs_base, daf_butler]
  • DM-24334: Create quickLookExp exposure type to support commissioning [obs_base, obs_lsst]
  • DM-24337: Fix raw formatter gen3 breakage [obs_base]
  • DM-24346: Don’t add fakes outside of the valid polygon area for the ccd [obs_subaru]
  • DM-24347: Allow component gets in gen3 butler to be None [obs_base, daf_butler]
  • DM-24352: Add auto transfer mode to gen3 ingest [obs_base, daf_butler]
  • DM-24365: Add relative symbolic link transfer mode to Gen 3 ingest [obs_base, daf_butler]
  • DM-24370: Support extensible scheduling in pipetask [ctrl_mpexec]
  • DM-24371: Implement fixed correction fixed PSF support decorrelation afterburner [pipe_tasks, ip_diffim]
  • DM-24375: Check time round trip issues with new time format [daf_butler]
  • DM-24376: afw should warn when FITS headers case may be modified (cfitsio > 3380) [afw]
  • DM-24378: Store instrument class with gen3 instrument registration and add API [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-24379: WriteSourceTable: add option to get local calib columns from calexp (or external calib) [pipe_tasks, obs_base]
  • DM-24385: Optimize posixStorage.search with relative path [daf_persistence]
  • DM-24392: Update testdata_jointcal to include Gaia+PS1 refcats [testdata_jointcal, jointcal]
  • DM-24404: Squash astropy ecsv read warnings [meas_algorithms]
  • DM-24429: Investigate change in fracDiaSourcesToSciSources in ap_verify CI [obs_decam]
  • DM-24434: symlink bug in posixDatastore.py on Ubuntu [daf_butler]
  • DM-24435: Freezing a config locks the registry(ies) of other instances of that config [pex_config]
  • DM-24450: Fix parsing of wavefront sensors [obs_lsst]
  • DM-24456: validateDrp matchedVisitMetricsTask broken [validate_drp]
  • DM-24466: Can’t run processCcd on multiple CCDs [obs_subaru]
  • DM-24472: Regenerate Gaia DR2 catalogs to correct coordinate error fields [testdata_jointcal, meas_algorithms]
  • DM-24495: Convert config overrides to use file in several obs packages [obs_decam, obs_subaru, obs_lsst]
  • DM-24515: Refactor gen3 butler.prune [daf_butler, ctrl_mpexec]
  • DM-24517: Reading of raws from existing imsim & phosim repos broken by DM-23584 [obs_lsst]
  • DM-24523: ap.verify.ingestion._findMatchingFiles excludes directories [ap_verify]
  • DM-24537: Add Isr calibration base class [ip_isr]
  • DM-24543: handle conflicting CLI subcommand names [obs_base, daf_butler]
  • DM-24545: remove unneeded explicit command names [obs_base, daf_butler]
  • DM-24546: Improve explanation of calibration and fluxCalibRadius in insertFakes.py code. [pipe_tasks]
  • DM-24553: Make a driver script for processCcdWithFakes [pipe_tasks, obs_subaru, obs_base]
  • DM-24555: Add ability to retrieve various Exposure components in gen3 butler. [daf_butler]
  • DM-24556: Add normalize method to Defects [cp_pipe, meas_algorithms]
  • DM-24559: write_curated_calibrations is not passing the butler registry to getInstrument [obs_decam, obs_subaru, obs_base, obs_lsst]
  • DM-24560: make ‘repo’ an argument, by convention always the first. [obs_base, daf_butler]
  • DM-24565: Un-deprecate old APIs related to afw.math.Background [afw]
  • DM-24569: Update obs_lsst with changes from observing runs [obs_lsst]
  • DM-24570: Fix lower and mixed-case metadata written to FITS files [meas_algorithms, pipe_tasks, meas_base]
  • DM-24584: create an ingestRaws butler command [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-24594: Create CollectionsManager implementation that uses an integer primary key [daf_butler]
  • DM-24612: Add indexes to dataset_collection tables [daf_butler]
  • DM-24614: Move dataset_location tables into manager/storage hierarchy [daf_butler]
  • DM-24616: Stop writing to gen2 butler directory during 2to3 conversion [obs_base, obs_lsst]
  • DM-24618: Fix leftover case warnings from FITS changes [meas_extensions_photometryKron]
  • DM-24658: Improve error reporting in connection string [daf_butler]
  • DM-24664: Resurrect HSC RC2 Gen3 repo bootstrap, part 2 [obs_decam, obs_subaru, obs_base, obs_lsst]
  • DM-24698: Clean up (at least) doc bug introduced in DM-21764 [daf_butler]
  • DM-24708: Quick fix of type mismatch runtime errors in decorrelation afterburner [ip_diffim]
  • DM-24716: Remove obs_lsstSim from lsst_ci package [lsst_ci]
  • DM-24721: w_2020_18 butler create does not work with Oracle [daf_butler]
  • DM-24734: Revive and profile RC2 QuantumGraph generation [pipe_base]
  • DM-24747: Remove obsolete Doxygen configuration [base]
  • DM-24756: Make sigma clipping configurable in PTC task [cp_pipe]
  • DM-24768: build race condition in daf_persistence [daf_persistence]
  • DM-24769: Improve connection string matching. [daf_butler]
  • DM-24780: Initial mypy configuration for daf_butler [daf_butler]
  • DM-24782: Add Sky Source Flag to SourceTable [obs_subaru]
  • DM-24786: New component column in datastore is too small [daf_butler]
  • DM-24796: pipetask’s graphviz dot files need to quote component dataset type names [ctrl_mpexec]
  • DM-24797: Store per-run information (configs, software versions) in butler repo [pipe_base, ctrl_mpexec]
  • DM-24804: create a ‘convert’ butler command [obs_decam, obs_subaru, obs_base, obs_lsst]
  • DM-24807: Investigate use of mypy for daf_butler datastores [daf_butler]
  • DM-24829: w_2020_19 butler write-curated-calibrations doesn’t work with Oracle [daf_butler]
  • DM-24851: Change Datastore.getUri to Datastore.getURIs [daf_butler, ctrl_mpexec]
  • DM-24857: Add processCcdWithFakesDriver to bin.src [pipe_drivers]
  • DM-24862: Update with latest scarlet and proxmin and test for performance [scarlet, meas_extensions_scarlet, proxmin]
  • DM-24864: Update processCcdWithFakes to add the sky_source column. [pipe_tasks]
  • DM-24892: Fix bug in gen2to3 when only special dataset types are being converted [obs_base]
  • DM-24906: Pin version of flake8 (and, if necessary, other tools) used in Travis checks [obs_cfht, cp_pipe, display_ds9, meas_extensions_convolved, meas_modelfit, pex_config, validate_drp, obs_test, ip_diffim, skymap, meas_extensions_simpleShape, pipe_drivers, base, meas_extensions_psfex, meas_base, obs_decam, daf_persistence, log, meas_algorithms, sconsUtils, pex_policy, astro_metadata_translator, meas_extensions_photometryKron, display_firefly, ap_association, jointcal, ip_isr, obs_lsst, ap_verify, geom, meas_extensions_scarlet, meas_astrom, display_matplotlib, meas_deblender, meas_extensions_shapeHSM, coadd_utils, shapelet, ctrl_execute, lsst_ci, utils, obs_sdss, pex_exceptions, pipe_base, ctrl_mpexec, ap_pipe, daf_base, verify, obs_base, daf_butler, astshim, sphgeom, synpipe, lsst_dm_stack_demo, fgcmcal, pipe_tasks, ctrl_platform_lsstvc, obs_subaru, obs_ctio0m9, afw, ctrl_pool, ctrl_orca]
  • DM-24907: Conda env missing psutil [ap_association, utils]
  • DM-24908: Change keys for imsim obs_lsst back to visit from expId [obs_lsst]
  • DM-24920: improve butler CLI mocking [obs_base, daf_butler]
  • DM-24923: eliminate redundant class name & name requirement in butler command test cases [obs_decam, obs_subaru, obs_base, obs_lsst]
  • DM-24924: Add new telescope name for ComCam and LSSTCam to obs_lsst [obs_lsst]
  • DM-24935: Add support for DataFrame and Table in ScienceSourceSelectorTask and ReferenceSourceSelectorTask [meas_algorithms]
  • DM-24937: move implementation of remaining butler commands to script folder [obs_base, daf_butler]
  • DM-24940: Move NamedKeyDict and NamedValueSet out of utils and clean up typing [pipe_base, daf_butler]
  • DM-24946: Propagate detectCoaddSources configs into multiBandDriver [obs_subaru]
  • DM-24972: Switch to using an explicit export of conda packages [scarlet]
  • DM-24975: Use GCS instead of AWS-S3 for the Butler datastore [daf_butler]
  • DM-24976: Rework testdata_jointcal dependencies [fgcmcal, testdata_jointcal, jointcal]
  • DM-24979: config_file_option does not save the ‘help’ input argument [daf_butler]
  • DM-24980: Fix flake8 failure in daf_base DateTime test [daf_base]
  • DM-24985: ‘file’ needs to be an @property when it uses a testdata pacakge [obs_decam, obs_subaru]
  • DM-24993: Drop afw case warnings down to DEBUG when reading fits files [afw]
  • DM-25000: Using components in PipelineTaskConnections no longer seems to work. [daf_butler, ctrl_mpexec]
  • DM-25014: Coadds converted to Gen 3 without abstract_filter [obs_base]
  • DM-25015: Create sublogger for pipeBase timer [pipe_base]
  • DM-25016: DM-21724 unpickling error appears again [pipe_base, ctrl_mpexec]
  • DM-25020: Update Sources.yaml with correct local background [obs_subaru]
  • DM-25040: ap_association uses physical filter in Gen 3 [ap_association]
  • DM-25072: Move gen3 afw formatters to obs_base [obs_base, daf_butler]
  • DM-25080: Finish static typing in daf.butler.registry [obs_base, daf_butler]
  • DM-25135: Restore HSC-R/HSC-R2 colorterms [obs_subaru]
  • DM-25149: Fix w18 ISR regression [ip_isr]
  • DM-25152: butler ingest raws does not register translators [obs_base]
  • DM-25153: Make it clear that gen3 instrument class paths don’t need .instrument [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam]
  • DM-25170: Fix to get predicted butler URIs [daf_butler]
  • DM-25177: pipelines.lsst.io broken by missing safeFileIo in daf_butler [daf_butler]
  • DM-25184: Make obs_subaru config overrides play nice with new jointcal filterMap config [obs_subaru]
  • DM-25187: daf_butler registry EllipsisType breaks pipelines.lsst.io doc build [daf_butler]
  • DM-25221: Backport fixes to w_2020_22 [psfex, jointcal, obs_lsst, obs_subaru, daf_butler]