.. _release-v20-0-0-tickets: ################################### Tickets Addressed in Release 20.0.0 ################################### - `DM-6075 `_: Document the need for abstract base tasks for tasks [pipe_base] - `DM-6981 `_: Add column setters for Flag types in catalogs [afw] - `DM-7584 `_: Fix missing code lines in doxygen page on using masked image locators [afw] - `DM-7718 `_: psfex tries to build with the wrong cc [psfex] - `DM-10190 `_: Implement a sparsity constraint with a cutoff [scarlet] - `DM-11097 `_: Document required entry points for Tasks [pipe_base] - `DM-11990 `_: Fix occasional 1-pixel shifts in Zogy [ip_diffim] - `DM-11991 `_: Understand and fix (if necessary) relative calibration for Zogy [ip_diffim] - `DM-13403 `_: numpy types fail in butler dataIds [daf_butler] - `DM-13440 `_: Check that amplifier gains are >= 0 and set to 1.0 if they are bad after warning [ip_isr] - `DM-14909 `_: Add disableCc=True to validate_drp SConstruct [validate_drp] - `DM-15536 `_: Add level of indirection in defining Visits from Exposures [pipe_tasks, pipe_base, ip_isr, obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-15813 `_: convert log to numpydoc status [log] - `DM-15890 `_: Switch butler schema to use TAI nanoseconds rather than datetime [obs_subaru, obs_base, daf_butler, obs_lsst] - `DM-16234 `_: Move qa_explorer/postprocess.py and writeObjectTable to pipe_tasks [pipe_tasks, obs_subaru, obs_base] - `DM-17169 `_: Crosstalk sources dataset does not cleanly fit Gen3 butler concept [pipe_base] - `DM-17566 `_: Formally deprecate discouraged C++ afw/geom components [geom, afw] - `DM-17597 `_: Update testdata_jointcal catalogs to be processed with gaia+ps1 [fgcmcal, jointcal, testdata_jointcal, validate_drp, obs_decam] - `DM-18529 `_: Add unique parts of aggregated dataids to the job metadata [validate_drp] - `DM-18683 `_: Review Craig's PR and merge [cp_pipe, obs_subaru, obs_base, obs_lsst] - `DM-19903 `_: Add columnar defect identification to findDefectsTask in cp_pipe please [cp_pipe] - `DM-20039 `_: Change fake source insertion tasks to use generators rather than appending to lists. [pipe_tasks] - `DM-20074 `_: Remove SubaruMakeCoaddTempExpTask after S19A AND problem fixed upstream [obs_subaru_data, meas_algorithms, obs_subaru] - `DM-20163 `_: Investigate using WCS Jacobian BoundedField in fgcmcal fit [fgcmcal, fgcm] - `DM-20551 `_: Add a number of bug-fixes and requested features to display_matplotlib [display_matplotlib] - `DM-20558 `_: Investigate rescaling the coadd variances for difference imaging templates [pipe_tasks] - `DM-20565 `_: Remove afwGeom aliases for geom [lsst_dm_stack_demo, meas_algorithms, afw, cp_pipe, jointcal, meas_modelfit, meas_astrom, meas_deblender] - `DM-20573 `_: Remove obs_lsstSim package from lsst_obs metapackage [testdata_jointcal, lsst_obs, lsst_apps] - `DM-20823 `_: Interface Spectractor with DM [obs_base, obs_test] - `DM-20841 `_: Investigate reports of incorrect meas_mosaic WCSs after rotation by 90deg [obs_subaru, afw] - `DM-21004 `_: Add writeInitialModel option to jointcal config [jointcal] - `DM-21155 `_: Implement QE curve interface as described in RFC-625 [meas_algorithms, pipe_tasks, obs_lsst, ap_verify, obs_subaru, obs_base, obs_test, obs_decam, obs_lsst_data] - `DM-21204 `_: Ensure CCS LSST data can be ingested by butler without collision [obs_lsst] - `DM-21207 `_: Remove deprecated Policy usage from pex_config, ip_diffim, and meas_algorithms [meas_algorithms, pex_config, ip_diffim] - `DM-21212 `_: Update existing cp_pipe tasks to pipelineTasks [astro_metadata_translator] - `DM-21221 `_: PTC task should produce a linearity model [cp_pipe] - `DM-21237 `_: BBox error in some DCR image differencing templates [ip_diffim] - `DM-21246 `_: Allow deferred passing of run/collection to Butler [pipe_tasks, obs_base, daf_butler] - `DM-21276 `_: Ensure all prescan/overscan accessors exist [afw] - `DM-21308 `_: Update pipe_tasks to allow for changing external calibrations [pipe_tasks, obs_subaru, obs_lsst] - `DM-21380 `_: Add a galaxy photometric repeatability metric to validate_drp [lsst_ci, validate_drp, verify_metrics] - `DM-21448 `_: Clean up DatasetRef comparisons and immutability [daf_butler, ctrl_mpexec] - `DM-21688 `_: Create multi plugins for DiaObject summary statistics [ap_association] - `DM-21711 `_: Remove cameraGeom interfaces deprecated in DM-18610 [afw, ip_isr] - `DM-21739 `_: Enable fringe and HSC y-band stray light correction in Gen3 IsrTask [obs_decam, obs_subaru, pipe_base, ip_isr] - `DM-21764 `_: Better encapsulate dataset storage in Registry [pipe_tasks, daf_butler] - `DM-21768 `_: Vectorize dataset insert API [daf_butler, ctrl_mpexec] - `DM-21795 `_: Rework Registry provenance objects to match prototype [daf_butler, pipe_base, ctrl_mpexec] - `DM-21809 `_: Check the variance plane operations and the decorrelation afterburner operations in the science convolution case [pipe_tasks] - `DM-21849 `_: Make runs a type of collection [pipe_base, ctrl_mpexec, obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-21869 `_: Make IngestTask callable by OODS [pipe_tasks] - `DM-21875 `_: Add StorageClass and Formatter support necessary to persist lsst.verify.Measurement in Gen3 repos [verify, daf_butler] - `DM-21877 `_: Create "marker" Butler dataset for PPDB [ap_association, ap_verify, obs_subaru, obs_base, obs_test, verify, obs_decam, ap_pipe] - `DM-21886 `_: Create PipelineTask driver for ap_pipe tasks that interact with the APDB [ap_association, ap_pipe] - `DM-21900 `_: More functionality for filename template generation [daf_butler] - `DM-21910 `_: Move lsst.verify.gen2tasks.MetricTask to lsst.verify.tasks [verify, ip_diffim] - `DM-21911 `_: Retrofit Gen 3 functionality onto MetricTask [ap_association, pipe_base, ap_verify, verify, ip_diffim] - `DM-21912 `_: Implement PpdbMetricTask.runQuantum [verify] - `DM-21917 `_: Convert DCR templates to PipelineTasks [pipe_tasks] - `DM-21950 `_: Update validate_drp to work with fgcmcal calibrations as an option [validate_drp, obs_base] - `DM-21965 `_: Investigate failures when running scarlet on fake images [meas_extensions_scarlet] - `DM-22027 `_: Re-make Gaia refcat to fix epoch [meas_algorithms] - `DM-22039 `_: Rename dax_ppdb to dax_apdb together with all dependencies [ap_verify, verify, ap_pipe, ap_association] - `DM-22062 `_: Add parquet support to Gen3 Butler [daf_butler] - `DM-22073 `_: Add matplotlib (output) support to Gen3 butler [daf_butler] - `DM-22093 `_: Store begin/end times of ap_pipe in ap_verify [verify] - `DM-22137 `_: Test proximal-ADAM scarlet branch on HSC [meas_extensions_scarlet, pipe_tasks] - `DM-22138 `_: validateDrp arguments (including brightSnr) not passed/parsed correctly [lsst_ci, validate_drp] - `DM-22162 `_: Add metadata writing to PipelineTask execution logic [pipe_base, ctrl_mpexec] - `DM-22173 `_: Support and document support for dates/times in query expressions [daf_butler] - `DM-22192 `_: Remove PixelScaleBoundedField [afw] - `DM-22199 `_: Add decorators for iterating over tests [utils] - `DM-22205 `_: Surprising error when passing a string to queryDimensions [daf_butler] - `DM-22207 `_: Update dev guide and deprecate_pybind11 message to reflect the fact that it works on classes, top [utils] - `DM-22222 `_: Add default processCcd configs to the processCcd pipeline [pipe_tasks] - `DM-22227 `_: measurePhotonTransferCurve.py fails to find raftName [cp_pipe, obs_base] - `DM-22233 `_: Remove astrometry_net_data dependency from pipe_tasks [lsst_ci, pipe_tasks] - `DM-22234 `_: Remove residual python_future usage from stack [obs_cfht, shapelet, display_firefly, ctrl_execute, ap_association, ctrl_orca, meas_modelfit, pipe_drivers, display_matplotlib, ctrl_pool, sphgeom, daf_persistence] - `DM-22235 `_: Fix deprecated collections imports [meas_base, base] - `DM-22255 `_: Move plotting IO out of plotting functions in coaddAnalysis [obs_subaru] - `DM-22256 `_: Remove SkipUnless PyArrow when PyArrow works with macos stack [pipe_tasks] - `DM-22260 `_: Metadata is not persisted when persisting an ExposureCatalog to fits [afw] - `DM-22283 `_: Removed unneeded calls to RegistryTask.addVisits [obs_decam, pipe_drivers] - `DM-22299 `_: Speed up specific diaCalculation plugins using fast pandas functionality [ap_association] - `DM-22301 `_: Printing Gen3 configs in a manner that is easily diff'd [pex_config, ctrl_mpexec] - `DM-22305 `_: Remove eups dependencies from data packages [validation_data_cfht, validation_data_decam, lsst_ci, afwdata] - `DM-22363 `_: Add ability for defefredDatasetHandles to retrieve dataset components [daf_butler] - `DM-22364 `_: Gen 3 Butler cannot be created using daf.butler.Config [daf_butler, ctrl_mpexec] - `DM-22371 `_: Add post-processing tasks to ci_hsc_gen2 [obs_base] - `DM-22372 `_: Fix docstring formatting [pipe_base] - `dm-22377 `_: Slight numeric differences in demo using ubuntu 19 (release candidate v19.0.0.rc1 [lsst_dm_stack_demo] - `DM-22386 `_: Jointcal fails unit tests with Astropy 4 [jointcal] - `DM-22389 `_: verify fails with Astropy 4 [verify] - `DM-22405 `_: Rewrite MeasurePhotonTransferCurveTask data types [cp_pipe, obs_base, obs_lsst] - `DM-22409 `_: Include meas_extensions_scarlet in pipelines.lsst.io [meas_extensions_scarlet] - `DM-22417 `_: Remove python future from obs_base [obs_base] - `DM-22427 `_: latitude and longitude are transposed in AUXTEL_LOCATION in obs_lsst [obs_lsst] - `DM-22453 `_: Update load to safe_load for yaml reading in validate_drp [validate_drp] - `DM-22469 `_: Add fgcmcal documentation tree [fgcmcal] - `DM-22470 `_: Remove all uses of future from fgcmcal and fgcm [fgcm] - `DM-22478 `_: Lightly restructure ap_association tasks and create new DiaObject/DiaSource pre-loading task. [ap_association, ap_pipe] - `DM-22479 `_: SQuaSH being spammed with timestamps [verify] - `DM-22481 `_: Do not delete datastore directory on error [daf_butler] - `DM-22485 `_: Fix docstring heading to make example appear [daf_butler] - `DM-22495 `_: validate_drp reportPerformance.py fails on new metrics [validate_drp] - `DM-22504 `_: Support for lsstDebug functionality in Gen3 middleware [ctrl_mpexec] - `DM-22505 `_: Create run method in ImageDifferenceTask and move all functionality from runDataRef except butler usage [pipe_tasks, ip_diffim] - `DM-22508 `_: Migrate Database interface from prototype to master and add implementations [daf_butler] - `DM-22519 `_: The GroupView iterator is broken [afw] - `DM-22541 `_: Add gen3 support classes to ImageDifferenceTask [pipe_tasks, ip_diffim] - `DM-22550 `_: Handle latiss diffuser filter in obs_lsst [obs_base, obs_lsst] - `DM-22576 `_: Add programmatic header fixups to astro_metadata_translator [astro_metadata_translator] - `DM-22599 `_: Develop PipelineTask unit test framework [pipe_base, daf_butler] - `DM-22605 `_: Add measured QE and read noise to lsstcam [obs_lsst_data, obs_lsst] - `DM-22609 `_: Start using Database in Registry [obs_base, daf_butler] - `DM-22641 `_: ap_verify CI broken [ap_verify] - `DM-22643 `_: convert visualizeVisit to gen3 [pipe_tasks] - `DM-22647 `_: Bug in isrMock.getCamera's use of CameraWrapper [ip_isr] - `DM-22648 `_: Add DcrModel to Gen3 butler [daf_butler] - `DM-22655 `_: Genericize gen2to3.py to be useable with any gen2 repo [pipe_base, obs_lsst, ip_isr, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-22659 `_: Fix bitrot with how brighter-fatter kernels are handled, add option to use new gain values [validation_data_cfht, jointcal, ip_isr, testdata_jointcal, obs_subaru] - `DM-22661 `_: isrTask failed to find master flat due to filter difference [ip_isr] - `DM-22677 `_: Modernize python scripts in daf_butler [daf_butler] - `DM-22680 `_: linearizer attempts to run in gen3 when LinearizerType=None [ip_isr] - `DM-22704 `_: filter 'diffuser' cannot be read [obs_base, obs_lsst] - `DM-22708 `_: Fix decam gen3 ingest [obs_decam, obs_lsst, obs_subaru, obs_base, afw, testdata_decam] - `DM-22718 `_: Typos in obs_lsst Mapper for difference image data products [obs_lsst] - `DM-22727 `_: Add numpy warnings catch to DiaCalculationPlugins [ap_association] - `DM-22728 `_: Handle immutable amplifiers in ip_isr test suite [ip_isr] - `DM-22741 `_: Remove ap_association subtasks from ap_pipe. Change to single DiaPipelineTask subtask. [ap_verify, ap_pipe, ap_association] - `DM-22770 `_: Remove duplication of BaseMapper [pipe_tasks, obs_base] - `DM-22771 `_: Resurrect HSC RC2 Gen3 repo bootstrap [obs_subaru, obs_base, daf_butler] - `DM-22776 `_: Update Gen3 isrTask for BF corrections [daf_butler, ip_isr] - `DM-22777 `_: Prune diaForcedSources in DiaForcedSourceTask [ap_association] - `DM-22788 `_: Responses tests fail with modern responses and old requests [verify] - `DM-22790 `_: pex_config FutureWarning reports wrong line number [pex_config] - `DM-22794 `_: obs_base tests should not use daf_butler test configs [obs_base] - `DM-22796 `_: pipe_tasks installs 200MB of temporary test output [pipe_tasks] - `DM-22797 `_: Various FITS header fixes [pipe_tasks, afw] - `DM-22798 `_: Remove (unused) ability for fgcmcal to run on a full repo without specifying any ids [fgcmcal] - `DM-22800 `_: Fix missing format % in pex_config [pex_config] - `DM-22803 `_: validate_drp and numpy1.17/astropy4/matplotlib3.1 fails [validate_drp] - `DM-22817 `_: Update conda base environment [eigen] - `DM-22818 `_: Matplotlib 3.1 bug triggered by fgcmcal on macOS [fgcmcal] - `DM-22952 `_: Add support of dcr coadds to ImageDifferenceTask in gen3 mode [pipe_tasks, ip_diffim] - `DM-22955 `_: Add spatially varying HSC NB filters to obs_subaru [obs_subaru] - `DM-22958 `_: CModel has unflagged flux=0 fluxSigma=NAN for 1% of sources [meas_modelfit] - `DM-22962 `_: Add a `--walltime` arg to ctrl_pool's BatchArgumentParser [ctrl_pool] - `DM-23008 `_: Add DCR model subfilters to Gen3 registry [pipe_tasks] - `DM-23023 `_: Simplify linearity corrections [obs_decam, obs_subaru, ip_isr] - `DM-23024 `_: Support multi-dataset single file ingest in daf_butler [obs_subaru, obs_base, daf_butler] - `DM-23026 `_: Make sure calibration columns are propagating properly in processCcdWithFakes [pipe_tasks] - `DM-23030 `_: Modify Photometry SDM Functor to use stored calibration value. [pipe_tasks] - `DM-23031 `_: Create dipole separation functor [pipe_tasks] - `DM-23033 `_: Updates to reference catalog creation documentation [meas_algorithms] - `DM-23036 `_: Add ability for fgcmcal to do calibrations on local background-corrected fluxes [fgcmcal] - `DM-23044 `_: PTC task should persist usable linearity models [pipe_tasks, cp_pipe, obs_lsst, ip_isr, obs_base] - `DM-23045 `_: Validate linearity correction [cp_pipe, ip_isr] - `DM-23048 `_: validate_drp uses deprecated matplotlib call [validate_drp] - `DM-23054 `_: Resolving formatting issues in lsst.utils.tests documentation [utils] - `DM-23056 `_: Suppress FutureWarnings from LSST code [meas_algorithms, utils, afw, pipe_tasks] - `DM-23062 `_: Preserve input headers [obs_base] - `DM-23063 `_: Request for ability to turn off checksumming in Gen3's POSIX datastore [daf_butler] - `DM-23065 `_: Test fgcmcal on NB0387 HSC data [fgcmcal, fgcm] - `DM-23067 `_: Delete commented code in tests/test_matchBackgrounds.py [pipe_tasks] - `DM-23071 `_: Update default aperture correction configuration in hsc to select bright stars [obs_subaru] - `DM-23073 `_: makeBrighterFatterKernel uses wrong PTC fit type [cp_pipe] - `DM-23074 `_: Make the schema of the output Object parquet files input-independent [pipe_tasks, obs_subaru] - `DM-23077 `_: Update default interpolation for Curve classes [meas_algorithms] - `DM-23078 `_: Add sky objects to the single frame processing step [lsst_dm_stack_demo, pipe_tasks] - `DM-23079 `_: Move opaque table Registry code into helper classes [daf_butler] - `DM-23080 `_: Move dimension Registry code into helper classes [daf_butler] - `DM-23083 `_: Update large masks for BF convolution issues [obs_subaru, ip_isr] - `DM-23085 `_: Deprecate SourceDetectionTask.makeSourceCatalog [synpipe, meas_algorithms, pipe_tasks, testdata_deblender, ip_diffim] - `DM-23090 `_: Update LATISS filters in obs_lsst to match commissioning filters [obs_lsst] - `DM-23091 `_: HSC-Y failed on w_2020_02 [obs_subaru] - `DM-23102 `_: Update python types for matchVisits and objectTable [obs_base] - `DM-23103 `_: Update functor unittests to no longer rely on test_multilevel_parq.csv.gz [pipe_tasks] - `DM-23104 `_: Augment ObjectTable to be useable for QA [pipe_tasks, obs_subaru] - `DM-23112 `_: Fix cgUtils.overlayCcdBoxes for rename of getAmpInfoCatalog [afw] - `DM-23129 `_: Update obs_base ingest RawFileData for multi-dataId files [obs_base] - `DM-23131 `_: Fix "unordered" map documentation in DetectorCollection getters [afw] - `DM-23149 `_: Fix LATISS data ingest for on sky data [obs_lsst] - `DM-23166 `_: Add __all__ to lsst.utils.deprecated module. [utils] - `DM-23171 `_: Add exposure group to metadata translator [astro_metadata_translator, obs_lsst] - `DM-23172 `_: Fix flake8 violations in astshim [astshim] - `DM-23173 `_: Implement RFC-650: update packages to prefer binary operators at the beginning of continued lines [display_ds9, skymap, pex_config, obs_test, ip_diffim, base, obs_decam, meas_extensions_simpleShape, meas_extensions_psfex, meas_base, daf_persistence, meas_algorithms, ap_association, geom, meas_extensions_scarlet, meas_astrom, meas_deblender, coadd_utils, obs_sdss, pex_exceptions, pipe_base, daf_base, sphgeom, pipe_tasks, afw, ap_pipe] - `DM-23174 `_: Consolidate daf_butler test code [daf_butler] - `DM-23178 `_: Convert some of afw to use f strings [afw] - `DM-23179 `_: revised color terms for HSC [obs_subaru] - `DM-23192 `_: Use correct weight maps in scarlet [scarlet, meas_extensions_scarlet, proxmin] - `DM-23195 `_: DM-21221 broke cp_pipe due to lack of tests [cp_pipe] - `DM-23196 `_: Add calibration storage to multiBand.py [pipe_tasks, meas_base] - `DM-23203 `_: Add tabular output to translate_header [astro_metadata_translator] - `DM-23206 `_: validate_drp crashes when trying to apply external skyWcs [validate_drp] - `DM-23208 `_: Add exposure group to gen3 registry [obs_base, daf_butler] - `DM-23212 `_: pipetask run with multiple "-i" command line arguments fails [ctrl_mpexec] - `DM-23213 `_: It's necessary to restart the butler to see newly ingested files [pipe_tasks] - `DM-23222 `_: Fix OBJECT ENGTEST date and RADEC [obs_lsst] - `DM-23223 `_: Allow translate_header to dump the fixed header [astro_metadata_translator] - `DM-23224 `_: Cross-check the schema column names in the Object table [obs_subaru] - `DM-23231 `_: Sort out visit vs exposure ID in gen 2 butler [obs_lsst] - `DM-23237 `_: Strange image types ingested for LATISS images [obs_lsst] - `DM-23238 `_: Add per row overscan removal [ip_isr] - `DM-23249 `_: New decam ingest tests need skipif for testdata_decam [obs_decam] - `DM-23258 `_: matchedVisitMetricsTask will crash if any visits have a missing source catalog [validate_drp] - `DM-23275 `_: Restore outfile option for butler makeRepo [daf_butler] - `DM-23277 `_: Update ImageDifferenceTask to remove v20 deprecation warnings [pipe_tasks, ip_diffim] - `DM-23278 `_: Fix RADEC for LATISS data on 27th Jan [obs_lsst] - `DM-23281 `_: FILTER and GRATING not concatenated in LATISS data [obs_lsst] - `DM-23282 `_: LATISS mapper doesn't find defects [obs_lsst] - `DM-23310 `_: Some ComCam images do not have LSST_NUM header [obs_lsst] - `DM-23313 `_: Fix dominant failure mode in deblending sources with 2-components [meas_extensions_scarlet] - `DM-23329 `_: Fix controller code in phosim and imsim data [obs_lsst] - `DM-23331 `_: Add default fgcmcal configuration files for HSC processing [fgcmcal, fgcm, obs_subaru] - `DM-23342 `_: Fix ingestDriver [pipe_drivers, pipe_tasks] - `DM-23343 `_: Mark Synpipe as deprecated [synpipe] - `DM-23352 `_: Add jointcal, skyCorr to forcedPhotCcd [pipe_tasks, pex_config, obs_subaru, obs_base, daf_butler, meas_base] - `DM-23359 `_: Allow pex_config configs to use __file__ [pex_config] - `DM-23362 `_: Old LATISS data has missing OBSID/DAYOBS [obs_lsst] - `DM-23364 `_: Flag edge sources and shift them in meas_extensions_scarlet [meas_extensions_scarlet] - `DM-23369 `_: Remove lsst.ip.isr.addDistortionModel [obs_cfht, ip_isr] - `DM-23371 `_: Remove lsst.obs.base.CameraMapper._extractAmpId [obs_subaru, obs_base] - `DM-23380 `_: Remove lsst.fgcmcal.FgcmFitCycleConfig.pixelScale [fgcmcal] - `DM-23396 `_: Function "overscanCorrection" in "isrFunctions.py" needs refactoring [obs_decam, obs_subaru, obs_lsst, ip_isr] - `DM-23397 `_: Remove refCols entries in obs_lsst calibrations [obs_lsst] - `DM-23414 `_: lsst_ci fails with astropy 4 and numpy >=1.17 [validate_drp] - `DM-23420 `_: ap_association does not work with numpy 1.18 and pandas 1.0 [ap_association] - `DM-23426 `_: Use PipelineTask test framework for MetricTask [ap_association, verify, ip_diffim] - `DM-23436 `_: apdb creation instructions outdated [ap_association, ap_pipe, ip_diffim] - `DM-23448 `_: Sort out defect generation and ingest for LATISS (and others) [cp_pipe, pipe_tasks, obs_lsst] - `DM-23449 `_: Butler can't get bias for LATISS using expId [obs_lsst] - `DM-23477 `_: pipe_base ScalarError can't be pickled [pipe_base] - `DM-23490 `_: Correct plate scale and boresight position for LATISS [obs_lsst] - `DM-23498 `_: Explore using PSF-matched warps for DCR coadds [pipe_tasks] - `DM-23503 `_: Butler gen3 datastore templates should handle slashes in data Ids [daf_butler] - `DM-23509 `_: obs_lsst failing LATISS plate scale test [obs_lsst] - `DM-23526 `_: Fix fgcmcal issues exposed during PDR2 run [fgcmcal, fgcm, obs_subaru] - `DM-23558 `_: DCR coadds are missing PhotoCalib [pipe_tasks, ip_diffim] - `DM-23566 `_: Provide a method to get dataset types from a Gen 2 butler [daf_persistence] - `DM-23584 `_: expId has wrong number of digits in comCamMapper.yaml template [obs_lsst] - `DM-23589 `_: Update LATISS filters in obs_lsst [obs_lsst] - `DM-23600 `_: Gen2 butler can't get bias for ComCam using expId [obs_lsst] - `DM-23616 `_: Run converted ap_verify testdata through gen3 pipeline [obs_decam, pipe_tasks, pipe_base, ap_pipe_testdata] - `DM-23623 `_: Measure crosstalk coefficients for AuxTel chip and add for use. [obs_lsst, obs_base, ip_isr] - `DM-23627 `_: Missing psfMatched_nImage definition [obs_base] - `DM-23630 `_: fgcmcal failure Ubuntu [fgcmcal] - `DM-23635 `_: Confusing help text for pipetask [ctrl_mpexec] - `DM-23638 `_: Add filename attribute to ParquetTable [pipe_tasks] - `DM-23651 `_: ap_pipe calls some deprecated things [validation_data_cfht, meas_algorithms, cp_pipe, ap_association, obs_lsst, obs_subaru, ip_diffim, afw, obs_decam, meas_base] - `DM-23652 `_: Add documentation for --show history to pipe_base [pipe_base] - `DM-23663 `_: Make insertFakes work for HSC artificial star tests [pipe_tasks, obs_base] - `DM-23671 `_: Improve dataset deletion in Gen3 butler [daf_butler] - `DM-23678 `_: Bug in s3Datastore when using temporary file [daf_butler] - `DM-23681 `_: Check, in the unit tests, that fitPtcAndNonLinearity returns what it is supposed to return. [cp_pipe] - `DM-23699 `_: Update fgcmcal default config format to remove possibility of index errors [fgcmcal, fgcm, obs_subaru] - `DM-23701 `_: pipetask-produced DOT for pipelines should show prerequisite inputs [ctrl_mpexec] - `DM-23702 `_: IsrTask shoud use regular Input for raw data [ip_isr] - `DM-23703 `_: ap_association fails if a new visit has 0 new diaObjects [ap_association] - `DM-23711 `_: Allow butler configs to use environment variables to find other configs [daf_butler] - `DM-23718 `_: Replace dots in gen3 file names [daf_butler] - `DM-23722 `_: Validate dataset type definitions in pipeline task connections [daf_butler, pipe_base, ip_isr] - `DM-23728 `_: Cleanup ci_hsc_gen2 to use new convert script instead of custom one [obs_base] - `DM-23729 `_: Generalize dataset to formatter mapping in obs packages [obs_decam, obs_base, daf_butler] - `DM-23762 `_: Check bad amps in LSSTCam are being tracked in the defects file [cp_pipe] - `DM-23778 `_: Write config and tests for obs_lsst gen2-gen3 convert [pipe_tasks, obs_lsst, obs_base, daf_butler, obs_lsst_data] - `DM-23798 `_: Fix scons install of bin [sconsUtils] - `DM-23827 `_: wrong python type for matchVisits_config [obs_base] - `DM-23828 `_: Create small test decam gen2 repo with calibs for test_convert2to3.py [testdata_decam, obs_decam] - `DM-23830 `_: Add cbp package to lsst_distrib [lsst_distrib] - `DM-23835 `_: Update environment with conda-forge third parties and conda compilers [obs_cfht, display_ds9, meas_modelfit, validate_drp, pex_config, skymap, obs_test, ip_diffim, base, meas_extensions_simpleShape, meas_extensions_psfex, proxmin, meas_base, daf_persistence, log, scarlet, sconsUtils, pex_policy, astro_metadata_translator, meas_extensions_photometryKron, meas_algorithms, ap_association, jointcal, ip_isr, obs_lsst, ap_verify, geom, meas_extensions_scarlet, meas_astrom, display_matplotlib, meas_deblender, meas_extensions_shapeHSM, coadd_utils, shapelet, psfex, utils, obs_sdss, pex_exceptions, pipe_base, daf_base, verify, obs_base, daf_butler, astshim, sphgeom, synpipe, lsst_dm_stack_demo, fgcmcal, pipe_tasks, jointcal_cholmod, fgcm, obs_subaru, firefly_client, obs_ctio0m9, afw, ctrl_pool] - `DM-23836 `_: DCR templates have incorrect variance [ip_diffim] - `DM-23846 `_: YAML files with python/object/apply fail in pyyaml>5.2.1 [obs_lsst] - `DM-23873 `_: Typo in cp_pipe makeBrighterFatterKernel.py [cp_pipe] - `DM-23928 `_: PropertySet not robust against duplicate and empty FITS cards [afw] - `DM-23931 `_: Allow butler.makeRepo to complain if a config already exists [daf_butler] - `DM-23946 `_: Change localWcs plugins to use a localGnomonicWcs transform. [pipe_tasks, meas_base] - `DM-23952 `_: Remove scarlet and proxmin from meas_deblender [pipe_tasks, meas_deblender] - `DM-23959 `_: CameraMapper._standardizeExposure should patch header [obs_base] - `DM-23971 `_: Undo TE1 "regression" [validate_drp] - `DM-23975 `_: Subfilter type inconsistency [pipe_tasks, daf_butler] - `DM-23976 `_: Move gen3 generic curated calibrations ingest code to obs_base [obs_decam, obs_subaru, obs_base, obs_lsst] - `DM-23980 `_: Standardize Gen3 instrument class names and location [obs_subaru, obs_lsst] - `DM-24013 `_: Fix bug in --output-run handling introduced in DM-21849 [ctrl_mpexec] - `DM-24018 `_: Failure to flatten or filter chained collections in queries [daf_butler] - `DM-24027 `_: CalibDate misinterpreted in curated calibration ingest [pipe_tasks] - `DM-24055 `_: Speed up table parsing in Defects [meas_algorithms] - `DM-24060 `_: Improve table creation efficiency in Defects [meas_algorithms] - `DM-24062 `_: Write Source Table transform tasks and Source.yaml for obs_subaru [pipe_drivers, pipe_tasks, obs_subaru, obs_base] - `DM-24244 `_: w_2020_13 makeButlerRepo.py missing left parens error with Oracle [obs_base, daf_butler] - `DM-24250 `_: Improve startup speed of fgcmcal and add checkpointing for restarts [fgcmcal] - `DM-24259 `_: Create “stub“ Gen2 HSC dataset for CI testing [ap_verify, obs_subaru, ap_pipe] - `DM-24260 `_: Create Gen3 versions of ap_verify datasets [obs_base] - `DM-24273 `_: pex_config test_history embeds too much tester knowledge [pex_config] - `DM-24285 `_: fitsExposureFormatter fails to read "Exposure" entries correctly [daf_butler] - `DM-24288 `_: Prototype datastore changes to support dataset disassembly/assembly [daf_butler] - `DM-24290 `_: Enable running ci_hsc_gen3 contents on AWS [pipe_tasks, daf_butler] - `DM-24314 `_: Support following links in Butler operations [daf_butler] - `DM-24324 `_: Generate alert packets in the alert production pipeline [ap_verify, ap_association] - `DM-24330 `_: add ability to run an obs_base command via the butler command [obs_base, daf_butler] - `DM-24334 `_: Create quickLookExp exposure type to support commissioning [obs_base, obs_lsst] - `DM-24337 `_: Fix raw formatter gen3 breakage [obs_base] - `DM-24346 `_: Don't add fakes outside of the valid polygon area for the ccd [obs_subaru] - `DM-24347 `_: Allow component gets in gen3 butler to be None [obs_base, daf_butler] - `DM-24352 `_: Add auto transfer mode to gen3 ingest [obs_base, daf_butler] - `DM-24365 `_: Add relative symbolic link transfer mode to Gen 3 ingest [obs_base, daf_butler] - `DM-24370 `_: Support extensible scheduling in pipetask [ctrl_mpexec] - `DM-24371 `_: Implement fixed correction fixed PSF support decorrelation afterburner [pipe_tasks, ip_diffim] - `DM-24375 `_: Check time round trip issues with new time format [daf_butler] - `DM-24376 `_: afw should warn when FITS headers case may be modified (cfitsio > 3380) [afw] - `DM-24378 `_: Store instrument class with gen3 instrument registration and add API [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-24379 `_: WriteSourceTable: add option to get local calib columns from calexp (or external calib) [pipe_tasks, obs_base] - `DM-24385 `_: Optimize posixStorage.search with relative path [daf_persistence] - `DM-24392 `_: Update testdata_jointcal to include Gaia+PS1 refcats [testdata_jointcal, jointcal] - `DM-24404 `_: Squash astropy ecsv read warnings [meas_algorithms] - `DM-24429 `_: Investigate change in fracDiaSourcesToSciSources in ap_verify CI [obs_decam] - `DM-24434 `_: symlink bug in posixDatastore.py on Ubuntu [daf_butler] - `DM-24435 `_: Freezing a config locks the registry(ies) of other instances of that config [pex_config] - `DM-24450 `_: Fix parsing of wavefront sensors [obs_lsst] - `DM-24456 `_: validateDrp matchedVisitMetricsTask broken [validate_drp] - `DM-24466 `_: Can't run processCcd on multiple CCDs [obs_subaru] - `DM-24472 `_: Regenerate Gaia DR2 catalogs to correct coordinate error fields [testdata_jointcal, meas_algorithms] - `DM-24495 `_: Convert config overrides to use file in several obs packages [obs_decam, obs_subaru, obs_lsst] - `DM-24515 `_: Refactor gen3 butler.prune [daf_butler, ctrl_mpexec] - `DM-24517 `_: Reading of raws from existing imsim & phosim repos broken by DM-23584 [obs_lsst] - `DM-24523 `_: ap.verify.ingestion._findMatchingFiles excludes directories [ap_verify] - `DM-24537 `_: Add Isr calibration base class [ip_isr] - `DM-24543 `_: handle conflicting CLI subcommand names [obs_base, daf_butler] - `DM-24545 `_: remove unneeded explicit command names [obs_base, daf_butler] - `DM-24546 `_: Improve explanation of calibration and fluxCalibRadius in insertFakes.py code. [pipe_tasks] - `DM-24553 `_: Make a driver script for processCcdWithFakes [pipe_tasks, obs_subaru, obs_base] - `DM-24555 `_: Add ability to retrieve various Exposure components in gen3 butler. [daf_butler] - `DM-24556 `_: Add normalize method to Defects [cp_pipe, meas_algorithms] - `DM-24559 `_: write_curated_calibrations is not passing the butler registry to getInstrument [obs_decam, obs_subaru, obs_base, obs_lsst] - `DM-24560 `_: make 'repo' an argument, by convention always the first. [obs_base, daf_butler] - `DM-24565 `_: Un-deprecate old APIs related to afw.math.Background [afw] - `DM-24569 `_: Update obs_lsst with changes from observing runs [obs_lsst] - `DM-24570 `_: Fix lower and mixed-case metadata written to FITS files [meas_algorithms, pipe_tasks, meas_base] - `DM-24584 `_: create an ingestRaws butler command [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-24594 `_: Create CollectionsManager implementation that uses an integer primary key [daf_butler] - `DM-24612 `_: Add indexes to dataset_collection tables [daf_butler] - `DM-24614 `_: Move dataset_location tables into manager/storage hierarchy [daf_butler] - `DM-24616 `_: Stop writing to gen2 butler directory during 2to3 conversion [obs_base, obs_lsst] - `DM-24618 `_: Fix leftover case warnings from FITS changes [meas_extensions_photometryKron] - `DM-24658 `_: Improve error reporting in connection string [daf_butler] - `DM-24664 `_: Resurrect HSC RC2 Gen3 repo bootstrap, part 2 [obs_decam, obs_subaru, obs_base, obs_lsst] - `DM-24698 `_: Clean up (at least) doc bug introduced in DM-21764 [daf_butler] - `DM-24708 `_: Quick fix of type mismatch runtime errors in decorrelation afterburner [ip_diffim] - `DM-24716 `_: Remove obs_lsstSim from lsst_ci package [lsst_ci] - `DM-24721 `_: w_2020_18 butler create does not work with Oracle [daf_butler] - `DM-24734 `_: Revive and profile RC2 QuantumGraph generation [pipe_base] - `DM-24747 `_: Remove obsolete Doxygen configuration [base] - `DM-24756 `_: Make sigma clipping configurable in PTC task [cp_pipe] - `DM-24768 `_: build race condition in daf_persistence [daf_persistence] - `DM-24769 `_: Improve connection string matching. [daf_butler] - `DM-24780 `_: Initial mypy configuration for daf_butler [daf_butler] - `DM-24782 `_: Add Sky Source Flag to SourceTable [obs_subaru] - `DM-24786 `_: New component column in datastore is too small [daf_butler] - `DM-24796 `_: pipetask's graphviz dot files need to quote component dataset type names [ctrl_mpexec] - `DM-24797 `_: Store per-run information (configs, software versions) in butler repo [pipe_base, ctrl_mpexec] - `DM-24804 `_: create a 'convert' butler command [obs_decam, obs_subaru, obs_base, obs_lsst] - `DM-24807 `_: Investigate use of mypy for daf_butler datastores [daf_butler] - `DM-24829 `_: w_2020_19 butler write-curated-calibrations doesn't work with Oracle [daf_butler] - `DM-24851 `_: Change Datastore.getUri to Datastore.getURIs [daf_butler, ctrl_mpexec] - `DM-24857 `_: Add processCcdWithFakesDriver to bin.src [pipe_drivers] - `DM-24862 `_: Update with latest scarlet and proxmin and test for performance [scarlet, meas_extensions_scarlet, proxmin] - `DM-24864 `_: Update processCcdWithFakes to add the sky_source column. [pipe_tasks] - `DM-24892 `_: Fix bug in gen2to3 when only special dataset types are being converted [obs_base] - `DM-24906 `_: Pin version of flake8 (and, if necessary, other tools) used in Travis checks [obs_cfht, cp_pipe, display_ds9, meas_extensions_convolved, meas_modelfit, pex_config, validate_drp, obs_test, ip_diffim, skymap, meas_extensions_simpleShape, pipe_drivers, base, meas_extensions_psfex, meas_base, obs_decam, daf_persistence, log, meas_algorithms, sconsUtils, pex_policy, astro_metadata_translator, meas_extensions_photometryKron, display_firefly, ap_association, jointcal, ip_isr, obs_lsst, ap_verify, geom, meas_extensions_scarlet, meas_astrom, display_matplotlib, meas_deblender, meas_extensions_shapeHSM, coadd_utils, shapelet, ctrl_execute, lsst_ci, utils, obs_sdss, pex_exceptions, pipe_base, ctrl_mpexec, ap_pipe, daf_base, verify, obs_base, daf_butler, astshim, sphgeom, synpipe, lsst_dm_stack_demo, fgcmcal, pipe_tasks, ctrl_platform_lsstvc, obs_subaru, obs_ctio0m9, afw, ctrl_pool, ctrl_orca] - `DM-24907 `_: Conda env missing psutil [ap_association, utils] - `DM-24908 `_: Change keys for imsim obs_lsst back to visit from expId [obs_lsst] - `DM-24920 `_: improve butler CLI mocking [obs_base, daf_butler] - `DM-24923 `_: eliminate redundant class name & name requirement in butler command test cases [obs_decam, obs_subaru, obs_base, obs_lsst] - `DM-24924 `_: Add new telescope name for ComCam and LSSTCam to obs_lsst [obs_lsst] - `DM-24935 `_: Add support for DataFrame and Table in ScienceSourceSelectorTask and ReferenceSourceSelectorTask [meas_algorithms] - `DM-24937 `_: move implementation of remaining butler commands to script folder [obs_base, daf_butler] - `DM-24940 `_: Move NamedKeyDict and NamedValueSet out of utils and clean up typing [pipe_base, daf_butler] - `DM-24946 `_: Propagate detectCoaddSources configs into multiBandDriver [obs_subaru] - `DM-24972 `_: Switch to using an explicit export of conda packages [scarlet] - `DM-24975 `_: Use GCS instead of AWS-S3 for the Butler datastore [daf_butler] - `DM-24976 `_: Rework testdata_jointcal dependencies [fgcmcal, testdata_jointcal, jointcal] - `DM-24979 `_: config_file_option does not save the 'help' input argument [daf_butler] - `DM-24980 `_: Fix flake8 failure in daf_base DateTime test [daf_base] - `DM-24985 `_: 'file' needs to be an @property when it uses a testdata pacakge [obs_decam, obs_subaru] - `DM-24993 `_: Drop afw case warnings down to DEBUG when reading fits files [afw] - `DM-25000 `_: Using components in PipelineTaskConnections no longer seems to work. [daf_butler, ctrl_mpexec] - `DM-25014 `_: Coadds converted to Gen 3 without abstract_filter [obs_base] - `DM-25015 `_: Create sublogger for pipeBase timer [pipe_base] - `DM-25016 `_: DM-21724 unpickling error appears again [pipe_base, ctrl_mpexec] - `DM-25020 `_: Update Sources.yaml with correct local background [obs_subaru] - `DM-25040 `_: ap_association uses physical filter in Gen 3 [ap_association] - `DM-25072 `_: Move gen3 afw formatters to obs_base [obs_base, daf_butler] - `DM-25080 `_: Finish static typing in daf.butler.registry [obs_base, daf_butler] - `DM-25135 `_: Restore HSC-R/HSC-R2 colorterms [obs_subaru] - `DM-25149 `_: Fix w18 ISR regression [ip_isr] - `DM-25152 `_: butler ingest raws does not register translators [obs_base] - `DM-25153 `_: Make it clear that gen3 instrument class paths don't need .instrument [obs_lsst, obs_subaru, obs_base, daf_butler, obs_decam] - `DM-25170 `_: Fix to get predicted butler URIs [daf_butler] - `DM-25177 `_: pipelines.lsst.io broken by missing safeFileIo in daf_butler [daf_butler] - `DM-25184 `_: Make obs_subaru config overrides play nice with new jointcal filterMap config [obs_subaru] - `DM-25187 `_: daf_butler registry EllipsisType breaks pipelines.lsst.io doc build [daf_butler] - `DM-25221 `_: Backport fixes to w_2020_22 [psfex, jointcal, obs_lsst, obs_subaru, daf_butler]