Tickets Addressed in Release 21.0.0

  • DM-4752: Build on Mac very slow due to running fc-list [sconsUtils]

  • DM-5304: Use psfex as the default PSF determiner [meas_extensions_psfex, obs_lsst, obs_subaru, pipe_tasks]

  • DM-5664: Delete or document and test config/psfex.py [meas_extensions_psfex]

  • DM-5684: Unused variables in meas_extensions_psfex library code [meas_extensions_psfex]

  • DM-6571: Remove the ‘makeSourceCatalog’ alias for the run method of SourceDetectionTask [meas_algorithms]

  • DM-8591: Follow-up pybind11 behavior with numpy.int64s as indices in Python 3 [afw]

  • DM-9081: testExposure.testGetWcs docstring is wrong, and tests should be assertIsNone [afw]

  • DM-9294: makeCamera.py has undefined variables [obs_decam]

  • DM-10339: Provide a named method to return the number of entries in a PropertyList [daf_base]

  • DM-13353: Prototype implementation of (optionally) compressed Datasets in Datastore [daf_butler, obs_base]

  • DM-14290: Do not raise generic Exceptions [afw, ip_diffim, ip_isr, utils]

  • DM-15465: Make utils optional for daf_butler [daf_butler]

  • DM-15815: convert daf_base to numpydoc status [daf_base]

  • DM-17538: Generate stack-produced (rather than CP) calibration products for DECam [obs_decam, pipe_tasks, testdata_decam]

  • DM-19389: Convert lsst.geom to numpydoc [geom]

  • DM-20070: Enhance PTC task to use Pierre^2’s code [cp_pipe, ip_isr, obs_base, obs_lsst]

  • DM-20770: Unhelpful error message from PessimisticPatternMatcherB [meas_astrom]

  • DM-20922: ap_verify can’t handle –id with empty argument [ap_verify]

  • DM-20994: Write how to make gen3 obs package guide [obs_base]

  • DM-21212: Update existing cp_pipe tasks to pipelineTasks [cp_pipe, daf_butler, obs_lsst, obs_subaru]

  • DM-21319: Add first metric to fakesAnalysis [obs_subaru, verify_metrics]

  • DM-21333: Implement afw.image.Filter replacement(s) [afw]

  • DM-21706: Add filter definitions to support BOT work [obs_lsst]

  • DM-21860: Add basic Gen3 butler support to obs_cfht [obs_cfht]

  • DM-21915: Support Gen 3 ingestion of ap_verify datasets [ap_verify, ap_verify_testdata, obs_base]

  • DM-21919: Run ap_verify end-to-end in Gen 3 [ap_association, ap_pipe, ap_verify, ctrl_mpexec, ip_diffim, pipe_base, pipe_tasks, verify]

  • DM-21939: Create Gen 3 AP Pipeline [ap_association, ap_pipe, pipe_tasks]

  • DM-22114: Remove deprecated makeDistortedTanWcs [afw]

  • DM-22187: Convert DPDD change requests to tickets and annotate Object.yaml [obs_subaru]

  • DM-22221: Create Task for finding satellite trails as a subtask of CompareWarpAssembleCoadd [pipe_tasks]

  • DM-22255: Move plotting IO out of plotting functions in coaddAnalysis [obs_lsst, obs_subaru]

  • DM-22310: Implement Astrometric Performance Metric AB1 in validate_drp [verify_metrics]

  • DM-22577: Create simple stack demo based on supported obs package [lsst_ci]

  • DM-22578: Remove obs_sdss and lsst_dm_stack_demo from stack builds [lsst_apps, lsst_ci, lsst_obs]

  • DM-22663: Reimplement make_apdb.py for Gen 3 [ap_pipe, ap_verify, pipe_base]

  • DM-22814: Remove afw APIs deprecated in DM-17566 [afw, daf_persistence, ip_diffim, meas_algorithms, meas_astrom, meas_base, meas_deblender, meas_extensions_psfex, meas_extensions_shapeHSM, meas_modelfit, obs_base, obs_decam]

  • DM-23173: Implement RFC-650: update packages to prefer binary operators at the beginning of continued lines [ip_isr]

  • DM-23276: Fix repr for gen3 Registry to output SQLite path [daf_butler]

  • DM-23302: Add ci_cpp package to allow calibration product generation to be tested [cp_pipe, ip_isr, obs_lsst]

  • DM-23344: Remove Synpipe from lsst_distrib [lsst_distrib]

  • DM-23529: Add cat to lsst_distrib (as sdm_schemas) [lsst_distrib]

  • DM-23551: skyCorrection error “No good pixels in image array” HSC visit=137268^137288 [pipe_drivers]

  • DM-23833: Add cbp package to pipelines_lsst_io [cbp]

  • DM-23983: Cannot apply crosstalk in Gen 3 DECam processing [daf_butler, ip_isr, obs_base, obs_decam, obs_decam_data, obs_lsst, obs_subaru, pipe_tasks, testdata_decam]

  • DM-23992: Cannot load refcats in Gen 3 DECam processing [meas_algorithms]

  • DM-24247: butler validation error in ci_hsc_gen3 [daf_butler]

  • DM-24262: Run HSC AP processing in CI using Gen 3 [ap_verify]

  • DM-24277: Apply proper motion and parallax while loading refcats in Jointcal [jointcal, meas_algorithms]

  • DM-24300: Create Gen3 repo from testdata_jointcal [testdata_jointcal]

  • DM-24319: Add support for new parquet source tables in fgcmcal [fgcmcal, obs_base, obs_subaru]

  • DM-24320: Add parquet source tables to testdata_jointcal [testdata_jointcal]

  • DM-24325: Include CCDData cutouts in alert packets [ap_association]

  • DM-24327: Include coadd cutouts in alert packets [ap_association, ap_pipe, obs_base, obs_decam, obs_subaru, pipe_tasks]

  • DM-24329: add sphinx documentation of the Click CLI in daf_butler [daf_butler]

  • DM-24331: Fix ImageDifferenceTask and related v21 deprecation warnings [pipe_tasks]

  • DM-24355: Make Database.sync transaction-friendly [daf_butler, obs_base, obs_cfht, obs_decam, obs_lsst, obs_subaru, skymap]

  • DM-24414: Implement –prune-replaced option in ctrl_mpexec [ctrl_mpexec, daf_butler]

  • DM-24438: Enable running ap_verify against ticket branches in Jenkins [verify]

  • DM-24439: Remove pex_policy usage from daf_persistence [daf_persistence]

  • DM-24475: Refactor S3/Posix usage to simplify ingest and export [daf_butler]

  • DM-24575: Add observational and detector metadata to Registry dimension tables [ap_verify_testdata, daf_butler, obs_base]

  • DM-24585: Update policies in obs_lsst to work with visualizeVisit.py [obs_lsst]

  • DM-24592: Get astrometry working for the AuxTel [obs_base, obs_lsst]

  • DM-24660: Improve Gen3 schema backwards compatibility [daf_butler]

  • DM-24696: Flatten Ap alert schemas [alert_packet, ap_association]

  • DM-24703: Make linearity a subclass of lsst.ip.isr.IsrCalib [cp_pipe, ip_isr, obs_subaru]

  • DM-24731: Create a test suite for (Dcr)AssembleCoaddTask [pipe_tasks]

  • DM-24760: Migrate measureCrosstalk.py to cp_pipe [cp_pipe, ip_isr, obs_base]

  • DM-24830: Create dataset class for processed bright star stamps [meas_algorithms, obs_base, obs_subaru]

  • DM-24844: Migrate ap_verify_testdata to obs_lsst [ap_verify, ap_verify_testdata]

  • DM-24855: Exclude sky sources from Ratio of DIASources to Direct Image Sources metric [ip_diffim]

  • DM-24926: Attempt to get footprints from afw diaSourceCatalog and automatically set the alert cutout size. [ap_association]

  • DM-24939: Remember which dataset types are in various collections [daf_butler]

  • DM-24961: Implement removal of SuprimeCam support from obs_subaru [obs_subaru, obs_subaru_data]

  • DM-25010: Make Alert serialization optional in diaPipe. [ap_association, ap_verify]

  • DM-25028: change butler create –config-file to –seed-config [daf_butler, obs_base]

  • DM-25030: make an import butler subcommand [daf_butler, obs_base]

  • DM-25101: butler ingest-raws CLI ingesting subsets of files in directory [daf_butler, obs_base]

  • DM-25114: Update internal fgcm math to always use 64-bit floats [fgcm, fgcmcal]

  • DM-25115: Implement zogy subtraction with non-spatial PSF [ip_diffim, pipe_tasks]

  • DM-25156: Gen 2->3 conversion of DECam repositories can give duplicate defects [obs_base, obs_lsst, obs_subaru]

  • DM-25159: Have jointcal fail if the chi2 increases dramatically between steps [jointcal]

  • DM-25164: Add diaForceSources to diaPipe loader and to alert packets [ap_association, dax_apdb]

  • DM-25171: Update translator for phosim to use nominal weather defaults [obs_lsst]

  • DM-25177: pipelines.lsst.io broken by missing safeFileIo in daf_butler [daf_butler]

  • DM-25178: Add parameter support to formatter configuration [daf_butler]

  • DM-25180: Add ingest time to registry [daf_butler]

  • DM-25184: Make obs_subaru config overrides play nice with new jointcal filterMap config [jointcal, obs_subaru]

  • DM-25187: daf_butler registry EllipsisType breaks pipelines.lsst.io doc build [daf_butler]

  • DM-25192: Switch from Travis to GitHub Actions in daf_butler [daf_butler]

  • DM-25196: make query-collections and query-dataset-types butler commands [daf_butler]

  • DM-25199: Distribute lsst.alert.packet on PyPI [alert_packet]

  • DM-25208: Fix broken test in lsst/alert_packet master [alert_packet]

  • DM-25210: Fix psfex regression in w18 [psfex]

  • DM-25216: ap_verify failure: ap_verify.py: error: no config field: diaPipe.doSerializeAlerts [ap_verify]

  • DM-25221: Backport fixes to w_2020_22 [daf_butler, jointcal, obs_lsst, obs_subaru, psfex]

  • DM-25222: Error with –init-only –skip-existing [ctrl_mpexec]

  • DM-25225: Fix a typo from DM-25016 writing the wrong qgraph [ctrl_mpexec]

  • DM-25236: Temporarily remove quantum tables from Registry [ctrl_mpexec, daf_butler]

  • DM-25246: Make version of RC2 bootstrap script that can convert reruns as well [daf_butler]

  • DM-25253: Warn when weights are ignored in statisticsStack [afw]

  • DM-25280: ap_association broken by changes to alert_packet [alert_packet, ap_association]

  • DM-25285: Refactor the fakes insertion code to make it easier to keep the configs up to date with the rest of the stack. [obs_subaru, pipe_tasks]

  • DM-25292: ComCam Instrument.name and translator instrument name differ [obs_lsst]

  • DM-25317: cleanup items for butler CLI [daf_butler, obs_base, obs_lsst]

  • DM-25323: Switch PropertySet in gen3 to YAML formatter [daf_butler, obs_base]

  • DM-25325: Write Packages object as YAML [base]

  • DM-25327: Change gen3 to write Packages in YAML format [daf_butler, obs_base]

  • DM-25329: Fix LATISS instrument pipeline definitions for cp_pipe [obs_lsst]

  • DM-25345: base fails to build standalone [base]

  • DM-25347: Integer out of range when trying to convert RC2 rerun using postgresql [daf_butler]

  • DM-25354: Extend registry schema to support metadata/configuration [daf_butler]

  • DM-25355: Add support for defining schema version defined by daf_butler [daf_butler]

  • DM-25373: Write Psf trampoline class [afw, meas_algorithms]

  • DM-25377: Update base.Packages serialization to support bytes [base]

  • DM-25379: psycopg2.OperationalError: SSL when running ci_hsc_gen3 [daf_butler]

  • DM-25385: begin pipetask command conversion to Click, implement the ‘build’ subcommand. [ctrl_mpexec, daf_butler, obs_base]

  • DM-25403: Clean up naming of gen3 formatters [ap_verify_testdata, daf_butler, obs_base, obs_decam, obs_lsst, obs_subaru, pipe_tasks]

  • DM-25407: ap_verify cannot handle curated crosstalk data in Gen 2 [ap_verify, ap_verify_testdata, obs_decam, obs_subaru]

  • DM-25411: Fix broken postgres test due to attribute manager [daf_butler]

  • DM-25416: Fix sphinx build for daf_butler [daf_butler]

  • DM-25424: Convert Defect to IsrCalib [cp_pipe, daf_butler, ip_isr, meas_algorithms, obs_base, obs_cfht, obs_decam, obs_lsst, obs_subaru, obs_test, pipe_tasks]

  • DM-25430: butler convert fails if gen2 repo is not an absolute path [obs_base]

  • DM-25431: Add conda env to base.Packages [base]

  • DM-25447: Support read-only components for butler get [ctrl_mpexec, daf_butler, obs_base, pipe_base]

  • DM-25450: Add Rowe Statistics to Squash [obs_subaru, verify_metrics]

  • DM-25458: Reorder instrument raw and calib collection names [obs_base, obs_decam, obs_lsst, obs_subaru]

  • DM-25461: Fix broken obs_lsst doc build [obs_lsst]

  • DM-25468: Add protection for repeated schema initialization [daf_butler]

  • DM-25470: Define better Gen2 path template for raw images in obs_lsst [obs_lsst]

  • DM-25477: Provide release notes for 20.0.0 [fgcmcal]

  • DM-25515: Fix references to https://lsst-web.ncsa.illinois.edu/~buildbot/doxygen [ip_isr, pipe_tasks]

  • DM-25516: numpydoc warnings in obs_base test utilities [obs_base]

  • DM-25627: make qgraph and run subcommands for pipetask [ctrl_mpexec, daf_butler, obs_base]

  • DM-25628: Update insertFakes to improve useability [pipe_tasks]

  • DM-25662: Add a column to the input fakes catalogue that specified if the source is to be added to the coadd or the calexp. [pipe_tasks]

  • DM-25673: Restructure obs_lsst configs for Gen3 [obs_lsst]

  • DM-25689: Problems converting RC2 with w_2020_25 [obs_subaru]

  • DM-25691: Write documentation on formatters, read-only components and parameters [daf_butler, obs_base]

  • DM-25692: Write gen3 formatter for Exposure Filter [daf_butler, obs_base]

  • DM-25698: Fix missing obs_decam dependencies [obs_decam]

  • DM-25705: Add tests for stray light in gen3 [obs_subaru, testdata_subaru]

  • DM-25711: Compare the gains produced by FULLCOVARIANCE in the PTC task (DM-20070) and with the pre-existing options EXPAPPROXIMATION and POLYNOMIAL [cp_pipe]

  • DM-25746: filterName in APDB needs to be one of g, r, i, z, or y [ap_association]

  • DM-25747: Update fakes code to add images directly [pipe_tasks]

  • DM-25748: Create pipeline task to generate fakes for AP processing and completeness tests [ap_pipe]

  • DM-25750: gen3 queryCollections does not work with regexes [daf_butler]

  • DM-25754: UnboundLocalError: local variable ‘n’ referenced before assignment [pipe_base]

  • DM-25760: add –glob to butler query-collections [daf_butler]

  • DM-25767: Add checking for duplicate keys in pipeline definitions [pipe_base]

  • DM-25768: Remove SdssShape computeFluxScale method [meas_base]

  • DM-25776: Refactor DataCoordinates and add custom containers for them. [daf_butler, ip_isr, obs_base, pipe_base, skymap]

  • DM-25779: Investigate jointcal chi2 changes in a large HSC run [jointcal]

  • DM-25782: Add an “detect_isPrimary” flag to single frame processing schema [pipe_tasks]

  • DM-25786: Cannot import sphgeom objects in Gen 3 [sphgeom]

  • DM-25806: Support parallel ap_verify ingestion in Gen 3 [ap_verify]

  • DM-25818: S3Datastore tests existence before writing [daf_butler]

  • DM-25826: lsst.alert.packet reader should iterate over alerts [alert_packet, ap_association]

  • DM-25855: Fix missing obs_decam dependencies, pt 2 [obs_decam]

  • DM-25867: Make sphgeom pip installable [sphgeom]

  • DM-25877: Rearrange organization of sphgeom pybind11 packaging [sphgeom]

  • DM-25881: Add extra logging to S3 datastore [daf_butler]

  • DM-25885: Export/Import of some datasets prints time warnings [daf_butler]

  • DM-25889: Update testdata_jointcal Gen3 HSC repo scripts [testdata_jointcal]

  • DM-25890: Patch gen2 mapper to report gen3 instrument class [obs_base, obs_decam, obs_lsst, obs_subaru]

  • DM-25903: CALIB_ID written by findDefects.py is wrong [cp_pipe]

  • DM-25914: Investigate sporadic failures in coverage.py [sconsUtils]

  • DM-25919: Add custom classes for registry query results [ap_verify, daf_butler, ip_isr, obs_base, pipe_base]

  • DM-25922: Ensure getVersionFromPythonModule() returns a string [base]

  • DM-25923: Add cache to yamlCamera.makeCamera [obs_base, obs_lsst]

  • DM-25930: Limit alert cutout size to twice the detection footprint size. [ap_association]

  • DM-25934: MeasurePhotonTransferCurveTask appears to ignore the defect mask [cp_pipe]

  • DM-25957: Remove unnecessary numpy usage from daf_butler [daf_butler]

  • DM-25970: ap_verify CI command line broken [ap_verify]

  • DM-25979: IngestIndexReferenceTask throws if coord_err_unit is None [meas_algorithms]

  • DM-25984: Investigate why the afw means of flat images are NANs for several amps of BOT data after DM-25934 [cp_pipe]

  • DM-25985: Save schema version(s) in the database [daf_butler]

  • DM-25990: Reprocess HSC COSMOS medium dataset with ap_pipe [ap_association, pipe_tasks]

  • DM-26004: Clean up qgraph show-workflow implementations [ctrl_mpexec, pipe_base]

  • DM-26007: defaultFilter is not used if a filterName is given to loadSkyCircle [jointcal, meas_algorithms, obs_subaru]

  • DM-26008: Add YAML representers to pex_config [daf_butler, obs_base, pex_config]

  • DM-26010: Simplify mypy configuration after ending use of namespace packages [daf_butler]

  • DM-26011: Intermittent failures in obs_lsst tests [obs_lsst, pipe_tasks]

  • DM-26015: Validate PipelineTaskConnections dimensions are iterables other than str [pipe_base]

  • DM-26040: Add AP timing metrics for DiaPipelineTask and all subtasks [ap_association, ap_verify, verify_metrics]

  • DM-26047: Defect calibration product filename collision [obs_lsst]

  • DM-26067: Fix bug in ptc.py that calls np.isnan on array (after DM-25984) [cp_pipe]

  • DM-26070: Add visit definition to ap_verify [ap_verify, ap_verify_testdata, obs_base, obs_lsst, obs_subaru]

  • DM-26082: Persist source-to-external reference matched catalogs in pipe_analysis to parquet [obs_base]

  • DM-26085: Fix fgcmcal zeropoint offset due to background offset [fgcm, fgcmcal]

  • DM-26088: Unmask LSST_LOG_CONFIG for setting log config [ctrl_mpexec]

  • DM-26103: make lsst.log config in butler command conditional [daf_butler]

  • DM-26119: Test dataset disassembly with ci_hsc_gen3 [daf_butler, obs_base]

  • DM-26131: Pipetask should handle SIGINT and back out of database transactions [ctrl_mpexec]

  • DM-26136: Improve handling of crashes in pipetask [ctrl_mpexec]

  • DM-26138: Add proper IMSIM filter definitions to obs_lsst [obs_lsst]

  • DM-26144: Allow Pipelines to inherit configs [pipe_base]

  • DM-26148: Add seaborn, conda, backoff, pytest-openfiles and postgres packages to conda environment [ap_association]

  • DM-26160: Fix test failure where URI special characters are in build directory [daf_butler, obs_base]

  • DM-26171: calibrations cannot be retrieved on the last day of a validity range [pipe_tasks]

  • DM-26175: ap_association fails with fastavro 0.24 [alert_packet]

  • DM-26181: Ensure that filters are defined in pipetask multiprocessing [ctrl_mpexec]

  • DM-26182: Fix zogy score image invalid value warning [ip_diffim]

  • DM-26187: Rename deblend.py in meas_deblender and meas_extensions_scarlet [meas_deblender, meas_extensions_scarlet, pipe_tasks]

  • DM-26198: Allow butler import to skip some dimensions [daf_butler]

  • DM-26203: Expose methods in sphgeom that provide chunk boundaries. [sphgeom]

  • DM-26205: Fix obs_lsst to handle corner raft data [obs_lsst]

  • DM-26206: Use the saturation levels from the per-amp yaml files if provided [obs_lsst]

  • DM-26217: Edge masking in IsrTask needs a “level” option to mask detector edges or amplifier edges. [ip_isr]

  • DM-26229: Investigate failure in daf_butler test in nightly build [daf_butler]

  • DM-26230: Improve pipetask dignostics on multiprocessing timeouts. [ctrl_mpexec]

  • DM-26237: Can’t rerun ap_verify on same repository in Gen 3 [ap_verify]

  • DM-26244: A few hot-fixes for pipe_analysis scripts post DM-22255 [obs_base, obs_lsst, obs_subaru, obs_test]

  • DM-26258: Fix problem with daf_butler tests failing with escaped characters in path [daf_butler]

  • DM-26260: ExposureFitsReader can’t read compressed darks/biases [afw]

  • DM-26265: ap_verify tests use fixed temp directories [ap_verify]

  • DM-26285: Add version parameter to pybind11 deprecation [utils]

  • DM-26301: Database work and cleanups in support of CALIBRATION collections [daf_butler]

  • DM-26304: Move PexConfigFormatter to obs_base [daf_butler, obs_base]

  • DM-26310: Add WebDAV support to daf_butler [daf_butler]

  • DM-26316: Allow bias generation to retain overscan signal [ip_isr]

  • DM-26317: Add camera caching to obs_decam and obs_subaru Gen3 Instrument [obs_decam, obs_subaru]

  • DM-26318: Fix WebDAV failures on Jenkins [daf_butler]

  • DM-26324: Investigate reproducible YAML export from butler [daf_butler]

  • DM-26327: Add support for x-flipped WCS in gen3 formatters [obs_base, obs_decam, obs_lsst]

  • DM-26331: Encapsulate and clean up timespans in Registry [ap_verify_testdata, daf_butler, obs_base, obs_subaru, skymap]

  • DM-26333: Add partial support for calibration collections [daf_butler]

  • DM-26336: Prototype and design work for dimensions/queries system improvements [daf_butler]

  • DM-26343: Fix extension usage in ButlerURI and Butler Ingest [daf_butler, obs_base]

  • DM-26371: Create a more robust Quantum graph object [ctrl_mpexec, daf_butler, pipe_base]

  • DM-26378: Refactor remote storage datastores [daf_butler, obs_base]

  • DM-26383: Add webdav datastore tests to daf_butler [daf_butler]

  • DM-26398: SourceDetectionTask alters input exposure image values in place [meas_algorithms]

  • DM-26403: Reorganize ButlerURI [daf_butler]

  • DM-26405: Move backoff retry from S3Datastore to ButlerURI [daf_butler]

  • DM-26407: Change how dimensions are stored in a Butler repository [ctrl_mpexec, daf_butler, obs_base, pipe_base]

  • DM-26414: Handle masked pixels in ip_isr’s MEDIAN_PER_ROW [ip_isr]

  • DM-26415: remove chained commands from pipetask2 [ctrl_mpexec, daf_butler]

  • DM-26418: Fix bug in insertFakes found during fakes RC processing. [pipe_tasks]

  • DM-26429: Improve DateSystem enum docs to clarify EPOCH [daf_base]

  • DM-26430: Fix bugs and easily-avoidable warnings in new timespan code [daf_butler]

  • DM-26438: Add “detect_isPrimary” flag to parquet Source Table [obs_subaru]

  • DM-26439: Add doProcessAllDataIds config option into insertFakes [pipe_tasks]

  • DM-26445: w34 ingest_raws fails with ci_hsc_gen3 and PostgreSQL with Timespan error [daf_butler]

  • DM-26452: Fix fringe filter inconsistency [ip_isr]

  • DM-26453: Add sqrt(var) as weight to EXPAPPROXIMATION and POLYNOMIAL fit residual in ptc.py [cp_pipe]

  • DM-26456: Create sky object performance metric trackers [verify_metrics]

  • DM-26476: Add “reason for observation” to Gen3 registry [daf_butler, obs_base]

  • DM-26483: –show history does not work with pipetask [ctrl_mpexec]

  • DM-26485: Add vectorized pure-python interface to convert ra/dec to and from x/y for SkyWcs [afw]

  • DM-26526: Can’t run RawIngestTask with processes != 1 [obs_base]

  • DM-26539: w_35 ci_hsc_gen pipeline.sh dying due to SSL Error [ctrl_mpexec]

  • DM-26545: Add spline linearizer. [cp_pipe, ip_isr, obs_base, obs_lsst]

  • DM-26550: Add observation_reason to ObservationInfo [astro_metadata_translator, obs_lsst]

  • DM-26553: Create gen3 unittests and test CreateApFakes pipeline tasks on data. [ap_pipe, pipe_tasks]

  • DM-26554: obs_decam missing configs for bias and flat construction in Gen2 [obs_decam]

  • DM-26567: Fix minor bug in Gen2 DCR diffim template generation [ip_diffim]

  • DM-26568: Instantiating a Butler with s3 datastore fails [daf_butler]

  • DM-26586: Investigate LengthError when running ProcessCcd on Decam galactic bulge images [pipe_tasks]

  • DM-26591: Include instrument data ID value when provided on pipetask command-line or Pipeline yaml file [ctrl_mpexec, pipe_base]

  • DM-26595: Have cp_pipe/ptc.py continue if there is a bad flat pair [cp_pipe]

  • DM-26597: fix –log-level help and default value [daf_butler]

  • DM-26599: Help Seth update sensor positions [obs_lsst]

  • DM-26600: Add ability to prune erroneous datasetTypes from gen3 registry [daf_butler]

  • DM-26601: butler convert should write curated calibrations even when there is no gen2 calibration registry [obs_base]

  • DM-26603: Update the stack version of scarlet, including scarlet_extensions [meas_extensions_scarlet, scarlet]

  • DM-26606: Make photodiode BOT data usable for Gen2 [cp_pipe]

  • DM-26613: Add butler command to make a discrete sky map [pipe_tasks]

  • DM-26615: Remove afw.filter dependence from DCR model [ip_diffim, pipe_tasks]

  • DM-26616: Gain slope is inverted in EXPAPPROXIMATION case in plotPtc.py [cp_pipe]

  • DM-26620: Fix ErfaWarning in butler time comparisons [daf_butler]

  • DM-26629: Convert calibration datasets to use CALIBRATION collections instead of calibration_label [ap_verify, ap_verify_testdata, cp_pipe, daf_butler, ip_isr, obs_base, obs_decam, obs_lsst, obs_subaru, pipe_base]

  • DM-26630: Column size for datastore filename is too short [daf_butler]

  • DM-26639: Add FGCM-application configs to obs_subaru to make them “official” [obs_subaru]

  • DM-26641: Problem with log plots and problem with bad amps in ptc.py [cp_pipe]

  • DM-26645: Add a default value for pipetask2 -j option [ctrl_mpexec]

  • DM-26647: Handle missing data in meas_extensions_scarlet [meas_extensions_scarlet]

  • DM-26654: insertFakes cleanCat() may clean stars without bulge/disk values [pipe_tasks]

  • DM-26662: ap_verify import slow in Gen 3 [ap_verify]

  • DM-26671: Use rubinenv in scipipe_conda_env and publish notice of it [obs_base, verify]

  • DM-26675: Changes to validate_drp to support treecorr > 4 [validate_drp]

  • DM-26677: Template retrieval KeyError [ip_diffim]

  • DM-26680: Integrate gen3 schema changes w38 [daf_butler]

  • DM-26683: Make dimension data export idempotent [daf_butler]

  • DM-26684: Add command-line tool for Butler.pruneCollection [daf_butler]

  • DM-26685: Add command-line tool for Registry.queryDatasets [daf_butler]

  • DM-26690: Add command-line tool for Registry.queryDataIds [daf_butler]

  • DM-26696: Add filename to fix_header and handle bad correction file [astro_metadata_translator, obs_base, obs_decam, obs_lsst]

  • DM-26697: Deprecate needing to pre-pair exposures in PTC task, and calculate an appropriate pair based on header data. [cp_pipe]

  • DM-26698: Add additional valid extensions to FitsGenericFormatter [obs_base]

  • DM-26725: Update the PTC dataset to be a proper IsrCalib [cp_pipe, ip_isr, obs_base, obs_lsst]

  • DM-26726: Add support for freezing configs in gen3 [ctrl_mpexec, pipe_base, pipe_tasks]

  • DM-26728: Growing open file count in pipetask [ctrl_mpexec]

  • DM-26729: measureCrosstalk has a typo in the debug code. [cp_pipe]

  • DM-26738: Change butler gen3 registry to band rather than abstract filter [ap_verify, ap_verify_testdata, ctrl_mpexec, daf_butler, meas_base, obs_base, obs_cfht, obs_decam, obs_lsst, obs_subaru, pipe_base, pipe_tasks, skymap]

  • DM-26739: Ensure logging includes PipelineTask label and quantum data ID [ctrl_mpexec]

  • DM-26757: Numpydoc conversion of meas_algorithms through detection.py [meas_algorithms]

  • DM-26774: Instrument-finding code incorrectly requires a data query [ctrl_mpexec, pipe_base, utils]

  • DM-26775: Add -d alias to ap_verify CLI [ap_verify]

  • DM-26778: Fix breakage in repo export from DM-26683 [daf_butler]

  • DM-26780: Allow click subcommands to take “presets” [ctrl_mpexec, daf_butler, obs_base]

  • DM-26783: DiaForcedSource on associated DiaObject off frame [ap_association]

  • DM-26811: Make the obs_lsst imSim instrument name respect the case of the imSim program name. [ap_verify, ap_verify_testdata, obs_lsst]

  • DM-26826: Fix docstring See also references in afw and ip_isr [afw, ip_isr]

  • DM-26832: Transient failure in ctrl_mpexec multiprocessing [ctrl_mpexec]

  • DM-26849: astshim incompatible with pybind11 2.5 [astshim]

  • DM-26856: Add butler remove-dataset-type command line subcommand [daf_butler]

  • DM-26862: Add focal-plane outlier rejection and focal-plane residual fits to fgcm [fgcm, fgcmcal, obs_subaru]

  • DM-26867: Mark obs_ctio0m9 as deprecated [obs_ctio0m9]

  • DM-26871: starlink-ast 9.2.1 appears to incorrectly simplify mappings [astshim]

  • DM-26872: Python 3.8 moves reported function start line, breaking flake8 annotations on decorators [afw, daf_base, daf_butler, daf_persistence, geom, log, meas_modelfit, shapelet]

  • DM-26874: rename pipetask2 to pipetask, remove old pipetask [ctrl_mpexec, obs_base, pipe_tasks]

  • DM-26939: Add unquie id to ap fakes catalog. [ap_pipe]

  • DM-26940: Add -j option to butler ingest-raws subcommand [ctrl_mpexec, daf_butler, obs_base]

  • DM-26943: Support newer headers in imsim data [obs_lsst]

  • DM-26944: Move certifyCalibrations.py content to a butler command [cp_pipe, daf_butler]

  • DM-26946: readFits in calibType.ptc seems to fail to read more than one table [ip_isr]

  • DM-26947: Minor follow-up work from DM-26629 [daf_butler, obs_base, pipe_base]

  • DM-26953: Update ap_verify system for calibration collections [ap_verify, ap_verify_testdata]

  • DM-26973: WebDAV datastore: Read token from file + other minor changes [daf_butler]

  • DM-26974: Fix timing issue in ctrl_mpexec unit test [ctrl_mpexec]

  • DM-26978: Change imSim/PhoSim gen3 instrument class names to be more explicit [ap_verify_testdata, obs_lsst]

  • DM-26999: Update fakes insertion code with an option to use fgcm. [pipe_tasks]

  • DM-27003: Create a ‘difference’ functor method [pipe_tasks]

  • DM-27007: Disable checksum calculation by default in gen3 datastore [daf_butler]

  • DM-27018: Fix problems with inconsistencies in calibrations gaps in gen2 to 3 conversion [obs_base]

  • DM-27027: Enable persistence of “source” parquet tables in obs_subaru [obs_subaru]

  • DM-27033: Integration of pre-middleware-release dimensions changes [ap_verify_testdata, ctrl_mpexec, daf_butler, obs_base]

  • DM-27034: Overhaul dimension construction and add ABCs [daf_butler, pipe_base]

  • DM-27035: Streamlining and integration after dimensions overhaul [daf_butler]

  • DM-27045: Reading defects from data packages is very slow [meas_algorithms]

  • DM-27046: Update an old QuantumGraph.traverse() usage [ctrl_mpexec]

  • DM-27060: pipe_base documentation fails to build due to BuildId [pipe_base]

  • DM-27066: Allow running a subset of pipeline labels [pipe_base]

  • DM-27071: butler ingest-raws crashes on one bad file [obs_base]

  • DM-27081: Fix daily breaking scarlet update [scarlet]

  • DM-27097: Enable -j in butler convert [obs_base]

  • DM-27099: Re-bridge ap_verify and ctrl_mpexec [ap_verify]

  • DM-27113: Convert RC2 w_2020_38 to gen3 with w_2020_42 stack [obs_base]

  • DM-27131: Fix pickling of Task with parent [ap_verify, obs_base, pipe_base]

  • DM-27140: Make concrete pipeline definitions usable and hide those that aren’t [ap_pipe, ctrl_mpexec, obs_subaru, pipe_base, pipe_tasks]

  • DM-27151: Allow record information such as detector name in dataIds [daf_butler]

  • DM-27157: constructDark.py fails on LSSTCam with ‘Image contains no Pixels’ [ip_isr]

  • DM-27158: constructDark.py fails on LSSTCam with ‘Too many CR pixels’ [obs_lsst]

  • DM-27159: measurePhotonTransferCurve.py fails on LSSTCam [cp_pipe]

  • DM-27162: For LSSTCam do not include empty second filter in filter name [obs_lsst]

  • DM-27165: Calibration ingestion produces registry where butler cannot find matching calib product [pipe_tasks]

  • DM-27171: Make gen3 ingest tests faster [obs_base]

  • DM-27180: NaNs in measurePhotonTransferCurve.py causing failures [cp_pipe, obs_lsst]

  • DM-27185: ptc.py fails with ptcFitType=FULLCOVARIANCE [cp_pipe, ip_isr]

  • DM-27208: Make meas_deblender and meas_extensions_scarlet catalogs consistent [meas_deblender, meas_extensions_scarlet, scarlet]

  • DM-27217: Avoid deadlocks in Database.sync [daf_butler]

  • DM-27222: change Registry.queryDatasets “deduplicate” to “findFirst” [daf_butler, ip_isr, pipe_base]

  • DM-27246: Lots of Doxygen warnings in afw [afw, base]

  • DM-27251: Introduce governor dimensions [ctrl_mpexec, daf_butler, pipe_base]

  • DM-27253: Add support for materializing dimension overlaps in old query system [daf_butler]

  • DM-27256: Add JSON support to butler Config [daf_butler]

  • DM-27266: Switch serialized Dimensions config to JSON inside database [daf_butler]

  • DM-27270: ap_verify failed on w_2020_43 [ap_verify]

  • DM-27280: Fix DataCoordinate sorting [daf_butler]

  • DM-27281: comcam images from NTS will not ingest in gen3 butler [obs_lsst]

  • DM-27293: Add expression parser support for bindparams and topological operators [daf_butler]

  • DM-27294: Certain log messages would crash a pipetask run with a S3Datastore [daf_butler]

  • DM-27298: Add observing_day and sequence_number to astro_metadata_translator [astro_metadata_translator, obs_lsst]

  • DM-27319: Examine behavior of IsrTask wrt doSaturation and doSaturationInterpolation [ip_isr]

  • DM-27321: Add observing day and other fixes to exposure/visit tables [daf_butler, obs_base]

  • DM-27339: BOT has unrecognised physical_filters [obs_lsst]

  • DM-27343: Butler Config reader has confusing error message if directory not found [daf_butler]

  • DM-27350: Fix handling of bad metadata translations in gen3 ingest [astro_metadata_translator, obs_base]

  • DM-27351: –long-log does not work with butler command line command [daf_butler]

  • DM-27354: Make sphgeom pip-installable (remove use of requirements.txt) [daf_butler, sphgeom]

  • DM-27358: starlink-ast 9.2.3 fails tests/test_skyWcs.py::checkTanWcs with default float tolerance [afw]

  • DM-27372: Bug in getting component from DeferredDatasetHandle [daf_butler]

  • DM-27373: Improve error message for missing butler_attributes table [daf_butler]

  • DM-27374: Add ingest time support to queryDatasets [daf_butler]

  • DM-27381: ctrl_mpexec fails (or may sometimes fail) on mac with python 3.8 in test_mpexec_faillure_failfast [ctrl_mpexec]

  • DM-27384: meas_extensions_psfex no longer has a utils module to document [meas_extensions_psfex]

  • DM-27390: Replace DimensionGraph.encode with something more stable [ctrl_mpexec, daf_butler]

  • DM-27397: Drop support for dataset type, governor restrictions in collection chaining [ctrl_mpexec, daf_butler, obs_base]

  • DM-27409: Remove seeing from visit record in registry [daf_butler]

  • DM-27418: Use PyYAML safe_dump() to write datasets [daf_butler]

  • DM-27425: Add gen3 HSC export.yaml to testdata_jointcal repo [testdata_jointcal]

  • DM-27426: Update YAML repo export files in ci_hsc_gen3 and pipelines_check [obs_base, obs_subaru]

  • DM-27427: Replace common daf_butler assertion failure with a better error message [daf_butler]

  • DM-27434: Add ctrl_bps to lsst_distrib package [lsst_distrib]

  • DM-27435: Unification of file-based datastores [daf_butler]

  • DM-27436: Invetigate scarlet failures in python 3.8 [scarlet]

  • DM-27437: Have maxMeanSignal (and minMeanSignal) be a list per amp, instead of a single number in the PTC task [cp_pipe]

  • DM-27438: Report in plots the latest point used in the PTC fit as the PTC turn off [cp_pipe]

  • DM-27490: Add docstrings for collections [ap_verify_testdata, daf_butler]

  • DM-27494: Fix race condition in saving DimensionGraphs to Registry [daf_butler]

  • DM-27535: ProcessCcd on gen3 broke in w_44 [meas_extensions_psfex]