FgcmFitCycleTask¶
FgcmFitCycleTask runs a single fgcm fit on the star observations generated from FgcmBuildStarsTask using the look-up table generated from FgcmMakeLutTask. This code is meant to be run multiple times until convergence, and the results output from one “fit cycle” are used as an input to the subsequent fit cycle. One final cleanup run is performed to generate the tables required for input to FgcmOutputProductsTask.
This is the third task in a typical fgcmcal processing chain. The first is FgcmMakeLutTask, the second is FgcmBuildStarsTableTask or FgcmBuildStarsTask, and the fourth is FgcmOutputProductsTask.
FgcmFitCycleTask is available as a command-line task, fgcmFitCycle.py.
Processing summary¶
FgcmFitCycleTask reads in the star observations and the look-up table, performs an atmosphere and instrument fit, and outputs fit parameters as well as a comprehensive list of QA plots. If the config option isFinalCycle is set to True then additional datafiles are output that are used by FgcmOutputProductsTask. Please see the cookbook for information on deciding when and how to set the isFinalCycle parameter to True.
Aside from the butler outputs (listed below), the FgcmFitCycleTask will output a new config file in the current working directory with recommended settings for the subsequent fit cycle. Furthermore, there are a wide range of diagnostic QA plots that are output by the task. For details on the contents of these QA plots, please see the cookbook as well as Burke, Rykoff, et al. 2018.
fgcmFitCycle.py command-line interface¶
Note that no --id arguments are used by fgcmFitCycle.py.
fgcmFitCycle.py REPOPATH [@file [@file2 ...]] [--output OUTPUTREPO | --rerun RERUN] [--configfile configfile] [other options]
Key arguments:
REPOPATH- The input Butler repository’s URI or file path.
Key options:
--configfile:- The config file to use. For all runs except the initial, this should be the config file output from the previous cycle.
See also
See Command-line task argument reference for details and additional options.
Python API summary¶
from lsst.fgcmcal.fgcmFitCycle import FgcmFitCycleTask
-
class
FgcmFitCycleTask Run Single fit cycle for FGCM global calibration
...
-
attribute
config Access configuration fields and retargetable subtasks.
-
method
runDataRef Run a single fit cycle for FGCM
...
See also
See the FgcmFitCycleTask API reference for complete details.
Butler datasets¶
When run as the fgcmFitCycle.py command-line task, or directly through the runDataRef method, FgcmFitCycleTask obtains datasets from the input Butler data repository and persists outputs to the output Butler data repository.
Note that configurations for FgcmFitCycleTask, and its subtasks, affect what datasets are persisted and what their content is.
Input datasets¶
camera- Camera geometry and detector object
fgcmLookupTable- FGCM look-up table produced by FgcmMakeLutTask
fgcmVisitCatalog- Catalog (
lsst.afw.table) of visit metadata produced by FgcmBuildStarsTask fgcmStarObservations- Catalog of star observations produced by FgcmBuildStarsTask
fgcmStarIds- Catalog of unique star ids, positions, and number of observations produced by FgcmBuildStarsTask
fgcmStarIndices- Catalog of indices linking unique star ids to star observations produced by FgcmBuildStarsTask
fgcmReferenceStars- Catalog of reference stars matched to unique star ids produced by FgcmBuildStarsTask
fgcmFitParameters- Catalog of fit parameters from previous fit cycle (if available)
Output datasets¶
fgcmFitParameters- Catalog of fit parameters. Not output if run as part of FgcmCalibrateTractTask
fgcmFlaggedStars- Catalog of flagged star ids, either bad or reserved from fit. Not output if run as part of FgcmCalibrateTractTask
fgcmZeropoints- Catalog of zero-point information. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTask fgcmAtmosphereParameters- Catalog of atmosphere parameters. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTask fgcmStandardStars- Catalog of standard stars from fit. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTask
Retargetable subtasks¶
No subtasks.
Configuration fields¶
aperCorrFitNBins¶
Number of aperture bins used in aperture correction fit. When set to 0no fit will be performed, and the config.aperCorrInputSlopes will be used if available.
aperCorrInputSlopeDict¶
Per-band specification of aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set this when there is insufficient data to fit the parameters (e.g. tract mode).
aperCorrInputSlopes¶
Aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set thiswhen there is insufficient data to fit the parameters (e.g. tract mode). If set, must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use aperCorrInputSlopeDict instead.
approxThroughput¶
Approximate overall throughput at start of calibration observations. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use approxThroughputDict instead.
approxThroughputDict¶
Per-band specification of the approximate overall throughput at the start of calibration observations. Must have one entry per band. Typically should be 1.0.
autoHighCutNSig¶
Number of sigma for automatic computation of (high) outlier cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).
autoPhotometricCutNSig¶
Number of sigma for automatic computation of (low) photometric cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).
brightObsGrayMax¶
Maximum gray extinction to be considered bright observation
ccdGrayFocalPlaneChebyshevOrder¶
Order of the 2D chebyshev polynomials for focal plane fit.
ccdGrayFocalPlaneDict¶
Per-band specification on whether to compute focal-plane residual (‘ccd gray’) corrections.
ccdGrayFocalPlaneFitMinCcd¶
Minimum number of ‘good’ CCDs required to perform focal-plane gray corrections. If there are fewer good CCDs then the gray correction is computed per-ccd.
ccdGrayMaxStarErr¶
Maximum error on a star observation to use in ccd gray (achromatic residual) computation
ccdGraySubCcd¶
Compute CCD gray terms on sub-ccd scale Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use ccdGraySubCcdDict instead.
ccdGraySubCcdChebyshevOrder¶
Order of the 2D chebyshev polynomials for sub-ccd gray fit.
ccdGraySubCcdDict¶
Per-band specification on whether to compute achromatic per-ccd residual (‘ccd gray’) on a sub-ccd scale.
ccdGraySubCcdTriangular¶
Should the sub-ccd gray chebyshev matrix be triangular to suppress high-order cross terms?
colorSplitBands¶
Band names to use to split stars by color. Must have 2 entries.
colorSplitIndices¶
Band indices to use to split stars by color Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use colorSplitBands instead.
connections¶
- Data type
lsst.pipe.base.config.Connections- Field type
ConfigField
Configurations describing the connections of the PipelineTask to datatypes
cycleNumber¶
FGCM fit cycle number. This is automatically incremented after each run and stage of outlier rejection. See cookbook for details.
deltaMagBkgOffsetPercentile¶
Percentile brightest stars on a visit/ccd to use to compute net offset from local background subtraction.
deltaMagBkgPerCcd¶
Compute net offset from local background subtraction per-ccd? Otherwise, use computation per visit.
doReferenceCalibration¶
Use reference catalog as additional constraint on calibration
expGrayErrRecoverCut¶
Maximum exposure gray error to be able to recover bad ccds via interpolation. Visits with more gray variance will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.
expGrayHighCut¶
Maximum (positive) exposure gray for a visit to be considered photometric. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expGrayHighCutDict instead.
expGrayHighCutDict¶
Per-band specification on maximum (positive) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be 0.2.
expGrayInitialCut¶
Maximum exposure/visit gray value for initial selection of possible photometric observations.
expGrayPhotometricCut¶
Maximum (negative) exposure gray for a visit to be considered photometric. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expGrayPhotometricCutDict instead.
expGrayPhotometricCutDict¶
Per-band specification on maximum (negative) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be -0.05.
expGrayRecoverCut¶
Maximum (negative) exposure gray to be able to recover bad ccds via interpolation. Visits with more gray extinction will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.
expVarGrayPhotometricCut¶
Maximum exposure variance to be considered possibly photometric Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expVarGrayPhotometricCutDict instead.
expVarGrayPhotometricCutDict¶
Per-band specification on maximum exposure variance to be considered possibly photometric. Must have one entry per band. Broad-band filters should be 0.0005.
filterMap¶
Mapping from ‘filterName’ to band. Deprecated: This field is no longer used, and has been deprecated by DM-28088. It will be removed after v22. Use physicalFilterMap instead.
fitBands¶
Bands to use in atmospheric fit. The bands not listed here will have the atmosphere constrained from the ‘fitBands’ on the same night. Must be a subset of
config.bandsfitFlag¶
Flag for which bands are directly constrained in the FGCM fit. Bands set to 0 will have the atmosphere constrained from observations in other bands on the same night. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use fitBands instead.
focalPlaneSigmaClip¶
Number of sigma to clip outliers per focal-plane.
freezeStdAtmosphere¶
Freeze atmosphere parameters to standard (for testing)
instrumentParsPerBand¶
Model instrumental parameters per band? Otherwise, instrumental parameters (QE changes with time) are shared among all bands.
instrumentSlopeMinDeltaT¶
Minimum time change (in days) between observations to use in constraining instrument slope.
isFinalCycle¶
Is this the final cycle of the fitting? Will automatically compute final selection of stars and photometric exposures, and will output zeropoints and standard stars for use in fgcmOutputProducts
maxCcdGrayErr¶
Maximum error on CCD gray offset to be considered photometric
maxIterBeforeFinalCycle¶
Maximum fit iterations, prior to final cycle. The number of iterations will always be 0 in the final cycle for cleanup and final selection.
minCcdPerExp¶
Minimum number of good CCDs per exposure/visit to be used in calibration fit. Visits with fewer good CCDs will have CCD zeropoints estimated where possible.
minExpPerNight¶
Minimum number of good exposures/visits to consider a partly photometric night
minStarPerCcd¶
Minimum number of good stars per CCD to be used in calibration fit. CCDs with fewer stars will have their calibration estimated from other CCDs in the same visit, with zeropoint error increased accordingly.
minStarPerExp¶
Minimum number of good stars per exposure/visit to be used in calibration fit. Visits with fewer good stars will have CCD zeropoints estimated where possible.
modelMagErrors¶
Should FGCM model the magnitude errors from sky/fwhm? (False means trust inputs)
multipleCyclesFinalCycleNumber¶
- Default
5- Field type
intRangeField- Range
- [2,10]
Final cycle number in multiple cycle mode. The initial cycle is 0, with limited parameters fit. The next cycle is 1 with full parameter fit. The final cycle is a clean-up with no parameters fit. There will be a total of (multipleCycleFinalCycleNumber + 1) cycles run, and the final cycle number cannot be less than 2.
outputStandardsBeforeFinalCycle¶
Output standard stars prior to final cycle? Used in debugging.
outputZeropointsBeforeFinalCycle¶
Output standard stars prior to final cycle? Used in debugging.
physicalFilterMap¶
Mapping from ‘physicalFilter’ to band.
precomputeSuperStarInitialCycle¶
Precompute superstar flat for initial cycle
randomSeed¶
Random seed for fgcm for consistency in tests.
refStarOutlierNSig¶
Number of sigma compared to average mag for reference star to be considered an outlier. Computed per-band, and if it is an outlier in any band it is rejected from fits.
refStarSnMin¶
Reference star signal-to-noise minimum to use in calibration. Set to <=0 for no cut.
requiredBands¶
Bands that are required for a star to be considered a calibration star. Must be a subset of
config.bandsrequiredFlag¶
Flag for which bands are required for a star to be considered a calibration star in the FGCM fit. Typically this should be the same as fitFlag. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use requiredBands instead.
saveMetadata¶
Flag to enable/disable metadata saving for a task, enabled by default.
sedFudgeFactors¶
Fudge factors for computing linear SED from colors. Must be same length as config.bands, and matched band-by-band. Deprecated: This field has been deprecated and will be removed after v20. Please use sedSlopeTermMap and sedSlopeMap.
sedboundaryterms¶
- Data type
lsst.fgcmcal.sedterms.SedboundarytermDict- Field type
ConfigField
Mapping from bands to SED boundary term names used is sedterms.
sedterms¶
- Data type
lsst.fgcmcal.sedterms.SedtermDict- Field type
ConfigField
Mapping from terms to bands for fgcm linear SED approximations.
sigFgcmMaxEGray¶
Maximum (absolute) gray value for observation in sigma_FGCM. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use sigFgcmMaxEGrayDict instead.
sigFgcmMaxEGrayDict¶
Per-band specification for maximum (absolute) achromatic residual (gray value) for observations in sigma_fgcm (raw repeatability). Broad-band filters should be 0.05.
sigmaCalFitPercentile¶
Magnitude percentile range to fit systematic error floor
sigmaCalPlotPercentile¶
Magnitude percentile range to plot systematic error floor
sigmaCalRange¶
Allowed range for systematic error floor estimation
starColorCuts¶
Encoded star-color cuts (to be cleaned up)
superStarSigmaClip¶
Number of sigma to clip outliers when selecting for superstar flats
superStarSubCcd¶
Compute superstar flat on sub-ccd scale Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use superStarSubCcdDict instead.
superStarSubCcdChebyshevOrder¶
Order of the 2D chebyshev polynomials for sub-ccd superstar fit. Global default is first-order polynomials, and should be overridden on a camera-by-camera basis depending on the ISR.
superStarSubCcdDict¶
Per-band specification on whether to compute superstar flat on sub-ccd scale. Must have one entry per band.
superStarSubCcdTriangular¶
Should the sub-ccd superstar chebyshev matrix be triangular to suppress high-order cross terms?
useQuadraticPwv¶
Model PWV with a quadratic term for variation through the night?
useRepeatabilityForExpGrayCuts¶
Use star repeatability (instead of exposures) for computing photometric cuts? Recommended for tract mode or bands with few exposures. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use useRepeatabilityForExpGrayCutsDict instead.
useRepeatabilityForExpGrayCutsDict¶
Per-band specification on whether to use star repeatability (instead of exposures) for computing photometric cuts. Recommended for tract mode or bands with few visits.