FgcmFitCycleTask¶
FgcmFitCycleTask runs a single fgcm fit on the star observations generated from FgcmBuildStarsTask using the look-up table generated from FgcmMakeLutTask. This code is meant to be run multiple times until convergence, and the results output from one “fit cycle” are used as an input to the subsequent fit cycle. One final cleanup run is performed to generate the tables required for input to FgcmOutputProductsTask.
This is the third task in a typical fgcmcal processing chain. The first is FgcmMakeLutTask, the second is FgcmBuildStarsTableTask or FgcmBuildStarsTask, and the fourth is FgcmOutputProductsTask.
FgcmFitCycleTask is available as a command-line task, fgcmFitCycle.py.
Processing summary¶
FgcmFitCycleTask reads in the star observations and the look-up table, performs an atmosphere and instrument fit, and outputs fit parameters as well as a comprehensive list of QA plots. If the config option isFinalCycle is set to True then additional datafiles are output that are used by FgcmOutputProductsTask. Please see the cookbook for information on deciding when and how to set the isFinalCycle parameter to True.
Aside from the butler outputs (listed below), the FgcmFitCycleTask will output a new config file in the current working directory with recommended settings for the subsequent fit cycle. Furthermore, there are a wide range of diagnostic QA plots that are output by the task. For details on the contents of these QA plots, please see the cookbook as well as Burke, Rykoff, et al. 2018.
fgcmFitCycle.py command-line interface¶
Note that no --id arguments are used by fgcmFitCycle.py.
fgcmFitCycle.py REPOPATH [@file [@file2 ...]] [--output OUTPUTREPO | --rerun RERUN] [--configfile configfile] [other options]
Key arguments:
REPOPATHThe input Butler repository’s URI or file path.
Key options:
--configfile:The config file to use. For all runs except the initial, this should be the config file output from the previous cycle.
See also
See Command-line task argument reference for details and additional options.
Python API summary¶
from lsst.fgcmcal.fgcmFitCycle import FgcmFitCycleTask
-
class
FgcmFitCycleTask Run Single fit cycle for FGCM global calibration
...
-
attribute
config Access configuration fields and retargetable subtasks.
-
method
runDataRef Run a single fit cycle for FGCM
...
See also
See the FgcmFitCycleTask API reference for complete details.
Butler datasets¶
When run as the fgcmFitCycle.py command-line task, or directly through the runDataRef method, FgcmFitCycleTask obtains datasets from the input Butler data repository and persists outputs to the output Butler data repository.
Note that configurations for FgcmFitCycleTask, and its subtasks, affect what datasets are persisted and what their content is.
Input datasets¶
cameraCamera geometry and detector object
fgcmLookupTableFGCM look-up table produced by FgcmMakeLutTask
fgcmVisitCatalogCatalog (
lsst.afw.table) of visit metadata produced by FgcmBuildStarsTaskfgcmStarObservationsCatalog of star observations produced by FgcmBuildStarsTask
fgcmStarIdsCatalog of unique star ids, positions, and number of observations produced by FgcmBuildStarsTask
fgcmStarIndicesCatalog of indices linking unique star ids to star observations produced by FgcmBuildStarsTask
fgcmReferenceStarsCatalog of reference stars matched to unique star ids produced by FgcmBuildStarsTask
fgcmFitParametersCatalog of fit parameters from previous fit cycle (if available)
Output datasets¶
fgcmFitParametersCatalog of fit parameters. Not output if run as part of FgcmCalibrateTractTask
fgcmFlaggedStarsCatalog of flagged star ids, either bad or reserved from fit. Not output if run as part of FgcmCalibrateTractTask
fgcmZeropointsCatalog of zero-point information. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTaskfgcmAtmosphereParametersCatalog of atmosphere parameters. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTaskfgcmStandardStarsCatalog of standard stars from fit. Only output if
isFinalCycleisTrue. Not output if run as part of FgcmCalibrateTractTask
Retargetable subtasks¶
No subtasks.
Configuration fields¶
aperCorrFitNBins¶
Number of aperture bins used in aperture correction fit. When set to 0no fit will be performed, and the config.aperCorrInputSlopes will be used if available.
aperCorrInputSlopeDict¶
Per-band specification of aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set this when there is insufficient data to fit the parameters (e.g. tract mode).
aperCorrInputSlopes¶
Aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set thiswhen there is insufficient data to fit the parameters (e.g. tract mode). If set, must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use aperCorrInputSlopeDict instead.
applyRefStarColorCuts¶
Apply color cuts to reference stars?
approxThroughput¶
Approximate overall throughput at start of calibration observations. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use approxThroughputDict instead.
approxThroughputDict¶
Per-band specification of the approximate overall throughput at the start of calibration observations. Must have one entry per band. Typically should be 1.0.
autoHighCutNSig¶
Number of sigma for automatic computation of (high) outlier cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).
autoPhotometricCutNSig¶
Number of sigma for automatic computation of (low) photometric cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).
bands¶
Bands to run calibration
brightObsGrayMax¶
Maximum gray extinction to be considered bright observation
ccdGrayFocalPlaneChebyshevOrder¶
Order of the 2D chebyshev polynomials for focal plane fit.
ccdGrayFocalPlaneDict¶
Per-band specification on whether to compute focal-plane residual (‘ccd gray’) corrections.
ccdGrayFocalPlaneFitMinCcd¶
Minimum number of ‘good’ CCDs required to perform focal-plane gray corrections. If there are fewer good CCDs then the gray correction is computed per-ccd.
ccdGrayMaxStarErr¶
Maximum error on a star observation to use in ccd gray (achromatic residual) computation
ccdGraySubCcd¶
Compute CCD gray terms on sub-ccd scale Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use ccdGraySubCcdDict instead.
ccdGraySubCcdChebyshevOrder¶
Order of the 2D chebyshev polynomials for sub-ccd gray fit.
ccdGraySubCcdDict¶
Per-band specification on whether to compute achromatic per-ccd residual (‘ccd gray’) on a sub-ccd scale.
ccdGraySubCcdTriangular¶
Should the sub-ccd gray chebyshev matrix be triangular to suppress high-order cross terms?
coatingMjds¶
Mirror coating dates in MJD
colorSplitBands¶
Band names to use to split stars by color. Must have 2 entries.
colorSplitIndices¶
Band indices to use to split stars by color Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use colorSplitBands instead.
connections¶
- Data type
lsst.pipe.base.config.Connections- Field type
Configurations describing the connections of the PipelineTask to datatypes
cycleNumber¶
FGCM fit cycle number. This is automatically incremented after each run and stage of outlier rejection. See cookbook for details.
deltaMagBkgOffsetPercentile¶
Percentile brightest stars on a visit/ccd to use to compute net offset from local background subtraction.
deltaMagBkgPerCcd¶
Compute net offset from local background subtraction per-ccd? Otherwise, use computation per visit.
doMultipleCycles¶
Run multiple fit cycles in one task
doPlots¶
Make fgcm QA plots.
doReferenceCalibration¶
Use reference catalog as additional constraint on calibration
epochMjds¶
Epoch boundaries in MJD
expGrayErrRecoverCut¶
Maximum exposure gray error to be able to recover bad ccds via interpolation. Visits with more gray variance will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.
expGrayHighCut¶
Maximum (positive) exposure gray for a visit to be considered photometric. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expGrayHighCutDict instead.
expGrayHighCutDict¶
Per-band specification on maximum (positive) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be 0.2.
expGrayInitialCut¶
Maximum exposure/visit gray value for initial selection of possible photometric observations.
expGrayPhotometricCut¶
Maximum (negative) exposure gray for a visit to be considered photometric. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expGrayPhotometricCutDict instead.
expGrayPhotometricCutDict¶
Per-band specification on maximum (negative) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be -0.05.
expGrayRecoverCut¶
Maximum (negative) exposure gray to be able to recover bad ccds via interpolation. Visits with more gray extinction will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.
expVarGrayPhotometricCut¶
Maximum exposure variance to be considered possibly photometric Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use expVarGrayPhotometricCutDict instead.
expVarGrayPhotometricCutDict¶
Per-band specification on maximum exposure variance to be considered possibly photometric. Must have one entry per band. Broad-band filters should be 0.0005.
filterMap¶
Mapping from ‘filterName’ to band. Deprecated: This field is no longer used, and has been deprecated by DM-28088. It will be removed after v22. Use physicalFilterMap instead.
fitBands¶
Bands to use in atmospheric fit. The bands not listed here will have the atmosphere constrained from the ‘fitBands’ on the same night. Must be a subset of config.bands
fitFlag¶
Flag for which bands are directly constrained in the FGCM fit. Bands set to 0 will have the atmosphere constrained from observations in other bands on the same night. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use fitBands instead.
fitMirrorChromaticity¶
Fit (intraband) mirror chromatic term?
focalPlaneSigmaClip¶
Number of sigma to clip outliers per focal-plane.
freezeStdAtmosphere¶
Freeze atmosphere parameters to standard (for testing)
instrumentParsPerBand¶
Model instrumental parameters per band? Otherwise, instrumental parameters (QE changes with time) are shared among all bands.
instrumentSlopeMinDeltaT¶
Minimum time change (in days) between observations to use in constraining instrument slope.
isFinalCycle¶
Is this the final cycle of the fitting? Will automatically compute final selection of stars and photometric exposures, and will output zeropoints and standard stars for use in fgcmOutputProducts
latitude¶
Observatory latitude
mapLongitudeRef¶
Reference longitude for plotting maps
mapNSide¶
Healpix nside for plotting maps
maxCcdGrayErr¶
Maximum error on CCD gray offset to be considered photometric
maxIterBeforeFinalCycle¶
Maximum fit iterations, prior to final cycle. The number of iterations will always be 0 in the final cycle for cleanup and final selection.
minCcdPerExp¶
Minimum number of good CCDs per exposure/visit to be used in calibration fit. Visits with fewer good CCDs will have CCD zeropoints estimated where possible.
minExpPerNight¶
Minimum number of good exposures/visits to consider a partly photometric night
minObsPerBand¶
Minimum good observations per band
minStarPerCcd¶
Minimum number of good stars per CCD to be used in calibration fit. CCDs with fewer stars will have their calibration estimated from other CCDs in the same visit, with zeropoint error increased accordingly.
minStarPerExp¶
Minimum number of good stars per exposure/visit to be used in calibration fit. Visits with fewer good stars will have CCD zeropoints estimated where possible.
modelMagErrors¶
Should FGCM model the magnitude errors from sky/fwhm? (False means trust inputs)
multipleCyclesFinalCycleNumber¶
- Default
5- Field type
- Range
[2,10]
Final cycle number in multiple cycle mode. The initial cycle is 0, with limited parameters fit. The next cycle is 1 with full parameter fit. The final cycle is a clean-up with no parameters fit. There will be a total of (multipleCycleFinalCycleNumber + 1) cycles run, and the final cycle number cannot be less than 2.
nCore¶
Number of cores to use
nExpPerRun¶
Number of exposures to run in each chunk
nStarPerRun¶
Number of stars to run in each chunk
outfileBase¶
Filename start for plot output files
outputStandardsBeforeFinalCycle¶
Output standard stars prior to final cycle? Used in debugging.
outputZeropointsBeforeFinalCycle¶
Output standard stars prior to final cycle? Used in debugging.
physicalFilterMap¶
Mapping from ‘physicalFilter’ to band.
precomputeSuperStarInitialCycle¶
Precompute superstar flat for initial cycle
quietMode¶
Be less verbose with logging.
randomSeed¶
Random seed for fgcm for consistency in tests.
refStarOutlierNSig¶
Number of sigma compared to average mag for reference star to be considered an outlier. Computed per-band, and if it is an outlier in any band it is rejected from fits.
refStarSnMin¶
Reference star signal-to-noise minimum to use in calibration. Set to <=0 for no cut.
requiredBands¶
Bands that are required for a star to be considered a calibration star. Must be a subset of config.bands
requiredFlag¶
Flag for which bands are required for a star to be considered a calibration star in the FGCM fit. Typically this should be the same as fitFlag. Must be same length as config.bands, and matched band-by-band. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use requiredBands instead.
reserveFraction¶
Fraction of stars to reserve for testing
saveMetadata¶
Flag to enable/disable metadata saving for a task, enabled by default.
sedFudgeFactors¶
Fudge factors for computing linear SED from colors. Must be same length as config.bands, and matched band-by-band. Deprecated: This field has been deprecated and will be removed after v20. Please use sedSlopeTermMap and sedSlopeMap.
sedboundaryterms¶
- Data type
- Field type
Mapping from bands to SED boundary term names used is sedterms.
sedterms¶
- Data type
- Field type
Mapping from terms to bands for fgcm linear SED approximations.
sigFgcmMaxEGray¶
Maximum (absolute) gray value for observation in sigma_FGCM. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use sigFgcmMaxEGrayDict instead.
sigFgcmMaxEGrayDict¶
Per-band specification for maximum (absolute) achromatic residual (gray value) for observations in sigma_fgcm (raw repeatability). Broad-band filters should be 0.05.
sigFgcmMaxErr¶
Maximum mag error for fitting sigma_FGCM
sigma0Phot¶
Systematic error floor for all zeropoints
sigmaCalFitPercentile¶
Magnitude percentile range to fit systematic error floor
sigmaCalPlotPercentile¶
Magnitude percentile range to plot systematic error floor
sigmaCalRange¶
Allowed range for systematic error floor estimation
starColorCuts¶
Encoded star-color cuts (to be cleaned up)
superStarSigmaClip¶
Number of sigma to clip outliers when selecting for superstar flats
superStarSubCcd¶
Compute superstar flat on sub-ccd scale Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use superStarSubCcdDict instead.
superStarSubCcdChebyshevOrder¶
Order of the 2D chebyshev polynomials for sub-ccd superstar fit. Global default is first-order polynomials, and should be overridden on a camera-by-camera basis depending on the ISR.
superStarSubCcdDict¶
Per-band specification on whether to compute superstar flat on sub-ccd scale. Must have one entry per band.
superStarSubCcdTriangular¶
Should the sub-ccd superstar chebyshev matrix be triangular to suppress high-order cross terms?
useQuadraticPwv¶
Model PWV with a quadratic term for variation through the night?
useRepeatabilityForExpGrayCuts¶
Use star repeatability (instead of exposures) for computing photometric cuts? Recommended for tract mode or bands with few exposures. May be 1 element (same for all bands) or the same length as config.bands. Deprecated: This field is no longer used, and has been deprecated by DM-23699. It will be removed after v20. Use useRepeatabilityForExpGrayCutsDict instead.
useRepeatabilityForExpGrayCutsDict¶
Per-band specification on whether to use star repeatability (instead of exposures) for computing photometric cuts. Recommended for tract mode or bands with few visits.
utBoundary¶
Boundary (in UTC) from day-to-day
washMjds¶
Mirror wash MJDs
Examples¶
See the cookbook for worked examples.