FgcmFitCycleTask

FgcmFitCycleTask runs a single fgcm fit on the star observations generated from lsst.fgcmcal.fgcmBuildStars.FgcmBuildStarsTask using the look-up table generated from FgcmMakeLutTask. This code is meant to be run multiple times until convergence, and the results output from one “fit cycle” are used as an input to the subsequent fit cycle. One final cleanup run is performed to generate the tables required for input to FgcmOutputProductsTask.

This is the third task in a typical fgcmcal processing chain. The first is FgcmMakeLutTask, the second is FgcmBuildStarsTableTask or lsst.fgcmcal.fgcmBuildStars.FgcmBuildStarsTask, and the fourth is FgcmOutputProductsTask.

Processing summary

FgcmFitCycleTask reads in the star observations and the look-up table, performs an atmosphere and instrument fit, and outputs fit parameters as well as a comprehensive list of QA plots. If the config option isFinalCycle is set to True then additional datafiles are output that are used by FgcmOutputProductsTask.

FgcmFitCycleTask will output a new config file in the current working directory with recommended settings for the subsequent fit cycle. Furthermore, there are a wide range of diagnostic QA plots that are output by the task. For details on the contents of these QA plots, please see the cookbook as well as Burke, Rykoff, et al. 2018.

Python API summary

from lsst.fgcmcal.fgcmFitCycle import FgcmFitCycleTask
classFgcmFitCycleTask(initInputs=None, **kwargs)

Run Single fit cycle for FGCM global calibration...

attributeconfig

Access configuration fields and retargetable subtasks.

methodrun(**kwargs) -> Struct)

Run task algorithm on in-memory data...

See also

See the FgcmFitCycleTask API reference for complete details.

Retargetable subtasks

No subtasks.

Configuration fields

aperCorrFitNBins

Default
10
Field type
int Field
Number of aperture bins used in aperture correction fit. When set to 0no fit will be performed, and the config.aperCorrInputSlopes will be used if available.

aperCorrInputSlopeDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification of aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set this when there is insufficient data to fit the parameters (e.g. tract mode).

applyRefStarColorCuts

Default
True
Field type
bool Field
Apply color cuts to reference stars?

approxThroughputDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification of the approximate overall throughput at the start of calibration observations. Must have one entry per band. Typically should be 1.0.

autoHighCutNSig

Default
4.0
Field type
float Field
Number of sigma for automatic computation of (high) outlier cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).

autoPhotometricCutNSig

Default
3.0
Field type
float Field
Number of sigma for automatic computation of (low) photometric cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).

bands

Default
[]
Field type
str ListField
Bands to run calibration

brightObsGrayMax

Default
0.15
Field type
float Field
Maximum gray extinction to be considered bright observation

ccdGrayFocalPlaneChebyshevOrder

Default
3
Field type
int Field
Order of the 2D chebyshev polynomials for focal plane fit.

ccdGrayFocalPlaneDict

Default
{}
Field type
DictField
Key type
str
Value type
bool
Per-band specification on whether to compute focal-plane residual (‘ccd gray’) corrections.

ccdGrayFocalPlaneFitMinCcd

Default
1
Field type
int Field
Minimum number of ‘good’ CCDs required to perform focal-plane gray corrections. If there are fewer good CCDs then the gray correction is computed per-ccd.

ccdGrayMaxStarErr

Default
0.1
Field type
float Field
Maximum error on a star observation to use in ccd gray (achromatic residual) computation

ccdGraySubCcdChebyshevOrder

Default
1
Field type
int Field
Order of the 2D chebyshev polynomials for sub-ccd gray fit.

ccdGraySubCcdDict

Default
{}
Field type
DictField
Key type
str
Value type
bool
Per-band specification on whether to compute achromatic per-ccd residual (‘ccd gray’) on a sub-ccd scale.

ccdGraySubCcdTriangular

Default
True
Field type
bool Field
Should the sub-ccd gray chebyshev matrix be triangular to suppress high-order cross terms?

coatingMjds

Default
(0.0,)
Field type
float ListField
Mirror coating dates in MJD

colorSplitBands

Default
('g', 'i')
Field type
str ListField
Required length
2
Band names to use to split stars by color. Must have 2 entries.

connections

Data type
lsst.pipe.base.config.Connections
Field type
ConfigField
Configurations describing the connections of the PipelineTask to datatypes

cycleNumber

Default
None
Field type
int Field
FGCM fit cycle number. This is automatically incremented after each run and stage of outlier rejection. See cookbook for details.

defaultCameraOrientation

Default
None
Field type
float Field
Default camera orientation for QA plots.

deltaAperFitMinNgoodObs

Default
2
Field type
int Field
Minimum number of good observations to use mean delta-aper values in fits.

deltaAperFitPerCcdNx

Default
10
Field type
int Field
Number of x bins per ccd when computing delta-aper background offsets. Only used when doComputeDeltaAperPerCcd is True.

deltaAperFitPerCcdNy

Default
10
Field type
int Field
Number of y bins per ccd when computing delta-aper background offsets. Only used when doComputeDeltaAperPerCcd is True.

deltaAperFitSpatialNside

Default
64
Field type
int Field
Healpix nside to compute spatial delta-aper background offset maps.

deltaAperInnerRadiusArcsec

Default
0.0
Field type
float Field
Inner radius used to compute deltaMagAper (arcseconds). Must be positive and less than deltaAperOuterRadiusArcsec if any of doComputeDeltaAperPerVisit, doComputeDeltaAperPerStar, doComputeDeltaAperMap, doComputeDeltaAperPerCcd are set.

deltaAperOuterRadiusArcsec

Default
0.0
Field type
float Field
Outer radius used to compute deltaMagAper (arcseconds). Must be positive and greater than deltaAperInnerRadiusArcsec if any of doComputeDeltaAperPerVisit, doComputeDeltaAperPerStar, doComputeDeltaAperMap, doComputeDeltaAperPerCcd are set.

deltaMagBkgOffsetPercentile

Default
0.25
Field type
float Field
Percentile brightest stars on a visit/ccd to use to compute net offset from local background subtraction.

deltaMagBkgPerCcd

Default
False
Field type
bool Field
Compute net offset from local background subtraction per-ccd? Otherwise, use computation per visit.

doComputeDeltaAperMap

Default
False
Field type
bool Field
Do the computation of delta-aper spatial maps? This is only used if doComputeDeltaAperPerStar is True,

doComputeDeltaAperPerCcd

Default
False
Field type
bool Field
Do the computation of per-ccd delta-aper background offsets?

doComputeDeltaAperPerStar

Default
True
Field type
bool Field
Do the computation of delta-aper mean values per star?

doComputeDeltaAperPerVisit

Default
False
Field type
bool Field
Do the computation of delta-aper background offsets per visit? Note: this option can be very slow when there are many visits.

doMultipleCycles

Default
False
Field type
bool Field
Run multiple fit cycles in one task

doPlots

Default
True
Field type
bool Field
Make fgcm QA plots.

doReferenceCalibration

Default
True
Field type
bool Field
Use reference catalog as additional constraint on calibration

epochMjds

Default
(0.0,)
Field type
float ListField
Epoch boundaries in MJD

expGrayErrRecoverCut

Default
0.05
Field type
float Field
Maximum exposure gray error to be able to recover bad ccds via interpolation. Visits with more gray variance will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.

expGrayHighCutDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification on maximum (positive) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be 0.2.

expGrayInitialCut

Default
-0.25
Field type
float Field
Maximum exposure/visit gray value for initial selection of possible photometric observations.

expGrayPhotometricCutDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification on maximum (negative) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be -0.05.

expGrayRecoverCut

Default
-1.0
Field type
float Field
Maximum (negative) exposure gray to be able to recover bad ccds via interpolation. Visits with more gray extinction will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.

expVarGrayPhotometricCutDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification on maximum exposure variance to be considered possibly photometric. Must have one entry per band. Broad-band filters should be 0.0005.

fitBands

Default
[]
Field type
str ListField
Bands to use in atmospheric fit. The bands not listed here will have the atmosphere constrained from the ‘fitBands’ on the same night. Must be a subset of config.bands

fitMirrorChromaticity

Default
False
Field type
bool Field
Fit (intraband) mirror chromatic term?

focalPlaneSigmaClip

Default
4.0
Field type
float Field
Number of sigma to clip outliers per focal-plane.

freezeStdAtmosphere

Default
False
Field type
bool Field
Freeze atmosphere parameters to standard (for testing)

instrumentParsPerBand

Default
False
Field type
bool Field
Model instrumental parameters per band? Otherwise, instrumental parameters (QE changes with time) are shared among all bands.

instrumentSlopeMinDeltaT

Default
20.0
Field type
float Field
Minimum time change (in days) between observations to use in constraining instrument slope.

isFinalCycle

Default
False
Field type
bool Field
Is this the final cycle of the fitting? Will automatically compute final selection of stars and photometric exposures, and will output zeropoints and standard stars for use in fgcmOutputProducts

latitude

Default
None
Field type
float Field
Observatory latitude

mapLongitudeRef

Default
0.0
Field type
float Field
Reference longitude for plotting maps

mapNSide

Default
256
Field type
int Field
Healpix nside for plotting maps

maxCcdGrayErr

Default
0.05
Field type
float Field
Maximum error on CCD gray offset to be considered photometric

maxIterBeforeFinalCycle

Default
50
Field type
int Field
Maximum fit iterations, prior to final cycle. The number of iterations will always be 0 in the final cycle for cleanup and final selection.

minCcdPerExp

Default
5
Field type
int Field
Minimum number of good CCDs per exposure/visit to be used in calibration fit. Visits with fewer good CCDs will have CCD zeropoints estimated where possible.

minExpPerNight

Default
10
Field type
int Field
Minimum number of good exposures/visits to consider a partly photometric night

minObsPerBand

Default
2
Field type
int Field
Minimum good observations per band

minStarPerCcd

Default
5
Field type
int Field
Minimum number of good stars per CCD to be used in calibration fit. CCDs with fewer stars will have their calibration estimated from other CCDs in the same visit, with zeropoint error increased accordingly.

minStarPerExp

Default
600
Field type
int Field
Minimum number of good stars per exposure/visit to be used in calibration fit. Visits with fewer good stars will have CCD zeropoints estimated where possible.

modelMagErrors

Default
True
Field type
bool Field
Should FGCM model the magnitude errors from sky/fwhm? (False means trust inputs)

multipleCyclesFinalCycleNumber

Default
5
Field type
int RangeField
Range
[2,10]
Final cycle number in multiple cycle mode. The initial cycle is 0, with limited parameters fit. The next cycle is 1 with full parameter fit. The final cycle is a clean-up with no parameters fit. There will be a total of (multipleCycleFinalCycleNumber + 1) cycles run, and the final cycle number cannot be less than 2.

nCore

Default
4
Field type
int Field
Number of cores to use

nExpPerRun

Default
1000
Field type
int Field
Number of exposures to run in each chunk

nStarPerRun

Default
200000
Field type
int Field
Number of stars to run in each chunk

outfileBase

Default
None
Field type
str Field
Filename start for plot output files

outputStandardsBeforeFinalCycle

Default
False
Field type
bool Field
Output standard stars prior to final cycle? Used in debugging.

outputZeropointsBeforeFinalCycle

Default
False
Field type
bool Field
Output standard stars prior to final cycle? Used in debugging.

physicalFilterMap

Default
{}
Field type
DictField
Key type
str
Value type
str
Mapping from ‘physicalFilter’ to band.

precomputeSuperStarInitialCycle

Default
False
Field type
bool Field
Precompute superstar flat for initial cycle

quietMode

Default
False
Field type
bool Field
Be less verbose with logging.

randomSeed

Default
None
Field type
int Field (optional)
Random seed for fgcm for consistency in tests.

refStarOutlierNSig

Default
4.0
Field type
float Field
Number of sigma compared to average mag for reference star to be considered an outlier. Computed per-band, and if it is an outlier in any band it is rejected from fits.

refStarSnMin

Default
50.0
Field type
float Field
Reference star signal-to-noise minimum to use in calibration. Set to <=0 for no cut.

requiredBands

Default
[]
Field type
str ListField
Bands that are required for a star to be considered a calibration star. Must be a subset of config.bands

reserveFraction

Default
0.1
Field type
float Field
Fraction of stars to reserve for testing

saveLogOutput

Default
True
Field type
bool Field
Flag to enable/disable saving of log output for a task, enabled by default.

saveMetadata

Default
True
Field type
bool Field
Flag to enable/disable metadata saving for a task, enabled by default.

sedboundaryterms

Data type
lsst.fgcmcal.sedterms.SedboundarytermDict
Field type
ConfigField
Mapping from bands to SED boundary term names used is sedterms.

sedterms

Data type
lsst.fgcmcal.sedterms.SedtermDict
Field type
ConfigField
Mapping from terms to bands for fgcm linear SED approximations.

sigFgcmMaxEGrayDict

Default
{}
Field type
DictField
Key type
str
Value type
float
Per-band specification for maximum (absolute) achromatic residual (gray value) for observations in sigma_fgcm (raw repeatability). Broad-band filters should be 0.05.

sigFgcmMaxErr

Default
0.01
Field type
float Field
Maximum mag error for fitting sigma_FGCM

sigma0Phot

Default
0.003
Field type
float Field
Systematic error floor for all zeropoints

sigmaCalFitPercentile

Default
(0.05, 0.15)
Field type
float ListField
Magnitude percentile range to fit systematic error floor

sigmaCalPlotPercentile

Default
(0.05, 0.95)
Field type
float ListField
Magnitude percentile range to plot systematic error floor

sigmaCalRange

Default
(0.001, 0.003)
Field type
float ListField
Allowed range for systematic error floor estimation

starColorCuts

Default
('NO_DATA',)
Field type
str ListField
Encoded star-color cuts (to be cleaned up)

superStarSigmaClip

Default
5.0
Field type
float Field
Number of sigma to clip outliers when selecting for superstar flats

superStarSubCcdChebyshevOrder

Default
1
Field type
int Field
Order of the 2D chebyshev polynomials for sub-ccd superstar fit. Global default is first-order polynomials, and should be overridden on a camera-by-camera basis depending on the ISR.

superStarSubCcdDict

Default
{}
Field type
DictField
Key type
str
Value type
bool
Per-band specification on whether to compute superstar flat on sub-ccd scale. Must have one entry per band.

superStarSubCcdTriangular

Default
False
Field type
bool Field
Should the sub-ccd superstar chebyshev matrix be triangular to suppress high-order cross terms?

useQuadraticPwv

Default
False
Field type
bool Field
Model PWV with a quadratic term for variation through the night?

useRepeatabilityForExpGrayCutsDict

Default
{}
Field type
DictField
Key type
str
Value type
bool
Per-band specification on whether to use star repeatability (instead of exposures) for computing photometric cuts. Recommended for tract mode or bands with few visits.

utBoundary

Default
None
Field type
float Field
Boundary (in UTC) from day-to-day

washMjds

Default
(0.0,)
Field type
float ListField
Mirror wash MJDs