FgcmFitCycleTask

FgcmFitCycleTask runs a single fgcm fit on the star observations generated from lsst.fgcmcal.fgcmBuildStars.FgcmBuildStarsTask using the look-up table generated from FgcmMakeLutTask. This code is meant to be run multiple times until convergence, and the results output from one “fit cycle” are used as an input to the subsequent fit cycle. One final cleanup run is performed to generate the tables required for input to FgcmOutputProductsTask.

This is the third task in a typical fgcmcal processing chain. The first is FgcmMakeLutTask, the second is FgcmBuildStarsTableTask or lsst.fgcmcal.fgcmBuildStars.FgcmBuildStarsTask, and the fourth is FgcmOutputProductsTask.

Processing summary

FgcmFitCycleTask reads in the star observations and the look-up table, performs an atmosphere and instrument fit, and outputs fit parameters as well as a comprehensive list of QA plots. If the config option isFinalCycle is set to True then additional datafiles are output that are used by FgcmOutputProductsTask.

FgcmFitCycleTask will output a new config file in the current working directory with recommended settings for the subsequent fit cycle. Furthermore, there are a wide range of diagnostic QA plots that are output by the task. For details on the contents of these QA plots, please see the cookbook as well as Burke, Rykoff, et al. 2018.

Python API summary

from lsst.fgcmcal.fgcmFitCycle import FgcmFitCycleTask
classFgcmFitCycleTask(initInputs=None, **kwargs)

Run Single fit cycle for FGCM global calibration...

attributeconfig

Access configuration fields and retargetable subtasks.

methodrun(**kwargs)

Run task algorithm on in-memory data...

See also

See the FgcmFitCycleTask API reference for complete details.

Retargetable subtasks

No subtasks.

Configuration fields

aperCorrFitNBins

Default
10
Field type

int Field

Number of aperture bins used in aperture correction fit. When set to 0no fit will be performed, and the config.aperCorrInputSlopes will be used if available.

aperCorrInputSlopeDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification of aperture correction input slope parameters. These are used on the first fit iteration, and aperture correction parameters will be updated from the data if config.aperCorrFitNBins > 0. It is recommended to set this when there is insufficient data to fit the parameters (e.g. tract mode).

applyRefStarColorCuts

Default
True
Field type

bool Field

Apply color cuts defined in starColorCuts to reference stars? These cuts are in addition to any cuts defined in refStarColorCuts

approxThroughputDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification of the approximate overall throughput at the start of calibration observations. Must have one entry per band. Typically should be 1.0.

autoHighCutNSig

Default
4.0
Field type

float Field

Number of sigma for automatic computation of (high) outlier cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).

autoPhotometricCutNSig

Default
3.0
Field type

float Field

Number of sigma for automatic computation of (low) photometric cut. Cut is based on exposure gray width (per band), unless useRepeatabilityForExpGrayCuts is set, in which case the star repeatability is used (also per band).

bands

Default
[]
Field type

str ListField

Bands to run calibration

brightObsGrayMax

Default
0.15
Field type

float Field

Maximum gray extinction to be considered bright observation

ccdGrayFocalPlaneChebyshevOrder

Default
3
Field type

int Field

Order of the 2D chebyshev polynomials for focal plane fit.

ccdGrayFocalPlaneDict

Default
{}
Field type

DictField

Key type

str

Value type

bool

Per-band specification on whether to compute focal-plane residual (‘ccd gray’) corrections.

ccdGrayFocalPlaneFitMinCcd

Default
1
Field type

int Field

Minimum number of ‘good’ CCDs required to perform focal-plane gray corrections. If there are fewer good CCDs then the gray correction is computed per-ccd.

ccdGrayMaxStarErr

Default
0.1
Field type

float Field

Maximum error on a star observation to use in ccd gray (achromatic residual) computation

ccdGraySubCcdChebyshevOrder

Default
1
Field type

int Field

Order of the 2D chebyshev polynomials for sub-ccd gray fit.

ccdGraySubCcdDict

Default
{}
Field type

DictField

Key type

str

Value type

bool

Per-band specification on whether to compute achromatic per-ccd residual (‘ccd gray’) on a sub-ccd scale.

ccdGraySubCcdTriangular

Default
True
Field type

bool Field

Should the sub-ccd gray chebyshev matrix be triangular to suppress high-order cross terms?

coatingMjds

Default
(0.0,)
Field type

float ListField

Mirror coating dates in MJD

colorSplitBands

Default
('g', 'i')
Field type

str ListField

Required length

2

Band names to use to split stars by color. Must have 2 entries.

connections

Data type

lsst.pipe.base.config.FgcmFitCycleConfigConnections

Field type

ConfigField

Configurations describing the connections of the PipelineTask to datatypes

cycleNumber

Default
None
Field type

int Field

FGCM fit cycle number. This is automatically incremented after each run and stage of outlier rejection. See cookbook for details.

defaultCameraOrientation

Default
None
Field type

float Field

Default camera orientation for QA plots.

deltaAperFitMinNgoodObs

Default
2
Field type

int Field

Minimum number of good observations to use mean delta-aper values in fits.

deltaAperFitPerCcdNx

Default
10
Field type

int Field

Number of x bins per ccd when computing delta-aper background offsets. Only used when doComputeDeltaAperPerCcd is True.

deltaAperFitPerCcdNy

Default
10
Field type

int Field

Number of y bins per ccd when computing delta-aper background offsets. Only used when doComputeDeltaAperPerCcd is True.

deltaAperFitSpatialNside

Default
64
Field type

int Field

Healpix nside to compute spatial delta-aper background offset maps.

deltaAperInnerRadiusArcsec

Default
0.0
Field type

float Field

Inner radius used to compute deltaMagAper (arcseconds). Must be positive and less than deltaAperOuterRadiusArcsec if any of doComputeDeltaAperPerVisit, doComputeDeltaAperPerStar, doComputeDeltaAperMap, doComputeDeltaAperPerCcd are set.

deltaAperOuterRadiusArcsec

Default
0.0
Field type

float Field

Outer radius used to compute deltaMagAper (arcseconds). Must be positive and greater than deltaAperInnerRadiusArcsec if any of doComputeDeltaAperPerVisit, doComputeDeltaAperPerStar, doComputeDeltaAperMap, doComputeDeltaAperPerCcd are set.

deltaMagBkgOffsetPercentile

Default
0.25
Field type

float Field

Percentile brightest stars on a visit/ccd to use to compute net offset from local background subtraction.

deltaMagBkgPerCcd

Default
False
Field type

bool Field

Compute net offset from local background subtraction per-ccd? Otherwise, use computation per visit.

doComputeDeltaAperMap

Default
False
Field type

bool Field

Do the computation of delta-aper spatial maps? This is only used if doComputeDeltaAperPerStar is True,

doComputeDeltaAperPerCcd

Default
False
Field type

bool Field

Do the computation of per-ccd delta-aper background offsets?

doComputeDeltaAperPerStar

Default
True
Field type

bool Field

Do the computation of delta-aper mean values per star?

doComputeDeltaAperPerVisit

Default
False
Field type

bool Field

Do the computation of delta-aper background offsets per visit? Note: this option can be very slow when there are many visits.

doMultipleCycles

Default
False
Field type

bool Field

Run multiple fit cycles in one task

doPlots

Default
True
Field type

bool Field

Make fgcm QA plots.

doReferenceCalibration

Default
True
Field type

bool Field

Use reference catalog as additional constraint on calibration

epochMjds

Default
(0.0,)
Field type

float ListField

Epoch boundaries in MJD

expGrayErrRecoverCut

Default
0.05
Field type

float Field

Maximum exposure gray error to be able to recover bad ccds via interpolation. Visits with more gray variance will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.

expGrayHighCutDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification on maximum (positive) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be 0.2.

expGrayInitialCut

Default
-0.25
Field type

float Field

Maximum exposure/visit gray value for initial selection of possible photometric observations.

expGrayPhotometricCutDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification on maximum (negative) achromatic exposure residual (‘gray term’) for a visit to be considered photometric. Must have one entry per band. Broad-band filters should be -0.05.

expGrayRecoverCut

Default
-1.0
Field type

float Field

Maximum (negative) exposure gray to be able to recover bad ccds via interpolation. Visits with more gray extinction will only get CCD zeropoints if there are sufficient star observations (minStarPerCcd) on that CCD.

expVarGrayPhotometricCutDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification on maximum exposure variance to be considered possibly photometric. Must have one entry per band. Broad-band filters should be 0.0005.

fitBands

Default
[]
Field type

str ListField

Bands to use in atmospheric fit. The bands not listed here will have the atmosphere constrained from the ‘fitBands’ on the same night. Must be a subset of config.bands

fitCcdChromaticityDict

Default
{}
Field type

DictField

Key type

str

Value type

bool

Specification on whether to compute first-order quantum efficiency (QE) adjustments. Key is band, and value will be True or False. Any band not explicitly specified will default to False.

fitMirrorChromaticity

Default
False
Field type

bool Field

Fit (intraband) mirror chromatic term?

focalPlaneSigmaClip

Default
4.0
Field type

float Field

Number of sigma to clip outliers per focal-plane.

freezeStdAtmosphere

Default
False
Field type

bool Field

Freeze atmosphere parameters to standard (for testing)

instrumentParsPerBand

Default
False
Field type

bool Field

Model instrumental parameters per band? Otherwise, instrumental parameters (QE changes with time) are shared among all bands.

instrumentSlopeMinDeltaT

Default
20.0
Field type

float Field

Minimum time change (in days) between observations to use in constraining instrument slope.

isFinalCycle

Default
False
Field type

bool Field

Is this the final cycle of the fitting? Will automatically compute final selection of stars and photometric exposures, and will output zeropoints and standard stars for use in fgcmOutputProducts

latitude

Default
None
Field type

float Field

Observatory latitude

mapLongitudeRef

Default
0.0
Field type

float Field

Reference longitude for plotting maps

mapNSide

Default
256
Field type

int Field

Healpix nside for plotting maps

maxCcdGrayErr

Default
0.05
Field type

float Field

Maximum error on CCD gray offset to be considered photometric

maxIterBeforeFinalCycle

Default
50
Field type

int Field

Maximum fit iterations, prior to final cycle. The number of iterations will always be 0 in the final cycle for cleanup and final selection.

minCcdPerExp

Default
5
Field type

int Field

Minimum number of good CCDs per exposure/visit to be used in calibration fit. Visits with fewer good CCDs will have CCD zeropoints estimated where possible.

minExpPerNight

Default
10
Field type

int Field

Minimum number of good exposures/visits to consider a partly photometric night

minObsPerBand

Default
2
Field type

int Field

Minimum good observations per band

minStarPerCcd

Default
5
Field type

int Field

Minimum number of good stars per CCD to be used in calibration fit. CCDs with fewer stars will have their calibration estimated from other CCDs in the same visit, with zeropoint error increased accordingly.

minStarPerExp

Default
600
Field type

int Field

Minimum number of good stars per exposure/visit to be used in calibration fit. Visits with fewer good stars will have CCD zeropoints estimated where possible.

mirrorArea

Default
None
Field type

float Field (optional)

Mirror area (square meters) of telescope. If not set, will try to estimate from camera.telescopeDiameter.

modelMagErrors

Default
True
Field type

bool Field

Should FGCM model the magnitude errors from sky/fwhm? (False means trust inputs)

multipleCyclesFinalCycleNumber

Default
5
Field type

int RangeField

Range

[2,10]

Final cycle number in multiple cycle mode. The initial cycle is 0, with limited parameters fit. The next cycle is 1 with full parameter fit. The final cycle is a clean-up with no parameters fit. There will be a total of (multipleCycleFinalCycleNumber + 1) cycles run, and the final cycle number cannot be less than 2.

nCore

Default
4
Field type

int Field

Number of cores to use Deprecated: Number of cores is deprecated as a config, and will be removed after v27. Please use pipetask run --cores-per-quantum instead.

nExpPerRun

Default
1000
Field type

int Field

Number of exposures to run in each chunk

nStarPerRun

Default
200000
Field type

int Field

Number of stars to run in each chunk

outfileBase

Default
None
Field type

str Field

Filename start for plot output files

outputStandardsBeforeFinalCycle

Default
False
Field type

bool Field

Output standard stars prior to final cycle? Used in debugging.

outputZeropointsBeforeFinalCycle

Default
False
Field type

bool Field

Output standard stars prior to final cycle? Used in debugging.

physicalFilterMap

Default
{}
Field type

DictField

Key type

str

Value type

str

Mapping from ‘physicalFilter’ to band.

precomputeSuperStarInitialCycle

Default
False
Field type

bool Field

Precompute superstar flat for initial cycle

quietMode

Default
False
Field type

bool Field

Be less verbose with logging.

randomSeed

Default
None
Field type

int Field (optional)

Random seed for fgcm for consistency in tests.

refStarColorCuts

Default
('NO_DATA',)
Field type

str ListField

Encoded star color cuts specifically to apply to reference stars. This is a list with each entry a string of the format band1,band2,low,high such that only stars of color low < band1 - band2 < high will be used as reference stars.

refStarMaxFracUse

Default
0.5
Field type

float Field

Maximum fraction of reference stars to use in the fit. Remainder will be used only for validation.

refStarOutlierNSig

Default
4.0
Field type

float Field

Number of sigma compared to average mag for reference star to be considered an outlier. Computed per-band, and if it is an outlier in any band it is rejected from fits.

refStarSnMin

Default
50.0
Field type

float Field

Reference star signal-to-noise minimum to use in calibration. Set to <=0 for no cut.

requiredBands

Default
[]
Field type

str ListField

Bands that are required for a star to be considered a calibration star. Must be a subset of config.bands

reserveFraction

Default
0.1
Field type

float Field

Fraction of stars to reserve for testing

saveLogOutput

Default
True
Field type

bool Field

Flag to enable/disable saving of log output for a task, enabled by default.

sedboundaryterms

Data type

lsst.fgcmcal.sedterms.SedboundarytermDict

Field type

ConfigField

Mapping from bands to SED boundary term names used is sedterms.

sedterms

Data type

lsst.fgcmcal.sedterms.SedtermDict

Field type

ConfigField

Mapping from terms to bands for fgcm linear SED approximations.

sigFgcmMaxEGrayDict

Default
{}
Field type

DictField

Key type

str

Value type

float

Per-band specification for maximum (absolute) achromatic residual (gray value) for observations in sigma_fgcm (raw repeatability). Broad-band filters should be 0.05.

sigFgcmMaxErr

Default
0.01
Field type

float Field

Maximum mag error for fitting sigma_FGCM

sigma0Phot

Default
0.003
Field type

float Field

Systematic error floor for all zeropoints

sigmaCalFitPercentile

Default
(0.05, 0.15)
Field type

float ListField

Magnitude percentile range to fit systematic error floor

sigmaCalPlotPercentile

Default
(0.05, 0.95)
Field type

float ListField

Magnitude percentile range to plot systematic error floor

sigmaCalRange

Default
(0.001, 0.003)
Field type

float ListField

Allowed range for systematic error floor estimation

starColorCuts

Default
('NO_DATA',)
Field type

str ListField

Encoded star-color cuts (using calibration star colors). This is a list with each entry a string of the format band1,band2,low,high such that only stars of color low < band1 - band2 < high will be used for calibration.

superStarPlotCcdResiduals

Default
False
Field type

bool Field

If plotting is enabled, should per-detector residuals be plotted? This may produce a lot of output, and should be used only for debugging purposes.

superStarSigmaClip

Default
5.0
Field type

float Field

Number of sigma to clip outliers when selecting for superstar flats

superStarSubCcdChebyshevOrder

Default
1
Field type

int Field

Order of the 2D chebyshev polynomials for sub-ccd superstar fit. Global default is first-order polynomials, and should be overridden on a camera-by-camera basis depending on the ISR.

superStarSubCcdDict

Default
{}
Field type

DictField

Key type

str

Value type

bool

Per-band specification on whether to compute superstar flat on sub-ccd scale. Must have one entry per band.

superStarSubCcdTriangular

Default
False
Field type

bool Field

Should the sub-ccd superstar chebyshev matrix be triangular to suppress high-order cross terms?

useExposureReferenceOffset

Default
False
Field type

bool Field

Use per-exposure (visit) offsets between calibrated stars and reference stars for final zeropoints? This may help uniformity for disjoint surveys.

useParquetCatalogFormat

Default
True
Field type

bool Field

Use parquet catalog format?

useQuadraticPwv

Default
False
Field type

bool Field

Model PWV with a quadratic term for variation through the night?

useRepeatabilityForExpGrayCutsDict

Default
{}
Field type

DictField

Key type

str

Value type

bool

Per-band specification on whether to use star repeatability (instead of exposures) for computing photometric cuts. Recommended for tract mode or bands with few visits.

utBoundary

Default
None
Field type

float Field

Boundary (in UTC) from day-to-day

washMjds

Default
(0.0,)
Field type

float ListField

Mirror wash MJDs