FgcmBuildStarsTask

FgcmBuildStarsTask finds all the single-visit sources in a repository (or a subset based on command-line parameters) and extracts all the potential photometric calibration stars for input into fgcm. This task additionally uses fgcm to match star observations into unique stars, and performs as much cleaning of the input catalog as possible. A tutorial on the steps of running fgcmcal are found in the cookbook.

This fgcmcal task runs on calexp source catalogs from visits constrained by the --id parameter on the command line. Best results are obtained when fgcmcal is run with full visits. If sourceTable_visit parquet tables are available, use of FgcmBuildStarsTableTask is recommended in place of this task.

In Gen2, due to limitations of the Gen2 Butler, optimal performance is obtained by specifying a single “reference” ccd on the command line (e.g. ccd=13) and setting the config variable checkAllCcds = True (which is the default). The alternative is to specify all the desired CCDs and set checkAllCcds = False, e.g., ccd=0..8^10..103. However, this is slower than the first option, and the improvement in speed in the first option is greater the more visits are specified. If instead you want to process all the visits in a rerun selected by filter, field, or some other dataid field, then by using a reference ccd and setting checkAllCcds = True you can speed things up by a factor of approximately 100 relative to the alternative (naming CCDs specifically).

Be aware that if a visit does not have a calexp available with the given reference CCD then it will be skipped. If this is possibly an issue, multiple reference ccds can be specified on the command line, although performance will degrade the more are specified.

At the current time, fgcmcal does not support Gen3.

This is the second task in a typical fgcmcal processing chain. The first is FgcmMakeLutTask, the third is FgcmFitCycleTask, and the fourth is FgcmOutputProductsTask.

FgcmBuildStarsTask is available as a command-line task, fgcmBuildStars.py.

Processing summary

FgcmBuildStarsTask runs this sequence of operations:

1. Finds unique visits and collates visit metadata, including exposure time, pointing, typical psf size, background level.

2. Reads in all sources, selecting good stars according to the chosen source selector.

3. Matches sources internally across bands to get a unique multi-band list of possible calibration stars.

4. Matches possible calibration stars to a reference catalog.

5. All results are stored in the output repo fgcm-process directory.

fgcmBuildStars.py command-line interface

fgcmBuildStars.py REPOPATH [@file [@file2 ...]] [--output OUTPUTREPO | --rerun RERUN] [--id] [other options]

Key arguments:

REPOPATH

The input Butler repository’s URI or file path.

Key options:

--id:

The data IDs to process.

See also

See Command-line task argument reference for details and additional options.

Python API summary

from lsst.fgcmcal.fgcmBuildStars import FgcmBuildStarsTask

classFgcmBuildStarsTask(butler=None, initInputs=None, **kwargs)

Build stars for the FGCM global calibration, using src catalogs...

attributeconfig

Access configuration fields and retargetable subtasks.

methodrun(**kwargs)

Run task algorithm on in-memory data...

methodrunDataRef(butler, dataRefs)

Cross-match and make star list for FGCM Input...

See also

See the FgcmBuildStarsTask API reference for complete details.

Butler datasets

When run as the fgcmBuildStars.py command-line task, or directly through the runDataRef method, FgcmBuildStarsTask obtains datasets from the input Butler data repository and persists outputs to the output Butler data repository. Note that configurations for FgcmBuildStarsTask, and its subtasks, affect what datasets are persisted and what their content is.

Input datasets

src

Full-depth source catalog (lsst.afw.table) produced by ProcessCcdTask

calexp

Calibrated exposure produced by ProcessCcdTask (for exposure metadata)

fgcmLookupTable

FGCM look-up table produced by FgcmMakeLutTask

Output datasets

fgcmVisitCatalog

Catalog (lsst.afw.table) of visit metadata

fgcmStarObservations

Catalog of star observations

fgcmStarIds

Catalog of unique star ids, positions, and number of observations

fgcmStarIndices

Catalog of indices linking unique star ids to star observations

fgcmReferenceStars

Catalog of reference stars matched to unique star ids.

Retargetable subtasks

fgcmLoadReferenceCatalog

Default

lsst.fgcmcal.fgcmLoadReferenceCatalog.FgcmLoadReferenceCatalogTask

Field type

ConfigurableField

FGCM reference object loader

sourceSelector

Default

'science'

Field type

Single-selection RegistryField

Choices

'science'

lsst.meas.algorithms.sourceSelector.ScienceSourceSelectorTask

'references'

lsst.meas.algorithms.sourceSelector.ReferenceSourceSelectorTask

'objectSize'

lsst.meas.algorithms.objectSizeStarSelector.ObjectSizeStarSelectorTask

'flagged'

lsst.meas.algorithms.flaggedSourceSelector.FlaggedSourceSelectorTask

'astrometry'

lsst.meas.algorithms.astrometrySourceSelector.AstrometrySourceSelectorTask

'matcher'

lsst.meas.algorithms.matcherSourceSelector.MatcherSourceSelectorTask

'diaCatalog'

lsst.ip.diffim.diaCatalogSourceSelector.DiaCatalogSourceSelectorTask

How to select sources

Configuration fields

apertureInnerInstFluxField

Default

'base_CircularApertureFlux_12_0_instFlux'

Field type

str Field

Full name of instFlux field that contains inner aperture flux for aperture correction proxy

apertureOuterInstFluxField

Default

'base_CircularApertureFlux_17_0_instFlux'

Field type

str Field

Full name of instFlux field that contains outer aperture flux for aperture correction proxy

ccdDataRefName

Default

'ccd'

Field type

str Field

dataRef name for the ‘ccd’ field, usually ‘ccd’ or ‘detector’.

checkAllCcds

Default

True

Field type

bool Field

Check repo for all CCDs for each visit specified. To be used when the full set of ids (visit/ccd) are not specified on the command line. For Gen2, specifying one ccd and setting checkAllCcds=True is significantly faster than the alternatives.

coarseNside

Default

8

Field type

int Field

Healpix coarse Nside for partitioning matches

densityCutMaxPerPixel

Default

1000

Field type

int Field

Density cut number of stars per pixel

densityCutNside

Default

128

Field type

int Field

Density cut healpix nside

doApplyWcsJacobian

Default

True

Field type

bool Field

Apply the jacobian of the WCS to the star observations prior to fit?

doModelErrorsWithBackground

Default

True

Field type

bool Field

Model flux errors with background term?

doReferenceMatches

Default

True

Field type

bool Field

Match reference catalog as additional constraint on calibration

doSubtractLocalBackground

Default

False

Field type

bool Field

Subtract the local background before performing calibration? This is only supported for circular aperture calibration fluxes.

filterMap

Default

{}

Field type

DictField

Key type

str

Value type

str

Mapping from ‘filterName’ to band. Deprecated: This field is no longer used, and has been deprecated by DM-28088. It will be removed after v22. Use physicalFilterMap instead.

instFluxField

Default

'slot_CalibFlux_instFlux'

Field type

str Field

Faull name of the source instFlux field to use, including ‘instFlux’. The associated flag will be implicitly included in badFlags

isolationRadius

Default

2.0

Field type

float Field

Isolation radius (arcseconds)

localBackgroundFluxField

Default

'base_LocalBackground_instFlux'

Field type

str Field

Full name of the local background instFlux field to use.

matchNside

Default

4096

Field type

int Field

Healpix Nside for matching

matchRadius

Default

1.0

Field type

float Field

Match radius (arcseconds)

minPerBand

Default

2

Field type

int Field

Minimum observations per band

nVisitsPerCheckpoint

Default

500

Field type

int Field

Number of visits read between checkpoints

physicalFilterMap

Default

{}

Field type

DictField

Key type

str

Value type

str

Mapping from ‘physicalFilter’ to band.

primaryBands

Default

None

Field type

str ListField

Bands for ‘primary’ star matches. A star must be observed in one of these bands to be considered as a calibration star.

psfCandidateName

Default

'calib_psf_candidate'

Field type

str Field

Name of field with psf candidate flag for propagation

referenceCCD

Default

13

Field type

int Field

Reference CCD for scanning visits

requiredBands

Default

()

Field type

str ListField

Bands required for each star

visitDataRefName

Default

'visit'

Field type

str Field

dataRef name for the ‘visit’ field, usually ‘visit’.

Examples

See the cookbook for worked examples.