ConvertReferenceCatalogTask¶
ConvertReferenceCatalogTask converts an external catalog for use as an LSST Science Pipelines reference catalog, using a Hierarchical Triangular Mesh (HTM) indexing scheme. The format and layout of the input data is configurable. The output data is a collection of lsst.afw.table.SimpleCatalog files identified by their HTM pixel. This task is not available as a pipeline task: see How to generate an LSST reference catalog for how to run the task.
Processing summary¶
ConvertReferenceCatalogTask uses Python multiprocessing to ingest multiple files in parallel, configured by n_processes.
Once it has generated the necessary multiprocessing file locks (one per output file: ~130,000 files for HTM depth=7), it performs the following steps for each input file:
Reads the file using the configured
file_readersubtask (default:ReadTextCatalogTask).Indexes the coordinates in the input data to determine which mesh pixel they go with, and thus which output file they will be written to.
Loops over the output pixels in this input file (where N is the number of sources in this pixel):
Acquires the lock for this output file.
Reads an existing output file and appends N new empty rows, or generates a new empty catalog with N rows.
Fills in the empty rows of the catalog with the converted values from the input data.
Writes the output file and releases the file lock.
Python API summary¶
from lsst.meas.algorithms.convertReferenceCatalog import ConvertReferenceCatalogTask
-
class
ConvertReferenceCatalogTask(*, output_dir=None, **kwargs) Class for producing HTM-indexed reference catalogs from external catalog data
...
- attributeconfig
Access configuration fields and retargetable subtasks.
See also
See the ConvertReferenceCatalogTask API reference for complete details.
Butler datasets¶
ConvertReferenceCatalogTask does not behave in the same manner as most LSST Tasks.
When run directly through the run method (e.g. via the convertReferenceCatalog commandline interface), ConvertReferenceCatalogTask reads input from a collection of non-LSST files, and writes to a specified directory.
Note that configurations for ConvertReferenceCatalogTask, and its subtasks, affect what the output dataset content is.
Output datasets¶
ref_catAn LSST-style reference catalog, consisting of one
lsst.afw.table.SimpleCatalogper HTM pixel.
Retargetable subtasks¶
file_reader¶
Task to use to read the files. Default is to expect text files.
manager¶
- Default
lsst.meas.algorithms.convertRefcatManager.ConvertRefcatManager- Field type
Multiprocessing manager to perform the actual conversion of values, file-by-file.
Configuration fields¶
coord_err_unit¶
Unit of RA/Dec error fields (astropy.unit.Unit compatible)
dataset_config¶
- Data type
lsst.meas.algorithms.convertReferenceCatalog.DatasetConfig- Field type
Configuration for reading the ingested data
dec_err_name¶
Name of Dec error column
dec_name¶
Name of Dec column (values in decimal degrees)
epoch_format¶
Format of epoch column: any value accepted by astropy.time.Time, e.g. ‘iso’ or ‘unix’
epoch_name¶
Name of epoch column
epoch_scale¶
Scale of epoch column: any value accepted by astropy.time.Time, e.g. ‘utc’
extra_col_names¶
Extra columns to add to the reference catalog.
full_position_information¶
Include epoch, proper motions, parallax, and covariances between sky coordinates, proper motion, and parallax in the schema. If true, a custom ConvertRefcatManager class must exist to compute the output covariances.
id_name¶
Name of column to use as an identifier (optional).
is_photometric_name¶
Name of column stating if satisfactory for photometric calibration (optional).
is_resolved_name¶
Name of column stating if the object is resolved (optional).
is_variable_name¶
Name of column stating if the object is measured to be variable (optional).
mag_column_list¶
The values in the reference catalog are assumed to be in AB magnitudes. List of column names to use for photometric information. At least one entry is required.
mag_err_column_map¶
A map of magnitude column name (key) to magnitude error column (value).
n_processes¶
Number of python processes to use when ingesting.
parallax_err_name¶
Name of parallax error column
parallax_name¶
Name of parallax column
parallax_scale¶
Scale factor by which to multiply parallax values to obtain units of milliarcsec
pm_dec_err_name¶
Name of proper motion Dec error column
pm_dec_name¶
Name of proper motion Dec column
pm_ra_err_name¶
Name of proper motion RA error column
pm_ra_name¶
Name of proper motion RA column
pm_scale¶
Scale factor by which to multiply proper motion values to obtain units of milliarcsec/year
ra_err_name¶
Name of RA error column
ra_name¶
Name of RA column (values in decimal degrees)
Examples¶
See How to generate an LSST reference catalog for a description of how to run the task to ingest the Gaia DR2 catalog.