DatasetIngestConfig

class lsst.ap.verify.DatasetIngestConfig

Bases: lsst.pex.config.config.Config

Settings and defaults for DatasetIngestTask.

The correct targets for this task’s subtasks can be found in the documentation of the appropriate obs package.

Because DatasetIngestTask is not designed to be run from the command line, and its arguments are completely determined by the choice of dataset, this config includes settings that would normally be passed as command-line arguments to IngestTask.

Attributes Summary

calibBadFiles	Names of calib files (no path; wildcards allowed) to not ingest, supersedes calibFiles.
calibFiles	Names of calib files (no path; wildcards allowed) to ingest from the dataset.
calibIngester	Task used to ingest flats, biases, darks, fringes, or sky.
calibValidity	Calibration validity period (days).
dataBadFiles	Names of raw science files (no path; wildcards allowed) to not ingest, supersedes dataFiles.
dataFiles	Names of raw science files (no path; wildcards allowed) to ingest from the dataset.
dataIngester	Task used to perform raw data ingestion.
defectIngester	Task used to ingest defects.
defectTarball	Name of tar.gz file containing defects.
defectValidity	Defect validity period (days).
history
refcats	Map from a refcat name to a tar.gz file containing the sharded catalog.

Methods Summary

compare(other[, shortcut, rtol, atol, output])	!Compare two Configs for equality; return True if equal
formatHistory(name, **kwargs)	!Format the specified config field’s history to a more human-readable format
freeze()	!Make this Config and all sub-configs read-only
items()	!Return the list of (field name, field value) pairs
iteritems()	!Iterate over (field name, field value) pairs
iterkeys()	!Iterate over field names
itervalues()	!Iterate over field values
keys()	!Return the list of field names
load(filename[, root])	!Modify this config in place by executing the Python code in the named file.
loadFromStream(stream[, root, filename])	!Modify this config in place by executing the python code in the provided stream.
names()	!Return all the keys in a config recursively
save(filename[, root])	!Save a python script to the named file, which, when loaded, reproduces this Config
saveToStream(outfile[, root])	!Save a python script to a stream, which, when loaded, reproduces this Config
setDefaults()	Derived config classes that must compute defaults rather than using the Field defaults should do so here.
toDict()	!Return a dict of field name: value
update(**kw)	!Update values specified by the keyword arguments
validate()	!Validate the Config; raise an exception if invalid
values()	!Return the list of field values

Attributes Documentation

calibBadFiles

Names of calib files (no path; wildcards allowed) to not ingest, supersedes calibFiles. (List, default [])

calibFiles

Names of calib files (no path; wildcards allowed) to ingest from the dataset. (List, default ['*.fits', '*.fz', '*.fits.gz'])

calibIngester

Task used to ingest flats, biases, darks, fringes, or sky. (ConfigurableInstance, default <class 'lsst.pipe.tasks.ingestCalibs.IngestCalibsConfig'>)

calibValidity

Calibration validity period (days). Assumed equal for all calib types. (int, default 9999)

dataBadFiles

Names of raw science files (no path; wildcards allowed) to not ingest, supersedes dataFiles. (List, default [])

dataFiles

Names of raw science files (no path; wildcards allowed) to ingest from the dataset. (List, default ['*.fits', '*.fz', '*.fits.gz'])

dataIngester

Task used to perform raw data ingestion. (ConfigurableInstance, default <class 'lsst.pipe.tasks.ingest.IngestConfig'>)

defectIngester

Task used to ingest defects. (ConfigurableInstance, default <class 'lsst.pipe.tasks.ingestCalibs.IngestCalibsConfig'>)

defectTarball

Name of tar.gz file containing defects. May be empty. Defect files may be in any format and directory layout supported by the obs package. (str, default None)

defectValidity

Defect validity period (days). (int, default 9999)

history

refcats

Map from a refcat name to a tar.gz file containing the sharded catalog. May be empty. (Dict, default {})

Methods Documentation

compare(other, shortcut=True, rtol=1e-08, atol=1e-08, output=None)

!Compare two Configs for equality; return True if equal

If the Configs contain RegistryFields or ConfigChoiceFields, unselected Configs will not be compared.

@param[in] other Config object to compare with self. @param[in] shortcut If True, return as soon as an inequality is found. @param[in] rtol Relative tolerance for floating point comparisons. @param[in] atol Absolute tolerance for floating point comparisons. @param[in] output If not None, a callable that takes a string, used (possibly repeatedly)

to report inequalities.

Floating point comparisons are performed by numpy.allclose; refer to that for details.

formatHistory(name, **kwargs)

!Format the specified config field’s history to a more human-readable format

@param[in] name name of field whose history is wanted @param[in] kwargs keyword arguments for lsst.pex.config.history.format @return a string containing the formatted history

freeze()

!Make this Config and all sub-configs read-only

items()

!Return the list of (field name, field value) pairs

iteritems()

!Iterate over (field name, field value) pairs

iterkeys()

!Iterate over field names

itervalues()

!Iterate over field values

keys()

!Return the list of field names

load(filename, root='config')

!Modify this config in place by executing the Python code in the named file.

@param[in] filename name of file containing config override code @param[in] root name of variable in file that refers to the config being overridden

For example: if the value of root is “config” and the file contains this text: “config.myField = 5” then this config’s field “myField” is set to 5.

@deprecated For purposes of backwards compatibility, older config files that use root=”root” instead of root=”config” will be loaded with a warning printed to sys.stderr. This feature will be removed at some point.

loadFromStream(stream, root='config', filename=None)

!Modify this config in place by executing the python code in the provided stream.

@param[in] stream open file object, string or compiled string containing config override code @param[in] root name of variable in stream that refers to the config being overridden @param[in] filename name of config override file, or None if unknown or contained

in the stream; used for error reporting

For example: if the value of root is “config” and the stream contains this text: “config.myField = 5” then this config’s field “myField” is set to 5.

@deprecated For purposes of backwards compatibility, older config files that use root=”root” instead of root=”config” will be loaded with a warning printed to sys.stderr. This feature will be removed at some point.

names()

!Return all the keys in a config recursively

save(filename, root='config')

!Save a python script to the named file, which, when loaded, reproduces this Config

@param[in] filename name of file to which to write the config @param[in] root name to use for the root config variable; the same value must be used when loading

saveToStream(outfile, root='config')

!Save a python script to a stream, which, when loaded, reproduces this Config

@param outfile [inout] open file object to which to write the config. Accepts strings not bytes. @param root [in] name to use for the root config variable; the same value must be used when loading

setDefaults()

Derived config classes that must compute defaults rather than using the Field defaults should do so here. To correctly use inherited defaults, implementations of setDefaults() must call their base class’ setDefaults()

toDict()

!Return a dict of field name: value

Correct behavior is dependent on proper implementation of Field.toDict. If implementing a new Field type, you may need to implement your own toDict method.

update(**kw)

!Update values specified by the keyword arguments

@warning The ‘__at’ and ‘__label’ keyword arguments are special internal keywords. They are used to strip out any internal steps from the history tracebacks of the config. Modifying these keywords allows users to lie about a Config’s history. Please do not do so!

validate()

!Validate the Config; raise an exception if invalid

The base class implementation performs type checks on all fields by calling Field.validate().

Complex single-field validation can be defined by deriving new Field types. As syntactic sugar, some derived Field types are defined in this module which handle recursing into sub-configs (ConfigField, ConfigChoiceField)

Inter-field relationships should only be checked in derived Config classes after calling this method, and base validation is complete

values()

!Return the list of field values