MetricsControllerTask¶
-
class
lsst.verify.gen2tasks.
MetricsControllerTask
(config=None, **kwargs)¶ Bases:
lsst.pipe.base.Task
A Task for executing a collection of
lsst.verify.tasks.MetricTask
objects.This class handles Butler input of datasets needed by metrics, as well as persistence of the resulting measurements.
Notes
MetricsControllerTask
is a stand-in for functionality provided by the Gen 3 Tasks framework. It will become redundant once we fully adopt that framework.Because
MetricsControllerTask
cannot support the full functionality of the Gen 3 framework, it places several restrictions on its metrics:- each
MetricTask
must measure a unique metric - no
MetricTask
may depend on the output of anotherMetricTask
- the granularity of the metrics is determined by the inputs to
runDataRefs
; configuration information specifying a different granularity is allowed but is ignored
Attributes Summary
measurers
The tasks to be executed by this object (iterable of lsst.verify.tasks.MetricTask
).Methods Summary
emptyMetadata
()Empty (clear) the metadata for this Task and all sub-Tasks. getAllSchemaCatalogs
()Get schema catalogs for all tasks in the hierarchy, combining the results into a single dict. getFullMetadata
()Get metadata for all tasks. getFullName
()Get the task name as a hierarchical name including parent task names. getName
()Get the name of the task. getSchemaCatalogs
()Get the schemas generated by this task. getTaskDict
()Get a dictionary of all tasks as a shallow copy. makeField
(doc)Make a lsst.pex.config.ConfigurableField
for this task.makeSubtask
(name, **keyArgs)Create a subtask as a new instance as the name
attribute of this task.runDataRefs
(datarefs[, customMetadata, …])Call all registered metric tasks on each dataref. timer
(name[, logLevel])Context manager to log performance data for an arbitrary block of code. Attributes Documentation
-
measurers
= []¶ The tasks to be executed by this object (iterable of
lsst.verify.tasks.MetricTask
).
Methods Documentation
-
emptyMetadata
()¶ Empty (clear) the metadata for this Task and all sub-Tasks.
-
getAllSchemaCatalogs
()¶ Get schema catalogs for all tasks in the hierarchy, combining the results into a single dict.
Returns: - schemacatalogs :
dict
Keys are butler dataset type, values are a empty catalog (an instance of the appropriate
lsst.afw.table
Catalog type) for all tasks in the hierarchy, from the top-level task down through all subtasks.
Notes
This method may be called on any task in the hierarchy; it will return the same answer, regardless.
The default implementation should always suffice. If your subtask uses schemas the override
Task.getSchemaCatalogs
, not this method.- schemacatalogs :
-
getFullMetadata
()¶ Get metadata for all tasks.
Returns: - metadata :
lsst.daf.base.PropertySet
The
PropertySet
keys are the full task name. Values are metadata for the top-level task and all subtasks, sub-subtasks, etc.
Notes
The returned metadata includes timing information (if
@timer.timeMethod
is used) and any metadata set by the task. The name of each item consists of the full task name with.
replaced by:
, followed by.
and the name of the item, e.g.:topLevelTaskName:subtaskName:subsubtaskName.itemName
using
:
in the full task name disambiguates the rare situation that a task has a subtask and a metadata item with the same name.- metadata :
-
getFullName
()¶ Get the task name as a hierarchical name including parent task names.
Returns: - fullName :
str
The full name consists of the name of the parent task and each subtask separated by periods. For example:
- The full name of top-level task “top” is simply “top”.
- The full name of subtask “sub” of top-level task “top” is “top.sub”.
- The full name of subtask “sub2” of subtask “sub” of top-level task “top” is “top.sub.sub2”.
- fullName :
-
getSchemaCatalogs
()¶ Get the schemas generated by this task.
Returns: - schemaCatalogs :
dict
Keys are butler dataset type, values are an empty catalog (an instance of the appropriate
lsst.afw.table
Catalog type) for this task.
See also
Task.getAllSchemaCatalogs
Notes
Warning
Subclasses that use schemas must override this method. The default implementation returns an empty dict.
This method may be called at any time after the Task is constructed, which means that all task schemas should be computed at construction time, not when data is actually processed. This reflects the philosophy that the schema should not depend on the data.
Returning catalogs rather than just schemas allows us to save e.g. slots for SourceCatalog as well.
- schemaCatalogs :
-
getTaskDict
()¶ Get a dictionary of all tasks as a shallow copy.
Returns: - taskDict :
dict
Dictionary containing full task name: task object for the top-level task and all subtasks, sub-subtasks, etc.
- taskDict :
-
classmethod
makeField
(doc)¶ Make a
lsst.pex.config.ConfigurableField
for this task.Parameters: - doc :
str
Help text for the field.
Returns: - configurableField :
lsst.pex.config.ConfigurableField
A
ConfigurableField
for this task.
Examples
Provides a convenient way to specify this task is a subtask of another task.
Here is an example of use:
class OtherTaskConfig(lsst.pex.config.Config): aSubtask = ATaskClass.makeField("brief description of task")
- doc :
-
makeSubtask
(name, **keyArgs)¶ Create a subtask as a new instance as the
name
attribute of this task.Parameters: - name :
str
Brief name of the subtask.
- keyArgs
Extra keyword arguments used to construct the task. The following arguments are automatically provided and cannot be overridden:
- “config”.
- “parentTask”.
Notes
The subtask must be defined by
Task.config.name
, an instance ofConfigurableField
orRegistryField
.- name :
-
runDataRefs
(datarefs, customMetadata=None, skipExisting=False)¶ Call all registered metric tasks on each dataref.
This method loads all datasets required to compute a particular metric, and persists the metrics as one or more
lsst.verify.Job
objects. Only metrics that successfully produce aMeasurement
will be included in a job.Parameters: - datarefs :
list
oflsst.daf.persistence.ButlerDataRef
The data to measure. Datarefs may be complete or partial; each generates a measurement at the same granularity (e.g., a dataref with only
"visit"
specified generates visit-level measurements).- customMetadata :
dict
, optional Any metadata that are needed for a specific pipeline, but that are not needed by the
lsst.verify
framework or by general-purpose measurement analysis code (these cases are handled by themetadataAdder
subtask). If omitted, only generic metadata are added. Both keys and values must be valid inputs toMetadata
.- skipExisting :
bool
, optional If this flag is set, MetricsControllerTask will skip computing metrics for any data ID that already has an output job file on disk. While this option is useful for restarting failed runs, it does not check whether the file is valid.
Returns: - struct :
lsst.pipe.base.Struct
A
Struct
containing the following component:jobs
: a list of collections of measurements (list
oflsst.verify.Job
). Each job in the list contains the measurement(s) for the corresponding dataref, and each job has at most one measurement for each element inself.measurers
. A particular measurement is omitted if it could not be created. IfskipExisting
is set, any jobs that already exist on disk are also omitted.
Notes
Some objects may be persisted, or incorrectly persisted, in the event of an exception.
- datarefs :
-
timer
(name, logLevel=10000)¶ Context manager to log performance data for an arbitrary block of code.
Parameters: - name :
str
Name of code being timed; data will be logged using item name:
Start
andEnd
.- logLevel
A
lsst.log
level constant.
See also
timer.logInfo
Examples
Creating a timer context:
with self.timer("someCodeToTime"): pass # code to time
- name :
- each