MemoryMetricTask¶
MemoryMetricTask
creates a resident set size Measurement
based on data collected by @timeMethod
.
It reads the raw timing data from the top-level PipelineTask
’s metadata, which is identified by the task configuration.
@timeMethod
measures the peak memory usage from process start, so the results can be contaminated by previous quanta (different tasks, data IDs, or both) run on the same process.
Interpret the results with care.
Because @timeMethod
gives platform-dependent results, this task may give incorrect results (e.g., units) when run in a distributed system with heterogeneous nodes.
Processing summary¶
MemoryMetricTask
searches the metadata for @timeMethod
-generated keys corresponding to the method of interest.
If it finds matching keys, it stores the maximum memory usage as a Measurement
.
Python API summary¶
from lsst.verify.tasks.commonMetrics import MemoryMetricTask
-
class
MemoryMetricTask
(**kwargs) A Task that computes the maximum resident set size using metadata produced by the `lsst.utils.timer.timeMethod` decorator
...
- attributeconfig
Access configuration fields and retargetable subtasks.
See also
See the MemoryMetricTask
API reference for complete details.
Butler datasets¶
Input datasets¶
metadata
The metadata of the top-level pipeline task (e.g.,
CharacterizeImageTask
,DiaPipeTask
) being instrumented. This connection is usually configured indirectly through thelabelName
template as"{labelName}_metadata"
.
Output datasets¶
measurement
The value of the metric. The dataset type should not be configured directly, but should be set changing the
package
andmetric
template variables to the metric’s namespace (package, by convention) and in-package name, respectively. Subclasses that only support one metric should set these variables automatically.
Retargetable subtasks¶
No subtasks.
Configuration fields¶
connections¶
- Data type
lsst.pipe.base.config.TimeMethodMetricConfigConnections
- Field type
Configurations describing the connections of the PipelineTask to datatypes
metadataDimensions¶
Override for the dimensions of the ‘metadata’ input, when instrumenting Tasks that don’t produce one metadata object per visit.
saveLogOutput¶
Flag to enable/disable saving of log output for a task, enabled by default.
saveMetadata¶
Flag to enable/disable metadata saving for a task, enabled by default. Deprecated: This field is deprecated and will be removed after v26.
target¶
The method to profile, optionally prefixed by one or more tasks in the format of lsst.pipe.base.Task.getFullMetadata()
.
Examples¶
from lsst.verify.tasks import MemoryMetricTask
config = MemoryMetricTask.ConfigClass()
config.connections.labelName = "diaPipe"
config.connections.package = "ap_association"
cofig.connections.metric = "DiaForcedSourceMemory"
config.target = "diaPipe:diaForcedSource.run"
task = MemoryMetricTask(config=config)
# config.connections provided for benefit of Pipeline
# but since we've defined it we might as well use it
metadata = butler.get(config.connections.metadata)
processCcdTime = task.run(metadata).measurement