SingleVisitListTaskRunner

class lsst.cp.pipe.utils.SingleVisitListTaskRunner(TaskClass, parsedCmd, doReturnResults=False)

Bases: lsst.pipe.base.TaskRunner

Subclass of TaskRunner for tasks requiring a list of visits per dataRef.

This transforms the processed arguments generated by the ArgumentParser into the arguments expected by tasks which require a list of visits to be supplied for each dataRef, as is common in lsst.cp.pipe code.

Such tasks’ run() methods tend to take two arguments, one of which is the dataRef (as usual), and the other is the list of visits. This list is supplied on the command line as documented, and this class parses that, and passes the parsed version to the run() method.

See lsst.pipe.base.TaskRunner for more information.

Attributes Summary

TIMEOUT

Methods Summary

__call__(args)	Run the Task on a single target.
getTargetList(parsedCmd, **kwargs)	Parse the visit list and pass through explicitly.
makeTask([parsedCmd, args])	Create a Task instance.
precall(parsedCmd)	Hook for code that should run exactly once, before multiprocessing.
prepareForMultiProcessing()	Prepare this instance for multiprocessing
run(parsedCmd)	Run the task on all targets.
runTask(task, dataRef, kwargs)	Make the actual call to runDataRef for this task.

Attributes Documentation

TIMEOUT = 2592000

Methods Documentation

__call__(args)

Run the Task on a single target.

Parameters:

args

Arguments for Task.runDataRef()

Returns:

struct : lsst.pipe.base.Struct

Contains these fields if doReturnResults is True:

• dataRef: the provided data reference.
• metadata: task metadata after execution of run.
• result: result returned by task run, or None if the task fails.
• exitStatus: 0 if the task completed successfully, 1 otherwise.

If doReturnResults is False the struct contains:

• exitStatus: 0 if the task completed successfully, 1 otherwise.

Notes

This default implementation assumes that the args is a tuple containing a data reference and a dict of keyword arguments.

Warning

If you override this method and wish to return something when doReturnResults is False, then it must be picklable to support multiprocessing and it should be small enough that pickling and unpickling do not add excessive overhead.

static getTargetList(parsedCmd, **kwargs)

Parse the visit list and pass through explicitly.

makeTask(parsedCmd=None, args=None)

Create a Task instance.

Parameters:

parsedCmd

Parsed command-line options (used for extra task args by some task runners).

args

Args tuple passed to TaskRunner.__call__ (used for extra task arguments by some task runners).

Notes

makeTask can be called with either the parsedCmd argument or args argument set to None, but it must construct identical Task instances in either case.

Subclasses may ignore this method entirely if they reimplement both TaskRunner.precall and TaskRunner.__call__.

precall(parsedCmd)

Hook for code that should run exactly once, before multiprocessing.

Notes

Must return True if TaskRunner.__call__ should subsequently be called.

Warning

Implementations must take care to ensure that no unpicklable attributes are added to the TaskRunner itself, for compatibility with multiprocessing.

The default implementation writes package versions, schemas and configs, or compares them to existing files on disk if present.

prepareForMultiProcessing()

Prepare this instance for multiprocessing

Optional non-picklable elements are removed.

This is only called if the task is run under multiprocessing.

run(parsedCmd)

Run the task on all targets.

Parameters:

parsedCmd : argparse.Namespace

Parsed command argparse.Namespace.

Returns:

resultList : list

A list of results returned by TaskRunner.__call__, or an empty list if TaskRunner.__call__ is not called (e.g. if TaskRunner.precall returns False). See TaskRunner.__call__ for details.

Notes

The task is run under multiprocessing if TaskRunner.numProcesses is more than 1; otherwise processing is serial.

runTask(task, dataRef, kwargs)

Make the actual call to runDataRef for this task.

Parameters:

task : lsst.pipe.base.CmdLineTask class

The class of the task to run.

dataRef

Butler data reference that contains the data the task will process.

kwargs

Any additional keyword arguments. See TaskRunner.getTargetList above.

Notes

The default implementation of TaskRunner.runTask works for any command-line task which has a runDataRef method that takes a data reference and an optional set of additional keyword arguments. This method returns the results generated by the task’s runDataRef method.