Pipeline¶
-
class
lsst.pipe.base.
Pipeline
(description: str)¶ Bases:
object
A
Pipeline
is a representation of a series of tasks to run, and the configuration for those tasks.Parameters: - description :
str
A description of that this pipeline does.
Methods Summary
addConfigFile
(label, filename)Add overrides from a specified file. addConfigOverride
(label, key, value)Apply single config override. addConfigPython
(label, pythonString)Add Overrides by running a snippet of python code against a config. addInstrument
(instrument, str])Add an instrument to the pipeline, or replace an instrument that is already defined. addTask
(task, str], label)Add a new task to the pipeline, or replace a task that is already associated with the supplied label. fromFile
(filename)Load a pipeline defined in a pipeline yaml file. fromIR
(deserialized_pipeline)Create a pipeline from an already created PipelineIR
object.fromPipeline
(pipeline)Create a new pipeline by copying an already existing Pipeline
.fromString
(pipeline_string)Create a pipeline from string formatted as a pipeline document. getInstrument
()Get the instrument from the pipeline. removeTask
(label)Remove a task from the pipeline. subsetFromLabels
(labelSpecifier)Subset a pipeline to contain only labels specified in labelSpecifier toExpandedPipeline
()Returns a generator of TaskDefs which can be used to create quantum graphs. toFile
(filename)Methods Documentation
-
addConfigFile
(label: str, filename: str)¶ Add overrides from a specified file.
Parameters:
-
addConfigOverride
(label: str, key: str, value: object)¶ Apply single config override.
Parameters: - label :
str
Label of the task.
- key: `str`
Fully-qualified field name.
- value : object
Value to be given to a field.
- label :
-
addConfigPython
(label: str, pythonString: str)¶ Add Overrides by running a snippet of python code against a config.
Parameters: - label :
str
The label used to identity the task associated with config to modify.
- pythonString: `str`
A string which is valid python code to be executed. This is done with config as the only local accessible value.
- label :
-
addInstrument
(instrument: Union[Instrument, str])¶ Add an instrument to the pipeline, or replace an instrument that is already defined.
Parameters: - instrument :
Instrument
orstr
Either a derived class object of a
lsst.daf.butler.instrument
or a string corresponding to a fully qualifiedlsst.daf.butler.instrument
name.
- instrument :
-
addTask
(task: Union[lsst.pipe.base.pipelineTask.PipelineTask, str], label: str)¶ Add a new task to the pipeline, or replace a task that is already associated with the supplied label.
Parameters: - task: `PipelineTask` or `str`
Either a derived class object of a
PipelineTask
or a string corresponding to a fully qualifiedPipelineTask
name.- label: `str`
A label that is used to identify the
PipelineTask
being added
-
classmethod
fromFile
(filename: str) → lsst.pipe.base.pipeline.Pipeline¶ Load a pipeline defined in a pipeline yaml file.
Parameters: - filename: `str`
A path that points to a pipeline defined in yaml format. This filename may also supply additional labels to be used in subsetting the loaded Pipeline. These labels are separated from the path by a colon, and may be specified as a comma separated list, or a range denoted as beginning..end. Beginning or end may be empty, in which case the range will be a half open interval. Unlike python iteration bounds, end bounds are INCLUDED. Note that range based selection is not well defined for pipelines that are not linear in nature, and correct behavior is not guaranteed, or may vary from run to run.
Returns: - pipeline: `Pipeline`
The pipeline loaded from specified location with appropriate (if any) subsetting
Notes
This method attempts to prune any contracts that contain labels which are not in the declared subset of labels. This pruning is done using a string based matching due to the nature of contracts and may prune more than it should.
-
classmethod
fromIR
(deserialized_pipeline: lsst.pipe.base.pipelineIR.PipelineIR) → lsst.pipe.base.pipeline.Pipeline¶ Create a pipeline from an already created
PipelineIR
object.Parameters: - deserialized_pipeline: `PipelineIR`
An already created pipeline intermediate representation object
Returns: - pipeline: `Pipeline`
-
classmethod
fromPipeline
(pipeline: lsst.pipe.base.pipelineIR.PipelineIR) → lsst.pipe.base.pipeline.Pipeline¶ Create a new pipeline by copying an already existing
Pipeline
.Parameters: - pipeline: `Pipeline`
An already created pipeline intermediate representation object
Returns: - pipeline: `Pipeline`
-
classmethod
fromString
(pipeline_string: str) → lsst.pipe.base.pipeline.Pipeline¶ Create a pipeline from string formatted as a pipeline document.
Parameters: - pipeline_string :
str
A string that is formatted according like a pipeline document
Returns: - pipeline: `Pipeline`
- pipeline_string :
-
getInstrument
()¶ Get the instrument from the pipeline.
Returns: - instrument :
Instrument
,str
, or None A derived class object of a
lsst.daf.butler.instrument
, a string corresponding to a fully qualifiedlsst.daf.butler.instrument
name, or None if the pipeline does not have an instrument.
- instrument :
-
removeTask
(label: str)¶ Remove a task from the pipeline.
Parameters: - label :
str
The label used to identify the task that is to be removed
Raises: - KeyError
If no task with that label exists in the pipeline
- label :
-
subsetFromLabels
(labelSpecifier: lsst.pipe.base.pipeline.LabelSpecifier) → lsst.pipe.base.pipeline.Pipeline¶ Subset a pipeline to contain only labels specified in labelSpecifier
Parameters: - labelSpecifier :
labelSpecifier
Object containing labels that describes how to subset a pipeline.
Returns: - pipeline :
Pipeline
A new pipeline object that is a subset of the old pipeline
Raises: - ValueError
Raised if there is an issue with specified labels
Notes
This method attempts to prune any contracts that contain labels which are not in the declared subset of labels. This pruning is done using a string based matching due to the nature of contracts and may prune more than it should.
- labelSpecifier :
-
toExpandedPipeline
() → Generator[TaskDef]¶ Returns a generator of TaskDefs which can be used to create quantum graphs.
Returns: - generator : generator of
TaskDef
The generator returned will be the sorted iterator of tasks which are to be used in constructing a quantum graph.
Raises: - NotImplementedError
If a dataId is supplied in a config block. This is in place for future use
- generator : generator of
-
toFile
(filename: str)¶
- description :