QuantumGraphSkeleton¶
- class lsst.pipe.base.quantum_graph_skeleton.QuantumGraphSkeleton(task_labels: Iterable[str])¶
Bases:
object
An under-construction quantum graph.
QuantumGraphSkeleton is intended for use inside
QuantumGraphBuilder
and its subclasses.- Parameters:
Notes
QuantumGraphSkeleton models a bipartite version of the quantum graph, in which both quanta and datasets are represented as nodes and each type of node only has edges to the other type.
Square-bracket (
getitem
) indexing returns a mutable mapping of a node’s flexible attributes.The details of the
QuantumGraphSkeleton
API (e.g. which operations operate on multiple nodes vs. a single node) are set by what’s actually needed by current quantum graph generation algorithms. New variants can be added as needed, but adding all operations that might be useful for some future algorithm seems premature.Attributes Summary
The set of dataset nodes that are not associated with any task.
The total number of edges.
The total number of nodes of all types.
Methods Summary
add_dataset_node
(parent_dataset_type_name, ...)Add a new node representing a dataset.
add_input_edge
(task_key, dataset_key[, ...])Add an edge connecting a dataset to a quantum that consumes it.
add_input_edges
(task_key, dataset_keys)Add edges connecting datasets to a quantum that consumes them.
add_output_edge
(task_key, dataset_key)Add an edge connecting a dataset to the quantum that produces it.
Add a new node representing a prerequisite input dataset.
add_quantum_node
(task_label, data_id, **attrs)Add a new node representing a quantum.
Find overall input datasets.
get_quanta
(task_label)Return the quanta for the given task label.
get_task_init_node
(task_label)Return the graph node that represents a task's initialization.
has_task
(task_label)Test whether the given task is in this skeleton.
Iterate over all quanta from any task, in topological (but otherwise unspecified) order.
iter_inputs_of
(quantum_key)Iterate over the datasets consumed by the given quantum.
iter_outputs_of
(quantum_key)Iterate over the datasets produced by the given quantum.
remove_dataset_nodes
(keys)Remove nodes representing datasets.
remove_input_edges
(task_key, dataset_keys)Remove edges connecting datasets to a quantum that consumes them.
Remove any dataset nodes that do not have any edges.
remove_quantum_node
(key, remove_outputs)Remove a node representing a quantum.
remove_task
(task_label)Fully remove a task from the skeleton.
update
(other)Copy all nodes from
other
toself
.Attributes Documentation
- global_init_outputs¶
The set of dataset nodes that are not associated with any task.
- n_edges¶
The total number of edges.
- n_nodes¶
The total number of nodes of all types.
Methods Documentation
- add_dataset_node(parent_dataset_type_name: str, data_id: DataCoordinate, is_global_init_output: bool = False, **attrs: Any) DatasetKey ¶
Add a new node representing a dataset.
- Parameters:
- parent_dataset_type_name
str
Name of the parent dataset type.
- data_id
DataCoordinate
The dataset data ID.
- is_global_init_output
bool
, optional Whether this dataset is a global init output.
- **attrs
Any
Additional attributes for the node.
- parent_dataset_type_name
- add_input_edge(task_key: QuantumKey | TaskInitKey, dataset_key: DatasetKey | PrerequisiteDatasetKey, ignore_unrecognized_quanta: bool = False) bool ¶
Add an edge connecting a dataset to a quantum that consumes it.
- Parameters:
- task_key
QuantumKey
orTaskInitKey
Identifier for the quantum node.
- dataset_key
DatasetKey
orPrerequisiteKey
Identifier for the dataset node.
- ignore_unrecognized_quanta
bool
, optional If
False
, do nothing if the quantum node is not already present. IfTrue
, the quantum node is assumed to be present.
- task_key
- Returns:
Notes
Dataset nodes that are not already present will be created.
- add_input_edges(task_key: QuantumKey | TaskInitKey, dataset_keys: Iterable[DatasetKey | PrerequisiteDatasetKey]) None ¶
Add edges connecting datasets to a quantum that consumes them.
- Parameters:
- task_key
QuantumKey
orTaskInitKey
Quantum to connect.
- dataset_keys
Iterable
ofDatasetKey
orPrequisiteDatasetKey
Datasets to join to the quantum.
- task_key
Notes
This must only be called if the task node has already been added. Use
add_input_edge
if this cannot be assumed.Dataset nodes that are not already present will be created.
- add_output_edge(task_key: QuantumKey | TaskInitKey, dataset_key: DatasetKey) None ¶
Add an edge connecting a dataset to the quantum that produces it.
- Parameters:
- task_key
QuantumKey
orTaskInitKey
Identifier for the quantum node. Must identify a node already present in the graph.
- dataset_key
DatasetKey
Identifier for the dataset node. Must identify a node already present in the graph.
- task_key
- add_prerequisite_node(parent_dataset_type_name: str, ref: DatasetRef, **attrs: Any) PrerequisiteDatasetKey ¶
Add a new node representing a prerequisite input dataset.
- Parameters:
- parent_dataset_type_name
str
Name of the parent dataset type.
- ref
DatasetRef
The dataset ref of the pre-requisite.
- **attrs
Any
Additional attributes for the node.
- parent_dataset_type_name
- add_quantum_node(task_label: str, data_id: DataCoordinate, **attrs: Any) QuantumKey ¶
Add a new node representing a quantum.
- Parameters:
- task_label
str
Name of task.
- data_id
DataCoordinate
The data ID of the quantum.
- **attrs
Any
Additional attributes.
- task_label
- extract_overall_inputs() dict[lsst.pipe.base.quantum_graph_skeleton.DatasetKey | lsst.pipe.base.quantum_graph_skeleton.PrerequisiteDatasetKey, lsst.daf.butler._dataset_ref.DatasetRef] ¶
Find overall input datasets.
- Returns:
- datasets
dict
[DatasetKey
orPrerequisiteDatasetKey
,DatasetRef
] Overall-input datasets, including prerequisites and init-inputs.
- datasets
- get_quanta(task_label: str) Set[QuantumKey] ¶
Return the quanta for the given task label.
- Parameters:
- task_label
str
Label for the task.
- task_label
- Returns:
- quanta
Set
[QuantumKey
] A set-like object with the identifiers of all quanta for the given task. The skeleton object’s set of quanta must not be modified while iterating over this container; make a copy if mutation during iteration is necessary.
- quanta
- get_task_init_node(task_label: str) TaskInitKey ¶
Return the graph node that represents a task’s initialization.
- Parameters:
- task_label
str
The task label to use.
- task_label
- Returns:
- node
TaskInitKey
The graph node representing this task’s initialization.
- node
- has_task(task_label: str) bool ¶
Test whether the given task is in this skeleton.
Tasks are only added to the skeleton at initialization, but may be removed by
remove_task
if they end up having no quanta.
- iter_all_quanta() Iterator[QuantumKey] ¶
Iterate over all quanta from any task, in topological (but otherwise unspecified) order.
- iter_inputs_of(quantum_key: QuantumKey | TaskInitKey) Iterator[DatasetKey | PrerequisiteDatasetKey] ¶
Iterate over the datasets consumed by the given quantum.
- Parameters:
- quantum_key
QuantumKey
orTaskInitKey
Quantum to iterate over.
- quantum_key
- Returns:
- datasets
Iterator
ofDatasetKey
orPrequisiteDatasetKey
Datasets consumed by the given quanta.
- datasets
- iter_outputs_of(quantum_key: QuantumKey | TaskInitKey) Iterator[DatasetKey] ¶
Iterate over the datasets produced by the given quantum.
- Parameters:
- quantum_key
QuantumKey
orTaskInitKey
Quantum to iterate over.
- quantum_key
- Returns:
- datasets
Iterator
ofDatasetKey
Datasets produced by the given quanta.
- datasets
- remove_dataset_nodes(keys: Iterable[DatasetKey | PrerequisiteDatasetKey]) None ¶
Remove nodes representing datasets.
- Parameters:
- keys
Iterable
ofDatasetKey
orPrerequisiteDatasetKey
Nodes to remove.
- keys
- remove_input_edges(task_key: QuantumKey | TaskInitKey, dataset_keys: Iterable[DatasetKey | PrerequisiteDatasetKey]) None ¶
Remove edges connecting datasets to a quantum that consumes them.
- Parameters:
- task_key
QuantumKey
orTaskInitKey
Quantum to disconnect.
- dataset_keys
Iterable
ofDatasetKey
orPrequisiteDatasetKey
Datasets to remove from the quantum.
- task_key
- remove_quantum_node(key: QuantumKey, remove_outputs: bool) None ¶
Remove a node representing a quantum.
- Parameters:
- key
QuantumKey
Identifier for the node.
- remove_outputs
bool
If
True
, also remove all dataset nodes produced by this quantum. IfFalse
, any such dataset nodes will become overall inputs.
- key
- remove_task(task_label: str) None ¶
Fully remove a task from the skeleton.
All init-output datasets and quanta for the task must already have been removed.
- Parameters:
- task_label
str
Name of task to remove.
- task_label
- update(other: QuantumGraphSkeleton) None ¶
Copy all nodes from
other
toself
.- Parameters:
- other
QuantumGraphSkeleton
Source of nodes. The tasks in
other
must be a subset of the tasks inself
(this method is expected to be used to populate a skeleton for a full from independent-subgraph skeletons).
- other