ProvenanceQuantumGraphReader

class lsst.pipe.base.quantum_graph.ProvenanceQuantumGraphReader(header: HeaderModel, pipeline_graph: PipelineGraph, zf: zipfile.ZipFile, decompressor: Decompressor, address_reader: AddressReader, page_size: int)

Bases: BaseQuantumGraphReader

A helper class for reading provenance quantum graphs.

Notes

The open context manager should be used to construct new instances. Instances cannot be used after the context manager exits, except to access the graph attribute`.

The various read_* methods in this class update the graph attribute in place.

Methods Summary

fetch_logs(nodes)	Fetch log datasets.
fetch_metadata(nodes)	Fetch metadata datasets.
fetch_packages()	Fetch package version information.
open(uri, *[, page_size, import_mode])	Construct a reader from a URI.
read_datasets([datasets])	Read information about the given datasets.
read_full_graph()	Read all bipartite edges and all quantum and dataset node attributes, fully populating the graph attribute.
read_init_quanta()	Read the thin graph, with all edge information and categorization of quanta by task label.
read_quanta([quanta])	Read information about the given quanta.

Methods Documentation

fetch_logs(nodes: Iterable[UUID | int]) → dict[uuid.UUID | int, list[lsst.daf.butler.logging.ButlerLogRecords | None]]

Fetch log datasets.

Parameters:

nodesIterable [ uuid.UUID ]

UUIDs or internal integer IDS of the log datasets themselves or of the quanta they correspond to.

Returns:

logsdict [ uuid.UUID or int, list [ lsst.daf.butler.ButlerLogRecords or None] ]

Logs for the given IDs. Each value is a list of lsst.daf.butler.ButlerLogRecords instances representing different execution attempts, ordered chronologically from first to last. Attempts where logs were missing will have None in this list.

fetch_metadata(nodes: Iterable[UUID | int]) → dict[uuid.UUID | int, list[lsst.pipe.base._task_metadata.TaskMetadata | None]]

Fetch metadata datasets.

Parameters:

nodesIterable [ uuid.UUID ]

UUIDs or internal integer IDs of the metadata datasets themselves or of the quanta they correspond to.

Returns:

metadatadict [ uuid.UUID or int, list [TaskMetadata] ]

Metadata for the given IDs. Each value is a list of TaskMetadata instances representing different execution attempts, ordered chronologically from first to last. Attempts where metadata was missing (not written even in the fallback extra provenance in the logs) will have None in this list.

fetch_packages() → Packages

Fetch package version information.

classmethod open(uri: str | ParseResult | ResourcePath | Path, *, page_size: int | None = None, import_mode: TaskImportMode = TaskImportMode.DO_NOT_IMPORT) → Iterator[ProvenanceQuantumGraphReader]

Construct a reader from a URI.

Parameters:

uriconvertible to lsst.resources.ResourcePath

URI to open. Should have a .qg extension.

page_sizeint, optional

Approximate number of bytes to read at once from address files and multi-block files. Note that this does not set a page size for all reads, but it does affect the smallest, most numerous reads. Can also be set via the LSST_QG_PAGE_SIZE environment variable.

import_modepipeline_graph.TaskImportMode, optional

How to handle importing the task classes referenced in the pipeline graph.

Returns:

readercontextlib.AbstractContextManager [ ProvenanceQuantumGraphReader ]

A context manager that returns the reader when entered.

read_datasets(datasets: Iterable[UUID | int] | None = None) → None

Read information about the given datasets.

Parameters:

datasetsIterable [uuid.UUID or int], optional

Iterable of dataset IDs or indices to load. If not provided, all datasets will be loaded. The UUIDs and indices of quanta will be ignored.

read_full_graph() → None

Read all bipartite edges and all quantum and dataset node attributes, fully populating the graph attribute.

Notes

This does not read logs, metadata, or packages ; those must always be fetched explicitly.

read_init_quanta() → None

Read the thin graph, with all edge information and categorization of quanta by task label.

read_quanta(quanta: Iterable[UUID | int] | None = None) → None

Read information about the given quanta.

Parameters:

quantaIterable [uuid.UUID or int], optional

Iterable of quantum IDs or indices to load. If not provided, all quanta will be loaded. The UUIDs and indices of datasets and special init quanta will be ignored.