ProvenanceQuantumGraphWriter

class lsst.pipe.base.quantum_graph.ProvenanceQuantumGraphWriter(output_path: str, *, exit_stack: ~contextlib.ExitStack, log_on_close: ~lsst.pipe.base.log_on_close.LogOnClose, predicted: ~lsst.pipe.base.quantum_graph._predicted.PredictedQuantumGraphComponents | ~lsst.pipe.base.quantum_graph._predicted.PredictedQuantumGraph, zstd_level: int = 10, cdict_data: bytes | None = None, loop_wrapper: ~collections.abc.Callable[[~collections.abc.Iterable[~lsst.pipe.base.quantum_graph._provenance._T]], ~collections.abc.Iterable[~lsst.pipe.base.quantum_graph._provenance._T]] = <function pass_through>, log: ~lsst.utils.logging.LsstLogAdapter | None = None)

Bases: object

A struct of low-level writer objects for the main components of a provenance quantum graph.

Parameters:

output_pathstr

Path to write the graph to.

exit_stackcontextlib.ExitStack

Object that can be used to manage multiple context managers.

log_on_closeLogOnClose

Factory for context managers that log when closed.

predictedPredictedQuantumGraphComponents

Components of the predicted graph.

zstd_levelint, optional

Compression level.

cdict_databytes or None, optional

Bytes representation of the compression dictionary used by the compressor.

loop_wrapperCallable, optional

A callable that takes an iterable and returns an equivalent one, to be used in all potentially-large loops. This can be used to add progress reporting or check for cancelation signals.

logLsstLogAdapter, optional

Logger to use for debug messages.

Attributes Summary

compressor

Object that should be used to compress all JSON blocks.

Methods Summary

write_init_outputs([assume_existence])	Write provenance for init-output datasets and init-quanta.
write_overall_inputs([loop_wrapper])	Write provenance for overall-input datasets.
write_packages()	Write package version information to the provenance graph.
write_quantum_provenance(quantum_id, ...)	Gather and write provenance for a quantum.
write_scan_data(scan_data)	Write the output of a quantum provenance scan to disk.

Attributes Documentation

compressor

Object that should be used to compress all JSON blocks.

Methods Documentation

write_init_outputs(assume_existence: bool = True) → None

Write provenance for init-output datasets and init-quanta.

Parameters:

assume_existencebool, optional

If True, just assume all init-outputs exist.

write_overall_inputs(loop_wrapper: ~collections.abc.Callable[[~collections.abc.Iterable[~lsst.pipe.base.quantum_graph._provenance._T]], ~collections.abc.Iterable[~lsst.pipe.base.quantum_graph._provenance._T]] = <function pass_through>) → None

Write provenance for overall-input datasets.

Parameters:

loop_wrapperCallable, optional

A callable that takes an iterable and returns an equivalent one, to be used in all potentially-large loops. This can be used to add progress reporting or check for cancelation signals.

write_packages() → None

Write package version information to the provenance graph.

write_quantum_provenance(quantum_id: UUID, metadata: TaskMetadata | None, logs: ButlerLogRecords | None) → None

Gather and write provenance for a quantum.

Parameters:

quantum_iduuid.UUID

Unique ID for the quantum.

metadataTaskMetadata or None

Task metadata.

logslsst.daf.butler.logging.ButlerLogRecords or None

Task logs.

write_scan_data(scan_data: ProvenanceQuantumScanData) → None

Write the output of a quantum provenance scan to disk.

Parameters:

scan_dataProvenanceQuantumScanData

Result of a quantum provenance scan.