MultiblockReader#

class lsst.pipe.base.quantum_graph.MultiblockReader(stream: IO[bytes], int_size: int)#

Bases: object

A helper object for reader multi-block files.

Attributes Summary

`int_size`	Number of bytes to use for all integers.
`stream`	A binary file-like object to read from.

Methods Summary

`open_in_zip`(zf, name, *, int_size)	Open a reader for a file in a zip archive.
`read_all_bytes_in_zip`(zf, name, *, int_size, ...)	Iterate over all of the byte blocks in a file in a zip archive.
`read_all_models_in_zip`(zf, name, model_type, ...)	Iterate over all of the models in a file in a zip archive.
`read_bytes`(address)	Read raw bytes from the multi-block file.
`read_model`(address, model_type, decompressor)	Read a single compressed JSON block.

Attributes Documentation

int_size: int = <dataclasses._MISSING_TYPE object>#: Number of bytes to use for all integers.

stream: IO[bytes] = <dataclasses._MISSING_TYPE object>#: A binary file-like object to read from.

Methods Documentation

classmethod open_in_zip(zf: zipfile.ZipFile, name: str, *, int_size: int) → Iterator[MultiblockReader]#

Open a reader for a file in a zip archive.

Parameters#

zfzipfile.ZipFile: Zip archive to read the file from.
namestr: Base name for the multi-block file; an extension will be added.
int_sizeint: Number of bytes to use for all integers.

Returns#

readercontextlib.AbstractContextManager [ MultiblockReader ]: Context manager that returns a reader when entered.

classmethod read_all_bytes_in_zip(zf: zipfile.ZipFile, name: str, *, int_size: int, page_size: int) → Iterator[bytes]#

Iterate over all of the byte blocks in a file in a zip archive.

Parameters#

zfzipfile.ZipFile: Zip archive to read the file from.
namestr: Base name for the multi-block file; an extension will be added.
int_sizeint: Number of bytes to use for all integers.
page_sizeint: Approximate number of bytes to read at a time.

Returns#

byte_iterIterator [ bytes ]: Iterator over blocks.

classmethod read_all_models_in_zip(zf: zipfile.ZipFile, name: str, model_type: type[_T], decompressor: Decompressor, *, int_size: int, page_size: int) → Iterator[_T]#

Iterate over all of the models in a file in a zip archive.

Parameters#

zfzipfile.ZipFile: Zip archive to read the file from.
namestr: Base name for the multi-block file; an extension will be added.
model_typetype [ pydantic.BaseModel ]: Pydantic model to validate JSON with.
decompressorDecompressor: Object with a decompress method that takes and returns bytes.
int_sizeint: Number of bytes to use for all integers.
page_sizeint: Approximate number of bytes to read at a time.

Returns#

model_iterIterator [ pydantic.BaseModel ]: Iterator over model instances.

read_bytes(address: Address) → bytes | None#

Read raw bytes from the multi-block file.

Parameters#

addressAddress: Offset and size of the data to read.

Returns#

databytes or None: Data read directly, or None if the address has zero size.

read_model(address: Address, model_type: type[_T], decompressor: Decompressor) → _T | None#

Read a single compressed JSON block.

Parameters#

addressAddress: Size and offset of the block.
model_typetype [ pydantic.BaseModel ]: Pydantic model to validate JSON with.
decompressorDecompressor: Object with a decompress method that takes and returns bytes.

Returns#

modelpydantic.BaseModel: Validated model.