MultiblockReader¶
- class lsst.pipe.base.quantum_graph.MultiblockReader(stream: IO[bytes], int_size: int)¶
Bases:
object
A helper object for reader multi-block files.
Methods Summary
open_in_zip
(zf, name, *, int_size)Open a reader for a file in a zip archive.
read_all_bytes_in_zip
(zf, name, *, int_size, ...)Iterate over all of the byte blocks in a file in a zip archive.
read_all_models_in_zip
(zf, name, model_type, ...)Iterate over all of the models in a file in a zip archive.
read_bytes
(address)Read raw bytes from the multi-block file.
read_model
(address, model_type, decompressor)Read a single compressed JSON block.
Methods Documentation
- classmethod open_in_zip(zf: zipfile.ZipFile, name: str, *, int_size: int) Iterator[MultiblockReader] ¶
Open a reader for a file in a zip archive.
- Parameters:
- zf
zipfile.ZipFile
Zip archive to read the file from.
- name
str
Base name for the multi-block file; an extension will be added.
- int_size
int
Number of bytes to use for all integers.
- zf
- Returns:
- reader
contextlib.AbstractContextManager
[MultiblockReader
] Context manager that returns a reader when entered.
- reader
- classmethod read_all_bytes_in_zip(zf: zipfile.ZipFile, name: str, *, int_size: int, page_size: int) Iterator[bytes] ¶
Iterate over all of the byte blocks in a file in a zip archive.
- Parameters:
- zf
zipfile.ZipFile
Zip archive to read the file from.
- name
str
Base name for the multi-block file; an extension will be added.
- int_size
int
Number of bytes to use for all integers.
- page_size
int
Approximate number of bytes to read at a time.
- zf
- Returns:
- classmethod read_all_models_in_zip(zf: zipfile.ZipFile, name: str, model_type: type[_T], decompressor: Decompressor, *, int_size: int, page_size: int) Iterator[_T] ¶
Iterate over all of the models in a file in a zip archive.
- Parameters:
- zf
zipfile.ZipFile
Zip archive to read the file from.
- name
str
Base name for the multi-block file; an extension will be added.
- model_type
type
[pydantic.BaseModel
] Pydantic model to validate JSON with.
- decompressor
Decompressor
Object with a
decompress
method that takes and returnsbytes
.- int_size
int
Number of bytes to use for all integers.
- page_size
int
Approximate number of bytes to read at a time.
- zf
- Returns:
- model_iter
Iterator
[pydantic.BaseModel
] Iterator over model instances.
- model_iter
- read_model(address: Address, model_type: type[_T], decompressor: Decompressor) _T | None ¶
Read a single compressed JSON block.
- Parameters:
- address
Address
Size and offset of the block.
- model_type
type
[pydantic.BaseModel
] Pydantic model to validate JSON with.
- decompressor
Decompressor
Object with a
decompress
method that takes and returnsbytes
.
- address
- Returns:
- model
pydantic.BaseModel
Validated model.
- model