chunk_iterable

lsst.utils.iteration.chunk_iterable(data: Iterable[Any], chunk_size: int = 1000) → Iterator[Tuple[Any, ...]]

Return smaller chunks of an iterable.

Parameters:

data : iterable of anything

The iterable to be chunked. Can be a mapping, in which case the keys are returned in chunks.

chunk_size : int, optional

The largest chunk to return. Can be smaller and depends on the number of elements in the iterator. Defaults to 1_000.

Yields:

chunk : tuple

The contents of a chunk of the iterator as a tuple. A tuple is preferred over an iterator since it is more convenient to tell it is empty and the caller knows it can be sized and indexed.