Progress¶

class lsst.daf.butler.Progress(name: str, level: int = 20)¶

Bases: object

Public interface for reporting incremental progress in the butler and related tools.

This class automatically creates progress bars (or not) depending on whether a handle (see ProgressHandler) has been installed and the given name and level are enabled. When progress reporting is not enabled, it returns dummy objects that can be used just like progress bars by calling code.

Parameters:

namestr: Name of the process whose progress is being reported. This is in general the name of a group of progress bars, not necessarily a single one, and it should have the same form as a logger name.
levelint, optional: A logging level value (defaults to logging.INFO). Progress reporting is enabled if a logger with name is enabled for this level, and a ProgressHandler has been installed.

Notes

The progress system inspects the level for a name using the Python built-in logging module, and may not respect level-setting done via the lsst.log interface. But while logging may be necessary to control progress bar visibility, the progress system can still be used together with either system for actual logging.

Methods Summary

`at`(level)	Return a copy of this progress interface with a different level.
`bar`([iterable, desc, total, skip_scalar])	Return a new progress bar context manager.
`is_enabled`()	Check whether this process should report progress.
`iter_chunks`(chunks[, desc, total, skip_scalar])	Wrap iteration over chunks of elements in a progress bar.
`iter_item_chunks`(items[, desc, total, ...])	Wrap iteration over chunks of items in a progress bar.
`set_handler`(handler)	Set the (global) progress handler to the given instance.
`wrap`(iterable[, desc, total, skip_scalar])	Iterate over an object while reporting progress.

Methods Documentation

at(level: int) → Progress¶

Return a copy of this progress interface with a different level.

Parameters:

levelint: A logging level value. Progress reporting is enabled if a logger with name is enabled for this level, and a ProgressHandler has been installed.

Returns:

progressProgress: A new Progress object with the same name as self and the given level.

bar(iterable: Iterable[_T] | None = None, desc: str | None = None, total: int | None = None, skip_scalar: bool = True) → ContextManager[ProgressBar[_T]]¶

Return a new progress bar context manager.

Parameters:

iterableIterable, optional: An arbitrary Python iterable that will be iterated over when the returned ProgressBar is. If not provided, whether the progress bar is iterable is handler-defined, but it may be updated manually.
desc: `str`, optional: A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).
totalint, optional: The total number of steps in this progress bar. If not provided, len(iterable) is used. If that does not work, whether the progress bar works at all is handler-defined, and hence this mode should not be relied upon.
skip_scalar: `bool`, optional: If True and total is zero or one, do not report progress.

Returns:

barContextManager [ ProgressBar ]: A context manager that returns an object satisfying the ProgressBar interface when it is entered.

is_enabled() → bool¶

Check whether this process should report progress.

Returns:

enabledbool: True if there is a ProgressHandler set and a logger with the same name and level as self is enabled.

iter_chunks(chunks: Iterable[_V], desc: str | None = None, total: int | None = None, skip_scalar: bool = True) → Generator[_V, None, None]¶

Wrap iteration over chunks of elements in a progress bar.

Parameters:

chunksIterable: An iterable whose elements are themselves iterable.
desc: `str`, optional: A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).
totalint, optional: The total number of steps in this progress bar; defaults to the sum of the lengths of the chunks if this can be computed. If this is provided or True, each element in chunks must be sized but chunks itself need not be (and may be a single-pass iterable).
skip_scalar: `bool`, optional: If True and there are zero or one chunks, do not report progress.

Yields:

chunk: The same objects that iteration over chunks would yield.

Notes

This attempts to display as much progress as possible given the limitations of the iterables, assuming that sized iterables are also multi-pass (as is true of all built-in collections and lazy iterators). In detail, if total is None:

if chunks and its elements are both sized, total is computed from them and full progress is reported, and chunks must be a multi-pass iterable.
if chunks is sized but its elements are not, a progress bar over the number of chunks is shown, and chunks must be a multi-pass iterable.
if chunks is not sized, the progress bar just shows when updates occur.

If total is True or an integer, chunks need not be sized, but its elements must be, chunks must be a multi-pass iterable, and full progress is shown.

If total is False, chunks and its elements need not be sized, and the progress bar just shows when updates occur.

iter_item_chunks(items: Iterable[tuple[_K, _V]], desc: str | None = None, total: int | None = None, skip_scalar: bool = True) → Generator[tuple[_K, _V], None, None]¶

Wrap iteration over chunks of items in a progress bar.

Parameters:

itemsIterable: An iterable whose elements are (key, value) tuples, where the values are themselves iterable.
desc: `str`, optional: A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).
totalint, optional: The total number of steps in this progress bar; defaults to the sum of the lengths of the chunks if this can be computed. If this is provided or True, each element in chunks must be sized but chunks itself need not be (and may be a single-pass iterable).
skip_scalar: `bool`, optional: If True and there are zero or one items, do not report progress.

Yields:

chunk: The same 2-tuples that iteration over items would yield.

Notes

This attempts to display as much progress as possible given the limitations of the iterables, assuming that sized iterables are also multi-pass (as is true of all built-in collections and lazy iterators). In detail, if total is None:

if chunks and its values elements are both sized, total is computed from them and full progress is reported, and chunks must be a multi-pass iterable.
if chunks is sized but its value elements are not, a progress bar over the number of chunks is shown, and chunks must be a multi-pass iterable.
if chunks is not sized, the progress bar just shows when updates occur.

If total is True or an integer, chunks need not be sized, but its value elements must be, chunks must be a multi-pass iterable, and full progress is shown.

If total is False, chunks and its values elements need not be sized, and the progress bar just shows when updates occur.

classmethod set_handler(handler: ProgressHandler | None) → None¶

Set the (global) progress handler to the given instance.

This should only be called in very high-level code that can be reasonably confident that it will dominate its current process, e.g. at the initialization of a command-line script or Jupyter notebook.

Parameters:

handlerProgressHandler or None: Object that will handle all progress reporting. May be set to None to disable progress reporting.

wrap(iterable: Iterable[_T], desc: str | None = None, total: int | None = None, skip_scalar: bool = True) → Generator[_T, None, None]¶

Iterate over an object while reporting progress.

Parameters:

iterableIterable: An arbitrary Python iterable to iterate over.
desc: `str`, optional: A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).
totalint, optional: The total number of steps in this progress bar. If not provided, len(iterable) is used. If that does not work, whether the progress bar works at all is handler-defined, and hence this mode should not be relied upon.
skip_scalar: `bool`, optional: If True and total is zero or one, do not report progress.

Yields:

element: The same objects that iteration over iterable would yield.

Navigation

Progress¶