Progress

class lsst.daf.butler.Progress(name: str, level: int = 20)

Bases: object

Public interface for reporting incremental progress in the butler and related tools.

This class automatically creates progress bars (or not) depending on whether a handle (see ProgressHandler) has been installed and the given name and level are enabled. When progress reporting is not enabled, it returns dummy objects that can be used just like progress bars by calling code.

Parameters:

name : str

Name of the process whose progress is being reported. This is in general the name of a group of progress bars, not necessarily a single one, and it should have the same form as a logger name.

level : int, optional

A logging level value (defaults to logging.INFO). Progress reporting is enabled if a logger with name is enabled for this level, and a ProgressHandler has been installed.

Notes

The progress system inspects the level for a name using the Python built-in logging module, and may not respect level-setting done via the lsst.log interface. But while logging may be necessary to control progress bar visibility, the progress system can still be used together with either system for actual logging.

Methods Summary

at(level)	Return a copy of this progress interface with a different level.
bar(iterable, desc, total, skip_scalar)	Return a new progress bar context manager.
is_enabled()	Check whether this process should report progress.
iter_chunks(chunks, desc, total, skip_scalar)	Wrap iteration over chunks of elements in a progress bar.
iter_item_chunks(items, _V]], desc, total, …)	Wrap iteration over chunks of items in a progress bar.
set_handler(handler)	Set the (global) progress handler to the given instance.
wrap(iterable, desc, total, skip_scalar)	Iterate over an object while reporting progress.

Methods Documentation

at(level: int) → lsst.daf.butler.core.progress.Progress

Return a copy of this progress interface with a different level.

Parameters:

level : int

A logging level value. Progress reporting is enabled if a logger with name is enabled for this level, and a ProgressHandler has been installed.

Returns:

progress : Progress

A new Progress object with the same name as self and the given level.

bar(iterable: Optional[Iterable[_T]] = None, desc: Optional[str] = None, total: Optional[int] = None, skip_scalar: bool = True) → AbstractContextManager[lsst.daf.butler.core.progress.ProgressBar[+_T][_T]]

Return a new progress bar context manager.

Parameters:

iterable : Iterable, optional

An arbitrary Python iterable that will be iterated over when the returned ProgressBar is. If not provided, whether the progress bar is iterable is handler-defined, but it may be updated manually.

desc: `str`, optional

A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).

total : int, optional

The total number of steps in this progress bar. If not provided, len(iterable) is used. If that does not work, whether the progress bar works at all is handler-defined, and hence this mode should not be relied upon.

skip_scalar: `bool`, optional

If True and total is zero or one, do not report progress.

Returns:

bar : ContextManager [ ProgressBar ]

A context manager that returns an object satisfying the ProgressBar interface when it is entered.

is_enabled() → bool

Check whether this process should report progress.

Returns:

enabled : bool

True if there is a ProgressHandler set and a logger with the same name and level as self is enabled.

iter_chunks(chunks: Collection[_V], desc: Optional[str] = None, total: Optional[int] = None, skip_scalar: bool = True) → Generator[_V, None, None]

Wrap iteration over chunks of elements in a progress bar.

Parameters:

chunks : Collection

A sized iterable whose elements are themselves both iterable and sized (i.e. len(item) works). If total is not provided, this may not be a single-pass iteration, because an initial pass to estimate the total number of elements is required.

desc: `str`, optional

A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).

total : int, optional

The total number of steps in this progress bar; defaults to the sum of the lengths of the chunks.

skip_scalar: `bool`, optional

If True and there are zero or one chunks, do not report progress.

Yields:

chunk

The same objects that iteration over chunks would yield.

iter_item_chunks(items: Collection[Tuple[_K, _V]], desc: Optional[str] = None, total: Optional[int] = None, skip_scalar: bool = True) → Generator[Tuple[_K, _V], None, None]

Wrap iteration over chunks of items in a progress bar.

Parameters:

items : Iterable

A sized iterable whose elements are (key, value) tuples, where the values are themselves both iterable and sized (i.e. len(item) works). If total is not provided, this may not be a single-pass iteration, because an initial pass to estimate the total number of elements is required.

desc: `str`, optional

A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).

total : int, optional

The total number of values in this progress bar; defaults to the sum of the lengths of the chunks.

skip_scalar: `bool`, optional

If True and there are zero or one items, do not report progress.

Yields:

chunk

The same items that iteration over chunks would yield.

classmethod set_handler(handler: Optional[lsst.daf.butler.core.progress.ProgressHandler]) → None

Set the (global) progress handler to the given instance.

This should only be called in very high-level code that can be reasonably confident that it will dominate its current process, e.g. at the initialization of a command-line script or Jupyter notebook.

Parameters:

handler : ProgressHandler or None

Object that will handle all progress reporting. May be set to None to disable progress reporting.

wrap(iterable: Iterable[_T], desc: Optional[str] = None, total: Optional[int] = None, skip_scalar: bool = True) → Generator[_T, None, None]

Iterate over an object while reporting progress.

Parameters:

iterable : Iterable

An arbitrary Python iterable to iterate over.

desc: `str`, optional

A user-friendly description for this progress bar; usually appears next to it. If not provided, self.name is used (which is not usually a user-friendly string, but may be appropriate for debug-level progress).

total : int, optional

The total number of steps in this progress bar. If not provided, len(iterable) is used. If that does not work, whether the progress bar works at all is handler-defined, and hence this mode should not be relied upon.

skip_scalar: `bool`, optional

If True and total is zero or one, do not report progress.

Yields:

element

The same objects that iteration over iterable would yield.