Namespace lsst::utils¶

namespace utils

Functions

std::string demangleType(std::string const _typeName)

std::size_t hashCombine(std::size_t seed)

Combine hashes

A specialization of hashCombine for a trivial argument list.

template<typename T, typename ...Rest> std::size_t hashCombine(std::size_t seed, const T &value, Rest... rest)

Combine hashes

This is provided as a convenience for those who need to hash a composite. C++11 includes std::hash, but neglects to include a facility for combining hashes.

Shall not throw exceptions.

Return

A combined hash for all the arguments after seed.

Template Parameters

TRest: the types to hash. All types must have a valid (in particular, non-throwing) specialization of std::hash.

Parameters

seed: An arbitrary starting value.
valuerest: The objects to hash.

To use it:

// Arbitrary seed; can change to get different hashes of same argument list
std::size_t seed = 0;
result = hashCombine(seed, obj1, obj2, obj3);

template<typename InputIterator> std::size_t hashIterable(std::size_t seed, InputIterator begin, InputIterator end)

Combine hashes in an iterable.

This is provided as a convenience for those who need to hash a container.

Shall not throw exceptions.

Return

A combined hash for all the elements in [begin, end).

Template Parameters

InputIterator: an iterator to the objects to be hashed. The pointed-to type must have a valid (in particular, non-throwing) specialization of std::hash.

Parameters

seed: An arbitrary starting value.
beginend: The range to hash.

To use it:

// Arbitrary seed; can change to get different hashes of same argument list
std::size_t seed = 0;
result = hashIterable(seed, container.begin(), container.end());

double nanojanskyToABMagnitude(double flux): Convert a flux in nanojansky to AB magnitude.

double ABMagnitudeToNanojansky(double magnitude): Convert an AB magnitude to a flux in nanojansky.

std::string getPackageDir(std::string const &packageName)

return the root directory of a setup package

Parameters

[in] packageName: name of package (e.g. “utils”)

Exceptions

lsst::pex::exceptions::NotFoundError: if desired version can’t be found

template<typename T> constexpr void assertValidHash()

Compile-time test of whether a specialization of std::hash conforms to the general spec.

The function itself is a no-op.

Template Parameters

T: The properties of std::hash<T> will be tested.

template<typename T> void assertHashesEqual(T obj1, T obj2)

Test that equal objects have equal hashes.

If objects of type T can be equal despite having different internal representations, you should include pairs of such objects.

Template Parameters

T: A hashable type.

Parameters

obj1obj2: Two equal objects.

Variables

const double referenceFlux = 1e23 * pow(10, ) * 1e9: The Oke & Gunn (1983) AB magnitude reference flux, in nJy (often approximated as 3631.0).

class Backtrace

#include <Backtrace.h>

Singleton, enables automatic backtraces on the following signals:

SIGABRT
SIGSEGV
SIGILL
SIGFPE

Note: Uses low level malloc and fprintf since higher level constructs may not be available when a signal is received.

template<typename Key, typename Value, typename KeyHash, typename KeyPred> class Cache

#include <Cache.h>

Cache of most recently used values

This object stores the most recent maxElements values, where maxElements is set in the constructor. Objects (of type Value) are stored by a key (of type Key; hence the need to provide a KeyHash and KeyPred), and the class presents a dict-like interface. Objects may be added to (add) and retrieved from (operator[]) the cache. For ease of use, an interface (operator()) is also provided that will check the cache for an existing key, and if the key is not present, generate it with a function provided by the user.

Note: Value and Key must be copyable.
Note: This header (Cache.h) should generally only be included in source files, not other header files, because you probably don’t want all of the boost::multi_index includes in your header. We suggest you se the CacheFwd.h file in your header instead, and hold the Cache as a std::unique_ptr.
Note: Python bindings (for pybind11) are available in lsst/utils/python/Cache.h.

namespace python

Functions

template<typename Key, typename Value, typename KeyHash = boost::hash<Key>, typename KeyPred = std::equal_to<Key>> void declareCache(py::module &mod, std::string const &name)

template<typename T, typename PyClass> void addSharedPtrEquality(PyClass &cls)

Add __eq__ and __ne__ methods based on two std::shared_ptr<T> pointing to the same address

Example:

Template Parameters

T: The type to which the std::shared_ptr points.
PyClass: The pybind11 class_ type; this can be automatically deduced.

lsst::afw::table records are considered equal if two std::shared_ptr<record> point to the same record. This is wrapped as follows for lsst::afw::table::BaseRecord, where cls is an instance of pybind11::class_<BaseRecord, std::shared_ptr<BaseRecord>>):

utils::addSharedPtrEquality<BaseRecord>(cls);

Note that all record subclasses inherit this behavior without needing to call this function.

template<class PyClass> void addOutputOp(PyClass &cls, std::string const &method)

Add __str__ or __repr__ method implemented by operator<<.

For flexibility, this method can be used to define one or both of __str__ and __repr__. It can also be used to define any Python method that takes no arguments and returns a string, regardless of name.

Template Parameters

PyClass: The pybind11 class_ type. The wrapped class must support << as a stream output operator.

Parameters

cls: The PyClass object to which to add a wrapper.
method: The name of the method to implement. Should be "__str__" or "__repr__".

template<class PyClass> void addHash(PyClass &cls)

Add __hash__ method implemented by std::hash.

Template Parameters

PyClass: The pybind11 class_ type. The wrapped class must have an enabled specialization of std::hash.

Parameters

cls: The PyClass object to which to add a wrapper.

std::size_t cppIndex(std::ptrdiff_t size, std::ptrdiff_t i)

Compute a C++ index from a Python index (negative values count from the end) and range-check.

Return

index in the range [0, size - 1]

Note

the size argument has type std::ptrdiff_t instead of std::size_t in order to to match the allowed range for the i argument.

Parameters

[in] size: Number of elements in the collection.
[in] i: Index into the collection; negative values count from the end

Exceptions

Python: IndexError if i not in range [-size, size - 1]

std::pair<std::size_t, std::size_t> cppIndex(std::ptrdiff_t size_i, std::ptrdiff_t size_j, std::ptrdiff_t i, std::ptrdiff_t j)

Compute a pair of C++ indices from a pair of Python indices (negative values count from the end) and range-check.

Return

a std::pair of indices, each in the range [0, size - 1]

Parameters

[in] size_i: Number of elements along the first axis.
[in] size_j: Number of elements along the second axis.
[in] i: Index along first axis; negative values count from the end
[in] j: Index along second axis; negative values count from the end

Exceptions

Python: IndexError if either input index not in range [-size, size - 1]

template<typename T> class PySharedPtr

#include <PySharedPtr.h>

A shared pointer that tracks both a C++ object and its associated PyObject.

Each group of PySharedPtr for a given object collectively counts as one reference to that object for the purpose of Python garbage collection.

A PySharedPtr is implicitly convertible to and from a std::shared_ptr to minimize API impact. Any shared_ptr created this way will (I think) keep the Python reference alive, as described above.

class TemplateInvoker

#include <TemplateInvoker.h>

A helper class for wrapping C++ template functions as Python functions with dtype arguments.

TemplateInvoker takes a templated callable object, a pybind11::dtype object, and a sequence of supported C++ types via its nested Tag struct. The callable is invoked with a scalar argument of the type matching the dtype object. If none of the supported C++ types match, a different error callback is invoked instead.

As an example, we’ll wrap this function:

template <typename T>
T doSomething(std::string const & argument);

TemplateInvoker provides a default error callback, which we’ll use here (otherwise you’d need to pass one when constructing the TemplateInvoker).

For the main callback, we’ll define this helper struct:

struct DoSomethingHelper {

    template <typename T>
    T operator()(T) const {
        return doSomething<T>(argument);
    }

    std::string argument;
};

The pybind11 wrapper for doSomething is then another lambda that uses TemplateInvoker::apply to call the helper:

mod.def(
    "doSomething",
    [](std::string const & argument, py::dtype const & dtype) {
        return TemplateInvoker().apply(
            DoSomethingHelper{argument},
            dtype,
            TemplateInvoker::Tag<int, float, double>()
        );
    },
    "argument"_a
);

The type returned by the helper callable’s operator() can be anything pybind11 knows how to convert to Python.

While defining a full struct with a templated operator() makes it more obvious what TemplateInvoker is doing, it’s much more concise to use a universal lambda with the decltype operator. This wrapper is equivalent to the one above, but it doesn’t need DoSomethingHelper:

mod.def(
    "doSomething",
    [](std::string const & argument, py::dtype const & dtype) {
        return TemplateInvoker().apply(
            [&argument](auto t) { return doSomething<decltype(t)>(argument); },
            dtype,
            TemplateInvoker::Tag<int, float, double>()
        );
    },
    "argument"_a
);

Note that the value of t here doesn’t matter; what’s important is that its C++ type corresponds to the type passed in the dtype argument. So instead of using that value, we use the decltype operator to extract that type and use it as a template parameter.

class WrapperCollection

#include <python.h>

A helper class for subdividing pybind11 module across multiple translation units (i.e. source files).

Merging wrappers for different classes into a single compiled module can dramatically decrease the total size of the binary, but putting the source for multiple wrappers into a single file slows down incremental rebuilds and makes editing unwieldy. The right approach is to define wrappers in different source files and link them into a single module at build time. In simple cases, that’s quite straightforward: pybind11 declarations are just regular C++ statements, and you can factor them out into different functions in different source files.

That approach doesn’t work so well when the classes being wrapped are interdependent, because bindings are only guaranteed to work when all types used in a wrapped method signature have been declared to pybind11 before the method using them is itself declared. Naively, then, each source file would thus have to have multiple wrapper-declaring functions, so all type-wrapping functions could be executed before any method-wrapping functions. Of course, each type-wrapping function would also have to pass its type object to at least one method-wrapping function (to wrap the types own methods), and the result is a tangled mess of wrapper-declaring functions that obfuscate the code with a lot of boilerplate.

WrapperCollection provides a way out of that by allowing type wrappers and their associated methods to be declared at a single point, but the method wrappers wrapped in a lambda to defer their execution. A single WrapperCollection instance is typically constructed at the beginning of a PYBIND11_MODULE block, then passed by reference to wrapper-declaring functions defined in other source files. As type and method wrappers are added to the WrapperCollection by those functions, the types are registered immediately, and the method-wrapping lambdas are collected. After all wrapper-declaring functions have been called, finish() is called at the end of the PYBIND11_MODULE block to execute the collecting method-wrapping lambdas.

Typical usage:

// _mypackage.cc

void wrapClassA(WrapperCollection & wrappers);
void wrapClassB(WrapperCollection & wrappers);

PYBIND11_MODULE(_mypackage, module) {
    WrapperCollection wrappers(module, "mypackage");
    wrapClassA(wrappers);
    wrapClassB(wrappers);
    wrappers.finish();
}

// _ClassA.cc

void wrapClassA(WrapperCollection & wrappers) {
    wrappers.wrapType(
        py::class_<ClassA>(wrappers.module, "ClassA"),
        [](auto & mod, auto & cls) {
            cls.def("methodOnClassA", &methodOnClassA);
        }
    );
}

// _ClassB.cc

void wrapClassB(WrapperCollection & wrappers) {
    wrappers.wrapType(
        py::class_<ClassB>(wrappers.module, "ClassB"),
        [](auto & mod, auto & cls) {
            cls.def("methodOnClassB", &methodOnClassB);
            mod.def("freeFunction", &freeFunction);
        }
    );
}

Note that we recommend the use of universal lambdas (i.e. auto & parameters) to reduce verbosity.

Navigation

Namespace lsst::utils¶