Class lsst::utils::python::WrapperCollection¶

class WrapperCollection¶

A helper class for subdividing pybind11 module across multiple translation units (i.e. source files).

Merging wrappers for different classes into a single compiled module can dramatically decrease the total size of the binary, but putting the source for multiple wrappers into a single file slows down incremental rebuilds and makes editing unwieldy. The right approach is to define wrappers in different source files and link them into a single module at build time. In simple cases, that’s quite straightforward: pybind11 declarations are just regular C++ statements, and you can factor them out into different functions in different source files.

That approach doesn’t work so well when the classes being wrapped are interdependent, because bindings are only guaranteed to work when all types used in a wrapped method signature have been declared to pybind11 before the method using them is itself declared. Naively, then, each source file would thus have to have multiple wrapper-declaring functions, so all type-wrapping functions could be executed before any method-wrapping functions. Of course, each type-wrapping function would also have to pass its type object to at least one method-wrapping function (to wrap the types own methods), and the result is a tangled mess of wrapper-declaring functions that obfuscate the code with a lot of boilerplate.

WrapperCollection provides a way out of that by allowing type wrappers and their associated methods to be declared at a single point, but the method wrappers wrapped in a lambda to defer their execution. A single WrapperCollection instance is typically constructed at the beginning of a PYBIND11_MODULE block, then passed by reference to wrapper-declaring functions defined in other source files. As type and method wrappers are added to the WrapperCollection by those functions, the types are registered immediately, and the method-wrapping lambdas are collected. After all wrapper-declaring functions have been called, finish() is called at the end of the PYBIND11_MODULE block to execute the collecting method-wrapping lambdas.

Typical usage:

// _mypackage.cc

void wrapClassA(WrapperCollection & wrappers);
void wrapClassB(WrapperCollection & wrappers);

PYBIND11_MODULE(_mypackage, module) {
    WrapperCollection wrappers(module, "mypackage");
    wrapClassA(wrappers);
    wrapClassB(wrappers);
    wrappers.finish();
}

// _ClassA.cc

void wrapClassA(WrapperCollection & wrappers) {
    wrappers.wrapType(
        py::class_<ClassA>(wrappers.module, "ClassA"),
        [](auto & mod, auto & cls) {
            cls.def("methodOnClassA", &methodOnClassA);
        }
    );
}

// _ClassB.cc

void wrapClassB(WrapperCollection & wrappers) {
    wrappers.wrapType(
        py::class_<ClassB>(wrappers.module, "ClassB"),
        [](auto & mod, auto & cls) {
            cls.def("methodOnClassB", &methodOnClassB);
            mod.def("freeFunction", &freeFunction);
        }
    );
}

Note that we recommend the use of universal lambdas (i.e. auto & parameters) to reduce verbosity.

Navigation

Class lsst::utils::python::WrapperCollection¶