Description
Required prerequisites
- Make sure you've read the documentation. Your issue may be addressed there.
- Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- Consider asking first in the Gitter chat room or in a Discussion.
Problem description
Consider a simple C++ visitor pattern:
class Data{};
using DataVisitor = std::function<void (const Data&)>;
class Adder {
public:
// interface to add two data objects, the result is passes to data visitor
virtual void operator()(const Data& first, const Data& second, const DataVisitor& visitor) const = 0;
};
void add(const Data& first, const Data& second, const Adder& adder, const DataVisitor& visitor) {
adder(first, second, visitor);
}
void add3(const Data& first, const Data& second, const Data& third, const Adder& adder, const DataVisitor& visitor) {
adder(first, second, [&] (const Data& first_plus_second) {
adder(first_plus_second, third, visitor);
});
}In this pattern, Data and Adder are abstract classes which can be used in two algorithms add and add3 which, as the name suggest, add two or three objects of the type Data and send the result to a visitor. This pattern is intended to eliminate allocating memory for Data pointer in case we need to store temporary results, e.g., see add3 algorithm.
Next, we define the corresponding python bindings:
class PyAdder : public Adder {
public:
void operator()(const Data& first, const Data& second, const DataVisitor& visitor) const override {
PYBIND11_OVERRIDE_PURE_NAME(void, Adder, "__call__", operator(), first, second, visitor);
}
};
class PyData : public Data {
public:
PyData() = default;
};
PYBIND11_MODULE(module, m) {
pybind11::class_<Adder, PyAdder>(m, "Adder")
.def(pybind11::init<>())
.def("__call__", &Adder::operator());
m.def("add", &add);
m.def("add3", &add3);
pybind11::class_<Data, PyData>(m, "Data")
.def(pybind11::init<>());
}Finally, let's test it in Python. Firs,t some boilerplate:
class Adder(module.Adder):
def __call__(self, first, second, visitor):
visitor(Data(first.value + second.value))
class Data(module.Data):
def __init__(self, value):
super().__init__()
self.value = value
def print_result(data):
print(data.value)Testing function add yields the expected result
module.add(Data(1), Data(2), Adder(), print_result) # prints 3However, testing add3 causes failure
module.add3(Data(1), Data(2), Data(3), Adder(), print_result) # raises RuntimeErrorThe error reads
Traceback (most recent call last):
File "*****/test_add.py", line 39, in test_add3_int
module.add3(Data(1), Data(2), Data(3), Adder(), print_result)
File "*****/test_add.py", line 8, in __call__
visitor(Data(first.value + second.value))
RuntimeError: Tried to call pure virtual function "Adder::__call__"
Chaising the problem through pybind11 sources, I found a possible culprit. In function get_type_override, we check if dispatch code is invoked from overridden function.
/* Don't call dispatch code if invoked from overridden function.
Unfortunately this doesn't work on PyPy. */
#if !defined(PYPY_VERSION)
PyFrameObject *frame = PyThreadState_Get()->frame;
if (frame != nullptr && (std::string) str(frame->f_code->co_name) == name
&& frame->f_code->co_argcount > 0) {
PyFrame_FastToLocals(frame);
PyObject *self_caller = dict_getitem(
frame->f_locals, PyTuple_GET_ITEM(frame->f_code->co_varnames, 0));
if (self_caller == self.ptr())
return function();
}
#elseIt returns an empty function if the second override is called from the same frame as the first override. Commenting this check renders a working example. I consider this as a bug, but please let consider proving me wrong.
My questions are:
- In which case it is important to prevent the second dispatch from overridden function?
- Is there a good way to fix things related to Visitor (recursive) or Double Dispatch patterns?
The link for reproducible and extended examples: https://bitbucket.org/yershov/recursive-dispatch/src/main/
Reproducible example code
https://bitbucket.org/yershov/recursive-dispatch/src/main/