Inheritance and py::cast

[aldanor]

Maybe I'm missing something obvious, but I couldn't figure why py::cast doesn't respect inheritance properly? Or does it? (I'm trying to get a Python object from a C++ object inside a function in order to do some more stuff with it before returning -- I'd expect py::cast to obey the same rules as return values)

In the example below, I would naturally expect A type to be returned from both functions, whereas for some reason it's B in the second example.

#include <pybind11/pybind11.h>
namespace py = pybind11;

struct A { }; struct B : A { };

PYBIND11_PLUGIN(pb11_inh) {
    py::module m("pb11_inh");
    py::class_<A>(m, "A").def(py::init());
    py::class_<B>(m, "B", py::base<A>()).def(py::init());
    m.def("f1", [](const B& b) { const A& a = b; return a; });
    m.def("f2", [](const B& b) { const A& a = b; return py::cast(a); });
    return m.ptr();
}

>>> from pb11_inh import A, B, f1, f2
>>> b = B()
>>> f1(b)
<pb11_inh.A at 0x7f2b004c4180>  # as expected
>>> f2(b)
<pb11_inh.B at 0x7f2b0191af10>  # ???

[wjakob]

In the first case, a new copy is made (the object returns a C++ reference, and the default return value policy specifies to make a copy in that case).

The py::cast<> function has a different return value policy for internal reasons. There, the default policy is to return the original object when a reference is encountered. Pybind11 recognizes that the pointer 0x7f2b0191af10 is known already, and it is mapped as an Python object instance of type "B". Python has no concept of casting an instance to a different type, so the "highest" known type in the class hierarchy is always used.

[wjakob]

(You can enforce a custom return value policy as a template argument to cast.)

[aldanor]

Thanks! That makes sense. (Albeit is a bit unobvious -- I don't think this is noted anywhere in the docs, just the source code)

[wjakob]

Many a bit more low level pybind11 functions like py::cast<> and the internal wrappers like float_ are poorly documented at the moment.

[aldanor]

Back to the example for a second, still can't make it work :) Do you mean changing the code in f2 like so? This still yields the object of type B on the Python side of things though.

const A& a = b; 
return py::cast<A>(a, py::return_value_policy::copy);

Upd: I don't think a single particular return value policy passed to cast<>() will make it return <A> in this example unless I've missed something.

[wjakob]

Ok, digging a bit more -- this is the expected behavior (which could be documented a bit more I suppose). My explanation at the beginning was bogus.

When you hand pybind11 an object that is known already, it will give you back the corresponding PyObject wrapper no matter what the return value policy is.

Return value policies specify what is done when pybind11 doesn't know the object yet -- in that case, the lifetime and ownership of the value needs to be clarified.

If you want to ensure that a copy is made, you'll have to modify your code as follows:

A a = b; 
return py::cast(a);

A return value policy doesn't need to be specified.

[aldanor]

Ok, now it makes sense! Thanks again 👍

[wjakob]

I took a stab at writing an improved explanation: fb6aed2

[aldanor]

Looks great 👍