__qualname__ for methods

[m-chaturvedi]

The __qualname__ attribute for methods created using pybind11 seems to be not consistent with that created by cpython.
Testing Repo: https://github.com/m-chaturvedi/reproduce_issues/tree/master/pybind_qualname
Related to #1171 via @jagerman .
PEP: https://www.python.org/dev/peps/pep-3155/

$  /bin/cat main.py
#!/usr/bin/python3
# Tested with python 3.7

import python_module
import pybind_module
from pprint import pprint

if __name__ == "__main__":
    assert python_module.Pet.get_data.__qualname__ == "Pet.get_data"
    assert python_module.Pet.__init__.__qualname__ == "Pet.__init__"

    assert pybind_module.Pet.get_data.__qualname__ == "PyCapsule.get_data"
    assert pybind_module.Pet.__init__.__qualname__ == "PyCapsule.__init__"

$  /bin/cat main.cpp
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;

class Pet {
 public:
        Pet() {
            c = 10;
        }
        ~Pet() {}
        int get_data() {
            return c;
        }

 private:
        int c;
};


PYBIND11_MODULE(pybind_module, m) {
    using rvp = pybind11::return_value_policy;
    py::class_<Pet>(m, "Pet")
        .def(py::init<>())
        .def("get_data", &Pet::get_data);
}

$  /bin/cat python_module.py

class Pet(object):

    def __init__(self):
        pass

    def get_data(self):
        return 10

$  /bin/cat CMakeLists.txt
cmake_minimum_required(VERSION 2.8.5)
project(test_pybind11 CXX)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# Tested with python 3.7
set(PATH_TO_PYBIND /home/chaturvedi/workspace/pybind11/build_master_ninja/install/share/cmake/pybind11)
# IMPORT pybind11
find_package(pybind11 REQUIRED CONFIG PATHS ${PATH_TO_PYBIND})

configure_file(main.py ${CMAKE_BINARY_DIR} COPYONLY)
configure_file(python_module.py ${CMAKE_BINARY_DIR} COPYONLY)
pybind11_add_module(pybind_module main.cpp)

[Skylion007]

Also could be due to self pointing to an unbound PyBind11 class and therefore not being converted properly.

[Skylion007]

May be related to: #2722

[XuehaiPan]

The logic for PyCFunctionObject.__qualname__ goes here:

https://github.com/python/cpython/blob/f84754b70506f03bcbf9fb0265e327d05a1a4b51/Objects/methodobject.c#L212-L244

    /* If __self__ is a module or NULL, return m.__name__
       (e.g. len.__qualname__ == 'len')

       If __self__ is a type, return m.__self__.__qualname__ + '.' + m.__name__
       (e.g. dict.fromkeys.__qualname__ == 'dict.fromkeys')

       Otherwise return type(m.__self__).__qualname__ + '.' + m.__name__
       (e.g. [].append.__qualname__ == 'list.append') */

In pybind11, we set fn->m_ml->def->ml_meth = py::cpp_function::dispatcher: L547-L548

pybind11/include/pybind11/pybind11.h

Lines 544 to 549 in 0116906

	rec->def = new PyMethodDef();
	std::memset(rec->def, 0, sizeof(PyMethodDef));
	rec->def->ml_name = rec->name;
	rec->def->ml_meth
	= reinterpret_cast<PyCFunction>(reinterpret_cast<void (*)()>(dispatcher));
	rec->def->ml_flags = METH_VARARGS | METH_KEYWORDS;

and set fn->m_self = rec_capsule which is a py::capsule object: L551-L553, L565

pybind11/include/pybind11/pybind11.h

Lines 551 to 565 in 0116906

	capsule rec_capsule(unique_rec.release(),
	detail::get_function_record_capsule_name(),
	[](void *ptr) { destruct((detail::function_record *) ptr); });
	guarded_strdup.release();
	object scope_module;
	if (rec->scope) {
	if (hasattr(rec->scope, "__module__")) {
	scope_module = rec->scope.attr("__module__");
	} else if (hasattr(rec->scope, "__name__")) {
	scope_module = rec->scope.attr("__name__");
	}
	}
	m_ptr = PyCFunction_NewEx(rec->def, rec_capsule.ptr(), scope_module.ptr());

We use the fn->m_self record capsule in the py::cpp_function::dispatcher function as the self argument to support overloading:

pybind11/include/pybind11/pybind11.h

Line 717 in 0116906

static PyObject *dispatcher(PyObject *self, PyObject *args_in, PyObject *kwargs_in) {

---

In best practice, the m_self field should be nullptr, a module object, or a type object. However, we have a PyCapsule object (the rec_capsule) in pybind11.

Then in https://github.com/python/cpython/blob/f84754b70506f03bcbf9fb0265e327d05a1a4b51/Objects/methodobject.c#L225-L230, we get:

type_qualname = Py_TYPE(fn->m_self).tp_name  // "PyCapsule"

this results in the following:

fn.__qualname__  # -> f'PyCapsule.{fn.__name__}'

[XuehaiPan]

I think the only way is to use currying the function_record in the dispatcher and make __self__ to be NULL in the PyCFunctionObject. Any thoughts?