How to pass a C++ object to another C++ object with Boost.Python
Question: I have some C++ code that defines two classes, A and B. B takes an instance of
A during construction. I have wrapped A with Boost.Python so that Python can
create instances of A, as well as subclasses. I want to do the same with B.

    
    
    class A {
        public:
            A(long n, long x, long y) : _n(n), _x(x), _y(y) {};
            long get_n() { return _n; }
            long get_x() { return _x; }
            long get_y() { return _y; }
        private:
            long _n, _x, _y;
    };
    
    class B {
        public:
            B(A a) : _a(a) {};
            doSomething() { ... };
        private:
            A _a;
    };
    

While wrapping B, I needed to work out how to pass an instance of A to B's
constructor. I did some digging and the
[solution](http://misspent.wordpress.com/2009/09/27/how-to-write-boost-python-
converters/) I found was to write a "converter" class:

    
    
    struct A_from_python_A {
        static void * convertible(PyObject* obj_ptr) {
            // assume it is, for now...
            return obj_ptr;
        }
    
        // Convert obj_ptr into an A instance
        static void construct(PyObject* obj_ptr,
                          boost::python::converter::rvalue_from_python_stage1_data* data) {
            // extract 'n':
            PyObject * n_ptr = PyObject_CallMethod(obj_ptr, (char*)"get_n", (char*)"()");
            long n_val = 0;
            if (n_ptr == NULL) {
                cout << "... an exception occurred (get_n) ..." << endl;
            } else {
                n_val = PyInt_AsLong(n_ptr);
                Py_DECREF(n_ptr);
            }
    
            // [snip] - also do the same for x, y
    
            // Grab pointer to memory into which to construct the new A
            void* storage = (
                (boost::python::converter::rvalue_from_python_storage<A>*)
                data)->storage.bytes;
    
            // in-place construct the new A using the data
            // extracted from the python object
            new (storage) A(n_val, x_val, y_val);
    
            // Stash the memory chunk pointer for later use by boost.python
            data->convertible = storage;
        }
    
        // register converter functions
        A_from_python_A() {
            boost::python::converter::registry::push_back(
                &convertible,
                &construct,
                boost::python::type_id<A>());
        }
    };
    

Then I register this with:

    
    
    BOOST_PYTHON_MODULE(interpolation_ext)
    {
        // register the from-python converter for A
        A_from_python_A();
    
        class_<A>("A", init<long, long, long>())
            ;
    
        class_<B>("B", init<object>())
            ;
    }
    

Convertible and construct are methods that answer the "is this convertible?"
and "how to convert?" questions respectively. I have observed that the
construct() method is non-trivial - it has to reach into A's PyObject*,
extract all relevant fields, then rebuild a C++ instance that it then passes
to B's constructor. Because A contains some private fields, it has to do this
via public access mechanisms (whereas with a pure Python object it wouldn't
have to, right?). This seems to work.

However, _is the field extraction in the 'construct' function really
necessary?_ It seems laborious. If A is a compound object, it could get very
complicated, and possibly require one converter to invoke another. I perhaps
understand the requirement if A is a Python class, but if the A instance
originated from the C++ side, is there a way to determine that this is the
case, and then simply get a handle (e.g. pointer) to this 'native' object, as
a shortcut?

Here's the associated python code:

    
    
    from my_ext import A, B
    a = A(1,2,3)
    b = B(a)
    b.doSomething()
    


Answer: In short, define `B`'s wrapper as:

    
    
    class_<B>( "B", init< A >() )
    

instead of

    
    
    class_<B>( "B", init< object >() )
    

When defining a wrapper for class in Boost.Python (at least in 1.50), the
`class_` template generates convert and construct functions. This allows `A`
to be converted to and constructed from a `A`'s wrapper. These `PyObject`
conversions have strict type-checking, and require that the following be true
in python: `isinstance( obj, A )`.

Custom converters are often used to support:

  * Automatic conversions to and from existing Python types. For example, converting `std::pair< long, long >` to and from a `PyTupleObject`.
  * Duck-typing. For example, having `B` accept class `D`, which is not derived from `A`, as long as `D` provides a compatible interface.

* * *

### Constructing `B` from an instance of `A`

Since `A` and `B` are neither existing Python types nor is duck-typing
required, custom converters are not necessary. For `B` to take an instance of
`A`, it can be as simple as specifying that `init` takes an `A`.

Here is a simplified example of `A` and `B`, where `B` can be constructed from
an `A`.

    
    
    class A
    {
    public:
      A( long n ) : n_( n ) {};
      long n() { return n_; }
    private:
      long n_;
    };
    
    class B
    {
    public:
      B( A a ) : a_( a ) {};
      long doSomething() { return a_.n() * 2; }
    private:
      A a_;
    };
    

And the wrappers would be defined as:

    
    
    using namespace boost::python;
    BOOST_PYTHON_MODULE(example)
    {
      class_< A >( "A", init< long >() )
        ;
    
      class_<B>( "B", init< A >() )
        .def( "doSomething", &B::doSomething )
        ;
    }
    

`B`'s wrapper explicitly indicates that it will be constructed from an `A`
object via `init< A >()`. Also, `A`'s interface is not fully exposed to the
Python objects, as no wrapper was defined for the `A::n()` function.

    
    
    >>> from example import A, B
    >>> a = A( 1 )
    >>> b = B( a )
    >>> b.doSomething()
    2
    

This also works for types that are derived from `A`. For example:

    
    
    >>> from example import A, B
    >>> class C( A ):
    ...     def __init__( self, n ):
    ...         A.__init__( self, n )
    ... 
    >>> c = C( 2 )
    >>> b = B( c )
    >>> b.doSomething()
    4
    

However, duck-typing is not enabled.

    
    
    >>> from example import A, B
    >>> class E: pass
    ... 
    >>> e = E()
    >>> b = B( e )
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    Boost.Python.ArgumentError: Python argument types in
        B.__init__(B, instance)
    did not match C++ signature:
        __init__(_object*, A)
    

* * *

### Constructing `B` from an object that is convertible to `A`.

To support the case where `B` can be constructed from an object that provides
a compatible interface, then custom converters are required. Although wrappers
were not previously generated for `A::n()`, lets continue with the statement
that an object can be converted to `A` if the object provides a `get_num()`
method that returns an `int`.

First, write an `A_from_python` struct that provides converter and
constructors functions.

    
    
    struct A_from_python
    {
      static void* convertible( PyObject* obj_ptr )
      {
        // assume it is, for now...
        return obj_ptr;
      }
    
      // Convert obj_ptr into an A instance
      static void construct(
        PyObject* obj_ptr,
        boost::python::converter::rvalue_from_python_stage1_data* data)
      {
        std::cout << "constructing A from ";
        PyObject_Print( obj_ptr, stdout, 0 );
        std::cout << std::endl;
    
        // Obtain a handle to the 'get_num' method on the python object.
        // If it does not exists, then throw.
        PyObject* n_ptr = 
          boost::python::expect_non_null( 
            PyObject_CallMethod( obj_ptr,
                                 (char*)"get_num",
                                 (char*)"()"  ));
    
        long n_val = 0;
        n_val = PyInt_AsLong( n_ptr );
        Py_DECREF( n_ptr );
    
        // Grab pointer to memory into which to construct the new A
        void* storage = (
          (boost::python::converter::rvalue_from_python_storage< A >*)
           data)->storage.bytes;
    
        // in-place construct the new A using the data
        // extracted from the python object
        new ( storage ) A( n_val );
    
        // Stash the memory chunk pointer for later use by boost.python
        data->convertible = storage;
      }
    
      A_from_python()
      {
        boost::python::converter::registry::push_back(
          &convertible,
          &construct,
          boost::python::type_id< A >() );
      }
    };
    

`boost::python::expect_non_null` is used to throw an exception if `NULL` is
returned. This helps provide the duck-typing guarantee that the python object
must provide a `get_num` method. If the `PyObject` is known to be an instance
of given type, then it is possible to use
[`boost::python::api::handle`](http://www.boost.org/doc/libs/1_50_0/libs/python/doc/v2/handle.html)
and
[`boost::python::api::object`](http://www.boost.org/doc/libs/1_50_0/libs/python/doc/v2/object.html)
to directly extract the type, and avoid having to generically make calls
through the `PyObject` interface.

Next, register the converter in the module.

    
    
    using namespace boost::python;
    BOOST_PYTHON_MODULE(example)
    {
      // register the from-python converter for A
      A_from_python();
    
      class_< A >( "A", init< long >() )
        ;
    
      class_<B>( "B", init< A >() )
        .def( "doSomething", &B::doSomething )
        ;
    }
    

No changes have occurred to `A`, `B`, or their associated wrapper definitions.
The auto-conversion functions were created, and then defined/registered within
the module.

    
    
    >>> from example import A, B
    >>> a = A( 4 )
    >>> b = B( a )
    >>> b.doSomething()
    8
    >>> class D:
    ...     def __init__( self, n ):
    ...         self.n = n
    ...     def get_num( self ):
    ...         return self.n
    ... 
    >>> d = D( 5 )
    >>> b = B( d )
    constructing A from <__main__.D instance at 0xb7f7340c>
    >>> b.doSomething()
    10
    >>> class E: pass
    ...
    >>> e = E()
    >>> b = B( e )
    constructing A from <__main__.E instance at 0xb7f7520c>
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: get_num
    

`D::get_num()` exists, and thus `A` is constructed from an instance of `D`
when `D` is passed to `B`'s constructor. However, `E::get_num()` does not
exists, and an exception is raised when trying to construct `A` from an
instance of `E`.

* * *

### An alternative conversion solution.

Implementing duck-typing via the C-API can become very complicated with larger
types. An alternative solution is perform the duck-typing in python, and
distribute the python file alongside the library.

`example_ext.py` will import the `A` and `B` types, as well as monkey patch
`B`'s constructor:

    
    
    from example import A, B
    
    def monkey_patch_B():
        # Store handle to original init provided by Boost.
        original_init = B.__init__
    
        # Construct an A object via duck-typing.
        def construct_A( obj ):
            return A( obj.get_num() )
    
        # Create a new init that will delegate to the original init.
        def new_init( self, obj ):
            # If obj is an instance of A, use it.  Otherwise, construct
            # an instance of A from object.
            a = obj if isinstance( obj, A ) else construct_A ( obj )
    
            # Delegate to the original init.
            return original_init( self, a )
    
        # Rebind the new_init.
        B.__init__ = new_init
    
    monkey_patch_B()
    

The only change required to the end-user is to import `example_ext` instead of
`example`:

    
    
    >>> from example_ext import A, B
    >>> a = A( 6 )
    >>> b = B( a )
    >>> b.doSomething()
    12
    >>> class D:
    ...     def __init__( self, n ):
    ...         self.n = n
    ...     def get_num( self ):
    ...         return self.n
    ... 
    >>> d = D( 7 )
    >>> b = B( d )
    >>> b.doSomething()
    14
    >>> class E: pass
    ... 
    >>> e = E()
    >>> b = B( e )
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "example_ext.py", line 15, in new_init
        a = obj if isinstance( obj, A ) else construct_A ( obj )
      File "example_ext.py", line 9, in construct_A
        return A( obj.get_num() )
    AttributeError: E instance has no attribute 'get_num'
    

Since the patched constructor guarantees an instance of `A` will be passed to
`B`, the `A_from_python::construct` will not get invoked. Hence, the missing
print statements in the output.

While this approach avoids the C-API, making it easier to perform duck-typing,
it does have one major trade-off in that it requires parts of the API to be
specifically patched for conversions. On the other hand, no patching is
required when auto-type conversion functions are available.

* * *

Also, for what it is worth, access control in both C++ and Python is intended
to protect against accidental misuse. Neither protect against deliberately
obtaining access to members with private visibility. It is much easier to do
in Python, but it is specifically permitted in the C++ Standard through
explicit template instantiations.

