Extracting unsigned char from array of numpy.uint8
Question: I have code to extract a numeric value from a python sequence, and it works
well in most cases, but not for a numpy array.

When I try to extract an unsigned char, I do the following

    
    
    unsigned char val = boost::python::extract<unsigned char>(sequence[n]);
    

where sequence is any python sequence and n is the index. I get the following
error:

    
    
    TypeError: No registered converter was able to produce a C++ rvalue of type 
    unsigned char from this Python object of type numpy.uint8
    

How can I successfully extract an unsigned char in C++? Do I have to
write/register special converters for numpy types? I would rather use the same
code that I use for other python sequences, and not have to write special code
that uses the `PyArrayObject*`.


Answer: One can register a custom from-python converter with Boost.Python that handles
conversions from NumPy array scalars, such as `numpy.uint8`, to C++ scalars,
such as `unsigned char`. A custom from-python converter registration has three
parts:

  * A function that checks if a `PyObject` is convertible. A return of `NULL` indicates that the `PyObject` cannot use the registered converter.
  * A construct function that constructs the C++ type from a `PyObject`. This function will only be called if `converter(PyObject)` does not return `NULL`.
  * The C++ type that will be constructed.

Extracting the value from the NumPy array scalar requires a few NumPy C API
calls:

  * [`import_array()`](http://docs.scipy.org/doc/numpy/reference/c-api.array.html#import_array) must be called within the initialization of an extension module that is going to use the NumPy C API. Depending on how the extension(s) are using the NumPy C API, other requirements for importing may need to occur.
  * [`PyArray_CheckScalar()`](http://docs.scipy.org/doc/numpy/reference/c-api.array.html#PyArray_CheckScalar) checks if a `PyObject` is a NumPy array scalar.
  * [`PyArray_DescrFromScalar()`](http://docs.scipy.org/doc/numpy/reference/c-api.array.html#PyArray_DescrFromScalar) gets the [data-type-descriptor](http://docs.scipy.org/doc/numpy/reference/c-api.types-and-structures.html#PyArray_Descr) object for an array scalar. The data-type-descriptor object contains information about how to interpret the underlying bytes. For example, its [`type_num`](http://docs.scipy.org/doc/numpy/reference/c-api.types-and-structures.html#PyArray_Descr.type_num) data member contains an [enum value](http://docs.scipy.org/doc/numpy/reference/c-api.dtype.html#enumerated-types) that corresponds to a C-type.
  * [`PyArray_ScalarAsCtype()`](http://docs.scipy.org/doc/numpy/reference/c-api.array.html#PyArray_ScalarAsCtype) can be used to extract the C-type value from a NumPy array scalar.

* * *

Here is a complete example demonstrating using a helper class,
`enable_numpy_scalar_converter`, to register specific NumPy array scalars to
their corresponding C++ types.

    
    
    #include <boost/cstdint.hpp>
    #include <boost/python.hpp>
    #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
    #include <numpy/arrayobject.h>
    
    // Mockup functions.
    
    /// @brief Mockup function that will explicitly extract a uint8_t
    ///        from the Boost.Python object.
    boost::uint8_t test_generic_uint8(boost::python::object object)
    {
      return boost::python::extract<boost::uint8_t>(object)();
    }
    
    /// @brief Mockup function that uses automatic conversions for uint8_t.
    boost::uint8_t test_specific_uint8(boost::uint8_t value) { return value; }
    
    /// @brief Mokcup function that uses automatic conversions for int32_t.
    boost::int32_t test_specific_int32(boost::int32_t value) { return value; }
    
    
    /// @brief Converter type that enables automatic conversions between NumPy
    ///        scalars and C++ types.
    template <typename T, NPY_TYPES NumPyScalarType>
    struct enable_numpy_scalar_converter
    {
      enable_numpy_scalar_converter()
      {
        // Required NumPy call in order to use the NumPy C API within another
        // extension module.
        import_array();
    
        boost::python::converter::registry::push_back(
          &convertible,
          &construct,
          boost::python::type_id<T>());
      }
    
      static void* convertible(PyObject* object)
      {
        // The object is convertible if all of the following are true:
        // - is a valid object.
        // - is a numpy array scalar.
        // - its descriptor type matches the type for this converter.
        return (
          object &&                                                    // Valid
          PyArray_CheckScalar(object) &&                               // Scalar
          PyArray_DescrFromScalar(object)->type_num == NumPyScalarType // Match
        )
          ? object // The Python object can be converted.
          : NULL;
      }
    
      static void construct(
        PyObject* object,
        boost::python::converter::rvalue_from_python_stage1_data* data)
      {
        // Obtain a handle to the memory block that the converter has allocated
        // for the C++ type.
        namespace python = boost::python;
        typedef python::converter::rvalue_from_python_storage<T> storage_type;
        void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;
    
        // Extract the array scalar type directly into the storage.
        PyArray_ScalarAsCtype(object, storage);
    
        // Set convertible to indicate success. 
        data->convertible = storage;
      }
    };
    
    BOOST_PYTHON_MODULE(example)
    {
      namespace python = boost::python;
    
      // Enable numpy scalar conversions.
      enable_numpy_scalar_converter<boost::uint8_t, NPY_UBYTE>();
      enable_numpy_scalar_converter<boost::int32_t, NPY_INT>();
    
      // Expose test functions.
      python::def("test_generic_uint8",  &test_generic_uint8);
      python::def("test_specific_uint8", &test_specific_uint8);
      python::def("test_specific_int32", &test_specific_int32);
    }
    

Interactive usage:

    
    
    >>> import numpy
    >>> import example
    >>> assert(42 == example.test_generic_uint8(42))
    >>> assert(42 == example.test_generic_uint8(numpy.uint8(42)))
    >>> assert(42 == example.test_specific_uint8(42))
    >>> assert(42 == example.test_specific_uint8(numpy.uint8(42)))
    >>> assert(42 == example.test_specific_int32(numpy.int32(42)))
    >>> example.test_specific_int32(numpy.int8(42))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    Boost.Python.ArgumentError: Python argument types in
        example.test_specific_int32(numpy.int8)
    did not match C++ signature:
        test_specific_int32(int)
    >>> example.test_generic_uint8(numpy.int8(42))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: No registered converter was able to produce a C++ rvalue of type
      unsigned char from this Python object of type numpy.int8
    

A few things to note from the interactive usage:

  * Boost.Python was able to extract `boost::uint8_t` from both `numpy.uint8` and `int` Python objects.
  * The `enable_numpy_scalar_converter` does not support promotions. For instance, it should be safe for `test_specific_int32()` to accept a `numpy.int8` object that is promoted to a larger scalar type, such as `int`. If one wishes to perform promotions: 
    * `convertible()` will need to check for compatible `NPY_TYPES`
    * `construct()` should use [`PyArray_CastScalarToCtype()`](http://docs.scipy.org/doc/numpy/reference/c-api.array.html#PyArray_ScalarAsCtype) to cast the extracted array scalar value to the desired C++ type.

