Using weak pointers that evaluate to a None type
Question: In my implementation for python integration into a c++ application, I am
adding support for nodes that might or might not be valid. Internally these
are stored as weak pointers, so I was thinking of having an isValid() method
that users can use before calling the exposed methods. If they call an exposed
method on an invalid node it would throw an exception.

However, I was wondering if it's possible to be a bit more pythonic than that.
Is it possible to internally check whether the pointer is valid before calling
the exposed method, and if it isn't making the python object None?

An example of what I want is here:

    
    
    >>> my_valid_node = corelibrary.getNode("valid_node")
    >>> my_valid_node.printName()
    valid_node
    

Now however, something somewhere else in the system might invalidate the node,
but from python's point of view, I want the node to become None.

    
    
    >>> my_valid_node.printName()
    Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'printName'
    

Can anyone think of a way to do this?


Answer: There is no _clean_ way to make a reference to an object become a reference to
`None` when an external event occurs. Nevertheless, when working towards a
Pythonic interface, one could:

  * Implement the [`__nonzero__`](https://docs.python.org/2/reference/datamodel.html#object.__nonzero__) method to allow the object to be evaluated in a boolean context.
  * Throw a Python exception when the `weak_ptr` fails to lock. One simple solution, would be to access a member attribute on a default constructed `boost::python::object`, as it references `None`.

Note that attribute lookup customization points, such as
[`__getattr__`](https://docs.python.org/2/reference/datamodel.html#object.__getattr__),
will not be sufficient enough as the object pointed to by `weak_ptr` may
expire between attribute access and dispatch to the C++ member functions.

* * *

Here is a complete minimal example based on the above details. In this
example, `spam` and `spam_factory`, a factory that instantiates `spam` objects
managed by `shared_ptr`, are considered legacy types. A `spam_proxy` auxiliary
class that references `spam` via `weak_ptr` and additionally auxiliary
functions help adapt the legacy types to Python.

    
    
    #include <string>
    #include <boost/make_shared.hpp>
    #include <boost/shared_ptr.hpp>
    #include <boost/smart_ptr/weak_ptr.hpp>
    #include <boost/python.hpp>
    
    /// Assume legacy APIs.
    
    // Mockup class containing data.
    class spam
    {
    public:
      explicit spam(const char* name)
       : name_(name)
      {}
    
      std::string name() { return name_; }
    
    private:
      std::string name_;
    };
    
    // Factory for creating and destroying the mockup class.
    class spam_factory
    {
    public:
      boost::shared_ptr<spam> create(const char* name)
      {
        instance_ = boost::make_shared<spam>(name);
        return instance_;
      }
    
      void destroy()
      {
        instance_.reset();
      }
    
    private:
      boost::shared_ptr<spam> instance_;
    };
    
    /// Auxiliary classes and functions to help obtain Pythonic semantics.
    
    // Helper function used to cause a Python AttributeError exception to
    // be thrown on None.
    void throw_none_has_no_attribute(const char* attr)
    {
      // Attempt to extract the specified attribute on a None object.
      namespace python = boost::python;
      python::object none;
      python::extract<python::object>(none.attr(attr))();
    }
    
    // Mockup proxy that has weak-ownership.
    class spam_proxy
    {
    public:
      explicit spam_proxy(const boost::shared_ptr<spam>& impl)
        : impl_(impl)
      {}
    
      std::string name() const  { return lock("name")->name(); }
      bool is_valid() const     { return !impl_.expired();     }
    
      boost::shared_ptr<spam> lock(const char* attr) const
      {
        // Attempt to obtain a shared pointer from the weak pointer.
        boost::shared_ptr<spam> impl = impl_.lock();
    
        // If the objects lifetime has ended, then throw.
        if (!impl) throw_none_has_no_attribute(attr);
    
        return impl;
      }
    
    private:
      boost::weak_ptr<spam> impl_;
    };
    
    // Use a factory to create a spam instance, but wrap it in the proxy.
    spam_proxy spam_factory_create(
      spam_factory& self,
      const char* name)
    {
      return spam_proxy(self.create(name));
    }
    
    BOOST_PYTHON_MODULE(example)
    {
      namespace python = boost::python;
    
      // Expose the proxy class as if it was the actual class.
      python::class_<spam_proxy>("Spam", python::no_init)
        .def("__nonzero__", &spam_proxy::is_valid)
        .add_property("name", &spam_proxy::name)
        ;
    
      python::class_<spam_factory>("SpamFactory")
        .def("create", &spam_factory_create) // expose auxiliary method
        .def("destroy", &spam_factory::destroy)
        ;
    }
    

Interactive usage:

    
    
    >>> import example
    >>> factory = example.SpamFactory()
    >>> spam = factory.create("test")
    >>> assert(spam.name == "test")
    >>> assert(bool(spam) == True)
    >>> if spam:
    ...     assert(bool(spam) == True)
    ...     factory.destroy() # Maybe occurring from a C++ thread.
    ...     assert(bool(spam) == False) # Confusing semantics.
    ...     assert(spam.name == "test") # Confusing exception.
    ... 
    Traceback (most recent call last):
      File "<stdin>", line 5, in <module>
    AttributeError: 'NoneType' object has no attribute 'name'
    >>> assert(spam is not None) # Confusing type.
    

One could argue that while the interface is Pythonic, the object's semantics
are not. With `weak_ptr` semantics not being too common in Python, one
generally does not expect the object referenced by a local variable to be
destructed. If `weak_ptr` semantics are necessary, then consider introducing a
means to allow the user to obtain shared ownership within a specific context
via the [context manager
protocol](https://docs.python.org/2/library/stdtypes.html#context-manager-
types). For example, the following pattern allows an object's validity to be
checked once, then be guaranteed within a limited scope:

    
    
    >>> with spam:                # Attempt to acquire shared ownership.
    ...     if spam:              # Verify ownership was obtained.
    ...         spam.name         # Valid within the context's scope.
    ...         factory.destroy() # spam is still valid.
    ...         spam.name         # Still valid.
    ...                           # spam destroyed once context's scope is exited.
    

Here is a complete extension of the previous example, where in the
`spam_proxy` implements the context manager protocol:

    
    
    #include <string>
    #include <boost/make_shared.hpp>
    #include <boost/shared_ptr.hpp>
    #include <boost/smart_ptr/weak_ptr.hpp>
    #include <boost/python.hpp>
    
    /// Assume legacy APIs.
    
    // Mockup class containing data.
    class spam
    {
    public:
      explicit spam(const char* name)
       : name_(name)
      {}
    
      std::string name() { return name_; }
    
    private:
      std::string name_;
    };
    
    // Factory for creating and destroying the mockup class.
    class spam_factory
    {
    public:
      boost::shared_ptr<spam> create(const char* name)
      {
        instance_ = boost::make_shared<spam>(name);
        return instance_;
      }
    
      void destroy()
      {
        instance_.reset();
      }
    
    private:
      boost::shared_ptr<spam> instance_;
    };
    
    /// Auxiliary classes and functions to help obtain Pythonic semantics.
    
    // Helper function used to cause a Python AttributeError exception to
    // be thrown on None.
    void throw_none_has_no_attribute(const char* attr)
    {
      // Attempt to extract the specified attribute on a None object.
      namespace python = boost::python;
      python::object none;
      python::extract<python::object>(none.attr(attr))();
    }
    
    // Mockup proxy that has weak-ownership and optional shared ownership.
    class spam_proxy
    {
    public:
      explicit spam_proxy(const boost::shared_ptr<spam>& impl)
        : shared_impl_(),
          impl_(impl)
      {}
    
      std::string name() const  { return lock("name")->name(); }
      bool is_valid() const     { return !impl_.expired();     }
    
      boost::shared_ptr<spam> lock(const char* attr) const
      {
        // If shared ownership exists, return it.
        if (shared_impl_) return shared_impl_;
    
        // Attempt to obtain a shared pointer from the weak pointer.
        boost::shared_ptr<spam> impl = impl_.lock();
    
        // If the objects lifetime has ended, then throw.
        if (!impl) throw_none_has_no_attribute(attr);
    
        return impl;
      }
    
      void enter()
      {
        // Upon entering the runtime context, guarantee the lifetime of the
        // object remains until the runtime context exits if the object is
        // alive during this call.
        shared_impl_ = impl_.lock();
      }
    
      bool exit(boost::python::object type,
                boost::python::object value,
                boost::python::object traceback)
      {
        shared_impl_.reset();
        return false; // Do not suppress the exception.
      }
    
    private:
      boost::shared_ptr<spam> shared_impl_;
      boost::weak_ptr<spam> impl_;
    };
    
    // Use a factory to create a spam instance, but wrap it in the proxy.
    spam_proxy spam_factory_create(
      spam_factory& self,
      const char* name)
    {
      return spam_proxy(self.create(name));
    }
    
    BOOST_PYTHON_MODULE(example)
    {
      namespace python = boost::python;
    
      // Expose the proxy class as if it was the actual class.
      python::class_<spam_proxy>("Spam", python::no_init)
        .def("__nonzero__", &spam_proxy::is_valid)
        // Support context manager protocol.
        .def("__enter__", &spam_proxy::enter)
        .def("__exit__", &spam_proxy::exit)
        .add_property("name", &spam_proxy::name)
        ;
    
      python::class_<spam_factory>("SpamFactory")
        .def("create", &spam_factory_create) // expose auxiliary method
        .def("destroy", &spam_factory::destroy)
        ;
    }
    

Interactive usage:

    
    
    >>> import example
    >>> factory = example.SpamFactory()
    >>> spam = factory.create("test")
    >>> with spam:
    ...     assert(bool(spam) == True)
    ...     if spam:
    ...         assert(spam.name == "test")
    ...         factory.destroy()
    ...         assert(bool(spam) == True)
    ...         assert(spam.name == "test")
    ... 
    >>> assert(bool(spam) == False)
    

The exact pattern may not be the most Pythonic, but it provides a clean way to
guarantee the object's lifetime within a limited scope.

