<story><title>Pb-jelly – A Protobuf code generation framework for Rust, developed at Dropbox</title><url>https://github.com/dropbox/pb-jelly</url></story><parent_chain><item><author>haberman</author><text>I work on the protobuf team at Google, and I&amp;#x27;m a big fan of Rust, though I haven&amp;#x27;t written much actual Rust except a bunch of Project Euler solutions.&lt;p&gt;For protobuf in C++, we&amp;#x27;ve been moving more and more in the direction of using arenas for memory allocation. When you parse a protobuf, it creates a tree of objects that are usually all deleted at the same time. Freeing an arena is much, much cheaper than traversing the tree of objects and calling free() on each one.&lt;p&gt;My dream has been that Rust protobuf could support arenas as well as C++, but use Rust&amp;#x27;s type system to make it all provably correct at compile time (in C++ the lifetime management is inherently manual and unsafe).  For absolute top performance, arenas will always beat trees of unique pointers (which I think corresponds to Rust&amp;#x27;s Box&amp;lt;&amp;gt; type).&lt;p&gt;I don&amp;#x27;t know Rust&amp;#x27;s type&amp;#x2F;lifetime system well enough to know if this is possible.  I was looking recently at arenas in Rust and I noticed that Rust&amp;#x27;s version of placement new seems to be stalled:&lt;p&gt;&amp;quot;Unfortunately the path forward for placement new in Rust does not look good right now, so I&amp;#x27;ve reverted this crate to work more like a memory heap where stuff can be put, but not constructed in place.&amp;quot;&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;docs.rs&amp;#x2F;light_arena&amp;#x2F;1.0.1&amp;#x2F;light_arena&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;docs.rs&amp;#x2F;light_arena&amp;#x2F;1.0.1&amp;#x2F;light_arena&amp;#x2F;&lt;/a&gt;&lt;p&gt;Does anyone know more about this?</text></item></parent_chain><comment><author>nipunn1313</author><text>Hey @haberman! One of the authors here.
We actually do have an arena-esque implementation built on top of pb-jelly internally, as it was needed for Magic Pocket.&lt;p&gt;It&amp;#x27;s built on top of the Blob traits exposed by pb-jelly. It&amp;#x27;s not yet open-source, but it would be a good candidate to do next!  It also definitely has unsafe code to your point. We open sourced the safe implementations that uses more standard types (Bytes&amp;#x2F;Buffer&amp;#x2F;Vec) first.&lt;p&gt;There&amp;#x27;s a decent amount of cleanup needed before we can opensource that as well, as much of it was built years ago, when rust ecosystem was less mature (eg Bytes&amp;#x2F;Buffer weren&amp;#x27;t around yet).&lt;p&gt;I like where you&amp;#x27;re thinking!</text></comment>
<story><title>Pb-jelly – A Protobuf code generation framework for Rust, developed at Dropbox</title><url>https://github.com/dropbox/pb-jelly</url></story><parent_chain><item><author>haberman</author><text>I work on the protobuf team at Google, and I&amp;#x27;m a big fan of Rust, though I haven&amp;#x27;t written much actual Rust except a bunch of Project Euler solutions.&lt;p&gt;For protobuf in C++, we&amp;#x27;ve been moving more and more in the direction of using arenas for memory allocation. When you parse a protobuf, it creates a tree of objects that are usually all deleted at the same time. Freeing an arena is much, much cheaper than traversing the tree of objects and calling free() on each one.&lt;p&gt;My dream has been that Rust protobuf could support arenas as well as C++, but use Rust&amp;#x27;s type system to make it all provably correct at compile time (in C++ the lifetime management is inherently manual and unsafe).  For absolute top performance, arenas will always beat trees of unique pointers (which I think corresponds to Rust&amp;#x27;s Box&amp;lt;&amp;gt; type).&lt;p&gt;I don&amp;#x27;t know Rust&amp;#x27;s type&amp;#x2F;lifetime system well enough to know if this is possible.  I was looking recently at arenas in Rust and I noticed that Rust&amp;#x27;s version of placement new seems to be stalled:&lt;p&gt;&amp;quot;Unfortunately the path forward for placement new in Rust does not look good right now, so I&amp;#x27;ve reverted this crate to work more like a memory heap where stuff can be put, but not constructed in place.&amp;quot;&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;docs.rs&amp;#x2F;light_arena&amp;#x2F;1.0.1&amp;#x2F;light_arena&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;docs.rs&amp;#x2F;light_arena&amp;#x2F;1.0.1&amp;#x2F;light_arena&amp;#x2F;&lt;/a&gt;&lt;p&gt;Does anyone know more about this?</text></item></parent_chain><comment><author>zozbot234</author><text>Rust objects can always be moved to arbitrary places in memory, so I&amp;#x27;m not sure how &amp;quot;in place&amp;quot; construction of a Rust object can even make sense unless it explicitly involves the Pin&amp;lt;P&amp;gt; feature to prevent moves. This is quite different from C&amp;#x2F;C++ where there&amp;#x27;s no default expectation that a constructed object can be bitwise-copied somewhere else in memory. Even Box&amp;lt;T&amp;gt; is really a special case since the object is meant to be accessed via an owning pointer or a (shared or mutable) reference; when accessing the underlying T by value, that always involves a move of the object so again there&amp;#x27;s no such thing as being &amp;quot;in place&amp;quot;.</text></comment>