title: Encoding/Serialization issues when using ICacheClient and protobuf in ServiceStack
body: <p>I'm using the current ServiceStack with protobuf serialization.</p>
  
  <p>When adding an ICacheClient to cache my responses, the binary answer sent from the cache client has a different encoding/binary serialization than the original response without any cache client.</p>
  
  <p>This is leading to deserialization problems on the client side, where I'm using a precompiled deserializer.</p>
  
  <p>For example, this protobuf exception occurs at the client side when using an ICacheClient on the ServiceStack server side:</p>
  
  <pre><code>OverflowException: Number overflow.
  ProtoBuf.ProtoReader.TryReadUInt32VariantWithoutMoving (Boolean trimNegative, System.UInt32&amp; value)
  ProtoBuf.ProtoReader.ReadUInt32Variant (Boolean trimNegative)
  ProtoBuf.ProtoReader.ReadUInt32 () MyModelSerializer.Read (My.Models.MyDataModel , ProtoBuf.ProtoReader )
  </code></pre>
  
  <p>(I believe this is a random exception, pretty sure there can occur other exceptions as well. Take this as an example.)</p>
  
  <p>This behaviour is the same with both MemoryCacheClient and Redis cache client.</p>
  
  <p>This is how I initialize protobuf:</p>
  
  <pre><code>ContentTypeFilters.Register(ContentType.ProtoBuf,
                          (reqCtx, res, stream) =&gt; ProtoBuf.Serializer.NonGeneric.Serialize(stream, res),
                           ProtoBuf.Serializer.NonGeneric.Deserialize);
  </code></pre>
  
  <p>The Cache Client is initialized like this:</p>
  
  <pre><code>container.Register&lt;ICacheClient&gt;(new MemoryCacheClient());
  </code></pre>
  
  <p>(Or Redis following <a href="https://github.com/ServiceStack/ServiceStack/wiki/Caching" rel="nofollow noreferrer">this guide</a>).</p>
  
  <p>This is how the cached protobuf-serialized data which goes over the wire looks like and differs from the uncached one:
  <img src="https://i.stack.imgur.com/GGc38.png" alt="Binary diff">
  <a href="https://i.stack.imgur.com/GGc38.png" rel="nofollow noreferrer">Big Image</a></p>
  
  <p>The uncached response seems to be encoded/serialized differently.</p>
  
  <p>I there anything I can do about this encoding issue to make Redis or the Memory-ICacheClient compatible with protobuf?</p>
  
  <hr>
  
  <p><strong>Update: I investigated a bit, these are my findings:</strong></p>
  
  <ul>
  <li><p>To be stored within the cache provider (whether it be Redis os Memory), the protobuf binary stream is "converted" to a string via a StreamReader in HttpResponseFilter->SerializeToString using "UTF8 without BOM"-Encoding, which itself is called via CacheClientExtensions->Cache().</p></li>
  <li><p><strong>First Problem</strong>: CacheClientExtensions->Cache() then returns the to-string-serialized DTO, making it yet already impossible for protobuf to deserialize. <strong>First Solution</strong>: Return the original DTO in CacheClientExtensions->Cache(). But that would only work for the first non-cached response, since the cached response won't yet be properly de-serialized. This takes us to the</p></li>
  <li><p><strong>Second Problem</strong>: Getting data out of the cache again requires a correct to-string-serialization of protobuf's binary data to put it into the cache in the first place. I know it would be working using via base64.</p></li>
  <li><p><strong>Third Problem</strong>: Currently there seems to be no way to replace the current stream-to-string conversion via StreamReader in HttpResponseFilter->SerializeToString at runtime, am I right?</p></li>
  <li><p><strong>Fourth Problem</strong>: When getting the data back from the cache, it has to be base64-decoded again. </p></li>
  </ul>