Simple PacketHandler class
My two cents : MemoryStream is a disposable object (inherited from Stream)

using (var reader = new BinaryReader(new MemoryStream(data)))


should grant you with a warning CA2000.

You may check this point with something like

Stream stream = null ;
try
{
   Stream = new MemoryStream(data) ;
   using (var reader = new BinaryReader(stream))
...
}
finally
{
    if (stream != null)
    {
        stream.Dispose() ;
    }
}


This should ensure you are freeing the stream in time.

Hope this helps !