How to encode nested Python Protobuf
Question: Been stumped on this for a while and pulling what is left of my hair out.

Sending non-nested Protobufs from Python to Java and Java to Python without an
issue with WebSockets. My problem is sending a nested version over a
WebSocket. I believe my issue is on the Python encoding side.

Your guidance is appreciated.

.proto file

    
    
    message Response {
      // Reflect back to caller
      required string service_name = 1;
    
      // Reflect back to caller
      required string method_name = 2;
    
      // Who is responding
      required string client_id = 3;
    
      // Status Code
      required StatusCd status_cd = 4;
    
      // RPC response proto
      optional bytes response_proto = 5;
    
      // Was callback invoked
      optional bool callback = 6 [default = false];
    
      // Error, if any
      optional string error = 7;
      //optional string response_desc = 6;
    }
    
    message HeartbeatResult {
        required string service = 1;
        required string timestamp = 2;
        required float status_cd = 3;
        required string status_summary = 4;
    }
    

A Heartbeat result is supposed to get sent in the reponse_proto field of the
Response Protobuf. I am able to do this in Java to Java but Python to Java is
not working.

I've included two variations of the python code. Neither of which works.

    
    
       def GetHeartbeat(self):
        print "GetHeartbeat called"
        import time
        ts = time.time()
        import datetime
        st = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
        heartbeatResult = rpc_pb2.HeartbeatResult()
        heartbeatResult.service = "ALERT_SERVICE"
        heartbeatResult.timestamp = st
        heartbeatResult.status_cd = rpc_pb2.OK
        heartbeatResult.status_summary = "OK"
    
        response = rpc_pb2.Response()
        response.service_name = ""
        response.method_name = "SendHeartbeatResult"
        response.client_id = "ALERT_SERVICE"
        response.status_cd = rpc_pb2.OK 
        response.response_proto = str(heartbeatResult).encode('utf-8')
    
        self.sendMessage(response.SerializeToString())
        print "GetHeartbeat finished"
    
       def GetHeartbeat2(self):
        print "GetHeartbeat called"
        import time
        ts = time.time()
        import datetime
        st = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
        heartbeatResult = rpc_pb2.HeartbeatResult()
        heartbeatResult.service = "ALERT_SERVICE"
        heartbeatResult.timestamp = st
        heartbeatResult.status_cd = rpc_pb2.OK
        heartbeatResult.status_summary = "OK"
    
        response = rpc_pb2.Response()
        response.service_name = ""
        response.method_name = "SendHeartbeatResult"
        response.client_id = "ALERT_SERVICE"
        response.status_cd = rpc_pb2.OK 
        response.response_proto = heartbeatResult.SerializeToString()
        self.sendMessage(response.SerializeToString())
        print "GetHeartbeat finished"
    

Errors on the Java server side are:

    
    
    (GetHeartbeat) Protocol message end-group tag did not match expected tag
    and
    (GetHeartbeat2)
    Message: [org.java_websocket.exceptions.InvalidDataException: java.nio.charset.MalformedInputException: Input length = 1
        at org.java_websocket.util.Charsetfunctions.stringUtf8(Charsetfunctions.java:80)
        at org.java_websocket.WebSocketImpl.deliverMessage(WebSocketImpl.java:561)
        at org.java_websocket.WebSocketImpl.decodeFrames(WebSocketImpl.java:328)
        at org.java_websocket.WebSocketImpl.decode(WebSocketImpl.java:149)
        at org.java_websocket.server.WebSocketServer$WebSocketWorker.run(WebSocketServer.java:593)
    Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:277)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:798)
        at org.java_websocket.util.Charsetfunctions.stringUtf8(Charsetfunctions.java:77)
    


Answer: Solution

Also posted this question on protobuf group

Credit to Christopher Head and Ilia Mirkin for providing input on the google
group

<https://groups.google.com/forum/#!topic/protobuf/Cp7zWiWok9I>

    
    
    response.response_proto = base64.b64encode(heartbeatResult.SerializeToString())
    self.sendMessage(response.SerializeToString())
    

FYI, Ilia also suggested base64 encoding the entire message but this seems to
be working at the moment.

