   Binary Serialized Data via TCP?
  Hi :D, Its me again. I started a thread not long ago and drifted into a different topic than what it began with. So my question is: How do you guys know what type of data has been sent from your players? I am only familiar with Strings, XML, and I've have heard of binary serilization.I am trying to decide on how to tell what data has been sent by my players - I was thinking of using binary serialization.As most people are more experienced than me on this topic. I was hoping for your opinions.Thanks :)Q   
  Message IDs are tacked onto the front of all messages describing the message format. They are normally represented via enumerations to save space (a short can be used which is 2 bytes).So you have all your messages set up like:Message ID// DataSo for example lets say a player requests to interact with an object54 // 2 bytes where 54 means "interact" often the enumerations are listed in an enum or something5432 // 2 bytes Object IDThis same kind of format goes for things like attacking or other actions client might perform.So you end up with "packets" in the format:size (2 bytes) then the messages one after another(This isn't the only way to handle things by the way. Since TCP is a stream you don't need the size. You can just throw messages down the stream. I used size because normally each "packet" of data is a critical state update where every message must be processed in that group at the same time.)   ;Quote:Original post by quddusaliquddusI am only familiar with Strings, XML, and I've have heard of binary serilization.<xml>  <player name="Abel">    <health>500</health>    <mana amount="120.55"/>  </player></xml>How do you know which types are 'name', 'health' and 'amount'?   ;  Thanks sirisian. That explains it well.Antheus - i was thinking along the lines of using a standard templates for each different type of data e.g.<Move>  <PlayerId>1</PlayerId>  <NewPosition><x>1</x><y>2</y><z>3</z></NewPosition></Move>Select XmlDoc.FirstChild.Name  Case "Move"  ' Code Goes HereEnd SelectBut obviously sirisians suggestion is far better   ;  Binary serialization essentially means that you take the machine representation of a type (such as an int) and stuff it into a buffer. You can then add more types (such as a string, a float, or whatever) to that buffer, and end up with:Buffer: [int][string][float]This is why the process is called serialization -- the data that you add to the buffer is laid out as a series of bytes. Often this process is simple. You create class -- let's call it SerialBuffer -- that maintains a variably sized array of unsigned chars. When adding a byte (unsigned char) to the buffer, you simply do this:void SerialBuffer::Write(unsigned char val){    m_buffer[m_position++] = val;}In this example, m_buffer would be the array of unsigned chars. m_position is the offset into the buffer. To add an integer you will have to either use memcpy(), or do some clever bit shifts to get the bytes in the right order.Note that you are working with the machine's native representation of the data types. On a regular PC, that means little endian byte order. Other platforms may use big endian or an entirely different scheme. Basically they determine which order the bytes that compose a larger type come in. This must be taken into consideration if you intend for your game to run on non-x86 platforms.Quote:Original post by Sirisian(This isn't the only way to handle things by the way. Since TCP is a stream you don't need the size. You can just throw messages down the stream.)Often this is not the case. Being a stream oriented protocol, TCP does not guarantee that two messages sent after one another will not get combined in the subsequent receive operation on the other end. That means that if you send two messages, you might read two and a half messages on the other end, and then read the other half the next time you call recv(). Therefore I recommend preceding all messages with two bytes containing the size of the message minus header, so that you can read exactly as many bytes as you expect.;  Im a little confused. Are we saying that the most efficient way to do it would be for data to be sent without using binary serialization - instead use integer representations of packet size, data type, and data?   ;Quote:Original post by WindryderQuote:Original post by Sirisian(This isn't the only way to handle things by the way. Since TCP is a stream you don't need the size. You can just throw messages down the stream.)Often this is not the case. Being a stream oriented protocol, TCP does not guarantee that two messages sent after one another will not get combined in the subsequent receive operation on the other end. That means that if you send two messages, you might read two and a half messages on the other end, and then read the other half the next time you call recv(). Therefore I recommend preceding all messages with two bytes containing the size of the message minus header, so that you can read exactly as many bytes as you expect.The message ID is normally a set size so you technically can just wait until you have the message ID then map it to the size of the message. It gets complicated so I wouldn't advise it. Always prepend the size myself. (You looked at my post when I was editing it. :\)Quote:Original post by quddusaliquddusIm a little confused. Are we saying that the most efficient way to do it would be for data to be sent without using binary serialization - instead use integer representations of packet size, data type, and data?I think you are confused. When people say binary serialization they are referring to the actual binary representation. Data types have a size. So an ASCII char is 1 byte. A short is 2 bytes (using basic numbers) and integer is 4 bytes. So basically you'd be creating a buffer of bytes and you're placing your raw binary data into it.This is more efficient since writing the number 4294967296 takes 4 bytes (an unsigned integer) and not "4294967296" which is 11 characters (counting null).Also the idea of a format as mentioned before is important. Antheus brought up an important point. XML has a format. When you get that string of data you are expecting something when you see the first tag. What we're saying is that you format the data with message IDs at the front and do something similar but much more compact.<Move><PlayerId>1</PlayerId><NewPosition><x>1</x><y>2</y><z>3</z></NewPosition></Move>50 something bytesin a binary format would beMessage ID // 2 bytesPlayerId   // 2 bytesx// 4 bytes (assuming float)y// 4 bytes12 bytes   ;  i now see what you mean. Thanks guys for the explanations.   ;  Have you guys made any mmo's?   ;  Yes.   