   Unique id
  I'm trying to implement a system that requires a sort of session id that must be unique for logins between multiple servers. The system I have implemented currently has one server that is acting as the master and all other servers as a slave, this is for more reasons than just the session id problem. The slaves request session ids in blocks from the master and use those until their supply runs low, then they request more. The master uses the same block system but they are returned locally. This has solved the problem so far because the master is controlling a single value.An issue arises because if a master server goes down one of the slaves are promoted to the new master. To keep the number unique when a master server updates its internal id it informs all the slaves and they update their own number in case they need to take over at any time. Then at the moment one of the slaves becomes a master it increases its number by a large amount (the equivalent of 1000 block requests) to cover any chance that number increases may have been lost while the master was going down. Although this solution seems to be good enough, I wanted to ask if anyone has dealt with this kind of unique id problem and might know another solution that will work and perhaps be superior to the one I implemented? I have a feeling that database servers handle this problem and might use a well researched solution I do not know about.   
  Three alternatives jump to my mind:1) Don't preallocate id blocks. Instead, just query the master server for each new id.  Make the transaction lightweight so that the id gets returned to the slave without allocating lots of master resources.  When the master server crashes, have all the slave servers report their active ids to the new master.  It can then determine where the sequence should resume.2) Locally generate ids.  These would have to be guid-like in nature.  If the id must be included in every network packet coming from the client (like if you're using a connectionless network protocol) then you may want to locally translate to an integer and index into a lookup table for the "real" id when forwarding messages to backend servers.  For bookkeeping purposes, you might still need to report these active "real" ids to new master servers as they come online.3) Make it so that the master server doesn't crash :);  Why do you need unique session ids? Wouldn't it be just as good with a signed authentication cookie, for example? What is the uniqueness of the session ID used for?   ;  Session ID may not be a good word to describe it. I use the ids to uniquely identify each logged in user in the cluster of servers. Basically I am using them as user ids except that they already have user ids in the database but I needed a new id to handle the case when two users are logged in at the same time. It is rare and currently one user is forced to log out, but must be handled. The id does not need to be globally unique just not already in use on the cluster so after a user logs off that id can be used again.   ;  How about using a file? Each time master allocates a new block, it writes to a file. Or database.Clearly you're dealing with persistent information here, that by definition needs to outlive the host.Quote:to cover any chance that number increases may have been lost while the master was going down. But if master is down, you cannot allocate new IDs, since it's the very master that does that. You just need to wait until it's up again, or fail after some time.Quote:The id does not need to be globally unique just not already in use on the cluster so after a user logs off that id can be used again.Is 32-bit int enough? What about 64? Since this information appears the need to persist only for the duration of run-time, but not indefinitely, why not just increment the counter.Even more - why not follow the web server's method, and simply store sessions as files? There's clean up issues, and sometimes the disk usage may be excessive, but it seems to work just fine for many sites, even busy ones. With that I mean empty files, where filenames identify the session.The problem there is somewhat different, but you can just imagine that your master server goes down each time user session expires, which in case of TCP is very frequently.   ;Quote:Original post by AntheusHow about using a file? Each time master allocates a new block, it writes to a file. Or database.That is true, keeping the number in the database. That would allow the slaves to get the number at the moment they become a master. That is a more stable solution then just incrementing 1000 blocks to cover the gap when a master fails.Quote:But if master is down, you cannot allocate new IDs, since it's the very master that does that. You just need to wait until it's up again, or fail after some time.Correct, and that is what I do already. There is a small window of time when a master goes down to when one of the slaves is promoted to a master and takes over. During this period user logins are queued but can expire if for some reason the window is too long (60 seconds currently). Quote:Is 32-bit int enough? What about 64? Since this information appears the need to persist only for the duration of run-time, but not indefinitely, why not just increment the counter.I forgot to mention that the ID is 64-bit, and it is actually an incrementing counter.Quote:Even more - why not follow the web server's method, and simply store sessions as files? There's clean up issues, and sometimes the disk usage may be excessive, but it seems to work just fine for many sites, even busy ones. With that I mean empty files, where filenames identify the session.The problem there is somewhat different, but you can just imagine that your master server goes down each time user session expires, which in case of TCP is very frequently.Would those files need to be on some sort of shared storage? The ids are used across all of the machines so I am not sure how to apply that to this problem.;  I read right over your post.Quote:Original post by humanapp1) Don't preallocate id blocks. Instead, just query the master server for each new id.  Make the transaction lightweight so that the id gets returned to the slave without allocating lots of master resources.  When the master server crashes, have all the slave servers report their active ids to the new master.  It can then determine where the sequence should resume.They are not actually preallocated because it is just a 64bit number incremented. The only reason for the blocks was that it appeared inefficient to me to request one id at a time when I can return 50 as a range because they are sequential.Quote:2) Locally generate ids.  These would have to be guid-like in nature.  If the id must be included in every network packet coming from the client (like if you're using a connectionless network protocol) then you may want to locally translate to an integer and index into a lookup table for the "real" id when forwarding messages to backend servers.  For bookkeeping purposes, you might still need to report these active "real" ids to new master servers as they come online.This would work but it seems like overkill due to the size of UUID structures, and I really don't need them to be completely unique every time. Just unique to the currently running cluster. Although I like the idea of a look up table that could make this option a little more feasible.Quote:3) Make it so that the master server doesn't crash :)Maybe some day the only reason a server goes down will be because of a software crash.;Quote:I use the ids to uniquely identify each logged in user in the cluster of servers.So an authentication cookie would be just fine?Imagine you created the following symbolic information tuple:global-userid:remote-ip-address:remote-udp-port:cookie-expiry-time:issuing-machineThis would uniquely identify a user for a given session (or cookie lifetime).Now, add a hash of this tuple plus a secret "key" that only your servers know:hash(global-userid:remote-ip-address:remote-udp-port:cookie-expiry-time:issuing-machine:server-secret-key)Now, a user connecting to any one of your servers can provide a tuple and its hash, and you can be sure that the information in the tuple is trusted (assuming the hash checks out when you re-verify it).The verification of the tuple (and sending the tuple) only needs to happen the first time a client talks to a specific server node. The server node can then issue a much smaller, transient, per-node user/session ID (on the order of 16 or 32 bits) that is used in each packet to identify the session. Given that ID, the user and other pertinent information can be looked up in a table. Just make sure that the provided ID is still valid, and comes from the correct remote IP/port.I have written more about this in the latest Game Programming Gems book (volume 7), and also have an article on my web site about it.;Quote:Original post by hplus0603So an authentication cookie would be just fine?I like it and I think I will use it for future use but for this it seems a little much. These ids are used in a very simple way. * User connects to one of the slave servers (or master is possible)* Server authenticates and gives the user an id, the id is only used on the cluster and is not even passed back to the user.* Server informs master of the login and sends all the details about the client master informs all the other servers as wellNow all the servers know the user is logged in and has all the basic information about the user. This is helpful for the friends system and queries against a user for any information that is not required to be stored in the database. One thing to note is that when a client connects to the cluster their connection continues to stay open as long as they are logged in. If the connection closes the user is logged out.So the ids are really just a look up index for a map structure so the servers can find what user an operation is referring to.Quote:global-userid:remote-ip-address:remote-udp-port:cookie-expiry-time:issuing-machineOne thing I didn't even think about that this would be perfect for is if the client did not have a persistent connection to the server and instead connected every time it made a request. Of course this is perfect for web login for that reason. In my case the id is used between the multiple servers not the client to server.;  Couldn't you just use the remote ip:port as the "client id" then?Instead of looking up in an array, look up in a hash table. Problem solved.   