   Windows IPC, heartbeats
  I'm working on transforming the command line interface to the C++ Middleware Writer (C++MW) from a run-once and exit program to being a server that will maintain a connection with the C++MW.  Besides maintaining the connection, we won't have to transmit/verify a password for every request.  I have both a UNIX and windows version of the command line interface and  have gotten the UNIX version working as a server and added a new, small program that communicates with the new server using UNIX domain sockets.  I'm thinking about the windows version now and am not as sure about the IPC there.  I guess the options are named pipes or regular sockets -- winsock.  Are there other options that are worth considering?  Are the advantages to using named pipes on windows analogous to using UNIX domain sockets on Linux?I'm also thinking about heartbeats between the C++MW and this new server I'll callCMWA (C++ Middleware Writer Ambassador).  I have some older books (2000 is the most recent) that discuss heartbeats and I haven't ever worked on them before.  I'm thinking of having the C++MW initiate a heartbeat message every so often.  Any book recommendations on this or links would be appreciated.  Thanks in advance.Brian Woodhttp://webEbenezer.net(651) 251-9384   
Quote:Are the advantages to using named pipes on windows analogous to using UNIX domain sockets on Linux?MySQL at very least uses named pipes.While in theory the overhead is smaller, the communication itself will effectively never be the bottleneck, unless the protocol were to be used for bulk data (megabytes and more in case of SQL), where TCP might be slightly worse off.Sockets on same windows machine bypass the network stack, so there is barely any benefit in using pipes. While it would safe minimal overhead of whatever this special TCP requires, the serialization still remains.For true IPC, shared memory would be used where all processes establish out-of-process shared memory, thereby avoiding serialization overhead altogether.IMHO - the communication overhead (vs. serialization, packing, processing) is the wrong thing to optimize, compared to added complexity. All that applies to LAN/WAN TCP connection handling can be directly reused for communication on localhost, it would carry over any heartbeat logic anyway.   ;  IDL compilation is a very batch-like, client-pulled operation. Persistent connections are pretty much only useful for low-latency, server-pushed operations. I think you have a significant mismatch between requirements and technology here.If I were to design an online IDL compiler, I'd just implement it as a HTTP REST interface. POST the IDL source, return the compiled data. Use SSL with HTTP authentication of some sort. A good HTTP client/server set-up will keep the connection persisted for performance reasons for a while (some # requests and some time), and you get all that for free.Separately, you might also ask yourself how much a potential client would trust your "middleware writer" IDL compiler solution, given the seemingly poor match between requirements and technology you show above.;Quote:Original post by AntheusIMHO - the communication overhead (vs. serialization, packing, processing) is the wrong thing to optimize, compared to added complexity. All that applies to LAN/WAN TCP connection handling can be directly reused for communication on localhost, it would carry over any heartbeat logic anyway.Makes sense.  I've dropped the use of named pipes on Windows and UNIX domain sockets on Linux and am using TCP sockets instead.Brian Wood;Quote:Original post by hplus0603IDL compilation is a very batch-like, client-pulled operation. Persistent connections are pretty much only useful for low-latency, server-pushed operations. I think you have a significant mismatch between requirements and technology here.If I were to design an online IDL compiler, I'd just implement it as a HTTP REST interface. POST the IDL source, return the compiled data. Use SSL with HTTP authentication of some sort. A good HTTP client/server set-up will keep the connection persisted for performance reasons for a while (some # requests and some time), and you get all that for free.I'm working on binary services here, though. Quote:Separately, you might also ask yourself how much a potential client would trust your "middleware writer" IDL compiler solution, given the seemingly poor match between requirements and technology you show above.I'm thinking about things a little differently now than when I started the thread.  As I mentioned in the reply to Antheus, I've dropped the UNIX domain sockets in part to be able to have multiple users, on various machines, be able to use a local C++MW Ambassador instance.   That way the C++MWA pays for the TCP connection and C++MW login costs once and users don't have to pay for that with every request.  I haven't tested this on the internet yet, but plan to over the next few days.  If you know of things we should be watching out for with the testing I'm interested in hearing about that.Brian WoodEbenezer Enterprises;Quote:Original post by wood_brianI'm working on binary services here, though. Yes, and my question is: "Why?" What are the metrics you're trying to improve, and what data do you have to show that the current values of those metrics are hurting your business?HTTP has a perfectly fine connection re-use mechanism already in place, which avoids paying the re-connection and re-authentication cost more than once. And HTTP works across the entire web, including from behind very restrictive firewalls that won't let anything else go through.;Quote:Original post by hplus0603Quote:Original post by wood_brianI'm working on binary services here, though. Yes, and my question is: "Why?" There are some advantages to not using XML in a project.  We have in my opinion a simpler development process than what you are advocating, but we don't have language neutrality.  Thankfully, C++ is >= other languages so it's not like users lose anything by being tied into the language.  Quote:HTTP has a perfectly fine connection re-use mechanism already in place, which avoids paying the re-connection and re-authentication cost more than once. And HTTP works across the entire web, including from behind very restrictive firewalls that won't let anything else go through.I admit we have to reinvent some wheels and in some ways life would be easier if we went in the direction you suggest, but believe we have a good approach.  The differences here may be similar to the differences between poetry and prose.  We've taken a poetic rather than a prosaic approach.  Brian Wood;Quote:HTTP has a perfectly fine connection re-use mechanism already in place, which avoids paying the re-connection and re-authentication cost more than once. And HTTP works across the entire web, including from behind very restrictive firewalls that won't let anything else go through.We may not agree 100% on this, but I thought of something we might agree on.  The introduction of the C++ Middleware Writer Ambassador may be welcomed by system administrators.  Rather than having to support n users that need access to the C++ Middleware Writer, they'll be able to funnel those users through the C++MWA.  So it seems to me there are a couple of perspectives that point toward this being a reasonable approach.  I hadn't thought of the network administration angle until you mentioned it.Brian Wood   ;Quote:There are some advantages to not using XML in a projectNowhere did I say XML. You can upload binary files just fine over HTTP, if you want.But you seem to be set on writing your own ambassador implementation, just like you're set on calling your non-standard IDL compiler/runtime a "middleware writer.";Quote:Original post by hplus0603Nowhere did I say XML. You can upload binary files just fine over HTTP, if you want.But you seem to be set on writing your own ambassador implementation, just like you're set on calling your non-standard IDL compiler/runtime a "middleware writer."Some C++ compilers don't conform exactly to the standard.  We're working at being more standard compliant.  We have support for std::move now and to my knowledge we're the only marshalling/serialization option with this support.  From my home page:Move semantics.  A config file parameter, 'Permit-std::move', is used to indicate that the generated code may use std::move.  The following shows an example of code generated whenPermit-std::move is turned on.template <typename B>voidReceive(B* buf, vector<deque<lil_string> >& abt1){  uint32_t headCount[2];  buf->Give(headCount[0]);  abt1.reserve(abt1.size() + headCount[0]);  for (; headCount[0] > 0; --headCount[0]) {    deque<lil_string> rep4;    buf->Give(headCount[1]);    for (; headCount[1] > 0; --headCount[1]) { lil_string rep5(buf); rep4.push_back(std::move(rep5));    }    abt1.push_back(std::move(rep4));  }}I have another question related to the development of the C++ Middleware Writer Ambassador.  I'm using syslog on Linux and am not sure what to use on Windows.  I've noticed there are syslog implementations available on Windows.  Thoughts on this subject would be helpful.   Using syslog on Windows is kind of tempting from a development standpoint.Brian WoodEbenezer Enterprises 