问:
"Separate Query from Modifier" and return values
I've heard it said that you should keep query and modifier functions separate from one another, and I agree with the sentiment. When trying to apply it, I often run into two issues/questions:

What, if anything, should a modifier return? Should it not return an indicator of success/failure? Should it instead return nothing, but throw an exception upon failure?

Should constructors be treated as modifiers (is the creation of an object a side-effect)? I suspect not, since a constructor that created an object but didn't return a reference to it would be pretty annoying.

答:
Actually answer to this is pretty simple, but not short.

What, if anything, should a modifier return? Should it not return an indicator of success/failure? Should it instead return nothing, but throw an exception upon failure?

First you have to ask yourself why you should not return anything from command (modifier). What we gain by that?

There is actually one advantage worth it - we can run that command asynchronously in a background, i.e. put command that creates new post to command queue, that is handled by other process, maybe written in another language.
By that you can just put command to queue and return http response immediately not waiting for post creation to happen, big performance gain, right?

Don't mistake returning value from command and returning http response. You must do the latter to inform client that putting command to queue was successful or not, but only that.
Raising exception from command is the same as returning value, so you cannot do that.

Now is the question, how to return id of the newly created post to client, because we want to let user edit that post immediately.

So, commands doesn't have return value, ok. But they have result, an effect.
In example if call turnLightOn command it doesn't return true or false to indicate that it worked - it's silly, but we can see with our own eyes that lights are on and we can see at night. That is an effect of command.
How do we represent effect in programming? By events of course.
Command that turned lights on, also publishes event LightsTurnedOn (past time) and client is subscribed to that event.

Going back to our post creation example:

client sends request to http server that he wants to create post with title 'Super post'

server receives that request and put it into command bus (queue) and returns HTTP response with code 200

process that is handling commands collects our newly put command whenever it is able to (has free workers), do what is have to do - creates post and publish event PostCreated(userId, postId)

client has to somehow subscribe for that event, and there are several ways:

query event service from time to time, to check if new event is published for us using AJAX (worst option),

connect to websocket server and wait for event there,

connect to Server Sent Events(SSE) server and wait for event (the best option)

event service is subcribed to event PostCreated so it knows when to forward that event to client via choosen protocol, i.e. SSE,

client receives event that his post was created and he can do whatever he wants with that

Profit.

Should constructors be treated as modifiers (is the creation of an object a side-effect)? I suspect not, since a constructor that created an object but didn't return a reference to it would be pretty annoying.

Side effect is everything that is not referential transparent.

It means that if your constructor does nothing else, but only returns reference to newly created object, then there is no side effect.
But if that constructor uses global values (get or set, doesn't matter) to create that object, then it's not referential transparent and it has side effect. Why?

```<code>import config

class Foo:
    self.bar = config.BAZ

foo1 = Foo()
config.BAZ = 'something different'
foo2 = Foo()

foo1 != foo2
</code>```

Body:
Actually answer to this is pretty simple, but not short.

What, if anything, should a modifier return? Should it not return an indicator of success/failure? Should it instead return nothing, but throw an exception upon failure?

First you have to ask yourself why you should not return anything from command (modifier). What we gain by that?

There is actually one advantage worth it - we can run that command asynchronously in a background, i.e. put command that creates new post to command queue, that is handled by other process, maybe written in another language.
By that you can just put command to queue and return http response immediately not waiting for post creation to happen, big performance gain, right?

Don't mistake returning value from command and returning http response. You must do the latter to inform client that putting command to queue was successful or not, but only that.
Raising exception from command is the same as returning value, so you cannot do that.

Now is the question, how to return id of the newly created post to client, because we want to let user edit that post immediately.

So, commands doesn't have return value, ok. But they have result, an effect.
In example if call turnLightOn command it doesn't return true or false to indicate that it worked - it's silly, but we can see with our own eyes that lights are on and we can see at night. That is an effect of command.
How do we represent effect in programming? By events of course.
Command that turned lights on, also publishes event LightsTurnedOn (past time) and client is subscribed to that event.

Going back to our post creation example:

client sends request to http server that he wants to create post with title 'Super post'

server receives that request and put it into command bus (queue) and returns HTTP response with code 200

process that is handling commands collects our newly put command whenever it is able to (has free workers), do what is have to do - creates post and publish event PostCreated(userId, postId)

client has to somehow subscribe for that event, and there are several ways:

query event service from time to time, to check if new event is published for us using AJAX (worst option),

connect to websocket server and wait for event there,

connect to Server Sent Events(SSE) server and wait for event (the best option)

event service is subcribed to event PostCreated so it knows when to forward that event to client via choosen protocol, i.e. SSE,

client receives event that his post was created and he can do whatever he wants with that

Profit.

Should constructors be treated as modifiers (is the creation of an object a side-effect)? I suspect not, since a constructor that created an object but didn't return a reference to it would be pretty annoying.

Side effect is everything that is not referential transparent.

It means that if your constructor does nothing else, but only returns reference to newly created object, then there is no side effect.
But if that constructor uses global values (get or set, doesn't matter) to create that object, then it's not referential transparent and it has side effect. Why?

import config

class Foo:
    self.bar = config.BAZ

foo1 = Foo()
config.BAZ = 'something different'
foo2 = Foo()

foo1 != foo2


答:
What, if anything, should a modifier return? Should it not return an indicator of success/failure? Should it instead return nothing, but throw an exception upon failure?

You will usually see void returned when an object is expected to modify its own state.  Idiomatically, this is somewhat useful -- the return value tells the developer to expect side effects in the operation.

Throwing an exception is the generally accepted practice for reporting the "failure" condition.

In event-sourced CQRS, you will sometimes see this alternative: that command methods return a (possibly empty) list of events that describe the changes in state that were made by the object in response to the command.

Should constructors be treated as modifiers (is the creation of an object a side-effect)? I suspect not, since a constructor that created an object but didn't return a reference to it would be pretty annoying.

Constructors are weird.  In many cases, the constructor call can be replaced with a call to a factory -- this doesn't really answer the question by itself, but if you can persuade yourself that the two choices should at least share the same answer, it offers some constraints.

Clearly the creation of the new object is a side effect, if the object can be thought to create itself.  But the factory doesn't necessarily change just because you have queried it for a new object.

Alternatively, you can think of each modification of an object as a change from the current state to the new state.  If you consider the constructor to be establishing the first state, then it isn't a transition between states, and shouldn't be considered a modification.  On the other hand, if you imaging all objects starting from the same "seed" state, then the constructor is the modification that takes an instance from the seed state to its first unique state.

Go not to the elves for counsel, for they will answer both no and yes.

ContentLicense: CC BY-SA 3.0

Title: "Separate Query from Modifier" and return values

Body:
I've heard it said that you should keep query and modifier functions separate from one another, and I agree with the sentiment. When trying to apply it, I often run into two issues/questions:

What, if anything, should a modifier return? Should it not return an indicator of success/failure? Should it instead return nothing, but throw an exception upon failure?

Should constructors be treated as modifiers (is the creation of an object a side-effect)? I suspect not, since a constructor that created an object but didn't return a reference to it would be pretty annoying.


问:
Development process used for the code on Apollo 11 missions?
The Apollo missions had technology no more complicated than a pocket calculator.

From link here, there's an information about Apollo Guidance Computer (AGC)

The on-board Apollo Guidance Computer (AGC) was about 1 cubic foot with 2K of 16-bit RAM and 36K of hard-wired core-rope memory with copper wires threaded or not threaded through tiny magnetic cores. The 16-bit words were generally 14 bits of data (or two op-codes), 1 sign bit, and 1 parity bit. The cycle time was 11.7 micro-seconds. Programming was done in assembly language and in an interpretive language, in reverse Polish.

So, I've stumbled upon some source code when I researched what was up there, and I've noticed great comments (eg. TEMPORARY, I HOPE HOPE HOPE)

```<code>VRTSTART    TS  WCHVERT
# Page 801
        CAF TWO     # WCHPHASE = 2 ---> VERTICAL: P65,P66,P67
        TS  WCHPHOLD
        TS  WCHPHASE
        TC  BANKCALL    # TEMPORARY, I HOPE HOPE HOPE
        CADR    STOPRATE    # TEMPORARY, I HOPE HOPE HOPE
        TC  DOWNFLAG    # PERMIT X-AXIS OVERRIDE
        ADRES   XOVINFLG
        TC  DOWNFLAG
        ADRES   REDFLAG
        TCF VERTGUID
</code>```

The actual programs in the spacecraft were stored in core rope memory, an ancient memory technology made by (literally) weaving a fabric/rope, where the bits were physical rings of ferrite material. "Core" memory is resistant to cosmic rays. The state of a core bit will not change when bombarded by radiation in Outer Space.

Virtual Apollo Guidance Computer (AGC) software is also on GITHUB!

Some part of documentation is here.

Another sample of source code with great comments.

```<code>033911,000064: 32,3017    06037        FLAGORGY        TC       INTPRET      #  DIONYSIAN FLAG WAVING

    034090,000243: 32,3241    13247        BZF      P63SPOT4               #  BRANCH IF ANTENNA ALREADY IN POSITION 1
    034091,000244:
    034092,000245: 32,3242    33254        CAF      CODE500                #  ASTRONAUT:     PLEASE CRANK THE
    034093,000246: 32,3243    04616        TC       BANKCALL               #                 SILLY THING AROUND
    034094,000247: 32,3244    20623        CADR     GOPERF1
    034095,000248: 32,3245    16001        TCF      GOTOP00H               #  TERMINATE
    034096,000249: 32,3246    13235        TCF      P63SPOT3               #  PROCEED        SEE IF HE'S LYING

    034101,000254: 32,3251    04635        TC       POSTJUMP               #  OFF TO SEE THE WIZARD ...
    034102,000255: 32,3252    74126        CADR     BURNBABY
</code>```

My question here is this:

How were the teams writing this much code able to make it functional given the tools at the time?

Because if you compile so much code that was used on Apollo 11... it'd take days, even weeks. I seriously doubt that programmers back then left everything to happen by chance.

答:
Like pretty much every other software project, it was done under aggressive deadline and quality pressure. Fortunately a large archive of material from software project manager Howard W. “Bill” Tindall, Jr. is available here.

If you sample the memos, you can get a very good sense of the normal conflict between time, features, and defects. It's worth noting that development proceeded over a number of years like the rest of the project.

Design of the system began in the second quarter of 1961, and NASA installed a Block I version in a spacecraft on September 22, 1965. Release of the original software (named CORONA) was in January 1966, with the first flight on August 25, 1966. Less than 3 years after that, designers achieved the final program objective
  (http://history.nasa.gov/computers/Ch2-5.html)

I can't find specific references to how the software was created, but given the date I can only assume that much of it was done on pen and paper, with "simulations" done manually. It seems there was a system for loading programs into the AGC with punch cards, presumably for "rapid" testing on the ground without the need to fabricate rope memory.


问:
Is it true that for Observer Pattern to work, there probably is some polling mechanism underneath?
The short question is: is it true that for Observer Pattern to work, there probably is some polling mechanism underneath? (update: in a network situation)

It was quite amazing that if on my Macbook Pro, I save a file Hello World.txt into the Dropbox folder, then I actually see it appearing on my other Macbook within 1 second or 2.

It is even more amazing that, on the other Macbook, if I do a search on Hello as a name match, and that Hello World.txt showed up there as a search result, and now I go to my Macbook Pro and rename the file as Hello World HiHi.txt, after 1 second or 2, then the new name actually showed up in the search result of the other Macbook.  That is, the search results probably observe on any folder changes, and folder observes on the Dropbox state changes remotely on the Dropbox website.

So I am guessing that the Dropbox app probably has a 2-way TCP/IP connection so that, the Dropbox server can initiate a message to my Macbook's Dropbox background app, saying "Hey, your folder content has changed on the server", and then the Dropbox background app on my Macbook will do the rename, and my folder object (the model, or the data object) will then notify the search folder object, "Hey, my content has changed", and the search folder object will then notify the folder view object, saying, "Hey, the search result has changed (and so you probably want to update the view on the screen)".

So I think this maybe roughly how it worked?  But the question is, if there was no TCP/IP two-way connection, then is it true that the Dropbox background app will have to poll the Dropbox server every 1 or 2 seconds to ask "did something changed?"  And that will be too busy for the Dropbox server.

However, isn't it true that even though there is a two-way communication set up, some where down beneath the TCP/IP layers (the 7 OSI layers), there is some kind of polling?

For example, the Dropbox server, at the very low level, cannot push an electron to my router, notifying my Macbook, because what if there is a firewall, then somehow, on my side, there must be some kind of polling at low level, or else the Dropbox server cannot push through my firewall.

Or, if viewed this way:  if I am using WiFi on my Macbook, there probably is no mechanism of Dropbox server pushing something to my wireless router, and then use electromagnetic wave to push to my Macbook, to notify me?  That is, my Macbook has to constantly ask the wireless router, "any thing new for me?"  Because if the Macbook doesn't ask -- if the Macbook crashed or if the Macbook wireless firmware crashed and is not asking, there really is no "pushing of data" from the wireless router to the Macbook.  The Macbook really has to "constantly ask".

Isn't it similar to the case that, if JavaScript on a browser had no two-way connection (before HTML5), somehow we can still have the web server "notify the browser's JavaScript code", but there will be some kind of polling underneath written in JavaScript, to simulate a two-way connection?

Update: I don't mean polling necessary to the server, but I just mean some kind of mechanism has to constantly look to see if anything new, versus if it is Observer Pattern inside of code in an app, then the CPU will just "jump" to the code to execute the observer's code, and therefore the observer really has to do nothing.  But for Observer pattern over a network, what I mean is, say, if you have your notebook computer using Wifi, and you turn off your Wifi, then the Observer Pattern will not work. Now you turn on the Wifi, but the Wifi firmware or hardware really has to constantly "sense" whether something new has come in, because Wifi signal has no way to "force into the computer", not like the Intel Processor can just "jump" to a chunk of code.  So if either firmware or hardware has to constantly "sense" whether there is something new, that's what I mean by polling -- the Observer cannot be totally waiting; some mechanism has to constantly check for something.

答:
Sure, if nobody automatically notifies you of a change, then you have to explicitly ask for an update, periodically or whenever you actually need it.

But whether you use TCP, UDP, or whatever as the underlying protocol is not important, the crucial thing is whether the communication is two-way or strictly one-way.


答:
In practice, this is often true, but only because the Observer pattern is often used as an abstraction on top of some existing polling mechanism. There's nothing stopping you from using Observers on top of non-polling mechanisms.

The typical example of an event/Observer system implemented on top of polling is keyboard/mouse events. The OS handles polling the hardware, and tells each application when it sees an even the application would be interested in.

My typical motivation for using an event/Observer system when no polling is going on is decoupling different parts of a program that shouldn't know too much about each other. For instance, if your program subscribes to new stack exchange posts, your Subscriber class may fire a NewPost event that all of your UI classes listen to. This way, the Subscriber never has to know what the UI classes are, or what events they care about. This page has some more detailed examples.

What is true at the very lowest level is that there must be some hardware or software active at all times in order to generate events. This may be as low-level as an incoming signal causing hardware interrupts, or as high-level as a "game loop" calling the AI players every frame to generate various events for the UI classes. I would not classify these two examples as "polling", though I suppose you could if you really wanted to.

Title: Is it true that for Observer Pattern to work, there probably is some polling mechanism underneath?

Body:
The short question is: is it true that for Observer Pattern to work, there probably is some polling mechanism underneath? (update: in a network situation)

It was quite amazing that if on my Macbook Pro, I save a file Hello World.txt into the Dropbox folder, then I actually see it appearing on my other Macbook within 1 second or 2.

It is even more amazing that, on the other Macbook, if I do a search on Hello as a name match, and that Hello World.txt showed up there as a search result, and now I go to my Macbook Pro and rename the file as Hello World HiHi.txt, after 1 second or 2, then the new name actually showed up in the search result of the other Macbook.  That is, the search results probably observe on any folder changes, and folder observes on the Dropbox state changes remotely on the Dropbox website.

So I am guessing that the Dropbox app probably has a 2-way TCP/IP connection so that, the Dropbox server can initiate a message to my Macbook's Dropbox background app, saying "Hey, your folder content has changed on the server", and then the Dropbox background app on my Macbook will do the rename, and my folder object (the model, or the data object) will then notify the search folder object, "Hey, my content has changed", and the search folder object will then notify the folder view object, saying, "Hey, the search result has changed (and so you probably want to update the view on the screen)".

So I think this maybe roughly how it worked?  But the question is, if there was no TCP/IP two-way connection, then is it true that the Dropbox background app will have to poll the Dropbox server every 1 or 2 seconds to ask "did something changed?"  And that will be too busy for the Dropbox server.

However, isn't it true that even though there is a two-way communication set up, some where down beneath the TCP/IP layers (the 7 OSI layers), there is some kind of polling?

For example, the Dropbox server, at the very low level, cannot push an electron to my router, notifying my Macbook, because what if there is a firewall, then somehow, on my side, there must be some kind of polling at low level, or else the Dropbox server cannot push through my firewall.

Or, if viewed this way:  if I am using WiFi on my Macbook, there probably is no mechanism of Dropbox server pushing something to my wireless router, and then use electromagnetic wave to push to my Macbook, to notify me?  That is, my Macbook has to constantly ask the wireless router, "any thing new for me?"  Because if the Macbook doesn't ask -- if the Macbook crashed or if the Macbook wireless firmware crashed and is not asking, there really is no "pushing of data" from the wireless router to the Macbook.  The Macbook really has to "constantly ask".

Isn't it similar to the case that, if JavaScript on a browser had no two-way connection (before HTML5), somehow we can still have the web server "notify the browser's JavaScript code", but there will be some kind of polling underneath written in JavaScript, to simulate a two-way connection?

Update: I don't mean polling necessary to the server, but I just mean some kind of mechanism has to constantly look to see if anything new, versus if it is Observer Pattern inside of code in an app, then the CPU will just "jump" to the code to execute the observer's code, and therefore the observer really has to do nothing.  But for Observer pattern over a network, what I mean is, say, if you have your notebook computer using Wifi, and you turn off your Wifi, then the Observer Pattern will not work. Now you turn on the Wifi, but the Wifi firmware or hardware really has to constantly "sense" whether something new has come in, because Wifi signal has no way to "force into the computer", not like the Intel Processor can just "jump" to a chunk of code.  So if either firmware or hardware has to constantly "sense" whether there is something new, that's what I mean by polling -- the Observer cannot be totally waiting; some mechanism has to constantly check for something.


问:
In general, is it worth using virtual functions to avoid branching?
There seems to be rough equivalents of instructions to equate to the cost of a branch miss virtual functions have a similar tradeoff:

instruction vs. data cache miss

optimization barrier

If you look at something like:

```<code>if (x==1) {
   p->do1();
}
else if (x==2) {
   p->do2();
}
else if (x==3) {
   p->do3();
}
...
</code>```

You could have a member function array, or if many functions depend on the same categorization, or more complex categorization exists, use virtual functions:

```<code>p->do()
</code>```

But, in general, how expensive are virtual functions vs branching
It is hard to test on enough platforms to generalize, so I was wondering if any one had a rough rule of thumb (lovely if it were as simple as 4 ifs is the breakpoint)

In general virtual functions are clearer and I would lean towards them.  But, I have several highly critical sections where I can change code from virtual functions to branches. I would prefer to have thoughts on this before I undertake this. (it's not a trivial change, or easy to test across multiple platforms)

答:
I wanted to jump in here among these already-excellent answers and admit that I've taken the ugly approach of actually working backwards to the anti-pattern of changing polymorphic code into switches or if/else branches with measured gains. But I didn't do this wholesale, only for the most critical paths. It doesn't have to be so black and white.

As a disclaimer, I work in areas like raytracing where correctness is not so difficult to achieve (and is often fuzzy and approximated anyway) while speed is often one of the most competitive qualities sought out. A reduction in render times is often one of the most common user requests, with us constantly scratching our heads and figuring out how to achieve it for the most critical measured paths.

Polymorphic Refactoring of Conditionals

First, it's worth understanding why polymorphism can be preferable from a maintainability aspect than conditional branching (switch or a bunch of if/else statements). The main benefit here is extensibility.

With polymorphic code, we can introduce a new subtype to our codebase, add instances of it to some polymorphic data structure, and have all the existing polymorphic code still work automagically with no further modifications. If you have a bunch of code scattered throughout a large codebase that resembles the form of, "If this type is 'foo', do that", you might find yourself with a horrible burden of updating 50 disparate sections of code in order to introduce a new type of thing, and still end up missing a few.

The maintainability benefits of polymorphism naturally diminish here if you just have a couple or even one section of your codebase that needs to do such type checks.

Optimization Barrier

I would suggest not looking at this from the standpoint of branching and pipelining so much, and look at it more from the compiler design mindset of optimization barriers. There are ways to improve branch prediction that apply to both cases, like sorting data based on sub-type (if it fits into a sequence).

What differs more between these two strategies is the amount of information the optimizer has in advance. A function call which is known provides a lot more information, an indirect function call which calls an unknown function at compile-time leads to an optimization barrier.

When the function being called is known, compilers can obliterate the structure and squash it down to smithereens, inlining calls, eliminating potential aliasing overhead, doing a better job at instruction/register allocation, possibly even rearranging loops and other forms of branches, generating hard-coded miniature LUTs when appropriate (something GCC 5.3 recently surprised me with a switch statement by using a hard-coded LUT of data for the results rather than a jump table).

Some of those benefits get lost when we start introducing compile-time unknowns into the mix, as with the case of an indirect function call, and that's where conditional branching can most likely offer an edge.

Memory Optimization

Take an example of a video game which consists of processing a sequence of creatures repeatedly in a tight loop. In such a case, we might have some polymorphic container like this:

```<code>vector<Creature*> creatures;
</code>```

Note: for simplicity I avoided unique_ptr here.

... where Creature is a polymorphic base type. In this case, one of the difficulties with polymorphic containers is that they often want to allocate memory for each subtype separately/individually (ex: using default throwing operator new for each individual creature).

That will often make the first prioritization for optimization (should we need it) memory-based rather than branching. One strategy here is to use a fixed allocator for each sub-type, encouraging a contiguous representation by allocating in large chunks and pooling memory for each sub-type being allocated. With such a strategy, it can definitely help to sort this creatures container by sub-type (as well as address), as that is not only possibly improving branch prediction but also improving locality of reference (allowing multiple creatures of the same subtype to be accessed from a single cache line prior to eviction).

Partial Devirtualization of Data Structures and Loops

Let's say you went through all these motions and you still desire more speed. It's worth noting that each step we venture here is degrading maintainability, and we'll already be at a somewhat metal-grinding stage with diminishing performance returns. So there needs to be a pretty significant performance demand if we tread into this territory, where we're willing to sacrifice maintainability even further for smaller and smaller performance gains.

Yet the next step to try (and always with a willingness to back out our changes if it doesn't help at all) might be manual devirtualization.

Version control tip: unless you're far more optimization-savvy than me, it can be worth creating a new branch at this point with a willingness to toss it away if our optimization efforts miss which may very well happen. For me it's all trial and error after these kinds of points even with a profiler in hand.

Nevertheless, we don't have to apply this mindset wholesale. Continuing our example, let's say this video game consists mostly of human creatures, by far. In such a case, we can devirtualize only human creatures by hoisting them out and creating a separate data structure just for them.

```<code>vector<Human> humans;               // common case
vector<Creature*> other_creatures;  // additional rare-case creatures
</code>```

This implies that all the areas in our codebase which need to process creatures need a separate special-case loop for human creatures. Yet that eliminates the dynamic dispatch overhead (or perhaps, more appropriately, optimization barrier) for humans which are, by far, the most common creature type. If these areas are large in number and we can afford it, we might do this:

```<code>vector<Human> humans;               // common case
vector<Creature*> other_creatures;  // additional rare-case creatures
vector<Creature*> creatures;        // contains humans and other creatures
</code>```

... if we can afford this, the less critical paths can stay as they are and simply process all creature types abstractly. The critical paths can process humans in one loop and other_creatures in a second loop.

We can extend this strategy as needed and potentially squeeze some gains this way, yet it's worth noting how much we're degrading maintainability in the process. Using function templates here can help to generate the code for both humans and creatures without duplicating the logic manually.

Partial Devirtualization of Classes

Something I did years ago which was really gross, and I'm not even sure it's beneficial anymore (this was in C++03 era), was partial devirtualization of a class. In that case, we were already storing a class ID with each instance for other purposes (accessed through an accessor in the base class which was non-virtual). There we did something analogical to this (my memory is a little hazy):

```<code>switch (obj->type())
{
   case id_common_type:
       static_cast<CommonType*>(obj)->non_virtual_do_something();
       break;
   ...
   default:
       obj->virtual_do_something();
       break;
}
</code>```

... where virtual_do_something was implemented to call non-virtual versions in a subclass. It's gross, I know, doing an explicit static downcast to devirtualize a function call. I have no idea how beneficial this is now as I have not tried this type of thing for years. With an exposure to data-oriented design, I found the above strategy of splitting up data structures and loops in a hot/cold fashion to be far more useful, opening up more doors for optimization strategies (and far less ugly).

Wholesale Devirtualization

I must admit that I've never gotten this far applying an optimization mindset, so I have no idea of the benefits. I have avoided indirect functions in foresight in cases where I knew there was only going to be one central set of conditionals (ex: event processing with only one central place processing events), but never started off with a polymorphic mindset and optimized all the way up to here.

Theoretically, the immediate benefits here might be a potentially smaller way of identifying a type than a virtual pointer (ex: a single byte if you can commit to the idea that there are 256 unique types or less) in addition to completely obliterating these optimization barriers.

It might also help in some cases to write easier-to-maintain code (versus the optimized manual devirtualization examples above) if you just use one central switch statement without having to split up your data structures and loops based on subtype, or if there's an order-dependency in these cases where things have to be processed in a precise order (even if that causes us to branch all over the place). This would be for cases where you don't have too many places that need to do the switch.

I would generally not recommend this even with a very performance-critical mindset unless this is reasonably easy to maintain. "Easy to maintain" would tend to hinge on two dominant factors:

Not having a real extensibility need (ex: knowing for sure that you have exactly 8 types of things to process, and never any more).

Not having many places in your code that need to check these types (ex: one central place).

... yet I recommend the above scenario in most cases and iterating towards more efficient solutions by partial devirtualization as needed. It gives you a lot more breathing room to balance extensibility and maintainability needs with performance.

Virtual Functions vs. Function Pointers

To kind of top this off, I noticed here that there was some discussion about virtual functions vs. function pointers. It is true that virtual functions require a little extra work to call, but that doesn't mean they are slower. Counter-intuitively, it may even make them faster.

It's counter-intuitive here because we're used to measuring cost in terms of instructions without paying attention to the dynamics of the memory hierarchy which tend to have a much more significant impact.

If we're comparing a class with 20 virtual functions vs. a struct which stores 20 function pointers, and both are instantiated multiple times, the memory overhead of each class instance in this case 8 bytes for the virtual pointer on 64-bit machines, while the memory overhead of the struct is 160 bytes.

The practical cost there can be a whole lot more compulsory and non-compulsory cache misses with the table of function pointers vs. the class using virtual functions (and possibly page faults at a large enough input scale). That cost tends to dwarf the slightly extra work of indexing a virtual table.

I've also dealt with legacy C codebases (older than I am) where turning such structs filled with function pointers, and instantiated numerous times, actually gave significant performance gains (over 100% improvements) by turning them into classes with virtual functions, and simply due to the massive reduction in memory use, the increased cache-friendliness, etc.

On the flip side, when comparisons become more about apples to apples, I've likewise found the opposite mindset of translating from a C++ virtual function mindset to C-style function pointer mindset to be useful in these types of scenarios:

```<code>class Functionoid
{
public:
    virtual ~Functionoid() {}
    virtual void operator()() = 0;
};
</code>```

... where the class was storing a single measly overridable function (or two if we count the virtual destructor). In those cases, it can definitely help in critical paths to turn that into this:

```<code>void (*func_ptr)(void* instance_data);
</code>```

... ideally behind a type-safe interface to hide the dangerous casts to/from void*.

In those cases where we're tempted to use a class with a single virtual function, it can quickly help to use function pointers instead. A big reason isn't even necessarily the reduced cost in calling a function pointer. It's because we no longer face the temptation to allocate each separate functionoid on the scattered regions of the heap if we're aggregating them into a persistent structure. This kind of approach can make it easier to avoid heap-associated and memory fragmentation overhead if the instance data is homogeneous, e.g., and only the behavior varies.

So there's definitely some cases where using function pointers can help, but often I've found it the other way around if we're comparing a bunch of tables of function pointers to a single vtable which only requires one pointer to be stored per class instance. That vtable will often be sitting in one or more L1 cache lines as well in tight loops.

Conclusion

So anyway, that's my little spin on this topic. I recommend venturing in these areas with caution. Trust measurements, not instinct, and given the way these optimizations often degrade maintainability, only go as far as you can afford (and a wise route would be to err on the side of maintainability).

Body:
I wanted to jump in here among these already-excellent answers and admit that I've taken the ugly approach of actually working backwards to the anti-pattern of changing polymorphic code into switches or if/else branches with measured gains. But I didn't do this wholesale, only for the most critical paths. It doesn't have to be so black and white.

As a disclaimer, I work in areas like raytracing where correctness is not so difficult to achieve (and is often fuzzy and approximated anyway) while speed is often one of the most competitive qualities sought out. A reduction in render times is often one of the most common user requests, with us constantly scratching our heads and figuring out how to achieve it for the most critical measured paths.

Polymorphic Refactoring of Conditionals

First, it's worth understanding why polymorphism can be preferable from a maintainability aspect than conditional branching (switch or a bunch of if/else statements). The main benefit here is extensibility.

With polymorphic code, we can introduce a new subtype to our codebase, add instances of it to some polymorphic data structure, and have all the existing polymorphic code still work automagically with no further modifications. If you have a bunch of code scattered throughout a large codebase that resembles the form of, "If this type is 'foo', do that", you might find yourself with a horrible burden of updating 50 disparate sections of code in order to introduce a new type of thing, and still end up missing a few.

The maintainability benefits of polymorphism naturally diminish here if you just have a couple or even one section of your codebase that needs to do such type checks.

Optimization Barrier

I would suggest not looking at this from the standpoint of branching and pipelining so much, and look at it more from the compiler design mindset of optimization barriers. There are ways to improve branch prediction that apply to both cases, like sorting data based on sub-type (if it fits into a sequence).

What differs more between these two strategies is the amount of information the optimizer has in advance. A function call which is known provides a lot more information, an indirect function call which calls an unknown function at compile-time leads to an optimization barrier.

When the function being called is known, compilers can obliterate the structure and squash it down to smithereens, inlining calls, eliminating potential aliasing overhead, doing a better job at instruction/register allocation, possibly even rearranging loops and other forms of branches, generating hard-coded miniature LUTs when appropriate (something GCC 5.3 recently surprised me with a switch statement by using a hard-coded LUT of data for the results rather than a jump table).

Some of those benefits get lost when we start introducing compile-time unknowns into the mix, as with the case of an indirect function call, and that's where conditional branching can most likely offer an edge.

Memory Optimization

Take an example of a video game which consists of processing a sequence of creatures repeatedly in a tight loop. In such a case, we might have some polymorphic container like this:

vector<Creature*> creatures;

Note: for simplicity I avoided unique_ptr here.

... where Creature is a polymorphic base type. In this case, one of the difficulties with polymorphic containers is that they often want to allocate memory for each subtype separately/individually (ex: using default throwing operator new for each individual creature).

That will often make the first prioritization for optimization (should we need it) memory-based rather than branching. One strategy here is to use a fixed allocator for each sub-type, encouraging a contiguous representation by allocating in large chunks and pooling memory for each sub-type being allocated. With such a strategy, it can definitely help to sort this creatures container by sub-type (as well as address), as that is not only possibly improving branch prediction but also improving locality of reference (allowing multiple creatures of the same subtype to be accessed from a single cache line prior to eviction).

Partial Devirtualization of Data Structures and Loops

Let's say you went through all these motions and you still desire more speed. It's worth noting that each step we venture here is degrading maintainability, and we'll already be at a somewhat metal-grinding stage with diminishing performance returns. So there needs to be a pretty significant performance demand if we tread into this territory, where we're willing to sacrifice maintainability even further for smaller and smaller performance gains.

Yet the next step to try (and always with a willingness to back out our changes if it doesn't help at all) might be manual devirtualization.

Version control tip: unless you're far more optimization-savvy than me, it can be worth creating a new branch at this point with a willingness to toss it away if our optimization efforts miss which may very well happen. For me it's all trial and error after these kinds of points even with a profiler in hand.

Nevertheless, we don't have to apply this mindset wholesale. Continuing our example, let's say this video game consists mostly of human creatures, by far. In such a case, we can devirtualize only human creatures by hoisting them out and creating a separate data structure just for them.

vector<Human> humans;               // common case
vector<Creature*> other_creatures;  // additional rare-case creatures

This implies that all the areas in our codebase which need to process creatures need a separate special-case loop for human creatures. Yet that eliminates the dynamic dispatch overhead (or perhaps, more appropriately, optimization barrier) for humans which are, by far, the most common creature type. If these areas are large in number and we can afford it, we might do this:

vector<Human> humans;               // common case
vector<Creature*> other_creatures;  // additional rare-case creatures
vector<Creature*> creatures;        // contains humans and other creatures

... if we can afford this, the less critical paths can stay as they are and simply process all creature types abstractly. The critical paths can process humans in one loop and other_creatures in a second loop.

We can extend this strategy as needed and potentially squeeze some gains this way, yet it's worth noting how much we're degrading maintainability in the process. Using function templates here can help to generate the code for both humans and creatures without duplicating the logic manually.

Partial Devirtualization of Classes

Something I did years ago which was really gross, and I'm not even sure it's beneficial anymore (this was in C++03 era), was partial devirtualization of a class. In that case, we were already storing a class ID with each instance for other purposes (accessed through an accessor in the base class which was non-virtual). There we did something analogical to this (my memory is a little hazy):

switch (obj->type())
{
   case id_common_type:
       static_cast<CommonType*>(obj)->non_virtual_do_something();
       break;
   ...
   default:
       obj->virtual_do_something();
       break;
}

... where virtual_do_something was implemented to call non-virtual versions in a subclass. It's gross, I know, doing an explicit static downcast to devirtualize a function call. I have no idea how beneficial this is now as I have not tried this type of thing for years. With an exposure to data-oriented design, I found the above strategy of splitting up data structures and loops in a hot/cold fashion to be far more useful, opening up more doors for optimization strategies (and far less ugly).

Wholesale Devirtualization

I must admit that I've never gotten this far applying an optimization mindset, so I have no idea of the benefits. I have avoided indirect functions in foresight in cases where I knew there was only going to be one central set of conditionals (ex: event processing with only one central place processing events), but never started off with a polymorphic mindset and optimized all the way up to here.

Theoretically, the immediate benefits here might be a potentially smaller way of identifying a type than a virtual pointer (ex: a single byte if you can commit to the idea that there are 256 unique types or less) in addition to completely obliterating these optimization barriers.

It might also help in some cases to write easier-to-maintain code (versus the optimized manual devirtualization examples above) if you just use one central switch statement without having to split up your data structures and loops based on subtype, or if there's an order-dependency in these cases where things have to be processed in a precise order (even if that causes us to branch all over the place). This would be for cases where you don't have too many places that need to do the switch.

I would generally not recommend this even with a very performance-critical mindset unless this is reasonably easy to maintain. "Easy to maintain" would tend to hinge on two dominant factors:

Not having a real extensibility need (ex: knowing for sure that you have exactly 8 types of things to process, and never any more).

Not having many places in your code that need to check these types (ex: one central place).

... yet I recommend the above scenario in most cases and iterating towards more efficient solutions by partial devirtualization as needed. It gives you a lot more breathing room to balance extensibility and maintainability needs with performance.

Virtual Functions vs. Function Pointers

To kind of top this off, I noticed here that there was some discussion about virtual functions vs. function pointers. It is true that virtual functions require a little extra work to call, but that doesn't mean they are slower. Counter-intuitively, it may even make them faster.

It's counter-intuitive here because we're used to measuring cost in terms of instructions without paying attention to the dynamics of the memory hierarchy which tend to have a much more significant impact.

If we're comparing a class with 20 virtual functions vs. a struct which stores 20 function pointers, and both are instantiated multiple times, the memory overhead of each class instance in this case 8 bytes for the virtual pointer on 64-bit machines, while the memory overhead of the struct is 160 bytes.

The practical cost there can be a whole lot more compulsory and non-compulsory cache misses with the table of function pointers vs. the class using virtual functions (and possibly page faults at a large enough input scale). That cost tends to dwarf the slightly extra work of indexing a virtual table.

I've also dealt with legacy C codebases (older than I am) where turning such structs filled with function pointers, and instantiated numerous times, actually gave significant performance gains (over 100% improvements) by turning them into classes with virtual functions, and simply due to the massive reduction in memory use, the increased cache-friendliness, etc.

On the flip side, when comparisons become more about apples to apples, I've likewise found the opposite mindset of translating from a C++ virtual function mindset to C-style function pointer mindset to be useful in these types of scenarios:

class Functionoid
{
public:
    virtual ~Functionoid() {}
    virtual void operator()() = 0;
};

... where the class was storing a single measly overridable function (or two if we count the virtual destructor). In those cases, it can definitely help in critical paths to turn that into this:

void (*func_ptr)(void* instance_data);

... ideally behind a type-safe interface to hide the dangerous casts to/from void*.

In those cases where we're tempted to use a class with a single virtual function, it can quickly help to use function pointers instead. A big reason isn't even necessarily the reduced cost in calling a function pointer. It's because we no longer face the temptation to allocate each separate functionoid on the scattered regions of the heap if we're aggregating them into a persistent structure. This kind of approach can make it easier to avoid heap-associated and memory fragmentation overhead if the instance data is homogeneous, e.g., and only the behavior varies.

So there's definitely some cases where using function pointers can help, but often I've found it the other way around if we're comparing a bunch of tables of function pointers to a single vtable which only requires one pointer to be stored per class instance. That vtable will often be sitting in one or more L1 cache lines as well in tight loops.

Conclusion

So anyway, that's my little spin on this topic. I recommend venturing in these areas with caution. Trust measurements, not instinct, and given the way these optimizations often degrade maintainability, only go as far as you can afford (and a wise route would be to err on the side of maintainability).


问:
Efficiently determining many-to-many subset relation
I'm doing market basket analysis. I have a set of transactions. Every transaction is a set of items that were bought. I then have a set of itemsets (i.e. a set of items) that I want to determine the support for. The support of an itemset is defined as the number of transactions of which the itemset is a subset. I'm only interested in those itemsets with a support above some threshold. For those who know, this is part of the Apriori algorithm which I'm trying to implement in C (yes, I know there are implementations available).

To clarify: a transaction is a set of items that were actually bought together by a customer. An itemset is a set of items that wasn't actually bought together but that you want to compute the support of. Say you have the transactions {Bread, Butter, Milk}; {Bread, Jam}; {Bread, Butter, Jam}. Then the support of the itemset {Bread, Butter} would be 2 (absolute) or 67%.

What I'm doing now:

Every transaction is stored as a binary search tree, and balanced once using Day-Stout-Warren.

For the itemsets with size 1, support is calculated by going through all transactions and then through the BST. This takes O(n*log(t)*1) per itemset, where t is the average size of a transaction and n the number of transactions. While computing this support, a sorted list of the matching transaction IDs is stored without additional computational effort.

The itemsets of size 2 and higher are always constructed as the union of two smaller itemsets. In this case, I simply take the sorted lists of matching transaction IDs, and compute the longest common subsequence, which in this particular case takes O(n) where n is the size of the largest list (because the lists are sorted, we can walk through them simultaneously).

Itemsets of size 2 and higher are only considered when they are the union of two smaller itemsets with sufficient support (which is the point of the Apriori algorithm).

This works, but isn't very efficient. I'm working in C, using a standard implementation of the binary search tree and a standard implementation of its exists function. Simply due to the computational complexity a lot of time is used in step 2 above: 93.53% on a typical run. I'd like to reduce this.

What could be a more efficient way to compute the support for itemsets of size 1?

The C implementation for reference:

```<code>typedef struct bs_node {
    ap_item data;
    struct bs_node* left;
    struct bs_node* right;
} bs_node;
typedef bs_node bs_tree;

bool bs_exists(bs_tree* tree, ap_item target) {
    if (tree == NULL)
        return false;
    else if (tree->data < target)
        return bs_exists(tree->right, target);
    else if (tree->data > target)
        return bs_exists(tree->left, target);
    else
        return true;
}
</code>```

答:
First, I'd recommend a better data structure for your transactions. Whereas a binary tree is preferable to a sequential list (assuming the list is more than about 10 items), it's still log(n) to find an item. Your code would execute a whole lot faster if your store receipt stored the items in a hash table rather than in a binary tree. Rather than O(n log(t)), finding all of the transactions that contain an item is O(n).

A Google search for "hashtable in C" gives multiple implementations that should work well.

Second, assuming that you're going to run the algorithm for multiple itemsets, it makes sense to pre-compute things so that you don't have to search the transactions at all. Instead, when the program starts go through all of the transactions and build a huge hash table of all items and the transactions in which they occur. So at startup you would have:

```<code>Butter: 1, 7, 9, 23, 86, 87, 92, ...
Bread: 2, 9, 33, 87, ...
Toothpaste: 8, 6, 12, 15, 43, ...
</code>```

The numbers are transaction Ids.

This is just an inverted index of the transactions. Rather than saying what items are in each transaction, it says what transaction contains the item.

Then, when somebody asks you to look for {Bread, Butter}, you quickly union those two sets.

If the inverted index is too large to fit into memory, you can easily create it in a database. Or, if you don't have a database, you use map-reduce techniques. The result is that you have the inverted index on disk and an index (surprise, another hash table) in memory. The index contains the item (i.e. Bread, Jam, etc.) and its location in the file.

Finally, a note on profiling:

Considering that the program doesn't do much at all in the single-item case but search for items in transactions, it's no surprise at all that it spends the vast majority of its time in the binary search. Even if you rewrite your code to use the hash table, it's still going to spend the vast majority of its time in the lookup code. Unless, of course, you pre-compute the single-item case as I suggested. Then all that lookup time is amortized over all of the searches.

Body:
First, I'd recommend a better data structure for your transactions. Whereas a binary tree is preferable to a sequential list (assuming the list is more than about 10 items), it's still log(n) to find an item. Your code would execute a whole lot faster if your store receipt stored the items in a hash table rather than in a binary tree. Rather than O(n log(t)), finding all of the transactions that contain an item is O(n).

A Google search for "hashtable in C" gives multiple implementations that should work well.

Second, assuming that you're going to run the algorithm for multiple itemsets, it makes sense to pre-compute things so that you don't have to search the transactions at all. Instead, when the program starts go through all of the transactions and build a huge hash table of all items and the transactions in which they occur. So at startup you would have:

Butter: 1, 7, 9, 23, 86, 87, 92, ...
Bread: 2, 9, 33, 87, ...
Toothpaste: 8, 6, 12, 15, 43, ...

The numbers are transaction Ids.

This is just an inverted index of the transactions. Rather than saying what items are in each transaction, it says what transaction contains the item.

Then, when somebody asks you to look for {Bread, Butter}, you quickly union those two sets.

If the inverted index is too large to fit into memory, you can easily create it in a database. Or, if you don't have a database, you use map-reduce techniques. The result is that you have the inverted index on disk and an index (surprise, another hash table) in memory. The index contains the item (i.e. Bread, Jam, etc.) and its location in the file.

Finally, a note on profiling:

Considering that the program doesn't do much at all in the single-item case but search for items in transactions, it's no surprise at all that it spends the vast majority of its time in the binary search. Even if you rewrite your code to use the hash table, it's still going to spend the vast majority of its time in the lookup code. Unless, of course, you pre-compute the single-item case as I suggested. Then all that lookup time is amortized over all of the searches.


问:
setSoftInputMode versus InputMethodManager
I have seen and used both of the methods to hide the on screen keyboard while coding Android:

```<code>getWindow().setSoftInputMode(WindowManager.LayoutParams.SOFT_INPUT_STATE_ALWAYS_HIDDEN);
</code>```

and:

```<code>InputMethodManager imm = (InputMethodManager) getSystemService(Context.INPUT_METHOD_SERVICE);
imm.hideSoftInputFromWindow(myEditText.getWindowToken(), 0);
</code>```

I have read through the documentation for both and I am still unclear as to how each is different, the same, which is a better method or practice to use, and which one should be used in which situations.

Here is an excerpt from the .setSoftInputMode documentation:

Specify an explicit soft input mode to use for the window, as per WindowManager.LayoutParams.softInputMode. Providing anything besides "unspecified" here will override the input mode the window would normally retrieve from its theme.

And here is an excerpt from the InputMethodManager documentation that seems to say nearly the same thing:

You can also control the preferred soft input state (open, closed, etc) for your window using the same windowSoftInputMode attribute.

More finer-grained control is available through the APIs here to
  directly interact with the IMF and its IME -- either showing or hiding
  the input area, letting the user pick an input method, etc.

So what is the difference in these two options to hide the Android soft keyboard, and does one have a benefit over the other? Is one more efficient? What are specific uses for each?

答: setSoftInputMode instructs the framework as to whether the soft input window should be shown or hidden when your activity is focused. The framework implements this behavior by calling on the methods of InputMethodManager. As alluded to in the documentation you quoted, you can exercise finer-grained control by disabling the automatic behavior (SOFT_INPUT_STATE_UNCHANGED) and then implementing the precise behavior you want with InputMethodManager.


答:
Actually, I think you've got it backwards. At the low level of the internet, there is no such thing as polling. Everything on the internet actually pushes.

Internet messages are made of packets, which are basically bunches of electrons travelling down wires. From the computer's perspective, packets simply show up either as electrons coming down a wire or electromagnetic waves being detected by the radio. Nothing ever asks for packets, they receive packets being sent along the communication medium.

When you visit a website, what happens is that you send a bunch of packets to the server, which responds by sending a bunch of packets back. We think of this as polling, because you sent the packets first instantiating the request. But at the low level of the internet, what happened was that you sent messages to the server, and the server just happened to send messages back to you later.

For example, the Dropbox server, at the very low level, cannot push an
  electron to my router, notifying my Macbook, because what if there is
  a firewall, then somehow, on my side, there must be some kind of
  polling at low level, or else the Dropbox server cannot push through
  my firewall.

No, the firewall accepts every electron in every packet that gets sent to it. It simply throws away those packets it deems suspect. In the case of dropbox, because your computer was talking to dropbox, it doesn't deem the packet suspect, and thus forwards the packets on.

That is, my Macbook has to constantly ask the wireless router, "any
  thing new for me?"

No, your Macbook doesn't ever ask if there is anything new. Rather it listens for a wireless signal from the router. The router basically assumes that the Macbook always wants the messages, and sends them out as soon they come in.

Isn't it similar to the case that, if JavaScript on a browser had no
  two-way connection (before HTML5), somehow we can still have the web
  server "notify the browser's JavaScript code", but there will be some
  kind of polling underneath written in JavaScript, to simulate a
  two-way connection?

No, javascript was restricted due to a quirk of how it was developed. The underlying internet technology has always fully supported two-way communication.


问:
Most Efficient way to Compute the sum of divisors of N (1 ≤ N ≤ 1 000 000 000)
I want to write a code which computes the sum of divisors of a number N(1 ≤ N ≤ 1 000 000 000), excluding the number itself.

However, I need an efficient way to do it. It suppose to print the answer under 1 second. I know it depends on hardware etc. But I have an i7 machine though it takes more than a second (for the worst case scenario which is 1000000000 ) because of my poor code.

So to make it more understandable:

EX:
Sample input: 10 --> Output should be: 8 (because we know 1 + 2 + 5 = 8)

EX:
Input: 20 --> Output suppose to be: 22 ( 1 + 2 + 4 + 5 + 10 = 22 )

So far i wrote this:

```<code>public class Main
{
public static void main(String[] args)
{
    @SuppressWarnings("resource")
    Scanner read = new Scanner(System.in);

    System.out.print("How many times you would like to try: ");
    int len = read.nextInt();

    for(int w = 0; w < len; w++)
    {
        System.out.print("Number: ");
        int input = read.nextInt();

        int remains = 1;
        int sum = remains;

        /* All I know we only need to check half of the given number as
           I learned ages ago. I mean (input / 2) :D */

        for(int i = 2; i <= input / 2; i++)
        {
            if(input % i == 0) sum += i;
        }

        System.out.print("Result: " + sum);
    }

}
}
</code>```

So the question is How to improve this solution ? How to make it more efficient or let me put it this way. Is there a way to do it more efficient ?

答:
You seem to be starting with code. That's completely wrong. You are solving a mathematical problem, so you need to solve the mathematics first.

Take the number 7 x 127 = 889. 7 and 127 are prime numbers. Can you think of a way to find all the divisors of 889, when I gave you the two prime factors? The answer: You get all divisors of 889 by taking one of the numbers (1, 7) and multiplying by one of the numbers (1, 127). That's four different products, including 7 x 127 = 889 which you didn't count. What's the sum of all the divisors, without finding all the divisors? Since you combined each number of (1, 7) with each number of (1, 127), the sum of all the divisors is (1 + 7) x (1 + 127) = 8 x 128 = 1024, including the 889 which you didn't count.

If you took the number 16 x 31 = 496, the possible products would be created by multiplying one of the numbers (1, 2, 4, 8, 16) by one of the numbers (1, 31), and the sum of all products is (1 + 2 + 4 + 8 + 16) x (1 + 31).

So to figure out the number of divisors of a number, you factor it into a product of distinct primes, then you figure out how to translate the maths above into code.

OwnerDisplayName: John Merih


问:
Drawbacks of a master error handler?
I'm thinking that I could simplify my life by making an exception handling class that I can just ship all exceptions to and it will handle appropriately.

Ideally:

```<code>def dostuff():
  try:
    dothis()
  except Exception e:
    Handler.handle(e)
</code>```

To avoid

```<code>def dostuff():
  try:
    dothis()
  except ValueError e:
    #handle ValueError
  except IndexError e:
    #handle IndexError
</code>```

The Handler class would have all the logic to handle exception appropriately and you could pass some arguments to specify how to handle the error, but there would be reasonable defaults to make thing easy.

This seems like either a great idea or a terrible one. Its possible that I will find it lacks the usefulness I'm looking for. But I feel like there may be some serious drawbacks that I'm not seeing. I don't think I've seen this approach often. Is there a reason?

I'm working in Python right now, but I think its not really a language specific question.

答: What you lose is context. You're passing the exception object e, but Handle.handle has no idea whether (say) IndexError happened at an input sanitiser, in which case it might trigger some kind of defensive mechanism; or whether it happened deep in an algorithm after sanitisation, in which case it could flag a problem to the developer. Because it doesn't have the context in which the exception was raised, Handle.handle wouldn't know which action to take.

Title: Drawbacks of a master error handler?


答:
The other answers already provide good theoretical arguments. I'd like to add the results of an experiment I have performed recently to estimate whether it would be a good idea to implement a virtual machine (VM) using a large switch over the op-code or rather interpret the op-code as an index into an array of function pointers. While this is not exactly the same as a virtual function call, I think it is reasonably close.

I have written a Python script to randomly generate C++14 code for a VM with an instruction set size picked randomly (albeit not uniformly, sampling the low range more densely) between 1 and 10000. The generated VM always had 128 registers and no RAM. The instructions are not meaningful and all have the following form.

inline void
op0004(machine_state& state) noexcept
{
  const auto c = word_t {0xcf2802e8d0baca1dUL};
  const auto r1 = state.registers[58];
  const auto r2 = state.registers[69];
  const auto r3 = ((r1 + c) | r2);
  state.registers[6] = r3;
}
```

<p>The script also generates dispatch routines using a <code>switch</code> statement…</p>

<pre class="lang-c++ prettyprint-override"><code>inline int
dispatch(machine_state& state, const opcode_t opcode) noexcept
{
  switch (opcode)
  {
  case 0x0000: op0000(state); return 0;
  case 0x0001: op0001(state); return 0;
  // ...
  case 0x247a: op247a(state); return 0;
  case 0x247b: op247b(state); return 0;
  default:
    return -1;  // invalid opcode
  }
}
</code>```

…and an array of function pointers.

inline int
dispatch(machine_state& state, const opcode_t opcode) noexcept
{
  typedef void (* func_type)(machine_state&);
  static const func_type table[VM_NUM_INSTRUCTIONS] = {
    op0000,
    op0001,
    // ...
    op247a,
    op247b,
  };
  if (opcode >= VM_NUM_INSTRUCTIONS)
    return -1;  // invalid opcode
  table[opcode](state);
  return 0;
}
```

Which dispatch routine was generated was chosen randomly for each generated VM.

For benchmarking, the stream of op-codes was generated by a randomly seeded (std::random_device) Mersenne twister random engine (std::mt19937_64).

The code for each VM was compiled with GCC 5.2.0 using the -DNDEBUG, -O3 and -std=c++14 switches. First, it was compiled using the -fprofile-generate option and profile data collected for simulating 1000 random instructions. The code was then re-compiled with the -fprofile-use option allowing optimizations based on the collected profile data.

The VM was then exercised (in the same process) four times for 50 000 000 cycles and the time for each run measured. The first run was discarded to eliminate cold-cache effects. The PRNG was not re-seeded between the runs so that they did not perform the same sequence of instructions.

Using this setup, 1000 data points for each dispatching routine were collected. The data was collected on a quad core AMD A8-6600K APU with 2048 KiB cache running 64 bit GNU/Linux without a graphical desktop or other programs running. Shown below is a plot of the average CPU time (with standard deviation) per instruction for each VM.

From this data, I could gain confidence that using a function table is a good idea except maybe for a very small number of op-codes. I do not have an explanation for the outliers of the switch version between 500 and 1000 instructions.

All source code for the benchmark as well as the full experimental data and a high resolution plot can be found on my website.

Title: In general, is it worth using virtual functions to avoid branching?

Body:
There seems to be rough equivalents of instructions to equate to the cost of a branch miss virtual functions have a similar tradeoff:

instruction vs. data cache miss

optimization barrier

If you look at something like:

```<code>if (x==1) {
   p->do1();
}
else if (x==2) {
   p->do2();
}
else if (x==3) {
   p->do3();
}
...
</code>```

You could have a member function array, or if many functions depend on the same categorization, or more complex categorization exists, use virtual functions:

```<code>p->do()
</code>```

But, in general, how expensive are virtual functions vs branching
It is hard to test on enough platforms to generalize, so I was wondering if any one had a rough rule of thumb (lovely if it were as simple as 4 ifs is the breakpoint)

In general virtual functions are clearer and I would lean towards them.  But, I have several highly critical sections where I can change code from virtual functions to branches. I would prefer to have thoughts on this before I undertake this. (it's not a trivial change, or easy to test across multiple platforms)

Body:
The other answers already provide good theoretical arguments. I'd like to add the results of an experiment I have performed recently to estimate whether it would be a good idea to implement a virtual machine (VM) using a large switch over the op-code or rather interpret the op-code as an index into an array of function pointers. While this is not exactly the same as a virtual function call, I think it is reasonably close.

I have written a Python script to randomly generate C++14 code for a VM with an instruction set size picked randomly (albeit not uniformly, sampling the low range more densely) between 1 and 10000. The generated VM always had 128 registers and no RAM. The instructions are not meaningful and all have the following form.

inline void
op0004(machine_state& state) noexcept
{
  const auto c = word_t {0xcf2802e8d0baca1dUL};
  const auto r1 = state.registers[58];
  const auto r2 = state.registers[69];
  const auto r3 = ((r1 + c) | r2);
  state.registers[6] = r3;
}

The script also generates dispatch routines using a switch statement…

inline int
dispatch(machine_state& state, const opcode_t opcode) noexcept
{
  switch (opcode)
  {
  case 0x0000: op0000(state); return 0;
  case 0x0001: op0001(state); return 0;
  // ...
  case 0x247a: op247a(state); return 0;
  case 0x247b: op247b(state); return 0;
  default:
    return -1;  // invalid opcode
  }
}

…and an array of function pointers.

inline int
dispatch(machine_state& state, const opcode_t opcode) noexcept
{
  typedef void (* func_type)(machine_state&);
  static const func_type table[VM_NUM_INSTRUCTIONS] = {
    op0000,
    op0001,
    // ...
    op247a,
    op247b,
  };
  if (opcode >= VM_NUM_INSTRUCTIONS)
    return -1;  // invalid opcode
  table[opcode](state);
  return 0;
}

Which dispatch routine was generated was chosen randomly for each generated VM.

For benchmarking, the stream of op-codes was generated by a randomly seeded (std::random_device) Mersenne twister random engine (std::mt19937_64).

The code for each VM was compiled with GCC 5.2.0 using the -DNDEBUG, -O3 and -std=c++14 switches. First, it was compiled using the -fprofile-generate option and profile data collected for simulating 1000 random instructions. The code was then re-compiled with the -fprofile-use option allowing optimizations based on the collected profile data.

The VM was then exercised (in the same process) four times for 50 000 000 cycles and the time for each run measured. The first run was discarded to eliminate cold-cache effects. The PRNG was not re-seeded between the runs so that they did not perform the same sequence of instructions.

Using this setup, 1000 data points for each dispatching routine were collected. The data was collected on a quad core AMD A8-6600K APU with 2048 KiB cache running 64 bit GNU/Linux without a graphical desktop or other programs running. Shown below is a plot of the average CPU time (with standard deviation) per instruction for each VM.

From this data, I could gain confidence that using a function table is a good idea except maybe for a very small number of op-codes. I do not have an explanation for the outliers of the switch version between 500 and 1000 instructions.

All source code for the benchmark as well as the full experimental data and a high resolution plot can be found on my website.


答:
My conjecture is that if you feel a need for such a helper, you're probably catching too many exceptions in the first place. What useful action can you take on an exception? You either know the cause of the error from the context (that is, you expect the exception) and can provide a reasonable fall-back strategy, then your generic handler won't help. Or you don't know what to do with the exception, then you should simply let it bubble up until somebody eventually will be able to act upon it in a reasonable way. As a last resort, you might have a “catch all” handler in your main function that says “sorry”, logs the exception and terminates the program. For many unexpected errors, this is the appropriate reaction.

Python is a little different from most other languages I know in that it encourages you to use exceptions much more as a form of ordinary control flow but I don't see how that would change the overall picture. Just as you don't have a generic helper function to handle return values, you probably don't want a generic exception handler.

Let me illustrate this with some example.

This is a reasonable use of catching an exception that can't really be automated away.

def get_foo_factor(fallback=10):
    try:
        return int(os.getenv('FOO_FACTOR'))
    except TypeError | ValueError:
        # Environment variable not set or not an integer.
        return fallback
```

<p>This one is completely ridiculous.</p>

<pre class="lang-py prettyprint-override"><code>def append_to_file(filename, message):
    try:
        with open(filename, 'a') as ostr:
            print(message, file=ostr)
    except TypeError:
        print("Not a string?", file=sys.stderr)
    except AttributeError:
        print("This should never happen!", file=sys.stderr)
</code>```

If your code has a lot of such except cascades, the best refactoring would probably be to simply delete them. For those few that remain, you'd probably find that your automated handler won't be of much use. The language already has a specialized syntax for making exception handling as convenient as possible, after all.

The only use-case for a handler I can see is if you're interfacing with a library that throws various exceptions and you want to translate them into your own scheme. In this case, something like the following might be appropriate.

def translate_libfoo_exception(exception):
    try:
        raise exception
    except FooErrorOne as e:
        return MyError(e)
    except FooErrorTwo as e:
        return MyOtherError(e)
    except FooErrorThree as e:
        return MyOtherError(e)
    except FooError as e:
        return MyGenericError(e)
    except Exception as e:
        return e
```

<p>It could then be used like this.</p>

<pre class="lang-py prettyprint-override"><code>try:
    foo.bar()
except FooErrorThree:
    # Needs special handling
    pass
except Exception as e:
    # Cannot be handled, translate and re-raise
    raise translate_libfoo_exception(e)
</code>```

Title: Drawbacks of a master error handler?

Body:
I'm thinking that I could simplify my life by making an exception handling class that I can just ship all exceptions to and it will handle appropriately.

Ideally:

```<code>def dostuff():
  try:
    dothis()
  except Exception e:
    Handler.handle(e)
</code>```

To avoid

```<code>def dostuff():
  try:
    dothis()
  except ValueError e:
    #handle ValueError
  except IndexError e:
    #handle IndexError
</code>```

The Handler class would have all the logic to handle exception appropriately and you could pass some arguments to specify how to handle the error, but there would be reasonable defaults to make thing easy.

This seems like either a great idea or a terrible one. Its possible that I will find it lacks the usefulness I'm looking for. But I feel like there may be some serious drawbacks that I'm not seeing. I don't think I've seen this approach often. Is there a reason?

I'm working in Python right now, but I think its not really a language specific question.

Body:
My conjecture is that if you feel a need for such a helper, you're probably catching too many exceptions in the first place. What useful action can you take on an exception? You either know the cause of the error from the context (that is, you expect the exception) and can provide a reasonable fall-back strategy, then your generic handler won't help. Or you don't know what to do with the exception, then you should simply let it bubble up until somebody eventually will be able to act upon it in a reasonable way. As a last resort, you might have a “catch all” handler in your main function that says “sorry”, logs the exception and terminates the program. For many unexpected errors, this is the appropriate reaction.

Python is a little different from most other languages I know in that it encourages you to use exceptions much more as a form of ordinary control flow but I don't see how that would change the overall picture. Just as you don't have a generic helper function to handle return values, you probably don't want a generic exception handler.

Let me illustrate this with some example.

This is a reasonable use of catching an exception that can't really be automated away.

def get_foo_factor(fallback=10):
    try:
        return int(os.getenv('FOO_FACTOR'))
    except TypeError | ValueError:
        # Environment variable not set or not an integer.
        return fallback

This one is completely ridiculous.

def append_to_file(filename, message):
    try:
        with open(filename, 'a') as ostr:
            print(message, file=ostr)
    except TypeError:
        print("Not a string?", file=sys.stderr)
    except AttributeError:
        print("This should never happen!", file=sys.stderr)

If your code has a lot of such except cascades, the best refactoring would probably be to simply delete them. For those few that remain, you'd probably find that your automated handler won't be of much use. The language already has a specialized syntax for making exception handling as convenient as possible, after all.

The only use-case for a handler I can see is if you're interfacing with a library that throws various exceptions and you want to translate them into your own scheme. In this case, something like the following might be appropriate.

def translate_libfoo_exception(exception):
    try:
        raise exception
    except FooErrorOne as e:
        return MyError(e)
    except FooErrorTwo as e:
        return MyOtherError(e)
    except FooErrorThree as e:
        return MyOtherError(e)
    except FooError as e:
        return MyGenericError(e)
    except Exception as e:
        return e

It could then be used like this.

try:
    foo.bar()
except FooErrorThree:
    # Needs special handling
    pass
except Exception as e:
    # Cannot be handled, translate and re-raise
    raise translate_libfoo_exception(e)


答:
Polling is a software construct and is generally inefficient for low level IO operations.

When you rename a file and the propagation goes to another linked pc, it's many forms of the observer pattern, from the click of the mouse to start the rename operation, to the changed file in the observed folder on the other pc.

Observer is a variant of publish-subscribe, and applies even at the network or data layer. See https://delog.wordpress.com/2012/09/14/applying-the-observer-pattern-for-protocol-parsing-and-handling/ for the TCP UDP addressing analogy.

Hardware is designed to generate interrupts when things happen.  Software subscribing to interrupts is also like the Observer pattern. Inside the hardware you can think of it as "constantly asking" or sensing if something important happened, so the software doesn't have to.


问:
Read K&R before learning low latency C++?
I have never really done a large amount of C programming but am in the middle of teaching myself low latency C++. Would it do more harm than good to read the K&R C programming book? I am a bit worried I will be reading about C conventions which may no longer hold in C++ (especially with C's fondness of global variables).</p>

答:
With a predominant focus on low-latency programming, probably K&R's C Programming Language is a bit basic and out-of-date. I still have fondness for that book but it's more of a piece of history now. Perhaps more useful would be to study more of those real-time fields like audio processing, game engines, data-oriented design, embedded system programming, as well as simply newer books on the C language.

I'm far from an expert on this subject as one who never faced the most demanding latency needs and never did any embedded system programming (hopefully I don't embarrass myself here), but it's an aesthetic I appreciate from seeing "average" C++ codebases in performance-critical fields end up losing their responsiveness quickly, only to reveal in a profiling session that these kinds of measurements have their hotspots distributed all over the place (leaving few glaring hotspots to optimize when trying to optimize for responsiveness rather than throughput of a single, heavy-duty operation).

A lot of the predominant focus I find in my field, at least, is throughput with a focus on optimizing bulky algorithms over homogeneous data. Much seems to be forgotten about keeping things highly responsive for all the disparate (or simply small) tasks in between. I like how Mike put it, that the constants here tend to be just as important as algorithmic complexity.

Yet throughput, latency, and deterministic response are all kind of fuzzy to me. This might be significantly oversimplified but I see latency concerns nowadays, and in the context of beefy hardware, to have to do with disparate tasks over non-homogeneous data, with a focus on throughput placing a predominant focus on uniform tasks over homogeneous data (any task which might be ideal for the GPU as one example). Techniques like data-oriented design still potentially apply towards latency demands, but with some tweaks.

One way to optimize for latency in this kind of perspective (possibly flawed) is to recognize that homogeneity and even contiguous memory layouts (and therefore often locality of reference) are often broken by disparate data types. From a machine level, data actually takes on a uniform and homogeneous data of bits and bytes (albeit with alignment requirements that vary based on the type of registers and instructions being used). The stack can store a wild mixture of data types for this reason, as it's modeled at the bits and bytes level rather than the notion of a homogeneous container of elements (and one way to improve latency is to utilize the hell out of it). Understanding this can help you reach down to the custom memory allocator level and start preallocating more disparate types of data in bulk, pooling from a common memory pool to provide requested memory chunks for lots of disparate data types.

In this sense, we're still kind of applying those bulky data-oriented design mindsets, but flattening away the disparity by reaching towards the lower-level, common denominator notion of bits and bytes in memory instead of "data type" (a meta concept which is often obliterated for the most part anyway at runtime). With that kind of adjustment, the idea of the more throughput-focused techniques and latency-focused techniques start to blur more and more together, since we're reducing the disparity in representation that can lead to hiccups in performance.

There's also other things that help like simply burning cycles more, using spin locks/CAS loops, things of that sort. A lot of these techniques are often described as evil and more likely to be misused than used properly, so it can get into a lot of taboo subjects leading to arguments with other developers (especially the most dogmatic) about what is "right". My suggestion is just go with what the measurements and responsiveness favor. Similar to how they teach artists in art school, once an artist becomes familiar and intimate with the classical rules, he is then (but only then) allowed to bend and break them.

In any case, K&R's C Programming Language is probably not the best book on this kind of subject (or even learning C in general), but there are plenty of other resources. Actually programming in C can help quite a bit. C and C++, as languages may share a lot in common with each other at the common denominator level, but there's a wildly different aesthetic along with idiomatic practices. C can make you more quickly get to the bare bits and bytes stage which is useful when improving your understanding of concepts like memory efficiency, if only for the aesthetics and idiomatic practices it promotes. The C++ aesthetic can be a wonderful complement to that in providing high-level, safe, abstract, and/or generic interfaces over these low-level optimization techniques which apply more at the implementation level.


问:
Are header-only libraries more efficient?
Assumptions

One of the advantages of header-only libraries for C++ is that they do not need to be compiled separately.

In C and C++ inline makes sense only if the function is defined in a header file*.

Traditionally, in C, .c/.h layout has been used, where the header represents the minimal public interface of the translation unit. Similarly, .cpp/hpp.

Question

Are header-only libraries generally more efficient code- and execution time wise than the traditional layout? If so, is this because of extensive inlining or other optimizations?

* - defining the function in a header allows the compiler to see the implementation during compilation of any translation unit and practically makes inlining code possible

答:
One of the advantages of header-only libraries for C++ is that they do not need to be compiled separately

No, that is not an advantage, quite the opposite - the main part of the library has to be compiled as often as it gets included, not just once. That will typically increase compile times. However, if you are referring to the advantages listed here in Wikipedia: that article is talking about decreased administrative overhead concerning the whole build, packaging and deployment process.

In C and C++ inline makes sense only if the function is defined in a header file*

This depends on the compiler/linker system, but I guess for most existing C and C++ compilers this is true.

Traditionally, in C, .c/.h layout has been used, where the header represents the minimal public interface of the translation unit. Similarly, .cpp/hpp.

That is mostly correct. C++ class headers often contain more than the minimal public interface - they typically contain also a lot of private stuff. To mitigate this, things like the PIMPL idiom are used. This is something like "the opposite" of a header-only library, it tries to minimize the necessary header content.

But to answer your main question: this is a trade-off. The more library code one puts into the header files, the more the compiler has a chance for optimizing the code for speed (if this really happens, or if the increasement is noteable, is a completely different question). On the other hand, too much code in headers increases the compile time. Especially in large C++ projects this can become a serious problem, see "Large Scale C++ Software Design" by John Lakos - though the book is a little bit outdated and some of the problems described in there are addressed by modern compilers, the general ideas/solutions are still valid.

In particular, when you are not using a stable (third party) library, but you are developing your own libs during your project, the compile times become apparent. Everytime you change something in the lib, you must change a header file, which will cause a recompile and linkage of all dependent units.

IMHO the popularity of header-only libs is caused by the popularity of template meta programming. For most compilers, templated libs must be header-only because the compiler can only start the main compile process when the type parameters are provided, and for full compilation and optimization the compiler must see "both at once" - the library code plus the template parameter values. That makes it impossible (or at least hard) to produce any "precompiled" compilation units for such a library.


问:
In TDD should I have to write Test first or Interface first?
I am learning TDD using c#, as far as I know test should drive the development, that is first write a failing test after write the bare minimum code to pass the test then do refactoring.

But it is also said that "Program to Interface, not Implementation", so write an interface first. This is where my confusion starts, If I am writing Interface first then it is violating two things

The code that is written for interface is not driven by test.

It is not the bare minimum obviously i can write it with a simple class.

Should I start by writing tests for interface also ? without any implementation what am I going to test ?

If this question sounds silly sorry for that, but I am utterly confused. May be I am taking things too literally.

答:
TDD or BDD would mean Doing your domain interfaces first and then writing tests against them by my interpretation.
an interface's implementation has an expected behavior.

it's still test before code because an interface contains no testable logic it is the structure you write a test against.

I Would Do it as follows

Write the Semi Formal Behavior (Given:When:Then:)

Write the Interface  (to host behavior encapsulating method)

Write the Test it Identifies (input the given, call the when, test the then)

Write/Alter the Concrete (Class that implements the interface) to Pass the Test


问:
Java Game. Preload objects and reuse vs create objects at runtime
I'm currently developing my first game in Java language.

The game is an endless runner with obstacles to avoid and enemies to kill. Unfortunatelly im still not experience enough and many times im just improvising. I have no idea if my method of loading of obstacles is good or bad in terms of performance (how fast a game runs, how much fps).

 Currently my code is designed in a way that I have a level class, and in level constructor I create all obstacles that will be used in that level. The obstacles are shown in the game window randomly. When an obstacle is needed, the state of obstacle is reset and the obstacle is rendered. When obstacle go offscreen rendering and update funtions are stopped. The obstacle object is still on the heap memory ready for reuse. So there is no creating objects on the fly and then removing the references.

 I have no idea if its good for how fast the game is running or not good or irrelevant? How about garbage collector is it good practise or not? I code it that way beacuse it was simplest for me because the obstacle objects are actually pretty complicated and it would be more difficult for me to code it the way of creating them on the fly randomly. And there is probably 40 or 50 objects allways ready for use? The objects are referenced through ArrayList of ArrayLists.

So is this acceptable design or complete garbage? What is your thoughts?

答:
You should implement both options, and profile your game

(More information about profiling: JVMMonitor)

The basic idea is:

-Create objects at runtime: Better use of memory (Might not be less memory used), but slower code (Less FPS).

-Preload Objects: Faster code (More FPS), but worse use of memory (Might not be more memory used, but you use that memory for more time).

IMHO, usually the best idea is to preload the objects and images. Still, you should test your case and decide what is best for your project.


答:
Object reuse is the core feature of an object pool.

If seeded while loading the level they can allow for expensive object initialsation without intoducing any performance spikes during gameplay.

Where the pool size is tuned to suit gameplay and platform requirements they usually improve realtime performance.

But remember to profile, it is important to test any asumptions.

Title: Java Game. Preload objects and reuse vs create objects at runtime

Body:
I'm currently developing my first game in Java language.

The game is an endless runner with obstacles to avoid and enemies to kill. Unfortunatelly im still not experience enough and many times im just improvising. I have no idea if my method of loading of obstacles is good or bad in terms of performance (how fast a game runs, how much fps).

 Currently my code is designed in a way that I have a level class, and in level constructor I create all obstacles that will be used in that level. The obstacles are shown in the game window randomly. When an obstacle is needed, the state of obstacle is reset and the obstacle is rendered. When obstacle go offscreen rendering and update funtions are stopped. The obstacle object is still on the heap memory ready for reuse. So there is no creating objects on the fly and then removing the references.

 I have no idea if its good for how fast the game is running or not good or irrelevant? How about garbage collector is it good practise or not? I code it that way beacuse it was simplest for me because the obstacle objects are actually pretty complicated and it would be more difficult for me to code it the way of creating them on the fly randomly. And there is probably 40 or 50 objects allways ready for use? The objects are referenced through ArrayList of ArrayLists.

So is this acceptable design or complete garbage? What is your thoughts?


问:
Best approach to develop php mysql project with multiple developers
We are 4 developers working on a PHP-mySQL web project. Currently we work together as described below:
All project files except DB is hosted at GitHub

All project files except DB is hosted at GitHub

DB is on our production server

We have local web server and php installed and working locally, connection to DB is remote. So database is in sync always.

Each developer pulls from GitHub, then pushes his changes to GitHub

Server pulls from GitHub to update the website

The problems we've faced so far is

Pulling from GitHub on the server is either manual or done by cron job. Which I don't feel it's the right choice.

Because our local machines use remote mySQL connection, we encounter time outs in executing queries.

So I don't know what would be a better approach for our situation?

答:
I think, the first - you must use some of continuous integration tools. The best of all I know is Jenkins. I think you have to make some research on this. This is not so difficult, but very useful.
Next I think that you must think about similar development environment. Docker help you with this.

Here is a link for Docker:https://docs.docker.com/get-started/
Here is a link for Jenkins: https://jenkins.io/doc/


答:
For website updates, you can set up a WebHook (Github -> Repository-Settings -> Webhooks). It can call a custom URL when a push to the repository has happened.

There's something wrong with either your application, your internet connection or MySQL Server. Your setup with a remote database is not uncommon, actually it should be able to maintain a stable connection.

--
Else: like @Evgeny said, you could use a CI Server (Jenkins, TeamCity, etc…) but it really depends on the complexity of your project. Because it's not a "no brainer" to setup such a pipeline with continuous integration.

Title: Best approach to develop php mysql project with multiple developers

Body:
We are 4 developers working on a PHP-mySQL web project. Currently we work together as described below:
All project files except DB is hosted at GitHub

All project files except DB is hosted at GitHub

DB is on our production server

We have local web server and php installed and working locally, connection to DB is remote. So database is in sync always.

Each developer pulls from GitHub, then pushes his changes to GitHub

Server pulls from GitHub to update the website

The problems we've faced so far is

Pulling from GitHub on the server is either manual or done by cron job. Which I don't feel it's the right choice.

Because our local machines use remote mySQL connection, we encounter time outs in executing queries.

So I don't know what would be a better approach for our situation?