问:
When decoupling methods is not a good idea?
It's a discussion about follow very strictly the SRP (Single Responsability Principle) vs. be more flexible when you write simple codes, like setting or getting properties values, even out of a direct context.

My first doubt is about where SRP should be included. I always read about SRP should be applied on classes. But and about methods, properties or traits (as in PHP), ...?

If this answer is no, maybe we can discard all text below.

To exemplify, let's define an example case. To be more easy understand, we'll write a singleton structure for a class called Example. Disregard IoC, for this case.

It should have two methods, basically:

getInstance(): returns an unique instance of your class; and

rebuildInstance(): it'll only destroy current instance and recreate it;

Basically to it works, getInstance() should check if the internal instance was set, create it if not, then return it. And rebuildInstance() should destroy current instance if it was defined and create a new, without return nothing.

Let's turn it more visual (in some pseudo-language):

class Example
{
    static private Example instance;

    static public Example getInstance()
    {
        if (!this.instance)
        {
            this.instance = new Example;
        }

        return this.instance;
    }

    static public void rebuildInstance()
    {
        if (this.instance)
        {
            destroy(this.instance);
        }

        this.instance = new Example;
    }
}
```

<p>So note that:</p>

<ul>
<li><code>getInstance()</code> have two responsabilities: 1. check and instantiate; 2. return.</li>
<li><code>rebuildInstance()</code> have two responsabilities: 1. check and destroy; 2. instantiate;</li>
</ul>

<p>You will see that both should instantiate, <em>so let consider it as an effect duplication</em>. To avoid that, I should decouple my methods to each one have a single responsability, then I need creates two other methods:</p>

<ul>
<li><code>buildInstanceWhenNeed()</code>;</li>
<li><code>destroyInstanceWhenNeed();</code></li>
</ul>

<p>It'll turn more clear what each methods does by name and by its structure. So our code will seems like:</p>

<pre class="lang-default prettyprint-override"><code>class Example
{
    static private Example instance;

    static private void buildInstanceWhenNeed()
    {
        if (!this.instance)
        {
            this.instance = new Example;
        }
    }

    static public Example getInstance()
    {
        self.buildInstanceWhenNeed();

        return this.instance;
    }

    static private void destroyInstanceWhenNeed()
    {
        if (this.instance)
        {
            destroy(this.instance);
        }
    }

    static public void rebuildInstance()
    {
        self.destroyInstanceWhenNeed();
        self.buildInstanceWhenNeed();
    }
}
</code>```

Okay.

Seems that now I'm respecting SRP completely on methods, very strictly. Each method have a single responsability without get out the class responsability. Except by the fact it have a singleton pattern inside of it, but I told to disregard IoC usage on start of topic.

So now I have additional questions:

Should I apply SRP on methods, decoupling it, to make sure that it have a single responsability on each method?

If yes. Should I do it even when the method don't duplicate any effect? For instance: if rebuildInstance() doesn't exists here, then I don't need decouple getInstance() because is the unique instance that will, in fact, create a new instance of class. So in this case, do I still need create buildInstanceWhenNeed() or not?

If yes, but lower priority. And about performance? I mean, when it really affect the runtime, decreasing considerably the speed of application because of ultra decoupled methods (which perhaps only affects interpreted languages), but with high specifications to each one.

答:
Single Responsibility Principle is ultimately a design principle created to ensure that "A class should have only one reason to change".  While this doesn't mean that it cannot apply to methods, it means that it is more strictly applicable to classes.

In determining the responsibility of your class or method, it is important to consider the consumer of your class or method.  Who/what will use this class and what is its intended purpose?  Is your class a business layer module designed to coordinate complex business tasks?  is your class a persistence class designed to commit data to your persistence module?  The consumers of these classes and their intended activities will often provide a hint as to whether or not your class is doing too much.

For that reason your example is not clear.  Who will use Example? What does Example do? If you haven't defined what it really does and who/what it is for, how can you say if it does too much?

Drilling down into your specific methods, I believe you're starting to touch on issues other than Single Responsibility Principle.  First off, your API is flawed as you expose all of your methods as public.  I believe destroyInstanceWhenNeeded() and buildInstanceWhenNeed() should be private.

With that resolved you can consider your individual methods.  Should buildInstanceWhenNeeded() exist?  Maybe, but that's more of a Don't Repeat Yourself problem.  Should destroyInstanceWhenNeeded() exist?  That's subjective based on a number of factors.  Does it improve the readability of rebuildInstance()?  Does it self-document some non-obvious behaviours or algorithms required in rebuildInstance()?

As you can see there's much to consider here.  To answer your questions:

Should I apply SRP on methods, decoupling it, to make sure that it have
  a single responsibility on each method?

Subjectively yes if it actually improves readability of code, if the decoupled method encapsulates logic that doesn't directly relate to the method based on its signature name, or if it helps self-document logic that may otherwise need a comment

If yes. Should I do it even when the method don't duplicate any effect? For instance: if rebuildInstance() doesn't exists here, then I
  don't need decouple getInstance() because is the unique instance that
  will, in fact, create a new instance of class. So in this case, do I
  still need create buildInstanceWhenNeed() or not?

Again, it's hard to say based on the issues with your example with regards to the consumer of your class, but I have given you reasons beyond duplication of effort for refactoring a method.

If yes, but lower priority. And about performance? I mean, when it really affect the runtime, decreasing considerably the speed of
  application because of ultra decoupled methods (which perhaps only
  affects interpreted languages), but with high specifications to each
  one.

If your method calls are so specific that you're running into performance issues with the stack, you may have made a mistake along the way.


问:
Should we avoid language features that C++ has but Java doesn't?
Suppose I am limited to use C++ by the environment in the project. Is it good to prevent the use of some language features that C++ has but Java doesn't have (e.g.: multiple inheritance, operator overloading)?

I think the reasons are:

As Java is newer than C++, if Java doesn't provide a feature that C++ has, it means that the feature is not good, so we should avoid using it.

C++ code with C++ specific features (e.g.: friend functions, multiple inheritance) can only be maintained or reviewed by C++ programmers, but if we just write C++ like Java (without C++ language specific feature), the code can be maintained or reviewed by both C++ and Java programmers.

You may be asked to convert the code to Java some day

Code without C++ specific features is usually more maintainable

Every C++ language specific feature (e.g.: multiple inheritance) should have alternatives to be implemented in Java. If it doesn't, that means the design pattern or code architecture is problematic.

Is that true?

答:
No, you should generally not write C++ like it was Java, and you should definitely not omit C++ language features that aren't present in Java.

For one thing, Java is garbage collected, and thus has no equivalent of the C++ "delete" keyword. Okay, so you implement a program without delete, because per your rules, it's not allowed.

Congratulations, you now have a memory leak ;). That's not theoretical, either - I've seen this exact situation happen in an open source game.

Similarly, Java doesn't have anything quite like C++ pointers, which rules out a lot of common function call conventions.

There are features of C++ which you should maybe avoid (see below), but "not being in Java" is not a good litmus test.

Better guidelines would be the following:

Write code that is appropriate for your language, environment, and tools.

Write code that your team can understand.

Make sure that your team is competent in the given domain.

Item (3) means that you should not have Java programmers code in C++ without training them in C++. There are some subtle but very important differences which they might not learn if they're trying to treat it like a weird dialect of Java.

Item (2) means that, if your team specifically is uncomfortable with multiple inheritance (for example), and if there's an adequate solution which doesn't use it, then it may be best to use that alternate solution. However, this depends on your team specifically. On the other hand, if your team is more uncomfortable with that alternate solution than with multiple inheritance, use multiple inheritance!

Lastly, there ARE language features of C++ that, arguably, one should avoid. If you want to know what these are, ask C++ programmers, rather than programmers of a different language. Some examples to start with are pointer arithmetic (no universal consensus) and goto.

ContentLicense: CC BY-SA 3.0


答:
I've debated whether to bother posting another answer when you already have a number that reach what seem to be entirely reasonable conclusions: that your idea is basically a disaster waiting to happen. I think, however, they've failed to point out some highly relevant reasons behind that conclusion.

The differences between Java and C++ run much deeper than the details upon which you seem to have focused.

For example, you talk about prohibiting multiple inheritance in C++. First, I'll point out that this is simply missing the point. Java defines "interfaces" as separate things from "classes". One class inherits from only one other class, but can implement an arbitrary number of interfaces.

C++ doesn't separate the two concepts that way. The closest analog C++ provides to implementing an interface in Java is inheriting from another class. As such, to keep the two aligned reasonably well at all, you probably need to use multiple inheritance in C++, but you want to restrict some of those base classes in roughly the same ways Java restricts an interface compared to a class (i.e., essentially that it specifies function signatures but, at least mostly, not implementations).

That's barely scratching the surface of the real differences between the two though. In reality, where Java code is likely to define interfaces, and classes that implement those interfaces, C++ code is a lot more likely to define some templates that will work with any class that meets its requirements (not just those defined specifically to implement the interface(s) it specifies).

If you honestly want code that's basically "Java using C++ syntax", you're almost certainly going to have to prohibit anything and everything similar to the latter though. You'll need to restrict your use of templates to roughly the level of "container of T" that Java generics support. Unfortunately, doing that will result in C++ code that's such a mess that nobody can maintain it as it exists now, not to mention the long-term possibility of translating it to some other programming language.

If you really want code that can be converted to some other language in the future, strive to make it as clean and readable as possible. The ideal is to write pseudo-code and still have it execute. Well-written modern C++ approaches that ideal much more closely than the subset you've suggested. No matter how you write it, if you ever translate to Java, you're going to have to actually translate the code, not just the syntax of the individual statements.

Title: Should we avoid language features that C++ has but Java doesn't?

Body:
Suppose I am limited to use C++ by the environment in the project. Is it good to prevent the use of some language features that C++ has but Java doesn't have (e.g.: multiple inheritance, operator overloading)?

I think the reasons are:

As Java is newer than C++, if Java doesn't provide a feature that C++ has, it means that the feature is not good, so we should avoid using it.

C++ code with C++ specific features (e.g.: friend functions, multiple inheritance) can only be maintained or reviewed by C++ programmers, but if we just write C++ like Java (without C++ language specific feature), the code can be maintained or reviewed by both C++ and Java programmers.

You may be asked to convert the code to Java some day

Code without C++ specific features is usually more maintainable

Every C++ language specific feature (e.g.: multiple inheritance) should have alternatives to be implemented in Java. If it doesn't, that means the design pattern or code architecture is problematic.

Is that true?


问:
Forking BSD project and distributing as AGPL
I am forking a BSD 2-clause licensed project, adding some AGPL-licensed files,
and re-distributing as AGPL.  The existing project comes with a LICENSE file,
and the files themselves have various copyright holders.

How do I modify the LICENSE file to make it clear that this project is distributed under the AGPL ?

Thanks!

答:
First of all, get advice from a qualified legal professional.

My (layman's) reading of 2-clause BSD license is that that would be permitted, provided that you retain the original copyright notices, and include the required disclaimer notice.

Bear in mind that when you fork someone elses codebase, their copyright is not "extinguished".  They will hold, and will continue to hold partial copyright on your codebase ... to the extent that your codebase includes parts of their code.  This is why you >>should<< talk to a lawyer.

I would also recommend talking to the original copyright holders first.  They may not care, but they might have views on whether forking is a good idea.  Generally speaking, forking a project leads to various inefficiencies ... to the detriment of both projects, and the people who use the software.


问:
REST API - Handling subresources
Suppose there is a primary resource "/accounts" which has Profile (name, national id, DOB), addresses and contacts(email, phones). I am considering them as sub-resources because they cannot exist without an account. To update them I am thinking two options

Option 1

PUT /accounts/{accountid}/address

PUT /accounts/{accountid}/contacts

PUT /accounts/{accountid}/profile

Option 2

PUT /accounts/{accountid}

(Depending on the presence/absence of address/phone/profile decide which updates to perform)

I am tempted to use option 1 because from implementation perspective each of the updates have own logic & process flow. A separate URI may keep implementation cleaner & manageable

Would it be incorrect to consider profile,address,contacts as subresources. if so, what would be an appropriate way of representing them

If they can be considered sub-resources which among the above is an appropriate option or is there a completely different option to be considered

答:
Would it be incorrect to consider profile,address,contacts as subresources.

No, anything you like can be a subresource.  REST doesn't care (a subresource is a resource), and using paths is appropriate URI design for hierarchical elements.

is there a completely different option to be considered?

You might want edits directed to finer resources though.  The PUT method is great for idempotent edits.  It's a little twitchy for partial edits - officially, the http specification "does not define how the PUT method affects the state of the origin server"; but it's widely understood that put is supposed to include a complete representation of the resource, with create or replace semantics.

For example: if you are always going to edit all three fields of the profile together, then using a single resource to modify the lot is appropriate.  On the other hand, if you want to support replacing the date of birth without bothering the other fields, then you'll surprised fewer people if you write the change to a dedicated subresource (ie: /accounts/{accountId}/profile/dob).

This is a general concern: do you update a resource by modifying it directly, or do you induce changes in it by modifying a subresource.  For instance, you seem to have multiple contacts in your definition -- it might be appropriate to accepts edits to contacts directly, or it may be that the representation of contacts is instead a reflection of the edits performed on contacts/email or contacts/phone.

Title: REST API - Handling subresources


问:
using CC-BY and GPL
I currently want to improve an application which is licensed under GPLv3. I wanted to improve it's design and Interface and use icons from google which is licensed under Creative Commons Attribution 4.0 International Public License.
On Gnu website it seems like it's compatible as I understand. But because of this line it should not be used on software. So I wanted to clarify If I can use them together

答:
The CC-BY 4.0 license is a very permissive license. The only requirement on you is that you give proper attribution where you got the icons from.

The Creative Commons licenses are designed to be used on artwork, literary works and the like. These licenses lack some considerations that are specific to software (such as the distinction between source code and (derived/compiled) binary code) that makes these licenses less suitable for use on software.

The caution "it should not be used on software" means that you should not write software and release that under on of the CC licenses, but it does not affect how you can combine artwork (such as icons) that is already under a CC license with a separate software product under a different license.

In particular, if you modify an existing application that is under the GPL license, then the software will always remain under the GPL license. Using icons that are licensed with CC-BY does not change that. The CC-BY license is also permissive enough that you can release the modified program with the new icons entirely under the GPL license.

Title: using CC-BY and GPL


问:
2D Animating in OpenGL
I am trying to animate a texture in OpenGL. I feel like it should be easy, as I know how to animate in SDL and other libraries but I am having trouble. I have a x,y,w,h that holds the texture coordinates for the frame I want. My vertex data has a x,y for the vertex, and a x,y for the texture coordinates. I know how to change these, but the problem is the width and height. How do I change these in a spritesheet? All I see is 2d vectors with no width or height. Am I overlooking something? Should I be doing this in the VBO, or in the vertex shader? I looked through several other related questions but they were all looking for what is the best strategy, or most optimizes or a unrelated to this animation question.

Edit I realized I was looking at this the wrong way. I can show a portion of the texture, then decided which portion to look at, in the vertex shader very easily.

答:
You are indeed on the right mental path.

When you draw geometry, the vertex shader runs separately on each vertex of the geometry and transforms the positions into clip space, as well as letting you manipulate any associated attributes.

The fragment shader is invoked on each pixel that is considered to be covered by the geometry, and the attributes of the three vertices are interpolated across the triangle.

If you want to alter the position of your sprite in the view, you would alter the position attribute, either by editing the vertex buffer, or editing parameters to your shader with which you can offset the geometry in space. Similarly, you can alter the texture coordinates to decide which region of the texture you're interested in sampling from.

If you only draw one or a few sprites, what you do doesn't matter overly much performance-wise.

If you're interested in drawing them in bulk, you will run into several bottlenecks:

recalculating and storing the vertices of all your sprites has a CPU and memory cost;

uploading the geometry has a bandwidth and latency hit;

each draw call has a lot of overhead in the driver.

A common solution to this is instancing. Instead of binding a single set of vertex buffers that contain the vertex attributes for a mesh, you bind two sets.

One of the sets will be a set of vertices, describing a single piece of geometry.
The other set represents distinct instances of that master geometry, containing per-instance attributes.

If you previously had vertex attributes pos and tc in a configuration like this:

```<code>vertices =
// position                        texcoord
{ {corner                        , tc_corner}
, {corner + float2(width, 0)     , tc_corner + float2(sprite_width, 0)            }
, {corner + float2(width, height), tc_corner + float2(sprite_width, sprite_height)}
, {corner + float2(0, height)    , tc_corner + float2(0, sprite_height)           } }
</code>```

you would with instances be able to separate out the things that vary per instance, leaving you with a vertex buffer that can be reused across multiple sprites:

```<code>pervertex =
// position      texcoord
{ {float2(0, 0), float2(0, 0)}
, {float2(1, 0), float2(1, 0)}
, {float2(1, 1), float2(1, 1)}
, {float2(0, 1), float2(0, 1)} }

perinstance =
// world_pos  world_size             texture_pos  texture_size
{ {corner,    float2(width, height), tc_corner,   float2(sprite_width, sprite_height)}
, another_instance
, more_instances... }
</code>```

From the perspective of the vertex shader, it's going to look just like a bunch of attributes, the distribution of per-vertex and per-instance attributes are done by the input assembly beforehand.

Your vertex shader would simply do the math along the lines of:

```<code>out_tc = texture_pos + tc * texture_size;
gl_Position = float4(world_pos + pos * world_size, 0.0, 1.0);
</code>```

Note that instancing is not necessary to do this kind of thing, you can emulate it reasonably well by multiple draw calls and passing the per-instance parameters in uniform variables to your shader program.

Title: 2D Animating in OpenGL

Body:
You are indeed on the right mental path.

When you draw geometry, the vertex shader runs separately on each vertex of the geometry and transforms the positions into clip space, as well as letting you manipulate any associated attributes.

The fragment shader is invoked on each pixel that is considered to be covered by the geometry, and the attributes of the three vertices are interpolated across the triangle.

If you want to alter the position of your sprite in the view, you would alter the position attribute, either by editing the vertex buffer, or editing parameters to your shader with which you can offset the geometry in space. Similarly, you can alter the texture coordinates to decide which region of the texture you're interested in sampling from.

If you only draw one or a few sprites, what you do doesn't matter overly much performance-wise.

If you're interested in drawing them in bulk, you will run into several bottlenecks:

recalculating and storing the vertices of all your sprites has a CPU and memory cost;

uploading the geometry has a bandwidth and latency hit;

each draw call has a lot of overhead in the driver.

A common solution to this is instancing. Instead of binding a single set of vertex buffers that contain the vertex attributes for a mesh, you bind two sets.

One of the sets will be a set of vertices, describing a single piece of geometry.
The other set represents distinct instances of that master geometry, containing per-instance attributes.

If you previously had vertex attributes pos and tc in a configuration like this:

vertices =
// position                        texcoord
{ {corner                        , tc_corner}
, {corner + float2(width, 0)     , tc_corner + float2(sprite_width, 0)            }
, {corner + float2(width, height), tc_corner + float2(sprite_width, sprite_height)}
, {corner + float2(0, height)    , tc_corner + float2(0, sprite_height)           } }

you would with instances be able to separate out the things that vary per instance, leaving you with a vertex buffer that can be reused across multiple sprites:

pervertex =
// position      texcoord
{ {float2(0, 0), float2(0, 0)}
, {float2(1, 0), float2(1, 0)}
, {float2(1, 1), float2(1, 1)}
, {float2(0, 1), float2(0, 1)} }

perinstance =
// world_pos  world_size             texture_pos  texture_size
{ {corner,    float2(width, height), tc_corner,   float2(sprite_width, sprite_height)}
, another_instance
, more_instances... }

From the perspective of the vertex shader, it's going to look just like a bunch of attributes, the distribution of per-vertex and per-instance attributes are done by the input assembly beforehand.

Your vertex shader would simply do the math along the lines of:

out_tc = texture_pos + tc * texture_size;
gl_Position = float4(world_pos + pos * world_size, 0.0, 1.0);

Note that instancing is not necessary to do this kind of thing, you can emulate it reasonably well by multiple draw calls and passing the per-instance parameters in uniform variables to your shader program.


答:
Allow me to go Zen on you:

Please do not incur in analysis paralysis.

By that reasoning you could as well call your method destroyInstanceWhenNeededButOnlyIfYouCallMe().

A single line of common code between two methods, specially an instantiation is not an itch that needs to be scratched. SRP doesn't go into that minutiae. It's about classes doing too much or having more than one reason to change.

The original design is OK. The second design is too much thinking for so little gain.

Title: When decoupling methods is not a good idea?

Body:
It's a discussion about follow very strictly the SRP (Single Responsability Principle) vs. be more flexible when you write simple codes, like setting or getting properties values, even out of a direct context.

My first doubt is about where SRP should be included. I always read about SRP should be applied on classes. But and about methods, properties or traits (as in PHP), ...?

If this answer is no, maybe we can discard all text below.

To exemplify, let's define an example case. To be more easy understand, we'll write a singleton structure for a class called Example. Disregard IoC, for this case.

It should have two methods, basically:

getInstance(): returns an unique instance of your class; and

rebuildInstance(): it'll only destroy current instance and recreate it;

Basically to it works, getInstance() should check if the internal instance was set, create it if not, then return it. And rebuildInstance() should destroy current instance if it was defined and create a new, without return nothing.

Let's turn it more visual (in some pseudo-language):

class Example
{
    static private Example instance;

    static public Example getInstance()
    {
        if (!this.instance)
        {
            this.instance = new Example;
        }

        return this.instance;
    }

    static public void rebuildInstance()
    {
        if (this.instance)
        {
            destroy(this.instance);
        }

        this.instance = new Example;
    }
}
```

<p>So note that:</p>

<ul>
<li><code>getInstance()</code> have two responsabilities: 1. check and instantiate; 2. return.</li>
<li><code>rebuildInstance()</code> have two responsabilities: 1. check and destroy; 2. instantiate;</li>
</ul>

<p>You will see that both should instantiate, <em>so let consider it as an effect duplication</em>. To avoid that, I should decouple my methods to each one have a single responsability, then I need creates two other methods:</p>

<ul>
<li><code>buildInstanceWhenNeed()</code>;</li>
<li><code>destroyInstanceWhenNeed();</code></li>
</ul>

<p>It'll turn more clear what each methods does by name and by its structure. So our code will seems like:</p>

<pre class="lang-default prettyprint-override"><code>class Example
{
    static private Example instance;

    static private void buildInstanceWhenNeed()
    {
        if (!this.instance)
        {
            this.instance = new Example;
        }
    }

    static public Example getInstance()
    {
        self.buildInstanceWhenNeed();

        return this.instance;
    }

    static private void destroyInstanceWhenNeed()
    {
        if (this.instance)
        {
            destroy(this.instance);
        }
    }

    static public void rebuildInstance()
    {
        self.destroyInstanceWhenNeed();
        self.buildInstanceWhenNeed();
    }
}
</code>```

Okay.

Seems that now I'm respecting SRP completely on methods, very strictly. Each method have a single responsability without get out the class responsability. Except by the fact it have a singleton pattern inside of it, but I told to disregard IoC usage on start of topic.

So now I have additional questions:

Should I apply SRP on methods, decoupling it, to make sure that it have a single responsability on each method?

If yes. Should I do it even when the method don't duplicate any effect? For instance: if rebuildInstance() doesn't exists here, then I don't need decouple getInstance() because is the unique instance that will, in fact, create a new instance of class. So in this case, do I still need create buildInstanceWhenNeed() or not?

If yes, but lower priority. And about performance? I mean, when it really affect the runtime, decreasing considerably the speed of application because of ultra decoupled methods (which perhaps only affects interpreted languages), but with high specifications to each one.


问:
Is the algorithm more important than the programming language?
During the current (2013) Google Code Jam contest, there was a problem that took C++ and Java people 200+ lines of code as compared to Python people that solved the same problem only using 40 lines of code.

Python is not directly comparable with C++ and Java but the difference in verbosity I thought might perhaps have an influence on the efficiency of the algorithm.

How important is knowing the right algorithm compared to the choice of language? Could an excellently implemented Python program be implemented in C++ or Java in a better way (using the same algorithm) and does this have any relation to the natural verbosity of certain programming languages?

答:
While extrapolating the "40 LoC vs 200 LoC" example, saying "well, only a fifth of the total LoC is obviously faster to write so it must be better" may seem tempting, I really think there is little truth to be found there.

Optimizing for fewest LoC is almost never a good idea in my opinion. Yes, every LoC written is a potential for bugs, and you never have to debug what you never wrote etcetc. The point is, optimize for readability and decoupledness. It does not matter if you solve a problem using a 20 lines big regex, as opposed to writing a module of 1k LoC. The regex will be an opaque wall, extremely prone to bugs, hard to understand, nightmarish to alter without changing it's behavior in unoredicatble ways etc.

Getting rid of boilerplate and verbosity that doesn't add any value is all well and good, but on the other hand, using something like Java or C#, having knowledge about design patterns and tools like resharper allow you SO much flexibility in refactoring the code, cleaning it up over time, breaking things down, etc., that would simply be MUCH harder had you written it as a much smaller python script or ruby app.

A very telling comparison:
I would rather have 100k LoC of test-covered decoupled C# code, filled with "overkill" stuff like strategy pattern, factories etc, rather than a 20k python app that just "gets stuff done". 5 times more code or not, the architecture wins every day.

I fully agree some kinds of work are easier and more convenient in some languages, but I believe more in choosing your language based on what tools you need and what the requirements are (and might be in the near future).


问:
Is a TCP client/server a good solution for a system which can be controlled by a GUI running on multiple platforms?
Consider a software which runs on a dedicated system (basically a Linux box), and controls some machinery. The system has all the required hardware interfaces for the task. The software also has a GUI for controlling said machinery.

However, this system should later be able to be used from external devices, so other software, using the same or similar GUI will be running on various PCs, smartphones, etc. The central SW will contain a TCP server through which the external devices will connect. The central device with its software is to be developed first, and the controlability from external devices will be a later add-on.

My proposal for the "central" software would be to split it into two separate programs. One would be a server without any GUI at all, which will handle all the controlling and hardware interfacing stuff, and a separate software would fulfill solely the roles of the GUI.

Of course, a good MVC separation is possible even within a single program, but the main advantage I see for splitting the program is that the GUI part will be much more easily portable to other systems. Actually, in case of a PC as an external device, the same software can be re-used 100%. For other systems, I guess it would be much easier to port a program 1 to 1, than to port only the GUI part of a more complex program.

The main disadvantage of this solution is that the initial system will be a more complicated, as the GUI needs to communicate via TCP with the controlling part, instead of directly inside the same program.

Are there other major advantages and disadvantages of splitting the application in two separate programs communicating over TCP?

The scope of the project is up to 2 years for a small, 2-3 person team.

答:
Your question might be too broad and is quite abstract (not enough implementation details are given: which operating system? which hardware: an Arduino like embedded processor is not the same as an ARM motherboard running some real-time variant of Linux; what machinery ? : a sewing machine is not the same as a nuclear power plant). Also, what kind of users? Are they all internal to a corporation or can they be "general public"?

However, did you consider making your software some specialized web server, using some HTTP server library like libonion or perhaps Wt ? (Of course HTTP is above TCP); then you would still split your software in two parts: one providing HTTP service, the other being some code (e.g. Javascript, AJAX, ...) running in browsers.

Then your GUI part would be mostly replaced by browsers (on variety of devices: desktops, smartphones, etc...) speaking HTTP (or HTTPS).

Of course, if you design your application as a specialized web server, you should take very early into account all the implications (notably security, DDOS attacks, ...) and you should be aware of web technologies. Of course, things could be different if all users are inside some small corporate intranet (then, you probably care a bit less about DDOS)

But on the client side, most end-user devices (desktops or laptops running Windows, Linux, MacOSX, tablets, smartphones) -those with a display at least- have already sophisticated HTML5 browsers, on which you could leverage.

You might consider other approaches, e.g. remote procedure calls using JSONRPC, using FastCGI to interface to some external existing web server, etc....

If you split your software in several parts, you need to define how they communicate and synchronize, and that is probably operating system and implementation specific (but you'l need some inter-process communication probably provided by some operating system, but you might have some unikernel approach). Without telling more about your software & system & machinery (i.e. physical purpose), you won't get really meaningful help.


答:
Yes, this separation of layers is always a good thing, even if its not going to be written for a distributed system. Many systems work by making UI or DB components as isolated as possible to allow for easier maintenance or replacement in the future.

Other system write the main program as a command line app and then write a GUI to drive that. That allows it to be scripted and used in a UI environment too. Its not excessively complicated to do this, you just write your program and then write a UI program later.

For your needs, I'd recommend an embedded HTTP webserver. There are many and they're very lightweight. You lose nothing by using one instead of a custom TCP server (as you still need to implement security etc) but gain all the standard features of the protocol. You also gain the ability to write your client in the server as a HTML website and control it via a web browser.

Title: Is a TCP client/server a good solution for a system which can be controlled by a GUI running on multiple platforms?

Body:
Consider a software which runs on a dedicated system (basically a Linux box), and controls some machinery. The system has all the required hardware interfaces for the task. The software also has a GUI for controlling said machinery.

However, this system should later be able to be used from external devices, so other software, using the same or similar GUI will be running on various PCs, smartphones, etc. The central SW will contain a TCP server through which the external devices will connect. The central device with its software is to be developed first, and the controlability from external devices will be a later add-on.

My proposal for the "central" software would be to split it into two separate programs. One would be a server without any GUI at all, which will handle all the controlling and hardware interfacing stuff, and a separate software would fulfill solely the roles of the GUI.

Of course, a good MVC separation is possible even within a single program, but the main advantage I see for splitting the program is that the GUI part will be much more easily portable to other systems. Actually, in case of a PC as an external device, the same software can be re-used 100%. For other systems, I guess it would be much easier to port a program 1 to 1, than to port only the GUI part of a more complex program.

The main disadvantage of this solution is that the initial system will be a more complicated, as the GUI needs to communicate via TCP with the controlling part, instead of directly inside the same program.

Are there other major advantages and disadvantages of splitting the application in two separate programs communicating over TCP?

The scope of the project is up to 2 years for a small, 2-3 person team.


答:
A few pitfalls you need to be aware of:

Authentication. Unless you want anyone in the world to operate your machine, you need to make sure only authorized people connect to your hardware controller. A good way to do this might be to use TLS with client- and server certificate. As a bonus this also gives you encryption to protect from eavesdroppers and man-in-the-middle attackers.

The client can not be trusted. You can not assume that the user (authorized or unauthorized) will use your unmodified client software. So make sure your protocol implementation on the server does not allow anything the user interface is not supposed to allow. All client-inputs need to be sanity-checked.

TCP/IP is not very real-time capable. Packets can be suddenly delayed for a whole bunch of reasons. UDP/IP has slightly better real-time capabilities, but has a whole bunch of even nastier surprises, like packets arriving in the wrong order or not at all. When it is important for your use-case that the machine executes commands with exact timing, you can not assume that the commands arrive at the server with the same time-differences between them as they left the client. For example: If you want the machine to run for exactly 5.637 seconds, then sending the on-message and 5.637 seconds later sending the off-message might not work because both messages will need a different time to arrive at the server. You would need to send a "Switch on, wait 5.637 seconds, switch off, now do it" command in one message.


问:
Common header file for C++ and JavaScipt
I have an app that runs a C++ server backend and Javascript on the client.  I would like to define certain strings once only, for both pieces of code.  For example, I might have a CSS class "row-hover" - I want to define this class name in one place only in case I change it later.

Is there an easy way to include, or read, some sort of common definitions file into both C++ and JavaScript?  Ideally as a compile / preprocessing step but any neat approach good.

答:
You can use macros in C/C++ to change identifiers as needed.  For javascript the obvious candidate is var since it is only needed in javascript ... it can be redefined as necessary to any c type.

c-header.h

```<code>//constant integers
#define var const int
#include "int-consts.js"
#undef var

//constant strings
#define var const char*
#include "string-consts.js"
#undef var

//arrays of strings
#define var const char**
#define new (const char*[])
#define Array(x,...)  {x,__VA_ARGS__}
#include "string-arrays.js"
#undef new
#undef var

//and so forth for other types
</code>```

int-consts.js

```<code>var swapMagicOffset = 4086;
</code>```

string-consts.js

```<code>var swapMagic = "SWAP-SPACE";
var swapMagic2 = "SWAPSPACE2";
</code>```

string-arrays.js

```<code>var cars = new Array("Ford", "Chevy", "Dodge");
</code>```

Body:
You can use macros in C/C++ to change identifiers as needed.  For javascript the obvious candidate is var since it is only needed in javascript ... it can be redefined as necessary to any c type.

c-header.h

//constant integers
#define var const int
#include "int-consts.js"
#undef var

//constant strings
#define var const char*
#include "string-consts.js"
#undef var

//arrays of strings
#define var const char**
#define new (const char*[])
#define Array(x,...)  {x,__VA_ARGS__}
#include "string-arrays.js"
#undef new
#undef var

//and so forth for other types

int-consts.js

var swapMagicOffset = 4086;

string-consts.js

var swapMagic = "SWAP-SPACE";
var swapMagic2 = "SWAPSPACE2";

string-arrays.js

var cars = new Array("Ford", "Chevy", "Dodge");


问:
Why Garbage Collection if smart pointers are there
These days, so many languages are garbage collected. It is even available for C++ by third parties. But C++ has RAII and smart pointers. So what's the point of using garbage collection? Is it doing something extra?

And in other languages like C#, if all the references are treated as smart pointers(keeping RAII aside), by specification and by implementation, will there still be any need of garbage collectors? If no, then why is this not so?

答:
The term garbage collection implies there is any garbage to collect. In C++ smart pointers come in multiple flavors, most importantly the unique_ptr. The unique_ptr is basically a single ownership and scoping construct. In a well designed piece of code most heap allocated stuff would normally reside behind unique_ptr smart pointers and ownership of those resources will be well defined at all times. There is hardly any overhead in unique_ptr and unique_ptr takes away most of the manual memory management problems that traditionally drove people to managed languages. Now that more cores running concurrently are becoming more common good, the design principles that drive code to use unique and well defined ownership at any point in time becomes more important for performance. The use of the actor model of computation allows for the construction of programs with a minimum amount of shared state between threads, and unique ownership plays a major role in making high performance systems make efficient use of many cores without the overhead of shared-between-threads data and the implied mutex requirements.

Even in a well designed program, especially in multi threaded environments, not everything can be expressed without shared data structures, and for those data structures that truly require, threads need to communicate. RAII in c++ works pretty well for lifetime concerns in a single threaded setup, in a multi threaded setup the lifetime of objects may not be completely stack hierarchically defined. For these situations, the use of shared_ptr offers a big part of the solution.  You create shared ownership of a resource and this in C++ is the only place we see garbage, but at such small quantities that a proper designed c++ program should be considered more to implement 'litter' collection with shared-ptr's than full fledged garbage collection as implemented in other languages. C++ simply does not have that much 'garbage' to collect.

As stated by others, reference counted smart pointers are one form of garbage collection, and a for that has one major issue. The example that is used mostly as drawback of reference counted forms of garbage collection is the problem with the creation of orphaned data structures connected with smart pointers to each other that create object clusters that keep each other from being collected. While in a program designed according to the actor-model of computation, the data structures not usually allow for such noncollectable clusters to arise in C++, when you use the broad shared data approach to multi threaded programming, as is used predominantly in a large part of the industry, these orphaned clusters can quickly become a reality.

So to sum it all up, if by shared pointer usage you mean the wide use of unique_ptr combined with the actor model of computation approach for multi threaded programming and the limited use of shared_ptr, than other forms of garbage collection don't buy you any added benefits. If however a shared everything approach would have you end up with shared_ptr all over the place, than you should consider either switching concurrency models or switching to a managed language that is more geared towards the wider sharing of ownership and concurrent access to data structures.

ContentLicense: CC BY-SA 2.5


问:
Should method comments be written as if everything will work?
If I have a method to look for a valid email address in an array, that could either:

Return the email address

Return false if an email wasn't found.

Throw an exception if an email was found, but it wasn't valid.

Should the method comment be written as though it would always work, or should it indicate that it might fail? For example:

```<code>/**
 * Attempts to return the email address from $data.
 * @param Array $data
 * @return Boolean
 * @throws My_Custom_Exception_InvalidEmailAddress
 */
</code>```

vs

```<code>/**
 * Returns the email address in the $data.
 * @param Array $data
 * @return Boolean
 * @throws My_Custom_Exception_InvalidEmailAddress
 *      Throw if an email key is found in $data, but the value is empty
 *      or if the value doesn't pass the Zend_Validate_Email check.
 */
</code>```

Generally, I go by the rule of, if I've wrote some "throw new..." in the method, I'll write it more like the first example, because I know there's a good chance that the method won't do what the comment says, but, you can tell that by the @throws annotation, so it seems a bit unnecessary.

Is either comment style "prefered", or is it more just a case of be consistent throughout the project?

答:
"Throws InvalidEmailAddress if an invalid email address is found" seems somewhat redundant.

Commenting what constitutes "invalid" would be a better use of the comment block, but otherwise comment the minimum necessary to understand what is intended, the more documentation you put in the more maintenance it will require as code changes.


答:
Throwing an exception, by definition, means that the method doesn't return whatever it's supposed to be returning. If you have a @throws annotation (and you should, always!), then the fact that the promised return value might not happen is completely obvious. There is nothing gained by complicating the documentation to say "Tries to return" or anything like that.

What you should always write is, as exactly as possible, under which circumstances this will happen. it doesn't matter if the actual throw statement is in your code or in third-party code, the only difference is that you're usually a bit more certain of the exact circumstances if it's in your own code.

Title: Should method comments be written as if everything will work?

Body:
If I have a method to look for a valid email address in an array, that could either:

Return the email address

Return false if an email wasn't found.

Throw an exception if an email was found, but it wasn't valid.

Should the method comment be written as though it would always work, or should it indicate that it might fail? For example:

```<code>/**
 * Attempts to return the email address from $data.
 * @param Array $data
 * @return Boolean
 * @throws My_Custom_Exception_InvalidEmailAddress
 */
</code>```

vs

```<code>/**
 * Returns the email address in the $data.
 * @param Array $data
 * @return Boolean
 * @throws My_Custom_Exception_InvalidEmailAddress
 *      Throw if an email key is found in $data, but the value is empty
 *      or if the value doesn't pass the Zend_Validate_Email check.
 */
</code>```

Generally, I go by the rule of, if I've wrote some "throw new..." in the method, I'll write it more like the first example, because I know there's a good chance that the method won't do what the comment says, but, you can tell that by the @throws annotation, so it seems a bit unnecessary.

Is either comment style "prefered", or is it more just a case of be consistent throughout the project?


答:
It is, as always, mostly about keeping it consistent.

One could argue that if you throw a checked FoundInvalidEmailAddressException you won't even need the throw because it provides duplicated information that the compiler will tell you anyway, but I'd write it like this:

```<code>/**
 * Returns the email address in the $data.
 * @param  Array $data //what format do you expect data to be?
 *         eg. [username, password]? [username, email]?
 * @return Boolean true if the email is found and valid, false otherwise
 * @throws InvalidEmailAddressException If there is an email for $data,
 *         but it fails validation. An email is invalid if...
 */
</code>```

Body:
It is, as always, mostly about keeping it consistent.

One could argue that if you throw a checked FoundInvalidEmailAddressException you won't even need the throw because it provides duplicated information that the compiler will tell you anyway, but I'd write it like this:

/**
 * Returns the email address in the $data.
 * @param  Array $data //what format do you expect data to be?
 *         eg. [username, password]? [username, email]?
 * @return Boolean true if the email is found and valid, false otherwise
 * @throws InvalidEmailAddressException If there is an email for $data,
 *         but it fails validation. An email is invalid if...
 */


问:
What are polymorphic messages?
I am reading Martin Fowler's 'Refactoring: Improving the Design of Existing Code'. I have not understood a section of the second chapter where Kent Beck describes the pros of indirection.

One of the pros listed is Encoding Conditional Logic and it is described as:

Objects have a fabulous mechanism, polymorphic messages, to flexibly
  but clearly express conditional logic. By changing explicit
  conditionals to messages, you can often reduce duplication, add
  clarity, and increase flexibility all at the same time.

What are polymorphic messages? What does he mean by 'change explicit conditionals to messages'?

答:
Another term would be "dynamic dispatch".  Below is an example using (somewhat non-idiomatic) C#.

Say we have the following list type:

public abstract class List<T> {
    public abstract bool isEmpty();
    public abstract T head();
    public abstract List<T> tail();
}

class Empty<T> : List<T> {
    public Empty() { }
    public override bool isEmpty() { return true; }
    public override T head() {
        throw new ApplicationException("head of empty list");
    }
    public override List<T> tail() {
        throw new ApplicationException("tail of empty list");
    }
}

class Cons<T> : List<T> {
    private T head;
    private List<T> tail;
    public Cons(T head, List<T> tail) {
        this.head = head;
        this.tail = tail;
    }
    public override bool isEmpty() { return false; }
    public override T head() { return head;  }
    public override List<T> tail() { return tail; }
}
```

<p>You could implement a length function as follows:</p>

<pre class="lang-cs prettyprint-override"><code>int length(List<T> xs) {
    if(xs.isEmpty()) {
        return 0;
    } else {
        return 1 + length(xs.tail());
    }
}
</code>```

Alternatively, using dynamic dispatch as Kent Beck suggests, we could eliminate the conditional by adding length as a method of List<T>.

public abstract class List<T> {
    /* ... */
    public abstract int length();
}

class Empty<T> : List<T> {
    /* ... */
    public override int length() {
        return 0;
    }
}

class Cons<T> : List<T> {
    /* ... */
    public override int length() {
         return 1 + tail.length();
    }
}
```

<p>This increases flexibility because we could now add another class with it's own implementation of <code>length</code> that can be smarter than our <code>length</code> function.</p>

<pre class="lang-cs prettyprint-override"><code>class Concat<T> : List<T> {
    private List<T> left, right;
    public Concat(List<T> left, List<T> right) {
        this.left = left;
        this.right = right;
    }
    /* ... */
    public override int length() {
        return left.length() + right.length();
    }
}
</code>```

As a bit of an aside, there are problems with the definition of length in Cons<T> that will cause it to use a lot of stack space and likely produce a stack overflow in real use.  The problem is that it isn't written in a tail recursive form.  That's easily corrected, but even if corrected, since virtually no OO language guarantees tail call elimination the benefits that Kent Beck mentions can't be achieved in this example.  In other words, the lack of tail call elimination significantly hampers one of the main benefits of OO style.

Title: What are polymorphic messages?

Body:
Another term would be "dynamic dispatch".  Below is an example using (somewhat non-idiomatic) C#.

Say we have the following list type:

public abstract class List<T> {
    public abstract bool isEmpty();
    public abstract T head();
    public abstract List<T> tail();
}

class Empty<T> : List<T> {
    public Empty() { }
    public override bool isEmpty() { return true; }
    public override T head() {
        throw new ApplicationException("head of empty list");
    }
    public override List<T> tail() {
        throw new ApplicationException("tail of empty list");
    }
}

class Cons<T> : List<T> {
    private T head;
    private List<T> tail;
    public Cons(T head, List<T> tail) {
        this.head = head;
        this.tail = tail;
    }
    public override bool isEmpty() { return false; }
    public override T head() { return head;  }
    public override List<T> tail() { return tail; }
}

You could implement a length function as follows:

int length(List<T> xs) {
    if(xs.isEmpty()) {
        return 0;
    } else {
        return 1 + length(xs.tail());
    }
}

Alternatively, using dynamic dispatch as Kent Beck suggests, we could eliminate the conditional by adding length as a method of List<T>.

public abstract class List<T> {
    /* ... */
    public abstract int length();
}

class Empty<T> : List<T> {
    /* ... */
    public override int length() {
        return 0;
    }
}

class Cons<T> : List<T> {
    /* ... */
    public override int length() {
         return 1 + tail.length();
    }
}

This increases flexibility because we could now add another class with it's own implementation of length that can be smarter than our length function.

class Concat<T> : List<T> {
    private List<T> left, right;
    public Concat(List<T> left, List<T> right) {
        this.left = left;
        this.right = right;
    }
    /* ... */
    public override int length() {
        return left.length() + right.length();
    }
}

As a bit of an aside, there are problems with the definition of length in Cons<T> that will cause it to use a lot of stack space and likely produce a stack overflow in real use.  The problem is that it isn't written in a tail recursive form.  That's easily corrected, but even if corrected, since virtually no OO language guarantees tail call elimination the benefits that Kent Beck mentions can't be achieved in this example.  In other words, the lack of tail call elimination significantly hampers one of the main benefits of OO style.


答:
When you send a message to an object, the object is free to respond to the message however it wishes. In particular, different objects may respond to the same message differently: that's the essence of ad-hoc polymorphism.

Doing different things is of course also what conditionals are all about. So, polymorphic messages are a form of conditional, and as it turns out a very powerful one. Here's an example implementing traditional conditionals (i.e. if/then/else) using polymorphic messages in Ruby, the gist of it boils down to this:

class TrueClass
  def iff(thn: ->{}, els: ->{})
    thn.()
  end
end

class FalseClass
  def iff(thn: ->{}, els: ->{})
    els.()
  end
end

(2 < 3).iff(thn: -> { puts '2 is less than 3' }, els: -> { puts '2 is greater than 3' })
# 2 is less than 3
```

<p>And a slightly more complete example in Scala:</p>

<pre class="lang-scala prettyprint-override"><code>sealed abstract trait Buul {
  def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): T
  def &&&(other: ⇒ Buul): Buul
  def |||(other: ⇒ Buul): Buul
  def ntt: Buul
}

case object Tru extends Buul {
  override def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): U = thn
  override def &&&(other: ⇒ Buul) = other
  override def |||(other: ⇒ Buul): this.type = this
  override def ntt = Fls
}

case object Fls extends Buul {
  override def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): V = els
  override def &&&(other: ⇒ Buul): this.type = this
  override def |||(other: ⇒ Buul) = other
  override def ntt = Tru
}

object BuulExtension {
  import scala.language.implicitConversions
  implicit def boolean2Buul(b: ⇒ Boolean) = if (b) Tru else Fls
}

import BuulExtension._

(2 < 3) { println("2 is less than 3") } { println("2 is greater than 3") }
// 2 is less than 3
</code>```

You will notice that this object-oriented encoding of conditionals and booleans is exactly the same as the Church Encoding of conditionals and booleans in the λ-calculus. That's no coincidence: OO is about behavioral abstraction, and the λ-calculus supports nothing but function (i.e. behavior) abstraction, prompting Cook to call it "the first OO language", only half joking.

Title: What are polymorphic messages?

Body:
I am reading Martin Fowler's 'Refactoring: Improving the Design of Existing Code'. I have not understood a section of the second chapter where Kent Beck describes the pros of indirection.

One of the pros listed is Encoding Conditional Logic and it is described as:

Objects have a fabulous mechanism, polymorphic messages, to flexibly
  but clearly express conditional logic. By changing explicit
  conditionals to messages, you can often reduce duplication, add
  clarity, and increase flexibility all at the same time.

What are polymorphic messages? What does he mean by 'change explicit conditionals to messages'?

Body:
When you send a message to an object, the object is free to respond to the message however it wishes. In particular, different objects may respond to the same message differently: that's the essence of ad-hoc polymorphism.

Doing different things is of course also what conditionals are all about. So, polymorphic messages are a form of conditional, and as it turns out a very powerful one. Here's an example implementing traditional conditionals (i.e. if/then/else) using polymorphic messages in Ruby, the gist of it boils down to this:

class TrueClass
  def iff(thn: ->{}, els: ->{})
    thn.()
  end
end

class FalseClass
  def iff(thn: ->{}, els: ->{})
    els.()
  end
end

(2 < 3).iff(thn: -> { puts '2 is less than 3' }, els: -> { puts '2 is greater than 3' })
# 2 is less than 3

And a slightly more complete example in Scala:

sealed abstract trait Buul {
  def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): T
  def &&&(other: ⇒ Buul): Buul
  def |||(other: ⇒ Buul): Buul
  def ntt: Buul
}

case object Tru extends Buul {
  override def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): U = thn
  override def &&&(other: ⇒ Buul) = other
  override def |||(other: ⇒ Buul): this.type = this
  override def ntt = Fls
}

case object Fls extends Buul {
  override def apply[T, U <: T, V <: T](thn: ⇒ U)(els: ⇒ V): V = els
  override def &&&(other: ⇒ Buul): this.type = this
  override def |||(other: ⇒ Buul) = other
  override def ntt = Tru
}

object BuulExtension {
  import scala.language.implicitConversions
  implicit def boolean2Buul(b: ⇒ Boolean) = if (b) Tru else Fls
}

import BuulExtension._

(2 < 3) { println("2 is less than 3") } { println("2 is greater than 3") }
// 2 is less than 3

You will notice that this object-oriented encoding of conditionals and booleans is exactly the same as the Church Encoding of conditionals and booleans in the λ-calculus. That's no coincidence: OO is about behavioral abstraction, and the λ-calculus supports nothing but function (i.e. behavior) abstraction, prompting Cook to call it "the first OO language", only half joking.

ContentLicense: CC BY-SA 4.0


问:
How to design a complex REST API considering DB performance?
I've been following some tutorials on how to design REST APIs, but I still have some big questions marks. All these tutorials show resources with relatively simple hierarchies, and I would like to know how the principles used in those apply to a more complex one. Furthermore, they stay at a very high/architectural level. They barely show any relevant code, let alone the persistence layer. I'm specially concerned about database load/performance, as Gavin King said:

you will save yourself effort if you pay attention to the database at
  all stages of development

Let's say my application will provide training for Companies. Companies have Departments and Offices. Departments have Employees. Employees have Skills and Courses, and certain Level of certain skills are required to be able to sign for some courses. The hierarchy is as as follows, but with :

-Companies
  -Departments
    -Employees
      -PersonalInformation
        -Address
      -Skills (quasi-static data)
        -Levels (quasi-static data)
      -Courses
        -Address
  -Offices
    -Address
```

<p>Paths would be something as:</p>

<pre class="lang-none prettyprint-override"><code>companies/1/departments/1/employees/1/courses/1
companies/1/offices/1/employees/1/courses/1
</code>```

Fetching a resource

So ok, when returning a company, I obviously don't return the whole hierarchy companies/1/departments/1/employees/1/courses/1 + companies/1/offices/../. I might return a list of links to the departments or the expanded departments, and have to take the same decission at this level: do I return a list of links to the department's employees or the expanded employees? That will depend on the number of departments, employees, etc.

Question 1: Is my thinking correct, is "where to cut the hierarchy" a typical engineering decission I need to make?

Now let's say that when asked GET companies/id, I decide to return a list of links to the department collection, and the expanded office information. My companies don't have many offices, so joining with the tables Offices and Addresses shouldn't be a big deal. Example of response:

GET /companies/1

200 OK
{
  "_links":{
    "self" : {
      "href":"http://trainingprovider.com:8080/companies/1"
      },
      "offices": [
            { "href": "http://trainingprovider.com:8080/companies/1/offices/1"},
            { "href": "http://trainingprovider.com:8080/companies/1/offices/2"},
            { "href": "http://trainingprovider.com:8080/companies/1/offices/3"}
      ],
      "departments": [
            { "href": "http://trainingprovider.com:8080/companies/1/departments/1"},
            { "href": "http://trainingprovider.com:8080/companies/1/departments/2"},
            { "href": "http://trainingprovider.com:8080/companies/1/departments/3"}
      ]
  }
  "name":"Acme",
  "industry":"Manufacturing",
  "description":"Some text here",
  "offices": {
    "_meta":{
      "href":"http://trainingprovider.com:8080/companies/1/offices"
      // expanded offices information here
    }
  }
}
```

<p>At the code level, this implies that (using Hibernate, I'm not sure how it is with other providers, but I guess that's pretty much the same) I won't put a collection of <code>Department</code> as a field in my <code>Company</code> class, because:</p>

<ul>
<li>As said, I'm not loading it with <code>Company</code>, so I don't want to load it eagerly</li>
<li>And if I don't load it eagerly, I might as well remove it, because the persistence context will close after I load a Company and there is no point in trying to load it afterwards (<code>LazyInitializationException</code>).</li>
</ul>

<p>Then, I'll put a <code>Integer companyId</code> in the <code>Department</code> class, so that I can add a department to a company.</p>

<p>Also, I need to get the ids of all the departments. Another hit to the DB but not a heavy one, so should be ok. The code could look like:</p>

```@Service
@Path("/companies")
public class CompanyResource {

    @Autowired
    private CompanyService companyService;

    @Autowired
    private CompanyParser companyParser;

    @Path("/{id}")
    @GET
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    public Response findById(@PathParam("id") Integer id) {
        Optional<Company> company = companyService.findById(id);
        if (!company.isPresent()) {
            throw new CompanyNotFoundException();
        }
        CompanyResponse companyResponse = companyParser.parse(company.get());
        // Creates a DTO with a similar structure to Company, and recursivelly builds
        // sub-resource DTOs such as OfficeDTO
        Set<Integer> departmentIds = companyService.getDepartmentIds(id);
        // "SELECT id FROM departments WHERE companyId = id"
        // add list of links to the response
        return Response.ok(companyResponse).build();
    }
}
```

```@Entity
@Table(name = "companies")
public class Company {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;

    private String name;

    private String industry;

    @OneToMany(fetch = EAGER, cascade = {ALL}, orphanRemoval = true)
    @JoinColumn(name = "companyId_fk", referencedColumnName = "id", nullable = false)
    private Set<Office> offices = new HashSet<>();

    // getters and setters
}
```

```@Entity
@Table(name = "departments")
public class Department {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;

    private String name;

    private Integer companyId;

    @OneToMany(fetch = EAGER, cascade = {ALL}, orphanRemoval = true)
    @JoinColumn(name = "departmentId", referencedColumnName = "id", nullable = false)
    private Set<Employee> employees = new HashSet<>();

    // getters and setters
}
```

Updating a resource

For the update operation, I can expose an endpoint with PUT or POST. Since I want my PUT to be idempotent, I can't allow partial updates. But then, if I want to modify the company's description field, I need to send the whole resource representation. That seems too bloated. The same when updating an employee's PersonalInformation. I don't think it makes sense having to send all the Skills + Courses together with that.

Question 2: Is PUT just used for fine-grained resources?

I've seen in the logs that, when merging an entity, Hibernate executes a bunch of SELECT queries. I guess that's just to check if anything has changed and update whatever information needed. The upper the entity in the hierarchy, the heavier and more complex the queries. But some sources advise to use coarse grained resources. So again, I'll need to check how many tables are too much, and find a compromise between resource granularity and DB query complexity.

Question 3: Is this just another "know where to cut" engineering decission or am I missing something?

Question 4: Is this, or if not, what is the right "thinking process" when designing a REST service and searching for a compromise between resource granularity, query complexity and network chattiness?

答:
I think you have complexity because you are starting with over-complication:

Paths would be something as:

companies/1/departments/1/employees/1/courses/1

  companies/1/offices/1/employees/1/courses/1

Instead I would introduce simpler URL scheme like this:

GET companies/
    Returns a list of companies, for each company
    return short essential info (ID, name, maybe industry)
```

<pre class="lang-none prettyprint-override"><code>GET companies/1
    Returns single company info like this:

    {
        "name":"Acme",
        "description":"Some text here"
        "industry":"Manufacturing"
        departments: {
            "href":"/companies/1/departments"
            "count": 5
        }
        offices: {
            "href":"/companies/1/offices"
            "count": 3
        }
    }

    We don't expand the data for internal sub-resources,
    just return the count, so client knows that some data is present.
    In some cases count may be not needed too.
</code>```

GET companies/1/departments
    Returns company departments, again short info for each department
```

<pre class="lang-none prettyprint-override"><code>GET departments/
    Here you need to decide if it makes sense to expose
    a list of departments or not.
    If not - leave only companies/X/departments method.

    Note, that you can also use query string to make this
    method "searchable", like:
        /departments?company=1 - list of all departments for company 1
        /departments?type=support - all 'support' departments for all companies
</code>```

GET departments/1
    Returns department 1 data
```

This way it answers most of your questions - you "cut" the hierarchy right away and you don't bind your URL scheme to the internal data structure.
For example, if we know employee ID, would you expect to query it like employees/:ID or like companies/:X/departments/:Y/employees/:ID?

Regarding PUT vs POST requests, from your question it is clear that you feel the partial updates will be more efficient for your data.
So I would just use POSTs.

In practice, you actually want to cache data reads (GET requests) and it is less important for data updates.
And update are often can't be cached regardless of what type of request you do (like if server automatically sets the update time - it will be different for every request).

Update: regarding the right "thinking process" - since it is based on HTTP, we can apply the regular way of thinking when designing the web-site structure.
In this case on the top we can have a list of companies and show a short description for each with a link to the "view company" page, where we show company details and links to offices / departments and so on.

Body:
I think you have complexity because you are starting with over-complication:

Paths would be something as:

companies/1/departments/1/employees/1/courses/1

  companies/1/offices/1/employees/1/courses/1

Instead I would introduce simpler URL scheme like this:

GET companies/
    Returns a list of companies, for each company
    return short essential info (ID, name, maybe industry)

GET companies/1
    Returns single company info like this:

    {
        "name":"Acme",
        "description":"Some text here"
        "industry":"Manufacturing"
        departments: {
            "href":"/companies/1/departments"
            "count": 5
        }
        offices: {
            "href":"/companies/1/offices"
            "count": 3
        }
    }

    We don't expand the data for internal sub-resources,
    just return the count, so client knows that some data is present.
    In some cases count may be not needed too.

GET companies/1/departments
    Returns company departments, again short info for each department

GET departments/
    Here you need to decide if it makes sense to expose
    a list of departments or not.
    If not - leave only companies/X/departments method.

    Note, that you can also use query string to make this
    method "searchable", like:
        /departments?company=1 - list of all departments for company 1
        /departments?type=support - all 'support' departments for all companies

GET departments/1
    Returns department 1 data

This way it answers most of your questions - you "cut" the hierarchy right away and you don't bind your URL scheme to the internal data structure.
For example, if we know employee ID, would you expect to query it like employees/:ID or like companies/:X/departments/:Y/employees/:ID?

Regarding PUT vs POST requests, from your question it is clear that you feel the partial updates will be more efficient for your data.
So I would just use POSTs.

In practice, you actually want to cache data reads (GET requests) and it is less important for data updates.
And update are often can't be cached regardless of what type of request you do (like if server automatically sets the update time - it will be different for every request).

Update: regarding the right "thinking process" - since it is based on HTTP, we can apply the regular way of thinking when designing the web-site structure.
In this case on the top we can have a list of companies and show a short description for each with a link to the "view company" page, where we show company details and links to offices / departments and so on.


问:
Splitting one class into subclasses to save memory
Suppose that you describe programs, which have a lot of AssignmentStatement(target, /*value*/Expression). There are other statements, like if-statement and for-statement and all of them may have optional label. So, our assignment must be coded like AssignmentStatement(name, target, /*value*/Expression) on the blackboard. Now, whenever an instance is created, it will allocate 8 bytes more to store the null reference to the name. You see, the problem is the waste of memory. Name is null for 95% of statements but it still consumes 8 bytes of memory. Advanced programmers may advise Option[Name] to consume even more. But let's focus on plain String names for the sake of the argument.

Because 95% of instances will carry null in their first field, we can separate them into a separate class. In order to save the memory, I feel it is better to split my classes into  those which have name and those which don't,

```<code>class NamedAssignmentStatement(name, target, /*value*/Expression)
class AnonymousAssignmentStatement(target, /*value*/Expression)
</code>```

This eliminates duplication (many instances carrying the same attributes) and saves memory. The only problem is the difficulty of handling the duplicated classes.

They are proliferating exponentially actually. The value expression actually may have one optional mode argument, which, likewise the name, is almost never used. It makes sense to save more memory to replace AssignmentStatement(target, mode, /*value*/Expression) with two classes, one which has mode and another which not.

Now, I have 4 classes instead of single AssignmentStatement(name, target, mode, /*value*/Expression). But that is not the end of story. The values can be either simple expressions, evaluated to value but can also be waveforms, evaluated to a list of (value, duration) pairs. To distinguish between value types, I can wrap the value expression into SimpleExpression(expression) or into Waveform(expression) to be used like AssignmentStatement(target, Waveform(Expression)). The problem is that instantiating extra wrapper per every assignment will consume ca 20 extra bytes as far as I know from JVM. Instead of using the wrapper class, I can save the memory by signaling the value type with the assignment type itself, which means that I need to split the assginment class once more

```<code>SimpleAssignmentStatement(target, /*waveform_*/expression)
WaveformAssignmentStatement(target, /*simple_*/expression)
</code>```

You see, I have got 8 classes instead of originally 1. It should be a hell to manage such system. Are there languages that address easy handling of such multiple case classes? To ask more generally, is the static typing and large number of classes vs dynamic memory allocation and abuse issue that I addressed here is a known in programming?

I see that programmers are pretty careless regarding the memory usage. They will answer that 8 bytes for null is a meaningless price, worth to ignore. But,

watch the pennies, and the pounds take care of themselves

The highest mountains are made of tiniest, invisible sub-micron particles, which, in case of computational process has a form of (hundreds of) millions of objects in memory which accumulate to the huge strucutres. The problem is not only the cost of the memory (which you say is dirt-cheap and should not be considered) but the PC performance bottleneck, which is the questoin of how well your data fits the cache size. When your data fits the cache, it operates thouthands times faster and I cannot feel free looking how simple assignment, which needs only couple of byes to encode the statement, target and value reference escalates enormously, to the tens of bytes.

A disclaimer: I do not ask about importance of microoptimization. You can add your considerations regarding it into your answer but I ask how to reclaim that memory, not whether it is worth. Please understand the difference between asked how and not asked if it is worth or not.

Here is some more food for thought that may help to answer. Suppose that expressions can be binary operation, like Add(a,b), Multiply(a,b) or Concatenate(a,b). You would encode them using one general class BinOp(opCode,arg1,arg2). On the other hand, one could make BinOp an interface and make opcode its method rather than a class field and use 3 classes to implement this interface. This again would save memory because all instaces of Add class would share the opcode at the class level instead of carrying "+" String in their opcode, causing huge duplication. Such subclassing would probably improve performance because instead 2-level cases, first determining that we are dealing with BinOp and then taking action depending on the opcode, we can take action directly since every subclassed binary operation now knows what action is expected on it. The action can be evaluate operation, which depends on the binary operation subclass. Probably, there are known techniques to deal with such multiple subclasses, which can encompass the exponential explosion of the classes that I address in my question.

Hint2 Scala makes you less productive claims that splitting one class into many case-subclasses makes you more productive actually, in the case-classes section. Probably, if you can understand that idea, you can probably tell if it already answers the question. He says that class explosion won't occur but I do not quite understand why.

答:
The program you describe is a bit complex, but it occurs to me you can leverage the Flyweight design pattern. With that pattern you save memory by making many objects share the same state.

In your case the shared state would all the null values that take 8 bytes each.

Those classes that will have values other than null sould simply override the extrinsic properties of the flyweight. Extrinsic methods are those that exposed a common state whereas intrinsic ones are those who will vary from one instance to another. Extrinsic state will not eat up memory because it will exists only once.

Image taken from here.


答:
I think the Flyweight pattern might be what you're looking for. This is designed to store the minimal amount of data and apply it to objects on an as-needed basis.

eg. imagine a word processor where each letter was an individual object. Obviously it'd run slower than Word on a 286! But if you stored each letter in memory directly, and loaded the letter's data into an object as the user needed to manipulate it, then you have all the functionality without the overhead.

Title: Splitting one class into subclasses to save memory

Body:
Suppose that you describe programs, which have a lot of AssignmentStatement(target, /*value*/Expression). There are other statements, like if-statement and for-statement and all of them may have optional label. So, our assignment must be coded like AssignmentStatement(name, target, /*value*/Expression) on the blackboard. Now, whenever an instance is created, it will allocate 8 bytes more to store the null reference to the name. You see, the problem is the waste of memory. Name is null for 95% of statements but it still consumes 8 bytes of memory. Advanced programmers may advise Option[Name] to consume even more. But let's focus on plain String names for the sake of the argument.

Because 95% of instances will carry null in their first field, we can separate them into a separate class. In order to save the memory, I feel it is better to split my classes into  those which have name and those which don't,

```<code>class NamedAssignmentStatement(name, target, /*value*/Expression)
class AnonymousAssignmentStatement(target, /*value*/Expression)
</code>```

This eliminates duplication (many instances carrying the same attributes) and saves memory. The only problem is the difficulty of handling the duplicated classes.

They are proliferating exponentially actually. The value expression actually may have one optional mode argument, which, likewise the name, is almost never used. It makes sense to save more memory to replace AssignmentStatement(target, mode, /*value*/Expression) with two classes, one which has mode and another which not.

Now, I have 4 classes instead of single AssignmentStatement(name, target, mode, /*value*/Expression). But that is not the end of story. The values can be either simple expressions, evaluated to value but can also be waveforms, evaluated to a list of (value, duration) pairs. To distinguish between value types, I can wrap the value expression into SimpleExpression(expression) or into Waveform(expression) to be used like AssignmentStatement(target, Waveform(Expression)). The problem is that instantiating extra wrapper per every assignment will consume ca 20 extra bytes as far as I know from JVM. Instead of using the wrapper class, I can save the memory by signaling the value type with the assignment type itself, which means that I need to split the assginment class once more

```<code>SimpleAssignmentStatement(target, /*waveform_*/expression)
WaveformAssignmentStatement(target, /*simple_*/expression)
</code>```

You see, I have got 8 classes instead of originally 1. It should be a hell to manage such system. Are there languages that address easy handling of such multiple case classes? To ask more generally, is the static typing and large number of classes vs dynamic memory allocation and abuse issue that I addressed here is a known in programming?

I see that programmers are pretty careless regarding the memory usage. They will answer that 8 bytes for null is a meaningless price, worth to ignore. But,

watch the pennies, and the pounds take care of themselves

The highest mountains are made of tiniest, invisible sub-micron particles, which, in case of computational process has a form of (hundreds of) millions of objects in memory which accumulate to the huge strucutres. The problem is not only the cost of the memory (which you say is dirt-cheap and should not be considered) but the PC performance bottleneck, which is the questoin of how well your data fits the cache size. When your data fits the cache, it operates thouthands times faster and I cannot feel free looking how simple assignment, which needs only couple of byes to encode the statement, target and value reference escalates enormously, to the tens of bytes.

A disclaimer: I do not ask about importance of microoptimization. You can add your considerations regarding it into your answer but I ask how to reclaim that memory, not whether it is worth. Please understand the difference between asked how and not asked if it is worth or not.

Here is some more food for thought that may help to answer. Suppose that expressions can be binary operation, like Add(a,b), Multiply(a,b) or Concatenate(a,b). You would encode them using one general class BinOp(opCode,arg1,arg2). On the other hand, one could make BinOp an interface and make opcode its method rather than a class field and use 3 classes to implement this interface. This again would save memory because all instaces of Add class would share the opcode at the class level instead of carrying "+" String in their opcode, causing huge duplication. Such subclassing would probably improve performance because instead 2-level cases, first determining that we are dealing with BinOp and then taking action depending on the opcode, we can take action directly since every subclassed binary operation now knows what action is expected on it. The action can be evaluate operation, which depends on the binary operation subclass. Probably, there are known techniques to deal with such multiple subclasses, which can encompass the exponential explosion of the classes that I address in my question.

Hint2 Scala makes you less productive claims that splitting one class into many case-subclasses makes you more productive actually, in the case-classes section. Probably, if you can understand that idea, you can probably tell if it already answers the question. He says that class explosion won't occur but I do not quite understand why.


问:
What's the historical reason of including DOS-stub in PE images?
I have been wondering this question recently. Why does PE images have to be executable in MS-DOS? I've read on the topic and most of the answers I've seen simply explain it with compatibility. But what this compatibility actually means and targets?

If it was intended to allow a single executable both to run on DOS and Windows - what was the problem to include two executables for each OS instead?

Some of the answers I've read explaining it with compatibility states that if ".exe" files were purely formatted in the COFF format with the PE signature this would lead to a crash in the system when executed on DOS. But I was wondering if that's actually true. Wasn't the MS-DOS checking for "MZ" starting signature and if not would it have been so expensive to include such check and also perhaps another to print message similar to the DOS-stub if the image was starting with "PE" (and so it was a PE executable instead)?

答:
Yes, MS-DOS included a check for the "MZ" signature.

I believe it (the stub) was originally included because the was a fair amount of time that Windows ran entirely as a "shell" around MS-DOS and early versions of Windows (before 3.0) were nearly unusable for running MS-DOS programs. As such, it was fairly routine for people to have Windows installed, and have Windows programs installed, but not run Windows most of the time--they'd only start up Windows when they ran the (few) particular programs they had that needed it (e.g., Excel).

I think Microsoft expected that to be the case (and were correct), and believed that under such circumstances it was probably more friendly to print out a readable error message than to just die with "not a valid executable", or something on that order.

There was also at least some idea of having actual hybrid executables--i.e., an executable that contained a real MS-DOS program, so if you invoked it from MS-DOS, it would run a DOS program to do what you'd asked, and if you ran it from Windows, it would run a native Windows program. Note that support for this was fairly limited though--the stub program couldn't be very large, so if you'd decided to use it this way in most cases the stub would probably have had to just spawn the program DOS program, rather than the DOS program really being contained in the stub itself.

I don't know of that happening commercially, but I do know of another variant that was sometimes used, and worked fairly well: an MS-DOS stub program that, instead of just printing an error message and exiting, actually executed Windows with itself (i.e., this executable) on the command line, so invoking the program from a DOS command prompt still executed it under Windows.


问:
Why is Python 3.4 still developed after the release of 3.5?
The lastest version of Python is 3.5.1. However, the latest release is 3.4.4. Why is 3.4.x still developed? Are there breaking changes in 3.5 with respect to 3.4?

https://www.python.org/downloads/

答:
The same reason we receive updates to Windows 7 after the release of Windows 8, 8.1, 10.…

The same reason you can take your car to the dealer for a service and new tyres, even though it's ten years old. It would be a bit rubbish if you had to buy a whole new car every time you broke a tail light.

The same reason you can build a new porch at the front of your home instead of moving house.

Incremental updates and patches to older versions of software is a crucial part of maintaining a project; you can't just expect everyone to perform major upgrades every few weeks (each of which may come with its own dependency/interface breakages).


答:
Like others have said this is likely a premature optimization. Have you check exactly how much memory are you wasting for empty optional fields? If you want to have multiple, optimal classes you might want to have single factory method, that select appropriate class by passed arguments, but as you have said it won't scale anyway with many optional parameters.

If you really want to squeeze memory usage I am afraid you will have to give up on type safety and use something similar to network protocols / file formats.

```<code>[0]: 0011
[1-4]: field1
[5-9]: field2
</code>```

Or:

```<code>[0]: 0100
[1-4]: field3
</code>```

Of course it makes more sense for files as hard drive/network are much slower that CPU and wasting few cycles to pack data as tight as possible is a good trade-off. For RAM just using unaligned data access might make you program 10 times slower and with above method you also have to calculate offset for each field for every write&read. But it saves memory, so it is worth it, right?

Body:
Like others have said this is likely a premature optimization. Have you check exactly how much memory are you wasting for empty optional fields? If you want to have multiple, optimal classes you might want to have single factory method, that select appropriate class by passed arguments, but as you have said it won't scale anyway with many optional parameters.

If you really want to squeeze memory usage I am afraid you will have to give up on type safety and use something similar to network protocols / file formats.

[0]: 0011
[1-4]: field1
[5-9]: field2

Or:

[0]: 0100
[1-4]: field3

Of course it makes more sense for files as hard drive/network are much slower that CPU and wasting few cycles to pack data as tight as possible is a good trade-off. For RAM just using unaligned data access might make you program 10 times slower and with above method you also have to calculate offset for each field for every write&read. But it saves memory, so it is worth it, right?


答:
For important software, it is common to support a release for multiple years. With Python, support is mostly in the form of bugfixes. You can see the changelog for the whole list of fixes between versions (see 3.5 full changelog).

Between versions such as 3.5 and 3.4, not only bugs were fixed but also new features were added. E.g. 3.5 added the @ matrix multiplication operator, and async-await style coroutine support. Organizations may want a fully-patched system without the uncertainty implied by a new feature, after having tested that a specific Python version works for them.

Given a version number such as 3.5.1, this allows you to quickly recognize the level of compatibility: for Python version 3, that would be the 5th feature release, and the 1st bug fix release for that feature release. Similarly, the 2.x branch is not being developed further, and is stuck at 2.7.x. However, this widely spread version will still receive bugfixes (currently at 2.7.11).

For further information, read the about the Python Development Cycle in the Developer Guide.

Title: Why is Python 3.4 still developed after the release of 3.5?

Body:
The lastest version of Python is 3.5.1. However, the latest release is 3.4.4. Why is 3.4.x still developed? Are there breaking changes in 3.5 with respect to 3.4?

https://www.python.org/downloads/


问:
Distributed transactions with Kafka
I need to implement a transaction that spans over loosely coupled (SOA+MOM) components.

When a particular event is received, FOO and BAR need to do a transactional operation: that is, both the transactions on FOO-DB and BAR-DB need to be successful or none of them. The consistency here is very important.

When the event is received (1), FOO does an operation on a database (2).

If the operation fails, nothing more happen.

If the operation is successful, a message is sent to BAR through a MOM (Kafka) (3).

BAR does an operation on another database (4).

If the operation fails, the operation previously done on FOO-DB must be reverted (5, 6).

If the operation is successful, all is fine.

Right now I'm using Kafka. I like its simplicity and speed, but I'm open to consider other solutions if they would make this situation easier to implement/maintain/extend.

I'm quite new to SOA and MOM architectures and patterns, so I'm wondering:

is this a common scenario/pattern?

how is this commonly implemented?

the simple means offered by Kafka are enough to reliably implement this or I would be better considering other solutions?

is the distributed transaction manager usually provided by the MOM or by the database? and if it is the DB, how can this be done using different DBs?

Sorry for the many questions, I hope they're not too much for a single question.
Thank you!

答:
I'm posting this comment as an answer because it's too long for a comment, but it does not give a workable solution to the OPs problem, only explains why what he wants to do is AFAIK impossible to implement 100% correctly.

You mention FOO-DB twice in your comment but not BAR-DB. Did you mean BAR-DB in your second instance? I'm going to assume you did.

Anyway, here's why what you wrote won't work: you update FOO-DB but with a "not validated" tag, it succeeds, you go to update BAR-DB, and once you confirm that update is successful, you remove the "not validated" tag from FOO-DB (which is just another update). But in that case, between the time you update BAR-DB and mark FOO-DB as validated, your combined databases are in an inconsistent state.

In essence, you only delayed the problem by one round. This is similar to the two generals problem, but not the exact same. Even though we can assume that the communications between these two nodes is guaranteed (because you're using Kafka), there is always an unspecified and unknown time delay, making an Atomic and Consistent update to the combined databases impossible.

I would like to point out that I am no databases expert. But for example, this was a problem where I worked with using the MongoDB cluster we had setup. Because we had multiple nodes for each database (replication), if you updated a record in the database and then read the record soon afterwords, there was no guarantee you would read the updated record. The writes took some (usually small but) unknown amount of time to propagate to all the replicas.

You actually haven't specified which Databases you're using, so I don't know the exact guarantees they provide, but since they are two entirely different clusters from what it sounds like, so no matter what the MOM you use to communicate between them is, they cannot perfectly coordinate.

The only solution I can think of, is to lock both databases, perform your updates, and then unlock both databases. Obviously this is not viable, but it is a thought experiment to show what kind of extreme measures it would take to accomplish this.

Title: Distributed transactions with Kafka