问:
Is the use of NoSQL Databases impractical for large datasets where you need to search by content?
I've been learning about NoSQL Databases for a week now.

I really understand the advantages of NoSQL Databases and the many use cases they are great for.

But often people write their articles as if NoSQL could replace Relational Databases. And there is the point I can't get my head around:

NoSQL Databases are (often) key-value stores.

Of course it is possible to store everything into a key-value store (by encoding the data in JSON, XML, whatever), but the problem I see is that you need to get some amount of data that matches a specific criterion, in many use cases. In a NoSQL database you have only one criterion you can search for effectively - the key. Relational Databases are optimized to search for any value in the data row effectively.

So NoSQL Databases are not really a choice for persisting data that need to be searched by their content. Or have I misunderstood something?

An example:

You need to store user data for a webshop.

In a relational database you store every user as an row in the users table, with an ID, the name, his country, etc.

In a NoSQL Database you would store each user with his ID as key and all his data (encoded in JSON, etc.) as value.

So if you need to get all users from a specific country (for some reason the marketing guys need to know something about them), it's easy to do so in the Relational Database, but not very effective in the NoSQL Database, because you have to get every user, parse all the data and filter.

I don't say it's impossible, but it gets a lot more tricky and I guess not that effective if you want to search in the data of NoSQL entries.

You could create a key for each country that stores the keys of every user who lives in this country, and get the users of a specific country by getting all the keys which are deposited in the key for this country. But I think this techique makes a complex dataset even more complex - it's harder to implement and not as effective as querying an SQL Database. So I think it's not a way you would use in production. Or is it?

I'm not really sure if I misunderstood something or overlooked some concepts or best practices to handle such use cases. Maybe you could correct my statements and answer my questions.

答:
Generally speaking, if your workflow is a perfect match for relational database queries, you'll find relational databases to be the most efficient approach.  Its kind of tautological, but its true.

The claim that many NoSQL advocates would make is that many workflows were actually massaged into a relational form, and would have been more effective before such massaging.  The validity of this claim is complicated to ascertain.  Clearly there are jobs that are very well described by SQL queries.  I can say from my experience that my particular relational programming tasks could have been done using NoSQL with nearly the same level of efficiency, if not more.  However, that's a very subjective statement based on narrow experience.

I have a feeling much of the sale of the NoSQL approach comes from the assumption of large databases.  The larger the database is, the more you must groom your workflow to support the larger datasets.  NoSQL seems to be better at supporting that grooming effort.  Thus, the larger the database, the more important NoSQL's features can potentially be.

To use the example, in SQL querying by country is just as slow as the NoSQL scan of all users, unless you explicitly told SQL to index the users table by country.  NoSQL can do the same, where you create an ordered key-value collection that is the index (just like SQL does under the hood) and maintain it.

The difference?  SQL engines had the concept of indexing the table built in.  This means you got to do less work (all you had to do was add an index to the table).  However, it also means you had less control.  For most cases, that loss of control is acceptable, in exchange for the SQL engine doing the work for you.  However, in massive datasets, you may want a different consistency model than the typical SQL ACID model.  You may want to use the BASE model which supports eventual consistency.  That could be very difficult in SQL, because the SQL engine is doing the work for you so it has to be done by the SQL engine's rules.  In NoSQL, those layers are typically exposed, letting you hack at them.


答:
Relational Databases are optimized to search for any value in the
  datarow effectively.

Don't confuse the ability to search on "any" value in a row with "every" value in a row. The most effective way to do this requires one or more indexes. You could have indexes include all the fields, but then you just hindered you're ability to make changes that require altering the index (inserts, updates, deletes). You (or your DBA) have to understand the data, the usage, bottlenecks etc.

ContentLicense: CC BY-SA 3.0

Title: Is the use of NoSQL Databases impractical for large datasets where you need to search by content?

Body:
I've been learning about NoSQL Databases for a week now.

I really understand the advantages of NoSQL Databases and the many use cases they are great for.

But often people write their articles as if NoSQL could replace Relational Databases. And there is the point I can't get my head around:

NoSQL Databases are (often) key-value stores.

Of course it is possible to store everything into a key-value store (by encoding the data in JSON, XML, whatever), but the problem I see is that you need to get some amount of data that matches a specific criterion, in many use cases. In a NoSQL database you have only one criterion you can search for effectively - the key. Relational Databases are optimized to search for any value in the data row effectively.

So NoSQL Databases are not really a choice for persisting data that need to be searched by their content. Or have I misunderstood something?

An example:

You need to store user data for a webshop.

In a relational database you store every user as an row in the users table, with an ID, the name, his country, etc.

In a NoSQL Database you would store each user with his ID as key and all his data (encoded in JSON, etc.) as value.

So if you need to get all users from a specific country (for some reason the marketing guys need to know something about them), it's easy to do so in the Relational Database, but not very effective in the NoSQL Database, because you have to get every user, parse all the data and filter.

I don't say it's impossible, but it gets a lot more tricky and I guess not that effective if you want to search in the data of NoSQL entries.

You could create a key for each country that stores the keys of every user who lives in this country, and get the users of a specific country by getting all the keys which are deposited in the key for this country. But I think this techique makes a complex dataset even more complex - it's harder to implement and not as effective as querying an SQL Database. So I think it's not a way you would use in production. Or is it?

I'm not really sure if I misunderstood something or overlooked some concepts or best practices to handle such use cases. Maybe you could correct my statements and answer my questions.


问:
Given a herd of horses, how do I find the average horn length of all unicorns?
The question above is an abstract example of a common problem I encounter in legacy code, or more accurately, problems resulting from previous attempts at solving this problem.

I can think of at least one .NET framework method that is intended to address this problem, like the Enumerable.OfType<T> method. But the fact that you ultimately end up interrogating an object's type at runtime doesn't sit right with me.

Beyond asking each horse "Are you a unicorn?" the following approaches also come to mind:

Throw an exception when an attempt is made to get the length of a non-unicorn's horn (exposes functionality not appropriate for each horse)

Return a default or magic value for the length of a non-unicorn's horn (requires default checks peppered throughout any code that wants to crunch horn stats on a group of horses that could all be non-unicorns)

Do away with inheritance and create a separate object on a horse that tells you if the horse is a unicorn or not (which is potentially pushing the same problem down a layer)

I have a feeling this is going to be best answered with a "non-answer." But how do you approach this problem and if it depends, what's the context around your decision?

I'd also be interested in any insights on whether this problem still exists in functional code (or maybe it only exists in functional languages that support mutability?)

This was flagged as a possible duplicate of the following question:
How to avoid downcasting?

The answer to that question assumes that one is in possession of a HornMeasurer by which all horn measurements must be made. But that's quite an imposition on a codebase that was formed under the egalitarian principle that everyone should be free to measure the horn of a horse.

Absent a HornMeasurer, the accepted answer's approach mirrors the exception-based approach listed above.

There's also been some confusion in the comments on whether horses and unicorns are both equines, or if a unicorn is a magical subspecies of horse. Both possibilities should be considered--perhaps one is preferable to the other?

答:
Short form of same answer at end requires reading book or web article.

Visitor Pattern

The problem has a mixture of horses and unicorns.
(Violating the Liskov substitution principle is a common problem in legacy codebases.)

Add a method to horse and all subclasses

```<code>Horse.visit(EquineVisitor v)
</code>```

Equine visitor interface looks something like this in java/c#

```<code>interface EquineVisitor {
  void visitHorse(Horse z);
  void visitUnicorn(Unicorn z);
}

Unicorn.visit(EquineVisitor v){
   v.visitUnicorn(this);
}

Horse.visit(EquineVisitor v){
   v.visitHorse(this);
}
</code>```

To measure horns we now write....

```<code>class HornMeasurer implements EquineVistor {
    void visitHorse(Horse h){} // ignore horses
    void visitUnicorn(Unicorn u){
         double len = u.getHornLength();
         totalLength+=len;
         unicornCount++;
    }

    double getAverageLength(){
          return totalLength/unicornCount;
    }

    double totalLength=0;
    int unicornCount=0;
}
</code>```

The visitor pattern is criticised for making refactoring and growth harder.

Short Answer: Use the design pattern Visitor to get double dispatch.

see also https://en.wikipedia.org/wiki/Visitor_pattern

see also http://c2.com/cgi/wiki?VisitorPattern  for discussion of visitors.

see also Design Patterns by Gamma et al.

Body:
Short form of same answer at end requires reading book or web article.

Visitor Pattern

The problem has a mixture of horses and unicorns.
(Violating the Liskov substitution principle is a common problem in legacy codebases.)

Add a method to horse and all subclasses

Horse.visit(EquineVisitor v)

Equine visitor interface looks something like this in java/c#

interface EquineVisitor {
  void visitHorse(Horse z);
  void visitUnicorn(Unicorn z);
}

Unicorn.visit(EquineVisitor v){
   v.visitUnicorn(this);
}

Horse.visit(EquineVisitor v){
   v.visitHorse(this);
}

To measure horns we now write....

class HornMeasurer implements EquineVistor {
    void visitHorse(Horse h){} // ignore horses
    void visitUnicorn(Unicorn u){
         double len = u.getHornLength();
         totalLength+=len;
         unicornCount++;
    }

    double getAverageLength(){
          return totalLength/unicornCount;
    }

    double totalLength=0;
    int unicornCount=0;
}

The visitor pattern is criticised for making refactoring and growth harder.

Short Answer: Use the design pattern Visitor to get double dispatch.

see also https://en.wikipedia.org/wiki/Visitor_pattern

see also http://c2.com/cgi/wiki?VisitorPattern  for discussion of visitors.

see also Design Patterns by Gamma et al.


答:
NoSQL is a rather vague term, since it basically covers all database systems which are not relational.

What you describe is a key-value store, which is a kind of database where a blob of data is stored under a key, and can be quickly looked up if you know the key. These databases are blazingly fast if you know the exact key, but as you say yourself, if you need to search or filter on multiple properties on the data, it will be slow and cumbersome.

Nobody in their right mind would claim that key-value stores can replace relational databases in general. However there may be particular use cases where are key-value store is a good fit. Key-value stores are often used for caching, since you typically cache items by id, but you don't need to perform ad-hoc queries over caches. For example the Stackoverflow site itself uses Redis (a key-value db) extensively, but only for output caching. The underlying canonical data is still persisted in a relational database.

So the answer is pretty obvious: Use a key-value store if you only need to store and lookup using a single key. Otherwise use a different kind of database. And if you are in doubt, use a relational database, since this is the most versatile kind of database, while the NoSQL databases are often optimized towards very particular use cases.


问:
How are POST/PUT/DELETE data passed to APIs?
I know that the data is passed through HTTP, but I'm not sure if I should be passing data through HTTP headers, or HTTP bodies. Which one is the convention for APIs?

Also, theoretically using PHP, which would be easier to process? Based on what I know, using headers seems to be the easiest method.

POST body

POST /api/v1/test_params.json HTTP/1.1
Host: rollrbla.de
Content-Length: 59
X-Target-URI: http://rollrbla.de
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Connection: Keep-Alive

Test-Parameter-1=Test-value-1&Test-Parameter-2=Test-Value-2
```

<h3>POST header</h3>

<pre class="lang-none prettyprint-override"><code>POST /api/v1/test_params.json HTTP/1.1
Test-Parameter-2: Test-Value-2
Host: rollrbla.de
Test-Parameter-1: Test-Value-1
Content-Length: 0
X-Target-URI: http://rollrbla.de
Connection: Keep-Alive
</code>```

Overall, what would be the best way to process this data, and what is the convention?

答:
I've looked at what a request looks like using body parameters and header parameters, and also some PHP examples (thanks to @RobertHarvey) and I've concluded that body parameters are the best way to go, as they do not require any special functions to read in PHP. I was just unsure of what they were, but now I understand completely.

POST Body

```<code>POST /api/v1/test_params.json HTTP/1.1
Host: rollrbla.de
Content-Length: 59
X-Target-URI: http://rollrbla.de
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Connection: Keep-Alive

Test-Parameter-1=Test-value-1&Test-Parameter-2=Test-Value-2
</code>```

... is the best way to go. Thanks to everybody who helped me reach this conclusion.

Body:
I've looked at what a request looks like using body parameters and header parameters, and also some PHP examples (thanks to @RobertHarvey) and I've concluded that body parameters are the best way to go, as they do not require any special functions to read in PHP. I was just unsure of what they were, but now I understand completely.

POST Body

POST /api/v1/test_params.json HTTP/1.1
Host: rollrbla.de
Content-Length: 59
X-Target-URI: http://rollrbla.de
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Connection: Keep-Alive

Test-Parameter-1=Test-value-1&Test-Parameter-2=Test-Value-2

... is the best way to go. Thanks to everybody who helped me reach this conclusion.


答:
The body.

Headers are supposed to describe the request, not encapsulate its payload.

Headers are meta-data.

Title: How are POST/PUT/DELETE data passed to APIs?

Body:
I know that the data is passed through HTTP, but I'm not sure if I should be passing data through HTTP headers, or HTTP bodies. Which one is the convention for APIs?

Also, theoretically using PHP, which would be easier to process? Based on what I know, using headers seems to be the easiest method.

POST body

POST /api/v1/test_params.json HTTP/1.1
Host: rollrbla.de
Content-Length: 59
X-Target-URI: http://rollrbla.de
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Connection: Keep-Alive

Test-Parameter-1=Test-value-1&Test-Parameter-2=Test-Value-2
```

<h3>POST header</h3>

<pre class="lang-none prettyprint-override"><code>POST /api/v1/test_params.json HTTP/1.1
Test-Parameter-2: Test-Value-2
Host: rollrbla.de
Test-Parameter-1: Test-Value-1
Content-Length: 0
X-Target-URI: http://rollrbla.de
Connection: Keep-Alive
</code>```

Overall, what would be the best way to process this data, and what is the convention?


答:
Well, it sounds like your semantic domain has an IS-A relationship, but you're a bit wary of using subtypes/inheritance to model this—particularly because of the runtime type reflection.  I think however that you're scared of the wrong thing—subtyping does indeed come with dangers, but the fact that you're querying an object at runtime is not the problem.  You'll see what I mean.

Object-oriented programming has leaned quite heavily on the notion of IS-A relationships, it has arguably leaned too heavily on it, leading to two famous critical concepts:

Circle-ellipse problem

Composition over inheritance

But I think there's another, more functional-programming-based way to look at IS-A relationships that perhaps doesn't have these difficulties.  First, we want to model horses and unicorns in our program, so we are going to have a Horse and an Unicorn type.  What are the values of these types?  Well, I'd say this:

The values of these types are representations or descriptions of horses and unicorns (respectively);

They are schematized representations or descriptions—they're not free-form, they're constructed according to very strict rules.

That may sound obvious, but I think one of the ways people get into issues like the circle-ellipse problem is by not minding those points carefully enough.  Every circle is an ellipse, but that doesn't mean that every schematized description of a circle is automatically a schematized description of an ellipse according to a different schema.  In other words, just because a circle is an ellipse doesn't mean that a Circle is an Ellipse, so to speak.  But it does mean that:

There is a total function that converts any Circle (schematized circle description) into an Ellipse (different type of description) that describes the same circles;

There is a partial function that takes an Ellipse and, if describes a circle, returns the corresponding Circle.

So, in functional programming terms, your Unicorn type doesn't need to be a subtype of Horse at all, you just need operations like these:

```<code>-- Convert any unicorn-description of into a horse-description that
-- describes the same unicorns.
toHorse :: Unicorn -> Horse

-- If the horse described by the given horse-description is a unicorn,
-- then return a unicorn-description of that unicorn, otherwise return
-- nothing.
toUnicorn :: Horse -> Maybe Unicorn
</code>```

And toUnicorn needs to be a right inverse of toHorse:

```<code>toUnicorn (toHorse x) = Just x
</code>```

Haskell's Maybe type is what other languages call an "option" type.  For example, the Java 8 Optional<Unicorn> type is either an Unicorn or nothing.  Note that two of your alternatives—throwing an exception or returning a "default or magic value"—are very similar to option types.

So basically what I've done here is reconstruct the concept IS-A relationship in terms of types and functions, without using subtypes or inheritance.  What I would take away from this is:

Your model needs to have a Horse type;

The Horse type needs to encode enough information to determine unambiguously whether any value describes a unicorn;

Some operations of the Horse type need to expose that information so that clients of the type can observe whether a given Horse is a unicorn;

The clients of the Horse type will have to use these latter operations at runtime to discriminate between unicorns and horses.

So this is fundamentally an "ask every Horse whether it is an unicorn" model.  You are wary of that model, but I think wrongly so.  If I give you a list of Horses, all that the type guarantees is that the things that the items in the list describe are horses—so you are, inevitably, going to need to do something at runtime to tell which of them are unicorns.  So there's no getting around that, I think—you need to implement operations that will do that for you.

In object-oriented programming, the familiar way of doing this is the following:

Have a Horse type;

Have Unicorn as a subtype of Horse;

Use runtime type reflection as the client-accessible operation that discerns whether a given Horse is an Unicorn.

This does have a big weakness, when you look at it from the "thing vs. description" angle that I presented above:

What if you have a Horse instance that describes a unicorn but is not an Unicorn instance?

Going back to the beginning, this is what I think is the really scary part about using subtyping and downcasts for modeling this IS-A relationship—not the fact that you have to do a runtime check.  Abusing the typography a bit, asking a Horse whether it's an Unicorn instance is not synonymous with asking a Horse whether it is an unicorn (whether it is a Horse-description of a horse that is also an unicorn).  Not unless your program has gone to great lengths to encapsulate the code that constructs Horses so that every time a client tries to construct a Horse that describes an unicorn, the Unicorn class is instantiated.  In my experience, seldom do programmers do things this carefully.

So I'd go with the approach where there is an explicit, non-downcast operation that converts Horses to Unicorns.  This could either be a method of the Horse type:

```<code>interface Horse {
    // ...
    Optional<Unicorn> toUnicorn();
}
</code>```

...or it could be an external object (your "separate object on a horse that tells you if the horse is a unicorn or not"):

```<code>class HorseToUnicornCoercion {
    Optional<Unicorn> convert(Horse horse) {
       // ...
    }
}
</code>```

The choice between these is a matter of how your program is organized—in both cases, you have the equivalent of my Horse -> Maybe Unicorn operation from above, you're just packaging it in different ways (that will admittedly have ripple effects on what operations the Horse type needs to expose to its clients).

Title: Given a herd of horses, how do I find the average horn length of all unicorns?

Body:
The question above is an abstract example of a common problem I encounter in legacy code, or more accurately, problems resulting from previous attempts at solving this problem.

I can think of at least one .NET framework method that is intended to address this problem, like the Enumerable.OfType<T> method. But the fact that you ultimately end up interrogating an object's type at runtime doesn't sit right with me.

Beyond asking each horse "Are you a unicorn?" the following approaches also come to mind:

Throw an exception when an attempt is made to get the length of a non-unicorn's horn (exposes functionality not appropriate for each horse)

Return a default or magic value for the length of a non-unicorn's horn (requires default checks peppered throughout any code that wants to crunch horn stats on a group of horses that could all be non-unicorns)

Do away with inheritance and create a separate object on a horse that tells you if the horse is a unicorn or not (which is potentially pushing the same problem down a layer)

I have a feeling this is going to be best answered with a "non-answer." But how do you approach this problem and if it depends, what's the context around your decision?

I'd also be interested in any insights on whether this problem still exists in functional code (or maybe it only exists in functional languages that support mutability?)

This was flagged as a possible duplicate of the following question:
How to avoid downcasting?

The answer to that question assumes that one is in possession of a HornMeasurer by which all horn measurements must be made. But that's quite an imposition on a codebase that was formed under the egalitarian principle that everyone should be free to measure the horn of a horse.

Absent a HornMeasurer, the accepted answer's approach mirrors the exception-based approach listed above.

There's also been some confusion in the comments on whether horses and unicorns are both equines, or if a unicorn is a magical subspecies of horse. Both possibilities should be considered--perhaps one is preferable to the other?

Body:
Well, it sounds like your semantic domain has an IS-A relationship, but you're a bit wary of using subtypes/inheritance to model this—particularly because of the runtime type reflection.  I think however that you're scared of the wrong thing—subtyping does indeed come with dangers, but the fact that you're querying an object at runtime is not the problem.  You'll see what I mean.

Object-oriented programming has leaned quite heavily on the notion of IS-A relationships, it has arguably leaned too heavily on it, leading to two famous critical concepts:

Circle-ellipse problem

Composition over inheritance

But I think there's another, more functional-programming-based way to look at IS-A relationships that perhaps doesn't have these difficulties.  First, we want to model horses and unicorns in our program, so we are going to have a Horse and an Unicorn type.  What are the values of these types?  Well, I'd say this:

The values of these types are representations or descriptions of horses and unicorns (respectively);

They are schematized representations or descriptions—they're not free-form, they're constructed according to very strict rules.

That may sound obvious, but I think one of the ways people get into issues like the circle-ellipse problem is by not minding those points carefully enough.  Every circle is an ellipse, but that doesn't mean that every schematized description of a circle is automatically a schematized description of an ellipse according to a different schema.  In other words, just because a circle is an ellipse doesn't mean that a Circle is an Ellipse, so to speak.  But it does mean that:

There is a total function that converts any Circle (schematized circle description) into an Ellipse (different type of description) that describes the same circles;

There is a partial function that takes an Ellipse and, if describes a circle, returns the corresponding Circle.

So, in functional programming terms, your Unicorn type doesn't need to be a subtype of Horse at all, you just need operations like these:

-- Convert any unicorn-description of into a horse-description that
-- describes the same unicorns.
toHorse :: Unicorn -> Horse

-- If the horse described by the given horse-description is a unicorn,
-- then return a unicorn-description of that unicorn, otherwise return
-- nothing.
toUnicorn :: Horse -> Maybe Unicorn

And toUnicorn needs to be a right inverse of toHorse:

toUnicorn (toHorse x) = Just x

Haskell's Maybe type is what other languages call an "option" type.  For example, the Java 8 Optional<Unicorn> type is either an Unicorn or nothing.  Note that two of your alternatives—throwing an exception or returning a "default or magic value"—are very similar to option types.

So basically what I've done here is reconstruct the concept IS-A relationship in terms of types and functions, without using subtypes or inheritance.  What I would take away from this is:

Your model needs to have a Horse type;

The Horse type needs to encode enough information to determine unambiguously whether any value describes a unicorn;

Some operations of the Horse type need to expose that information so that clients of the type can observe whether a given Horse is a unicorn;

The clients of the Horse type will have to use these latter operations at runtime to discriminate between unicorns and horses.

So this is fundamentally an "ask every Horse whether it is an unicorn" model.  You are wary of that model, but I think wrongly so.  If I give you a list of Horses, all that the type guarantees is that the things that the items in the list describe are horses—so you are, inevitably, going to need to do something at runtime to tell which of them are unicorns.  So there's no getting around that, I think—you need to implement operations that will do that for you.

In object-oriented programming, the familiar way of doing this is the following:

Have a Horse type;

Have Unicorn as a subtype of Horse;

Use runtime type reflection as the client-accessible operation that discerns whether a given Horse is an Unicorn.

This does have a big weakness, when you look at it from the "thing vs. description" angle that I presented above:

What if you have a Horse instance that describes a unicorn but is not an Unicorn instance?

Going back to the beginning, this is what I think is the really scary part about using subtyping and downcasts for modeling this IS-A relationship—not the fact that you have to do a runtime check.  Abusing the typography a bit, asking a Horse whether it's an Unicorn instance is not synonymous with asking a Horse whether it is an unicorn (whether it is a Horse-description of a horse that is also an unicorn).  Not unless your program has gone to great lengths to encapsulate the code that constructs Horses so that every time a client tries to construct a Horse that describes an unicorn, the Unicorn class is instantiated.  In my experience, seldom do programmers do things this carefully.

So I'd go with the approach where there is an explicit, non-downcast operation that converts Horses to Unicorns.  This could either be a method of the Horse type:

interface Horse {
    // ...
    Optional<Unicorn> toUnicorn();
}

...or it could be an external object (your "separate object on a horse that tells you if the horse is a unicorn or not"):

class HorseToUnicornCoercion {
    Optional<Unicorn> convert(Horse horse) {
       // ...
    }
}

The choice between these is a matter of how your program is organized—in both cases, you have the equivalent of my Horse -> Maybe Unicorn operation from above, you're just packaging it in different ways (that will admittedly have ripple effects on what operations the Horse type needs to expose to its clients).


问:
How to handle 50% of worse than average sprints?
My understanding of Scrum is that I determine the work my team can take on in the next sprint by averaging the number of completed points for the past several sprints.

This is an average, so if history repeats itself, this sprint it is 50% likely that we are taking on too few story points, and 50% likely that we are taking on too many story points.

In the case we have taken on too many story points we could:

Fail to complete the sprint. This means we will fail to meet our sprint commitment half the time.

Work extra to catch up. The problem is that this ratchets only one way. We will accomplish the sprint, and the number of story points completed will reflect that. Since we always finish, over time, our average will trend upward to the point where we are always accomplishing a large number of story points and staying late.

Is my understanding of average velocity and sprint commitments correct?

If so, what should we do for the 50% of sprints where we are behind average?

If not, what have I gotten wrong?

答:
Is my understanding of average velocity and sprint commitments correct?

Unfortunately, you've been misinformed about a few things regarding Sprint Planning in Scrum. First, the Development Team (DT) is

...structured and empowered by the organization to organize and manage their own work. - Scrum Guide

The word for this is self-organizing. This includes the work of forecasting the work that will be done in a given Sprint. The DT is not told what it will work on each sprint, it is rather empowered to choose its own work. The DT might need information like historical velocity, a well-refined Product Backlog, and the next Sprint's DT capacity to create a forecast. Ultimately, it's the DT's determination of what can and cannot be accomplished in a Sprint that should prevail in their forecast; they should not be told how much work they will do.

Notice also, forecast and not commitment. The c-word was removed from the Scrum Guide because it was being used to abuse the Development Team. Forecast is the preferred term.

Regarding the probability of missing the Sprint forecast on the low or high end, I don't see that as being of importance. At some point, more analysis does not yield better accuracy and I think we're beyond that point now.

Also, a Sprint can only be "Canceled;" it can never fail. A Sprint is canceled only when the goal of the sprint becomes completely obsolete and irrelevant. This is very rarely the case. If a forecast is incorrect, you just keep calm and Scrum on. Have you retrospective. Next Sprint, your forecasts will be better :).

ContentLicense: CC BY-SA 4.0


问:
Synchronous facade hiding asynchronous web service
I'm maintaining a system providing a typical synchronous web service REST API. It routes each request to one of several possible backend services. Now there is a new backend I want to use, but there is a problem - it has an asynchronous API - I will need to do a request, and then wait for a callback request.

I'm thinking that it's possible to hide this asynchronicity behind a syncronizing facade. This facade will call the async API and then put the incoming web requests "on hold". When the callback endpoint is called a few seconds later it will resume the correct web request and respond to the original caller. If the callback does not arrive within let's say 10 seconds I will stop waiting, and respond with an error.

A complicating factor is that I have several web nodes behind a load balancer, but I'm thinking I can use a distributed storage solution like Redis to publish incoming callbacks, so that the correct web node can pick it up.

The question is: Is this feasible? Or do you believe it will be really difficult to get it to work properly? Any experience you can share or links to relevant resources would be much appreciated. Any other aproach to using an asynchronous web service in another service which needs to be syncronous?

Some pseudo code of what I'm thinking:

```<code>define rest-endpoint "/foobar" {
    call async backend
    until 10 seconds has passed {
        sleep 1 second
        result = redis.get(some_correlation_id)
        if (result)
            return new Response(result)
    }
    return new Response("failed")
}

define rest-endpoint "/callback" {
    redis.push(some_correlation_id)
}
</code>```

Note that if I time out and respond "failed" I don't (in my particular case) have to create any kind of compensating transaction towards the backend, and the client is also free to retry the same request later.

For the record: The system is implemented on the .NET stack, but I'm interested in any technology which would make implementing something like this simpler.

Also note that I would probably not actually use sleep in a loop, but some kind of Task / future abstraction.

答:
I think making clients wait for their web requests is not optimal. Even seconds are too much.

A better way in my opinion is to immediately return the web request, run the asynchronous service and either make the client poll in regular intervals or much better setup a communication channel with the client in the form of websockets for example.

Edit:

Given the new info that the client cannot be changed I think the best approach would be:

Each web node has an instance Id,
when you call the async service you give it the endpoint /callback/instanceId and this endpoint calls WebNodes[instanceId].sendResponse().

You need to keep a list of webnodes like that but you don't have to poll redis all the time to see when answer is back, doing poll and thread sleep in a loop is unecessary.


答:
Your assertions about relational databases are all true, up until the point where you have so much data you can't fit a copy of it on a single server anymore.  Then you start running into all sorts of interesting problems.  How do you split up your tables so most of your queries can run on a single server?  How many copies of the data do you make?  How do you deal with inconsistencies between those copies?  How do you keep a user's data in a data center that's relatively close to him or her geographically?

These goals often conflict with each other.  A lot of twitter users follow people from all over the world.  Should twitter's database be geographically optimized for reading tweets or writing tweets?

It turns out when you deal with that kind of scale, you start inventing solutions, adding redundancies, and imposing restrictions that very much resemble a NoSQL database.  If you can fit all your data on one box, you're only getting the restrictions and have no need for the benefits.


答:
I've been using couchdb for two years now.  It's mostly used for content management and configuration.

For hierarchical relationships are much easier to manage when you can visualize them.  For read-mostly data, it's easier to edit JSON than it is to write an UPDATE statement in many cases.  Doesn't take a programmer, actually, to edit JSON. And SQL gives you rows and columns, which you then have to map into some sort of object structure.

You also get a performance boost because you're not joining 10-20 tables on complex queries.  Couchdb views are very fast because the javascript they are based on are not executed at query time.

Most programmers understand Javascript, and most programmers struggle with SQL occasionally.

In Couchdb, a view can be thought of as an abstract of a JSON document. How the view data is structured is up to you (you are not constrained by the original hierarchy).

I wouldn't use Couchdb for highly transactional data, but for semi-static data with a parts-explosion type structure, it is MUCH easier to work with than SQL.

Note though, that there is no clear 'normalization' that can be applied (though avoiding duplication of data is a worthy goal), and there is essentially and 'optimistic' update strategy akin to optimistic locking.


答:
While I agree with your premise that NoSQL is not a panacea for all database woes, I think you misunderstand one key point.

In NoSQL database you have only one criterion you can search for effectively - the key.

This is clearly not true.

For example MongoDB supports indices. (from https://docs.mongodb.org/v3.0/core/indexes-introduction/)

Indexes support the efficient execution of queries in MongoDB. Without
  indexes, MongoDB must perform a collection scan, i.e. scan every
  document in a collection, to select those documents that match the
  query statement. If an appropriate index exists for a query, MongoDB
  can use the index to limit the number of documents it must inspect.

Indexes are special data structures [1] that store a small portion of
  the collection’s data set in an easy to traverse form. The index
  stores the value of a specific field or set of fields, ordered by the
  value of the field. The ordering of the index entries supports
  efficient equality matches and range-based query operations. In
  addition, MongoDB can return sorted results by using the ordering in
  the index.

As does couchbase (from http://docs.couchbase.com/admin/admin/Views/views-intro.html)

Couchbase views enable indexing and querying of data.

A view creates an index on the data according to the defined format
  and structure. The view consists of specific fields and information
  extracted from the objects in Couchbase.

In fact anything that calls itself a NoSQL database rather than a key-value store should realy support some kind of indexing schemes.

In fact, it is often the flexibility of these index schemes that makes NoSQL shine. In my opinion, the language used to define the NoSQL indices are often more expressive or natural than SQL, and since they usually live outside the table, you don't need to change your table schemas to support them. (Not to say you cant do similar things in SQL but to me it feels like there is a lot more hoop-jumping involved).


问:
Detecting Persons In Seats from Image
I am attempting to build an attendance program which could analyze if someone is sitting in a seat. The seats are fixed making knowing where each seat is easy, but I don't really know where to start to determine if there is someone in the seat. I have tried to do some research but it is all quite complex. I thought that starting with some basic code to highlight contrast would help such as:

```<code>#!/usr/bin/env python

from PIL import Image
from pylab import *

im = array(Image.open('images/14.jpg'))

figure('ORIG')

imshow(im)

figure('MOD')

c = copy(im)
c = c[:,:,0]
c = 255 - floor(4 * c**(1/2))
gray()
imshow(c)

show()
</code>```

Which does help:

Where do I go from here? Are there any good resources for image processing? Is what I have so far a good approach or is there a superior one?

答:
Posting here because this is too long for a comment, but I'm not sure how good of an answer it is.

I don't mean to be a Debbie Downer, but you haven't really done anything besides make the image gray-scale and increase contrast. Computer vision is a vastly complex topic -- you can't just write a simple Pythons script.

You should start with the fundamentals and work your way up. Be forewarned, computer vision involves a lot of math. You're not going to be able to figure out on your own how to recognize people in photographs on your own -- a lot of work has been put into this by very smart people over the years, and we're still not perfect at it.

Of course there are good resources; how could there not be at this point? I took a computer vision course in college, and we used Computer Visions: Algorithms and Applications by Richard Szeliski. It is a difficult read, but well-written and comprehensive. There is a chapter on human-recognition, with pointers to the literature. You should find an algorithm and try to implement it correctly (this is harder than you would think).

Also, FYI, you may want to switch from Python to Matlab. It has fantastic image processing capabilities and libraries.

Title: Detecting Persons In Seats from Image


问:
Domain Services vs. Factories vs. Aggregate Roots
After dealing with DDD for months now, I'm still confused about the general purposes of domain services, factories and aggregate roots in relation to each other, e.g. where they overlap in their responsibility.

Example: I need to 1) create a complex domain entity in a saga (process manager) which is followed by 2) a certain domain event that needs to be handled elsewhere whereas 3) the entity is clearly an aggregate root that marks a bounded context to some other entities.

The factory IS be responsible for creation of the entity/aggregate root

The service CAN create the entity, since he also throws the domain event

The service CAN act as a aggregate root (create 'subentity' of 'entity' with ID 4)

The aggregate root can create and manage 'subentities'

When I introduce the concept of a aggregate root as well as a factory to my domain, a service seems no longer needed. However, if I'm not, the service can handle everything needed as well with the knowledge and dependencies he has.

Code Example
based on the ubiquitous language of a car repair shop

```<code>public class Car : AggregateRoot {

    private readonly IWheelRepository _wheels;
    private readonly IMessageBus _messageBus;

    public void AddWheel(Wheel wheel) {
        _wheels.Add(wheel);
        _messageBus.Raise(new WheelAddedEvent());
    }

}

public static class CarFactory {

    public static Car CreateCar(string model, int amountofWheels);

}
</code>```

..or...

```<code>public class Car {

    public ICollection<Wheel> Wheels { get; set; }

}

public interface ICarService {

    Car CreateCar(args);
    void DeleteCar(args);
    Car AddWheel(int carId, Wheel wheel);

}
</code>```

答:
After dealing with DDD for months now, I'm still confused about the general purposes of domain services, factories and aggregate roots in relation to each other, e.g. where they overlap in their responsibility.

The aggregate root is responsible for ensuring that the state is consistent with the business invariant.  In CQRS lingo, you change the model by issuing commands to an aggregate root.

The domain service is a query support mechanism for the aggregates.  For example, a domain service might support a calculation.  The command handler passes the domain service to the aggregate, the aggregate (processing a command) needs the result of the calculation, so it will pass to the domain service the parts of its state that are needed as inputs to the calculation.  The domain service makes the calculation and returns the result, and the aggregate then gets to decide what to do with the result.

"Factory" can mean a couple of things.  If you are using the factory to create an entity that is within the aggregate boundary, then it's just an implementation detail -- you might use it if the construction of the object is complicated.

In some circumstances, the term "Repository" is used.  This usually implies that it is part of the persistence layer (not part of the domain model) responsible for creating aggregate roots.  Roughly, the command handler (which is part of the application) validates a command, then uses the Repository to load the aggregate root that the command is addressed to.  The command handler will then invoke the specified method on the aggregate root, passing in parameters from the command object, and possibly passing in the domain service as an argument as well.

In your examples, the key question to ask is where the responsibility for deciding whether a command should be run lives.  The application is responsible for making sure that the command is well formed (all the command data is present, the data was translated into values recognized by the domain without throwing validation errors, and so on).  But who gets to decide "No, you don't get to add a wheel right now -- the business rules don't allow it!"

In the DDD world, that is unquestionably the responsibility of the aggregate root.  So any logic to determine that should be in the aggregate root, and the ICarService goes away.

(The alternative implementation, where the aggregate exposes its state, and the cars service checks the business rules and manipulates the state of the object, is regarded as an anti pattern -- an example of an "anemic" aggregate.  "Setters" in an aggregate is a code smell.  "Getters" in an aggregate are often a code smell -- especially in CQRS, where the responsibility for supporting queries is supposed to be "somewhere else" -- in the read model.)


问:
License for forked software library
I'm writing an application for which I want to use some 20-year old open source code. The original code is kind of antiquated in certain aspects, for example, it's written in K&R C rather than C99; it has to manually compute certain things that are now defined in C standard library headers; there are no tests; and so on and so forth. All that the original code says by way of licensing is "Released in the public domain by so and so"; no BSD/MIT/GPL, just "public domain".

In addition to my original application, I'd like to release my own fork of the legacy library with a few of changes, chiefly to modernize it and to add unit tests. I've contacted the original author about it, but have not heard back from him. Am I within my rights to release my fork with a liberal (BSD/MIT) license? I'm not trying to make any money off of either my update of the legacy library or off of the application built on top of it.

答:
IANAL but I did find an interesting piece by an IP lawyer in the US, essentially declaring that the 'public domain is not a license':

Just as there is nothing in the law that permits a person to dump
  personal property on a public highway, there is nothing that permits
  the dumping of copyrighted works into the public domain, except as
  happens in due course when any applicable copyrights expire. Until
  those copyrights expire, no mechanism is in the law by which an owner
  of software can simply elect to place it in the public domain.

It would appear that these caveats relate more to fear that the public-domain-ness of the work could be retracted any time by the original author.

Title: License for forked software library


答:
Relevant XKCD:

On the upside, you might be able to degenerate this task into hand-created bounding boxes for each seat. Take each seat and generate a "base line" from a photo that has no people in any seat under normal expected lighting conditions.

Then when a picture to record attendance is taken - and assuming no smarty pants moves the camera or plays with the lights too much - you can compare each "seat" to the base line. If some measure of difference is sufficiently high (that's not a trivial bit of math), someone must be sitting there. Or maybe someone's shirt looks a bit too much like a seat, or someone sits their coat on the seat, or someone's arm extends too much into the seat.

You can certainly get something to "work", for some arbitrarily low value of work. But for anything else you'll need...well, probably months of hard work and an intensive nose-dive into computer vision. You'll need to expect that the result might be cool (if it ever even works at all), but will very likely be terribly unreliable and unusable outside an academic context.

Reference requests are of course off-topic here, but you might want to consider Programming Computer Vision with Python...but I'm going to go out on a limb and guess you can't do an import see_people and be done in an afternoon...but it's python so I guess I can't say for sure (I'm pretty sure you can't, though).

Title: Detecting Persons In Seats from Image

Body:
I am attempting to build an attendance program which could analyze if someone is sitting in a seat. The seats are fixed making knowing where each seat is easy, but I don't really know where to start to determine if there is someone in the seat. I have tried to do some research but it is all quite complex. I thought that starting with some basic code to highlight contrast would help such as:

```<code>#!/usr/bin/env python

from PIL import Image
from pylab import *

im = array(Image.open('images/14.jpg'))

figure('ORIG')

imshow(im)

figure('MOD')

c = copy(im)
c = c[:,:,0]
c = 255 - floor(4 * c**(1/2))
gray()
imshow(c)

show()
</code>```

Which does help:

Where do I go from here? Are there any good resources for image processing? Is what I have so far a good approach or is there a superior one?


问:
Interface Segregation Principle: What to do if interfaces have significant overlap?
From Agile Software Development, Principles, Patterns, and Practices: Pearson New International Edition:

Sometimes, the methods invoked by different groups of clients will overlap. If the overlap is small, then the
  interfaces for the groups should remain separate. The common functions should be declared in all the overlapping
  interfaces. The server class will inherit the common functions from each of those interfaces, but it will implement
  them only once.

Uncle Bob, talks about the case when there is minor overlap.

What should we do if there is significant overlap?

Say we have

```<code>Class UiInterface1;
Class UiInterface2;
Class UiInterface3;

Class UiIterface : public UiInterface1, public UiInterface2, public UiInterface3{};
</code>```

What should we do if there is significant overlap between UiInterface1 and UiInterface2?

答:
Casting

This is almost certainly going to be a complete tangent to the cited book's approach, but one way to conform better to ISP is to embrace a casting mindset at one central area of your codebase using a QueryInterface COM-style approach.

A lot of the temptations to design overlapping interfaces in a pure interface context often comes from the desire to make interfaces "self-sufficient" more than performing one precise, sniper-like responsibility.

For example, it might seem odd to design client functions like this:

```<code>// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `position` and `parenting` parameters should point to the
// same object.
Vec2i abs_position(IPosition* position, IParenting* parenting)
{
     const Vec2i xy = position->xy();
     auto parent = parenting->parent();
     if (parent)
     {
         // If the entity has a parent, return the sum of the
         // parent position and the entity's local position.
         return xy + abs_position(dynamic_cast<IPosition*>(parent),
                                  dynamic_cast<IParenting*>(parent));
     }
     return xy;
}
</code>```

... as well as quite ugly/dangerous, given that we're leaking the responsibility to do error-prone casting to the client code using these interfaces and/or passing the same object as an argument multiple times to multiple parameters of the same function. So we end up often wanting to design a more diluted interface which consolidates the concerns of IParenting and IPosition in one place, like IGuiElement or something like that which then becomes susceptible to overlapping with the concerns of orthogonal interfaces which will likewise be tempted to have more member functions for the same "self-sufficiency" reason.

Mixing Responsibilities vs. Casting

When designing interfaces with a totally distilled, ultra-singular responsibility, the temptation will often be to either accept some downcasting or consolidate interfaces to fulfill multiple responsibilities (and therefore tread on both ISP and SRP).

By using a COM-style approach (just the QueryInterface part), we play to the downcasting approach but consolidate the casting to one central place in the codebase, and can do something more like this:

```<code>// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `obj` should implement `IPosition` and optionally `IParenting`.
Vec2i abs_position(Object* obj)
{
     // `Object::query_interface` returns nullptr if the interface is
     // not provided by the entity. `Object` is an abstract base class
     // inherited by all entities using this interface query system.
     IPosition* position = obj->query_interface<IPosition>();
     assert(position && "obj does not implement IPosition!");
     const Vec2i xy = position->xy();

     IParenting* parenting = obj->query_interface<IParenting>();
     if (parenting && parenting->parent()->query_interface<IPosition>())
     {
         // If the entity implements IParenting and has a parent,
         // return the sum of the parent position and the entity's
         // local position.
         return xy + abs_position(parenting->parent());
     }
     return xy;
}
</code>```

... of course hopefully with type-safe wrappers and all that you can build centrally to get something safer than raw pointers.

With this, the temptation to design overlapping interfaces is often mitigated to the absolute minimum. It allows you to design interfaces with very singular responsibilities (sometimes just one member function inside) that you can mix and match all you like without worrying about ISP, and getting the flexibility of pseudo-duck typing at runtime in C++ (though of course with the trade-off of runtime penalties to query objects to see if they support a particular interface). The runtime part can be important in, say, a setting with a software development kit where the functions will not have the compile-time information of plugins in advance that implement these interfaces.

Templates

If templates are a possibility (we have the necessary compile-time info in advance which is not lost by the time we get hold of an object, i.e.), then we can simply do this:

```<code>// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `obj` should have `position` and `parent` methods.
template <class Entity>
Vec2i abs_position(Entity& obj)
{
     const Vec2i xy = obj.xy();
     if (obj.parent())
     {
         // If the entity has a parent, return the sum of the parent
         // position and the entity's local position.
         return xy + abs_position(obj.parent());
     }
     return xy;
}
</code>```

... of course in such a case, the parent method would have to return the same Entity type, in which case we probably want to avoid interfaces outright (since they'll often want to lose type information in favor of working with base pointers).

Entity-Component System

If you start pursuing the COM-style approach further from a flexibility or performance standpoint, you'll often end up with an entity-component system similar to what game engines apply in the industry. At that point you'll be going completely perpendicular to a lot of object-oriented approaches, but ECS might be applicable to GUI design (one place I've contemplated using ECS outside of a scene-oriented focus, but considered it too late after settling on a COM-style approach to try there).

Note that this COM-style solution is completely out there as far as GUI toolkit designs go, and ECS would be even more, so it's not something which will be backed by a lot of resources. Yet it'll definitely allow you to mitigate the temptations to design interfaces which have overlapping responsibilities to an absolute minimum, often making it a non-concern.

Pragmatic Approach

The alternative, of course, is relax your guard a bit, or design interfaces at a granular level and then start inheriting them to create coarser interfaces that you use, like IPositionPlusParenting which derives from both IPosition and IParenting (hopefully with a better name than that). With pure interfaces, it shouldn't violate ISP quite as much as those monolithic deep-hierarchical approaches commonly applied (Qt, MFC, etc, where the documentation often feels the need to hide irrelevant members given the excessive level of violating ISP with those kinds of designs), so a pragmatic approach might simply accept some overlap here and there. Yet this kind of COM-style approach avoids the need to create consolidated interfaces for every combination you'll ever use. The "self-sufficiency" concern is completely eliminated in such cases, and that will often eliminate the ultimate source of temptation to design interfaces which have overlapping responsibilities which want to fight with both SRP and ISP.

Body:
Casting

This is almost certainly going to be a complete tangent to the cited book's approach, but one way to conform better to ISP is to embrace a casting mindset at one central area of your codebase using a QueryInterface COM-style approach.

A lot of the temptations to design overlapping interfaces in a pure interface context often comes from the desire to make interfaces "self-sufficient" more than performing one precise, sniper-like responsibility.

For example, it might seem odd to design client functions like this:

// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `position` and `parenting` parameters should point to the
// same object.
Vec2i abs_position(IPosition* position, IParenting* parenting)
{
     const Vec2i xy = position->xy();
     auto parent = parenting->parent();
     if (parent)
     {
         // If the entity has a parent, return the sum of the
         // parent position and the entity's local position.
         return xy + abs_position(dynamic_cast<IPosition*>(parent),
                                  dynamic_cast<IParenting*>(parent));
     }
     return xy;
}

... as well as quite ugly/dangerous, given that we're leaking the responsibility to do error-prone casting to the client code using these interfaces and/or passing the same object as an argument multiple times to multiple parameters of the same function. So we end up often wanting to design a more diluted interface which consolidates the concerns of IParenting and IPosition in one place, like IGuiElement or something like that which then becomes susceptible to overlapping with the concerns of orthogonal interfaces which will likewise be tempted to have more member functions for the same "self-sufficiency" reason.

Mixing Responsibilities vs. Casting

When designing interfaces with a totally distilled, ultra-singular responsibility, the temptation will often be to either accept some downcasting or consolidate interfaces to fulfill multiple responsibilities (and therefore tread on both ISP and SRP).

By using a COM-style approach (just the QueryInterface part), we play to the downcasting approach but consolidate the casting to one central place in the codebase, and can do something more like this:

// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `obj` should implement `IPosition` and optionally `IParenting`.
Vec2i abs_position(Object* obj)
{
     // `Object::query_interface` returns nullptr if the interface is
     // not provided by the entity. `Object` is an abstract base class
     // inherited by all entities using this interface query system.
     IPosition* position = obj->query_interface<IPosition>();
     assert(position && "obj does not implement IPosition!");
     const Vec2i xy = position->xy();

     IParenting* parenting = obj->query_interface<IParenting>();
     if (parenting && parenting->parent()->query_interface<IPosition>())
     {
         // If the entity implements IParenting and has a parent,
         // return the sum of the parent position and the entity's
         // local position.
         return xy + abs_position(parenting->parent());
     }
     return xy;
}

... of course hopefully with type-safe wrappers and all that you can build centrally to get something safer than raw pointers.

With this, the temptation to design overlapping interfaces is often mitigated to the absolute minimum. It allows you to design interfaces with very singular responsibilities (sometimes just one member function inside) that you can mix and match all you like without worrying about ISP, and getting the flexibility of pseudo-duck typing at runtime in C++ (though of course with the trade-off of runtime penalties to query objects to see if they support a particular interface). The runtime part can be important in, say, a setting with a software development kit where the functions will not have the compile-time information of plugins in advance that implement these interfaces.

Templates

If templates are a possibility (we have the necessary compile-time info in advance which is not lost by the time we get hold of an object, i.e.), then we can simply do this:

// Returns the absolute position of an entity as the sum
// of its own position and the position of its ancestors.
// `obj` should have `position` and `parent` methods.
template <class Entity>
Vec2i abs_position(Entity& obj)
{
     const Vec2i xy = obj.xy();
     if (obj.parent())
     {
         // If the entity has a parent, return the sum of the parent
         // position and the entity's local position.
         return xy + abs_position(obj.parent());
     }
     return xy;
}

... of course in such a case, the parent method would have to return the same Entity type, in which case we probably want to avoid interfaces outright (since they'll often want to lose type information in favor of working with base pointers).

Entity-Component System

If you start pursuing the COM-style approach further from a flexibility or performance standpoint, you'll often end up with an entity-component system similar to what game engines apply in the industry. At that point you'll be going completely perpendicular to a lot of object-oriented approaches, but ECS might be applicable to GUI design (one place I've contemplated using ECS outside of a scene-oriented focus, but considered it too late after settling on a COM-style approach to try there).

Note that this COM-style solution is completely out there as far as GUI toolkit designs go, and ECS would be even more, so it's not something which will be backed by a lot of resources. Yet it'll definitely allow you to mitigate the temptations to design interfaces which have overlapping responsibilities to an absolute minimum, often making it a non-concern.

Pragmatic Approach

The alternative, of course, is relax your guard a bit, or design interfaces at a granular level and then start inheriting them to create coarser interfaces that you use, like IPositionPlusParenting which derives from both IPosition and IParenting (hopefully with a better name than that). With pure interfaces, it shouldn't violate ISP quite as much as those monolithic deep-hierarchical approaches commonly applied (Qt, MFC, etc, where the documentation often feels the need to hide irrelevant members given the excessive level of violating ISP with those kinds of designs), so a pragmatic approach might simply accept some overlap here and there. Yet this kind of COM-style approach avoids the need to create consolidated interfaces for every combination you'll ever use. The "self-sufficiency" concern is completely eliminated in such cases, and that will often eliminate the ultimate source of temptation to design interfaces which have overlapping responsibilities which want to fight with both SRP and ISP.


问:
What does it mean to declare a volatile variable?
Many low level programs use the volatile keyword for types for memory mapping and such, however I'm sort of confused as to what it REALLY does in the background.  In other words, what does it mean when the compiler doesn't "optimize away" the memory address?

答:
volatile means some other processor or I/O device or something can change the variable out from under you.

With an ordinary variable, your program's steps are the only thing that will change it. So for instance if you read 5 from a variable and you don't change it, it'll still contain 5. Since you can rely on that, your program doesn't have to take the time to read the variable again next time you want to use it. The C++ compiler is smart to generate code that just remembers the 5.

But you could read it as 5, then perhaps the system loads data from disk into that memory, changing it to 500. If you want your program to read the fresh value 500, you need the compiler to not be too smart about using the previously-read 5. You need to tell it to reload the value every time. That's what volatile does.

An analogy for 5-year olds

Let's say you put a big sheet of paper on a table. In one corner of the paper, you write down the current score of an ongoing game, 3 to 4. Then you go to the opposite side of the table and start writing a story about the game. Your friend who's watching the game updates the score in that corner as the game proceeds. She erases 3 to 4 and writes 3 to 5.

When you go to put the game score into your story, you could either:

Write down the last score you read, 3 to 4, merrily assuming it didn't change (or not minding if it did), or

Walk around to the opposite side of the table to read the current score (which happens to be 3 to 5 now), and walk back. That's how a volatile variable acts.


问:
Domain Driven Design - designing Aggregate Roots
Imagine you're designing an application for organizing Workshops.
Workshop itself is quite complicated - it acts as a state machine with multiple possible states and transitions between them.

We're using DDD, CQRS and Event Sourcing.

And now, the use cases:

1) one should be able to assign TodoItems to a Workshop and perform CRUD operations on them

2) one should be able to submit additional Resources for a given Workshop (slides, photos, zip/tar.gz archives etc.) and also perform CRUD operations on them

3) one should be able to add a PossibleTerm (with a Lecturer, Date and Room specified) to a Workshop. Also, there should be a way to modify/delete them.

Later on there is a Voting on those terms, system tries to book room for few terms (let's say, for two terms) that won Voting and Users should be split between those Terms.

Here I have following solutions:

1) Model TodoItems, Resources and PossibleTerms as a part of a Workshop.

Pros:

everything is where it conceptually belongs to

no eventual consistency (not a big problem probably...)

Cons:

Workshop aggregate is cluttered with all those CRUD methods for all stuff.

According to Implementing Domain Driven Design by Vaughn Vernon, creating huge  aggregate roots is an antipattern

2) Create aggregate roots for holding all those "items" (e.g. TodoItemList, PossibleTermList, Resources or whatever you're gonna call them).

Pros:

Workshop aggregate is smaller, it's not related with those Lists at all (the only connection would be that for example PossibleTermList holds a reference to WorkshopId)

Cons:

when CreateWorkshopCommand arrives, you're forced to create not only a Workshop, but also additional aggregates as they need to exist after Workshop creation. To avoid this, there could be some kind of CreationalSaga, listening for WorkshopCreatedEvent and producing proper commands to create relatives, but I don't think this is a good idea.

3) Model PossibleTerms, Resource, TodoItem as aggregate roots itself

Pros:

no need for creating 'aggregate wrappers' (as in 2))

Cons:

again, you're cluttering Workshop with at least factory methods

it is hard for me to implement term-choosing feature - for counting votes and later on assigning User to a particular Term you need to query your repository for all Terms matching your Workshop ID and so on - this leads to operations on multiple aggregates within one transaction.

Question: which solution sound best for you? Maybe you have differents ideas?

(Bonus question: I need to perform CRUD operations on individual Resources/TodoItem. I'd rather model then as value objects, but how are they then identified?)

答:
It seems to me that the worst choice is trying to model Lists.  The point of an aggregate is to ensure a business invariant.  What invariant does a collection enforce?  If the addItem command is invoked, is the list allowed to say no?

Making all of the entities part of the Workshop aggregate is a natural fit -- especially if the other entities don't "make sense" outside of the context of a workshop.  If you've got a business invariant that constrains multiple entities, then you need those entities to be part of the same aggregate.

But it does mean that you've got contention: Alice, working on ToDoItem:1, and Bob, working on ToDoItem:2, can't save their changes at "the same time" because of the write conflict.

Option #3, where each entity is its own aggregate, takes care of the contention problem.  Alice and Bob can commit their changes without conflict, the ToDoItems can enforce their own invariant.

It is true that the entities have different life cycles, and you need to be ok with that.  You also need to work a bit harder to enforce an invariant that spans multiple aggregates.

The ubiquitous language should offer hints as to whether the components of a workshop need to be part of the same transaction as the workshop itself (if there's already a business process for mitigating contingencies, then you probably don't have a transaction consistency requirement.

That said, if your domain is CRUD (if the model isn't allowed to veto the judgment of the human operators), then trying to invent "aggregates" out of nothing may not be the best play.


问:
Where should an object in CQRS+ES be fully initialized: in the constructor, or when applying the first event?
There appears to be widespread agreement in the OOP community that the class constructor should not leave an object partly or even fully uninitialized.

What do I mean by "initialization"? Roughly speaking, the atomic process that brings a newly created object into a state where all of its class invariants hold. It should be the first thing that happens to an object, (it should only run once per object,) and nothing should be permitted to get hold of an un-initialized object. (Thus the frequent advice to perform object initialization right in the class constructor. For the same reason, Initialize methods are often frowned upon, as these break apart the atomicity and make it possible to get hold of, and use, an object that is not yet in a well-defined state.)

Problem: When CQRS is combined with event sourcing (CQRS+ES), where all state changes of an object are caught in an ordered series of events (event stream), I am left wondering when an object actually reaches a fully-initialized state: At the end of the class constructor, or after the very first event has been applied to the object?

Note: I'm refraining from using the term "aggregate root". If you prefer, substitute it whenever you read "object".

Example for discussion: Assume that each object is uniquely identified by some opaque Id value (think GUID). An event stream representing that object's state changes can be identified in the event store by the same Id value: (Let's not worry about correct event order.)

interface IEventStore
{
    IEnumerable<IEvent> GetEventsOfObject(Id objectId);
}
```

<p>Assume further that there are two object types <code>Customer</code> and <code>ShoppingCart</code>. Let's focus on <code>ShoppingCart</code>: When created, shopping carts are empty and must be associated with exactly one customer. That last bit is a class invariant: A <code>ShoppingCart</code> object that is not associated to a <code>Customer</code> is in an invalid state.</p>

<p>In traditional OOP, one might model this in the constructor:</p>

<pre class="lang-cs prettyprint-override"><code>partial class ShoppingCart
{
    public Id Id { get; private set; }
    public Customer Customer { get; private set; }

    public ShoppingCart(Id id, Customer customer)
    {
        this.Id = id;
        this.Customer = customer;
    }
}
</code>```

I am however at a loss how to model this in CQRS+ES without ending up with deferred initialization. Since this simple bit of initialization is effectively a state change, wouldn't it have to be modelled as an event?:

partial class CreatedEmptyShoppingCart
{
    public ShoppingCartId { get; private set; }
    public CustomerId { get; private set; }
}
// Note: `ShoppingCartId` is not actually required, since that Id must be
// known in advance in order to fetch the event stream from the event store.
```

This would obviously have to be the very first event in any ShoppingCart object's event stream, and that object would only be initialized once the event were applied to it.

So if initialization becomes part of the event stream "playback" (which is a very generic process that would likely work the same, whether for a Customer object or a ShoppingCart object or any other object type for that matter)…

Should the constructor be parameter-less and do nothing, leaving all work to some void Apply(CreatedEmptyShoppingCart) method (which is much the same as the frowned-upon Initialize())?

Or should the constructor receive an event stream and play it back (which makes initialization atomic again, but means that each class' constructor contains the same generic "play back & apply" logic, i.e. unwanted code duplication)?

Or should there be both a traditional OOP constructor (as shown above) that properly initializes the object, and then all events but the first are void Apply(…)-ied to it?

I do not expect of answer to provide a fully working demo implementation; I'd already be very happy if someone could explain where my reasoning is flawed, or whether object initialization really is a "pain point" in most CQRS+ES implementations.

答:
Note: if you don't want to use aggregate root, "entity" encompasses most of what you are asking about here while side stepping the concerns about transaction boundaries.

Here's another way of thinking about it: an entity is identity + state.  Unlike a value, an entity is the same guy even when his state changes.

But the state itself can be thought of as a value object.  By this I mean, the state is immutable; the history of the entity is a transition from one immutable state to the next -- each transition corresponds to an event in the event stream.

```<code>State nextState = currentState.onEvent(e);
</code>```

The onEvent() method is a query, of course -- currentState isn't changed at all, instead currentState is calculating the arguments used to create the nextState.

Following this model, all instances of Shopping cart can be thought of as starting from the same seed value....

```<code>State currentState = ShoppingCart.SEED;
for (Event e : history) {
    currentState = currentState.onEvent(e);
}

ShoppingCart cart = new ShoppingCart(id, currentState);
</code>```

Separation of concerns - the ShoppingCart processes a command to figure out what event should be next; the ShoppingCart State knows how to get to the next state.

Body:
Note: if you don't want to use aggregate root, "entity" encompasses most of what you are asking about here while side stepping the concerns about transaction boundaries.

Here's another way of thinking about it: an entity is identity + state.  Unlike a value, an entity is the same guy even when his state changes.

But the state itself can be thought of as a value object.  By this I mean, the state is immutable; the history of the entity is a transition from one immutable state to the next -- each transition corresponds to an event in the event stream.

State nextState = currentState.onEvent(e);

The onEvent() method is a query, of course -- currentState isn't changed at all, instead currentState is calculating the arguments used to create the nextState.

Following this model, all instances of Shopping cart can be thought of as starting from the same seed value....

State currentState = ShoppingCart.SEED;
for (Event e : history) {
    currentState = currentState.onEvent(e);
}

ShoppingCart cart = new ShoppingCart(id, currentState);

Separation of concerns - the ShoppingCart processes a command to figure out what event should be next; the ShoppingCart State knows how to get to the next state.


问:
Understanding Generic Repository Pattern
I am trying to understand the generic repository implementation.  I have seen this line (or similar to it) in many examples:

```<code>public interface IRepository<TEntity> : IDisposable where TEntity : IEntity
</code>```

Can some explain the parts to this?  Also, I'm having a hard time understanding the IEntity and how it plays into the interface.

```<code>public interface IEntity
    {
        int Id { get; set; }
    }

public interface IRepository<TEntity> : IDisposable where TEntity : IEntity
    {
        IQueryable<TEntity> GetAll();
        void Delete(TEntity entity);
        void Add(TEntity entity);
    }
</code>```

答:
Starting from the bottom; if you want to know if two values are the same, you just check that they are equal.  Think about 7.  No matter what happens between now and the heat death of the universe, 7 is never going to equal 2.

On the other hand, entities do change.  Today, Bob is 2.  In the due course of time, Bob will be 7.  Even though he isn't 2 any more, he is still Bob.  That's identity, the thing that we use to link different states of history across time.
Put a slightly different way, if we want to know if "Bob" and "the kid in the hat" are the same person, we don't ask their ages, or their names (which could be coincident), we check their identity.

So IEntity is an interface defines an identity.  In sample implementations like this, the actual thing you are modeling (Person) will implement IEntity, and will have some id:int as part of its state.  (A settable id is kind of weird though; the id is usually immutable).

The IRepository interface, now, promises that it can be disposed, and also that the methods that is supports all corresponding to a consistent (but as yet unspecified) type.  However, the type is constrained: this interface for repositories promises that the type is going to be one that implements IEntity.

so you might see

```<code>class Dog : IEntity {
    ...
}

class Person : IEntity {
    ...
}
</code>```

and matching repositories

```<code>class DogRepository : IRepository<Dog>
{
    IQueryable<Dog> GetAll();
    void Delete(Dog entity);
    void Add(Dog entity);
}
</code>```

But you aren't going to see, for instance

```<code>class AgeRepository : IRepository<Age> {
    ...
}
</code>```

Because Age, being a value, isn't going to implement IEntity.

Body:
Starting from the bottom; if you want to know if two values are the same, you just check that they are equal.  Think about 7.  No matter what happens between now and the heat death of the universe, 7 is never going to equal 2.

On the other hand, entities do change.  Today, Bob is 2.  In the due course of time, Bob will be 7.  Even though he isn't 2 any more, he is still Bob.  That's identity, the thing that we use to link different states of history across time.
Put a slightly different way, if we want to know if "Bob" and "the kid in the hat" are the same person, we don't ask their ages, or their names (which could be coincident), we check their identity.

So IEntity is an interface defines an identity.  In sample implementations like this, the actual thing you are modeling (Person) will implement IEntity, and will have some id:int as part of its state.  (A settable id is kind of weird though; the id is usually immutable).

The IRepository interface, now, promises that it can be disposed, and also that the methods that is supports all corresponding to a consistent (but as yet unspecified) type.  However, the type is constrained: this interface for repositories promises that the type is going to be one that implements IEntity.

so you might see

class Dog : IEntity {
    ...
}

class Person : IEntity {
    ...
}

and matching repositories

class DogRepository : IRepository<Dog>
{
    IQueryable<Dog> GetAll();
    void Delete(Dog entity);
    void Add(Dog entity);
}

But you aren't going to see, for instance

class AgeRepository : IRepository<Age> {
    ...
}

Because Age, being a value, isn't going to implement IEntity.


问:
Amazon Echo Development on LAN
I have followed the tutorial from Amazon to get started with the Echo.  I made a skill and setup an application server on their AWS Lambda for basic testing.

I have a few questions about the Echo, skills and AWS Lambda:

When I create a skill does it reside on the Echo itself or does the echo have to go out to the internet and retrieve it?  If it goes out to the internet, is there any way I can develop a skill and store it locally on the Echo? (I don't want the Echo to use the internet).

The last part to the process for answering and processing a vocal command by the echo is an application server, usually AWS Lambda provided by Amazon.  What do I need to setup an application server on the LAN? (I don't want the Echo to use the internet).  I understand they had a software called JAWS(now serverless-serve) but I feel like I'm missing something else.

I also see I'm able to export an AmazonMachineImage.  Does this mean I can set it up on my own VM?  Or is it for AWS only?

答:
When I create a skill does it reside on the Echo itself or does the echo have to go out to the internet and retrieve it?

The code driving the skill is hosted either as a Lambda function (in which case it resides in AWS' infrastructure), or as a REST endpoint hosted on your own infrastructure.

...is there any way I can develop a skill and store it locally on the Echo? (I don't want the Echo to use the internet).

No, your Echo always uses the public Internet for communication. Even if you were able to host your Skill's code on infrastructure local to your network, Amazon's systems must act as an intermediary for authentication, voice parsing, and so on. Not to mention that the only way to load a Skill on to your device is via the Alexa Store (or, in during development, through the Amazon Developer Console).

I understand they had a software called JAWS(now serverless-serve) but I feel like I'm missing something else.

JAWS (now called serverless) is a scaffolding tool and framework that helps you deploy code to Lambda, API Gateway, and DynamoDB. It's scope is a little more broad than what one requires to build a simple Alexa Skill.

You might be interested in alexa-app, a simple Javascript abstraction which is simpler to use than Amazon's Alexa API. (Full disclosure, I'm a maintainer of alexa-app).

Title: Amazon Echo Development on LAN


问:
How to communicate between cooperative tasks "first me, then you"?
I am in the final stages of development of a simple embedded system. The device performs PID coefficient estimation and then instantiates a PID controller with the estimated coefficients.

The architecture of the program is cooperative multitasking. The file main.c defines a series of void task_...(void) functions and calls them one after the other upon each timer tick.

I would like to perform the operation void task_pid_tune(void) for as long as it takes. Then, I want to perform the operation pid_configure(coefficients) once. From then on, I want to perform the operation pid_run() instead of the previous two.

Currently all 4 tasks (pid_tune() is not implemented yet) run in an extremely simple way

```<code>while( wait_for_timer_tick() )
{
    task1();
    task2();
    task3();
    task4();
 }
</code>```

I would prefer to keep it that way, instead of writing logic inside this function. I am writing in C99. I am fine with static and global variables, as well as public getters.

Please advise how shall I implement communication between void task_pid_tune(void) and void task_pid_run(void). Please keep in mind that the latter will be called much more often than the former, hence performance is important in that case.

Clarification.

No task is allowed to be blocking. In other words, all scheduled tasks must return before the next timer tick.

Consequently, the required execution order is:

```<code>    task1();
    task2();
    task3();
    task4();
    task_pid_tune();
    // this loops until pid tuning is ready
    // that is, pid_tune() samples some signal once per system tick
    // and at some point, it gets happy and provides estimated pid coefficients

    task1();
    task2();
    task3();
    task4();
    task_pid_configure();  // this runs only one

    task1();
    task2();
    task3();
    task4();
    task_pid_run();
    // This loops forever.
</code>```

答:
An easy and readable way to switch between the PID tasks is to use a function pointer, like this:

```<code>void (*pid_task)() = &task_pid_tune;

int main() {
  //... startup code

  while( wait_for_timer_tick() )
  {
    task1();
    task2();
    task3();
    task4();
    pid_task();
  }
}

void task_pid_run()
{
  //... run the PID
}

void task_pid_configure()
{
  //... perform configuration

  pid_task = &task_pid_run;
}

void task_pid_tune()
{
  //... perform PID tuning

  if (tuning done)
    pid_task = &task_pid_configure;
}
</code>```

Here the pid_tune and pid_configure tasks determine when a new task should replace them and which task that should be. The main execution loop just executes whichever PID task is currently assigned to the function pointer for execution.

Body:
An easy and readable way to switch between the PID tasks is to use a function pointer, like this:

void (*pid_task)() = &task_pid_tune;

int main() {
  //... startup code

  while( wait_for_timer_tick() )
  {
    task1();
    task2();
    task3();
    task4();
    pid_task();
  }
}

void task_pid_run()
{
  //... run the PID
}

void task_pid_configure()
{
  //... perform configuration

  pid_task = &task_pid_run;
}

void task_pid_tune()
{
  //... perform PID tuning

  if (tuning done)
    pid_task = &task_pid_configure;
}

Here the pid_tune and pid_configure tasks determine when a new task should replace them and which task that should be. The main execution loop just executes whichever PID task is currently assigned to the function pointer for execution.


答:
In my programming environment (MacOS X / iOS) it is quite trivial - I create a semaphore, start the asynchronous request, the asynchronous request will unlock the semaphore when it's done, and then I lock the semaphore (which will wake up when the asynchronous request is done). Maybe six or eight lines of code. </p>

<p>On the other hand, synchronous requests, while being easier to handle, have the negative effect that you have to wait for the result, which for anything trying to show a user interface is an absolute no-no. So such synchronous requests should only be used on a background thread. </p>

Title: Synchronous facade hiding asynchronous web service

Body:
I'm maintaining a system providing a typical synchronous web service REST API. It routes each request to one of several possible backend services. Now there is a new backend I want to use, but there is a problem - it has an asynchronous API - I will need to do a request, and then wait for a callback request.

I'm thinking that it's possible to hide this asynchronicity behind a syncronizing facade. This facade will call the async API and then put the incoming web requests "on hold". When the callback endpoint is called a few seconds later it will resume the correct web request and respond to the original caller. If the callback does not arrive within let's say 10 seconds I will stop waiting, and respond with an error.

A complicating factor is that I have several web nodes behind a load balancer, but I'm thinking I can use a distributed storage solution like Redis to publish incoming callbacks, so that the correct web node can pick it up.

The question is: Is this feasible? Or do you believe it will be really difficult to get it to work properly? Any experience you can share or links to relevant resources would be much appreciated. Any other aproach to using an asynchronous web service in another service which needs to be syncronous?

Some pseudo code of what I'm thinking:

```<code>define rest-endpoint "/foobar" {
    call async backend
    until 10 seconds has passed {
        sleep 1 second
        result = redis.get(some_correlation_id)
        if (result)
            return new Response(result)
    }
    return new Response("failed")
}

define rest-endpoint "/callback" {
    redis.push(some_correlation_id)
}
</code>```

Note that if I time out and respond "failed" I don't (in my particular case) have to create any kind of compensating transaction towards the backend, and the client is also free to retry the same request later.

For the record: The system is implemented on the .NET stack, but I'm interested in any technology which would make implementing something like this simpler.

Also note that I would probably not actually use sleep in a loop, but some kind of Task / future abstraction.

Body:
In my programming environment (MacOS X / iOS) it is quite trivial - I create a semaphore, start the asynchronous request, the asynchronous request will unlock the semaphore when it's done, and then I lock the semaphore (which will wake up when the asynchronous request is done). Maybe six or eight lines of code.

On the other hand, synchronous requests, while being easier to handle, have the negative effect that you have to wait for the result, which for anything trying to show a user interface is an absolute no-no. So such synchronous requests should only be used on a background thread.


答:
volatile means two things:

The value of the variable may change without any code of yours changing it. Therefore whenever the compiler reads the value of the variable, it may not assume that it is the same as the last time it was read, or that it is the same as the last value stored, but it must be read again.

The act of storing a value to a volatile variable is a "side effect" which can be observed from the outside, so the compiler is not allowed to remove the act of storing a value; for example if two values are stored in a row, then the compiler must actually store the value twice.

As an example:

```<code>i = 2;
i = i;
</code>```

The compiler must store the number two, read the variable i, store the variable that it read into i.

There is another situation: If a function uses setjmp and then longjmp is called, all volatile local variables of the function are guaranteed to have the last value stored - this is not the case with non-volatile local variables.

Title: What does it mean to declare a volatile variable?

Body: Many low level programs use the volatile keyword for types for memory mapping and such, however I'm sort of confused as to what it REALLY does in the background.  In other words, what does it mean when the compiler doesn't "optimize away" the memory address?

Body:
volatile means two things:

The value of the variable may change without any code of yours changing it. Therefore whenever the compiler reads the value of the variable, it may not assume that it is the same as the last time it was read, or that it is the same as the last value stored, but it must be read again.

The act of storing a value to a volatile variable is a "side effect" which can be observed from the outside, so the compiler is not allowed to remove the act of storing a value; for example if two values are stored in a row, then the compiler must actually store the value twice.

As an example:

i = 2;
i = i;

The compiler must store the number two, read the variable i, store the variable that it read into i.

There is another situation: If a function uses setjmp and then longjmp is called, all volatile local variables of the function are guaranteed to have the last value stored - this is not the case with non-volatile local variables.


问:
What are the security risks/vulnerabilities every C programmer must be aware of?
There are many security risks coming from having close contact to the hardware as opposed to using well-tested and proved APIs from high level programming languages. It is much easier to cause a buffer overflow in C than in a language such as Java.

What are the risks or vulnerabilities (e.g. buffer overflows) that every C programmer should be aware of (I.E. vulnerabilities relevant to C programmers)? What problems could these lead to? How to avoid them, and what are common mistakes causing these to occur in programs?

答:
There is just one security risk: The fact that there are people outside who will do their best to catch any vulnerability in your software and to exploit it for their own gain. Everything else follows from there.

So when you think "nobody in their right mind would ... ", then you need to think immediately "except someone who wants to hack into other people's computers would do exactly that".

The biggest consequence is that whenever you react to events outside (for example, by processing data delivered from the outside), you must assume that this data was under control of your worst enemy.


答:
(non-)volatile is a hint for the compiler how to optimize code (from generated assembly-code point of view):

non volatile means that your current compiler decides where the variable will be located or how the variable-s value is teransfered to a subroutine

in a fixed memory adress,

on the stack [relative to the processors current stackpointer],

on the heap [relative to the processors current basepointer],

in a processor register,

...

volatile means that the compiler cannot optimize the variable because something else outside of the main-cpu-s controll (i.e. a seperate io-processer) can change this value.


答:
Abstract explanation

Both C and C++ have a concept of an abstract machine. When the code uses the value of some variable, the abstract machine says the implementation has to access the value of that variable. Code of the form statement_A; statement_B; statement_C; has to be executed in exactly the order specified. Expressions common to those three statements must be re-calculated each time they occur.

Per the abstract machines, given the sequence of statements statement_A; statement_B; statement_C;, the implementation must first perform statement_A in its entirety, then statement_B, and finally  statement_C. The implementation cannot remember that you assigned age the value of 5 . Every statement that references age must instead access the value of that variable.

There would be no need for the volatile keyword if implementations strictly executed C or C++ code per the abstract machine specifications. The C and C++ abstract machines have no concept of registers, no concept of common subexpressions, and execution order is strict.

Both languages also have as-if rules. An implementation is compliant with the standard so long as that implementation behaves as if it had executed things per the abstract machine specification. The compiler can assume non-volatile variables don't change values between assignments. So long as it doesn't break the as-if rule, the sequence statement_A; statement_B; statement_C; might be implemented by executing part of statement_C, then part of statement_A, then all of statement_B, then the rest of statement_A, and finally the rest of statement_C.

Those as-if rules do not apply to volatile variables. With regard to volatile variables and functions, an implementation has to do exactly what you told it to do, and exactly in the order you told it to do things.

There's a downside to the abstract machine specification: It's slow. One positive aspect of C and C++ compared to other languages is that they are quite fast. This would not be the case if code was executed per these abstract machines. The as-if rules are what enable C and C++ to be so fast.

ELI5 answer

what does it mean when the compiler doesn't "optimize away" the memory address?

"Optimizing away" a memory address is an advanced concept, something that is not within the realm of the capabilities of a five year old. Compliant five year olds will do exactly what you tell them to do, no more, no less. With volatile, you are telling the implementation to act like it's five: No thinking, no fancy optimizations. Instead, the implementation has to do exactly what the code tells it to do.


问:
Is this example implementation of the Observer pattern well-written?
It seems as if this example implementation of the Observer pattern is drawn from the book Headfirst Design Patterns, OReilly, which I am currently reading. Here is a UML diagram from the book

It's not very cleanly visible, but the methods, composing the Subject Interface are:

registerObserver()

removeObserver()

notifyObservers()

What I am skeptical about is the last method. Why would the clients of the interface know about the specific way in which they are called? IMHO the place of this method is inside the concrete subject implementation -ConcreteSubject.

答:
notifyObservers is indeed something internal to the implementation of Subject and should not necessarily be public.

It is helpful for a helper class like ObserverSet. To which you would forward the registerObserver and removeObserver calls and then internally call notifyObservers on.