问:
Caching at business layer vs Caching at Data Layer
I have always worked on projects where caching was done on DAL, basically just when you are about to make the call to database, it checks if data is already there in the cache and if it is, it just doesn't make the call and instead returns that data.

I just recently read about caching at business layer, so basically caching the entire business objects. One advantage I can see straight away is much better response times.

When would you prefer one over the other? and Is caching at Business Layer a common practice?

答:
This is probably too broad for a definitive answer.
Personally, I feel that a data access layer is the better place for caching, simply because it is supposed to be very simple - records go in and out and that's it.

A business layer implements many additional rules of higher complexity, so it's better if it doesn't also have to manage per-object availability concerns in addition to multiple-object consistency concerns in the same class(or even the same method) - that would be a blatant violation of the SRP.

(Of course, I only reached that insight after my service classes had grown to unmanageable complexity when they tried to do both caching and configuration simultaneously. There is no better teacher than experience, but the price sure is steep.)


问:
How should I annotate, that a class is immutable in Java?
I recently stumbled upon how useful immutable objects are and that e.g. if you pass elements to a constructor and your class should be immutable, you have to copy these elements if they are not immutable themselves.

This requires a lot of checking or knowledge about my project, because if I have

```<code>public A(B foo)
</code>```

and B is not immutable, A I'd have to copy B. Now imagine B seems immutable, but itself has mutable classes in the constructor and so on.

Is there a standard or best-practice for documenting if a class is immutable in Java? It seems there is no @immutable keyword in Javadoc.

The @Immutable annotation seems to be something totally different for auto class generation and not part of standard Java.

答:
Looking at the documentation of the String class (bold text is something I did):

The String class represents character strings. All string literals in
  Java programs, such as "abc", are implemented as instances of this
  class.
  Strings are constant; their values cannot be changed after they are created. String buffers support mutable strings. Because String
  objects are immutable they can be shared.

As you can see, there is no particular labeling or other markers, however, they do specify in the JavaDoc describing the class that the String class represents a constant, immutable object.

Thus, assuming that you followed the instructions provided in this Oracle tutorial to create an immutable class, to make sure that the immutability of the class is documented, you should just make sure you mention in it the in the JavaDoc describing the class and what it does.

If that still does not suit what you need, you could take a look at how you could build custom JavaDoc tags here.

ContentLicense: CC BY-SA 3.0


问:
How to earn team members respect as team lead
If you were hired into a new company as a team lead (say a team of 10) one of the important things to do is to earn the respect of the members of the team. In the early days the new team lead may know nothing of the team culture, code base and business domain: in other words, is a complete neophyte. </p>

<p>How does one go about this? What are the do's and don'ts? </p>

答:
Now that you have got some good answers, I will suggest to take initiative and start something new. Something like Friday free tech talk meeting, in which you can show them how to solve Rubik's cube.
It's a kind of development forum, where each member is free to talk about the technology or the product that they like, or they have worked with. Let it be open source or the personal or the college project. The idea is to create the environment where the developers will participate on their own. They can talk freely about anything in the meeting.

In my previous company, we used to have something like this, but it was concentrated around the patents. Each person will come up with the idea, to use the product in a new way. In the meeting we will discuss the idea, their advantages, disadvantages, challenges etc.

In the current organization there was the same idea, but somehow unfortunately it is turned into the progress meeting, revolving around the project. So that's another lesson. Developers should not be forced about anything in this forum, but rather environment should be created so they should participated on their own.

ContentLicense: CC BY-SA 2.5


问:
Do wildcards in Java generics restrict or increase flexibility?
I have read many sources about wildcards and Java generics. Even though I have seen many explanations and answers, none of them seems correct.

The question is very simple: Do wilcards in Java generics increase flexibility?

My understand is "No". Wildcards in Java generics (wilcards) actually decrease flexibility.
When I first looked at and used Java generics mechanism, I rarely/never used wildcards. Very often I saw and used the form <T> abc(T xt). One day I got a test question about super and later I started looking at wildcards more closely.

Many sources and in particular: Effective Java 2nd gives an example as follows to show how wildcards increase flexibility.

```<code>public class Stack<E> {
    ...
    public void pushAll(List<E> list) {...}
    ...
}
</code>```

The books goes on and says that since we should be able to use List of anything extending E as argument to pushAll(List<E> list), this is not flexible enough. Therefore, the book suggest changing the signature of the method to pushAll(List<? extends E> list). Now if E is Number then we can even use List<Integer> as argument for pushAll(List<? extends Number> list). And that we have increased flexibility (of the API).

My understanding is that the other solution is not to use wildcards but to use the regular typing. In that case, if we change the method signature to

```<code>public <T extends E> void pushAll(List<T> list)
</code>```

we will achieve the same thing that the wildcard version wants to achieve.
And now in the body of the method, not only we can read from list, we can also write into it. If we use the the wildcard version, we cannot write into that list any value other than null. So in this view, wildcards actually decrease flexibility, not increase.

All in all, it looks to me that wildcards are actually used to decrease flexibility so that <? extends X> discourages people from writing into the object while <? super X> discourages people from reading the object (try it and you will get only Object as type of the returned value). The following example illustrates the <? super X> case.

```<code>public void popAll(List<? super E> list) {
    Object x = list.get(0) // Only get Object as type here, reading is discouraged
}
</code>```

We can change the signature to public <T extends E> void popAll(List<T> list).
Now we can write or read as we want. There is no restriction.

So to sum up: All usages of wildcards can be substituted by typed parameter in the manner of <T extends E> (so there can be no need for the keyword super) . Using wildcards, we lose flexibility in the ability to both read and write into the object. Therefore, wildcards decrease flexibility.

Is it therefore true that wildcards in Java actually decrease flexibility?  If not, how is my reasoning flawed?

[My question is flawed as per missing the cases as mentioned by @Doval in the comment under his answer.]

答:
Item 28 of the good book (Effective Java 2e) is 'Use bounded wildcards to increase API flexibility'.

Yes bounded wildcards increase flexibility over type signitures without bounds, but they lack the ability to express some concepts that can achieved with non-wildcard type bounds.

Do not use wildcard types as return types. Rather than providing additional flexibility for your users, it would force them to use wildcard types in client code.

So when a type appears in the return type, it must not be a wildcard. It is also best not to wildcards if a type appears multiple times in the type signiture.

So why would you ever want bounded wildcards?

There is a duality between type parameters and wildcards, and many methods can be declared using one or the other. [...]
  As a rule, if a type parameter appears only once in a method declaration, replace it with a wildcard.

They can be easier for people using your API to read.

However, Effictive Java 2E was written for Java5, which was a while back, and Java7 came with improvements to the inference for type parameters and the diamond operator. Perhaps in this more modern age type parameters should be prefered.


问:
In what case a use case should be written
I am trying to figure out some questions to help me decide whether a use case should be written for that requirement or not. Is it neccesesary to have a use case for each requirement?

E.g I have a functional requirement which is about creating a new DB table for the new parameters that will be selected from the UI. The FR also states the what ariables should present each parameter in the table etc.

答:
A functional requirement wouldn't state that a new DB table needs to be created so that different layers of your application can relay user input from the UI.  And it certainly shouldn't specify the schema of the table.

What you have is not a Function Requirement.  It may have started off as one but it looks like somebody along the way has suggested some implementation details.

Do you still have the original FR?  If so, you should be able to write use cases around that (which wouldn't reference any implementation details).


问:
Trying to process partial POST of data in REST when some data is successful?
I have a resource that accepts array of numbers. Each number is proceed interdependently, and computation fails are expected. How should I communicate to API client "some of your numbers couldn't be proceed, but rest was successfully forwarded"?</p>

答:
It seems the most straightforward method is just returning a XML or JSON, with the numbers and status.

Per your description, I imagine a structure like this would suffice:

```<code>{
   requestDate: '2015-02-05 12:32'
   results: [{
       number: 1,
       status: 'forwarded',
   },
   {
       number: 2,
       status: 'forwarded',
   },
   {
       number: 3,
       status: 'fail',
   }]
}
</code>```

I also think a HTTP Status of 200 is most indicated, as the client request was successfully processed. Also you can read in the spec (emphasis mine):

10.2.1 200 OK

...

POST an entity describing or containing the result of the action;

Edit: Researching a little more about the status codes, I think it may be better to use 207 Multi-Status:

The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.[4]

Body:
It seems the most straightforward method is just returning a XML or JSON, with the numbers and status.

Per your description, I imagine a structure like this would suffice:

{
   requestDate: '2015-02-05 12:32'
   results: [{
       number: 1,
       status: 'forwarded',
   },
   {
       number: 2,
       status: 'forwarded',
   },
   {
       number: 3,
       status: 'fail',
   }]
}

I also think a HTTP Status of 200 is most indicated, as the client request was successfully processed. Also you can read in the spec (emphasis mine):

10.2.1 200 OK

...

POST an entity describing or containing the result of the action;

Edit: Researching a little more about the status codes, I think it may be better to use 207 Multi-Status:

The message body that follows is an XML message and can contain a number of separate response codes, depending on how many sub-requests were made.[4]


答:
Caching on the DAL is straightforward and simple

Your DAL is the central data access layer, which means that any and all data access can be controlled through the classes there. As both reading and persisting happens on those layers it is equally easy to clear or update cache entries as changes happen.

Caching in the business is flexible

Caching on the business gives developers the flexibility to determine if the concrete usage of an object will benefit from caching. Depending on the structure of the application back-end services or automated processes might change data that is cached in other parts. With caching in the business a developer can determine if a certain business object will have possible stale data and gain performance, or have the most up-to-date state of a business object at the expense of performance.

Title: Caching at business layer vs Caching at Data Layer

Body:
I have always worked on projects where caching was done on DAL, basically just when you are about to make the call to database, it checks if data is already there in the cache and if it is, it just doesn't make the call and instead returns that data.

I just recently read about caching at business layer, so basically caching the entire business objects. One advantage I can see straight away is much better response times.

When would you prefer one over the other? and Is caching at Business Layer a common practice?


答:
Questions to decide whether to write a use case:

Will (or might) a use case provide more information to the business, the developers, and/or the user about:

whether the requirement should be implemented;

what priority the implementation has?

Will a use case give the developers more information about what functionality to build?

Is the requirement about stand alone functionality or is it an inseparable part of something else?

[Edit, as requested per comment]

To distinguish whether a requirement is stand alone functionality or not you can consider the following:

Can a system that implements this requirement and another system, that is completely the same as the first except in that it does not implement the requirement, both (in theory) exist?

For example, if you have an access control system allowing people entry to a building. The use cases "allow an authorised person to enter the building" and "allow an authorised person to leave the building" are two independent use cases, even though they are very similar from an implementation/design perspective
and when "allow an authorised person to leave the building" is not implement while "allow an authorised person to enter the building" is implemented the building would become very crowed, hypothetically we could implement one and not the other.

If so, then this is a good candidate for a use case.

Is this requirement an extension of some other requirement?  I.e. some other use case must be implemented for this use case to make sense.

For example, "authenticate persons on sight within 100ms" is a refinement, i.e. an extension, of the more general "authenicate persons".  Another example, "allow only authorised person to enter the building" can not exist without "enable ersons to enter the building".

If so, then you can often still model it a separate use case and most methodologies do have some way that you can specify Use Case A extends Use Case B.

The third category are requirements that are inseparable.

[Currently I can't give an example.]

When you have a group of requirements that belong together, write a single use case for them.

Does the software engineering methodology you follow or the project procedure require use case documentation?

Title: In what case a use case should be written

Body:
I am trying to figure out some questions to help me decide whether a use case should be written for that requirement or not. Is it neccesesary to have a use case for each requirement?

E.g I have a functional requirement which is about creating a new DB table for the new parameters that will be selected from the UI. The FR also states the what ariables should present each parameter in the table etc.


问:
Web API alternatives to save a file to server's file system
We have a SQL Server 2012 Enterprise edition and an ASP.NET Web API 2.2 web service. The web API is the public interface for the database.

We need to end a production order doing:

Changing a value in Production orders table.

Generate a XML with all data related to this Production order.

Save that XML file into server's file system.

I have thought to do it with a POST to Web API, but I don't know if I have to do the three previous steps ( Changing value, generating XML and saving it) in a Stored Procedure or maybe in the Web API Controller.

Maybe, if I do it with the API controller, it will take a long time and the client will time out.

答:
Your three steps belong in the Model, not the Controller.

If it's a long-running operation, make it an asynchronous one.  See Using Asynchronous Methods in ASP.NET MVC 4 and C#5, ASP.NET MVC 4, and asynchronous Web applications for more information.


问:
Functional Requirements, Non-behavioral specifications, Acceptance Criteria and the gap
I am struggling in my brain about what the right answer is here and where the gap in our requirements gathering process is on this team.

Non-technical BA - their role is to write Business and User requirements in the form of User Stories.  When there is a compelling business reason for defining a general system behavior though they will write user stories with a system actor. They define Acceptance Criteria for these stories.

Business Systems Analysts and SE's - our role is to take Business and User requirements and derive Functional Requirements from these.  We will generally define system actors and define system behavior with traceability back to business requirements and user stories.

The gap I am confused about is that the Business Requirements and User stories generally don't define the necessary details needed to describe functional components and I am not sure they even should.  The Non-Technical BA's wouldn't understand the intricacies of how to do this even if we asked them to so it seems they belong in the Functional Requirements, however according to the formal definition of functional requirements they are to define the behavior of systems:

A function is described as a set of inputs, the behavior, and outputs (see also software). Functional requirements may be calculations, technical details, data manipulation and processing and other specific functionality that define what a system is supposed to accomplish.
  Functional Requirements (Wikipedia)

Everybody on the team is dealing with this differently right now.  Our approved document for capturing FR's is limited in a way that forces us to define functions of a component and not details of the component itself.  Some BSA's and SE's are defining functions with empty conditions and acceptance criteria and using the Inputs field to define properties of the component.  Others still are not even capturing the information or they have it scattered in notes that they attach to functions.

Others still argue with the Non-Technical BA's claiming their requirements aren't specific enough and the Scrum master's answer is encouraging us to split things into total obfuscated oblivion with the hope that things become more estimable.

Is my understanding of the definition of functional requirements limited here?

CLARIFICATION: I felt I should clarify that when I say Business Requirements I mean VERY high level vision statements from a management level.  They are almost irrelevant to the project in all but water cooler talk.  The actual used requirements are User Stories.

答:
Others still argue with the Non-Technical BA's claiming their requirements aren't specific enough and the Scrum master's answer is encouraging us to split things into total obfuscated oblivion with the hope that things become more estimable.

They are probably correct, although not about the obfuscation.

Requirements should be specific enough so that you can write a test for the requirement and have the customer agree that, if the test passes, the requirement is fulfilled and you can declare success.

If you still need more specificity to allow better estimates and provide better detail for the programmers to follow, then write a Software Design Specification.  This would include things like Class Diagrams and Method Descriptions (inputs, outputs, purpose, etc.)  The customer would not get involved in this... these are instructions just for the developers.  The instructions do map directly to testable requirements, however.


问:
Avoiding closures
I have a data structure in the form of a tree.  Each node represents a cardboard box.  The boxes can contain child boxes.

```<code>public class CardboardBox
{
   public int Id {get; set;}
   public int ParentBoxId {get; set;}

   // [...]
}
</code>```

I'm storing those objects in a flat list rather than a tree structure:

```<code>public class CardboardBoxList : List<CardboardBox>
{
}
</code>```

I want to be able to traverse the tree so I made a 'for each' delgate like this:

```<code>public class CardboardBox
{
   // [...]

   public delegate void ForEachDelegate(int childId);

   public void ForEachChildBox(CardboardBoxList fullListOfBoxes, ForEachDelegate forEachDelegate)
   {
      List childIdList = fullListOfBoxes.GetChildIdList(this.Id);

      foreach(int childId in childIdList)
      {
         forEachDelegate(childId);
      }
   }
}
</code>```

This is all good because now I can easily recurse through the box-tree with:

```<code>someBox.ForEachChild(fullListOfBoxes, childId =>
{
   CardboardBox currentBox = fullListOfBoxes.GetById(childId);

   // [... do something with it ...]
};
</code>```

The problem is, I have code that wants to go through the boxes and build some other data structure.  And I need to have some variables outside the delegate call like this:

```<code>NewDataStructure newDataStructure = new NewDataStructure();

int currentParentNodeId = newDataStructure.AddRootNode();

someBox.ForEachChild(fullListOfBoxes, childId =>
{
   CardboardBox currentBox = fullListOfBoxes.GetById(childId);

   NewDataStructureNode newNode = new NewDataStructureNode();
   newNode.Color = currentBox.Color;

   newDataStructure.AddNodeToParent(newNode, currentParentNodeId);
};
</code>```

So it needs to create a closure for currentParentNodeId as it builds the new tree structure.  But I want to try to avoid using closures if possible, just to keep the code simple.

I realize closures just come with the territory when using delegates.  In a way it is kind of the point of using delegates.  You want to have an anonymous method that interacts with the code around it, rather than having to define a whole new method like the old days.

I just wonder if there is a better way here.  The CardboardBox tree data structure will be used extensively in this code and will sometimes need closures on all types of different variables, so I can't just add a new parameter to the delegate type.  I would need to create a new delegate and a new ForEachChild method for each delegate type.

I could try adding a generic parameter to the delegate and just pass in a kind of context parameter, but it still doesn't feel clean.  In the end I'm trying to avoid having any closures if possible but I am not sure if I am trying to hard.

Please note this code is all paraphrased and the (probably lame) example of CardboardBox is really a container-type of object in the real code.  The NewDataStructure is also a tree in the real code but it does not necessarily have the same structure as the tree it is built from.

I posted this question on StackOverflow yesterday but the question was deemed too vague, understandably.  I have tried to re-write it more clearly but I admit it is still fairly unclear.  I have also done a fair bit of searching over the last few days and reading up about delegates, anonymous methods and closures which has been helpful.  But I still don't feel comfortable just using closures willy-nilly.  So I'd appreciate any advice or just opinions on all this.

Disclaimer: I have used c# for many years so if there are any syntax errors I don't need you to point those out.  I am paraphrasing the real code off the top of my head.

答:
Why don't you design the interface of CardBoardBox utilizing an IEnumerable:

```<code>public class CardboardBox
{
   // [...]

   public IEnumerable<int> GetChildBoxIDs(CardboardBoxList fullListOfBoxes)
   {
      List childIdList = fullListOfBoxes.GetChildIdList(this.Id);

      foreach(int childId in childIdList)
      {
         yield return childId;
      }
      // remark: this function could be arbitrarily more complex
   }
}
</code>```

In your example, the calling code then looks like this:

```<code>foreach(var childId in someBox.GetChildBoxIDs(fullListOfBoxes))
{
    CardboardBox currentBox = fullListOfBoxes.GetById(childId);

  // [... do something with it ...]
}
</code>```

So the caller does not need to provide an anonymous function any more.

Title: Avoiding closures

Body:
Why don't you design the interface of CardBoardBox utilizing an IEnumerable:

public class CardboardBox
{
   // [...]

   public IEnumerable<int> GetChildBoxIDs(CardboardBoxList fullListOfBoxes)
   {
      List childIdList = fullListOfBoxes.GetChildIdList(this.Id);

      foreach(int childId in childIdList)
      {
         yield return childId;
      }
      // remark: this function could be arbitrarily more complex
   }
}

In your example, the calling code then looks like this:

foreach(var childId in someBox.GetChildBoxIDs(fullListOfBoxes))
{
    CardboardBox currentBox = fullListOfBoxes.GetById(childId);

  // [... do something with it ...]
}

So the caller does not need to provide an anonymous function any more.


答:
Data access and persistence/storage layers are irresistibly natural places for caching. They're doing the I/Os, making them handy, easy place to insert caching. I daresay that almost every DAL or persistence layer will, as it matures, be given a caching function--if it isn't designed that way from the very start.

The problem is intent. DAL and persistence layers deal with relatively low-level constructs--for example, records, tables, rows, and blocks. They don't see the "business" or application-layer objects, or have much insight into how they're being used at higher levels. When they see a handful of rows or a dozen blocks being read or written, it's not clear that they represent. "The Jones account we're currently analyzing" doesn't look much different from "some basic taxation rate reference data the app needs just once, and to which it won't refer again." At this layer, data is data is data.

Caching at the DAL/persistence layer risk having the "cold" tax reference data sitting there, pointlessly occupying 12.2MB of cache and displacing some account information that will, in fact, be intensively used in just a minute. Even the best cache managers are dealing with scant knowledge of the higher level data structures and connections, and little insight as to what operations are coming soon, so they fall back to guesstimation algorithms.

In contrast, application- or business-layer caching isn't nearly so neat. It requires inserting cache management operations or hints in the middle of other business logic, which makes the business code more complex. But the tradeoff is: Having more knowledge of how macro-level data is structured and what operations are coming up, it has a much better opportunity to approximate optimal ("clairvoyant" or "Bélády Min") caching efficiency.

Whether inserting cache management responsibility into business/application code makes sense is a judgment call, and will vary by applications. In many cases, while it's known that DAL/persistence layers won't get it "perfectly right," the tradeoff is that they can do a pretty good job, that they do so in an architecturally "clean" and much more intensively testable way, and that low-level catching avoids increasing the complexity of business/app code.

Lower complexity encourages higher correctness and reliability, and faster time-to-market. That is often considered a great tradeoff--less perfect caching, but better-quality, more timely business code.


问:
When to use nested classes and when to use namespaces?
I have really never made use of namespaces, and am considering them at the moment.

My situation. I have a class Reverb. It contains as members instances of other classes, declared in the same header file, above it. Now there are appearing more and more data structures, relevant to the Reverb class e.g. 3 different representations of the user settings.

So I made a class Settings, that contains 3 sub-classes for the 3 representations and some methods to convert between them.

My question is where to place that?

Put it inside Reverb? Then the Reverb API becomes quite bloated and scroll-heavy.

Put it in file scope? Any user of the header will be unpleasantly surprised to find a class Settings, that actually means ReverbSettings.

Create a namespace Reverb and put everything in it? Then what to do with the class Reverb? It would be strange to ask for Reverb::Reverb.

Any better solution?

答:
The questions to ask are can a nested class exist without its enclosing class? Does a nested class need intimate visibility into its enclosing class?

I tend to avoid nested classes in general because of the tight coupling. I would rather refactor the nested class to its own top-level class and use interfaces/pure virtual base classes to give it a reference to what used to be its enclosing class. Often what I find is I can have smaller classes that are more laser-focused on a single task and are reusable.

In C++ you also have the friend keyword which can provide visibility into class internals. While it should be used sparingly, it is useful for those cases where two classes need to work closely together but do not represent quite the same abstraction.

Short version: there is nothing wrong with nesting classes, but prefer namespaces. In rare cases where it is warranted, friend can also be used.


问:
Is there a name for the problem of representing state twice and then getting out of sync?
I have a system that has configurable state values, like BoxCount, "Number of Visible Boxes on the screen".   If I am using some heavyweight set of patterns (like MVC) that enforce the creation of multiplicities of classes, and if I follow design patterns, I often find the case where some tree of objects exists like this:

CModelBox = a model class dealing with some box count

CViewBox = a view class dealing with showing the count of boxes

CControllerBox = a controller class dealing with some box count.

If each of the above had a field (a member variable) called Count, and that one fact (how many boxes?) is repeated in the implementation, and then somehow has to be "updated" in every which way, is there a name for this problem? Is it known by some name, as an anti-pattern, or by some other formal name that might help me do some research on how people identify, and remediate these problems in their application designs?

The very specific problem I observe most often is that any time you have two fields instead of one, you CAN have a difference, and then the question is, does the difference have meaning, or is it an accident. If it's an accidental (and permanent rather than temporary) difference, a failure to keep things in sync, then it's a lurking bug.

Update: I suspect that people who build systems according to "SOLID" OOP principles might have a definitive name for this problem, and if they do, that's the particular answer I want. This is not really an opinion, it should be backed up by some citation from a SOLID OOP source, such as Uncle Bob, or one of his minions.

答:
"Wet."

More precisely, failure to "stay DRY."


问:
How to setup an agile startup without the customer?
In agile, there is an emphasis on customer involvement. They provide user stories, provide feedback, and a lot of decisions are up to them.

There are situations that there is no customer for the software at the time of development. If we are going to ship a game -a software that people will be its customers after shipment- or if we are going to create a new service -like a new social network- and we are going to own it, there won't be any client present at the time of development.

I have a few questions for these environments:

Is there any software development methodology/framework specially designed for such cases?

Is agile applicable to such environments? If yes, How?

Update: It would be much interesting to know about a similar project developed agile.

答:
You need someone to be a proxy for the customer. Someone in the team who takes on the role of Product Owner, or whatever it is called in your methodology of choice, and puts themselves in the position of a potential customer.

Note that this is a difficult job! But choosing the right thing to build is one of the difficult problems any start-up must face!


答:
... who will the future client be? If you can't answer that, you probably have more significant problems than "how to run agile" since you are adopting a, "if we make something cool, people will buy it!" perspective.

You can do a variety of techniques to create an identity for this client. Perhaps a user persona acted out by your team members.

Having a documented "this is a sample user, here are needs of this user" makes it a lot easier to act this out correctly as well.

Title: How to setup an agile startup without the customer?

Body:
In agile, there is an emphasis on customer involvement. They provide user stories, provide feedback, and a lot of decisions are up to them.

There are situations that there is no customer for the software at the time of development. If we are going to ship a game -a software that people will be its customers after shipment- or if we are going to create a new service -like a new social network- and we are going to own it, there won't be any client present at the time of development.

I have a few questions for these environments:

Is there any software development methodology/framework specially designed for such cases?

Is agile applicable to such environments? If yes, How?

Update: It would be much interesting to know about a similar project developed agile.


答:
Replace the word "customer" with "stakeholder". For a contract-software development gig, the primary stakeholder is the customer footing the bill.

For a speculative software or a game, the primary stakeholders are the owners of the company, or whoever else is putting the up front investment to make this happen. The stakeholder in that case often times has plenty of stuff to do other than programming, but needs to be kept up to date and involved with the development process. That person is responsible for imagining what the customer is going to want, or is responsible for selecting someone else to fulfill that role and serve as the stakeholder.

Then, read this: Five Worlds


答:
I call this "redundancy".
Is redundancy good or bad?

It's good if it exists for the purpose of correcting errors,
and if there is a mechanism in place for correcting those errors.

It's bad if it's just "there", because of just what you describe.
It can get inconsistent, and then you're at sea.

One method that is sold as a technique to prevent the inconsistency is notifications.
I don't care for those, for a variety of reasons.

Rather, what I try to do is:

Minimize redundancy, so as to minimize the opportunity for inconsistency. This may mean sharing data, or something like that. This is the reason databases are "normalized".

When some redundancy is unavoidable, as it often is, understand how to deal with it.
I first try to understand which representation is "the boss", so if different data structures disagree, I can tell which one to believe.
Then, I always try to follow a policy of tolerating inconsistency.
I try to have a method to reconcile differences where they occur.
I tend to use algorithms like "merge" and "diff" to do this.
This opposes the strategy of using notifications, but I find it is much more reliable and efficient.

P.S. As an example, a long time ago I stumbled on differential execution which is one way to manage redundancy between a program's state and its UI.

Title: Is there a name for the problem of representing state twice and then getting out of sync?

Body:
I have a system that has configurable state values, like BoxCount, "Number of Visible Boxes on the screen".   If I am using some heavyweight set of patterns (like MVC) that enforce the creation of multiplicities of classes, and if I follow design patterns, I often find the case where some tree of objects exists like this:

CModelBox = a model class dealing with some box count

CViewBox = a view class dealing with showing the count of boxes

CControllerBox = a controller class dealing with some box count.

If each of the above had a field (a member variable) called Count, and that one fact (how many boxes?) is repeated in the implementation, and then somehow has to be "updated" in every which way, is there a name for this problem? Is it known by some name, as an anti-pattern, or by some other formal name that might help me do some research on how people identify, and remediate these problems in their application designs?

The very specific problem I observe most often is that any time you have two fields instead of one, you CAN have a difference, and then the question is, does the difference have meaning, or is it an accident. If it's an accidental (and permanent rather than temporary) difference, a failure to keep things in sync, then it's a lurking bug.

Update: I suspect that people who build systems according to "SOLID" OOP principles might have a definitive name for this problem, and if they do, that's the particular answer I want. This is not really an opinion, it should be backed up by some citation from a SOLID OOP source, such as Uncle Bob, or one of his minions.


答:
There are many possibilities open for you for situations such as this.

For the scrums meetings and day to day, as others mentioned, you should appoint a member of your team, typically the one with a more "visionnary" feel for your game, to act as the stakeholder.  Do make sure that the person that is signing the checks is also represented, at least for your sprint reviews and sprint planning.

Next you should definitively get a broader feedback, play tests for example would be a great source of information.  Usability testing is also good, they do overlap but both target different parts of your system (first checks if it's fun, the other checks how easy it is to get into it).

Stakeholders are an important part of the scrum methodology and should not be overlooked.  They are in big part what keeps the dreams and wishes of the team firmly grounded with their perspective.  So if you decide to play the role internally do make the extra effort to get external feedback as often as possible.


答: I would say you should put yourself in the shoes of your customers. Try to do a SWOT analysis to figure out why would you buy such a software and decide on the potential requirements that a customer would request. From there you can I would recommend reading Business Model Generation written by Alex Osterwalder. I think this is a question that relates more of a how to build a software that is desirable. </p>


答:
Often Agile seems geared towards B2B interactions, where your "customer" is another company with a few representatives who are there to tell your group what they want.  From what I understand, you are talking about the situation where you are making a product for mass distribution.  In such cases, your customers are millions of people you hope want to use your product.  You don't really have anyone to invite to your review.

The way we handle this is with a separate position, a "Product Manager", who is in essence responsible for acting as the customer.  They will analyze the market, do competitive analysis, survey customers, do user testing, and anything else to try to understand the target customers.  That person acts as your customer.  They end up working closely with the product owner in that regard.

If you are small, and can't have a separate product manager, the product owner can often double-up on that role.  Though it is generally better to have someone separate, as the product owner also has to manage technical stakeholders.


问:
How do you learn a language without actual tools?
I will bring a concrete case here.

I have a project involving Sketch and Arduino. Since I was they only guy willing in the group, I had to take the Sketch part. The problem is that our informatics group is part of an bigger organisation, and the organisation doesn't allow us to take the Arduinos with us at home. We are there for only 2 hours of week or so, and learning anything there is out of the question (Noise, other projects, etc). So I am stuck here without anything physical, only books. I don't even have a book at that, by the way.

The same case can be made for learning iPhone developing without running the apps, or whatnot. So how do you proceed here?

答:
It sounds like Zen philosophy but the simple answer is "Be the Computer". Let me tell you a story to explain what I mean...

When I was a kid an adult friend/mentor logged me into a mainframe computer at his college to play a text-based game called "StarTrek" using a teletype printer with roll-feed paper (CRT monitors were rare back then). I fell in love with the game and wanted to learn how it all worked but I was not even supposed to be using the computer let alone have allocated programming time and resources. Home computers did not even exist (Yes, I am OLD!).

The program was written in BASIC and had about 2,000 lines of code. My friend knew how much I wanted to learn so he printed out the entire source code listing onto a scroll of paper that was about 30 feet long. He also provided samples of input and output with several complete sessions of the game on other scrolls of paper. In the end I had a couple hundred feet of printouts.

Right after I got the listing my dad got a new job and we had to move so I lost easy access to my friend for questions (no home computers/email/etc again). The first thing I did when we got settled was go to my local library and check out the only book they had on BASIC. Using these I examined each line, analyzed what it did, and then using plain old fashioned notebook & graph paper began to execute the code line-by-line. I tracked the variables, built the strings, and used dice to generate random numbers. In modern day terminology we call this "single-stepping".

With the printouts, one book, no instructor and no live computer I learned how to program and also how computer language interpreters work. Took me about 3 weeks of focused study to master the language.

Later, I did the same thing with the entire microcode OS of the TRS-80 (Level I) written in Z80 assembler. About 4,000 lines of machine mnemonic and hex code. I was the CPU, the bus, the memory core, and the hardware interrupts. I became a Z80 microprocessor and a TRS-80 computer system and in about 4 months of on-and-off study I mastered Z80 assembly programming and learned exactly how microprocessors work. I also learned a little about motherboard design as a side effect.


答:
You may already know about this but you want to look at the books/ movements "Lean Startup" by Eric Ries and "Running Lean by Ash Maurya.

Both cover concepts and methods of determining with incremental design, prototyping and development,  discovering who your customers might be, what your product needs to be and where it needs to go, and in general, continuously testing your premises for even wanting to develop your product.  They help you ensure that you don't spend a ton of money up-front in building stuff that no one might want.

Once you start actually building the product, techniques such as Scrum and Kanban can help you build incrementally/ continually - and prioritize what you should build by focusing on risk/ cost of delay and other factors.

This is a very basic response to your question - but one needs to better understand your situation for that.  And if you haven't read those two books, you MUST!

HTH


问:
Does the JVM's Garbage Collector clean the entire JVM HotSpot memory (C-Heap, Java Heap and Permanent Generation space) or just Java Heap?
I know that the JVM has some JVM HotSpot memory which is further divided into three areas:

Java Heap

Permanent Generation Space

Native Heap (C-Heap)

I know that Java has automatic garbage collection mechanism for Heap Memory.

So, I want to confirm that in my last statement Heap refers to both Java Heap and Native Heap (C-Heap) or just Java Heap.

Also, what about Permanent Generation Space? Is it also considered as some kind of Heap or what?

答: The GC will collect both the Java Heap and the permgen, but not the native heap. The native heap is manually managed by whatever C code owns it. The permgen, on the other hand, is managed by the JVM garbage collector (though not necessarily as efficiently as the rest of the heap, as certain GC algorithms will only compact the permgen when absolutely necessary).</p>


问:
Serving a file from a private directory
I have a PDF that needs to be accessible for multiple unrelated areas across a website. I don't want to reveal anything about where the file is downloaded from and I want to only upload the file once.

Is it possible to serve a file stored in the private space, as opposed to storing it in a public directory. If so, can someone point me in the correct direction or give an example?

答:
PHP provides the int fpassthru(resource $handle) function. You open a file, and then have fpassthru send the file outby simply copying it to the output without storing it, allowing potentially very large files to be sent.

```<code><?php
// open the file in a binary mode
$name = './img/ok.png';
$fp = fopen($name, 'rb');

// send the right headers
header("Content-Type: image/png");
header("Content-Length: " . filesize($name));

// dump the picture and stop the script
fpassthru($fp);
exit;
</code>```

If you can, it is better to avoid using PHP to do anything other than setting up the initial download, and then handing off the actual download to the webserver to control (and so release a PHP process). If you control the server, you can install/enable mod_xsendfile (for Apache) or the Nginx equivalent, X-accel.

Title: Serving a file from a private directory

Body:
PHP provides the int fpassthru(resource $handle) function. You open a file, and then have fpassthru send the file outby simply copying it to the output without storing it, allowing potentially very large files to be sent.

<?php
// open the file in a binary mode
$name = './img/ok.png';
$fp = fopen($name, 'rb');

// send the right headers
header("Content-Type: image/png");
header("Content-Length: " . filesize($name));

// dump the picture and stop the script
fpassthru($fp);
exit;

If you can, it is better to avoid using PHP to do anything other than setting up the initial download, and then handing off the actual download to the webserver to control (and so release a PHP process). If you control the server, you can install/enable mod_xsendfile (for Apache) or the Nginx equivalent, X-accel.


问:
How QA/testing process can be evaluated?
I asked this question on Programmers about a crazy idea to evaluate how testers are doing their job. From the answers/comments, it seems community also considers that as a crazy idea. This is a spin off from that question.

From one of the answers in that question

On the other hand, we tend to let the customers verify the work of the
  QA teams, which is possibly not ideal. It is a very powerful feedback
  loop though.

As the answerer himself says, it is not ideal. The whole point of QA is to make sure that highest quality product gets delivered to the end user. We want our testers to report problems, not end users.

So my question is , if John is a Project Manager, what are the methods he can use to evaluate QA process? How he can identify QA guys who are doing it correctly or incorrectly? What are the obvious little things that should be noticed and acted upon?

答:
Ultimately, the metrics that you choose depend on the specific questions that you want to answer about the quality of your product and processes:

If you want to know how good your defect reports submitted are, track the number that are withdrawn or marked as not reproducible. You'd also want to track what organization (development, quality, customer support, etc.) submitted it. You want a high percentage of the reports to contain sufficient information to reproduce and then fix the issues.

If you want to know how effective your testing is, look at test coverage. If you are doing requirements based coverage, map test cases to requirements and ensure that all requirements are covered. If your quality team is writing test code at a unit and integration level, look at code coverage.

Look at defect containment metrics, such as total defect containment effectiveness (TDCE). Use the equation TDCE = (pre-release defects found) / (pre-release defects + post-release defects). You may need to be careful what you consider a pre-release / post-release defect. If you only want to count the effectiveness of your back-end quality activities, pre-release defects is only defects found by the quality organization and post-release defects is anything found after they finish their work. You could consider pre-release to be all defects found by anyone before release to a customer and post-release defects to be those found by customers or in-house after the release.

Number of defects found per phase. This depends on your process, but examples would be requirements, design, code and unit test, integration test, acceptance test, post-release.

Something that is important to consider, though, is that your developers should be involved in quality assurance. They should be reviewing the requirements to ensure they are correct and usable, the designs to ensure that they can implement the software, the code should be both reviewed and unit tested. QA is more of a process to give you confidence that the product is of high quality, and not an organization that inspects or tests the final product.

OwnerDisplayName: Krishnabhadra

OwnerDisplayName: Thomas Owens


答:
Quality management, both assurance and control, is risk mitigation against poor performance and defects, both of which are probabilistic.  Part of the probabilistic outcome is aleatory, meaning due to random variability where no action has any real effect.  Trying to decipher how well your QM activity is doing when you have both random and non random effects would be near impossible.  In other words, you will credit or blame QM on either a favorable and unfavorable result that would have occurred anyway.

There is also something odd about doing a quality analysis on the function that does quality analysis on the rest of the team and processes and outputs.  In a project environment, where time and money are always constrained, I doubt that would yield any real value.

At the end of day, you measure your performance and outputs that the entire project team is a part of, including the quality management team.  If performance and outputs are meeting objectives, then I believe it is safe to assume your quality management capability is partly to blame.  If objectives are not being met, and your quality management team are producing findings that require attention, whether during QA or QC, then that is another good sign they are performing.  If objectives are not being met but neither QA nor QC are introducing any findings, then that is a good sign they are under performing themselves.  I think it is as easy as this; it should not require any more complex analysis than this.

Title: How QA/testing process can be evaluated?

Body:
I asked this question on Programmers about a crazy idea to evaluate how testers are doing their job. From the answers/comments, it seems community also considers that as a crazy idea. This is a spin off from that question.

From one of the answers in that question

On the other hand, we tend to let the customers verify the work of the
  QA teams, which is possibly not ideal. It is a very powerful feedback
  loop though.

As the answerer himself says, it is not ideal. The whole point of QA is to make sure that highest quality product gets delivered to the end user. We want our testers to report problems, not end users.

So my question is , if John is a Project Manager, what are the methods he can use to evaluate QA process? How he can identify QA guys who are doing it correctly or incorrectly? What are the obvious little things that should be noticed and acted upon?

OwnerDisplayName: David E PMP


答:
Taking an ultra-simplistic view...

Defects are introduced into a product by the development operation. We (or more often, management) might say "If I get better developers (for "better" developers read "better people and better processes and practises) I will get fewer defects". This is undeniably true, but only up to a point- At some unknown point on the curve you will not be able to reduce defects by hiring "better" developers, but you will still get defects because, as we all know (except for management) it is technically and practically impossible to produce defect-free code.

So you hire testers to ensure that a) the product does what it is supposed to do so you all get paid and b) they find every single remaining bug left in the code... Except that you have a similar curve- At one level, hiring better testers means they will find more defects, but just like you cannot guarantee bug-free programming, nor can you ever do enough testing to guarantee you have found all the bugs. Some will always remain. And it is the I.T. equivalent of Sods Law that even though the remaining defects are so obscure your testing coverage does not find them, on day 1 the users will (or thereabouts).

The trick is knowing whether your developers are sufficiently good and your testers are sufficiently good that you are producing the best possible outputs to the users, accepting the fact that there will always be some residual defects after testing no matter how much you do.

But let's suppose you have cause to believe that some element of the delivery operation is not operating to maximum possible quality and the users are seeing more defects than they could be. Where do you go from there?

Well, measure the amount and types of defect found in System testing, in UAT testing, and then after the product goes Live. Now you know how many "findable" defects there were in your product release. You can then assess why UAT didn't find the ones that were reported in Live, and why System testing didn't find the ones that were reported in UAT. Lastly you can assess why the defects made it out of the development shop at all.

You will get a wide range of effects and answers, but when you have them you can devise a cost/benefit analysis for mitigating whatever issues you find. Then when you have mitigated those effects to the point where spending more time or money wouldn't give you any appreciable difference in the residual defect rates you have achieved all you can achieve and you have the best practical delivery operation possible for you...

After that, issues will still get found, and users will still get annoyed and management will still ask why it is possible to deliver code with defects "even after spending all this money to make our delivery the best it can be", but you will know you have, as an organisation, done the best you can and you may rest easy at night :)

OwnerDisplayName: Marv Mills


问:
Why was Python's popularity so sudden?
Python first appeared in 1991, but it was somewhat unknown until 2004, if the TIOBE rankings quantify anything meaningful.

What happened?  What caused the interest in this 13 year old language to go through the roof? Is there a reason that Python wasn't considered a real competitor to Perl in its first decade of existence? Is there a reason that Python didn't continue in relative obscurity for another ten years?

I personally think that Python is a very nice language, and I'm glad that I'm not the only one. But it doesn't have corporate backing or a killer feature that would explain a sudden rise to relevance.

Does anyone know the story?

答: The reshuffle in TIOBE index in 2004 is down to Google changing their search. There really isn't much you can learn from TIOBE - it's utter nonsense. I don't believe there has been any sudden jump in Python popularity. Certainly not because one company happens to use it internally for some of their stuff.</p>

Title: Why was Python's popularity so sudden?


答:
The answer to your question is probably 2-stage.

First, QA has a policing role. This can be measured in nearly any project and you could use a fairly simple metric like ratio of defects caught by QA vs number of defects discovered (by anyone) in production. This will you how good your QA team is at catching defects and it's not a bad place to start. If a lot of defects are getting through, you can see if there are trends in the types of defects coming through or maybe defects are most often getting through when QA gets overwhelmed close to deadlines.

However, too many organizations stop here and that is both financially wasteful and ultimately ineffective. You can't solve the defect problem with testing alone. You want to leverage the perspective of your QA experts throughout the process. When I was managing a QA team, we'd have people embedded in all of the agile teams. When reviewing user stories and tasks, they QA team member would asking questions about use cases and potential hangups that would become test cases, but would more importantly help frame the work the developer would do. They'd also be present for architectural and tactical discussions as well as code reviews to create a broader perspective in the team. Finally, if your QA team members are asked to make quality their priority, they can provide a check against the pressures on developers to rush through tasks that we all know can occur in projects.

Leveraging the QA team as more than just a policing force will bring down the total number of defects created in development. Obviously, catching 95% of 100 defects in a project is a much more palatable scenario than 95% of 1000 defects. Once things are more integrated like this, you can look at the total generated defects as a measure, but you have to remember at that point that you're talking about a measure that reflects the whole organization, not just QA anymore.

OwnerDisplayName: Daniel


答:
If you're looking for quantitative metrics that show if QA is doing its job, here are a couple common ones:

number of open defects and # of total defects, the ratio between the two, the change in the ratio throughout the iteration.

number of P1, P2. P3. P4 defects found in pre-prod environment vs production environment

Average Defect cycle time

Here are some more qualitative things to look at that are strong indicators of performance:

How well is the defect documented?  Does it at least have a severity, actual outcome, expected outcome documented?

Does your QA team write code?  Do they automate tests?  Do they write an automated test when a defect is fixed?

Does your QA write test cases?  When do they write them - before a story is developed or right at the end when it needs testing?

Do they perform regressions?  When?  What is regressed in terms of business critical functionality?  Is the regression automated?

Can your QA justify, in terms of business value, what they decided to test and how?

During retrospectives, does QA provide suggestions for improvement?  Does the team listen?

How is the communication between QA and developers?  What does Dev say about the value added by QA on the team?

OwnerDisplayName: WBW


问:
Replace repeated timestamp with variable in tests
I have couple hundred tests and I work with date/time a lot. In some tests, I aim for output format, elsewhere, I check date ranges. Therefore, my tests have lots of these:

```<code>FactoryGirl.create(:foo, ended_at: Time.zone.local(2014, 5, 5, 22, 15))
</code>```

I thought, maybe there's a shortcut to these, since in some test files, I only work with few timestamps.

```<code>FactoryGirl.create(:foo, ended_at: may_5_22_15)
</code>```

Where may_5_22_15 would be defined in method or using let.

Is this a good way to go?

Update: More context (thanks @Snowman for pointing out).

Motivation for this is to improve readability of testsuite. It makes it easier to me to read "Foo ended at May 5th by 22:15" rather than parsing a bunch of numbers in Time constructor.

The model has bunch of date/time attributes (ended_at, started_at, closed_at...) and the more specs I write, the more Time constructors I use and re-use. So I thought maybe I could do something like this:

```<code>let(:may_5_22_15) { Time.zone.local(2014, 5, 5, 22, 15) }
</code>```

What I'd like to know is whether this is a good practice or not, in terms of test suite maintenance, readability or if there are any reasons why not to do it.

答: I am not sure where the ended_at method is defined, but there is no reason why you could not extend  it so that it would take whatever kind of string input you want using Time.parse


答:
If this process really does take a long time, you could also consider moving the long-running work out of process.

For example, when the request is handled by your Web API, you could write a message into something like a Message Queue and have MSMQ on the server activate a separate process to perform whatever work is indicated by that message.

How that work might be carried out is kind of orthogonal here - maybe a stored procedure works best in your case, or maybe a driver app to run business logic coded in a class library, etc. The point is that you can do this work without having to worry about it being in the context of a Web API thread - you can return immediately to the client and they don't have to wait for the long running process to complete, and you don't have to worry about timeouts on their connection, etc.

The caller can then poll for status update on the job, or you could use web sockets to push status back to the caller, or even use a separate Message Queue to write completion messages which the caller is monitoring, etc.

Of course this can only work if the caller is willing to accept a fully asynchronous workflow :)

Title: Web API alternatives to save a file to server's file system

Body:
We have a SQL Server 2012 Enterprise edition and an ASP.NET Web API 2.2 web service. The web API is the public interface for the database.

We need to end a production order doing:

Changing a value in Production orders table.

Generate a XML with all data related to this Production order.

Save that XML file into server's file system.

I have thought to do it with a POST to Web API, but I don't know if I have to do the three previous steps ( Changing value, generating XML and saving it) in a Stored Procedure or maybe in the Web API Controller.

Maybe, if I do it with the API controller, it will take a long time and the client will time out.


答:
Be a bullshit umbrella.

Management is going to pawn tasks upon you. Don't then pawn those tasks off onto your subordinates. Work to shelter your subordinates from the crap that rains from above so they can focus on getting their jobs done.

Title: How to earn team members respect as team lead

Body:
If you were hired into a new company as a team lead (say a team of 10) one of the important things to do is to earn the respect of the members of the team. In the early days the new team lead may know nothing of the team culture, code base and business domain: in other words, is a complete neophyte. </p>

<p>How does one go about this? What are the do's and don'ts? </p>


问:
How to efficiently store big time series data?
I need to store and to be able to query some very large amounts time series data.

Properties of the data are as follows:

number of series : around 12.000 (twelve thousand)

number of data points, globally : around 500.000.000 per month (five hundred millions)

mixed value types: the majority of data points are floating point values, the rest are strings

sampling period : variable between series as well as within a series

timestamps : millisecond precision

data retention period : several years, without decay or downsampling

data archives need to be built in near realtime, but a reasonable delay (~1 hour) is acceptable

past data can be rebuilt if needed, but at a high cost

sometimes, but quite rarely, some past data needs to be updated

Properties of envisioned queries:

most of the queries against the data will be timestamp-based queries; ranging from one day to several months/years. 90%+ will be queries on the most recent data

Other requirements:

the solution must be free as in free beer and preferably opensource

My initial thought thought was to use PyTables / Pandas with HDF5 files as storing backend instead of an SQL database.

Questions :

Assuming PyTables / Pandas is the "best" route, would it be better to split the data in several HDF files, each one spanning a given period of time, or put everything in a single file that would then become huge ?

Should I go and prefer the fixed or the table format ? To me, fixed format looks OK if I keep one HDF file per month, as this way a whole series probably fits in RAM and I can slice in-memory without needing a table format index. Am I correct ?

And if that's not the best approach, how should I structure this data store or what technologies should I be considering?  I'm not the first to tackle storing large sets of time series data, what is the general approach to resolving this challenge?

Other approaches I have considered :

array databases: they are a superb fit for time series with constant sampling period, as you then only need to store start and end times and sampling period of the array, and then only values in the array itself and indexing is easy. But with variable sampling periods within series themselves, I need to keep a closer timestamp->value relation, that in my view is not such a good fit for array DBMS.

standard SQL database with timestamp,paramID,value as columns but by their nature they request a lot of disk I/O for any query

答:
you might want to checkout column-oriented databases. I am not sure what you mean by array databases but with my suggested approach you can have a dynamic number of values per time frame. You can also have multiple values for the same timestamp. The interesting part is that if you have values measured at the same timestamp you can save them as additional columns (e.g. a sensor that measures temperature and humidity, in stock trading price and size of an trade, ...). Because of the column-oriented nature you can have tables with 100 columns but if your query only accesses five columns the database reads only the data of the five columns.

I wrote a series about creating your own time series database, you might want to have a look at it:

Part 1: https://medium.com/@hellomichibye/column-oriented-database-introduction-part-1-572e5780aebb

Part 2: https://medium.com/@hellomichibye/column-oriented-database-draft-part-2-21199a2de18a

OwnerDisplayName: flyingmig


问:
Relation between object orientation and algorithms
As I read some algorithms textbooks, they are full of clever procedures for some problems (sorting, shortest path) or some general methods (recursive algorithms, divide and conquer, dynamic programming...). I found few traces of object oriented programming there; (Why they are more procedure-oriented?).

Then I was thinking:

What is the relation between algorithms and OOP? Are they two independent topics?

Are there some problems which can only be presented and solved by OOP?

How can OOP help algorithms? Or in which direction it can affect it?

答:
First, lets define what we mean by OOP. By OOP I primarily mean:

Encapsulation and hiding of details in class.

Polymorphism of behavior through inheritance and virtual methods.

Now, to answer your question:

What is the relation between algorithms and OOP? Are they two independent topics?

Yes, algorithms and OOP are independent topics.

Are there some problems which only can be presented and solved by OOP?

No. OOP primarily offers convenience with its object-oriented-structuring of code.
OOP allows the programmer to read (and write) code more easily and makes it easier to understand how code works. It neither increases nor decreases the (algorithmic) procedure behind a code.

How OOP can help algorithms? Or in which direction it can affect it?

Like I said above. Both points, Encapsulation & Polymorphism, apply here. Being able to hide details of algorithms and their data structures can help reason about the whole thing. Many algorithms contain details you don't want users of these algorithms messing around with. Hiding those details helps a lot.

The ability to have polymorphic behavior is great too. List is defined as being able to add/remove/clear the items anywhere in the collection. But it can be implemented as resizable array, double-linked or single-linked, etc. Having single API for multiple implementations can help reuse.

Why algorithms textbook are more procedure-oriented?

Like I said, OOP is not necessary to implement an algorithm. Also, many algorithms are old and created when OOP was still not widespread. So it might also be a historical thing.

ContentLicense: CC BY-SA 4.0


答:
Algorithms and OOP are two disparate terms, which have only in common, that they are CS-terms. Simply - An algorithm is like a cooking recipe: to do x you need the following ingredients and do step 1,2,3,4,5,6... then you have your meal prepared.

That said, it seems a natural fit for algortihms to be described in a procedural way. Procedural means nothing other than: first do x and then do y.

A common problem is: »How to sort a set of x?«.
An easy to understand solution is bubble-sort:

Iterate over the set from the last element as long as you have not reached the first element and while iterating

Start a 2nd iteration from the beginning to the current element of the first iteration and

Compare the current element of (2) with its successor

If greater, swap positions

That is the algorithmic / verbal description of the bubblesort-algorithm.

Here comes a procedural / pseudocode implementation

```<code>bubbleSort(Array A)
  for (n=A.size; n>1; n=n-1){
    for (i=0; i<n-1; i=i+1){
      if (A[i] > A[i+1]){
        A.swap(i, i+1)
    }
  }
}
</code>```

That was easy.

How does that releate to OOP?
You could use this algorithm to treat collections (an object itself) of objects:

Example in Javascript (though no clean OO-Lingo, but with near to no boilerplate and easy to understand)

```<code>objects =[{"name":"Peter"}, {"name":"Paul"}, {"name":"Mary"}]

compareObjects=function(x,y){ return x.name>y.name };

sorted = objects.sort(compareObjects)

console.log(sorted)
</code>```

We have a) a collection objects, b) a method common to this collection sort which contains / abstracts away the sorting algorithm and c) our objects Peter, Paul and Mary. The Specification for the sorting is found here.

What is the relation between algorithms and OOP? Are they two independent topics?

From what was said, it should be clear, the answer should be: yes, they are independend.

How OOP can help algorithms? Or in which direction it can affect it?

OOP is just one programming style. It can not help in any kind. Otherwise an algorithm could be implemented in an OO language to do something to objects (as shown)

Are there some problems which only can be presented and solved by OOP?

I can not think of one (but that doesn't mean, that it is impossible). But if you look it the other way around: OOP is useful, if you want to model some problems and solve ith with an appropriate algorithm. Say you have a record of friends you could model them as objects with properties and if you want a list of friends sorted in any way, you could use the example-code given above to do exactly that.

Why algorithms textbook are more procedure-oriented?

As said: it is more natural, since procedural is the character of algorithms.

Title: Relation between object orientation and algorithms

Body:
As I read some algorithms textbooks, they are full of clever procedures for some problems (sorting, shortest path) or some general methods (recursive algorithms, divide and conquer, dynamic programming...). I found few traces of object oriented programming there; (Why they are more procedure-oriented?).

Then I was thinking:

What is the relation between algorithms and OOP? Are they two independent topics?

Are there some problems which can only be presented and solved by OOP?

How can OOP help algorithms? Or in which direction it can affect it?

Body:
Algorithms and OOP are two disparate terms, which have only in common, that they are CS-terms. Simply - An algorithm is like a cooking recipe: to do x you need the following ingredients and do step 1,2,3,4,5,6... then you have your meal prepared.

That said, it seems a natural fit for algortihms to be described in a procedural way. Procedural means nothing other than: first do x and then do y.

A common problem is: »How to sort a set of x?«.
An easy to understand solution is bubble-sort:

Iterate over the set from the last element as long as you have not reached the first element and while iterating

Start a 2nd iteration from the beginning to the current element of the first iteration and

Compare the current element of (2) with its successor

If greater, swap positions

That is the algorithmic / verbal description of the bubblesort-algorithm.

Here comes a procedural / pseudocode implementation

bubbleSort(Array A)
  for (n=A.size; n>1; n=n-1){
    for (i=0; i<n-1; i=i+1){
      if (A[i] > A[i+1]){
        A.swap(i, i+1)
    }
  }
}

That was easy.

How does that releate to OOP?
You could use this algorithm to treat collections (an object itself) of objects:

Example in Javascript (though no clean OO-Lingo, but with near to no boilerplate and easy to understand)

objects =[{"name":"Peter"}, {"name":"Paul"}, {"name":"Mary"}]

compareObjects=function(x,y){ return x.name>y.name };

sorted = objects.sort(compareObjects)

console.log(sorted)

We have a) a collection objects, b) a method common to this collection sort which contains / abstracts away the sorting algorithm and c) our objects Peter, Paul and Mary. The Specification for the sorting is found here.

What is the relation between algorithms and OOP? Are they two independent topics?

From what was said, it should be clear, the answer should be: yes, they are independend.

How OOP can help algorithms? Or in which direction it can affect it?

OOP is just one programming style. It can not help in any kind. Otherwise an algorithm could be implemented in an OO language to do something to objects (as shown)

Are there some problems which only can be presented and solved by OOP?

I can not think of one (but that doesn't mean, that it is impossible). But if you look it the other way around: OOP is useful, if you want to model some problems and solve ith with an appropriate algorithm. Say you have a record of friends you could model them as objects with properties and if you want a list of friends sorted in any way, you could use the example-code given above to do exactly that.

Why algorithms textbook are more procedure-oriented?

As said: it is more natural, since procedural is the character of algorithms.


问:
Place variables frequently used by the same function on the heap?
Say that I have the following situation:

```<code>void myFunc()
{
    int x;
    //Do something with x
}
</code>```

"x" is placed on the stack which is no doubt fast.

Now, "myFunc" is called very frequently, lets say 10 times per second.
Is it plausible to do something like this:

```<code>int x;
void myFunc()
{
    //Do something with x
}
</code>```

so that x gets allocated in the applications data segment. but it is allocated only once. Since "myFunc" is called so frequently, does the second approach deliver any performance benefits?

答:
Allocating a variable on the stack and deallocating it is a simple addition and subtraction of the stack pointer. Given that it happens anyway when entering a function means that local variables are so cheap that trying to optimize them to anything else will generally incur more cost.

Putting it in the data segment will incur a cache cost, the stack will usually be in cache.

The biggest disadvantage is that you lose reentrant properties. Meaning that recursion won't work and that it's not thread-safe.

The other big disadvantage is that letting the variable escape scope is that the optimizer can't make it a register-only variable, which never reaches RAM to begin with.


问:
Should factory make context or concrete strategies objects in strategy-factory pattern?
In a typical strategy pattern

```<code>class Strategy
{
public:
    virtual int execute() const = 0;
}

class StrategyA : public Strategy
{
public:
    int execute() const override;
}

class StrategyB : public Strategy
{
public:
    int execute() const override;
}

class Context
{
public:
    Context() = delete;
    Context(Strategy* the_strategy);
    int execute() const;
private:
    Strategy* the_strategy_;
}
</code>```

Should it be preferred to use a Context factory with the correct strategy injected

```<code>class ContextFactory
{
public:
    std::unique_ptr<Context> make(/*some parameters*/);
}

void ContextUser(/*some parameters*/)
{
    ContextFactory a_context_factory;
    auto a_context = a_context_factory.make(/*some parameters*/);
}
</code>```

or a Strategy factory, leaving injection to the caller?

```<code>class StrategyFactory
{
public:
    std::unique_ptr<Strategy> make(/*some parameters*/);
}

void ContextUser(/*some parameters*/)
{
    StrategyFactory a_strategy_factory;
    Context a_context(a_strategy_factory.make(/*some parameters*/));
}
</code>```

It seems to me the Context factory should be preferred as:

Less work for the caller.

Factory can manage lifetime of Strategy objects.

Facilitates design change; if the strategy pattern was latter changed to another design, say inheritance, the client code wouldn't change, only the factory.

答:
You bind your factory to Context. Here's some disadvantages for your consideration.

More annoying to implement factories.

If you want to reuse the strategies for something other than Context, you have a problem.

If Context needs a second, independent strategy, you have a problem. With one factory responsible for creating the context, you now need N*M factories, one for each strategy combination.


问:
Open closed principle vs abstraction leaking (Java enums)
In Java, an enum is not a plain replacement for a number (like in C/C++), but a family of objects which can have properties. For instance

```<code>public enum Order {
  NAME("Ordering by name"),
  SURNAME("Ordering by surname");

  private String comment;

  private Order(String comment) {
    this.comment = comment;
  }

  public String getComment() {
    return this.comment;
  }
}
</code>```

And then, I could do:

```<code>System.out.println(Order.NAME.getComment());
</code>```

Now, imagine I define an API like:

```<code>public interface MyApi {
  public List<People> getAllPeople(Order order);
}
</code>```

I can do several things in the implementation:

A switch with the enum, defining the ORDER BY clause as needed for each value.

Add a new parameter to the enum:

an extreme example would be, with SQL, NAME("Ordering by Name", " ORDER BY peo_name") 1, and just do

if (order != null) { sql += order.getOrderClause(); }

Using JPA, I would use the property and not the column name, so I could do `NAME("Ordering by Name", "name") 2 and do

if (order != null) {jpql += " ORDER BY p." + order.getOrderAttribute(); }

On the plus side, I find it as the most straightforward way of implementing the Open/Closed principle (if I add a new ordering criteria, I just need to add the respective enum instance). On the less positive way, I am exposing internal details of my server to the client (although the client is not able to "inject" a different value to profit from it).

What kind of problems can I expect from this breach of abstraction? Is it worth it the simplicity of coding? There is any -hopefully- simple3 way to improve this design (the JPA one)?

1 peo_name being the name of the column in the DB.

2 name would be an attribute of People, that is, there exists a getName() method in People.

3 One alternative I can think of would be using a new enum, Order2, as part of the API, and then link each instance of such enum to one instance of Order, but I think it is complicated and somewhat breaks the open/closed principle (to add a sorting criteria I would need to modify both Order and Order2).

答:
Abstraction leaks are a problem only to the degree that callers may rely on undocumented internals, and mysteriously break when you change those internals. For instance (since we're talking about ordering), if you return a list of things and it happens to always be sorted, callers might assume that this is part of the contract and break if you reimplement the method in a different way.

In this case, the leak is a problem to the degree that callers will come to expect the particular value that the additional method returns and depend on it. That seems unlikely to me, but you probably know your users better than we do, so it's your call. In general, I like utility methods in enums very much, precisely for the reasons you give.


问:
What does one mean by UI Independent client logic?
I was reading this excellent post on Gmail's Blog.

And I came across this line in the end which says,

If you’re building an application that (a) has significant UI
  independent client logic

What does one mean by UI Independent client logic?

Can someone please explain this with some examples?

答:
Let's assume you have an application that looks at the last 100 payments performed with a creditcard and displays a graph for these with the payments grouped into different categories (food, car, sports,...). Instead of the server performing all the grouping and calculating you decide to take advantage of the processing power on the clients (which is what google does for GMail) and only send the clients a list of payments.

There would be logic that takes the payments, groups them into the categories and calculates the totals for the different categories. That logic does not depend on the UI but is performed on the client side and is UI independent client logic.

On the other hand, there must also be logic to render these things onto a graph. This will probably involve drawing or rendering images and will require logic to see how big a certain part of the graph needs to be rendered. This is logic that depends on the UI and is performed on the client side so UI dependent client logic.

Now, if you have different applications (native apps, mobile website) they would all have the same logic to group the payments. What would differ between these applications would be the way the graph needs to be rendered. Hence the former is UI independent, the latter UI dependent logic.


答:
UI Independent client logic

For most applications, this would be the model in the MVC, MVP, MVVM, … patterns.

The model is all the data and business logic needed to run an application.

In old incarnations of the web, this was all done server side with the browser only presenting views to the user. Now, more applications are running some/most/all of the model within the browser. Ideally, an application should run as a set of APIs without any reference to views (DOM elements) or view binding logic.

Body:
I was reading this excellent post on Gmail's Blog.

And I came across this line in the end which says,

If you’re building an application that (a) has significant UI
  independent client logic

What does one mean by UI Independent client logic?

Can someone please explain this with some examples?


问:
Use Case Diagram. Can a system be its own actor?
Hello I got the definition below for an actor. That means the system cannot be an actor to itself. I was wondering what if the system is supposed to display certain elements on a screeen. In that case would the system be its own actor in the system? Probably not . Then how would I present that in a use case diagram ?

Actors: An actor is a person, organization, or external system that
  plays a role in one or more interactions with your system. Actors are
  drawn as stick figures. - See more at:
  http://www.agilemodeling.com/artifacts/useCaseDiagram.htm#sthash.1jE5S2CY.dpuf"

答:
The system can display certain elements on a screen, means that an actor can "Check those elements on the screen". Try not to describe your system from the point of view of the "system", but from the "Users" point of view: what can they do with the system?

I believe an example would help:

Let's say I want to talk about an ATM as a system. The use cases the "Bank Customer" can use:

Withdraw Money

Check Account Balance

It would not be ok to describe from the Banks point of view. But If you try you should get something like: The system can:

Give money on request (maps to withdraw money)

Display balance (maps to Check acct. balance)


答:
Who are the viewers of the screen? Aren't they the ones who "want" to view the information?

About the question "Can a system be its own actor?"

Perhaps a system can be its own actor.  If you view a system as an intelligent goal driven agent that is proactive towards its own goals and acting based on its beliefs, desires, obligations, and intentions (see, amongst others the work of the Intelligent Systems group at the Utrecht University), then one could vie that system as its on actor.

But let me emphasize, I don't think this way of modelling would help you for the display system.

Title: Use Case Diagram. Can a system be its own actor?

Body:
Hello I got the definition below for an actor. That means the system cannot be an actor to itself. I was wondering what if the system is supposed to display certain elements on a screeen. In that case would the system be its own actor in the system? Probably not . Then how would I present that in a use case diagram ?

Actors: An actor is a person, organization, or external system that
  plays a role in one or more interactions with your system. Actors are
  drawn as stick figures. - See more at:
  http://www.agilemodeling.com/artifacts/useCaseDiagram.htm#sthash.1jE5S2CY.dpuf"


答:
Algorithms = tell the "How" story (i.e. how to manipulate input data using data structures in time to produce desired results)

OOP = a "Methodology" facilitated by OO-languages to write programs (=algorithms + data structs) that gives you memory safety and abstraction

OOP is just a paradigm of implementing algorithms.

Good analogy: movies

You can record scenes by employing a stuntman or not. Screenplay (algorithm) does not change. People shouldn't see any difference in final result.

EDIT: you could try out a good quality MOOC: https://www.coursera.org/course/algs4partI which interleaves the discussed topics (especially OOP approach) and gives the essence of what you ask here.


答:
What is the relation between algorithms and OOP? Are they two independent topics?

Algorithms are about how to solve a problem (How to generate output from the given input), OOP is about how to formulate or express our solution (the steps of the algorithm).

An algorithm can be described in natural language or in assembly language, but the concepts we have in a natural language help us to write and understand it better. For example the algorithm for bubble-sort could be:

```<code>bubbleSort(Array A)
  for (n=A.size; n>1; n=n-1){
    for (i=0; i<n-1; i=i+1){
      if (A[i] > A[i+1]){
        A.swap(i, i+1)
    }
  }
}
</code>```

To hide the details of swap and to relate it to the A we used an OOP syntax and feature, then OO makes the algorithm closer to our natural language and understanding.

Are there some problems which can only be presented and solved by OOP?

No, if you consider that any program (or algorithm) on a computer will be translated to a set of instructions executed on CPU (Turing Machine) and if we regard these instructions as the ultimate algorithm which solves the problem on a computer, then OOP can't do something further. It just makes it closer to the human understanding and reasoning. It's a way to package our procedures and data structures.

How OOP can help algorithms? Or in which direction it can affect it?

It may help to state or formulate an algorithm easier or more understandable. It can hide details and provide a big picture of the solution.

In theory, the algorithm is first and implementing it the second. But in reality, we can't be sure that our algorithm works as expected until we trace it or generate the expected output. Computers help us to do that, but you don't expect to write it in machine language (assembly).

In this regard, OOP facilitate the implementation, test and refinement of our algorithm on computers and write it for a computer in a language near to our natural language.

Body:
What is the relation between algorithms and OOP? Are they two independent topics?

Algorithms are about how to solve a problem (How to generate output from the given input), OOP is about how to formulate or express our solution (the steps of the algorithm).

An algorithm can be described in natural language or in assembly language, but the concepts we have in a natural language help us to write and understand it better. For example the algorithm for bubble-sort could be:

bubbleSort(Array A)
  for (n=A.size; n>1; n=n-1){
    for (i=0; i<n-1; i=i+1){
      if (A[i] > A[i+1]){
        A.swap(i, i+1)
    }
  }
}

To hide the details of swap and to relate it to the A we used an OOP syntax and feature, then OO makes the algorithm closer to our natural language and understanding.

Are there some problems which can only be presented and solved by OOP?

No, if you consider that any program (or algorithm) on a computer will be translated to a set of instructions executed on CPU (Turing Machine) and if we regard these instructions as the ultimate algorithm which solves the problem on a computer, then OOP can't do something further. It just makes it closer to the human understanding and reasoning. It's a way to package our procedures and data structures.

How OOP can help algorithms? Or in which direction it can affect it?

It may help to state or formulate an algorithm easier or more understandable. It can hide details and provide a big picture of the solution.

In theory, the algorithm is first and implementing it the second. But in reality, we can't be sure that our algorithm works as expected until we trace it or generate the expected output. Computers help us to do that, but you don't expect to write it in machine language (assembly).

In this regard, OOP facilitate the implementation, test and refinement of our algorithm on computers and write it for a computer in a language near to our natural language.


答:
In addition to the great answers, I will mention an additional conceptual commonality between OOP and Algorithms.

Both OOP and Algorithms strongly emphasize the use of preconditions and postconditions to ensure the correctness of code.

In general, this is standard practice in all areas of computer science; however, this guiding principle results in an evolutionary path in OOP that makes it mutually beneficial to implement algorithms in OOP environment.

In OOP, a group of objects that can satisfy the same contract (preconditions and postconditions) can be made to implement an interface. The user of such interface will not need to know which implementation is used in the underlying object, except in some rare situations (in which leaky abstraction happens).

An algorithm is an implementation of steps used to perform a computation, one that will take the precondition and produce the postcondition.

Therefore, one can borrow the idea of abstraction in the way of preconditions and postconditions, and apply it to algorithms. You will find that sometimes complicated algorithms can be decomposed into smaller steps, and these smaller steps can permit different implementation strategies as long as the same preconditions and postconditions are met.

By implementing algorithms in OOP, one can make these smaller steps interchangeable.

Finally, keep in mind that FP and OOP are not mutually exclusive. Anything described above can be equally applicable to FP as well.


问:
"Use map instead of class to represent data" -Rich Hickey
In this video by Rich Hickey, the Clojure creator, he advises to use map to represent data instead of using a class to represent it, as done in Java. I don't understand how it can be better, since how can the API user know what the input keys are if they are simply represented as maps.

Example:

```<code>PersonAPI {
    Person addPerson(Person obj);
    Map<String, Object> addPerson(Map<String, Object> personMap);
}
</code>```

In the second function, how does the API user know what the needed inputs are for creating a person?

答:
Exagg'itive Summary (TM)

You get a few things.

Prototypal inheritance and cloning

Dynamic addition of new properties

Co-existence of objects of different versions (specification levels) of the same class.

Objects belonging to the more recent versions (specification levels) will have extra "optional" properties.

Introspection of properties, old and new

Introspection of validation rules (discussed below)

There's one fatal drawback.

Compiler doesn't check for misspelled strings for you.

Automatic refactoring tools won't rename property key names for you - unless you pay for the fancy ones.

The thing is, you can get introspection by using, um, introspection. This is what usually happens:

Enable reflection.

Add a large introspection library into your project.

Mark off various object methods and properties with attributes or annotations.

Let the introspection library do the magic.

In other words, if you don't ever need to interface with FP, you don't have to take Rich Hickey's advice.

Last, but not the least (nor the prettiest), although using String as property key makes the most straightforward sense, you don't have to use Strings. Many legacy systems, including Android™, uses integer IDs extensively through the entire framework to refer to classes, properties, resources, etc.

Android is a trademark of Google Inc.

You can also make both worlds happy.

For the Java world, implement the getters and setters as usual.

For the FP world, implement the

Object getPropertyByName(String name)

void setPropertyByName(String name, Object value) throws IllegalPropertyChangeException

List<String> getPropertyNames()

Class<?> getPropertyValueClass(String name)

Inside these function, yes, ugly code, but there are IDE plugins that will fill that up for you, using... uh, a smart plugin that reads your code.

The Java side of things will be just as performant as usual. They'll never use that ugly part of the code. You might even want to hide it from Javadoc.

The FP side of the world can write whatever "leet" code they want, and they typically don't yell at you about the code being slow.

In general, using a map (property bag) in place of object is commonplace in software development. It is not unique to functional programming or any particular types of languages. It may not be an idiomatic approach for any given language, but there are situations which calls for it.

In particular, serialization/deserialization often requires a similar technique.

Just some general thoughts regarding "map as object".

You still have to provide a function for validating such a "map as object". The difference is that "map as object" allows for more flexible (less restrictive) validation criteria.

You can easily add addition fields to the "map as object".

To provide a specification of the minimum requirement of a valid object, you will need to:

List the "minimally required" set of keys expected in the map

For each key whose value needs to be validated, provide a value validation function

If there are validation rules that need to check multiple key values, provide that as well.

What's the benefit? Providing the specification this way is introspective: you can write a program to query the minimally required set of keys, and to obtain the validation function for each key.

In OOP, all of these are rolled up into a black box, in the name of "encapsulation". In place of machine-readable validation logic, the caller can only read human-readable "API documentation" (if fortunately it exists).