问:
Java App as Tomcat Service?
I currently have a basic j2ee website. Currently, a jsp frontend gets data from the backend (the java program, lets call it object "Score"). The backend gets this data from a web site.

```<code><% Main m = new Main(5); %>
<table style="width: 100%">
  <tr>
    <th align="left">NBA</th>
    <th style="text-align: right"><%=m.getTime("nba")%></th>
  </tr>
</table>
<% Iterator itrNba = m.getNbaM().entrySet().iterator();
   for (Map.Entry<Integer, Match> entry : m.getNbaM().entrySet()) {
     Match value = entry.getValue(); %>
</code>```

However, this only happens once. If the user wants new data, they have to refresh the page, the backend will run the algorithm to populate some objects (HashMaps), and the jsp displays the values.

The backend data is sports scores and I want this to be a live feed. Therefore I have to make the back end poll the source, and the front end poll the backend (not necessarily the same intervals either).

How can I make my data objects persist? With my current implementation, every time the page is loaded, a new Score, when Score should be the same for all clients (the frontend jsp). So I made my data objects static and I put the main entry point in a loop, and it isnt working.

Im thinking my main java Scores object needs to be run as a service. Do I need to use servlets here?

Is there an easy way to accomplish what I'm describing?

答:
So, lets think about how the current approach works.

You've got a jsp that is invoked

It creates a Main object.

The Main object goes out and does some work and provides this information via its fields or method after control has returned.

The jsp accesses these fields or methods and presents them to the end user.

This has a number of issues with scalability that you're seeing.  You could be doing what appears to be a denial of service on the target site of the data.  Thats not good (though that said, they've probably got better caching and network than you do - still, it isn't good). The work that you do on your own system isn't shared between requests and so every request has to do the work again (even if nothing is changed).

There are two parts to this application, and as written there is a clear separation between the two.  That makes things a little bit easier.

If this is transient data (you don't need to persist it for more than a short period of time), the easiest thing to do would be to store it in the Context.  There are several contexts available - Application (all of Tomcat), Servlet (that servlet's info), Session (one user's session info), and Request (note: this is framework dependent - you've got things like Spring, PrimeFaces, or your own ad hoc).

Now, I'm going to start out with you could do it this way, but its a way that is rather... undisciplined.  The lack of discipline may make it easier to implement now, but harder to extend properly later.  So make sure you keep reading after seeing the "it could be done this way".

You could create an object that conforms to the JavaBeans naming standards that has the data and the timestamp (for caching) and stick it in the application context.

It would look something like:

```<code>package com.mycompany;

public class MyBean {

  private Date date;
  private int score;

  public Date getDate() { return date;  }
  public void setDate(Date arg) {  date = arg; }
  public int getScore() { return score;  }
  public void setScore(int arg) { score = arg; }
}
</code>```

Register it as part of the context of the application in web.xml and then have Main look it up, check to see if it needs to fetch new data or not, and if so, fetch the data from remote and store the data in the bean.  A bit more of this spelled out in Tomcat Standard Resource Factories in the Tomcat docs.  Essentially, you are creating a Singleton that you can access.  I'm not going to say that this is a good approach, but it will work.

The thing is, thats kind of ugly.  Yea, you can store the data there, but its still kind of ugly. You shouldn't have any problem with a simple class like the one described, but if you have something more complex, its possible to get race conditions where two requests both decide that they want to fetch the data and each store part of the data which is mismatched.

This also really doesn't scale.  If you have two servers (for redundancy) you could get one with the data from 10 minutes ago and one from 5 minutes ago.  And you hit the page and you get one answer, then you hit it again and you get another answer, and then you hit it again and you get the first answer.  With game scores, this could be very confusing.

And here's where I'm going to start getting architect astronauty.

You need to have the data somewhere.  Put it in a database.  It could be a classic relational one like MySQL; or if the data fits the appropriate model, a NoSQL database of some sort might be appropriate.  The specifics of which NoSQL database would be the right one depends significantly on the type of data you are storing and how you want to access it (and remember, if you are dealing with multiple databases for redundancy, the eventual consistency of some flavors of NoSQL can bite you hard).

So, you've got a database.  You also have a process that runs outside of the Tomcat container that puts the information in it.  This could be a quick script or its own Java application.  Its just fetching data from some source and reshaping it to fit in the correct database.  It runs once a minute or so (depending on what your needs are).  Thats all, it fetches data, and puts it in the database.

Now, instead of Main fetching the data from the site, it fetches the data from the database.  This is fast.  Frankly, you shouldn't be doing it from a separate class at this point - just fetch the data.  It also fetches a complete record when it does so that you don't get race conditions in there.

You could stick it right in the JSP, though thats seen as rather ugly.  I'll tell you up front that this offends my sensibilities.

What you've got right now is a model 1 application.  It works, and its easy to write and deploy.  But it mixes the business and presentation logic in the JSP.  You've also got very limited sharing of code between JSPs which means there is lots of potential code duplication which makes maintain a mess.

You really should look at a model 2 approach which has a servlet handling the 'traffic cop' aspect of the web application, a data access layer that fetches the data from some data source, and the jsp that gets the request from the servlet, pulls out the appropriate info stored in beans and uses Expression Language (often written EL) instead of scriptlets in the jsp.

Yes, this is a bit more ceremony in creating it.  However it's a well known and standardized model that another Java programmer could come in and immediately recognize and build on with less work.  It also makes it feasible to test small parts of it to make sure it all still works (JSPs are notoriously difficult to unit test).

You could do this with straight up servlets (which is quite doable), or look at using a framework such as Spring, Stripes, or Play to make it easier to work with.  The frameworks put a structure around the code and how it is written - adding some discipline to the layout and enforcing it to ultimately make a higher quality product.

OwnerDisplayName: Allen

Title: Java App as Tomcat Service?

Body:
So, lets think about how the current approach works.

You've got a jsp that is invoked

It creates a Main object.

The Main object goes out and does some work and provides this information via its fields or method after control has returned.

The jsp accesses these fields or methods and presents them to the end user.

This has a number of issues with scalability that you're seeing.  You could be doing what appears to be a denial of service on the target site of the data.  Thats not good (though that said, they've probably got better caching and network than you do - still, it isn't good). The work that you do on your own system isn't shared between requests and so every request has to do the work again (even if nothing is changed).

There are two parts to this application, and as written there is a clear separation between the two.  That makes things a little bit easier.

If this is transient data (you don't need to persist it for more than a short period of time), the easiest thing to do would be to store it in the Context.  There are several contexts available - Application (all of Tomcat), Servlet (that servlet's info), Session (one user's session info), and Request (note: this is framework dependent - you've got things like Spring, PrimeFaces, or your own ad hoc).

Now, I'm going to start out with you could do it this way, but its a way that is rather... undisciplined.  The lack of discipline may make it easier to implement now, but harder to extend properly later.  So make sure you keep reading after seeing the "it could be done this way".

You could create an object that conforms to the JavaBeans naming standards that has the data and the timestamp (for caching) and stick it in the application context.

It would look something like:

package com.mycompany;

public class MyBean {

  private Date date;
  private int score;

  public Date getDate() { return date;  }
  public void setDate(Date arg) {  date = arg; }
  public int getScore() { return score;  }
  public void setScore(int arg) { score = arg; }
}

Register it as part of the context of the application in web.xml and then have Main look it up, check to see if it needs to fetch new data or not, and if so, fetch the data from remote and store the data in the bean.  A bit more of this spelled out in Tomcat Standard Resource Factories in the Tomcat docs.  Essentially, you are creating a Singleton that you can access.  I'm not going to say that this is a good approach, but it will work.

The thing is, thats kind of ugly.  Yea, you can store the data there, but its still kind of ugly. You shouldn't have any problem with a simple class like the one described, but if you have something more complex, its possible to get race conditions where two requests both decide that they want to fetch the data and each store part of the data which is mismatched.

This also really doesn't scale.  If you have two servers (for redundancy) you could get one with the data from 10 minutes ago and one from 5 minutes ago.  And you hit the page and you get one answer, then you hit it again and you get another answer, and then you hit it again and you get the first answer.  With game scores, this could be very confusing.

And here's where I'm going to start getting architect astronauty.

You need to have the data somewhere.  Put it in a database.  It could be a classic relational one like MySQL; or if the data fits the appropriate model, a NoSQL database of some sort might be appropriate.  The specifics of which NoSQL database would be the right one depends significantly on the type of data you are storing and how you want to access it (and remember, if you are dealing with multiple databases for redundancy, the eventual consistency of some flavors of NoSQL can bite you hard).

So, you've got a database.  You also have a process that runs outside of the Tomcat container that puts the information in it.  This could be a quick script or its own Java application.  Its just fetching data from some source and reshaping it to fit in the correct database.  It runs once a minute or so (depending on what your needs are).  Thats all, it fetches data, and puts it in the database.

Now, instead of Main fetching the data from the site, it fetches the data from the database.  This is fast.  Frankly, you shouldn't be doing it from a separate class at this point - just fetch the data.  It also fetches a complete record when it does so that you don't get race conditions in there.

You could stick it right in the JSP, though thats seen as rather ugly.  I'll tell you up front that this offends my sensibilities.

What you've got right now is a model 1 application.  It works, and its easy to write and deploy.  But it mixes the business and presentation logic in the JSP.  You've also got very limited sharing of code between JSPs which means there is lots of potential code duplication which makes maintain a mess.

You really should look at a model 2 approach which has a servlet handling the 'traffic cop' aspect of the web application, a data access layer that fetches the data from some data source, and the jsp that gets the request from the servlet, pulls out the appropriate info stored in beans and uses Expression Language (often written EL) instead of scriptlets in the jsp.

Yes, this is a bit more ceremony in creating it.  However it's a well known and standardized model that another Java programmer could come in and immediately recognize and build on with less work.  It also makes it feasible to test small parts of it to make sure it all still works (JSPs are notoriously difficult to unit test).

You could do this with straight up servlets (which is quite doable), or look at using a framework such as Spring, Stripes, or Play to make it easier to work with.  The frameworks put a structure around the code and how it is written - adding some discipline to the layout and enforcing it to ultimately make a higher quality product.


问:
When is inlining worth it?
Modern compilers often inline functions when they decide it is worth it.

But here comes my question: how does one define if it is optimal to inline function at given time, or more important how to decide that it should be avoided?

答:
It's a trade off between pushing the parameters and registers on the stack and a jump plus the jump back and restoring it all vs. code bloat.

Another factor is the possibility of other optimization available on the inlined code. Particularly when some of the parameters are known.

For example inlining printf when called with a static string will let the compiler eliminate parsing the string at runtime and emit the formatting code directly. In other words printf("%d", x); can get optimized to a itoa and a write.

ContentLicense: CC BY-SA 3.0

Title: When is inlining worth it?


问:
Should you mock key-value stores/aggregators
I've always been taught that your tests should run in isolation, and other than a db  for ActiveRecord etc., your tests should not rely on any external software/services. e.g. I always mock my other web-service responses but what about responses from web based key-value stores and aggregators like elasticsearch, redis, and statsd? My intuition would be to mock those as well as I am not testing if they work I am testing my apps IO with them. However I see examples around where people uses namespace strategies to test their elasticsearch and statsd related code.

How much should I mock when it comes to web based services?

thanks!

答:
Treat them as an ordinary database.

When you are testing business code which uses a database, you mock the database in order to test just the business code (as well as for making tests slightly faster). The same applies to key-value stores.

What you may have seen is:

Either integration and system tests which, indeed, rarely use mocks. Usually, a system test will run with a database filled with test data. Some system tests may even use third-party web services (some system tests may still mock those web services if you pay per request).

Or simply unit tests written by a person who is unaware of proper testing techniques.


问:
Open-source licenses and modifications
I see in the GPL license FAQ :

Does the GPL require that source code of modified versions be posted
to the public? (#GPLRequireSourcePostedPublic)

The GPL does not require you to release your modified version, or any part of it.
You are free to make modifications and use them privately, without ever
releasing them. This applies to organizations (including companies), too; an
organization can make a modified version and use it internally without ever
releasing it outside the organization.

I always saw the GPL license as a more restrictive license than MIT, BSD, etc., and as a way of being sure that if modifications/improvements are done (and/or used in a commercial context), the improvements would benefit to the original author and the whole community. Because of the FAQ answer mentionned here, it seems that I was wrong.

Question:

Doesn't the GPL license help to be sure that improvements of a software can benefit to the whole community? If so, are there some other well-known licenses that care about this question?

Let's take an example. Is there an open-source license that requires that if somebody does a new JSBIN by reusing parts of https://github.com/jsbin/jsbin,  then this new "online service" would need to be open-source?
It seems that GPL does not offer this, according to the point discussed in the first part of this question.

答:
Yes, undistributed improvements don't need to have source code released. This was an intentional decision when designing the GPL; you're allowed to modify the code for your own purposes without needing to tell anyone what you're doing, or to distribute anything related to it. The GPL was never intended to say "if you improve the code, you must release the improvements", but rather "if you do release the improvements, you must release them as free software."

The goal of the GPL is definitely not to ensure that any improvements benefit the whole community. Instead, the GPL is designed to help give users the four freedoms defined by the FSF. With permissive licenses, open-source software can be incorporated into closed-source software with restrictions that the FSF doesn't think are ethical (in some cases, a closed-source program can completely displace the FOSS program it's derived from, and the FSF certainly doesn't want that to happen); thus, the GPL requires anyone distributing GPLed software or derivatives of GPLed software to give users those freedoms. Otherwise, closed-source software could use open-source code but not vice versa, giving closed-source a competitive advantage.

This only applies to users of the software; while users must be able to distribute it, non-users have no right to obtain it (they have to find a user who wants to distribute it to them). Freedom 3 says that users must be able to let the community benefit from their changes, but if you don't want the community to do that, you're free to not distribute them. The only rule is that any users of your changes must also get the four freedoms, so you must GPL your derived work if you release it. Non-users have no rights relating to your software; that's just not what the GPL is for. Internal corporate use is a bit of a special case (because your helpdesk people likely don't have access to the code of your modified GPL ticket management system for administrative reasons); there, since the employees are employees using it just as part of their job, the FSF considers the "user" to be the company, not the employees. Likewise, using a public kiosk doesn't make you the user of the software for free software purposes, so you don't get source code.

That was the original purpose: the community often gets improvements, but that's a side effect of ensuring user freedom. Someone who just wants to do something for their own purposes doesn't have to care. However, with the Internet, the line was blurred between "internal use" and "sharing with the world" -- strictly speaking it's similar to public kiosks, but it feels kinda different. With the Internet, you can use your program as part of something you're offering the whole world, while never actually distributing it under the GPL's meaning. While modifications generally got distributed under the GPL, the Internet means that many kinds of software (e.g. database software) can easily be used to provide a service to the general public without being distribution under the GPL (I think this vs. kiosks is mostly a matter of scale and context), so the FSF added a different license (the AGPL) that says that if you host an AGPL program for people to interact with (but only running it on your server), and you modify it, you have to share the modifications. The AGPL isn't that commonly used, though.

Title: Open-source licenses and modifications


问:
How can a company sell licenses for GPLed software without violating rights of others
There are tons of questions like "Can I use GPLed code X in my commercial project?" but I found nothing which answers my question.

I take ExtJS as an example, but actually you can any commercial / community edition dual licensed software.

If I contribute to the community edition, isn't my contribution GPLed?

Even if my contribution was small (like a bug fix) how come they're allowed to sell it?

Disclaimer: I know, we're not lawyers. Just wondering.

答:
Dual licences

Software licences are not exclusive, you can give the same code to everyone under terms of licence A, and to some people (who pay you) under the different terms of licence B. Of course, that refers to your code. If other people develop and distribute additions to your system under the GPL, then you can always reintroduce them back to your system, but if you want to futher re-licence code developed by others, then you need their permission - such as the Contributor License Agreements that are used by some organizations.

You can sell GPL software

Also, GPL allows you to sell that software, and it doesn't require to offer the software to everybody. In essence, it requires to give a copy of the source code and the relevant rights when you distribute the code and nothing much else.

You can distribute GPL software to select paying customers only or request payment of $x per seat - with the caveat that you must give them the software, the source, and they gain the full GPL rights, including a right to use, modify and freely redistribute. This doesn't require any extra permissions - I am allowed to do that with your code that you published under GPL, even if I didn't contribute a single thing to it.

In practice this means that you need to offer a sufficient added value for people to keep paying you instead of obtaining the software from any of your customers (which are also allowed to resell it or publish it freely). It can be feasible if you sell it as a package that involves the GPL software and also some complementary services, consultations or support; others might redistribute the software but you can still charge a solid premium for the whole package.


问:
Large internal features on kanban
This days I'm reading some tutorials/advices about Kanban process and Agile methodologies, but I have two big questions about this Kanban methodology.

What I understood is user stories are about adding bussiness value, like a feature which our customers can use. The main problem is we have a lot of features that looks trivial (simple form that manages synonyms for solr, for example) but are hard to implement and may take several days/weeks to complete, due to scaling/coordination details on our cluster (refreshing the synonyms using zk and reloading from database, for exmaple).

How do you break down this user story into smaller ones since Kanban specificies every user story has to be independent? Can I break it down into user stories about implementation details? (implement coordination algorithm, implement X storage service, etc)

And my last question is about architectural changes/refactoring. Since it doesn't add any business value (invisible to our customers) how do you reflect this tasks in this methodology?

答:
This is a challenge in--dare I say, weakness of--many of the Agile methodologies. By focusing so strongly and intensely on user stories, user-visible value, and user feedback, they make it hard(er) to focus on "internal" improvements that work down "technical debt" or that prepare the project/codebase for future user-visible and user-appreciable improvements. Internal improvements are, almost by definition, not user-visible--certainly not at in a systematic, easy-to-value, or leading indicator sense.

In the grand scheme of things, Agile is a reaction to and/or flight from previous methodologies that were all-architecture-all-the-time and deeply pre-planned, with little in the way of incrementalism, feedback loops, or developer input. While the shift has been valuable, the resulting eschewing of much of the deep architecture, long-horizon planning, and internal focus that software projects (whatever their many and genuine flaws) used to have, you could argue that some previous virtues are harder to enjoy. The more extreme the form of Agile, the more likely it is to wholeheartedly disdain preparatory work.

If you want the pattern name for this, it's "throwing the baby out with the bathwater."

Title: Large internal features on kanban


答:
Kanban is an approach that is one that focuses on trying to identify work in progress (and minimize it).  Thats the core concept in it.  It is based on the just in time workflows of Japanese car companies.  Stuff that is stuck somewhere isn't moving as fast as it could through the workflow.  Often it is paired with scrum in which case its known as scrumban though this doesn't have much to do with the agile principles.

For kanban to work well, the stories or tasks should be similar sized.  Having a minivan go through the process that is designed to build scooter sized things will cause problems on an assembly line.  So to it will have problems trying to push a task that takes a month through a workflow that wants to be pushing through tasks that are a week long.  It can be done, but you're going to need to realize what it implies to the rest of the workflow.

Stories or tasks can have dependencies.  Just one needs to make sure that the dependency is complete before the next one moves forward.  Consider the situation where you have two tasks, A and B.  A must be done before B can be coded.  If you have both of these tasks in the backlog at the same time, someone might take B first - this would be bad as it would have a task that is some state that cannot be moved forward - work in progress.  Furthermore, if A is started and is partly done then B is started you can again have the situation where there is work in progress that cannot move forward.

That is why there is an emphasis on independent tasks.  However, that does not mean that all tasks must be independent - just that you shouldn't have two dependent tasks being worked on simultaneously or in the wrong order.  The control for this is the ability to put things in the backlog.

With attention paid to similar sized stories moving through the process and limiting how much can be in a given column at one time it is then possible to give better feedback to the customer about how long it will take for something to go through the workflow.

Thus, you may absolutely break it down into implementation stories or tasks.  If you are using a physical kanban board, consider using a particular color sticky note for a partial detail of a larger story so that these can be seen and understood (possibly including the dependency information on them) so that they can be seen and followed more quickly when trying to get the big picture of the board.

There is nothing in kanban that has a parallel with the oft seen applications of agile that attempt to minimize design time (note, there's nothing in the principles behind the Agile Manifesto that suggests no design either though it may be a reaction to the "Welcome changing requirements, even late in development" or "Working software is the primary measure of progress" being interpreted as "design is worthless" - but I doubt any of the signatories of the Manifesto would make that claim).  When one looks at various demos of kanban boards one sees things like:

That first image has a 'specify' column.  The second one shows an 'Analysis' and 'Design' column.

As described in the principles behind the Agile Manifesto the key thing is to have working software and work with the customer.  The individual responsible for putting tasks into the backlog is perfectly capable of putting internal refactoring tasks in there.  I've also heard of managers doing entire scrum style sprints that are 80% internal tasks when necessary, or have a Friday refactoring day where you work on code cleanup on Fridays.  The wise manager/product owner will do this because without addressing the internal (non-enduser facing tasks) the size of the tasks that can go through become smaller and smaller (and estimations for similar sized things become bigger and bigger over time) as increased technical debt is accumulated on the product.

There is nothing in agile or kanban that says you cannot have tasks of the nature 'refactor library XYZ'. If the product owner approves of the task to go through (or however the workflow for putting things in the backlog is) then do it.  As mentioned, if you don't do it, development will eventually grind to a halt.

I will point out that not doing this goes against the principles: "Continuous attention to technical excellence and good design enhances agility."  By keeping the code base in a constant state of a given quality one can do a better job at predicting how long the entire project will take (rather than having the velocity get slower as things get harder because of the technical debt).

In theory agile processes are supposed to be constantly refactoring the code, though this isn't something I've often seen.  In part it has to do with how empowered the programmers are to say "this is how long something will take".  In the situation of consultants working on a project, there is often a fair bit of "this will take as long as I say it will take," and take into account the constant refactoring.  Such a statement is harder to do with in-house development where its a "it will take a week" ➔ "can you get it done in 3 days if you cut corners/don't do that refactoring thing?" ➔ "well, yea..." ➔ "great! 3 days it is."  This is not so much a problem with agile, but rather the empowerment of the developer in the organization.

Related reading:

What is Scrumban?

Lean Software Scrum-ban

Lean Software Kanban bootstrap I will point out that this post is specifically addressing the very large project issue that deals with breaking down large stories into smaller ones.

Title: Large internal features on kanban

Body:
This days I'm reading some tutorials/advices about Kanban process and Agile methodologies, but I have two big questions about this Kanban methodology.

What I understood is user stories are about adding bussiness value, like a feature which our customers can use. The main problem is we have a lot of features that looks trivial (simple form that manages synonyms for solr, for example) but are hard to implement and may take several days/weeks to complete, due to scaling/coordination details on our cluster (refreshing the synonyms using zk and reloading from database, for exmaple).

How do you break down this user story into smaller ones since Kanban specificies every user story has to be independent? Can I break it down into user stories about implementation details? (implement coordination algorithm, implement X storage service, etc)

And my last question is about architectural changes/refactoring. Since it doesn't add any business value (invisible to our customers) how do you reflect this tasks in this methodology?


答:
Kanban specificies every user story has to be independent

Where did you get this from? Kanban does not require the units of work to be "user stories", and there is nothing in Kanban forbidding you to develop a component first, and then a second component using the first, and so on. And each of this components can still follow a typical Kanban workflow, with analysis at the beginning, and tests and deployment at the end. You may just have to rethink what "deployment" means in this context.

If you have a big feature to be developed over many weeks, you will have to break this down into small subfeatures, or even better, components - at least, if you trying to develop any kind of reasonable architecture. Of course, some intermediate components may not provide any business value to your customer on its own. But actually, since you will develop tests for each subfeature or component, the working tests can be used to proof that a specific component is "ready". And the component provides "business value" for the developer in your team who is going to use the component. So the only thing you have to adapt in your interpretation of Kanban is just your "defintion of done", when it comes to internal components.

Said that, this should not stop you from thinking about how to organize the development process in a way you can deliver value to the final customer as early as possible, and in smaller steps. Using your example: a form with some complex algorithm in the background itself might be developed first with a very basic, simple "dummy" algorithm, and the algorithm is improved or refined in a second step. A storage service might need some kind of administrative console, and / or some kind of logging mechanics to make it work - something which counts as "business value", too.

Title: Large internal features on kanban


问:
Generating test data for a search application
I have a general question about testing search applications, and what I'm looking for is pointers to resources on the topic that I can go and research on my own. I've tried semi-informed, semi-undirected googling, but that's yielding a lot of distractions and blind alleys (or maybe my search skills aren't all that sharp).

A bit of setup first. When I say "a search application", what I mean this:

you have some data sources which you can put together in a search index

your application has an API that takes as input a search query (keywords and optionally other stuff), and its output is a relevance-ranked list of results from the search index.

there is a whole bunch of business logic on top of just retrieving the results from the index - the final result set in the output could have a large edit distance from the original result set from search.

assume that in real life, the index is large and takes a while to build

The task is to write tests for the application. The basic structure of a test is "given search request X, I expect response Y consisting of relevance-ordered results". The problem, therefore, is: what is a good strategy for generating the underlying data for the tests?

Here are some approaches I'm aware of (and have used in practice):

Don't generate test data. Start with a real index, and apply targeted modifications to it to "introduce" edge cases for your tests as needed. Upside: close to real life. Downsides: large test index; has to be rebuilt every time some change is made to the indexing scheme; most of it is unused by existing test cases.

Generate fake data such that for each request X there is a well-defined, intentionally constructed set of results Y that will be returned. Upsides: full control over the search data; only as much data as needed for the tests, quicker and easier to change. Downsides: still have to rebuild everything in the indexing scheme changes; not necessarily realistic data, which may leave aspects of the system untested or under-tested; too much flexibility and test-specific domain knowledge separate from the real-life domain of the application.

Actually that's where my current knowledge ends. Something tells me there is either a nice middle ground which allows for testing flexibility without deviating from how the application works in real life, or a completely different testing approach that does away with these concerns. What approaches might you consider?

答:
Short answer: you need both

fake data, with well defined input X and output Y

real-world data, probably with the modifications you suggested

Use the first one especially when doing TDD (as your tag indicated), and after you have the basic algorithm ready, use the second kind of data for integration or acceptance tests. The first kind of tests will prevent you from the need of running the (probably slow) second kind of tests more often than necessary.

Something tells me  there is either a nice middle ground which allows for testing flexibility without deviating from how the application works in real life or a completely different testing approach

Sorry, but there is no "magic bullet" so far. Testing complex algorithms is hard work, sometimes difficult, requiring analytic skills. There are whole books written about how to construct test cases efficiently, and the techniques described, for example, by Glenford Myers in his book about software testing, which was published first 1979 AFAIK, are still valid today.


问:
Design for a shared checkout
I am currently working on a ecommerce system that is slightly different in structure to a typical ecommerce system in that you have multiple stores, accessing the same database from different URLs.

So for example i might have:

http://www.site1.com

http://www.site2.com

the above are essentially the same store, they point to the same database, in the main will share the same products (this may seem weird but these are the requirements of the client).  The only difference is, based on the URL the users, baskets, orders etc are split so that they do not cross each other. I.e. the orders/baskets/users of site1 are not visible to site2 so a pretty standard single database multitenant model.

The above is implemented, i am just slightly stuck on how to implement the checkout process.  The requirement is to have a shared checkout, so instead of:

https://www.site1.com/checkout

you would have (for clarity this will link to the same database as the above sites)

https://checkout.somesite.com/storeid/basketid

You could tie this in to the way it is handled at Shopify, multiple stores one checkout process.

The checkout process in the main is not difficult, i can use the storeId and basketId to identify what i need to identify in terms of create an order from a basket and assigning to a store if the user is a guest.  The issue i am facing is what to do when an existing user wants to checkout and login so that the order is associated to their account and some basic information (Email Address, Billing Address) is pre-populated for them.

The checkout process (https://checkout.somesite.com/storeid/basketid) does not concern itself with logins or account management of any sort, its just there to facilitate checkout so users, if they are at the beginning of the checkout process and wish to login will be redirected back to the corresponding sites login page.  From there i am thinking about this process, but just wanted to share it to see if there is something glaringly wrong with it:

User Logs in with correct credentials

Some object (may be the Cart or an intermediary CartOrder object) is updated to contain the id of the user, a unique session key (possibly a GUID) and a timestamp to state the expiry of the checkout session

User is redirected back to the checkout site with the session id appended in the querystring (i can't use cookies as the sites are on different domains)

Option: At this point i may create a local session to hold onto the value for the remainder of the checkout process making sure it is still valid along the way (a new login would also expire the session key)

Checkout!

Now there is a slight concern in that placing the session id in the query string it could be picked up by some malicious person, used and some information (name, address) could be visible. On the flip side to that, the checkout will be all SSL, the sessions are short lived and not really able to spoof accounts (or do anything on behalf of the account it represents) so i am not sure if it is being over thought in this process?

答:
Apologies if I misunderstood the question and requirements.

Assumptions

Three unique web sites

shop1.com

shop2.com

checkout.com

Shopping at site 1 or site 2. When you checkout you redirect to site 3.

I would take a look at browser sessions.  They usually have a unique ID across tabs in the browser.

My implementation would be something along the lines of.

•   User shops at shop1.com

•   User does a checkout.  Login required.

•   Capture browser session ID.  Open a socket from the shop1.com server to the checkout.com server and pass it the unique browser ID along with any other info that you require.

•   Perform redirect to checkout.com

•   Get unique browser ID and check if the checkout.com server knows about it.

This way you don't need to use special URL's and parameters and you can pass information between the sites.

Title: Design for a shared checkout


问:
What product is better: ADFSv2, Ping Identity, CS Siteminder?
I'm comparing different STS's for claims based authentication and am trying when it's appropriate to use each one.

It appears that ADFSv2 should be used in all Windows Environments.  Ping and Siteminder are more for the Unix side... although a sales rep said Ping was better for Windows, I didn't really understand "why"

Any comparative information is much appreciated!

答:
How to download Siteminder policy server for windows 2012 server and which version i have to download.

while downloading a hotfix i am getting this message>This solution belongs to a product for which you do not have an active license.

ContentLicense: CC BY-SA 2.5


问:
Scrum: Is it ok for the design/UX of user story to occur in the same sprint as implementation
I am currently in a sprint (two weeks) where the designer is tasked with defining the requirements and UX of a particular user story.

In the same sprint, I am to implement this design. During sprint planning, I had to take a wild guess as to how long this undefined user story would take.

Today I finally received the design. Unfortunately, the design is incomplete/vague and is more akin to a client's requirements than a design. However, from this, I can still see that I have not nearly estimated enough.

To make matters worse this is not the first time. In the last sprint, the exact same thing happened. I flagged it in our retrospective and the scrum master did not have an answer of how to solve this, instead of saying "that's just development for you". Ironically he gets annoyed if the burn down is not on target...

Now I am going to have to ask/work with the designer to get his job done. This is going to hold me up as I have completed all my other tasks.

So my question is

 A) <s>how do you handle dependencies in sprint planning?</s> EDIT: Is it ok for the design/UX of user story to occur in the same sprint as implementation

 B) how should I now approach the sprint? Re-estimate the current user story and watch the burndown turn into a burn up and be viewed as incompetent/unproductive? Or add a new task to the current sprint along the lines of "help the designer create a suitable design"

答:
During sprint planning I had to take a wild guess as to how long this
  undefined user story would take.

That's the mistake you did. Nobody can force the team to accept a task in the sprint and it's your job to state that the task can not be estimated and accepted in the sprint unless there is at least wireframe (for example). It seems that your scrum master is actually a project manager, which only makes the things worse.

How to approach such task if it's necessary to accomplish it within the sprint (for valid business reasons)? Well, first you have to set deadline for the design, after which you won't be able to implement it. For example: the story is accepted, but the design should be delivered within the first week and implemented within the second. Next, you have to work very closely with the designer to ensure it will be possible to be implemented in the sprint. Scrum assumes cross-functional team and it's up to you to organize your work with the designer. Having said that, if you accept the task in the sprint - which is up to the team to decide - it's up to the team to manage the work in a way that it is completed within the sprint, and to manage the associated risks. You should not wait for another one to finish his/her job to get your job done. It's possible that you are about to reveal a bigger dysfunction in your organization.

ContentLicense: CC BY-SA 4.0


问:
A separate endpoint specifically for sync in a REST api
I am working on a multi client/single server app. Communication between the two happens via RESTful API. There are several resources that need to stay in sync across all clients, including images captured from camera, likes and comments on these images, user profiles and user followers/followings. My current approach involves GETing all resources changed on server since last sync, like GET /images?since=<last-sync-timestamp>, GET /users/:id/followers?since=<last-sync-timestamp> etc. and POSTing all client changes since last sync to the same endpoints, like POST /images, POST /user/:id/followers

The main drawback to this approach is way too many network calls - For each resource I need to sync, I need to perform at least 2 network calls.

So I am thinking about introducing a new endpoint like /sync which provides all syncable resources that have changed since last sync timestamp. Both client and server will need to parse the JSON and put the respective data in appropriate tables, but now, no matter how many resources I have to sync, I will only need to perform two network calls, but change the structure of the JSON as reosurces get added/removed.

Any thoughts on why the second approach is superior/inferior to the first one?

答:
In REST, you should think of URIs as identifying an addressable resource. So as long as you think of a "sync" as a noun ;) you can define that resource in whatever manner suits your requirements.

At surface level, in REST we talk about collection resources (like the collection of users) and individual resources (like a single user), so a resource that is an aggregate of other resources is in keeping with that.

The idea of "a resource" is extremely broad. I don't know that there's a precise definition. So an aggregate resource that bundles up the smaller resources saves bandwidth and reduces latency sounds like a good idea.


问:
In a Table Data Gateway, where do I put queries that use joins
The book Pattern of Enterprise Application Architecture has the following description over the Table Data Gateway pattern:

“A Table Data Gateway holds all the SQL for accessing a single table or view: selects, inserts, updates, and deletes. Other code calls its methods for all interaction with the database.”

They do not say anything about queries that use multiple tables. Where do I have to put a query that uses multiple tables and generates a result that has as much in common for every single table?

答:
Despite appearing in a book authored by Martin Fowler (for whom I gave a lot of respect), I can't understand why one would use this pattern. As you have realised, it is pretty limited as most business operations require accessing multiple tables, and sometimes multiple data sources (databases, ldap repositories etc). A secondary concern is performance - you would need to fudge joins between tables at the Java level. When large tables are involved, SQL will generally do this much more elegantly and quicker.

I think the Data Access Object (DAO) pattern is far more useful. The interface should reflect business actions (e.g. placeOrder(), shipOrder(), chargeCreditCard(), listOutstandingOrdersForCustomer() etc) and should hide the fact that multiple tables/repositories etc are involved in a transaction. This also aids testing, as you can easily stub out the database with mock code.


问:
Confusion about inheritance
I know I might get downvoted for this, but I'm really curious.

I was taught that inheritance is a very powerful polymorphism tool, but I can't seem to use it well in real cases.

So far, I can only use inheritance when the base class is an abstract class.

Examples :

If we're talking about Product and Inventory, I quickly assumed that a Product is an Inventory because a Product must be inventorized as well. But a problem occured when user wanted to sell their Inventory item. It just doesn't seem to be right to change an Inventory object to it's subtype (Product), it's almost like trying to convert a parent to it's child.

Another case is Customer and Member. It is logical (at least for me) to think that a Member is a Customer with some more privileges. Same problem occurred when user wanted to upgrade an existing Customer to become a Member.

A very trivial case is the Employee case. Where Manager, Clerk, etc can be derived from Employee. Still, the same upgrading issue.

I tried to use composition instead for some cases, but I really wanted to know if I'm missing something for inheritance solution here.

My composition solution for those cases :

Create a reference of Inventory inside a Product. Here I'm making an assumption about that Product and Inventory is talking in a different context. While Product is in the context of sales (price, volume, discount, etc), Inventory is in the context of physical management (stock, movement, etc).

Make a reference of Membership instead inside Customer class instead of previous inheritance solution. Therefor upgrading a Customer is only about instantiating the Customer's Membership property.

This example is keep being taught in basic programming classes, but I think it's more proper to have those Manager, Clerk, etc derived from an abstract Role class and make it a property in Employee.

I found it difficult to find an example of a concrete class deriving from another concrete class.

Is there any inheritance solution in which I can solve those cases?

Being new in this OOP thing, I really really need a guidance.

Thanks!

答:
I think you should google Peter Coad and read his excellent articles on color data modelling. A good introduction is http://www.step-10.com/SoftwareDesign/ModellingInColour/

Although not explicitly stated in the above article, a Coad modelling workshop I attended used some simple tests to decide when inheritance was the right way to model the data:

Inheritance should be used sparingly. Part of this reasoning is to accommodate modern OO languages that only support single inheritance.

Inheritance should only be used when the subclass applies for the full life of an object. For instance Cat and Dog can extend Animal, as a cat never becomes a dog. But Manager should not extend Employee, as an employee can become a manager. Manager is an employee playing a time-constrained role, not an inheritance relationship. Coad has invented role and moment-interval classes for modelling such roles.

As you note, changing the class type of an entity at runtime feels wrong, and is definitely code smell.

Title: Confusion about inheritance


问:
How do you handle versioning in a multi-sided project?
I know it's a broad question so I'll try to be as specific as possible.
This question is more an "organisational" question than a technical one.

We have a multi-sided project with these main components:

A server, hosting the core business logic (data models)

A backoffice for clients which uses the core business logic

An application API (REST) which uses the core business logic too

There are smartphone apps (iOS and android) using the application API

There are another tablet app (android) different from the smartphone one using the same application API.

Soon, I'll be in production with active clients. And as any project, I'll need to maintain all different components over time. That means all of the following can be upgraded:

the code of core business logic in the server (used by the back-office, the API, and as a side effect, by mobile apps)

the API itself (used by both smartphone and tablet apps)

all the mobile apps (via appstore/googleplay)

Of course, server-side parts (core business logic code and API code) can be changed immediately by myself. However, new mobile apps must be downloaded by the clients on the appstore/googleplay, and I can't be sure they're up to date.

Could you provide any guidance, good practices tips, to make theses upgrades smooth and, unrisked for the client?

Which component do I need to "version"? How to ensure everything works even if the client does not upgrade its mobile app? Should I force him to upgrade to make my work easier?

In a word, how should I organize to make my multi-sided-project live over time?

答:
I recommend you install a continuous integration server, hook it up to your code repository and a snapshot/release repository and automate your builds. This will have a number of advantages:

Every component will be versioned when it is released. This includes low-level libraries as well as your final products.

Every code commit will trigger a snapshot build. This helps keep your developers honest, especially if you use the facility to email the culprit who broke the build.

Every build can run unit tests. This markedly improves code quality.

Every release will be tagged, so if a production fix is required, it is easy to branch from the tag and do the fix.

You will need to maintain a compatibility register of some sort. (e.g. BackOffice 1.0.23 is compatible with REST API 2.0.267 etc). I don't know of a tool that will help in this area.

My experience has been with open source tools: SVN, Maven 2, Jenkins and Nexus, but there are alternatives to all of these.

Don't underestimate the learning time to get your team up to speed. But once they are up to speed, they will never go back.


问:
Why is it you never get as much done as you'd planned?
I always start the day thinking "I'll easily get this done by the end of the day" and set what looks like a realistic target.

So why do I never hit it? The task always ends up taking 3x longer due to unforeseen bugs, last minute changes etc.

Is it just me?  I don't seem to be getting any better predicting what can be done in a day.

答:
Every programmer, once in a great while, has a perfect day.  You wake up 5 minutes before your alarm goes off feeling great.  Breakfast is made and on the counter, along with fresh coffee, so you can grab something and head out the door.  During your commute you hit every green light, and traffic seems to be especially light.  Contemplating the day ahead of you, you are able to fully understand the design and consequences of the task ahead of you, which has been well planned with firm requirements.

You get to work and you find that you have no important emails, no voicemails waiting, and your co-workers are either out or in meetings that you don't have to attend.  You fire up your editor and are immediately in the zone, you can feel the structure of the code and see your data structures and algorithms fitting into place within a beautiful and cohesive whole. Thoughts flow through your hands to the keyboard, entering perfectly formed code that is elegant, maintainable, and with not a bug to be found.

During the day you work with no interruptions, the office is quiet and you are so focused that you're never tempted to spend any time catching up on the news, blogs, etc. When you compile and run your tests, you find that everything works without a hitch, of course you knew it would, and at the end of the day you commit with no conflicts.  Glancing at the clock on your way out you realize you put in 12 hours and it felt like a brief 20 minute coding session.

That day, that perfect day, is what we assume we'll have every time we have to estimate anything.


问:
Why does SortedDictionary not implement IReadOnlyDictionary?
The .NET type SortedDictionary<TKey, TValue> does not implement the interface IReadOnlyDictionary<TKey, TValue>, although Dictionary<TKey, TValue> does.

It is trivial to make SortedDictionary implement IReadOnlyDictionary by subclassing it (< 10 LOCs).

But I guess there are good reasons why the framework developers did not do it and that’s why I shouldn’t do it. What are those reasons?

答:
Looking at MSDN today, one can see that the MSDN documentation page for SortedDictionary<TKey, TValue>, SortedDictionary<TKey, TValue> does implement IReadOnlyDictionary<TKey, TValue>.

According to the internet archive, that was not the case on November 14th, 2013.

Given that information, it is likely that it was simply an oversight from Microsoft, fixed in a recent .NET point update.


问:
Collections, relationships and tracking changes (in DDD)
Let's say we have a Book that has a List of Authors. Order of Authors is important.

We then have the update page, where user can 1) add new author, 2) remove existing author or 3) change the order. What I have as the input is the list of new authors.

I find this problem often - there is often a lot of models with this kind of relationship. Sometimes the order is not important (so we use Set) but more-less is like this.

In my business method, I need to:

detect new Authors, so I can insert them

detect removed Authors, so I can remove them

assume everything else is updated.

I can not simply reapply authors (delete and then insert) since there is additional data related to a book and author.

Doing this manually every time is error-prone. Is there a better way to track changes in related collections?

Should I track changes on UI instead (as this is where they happens) and then provide more input then just one list of all users: for example, list of changes, list of removals? In that case my business method would take more parameters, i.e. lists for each type of change (removal, etc).

Finally, I was thinking into storing set of events per update, so I can repeat them on model easily.

Should I handle this margining in model or in my service method?

Any wisdom on this?

答:
If authors are Value Objects, you can easily re-create the whole list each time because only their value matters, you don't have to relate them to identified objects. This might be at the cost of losing business intent though, since you don't capture what the user precisely wants to do.

If they are Entities, I suggest sending granular commands to Book -- AddAuthor, RemoveAuthor, ChangeAuthorOrder. The latter will have to include distinct order indexes for all authors, otherwise the Book can get in an inconsistent state.


问:
Database design for polymorphic data
I have an application that needs to log communications with users over several different mediums: Email, SMS, Voice, Website Announcements, etc.. in a traditional database.

I have considered 3 approaches to modeling these different types of data:

Single Table Inheritance:

Store them all together in a single table (i.e., comm_message) with discriminator field of some sort to indicate the type of communications (e.g. message_type).  This means that some fields in the table won't be used for each type--and it means that the same message may be duplicated in several different rows in the table (if the message is sent via more than one medium).

Message "has" transports

Have a message table (comm_message) and then transports table (comm_transports) with the various different mediums of communication.  A many-to-many relationship between messages and transports would mean one row for each message in the message table--but that row might have several different transports.  If additional specific information is needed for a particular transport, it could be in it's own table (i.e. comm_sms, comm_email, etc..) that is linked off the many-to-many table. I.e., a "has-a" approach.

Class Table Inheritance:

Create a base message table (comm_message) and then another table for each medium with fields specific to it (inheritance).  My ORM (LLBLGen) would facilitate this approach by using shared PKs for the different tables.  In this approach there would be a row in the base table (comm_message), plus rows in each of the related tables for each transport (comm_email, comm_sms, etc.) but there would be no many-to-many relationship.  Rather the records across different tables would share the same PK (1-1).  This would be more of an "is-a" approach.

Context:  This is a medium sized application (around 100 tables) that I'll be maintaining for many years--so I'd like to get this "right".  I'll often need to present all the communications info together in the UI in a grid, reports, etc..

Which should I use?  Why?

答: Another option is to use a database that natively supports inheritance, e.g. PostgreSQL.</p>

Title: Database design for polymorphic data


答:
I will try to answer with the info you gave me.

You wrote

I need to:

detect new Authors, so I can insert them

detect removed Authors, so I can remove them

assume everything else is updated.

So I think that your approach is too CRUD for DDD and aggregate roots. My 2 cents your business method is something like Update(Book) and the parameter is the modified Book (its data, isbn or title for example, and/or its author collection) and that is why you have to "detect" changes in the aggregate root and why you are in trouble about how to do that.

I would implement a command pattern and segregate every user-case into a command.

Edit book info

Add author

Remove author

Edit author

Change author order

This will need a good task-based UI design.

Then, you can choose the most suitable strategy:

Execute every command instantly and save into persistence. Get Book aggregate root, modify book data or author collection (depends of the command) and update persistence. No need of tracking changes.

Execute every command instantly in the aggregate root in memory and wait user push "Save" button. Then go to persistence. If you store the commands executed in the aggregate until persistence is updated you already know what update in persistence.  Changes tracked by the commands executed in the aggregate.

Store a list of generated commands by the user and execute it in a batch when user push "Save"; use (1) or (2) strategy to update aggregate and persistence.

Last words and tips:

Even UI design affects the architecture and patterns you can use. Think in everything as a whole.

Every strategy listed above has its uses. Depends of your domain, context, environment, technologies you use, etc.

You can use several strategies. No need to stick just in one in every bounded context.

Your domain has to model user-case actions. Don't do just Update(Book). Model every action (every command execution) in its own function and apply its relevant domain rules and invariants.

Title: Collections, relationships and tracking changes (in DDD)

Body:
Let's say we have a Book that has a List of Authors. Order of Authors is important.

We then have the update page, where user can 1) add new author, 2) remove existing author or 3) change the order. What I have as the input is the list of new authors.

I find this problem often - there is often a lot of models with this kind of relationship. Sometimes the order is not important (so we use Set) but more-less is like this.

In my business method, I need to:

detect new Authors, so I can insert them

detect removed Authors, so I can remove them

assume everything else is updated.

I can not simply reapply authors (delete and then insert) since there is additional data related to a book and author.

Doing this manually every time is error-prone. Is there a better way to track changes in related collections?

Should I track changes on UI instead (as this is where they happens) and then provide more input then just one list of all users: for example, list of changes, list of removals? In that case my business method would take more parameters, i.e. lists for each type of change (removal, etc).

Finally, I was thinking into storing set of events per update, so I can repeat them on model easily.

Should I handle this margining in model or in my service method?

Any wisdom on this?


答:
The way I usually do this is based on a full update using a structured representation of the record. Some would call this a Data Transfer Object (DTO). Something like:

```<code>{ # book
  'id': 123,
  'title': 'The Art of Computer Programming',
  'authors': [
    {
       'id': 456,
       'name': 'Donald Knuth'
    }
  ]
}
</code>```

In the list of authors, if the author record already exists, then it gets an id. If there's no id, it's a new record. Deleted authors simply do not appear in the list.

Depending on the application front-end, this is generated in different ways:

In a traditional HTML forms application, the server decodes the posted form data

In an Ajax application, JSON is generated on the client and sent to the server

Mobile and fat clients apps work somewhat like an Ajax app

I always use libraries so I don't generate this by hand: the library generates it based on a structured definition of the form.

The next step is to validate the DTO, and apply it to your domain objects. There are some subtleties to doing it right. In particular, where an author already has an ID, you need to check the ID is already associated with that book. If not, you risk parameter tampering attacks.

Again, I always use libraries to do this. ToscaWidgets has great support for this, again doing all the validation based on a structured definition of the form.

The final step is to persist your domain objects. Most ORMs do this automatically.

This pattern lets your perform complex updates reliably. I've always like the style of large forms with sub-forms - rather than forcing users to click through multiple pages. I use this pattern a lot for highly complex forms, including multiple levels of nesting, and it works great.

Body:
The way I usually do this is based on a full update using a structured representation of the record. Some would call this a Data Transfer Object (DTO). Something like:

{ # book
  'id': 123,
  'title': 'The Art of Computer Programming',
  'authors': [
    {
       'id': 456,
       'name': 'Donald Knuth'
    }
  ]
}

In the list of authors, if the author record already exists, then it gets an id. If there's no id, it's a new record. Deleted authors simply do not appear in the list.

Depending on the application front-end, this is generated in different ways:

In a traditional HTML forms application, the server decodes the posted form data

In an Ajax application, JSON is generated on the client and sent to the server

Mobile and fat clients apps work somewhat like an Ajax app

I always use libraries so I don't generate this by hand: the library generates it based on a structured definition of the form.

The next step is to validate the DTO, and apply it to your domain objects. There are some subtleties to doing it right. In particular, where an author already has an ID, you need to check the ID is already associated with that book. If not, you risk parameter tampering attacks.

Again, I always use libraries to do this. ToscaWidgets has great support for this, again doing all the validation based on a structured definition of the form.

The final step is to persist your domain objects. Most ORMs do this automatically.

This pattern lets your perform complex updates reliably. I've always like the style of large forms with sub-forms - rather than forcing users to click through multiple pages. I use this pattern a lot for highly complex forms, including multiple levels of nesting, and it works great.


问:
Performance of Class methods vs singleton instance methods
In my app, I'm using both strategies:

I have a Utils class, it only has class methods that I call for doing common global methods, like NSString validation, etc.

I have an app-wide singleton class which I instantiate during app Did Finish Launching, and use this throughout the app to hold global objects / state, perform long-running processes, etc.

My question is, in the terms of performance which approach is better? I'm happy with both the approaches, but I can do an inexpensive, but effortful migration to move all the Utils class methods to instance methods in the app-wide singleton if this would give me performance gains. The Utils methods get called a lot in my app, but I don't really notice any performance penalties in Instruments.

Likewise, I can go the other way and move at least a few of the somewhat general purpose functions I end up adding to the singleton to now be class methods in the Utils class as well.

Does any Objective-C guru have some concrete benchmarks or even strong opinions on which approach is more performant?

答:
To start with if you really need to shave those extra few nanoseconds off your Utils C++ functions (potentially inlined) are probably the fastest thing on offer to you.

As to the question of whether class methods or singleton instance methods are more performant I will not claim to know for sure but a quick look at the objective-c runtime shipped with OSX 10.10 shows that _class_getClassMethod calls _class_getInstanceMethod which might lead you to think that instance methods would be faster.

Looking into this further I found that _object_getMethodImplementation calls _class_getMethodImplementation. Which would point to class methods being slightly faster to load.

All this is pretty moot though as in any instance you have 1 or 2 more function pointer accesses. This would only be noticeable if your methods are being called at least millions of times (and at that point optimising your method implementations is probably time better spent).

Stylistically, I agree with what you currently have, using singletons where state is required and class methods where it is not.


问:
Should Form Object be considered a DTO?
I got this from one of online video tutorials. Author binds form data not to Value Object directly, but first to validable DTO, and THEN to Value Object, using service.

Example:

```<code>package com.foo.spring.dto;

import javax.validation.constraints.NotNull;
import javax.validation.constraints.Pattern;
import javax.validation.constraints.Size;

public class SignupForm {

    @NotNull
    @Size(min=1, max=255, message="{sizeError}")
    @Pattern(regexp="[A-Za-z0-9._%-=]+@[A-Za-z0-9.=]+\\.[A-Za-z]{2,4}", message="{emailValidationError}")
    private String email;

    @NotNull
    @Size(min=1, max=100, message="{sizeError}")
    private String name;

    @NotNull
    @Size(min=1, max=30, message="{sizeError}")
    private String password;

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    @Override
    public String toString() {
        return "SignupForm [email=" + email + ", name=" + name + ", password="
                + password + "]";
    }

}
</code>```

Part of controller:

```<code>@RequestMapping(value = "/signup", method = RequestMethod.POST)
    public String signup(
            @ModelAttribute("signupForm") @Valid SignupForm signupForm,
            BindingResult result, RedirectAttributes redirectAttributes) {

        if (result.hasErrors()) {
            return "signup";
        }

        userService.signup(signupForm);

        MyUtil.flash(redirectAttributes, "success", "signupSuccess");

        Logger.debug(signupForm.toString());

        return "redirect:/";
    }
</code>```

Part of service:

```<code>@Override
    public void signup(SignupForm signupForm) {

        User user = new User();
        user.setEmail(signupForm.getEmail());
        user.setName(signupForm.getName());
        user.setPassword(signupForm.getPassword());
        userRepository.save(user);
    }
</code>```

My question is - is this a real DTO? Is this considered a good practice? (I know some of you hate this term)Or should it be place somewhere else on other layer? I mean, sure, you can perform object validation on Entity directly, but there are many cases (complex forms) where you are calling and using multiple entities at once, so you want to move validation to more specific class.

答:
It is not a real DTO, according to this definition of DTO. In that definition, a DTO is a streamlined, packaged object or group of objects without real behaviour.

On the other hand, it is certainly a transfer object, by definition, and it certainly (likely) can package more than one object. Putting validation, especially inter-object validation in it makes sense to me.

The don't repeat yourself (DRY) principle applies here. Validation of the business object has to be done whether you get your data from a a database, a web front end, a desktop GUI, or inter-application communication. That means that you need validation in the business object.

That being said, there are certain kinds of validation that only make sense for web-page transfer objects, particularly those that involve translation of encodings, e.g., CP1290 to UTF-8. These are the kinds of validations that you need to do on the web form.

Title: Should Form Object be considered a DTO?


答:
It's not clear to me what you try do, it can be one of two very different things that you want to test, and you should not conflate them together:

Test that something is not awfully broken with your system - i.e. "write testcases". In this case, it's best if you generate some fake (probably relatively small) corpus of documents that you index, and  write precision/recall tests against that.

Actually measure the performance of the "search system" in the sense of measuring "how well it serves the user in real life". E.g. this is something that you are likely to do if you want to, say, build a competitor to Google (you'd want to evaluate how good is your search when compared to Google; or, if you're already working at google, you'd want to measure "how good is my new Hummingbird algorithm, when compared to Panda and Penguin?").
If this is what you measure, then this is a completely different goal, and you shouldn't measure anything synthetic, but instead you should measure actual user behaviour. There are AFAIK two widely-accepted (and used) performance measures:

2.a Measure "time-to-answer" for the typical user (how long does it take for a user of the system to 'succeed'? This is especially applicable where you have a good measure for success - e.g. if you're working for a stock photography site, success may mean "image was downloaded or added to wishlist"). Note that it's likely that the user will perform more than one query in a single session, you will actually assume that all those queries are related and that the user may have difficulties expressing his need, and may try different searches, all with the same goal.

2.b Measure Mean Average Precision (see http://en.wikipedia.org/wiki/Information_retrieval#Mean_average_precision for a quick definition, and http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/ for a nice simple explanation of what MAP really is).

In the wikipedia article you'll find other performance& correctness measures, but these two are all you really need to know :)

Title: Generating test data for a search application

Body:
I have a general question about testing search applications, and what I'm looking for is pointers to resources on the topic that I can go and research on my own. I've tried semi-informed, semi-undirected googling, but that's yielding a lot of distractions and blind alleys (or maybe my search skills aren't all that sharp).

A bit of setup first. When I say "a search application", what I mean this:

you have some data sources which you can put together in a search index

your application has an API that takes as input a search query (keywords and optionally other stuff), and its output is a relevance-ranked list of results from the search index.

there is a whole bunch of business logic on top of just retrieving the results from the index - the final result set in the output could have a large edit distance from the original result set from search.

assume that in real life, the index is large and takes a while to build

The task is to write tests for the application. The basic structure of a test is "given search request X, I expect response Y consisting of relevance-ordered results". The problem, therefore, is: what is a good strategy for generating the underlying data for the tests?

Here are some approaches I'm aware of (and have used in practice):

Don't generate test data. Start with a real index, and apply targeted modifications to it to "introduce" edge cases for your tests as needed. Upside: close to real life. Downsides: large test index; has to be rebuilt every time some change is made to the indexing scheme; most of it is unused by existing test cases.

Generate fake data such that for each request X there is a well-defined, intentionally constructed set of results Y that will be returned. Upsides: full control over the search data; only as much data as needed for the tests, quicker and easier to change. Downsides: still have to rebuild everything in the indexing scheme changes; not necessarily realistic data, which may leave aspects of the system untested or under-tested; too much flexibility and test-specific domain knowledge separate from the real-life domain of the application.

Actually that's where my current knowledge ends. Something tells me there is either a nice middle ground which allows for testing flexibility without deviating from how the application works in real life, or a completely different testing approach that does away with these concerns. What approaches might you consider?


问:
How do you calculate the CPU usage?
Suppose that in a single processor system, an average of 6 process arrive each minute. Each process needs an average service time of 8 seconds. How would you calculate the fraction of time the CPU is kept busy ?

What I have done is -:
Since a 6 new processes arrive each minute and 8 seconds is required to complete one then in total 6x8=48 seconds would be required. So the CPU would be kept busy for 48 seconds out of 60 seconds.

Am I right ?

答:
In the usual terminology, processes don't arrive. Processes roughly corresponds to the programs running on a machine. What can arrive (and cause some processing to be done) are requests and/or events. I will assume you meant those when using "process" in the question.

Assuming that the processing of a request is CPU-bound (meaning that a single request keeps the CPU fully occupied for 8 seconds, with no spare time to do anything else), then your calculation of

```<code>6 requests/minute * 8 seconds processing time = 48/60 = 80% load
</code>```

is a reasonable estimate for the average processor load.

If the processing of a single request isn't CPU-bound, then the 8 second response time is not a correct figure for calculating CPU load, because you need to subtract the time that the process is waiting for external systems, like the disk drive or database.

Title: How do you calculate the CPU usage?

Body:
In the usual terminology, processes don't arrive. Processes roughly corresponds to the programs running on a machine. What can arrive (and cause some processing to be done) are requests and/or events. I will assume you meant those when using "process" in the question.

Assuming that the processing of a request is CPU-bound (meaning that a single request keeps the CPU fully occupied for 8 seconds, with no spare time to do anything else), then your calculation of

6 requests/minute * 8 seconds processing time = 48/60 = 80% load

is a reasonable estimate for the average processor load.

If the processing of a single request isn't CPU-bound, then the 8 second response time is not a correct figure for calculating CPU load, because you need to subtract the time that the process is waiting for external systems, like the disk drive or database.


问:
Would you use C, today, for a software project?
If yes, where and why would you use it?

If no, please provide an explanation to why C is not acceptable to you.

答: I would use C if I was writing an operating system. Since that is not going to happen in the next twenty years, unless I hit lotto and have nothing else to do but make my own awesome Linux distro, I'll probably just stick to C#, Java, Python, etc, etc. I haven't used C in a very long time but I always enjoyed using it; I think though, these days my head is so wrapped around OO if I have to go back to it it'd take me a bit to get rolling again.</p>


问:
Preferred way of handling errors when loading an object from a file
If I want to load an object from a file, there are a number of things that can go wrong. Thus, one needs a way of handling errors when doing so. In some languages, like haskell, one can return a Maybe object that might contain the newly created object. In C++, although it is in principle possible to create such a convenience class, it is usually not the most straight-forward way to do things. Below I list various ways errors could be handled when loading an object from a file in C++.

Return a pointer:

```
<code>
Object* load_from_file(const char* filename);
</code>
```

By returning a pointer, we can return nullptr in case a problem occured while loading the object.

Take a reference to a default constructed object and return false if a problem occured:

```
<code>
bool load_from_file(const char* filename, Object& obj);
</code>
```

Using this method, a separate initialization method needs to be implemented for the object, which will be called after loading it from the file.

Throw an exception inside the load_from_file function:

```
<code>
Object load_from_file(const char* filename);
</code>
```

Generally, I believe, throwing an exception in such a case might be bad, as this is not really an "exceptional" case.

Load data to an intermediate struct:

```
<code>
struct ObjectData{
    //data
};
bool load_from_file(const char* filename, ObjectData& data);
//example use:
ObjectData data;
if(load_from_file("path_to_some_file", data)){
    Object(data);
    //etc...
}
</code>
```

The difference here with the second method is that you do not need to create an initialization method for Object.

Generally, what is the preferred method for the problem described above, or which method is preferred in which cases?

答:
Don't confuse "exception" with "rare"

The main point of exceptions and exception handling is keeping different
types of code apart:

The code that provides the main functionality (the "happy path" code)

Everything else: The code that handles all those 973 other cases where
not everything has worked out wonderfully and precisely to your
main functionality's liking.

Pulling the code for the not-quite-happy statements out of your
happy path code makes the latter nicely simple and understandable.
That's what exception handling is about, really (plus keeping the other
973 cases apart from one another as well; plus making sure error
handling of errors that occurred during error handling will happen
reliably as well).

So your formulation "there are a number of things that can go wrong"
is a good indication that an exception is probably the most
maintainable-in-the-long-run approach for your case.
And that remains true even if the exceptions are more frequent
than the happy path case.


答:
Generally, what is the preferred method for the problem described above, or which method is preferred in which cases?

The prefered method looks like this:

```<code>class Object { ... };

std::istream& operator>>(std::istream& in, Object& obj) { /* classic "in" operator*/ }
</code>```

Client code (1):

```<code>Object o;
if(std::cin >> o) {
    // do something with o
}
</code>```

Client code (2):

```<code>Object o;
std::cin.exceptions(std::ios::failbit);
std::cin >> o; // will throw on failure
// do something with o
</code>```

It is up to your implementation of the input operator, to set the failbit for in (which may or may not throw an exception), in case the values read from the stream are valid for the data types, but not valid for being placed into the Object instance.

This is prefered, because:

it is flexible wrt the stream type (with this code, you can just as well use a std::stringstream instance and serialize your data to a string)

it doesn't impose the error handling strategy (client code gets to decide if an exception should be thrown or not)

it minimizes knowledge cf. Law of Demeter (the loading function doesn't need to know what a file - or a file path - is)

it integrates and scales properly with the built-in C++ i/o support; that means, you can easily do this:

```<code>Object a;
int b;
std::string word;

std::cin >> a >> b >> word; // a is read the same as any int or std::string
</code>```

Downsides:
- unless you are careful on reading, errors may slip by (consider my first example, without the if as an example).

Edit: An extra advantage of writing i/o code with i/o-stream operators, is that you get free integration with other code that uses std::i/o-streams. For example, after you define your operator, you could use boost::lexical_cast with Object instances:

```<code>#include <boost/lexical_cast.hpp>

std::ostringstream buffer;
Object a = { /* data here */ };
buffer << a;
auto b = boost::lexical_cast<Object>(buffer.str());
</code>```

Title: Preferred way of handling errors when loading an object from a file

Body:
If I want to load an object from a file, there are a number of things that can go wrong. Thus, one needs a way of handling errors when doing so. In some languages, like haskell, one can return a Maybe object that might contain the newly created object. In C++, although it is in principle possible to create such a convenience class, it is usually not the most straight-forward way to do things. Below I list various ways errors could be handled when loading an object from a file in C++.

Return a pointer:

```
<code>
Object* load_from_file(const char* filename);
</code>
```

By returning a pointer, we can return nullptr in case a problem occured while loading the object.

Take a reference to a default constructed object and return false if a problem occured:

```
<code>
bool load_from_file(const char* filename, Object& obj);
</code>
```

Using this method, a separate initialization method needs to be implemented for the object, which will be called after loading it from the file.

Throw an exception inside the load_from_file function:

```
<code>
Object load_from_file(const char* filename);
</code>
```

Generally, I believe, throwing an exception in such a case might be bad, as this is not really an "exceptional" case.

Load data to an intermediate struct:

```
<code>
struct ObjectData{
    //data
};
bool load_from_file(const char* filename, ObjectData& data);
//example use:
ObjectData data;
if(load_from_file("path_to_some_file", data)){
    Object(data);
    //etc...
}
</code>
```

The difference here with the second method is that you do not need to create an initialization method for Object.

Generally, what is the preferred method for the problem described above, or which method is preferred in which cases?

Body:
Generally, what is the preferred method for the problem described above, or which method is preferred in which cases?

The prefered method looks like this:

class Object { ... };

std::istream& operator>>(std::istream& in, Object& obj) { /* classic "in" operator*/ }

Client code (1):

Object o;
if(std::cin >> o) {
    // do something with o
}

Client code (2):

Object o;
std::cin.exceptions(std::ios::failbit);
std::cin >> o; // will throw on failure
// do something with o

It is up to your implementation of the input operator, to set the failbit for in (which may or may not throw an exception), in case the values read from the stream are valid for the data types, but not valid for being placed into the Object instance.

This is prefered, because:

it is flexible wrt the stream type (with this code, you can just as well use a std::stringstream instance and serialize your data to a string)

it doesn't impose the error handling strategy (client code gets to decide if an exception should be thrown or not)

it minimizes knowledge cf. Law of Demeter (the loading function doesn't need to know what a file - or a file path - is)

it integrates and scales properly with the built-in C++ i/o support; that means, you can easily do this:

Object a;
int b;
std::string word;

std::cin >> a >> b >> word; // a is read the same as any int or std::string

Downsides:
- unless you are careful on reading, errors may slip by (consider my first example, without the if as an example).

Edit: An extra advantage of writing i/o code with i/o-stream operators, is that you get free integration with other code that uses std::i/o-streams. For example, after you define your operator, you could use boost::lexical_cast with Object instances:

#include <boost/lexical_cast.hpp>

std::ostringstream buffer;
Object a = { /* data here */ };
buffer << a;
auto b = boost::lexical_cast<Object>(buffer.str());


问:
When is inlining worth it?
Modern compilers often inline functions when they decide it is worth it.

But here comes my question: how does one define if it is optimal to inline function at given time, or more important how to decide that it should be avoided?

答:
Unless the call is on a really really hot path (executed gazillions of times),
the most relevant aspect is code size: If it shrinks, inline. If it grows, don't.
Unless your routine is really small, non-inlined code
may be more cache-friendly and thus faster for routines called in several places.

On the really hot path, the man issue will tend to be execution time:
time no longer required
for pushing arguments and return address, branching, retrieving result,
returning. The most important aspect may be the improved branching prediction.

Title: When is inlining worth it?


问:
Is generating a large random number more unique than generating each digit in that number?
I'm using a PRNG (PHP's default rand() implementation) to generate unique 16 digit numbers. Does it make sense to generate digits one by one, or the number at once?

I'm using PHP.

答:
In principle each arithmetic pseudo-random number generator (such as PHP's) has a finite period after which the same sequence of numbers will repeat.
And most generators will generate a fixed minimum number of bits (often 32) for you internally at each step, no matter how large or small the range of results is that you asked for.

So generating each digit separately will eat up the sequence of your generator more quickly and thus make the resulting sequence of complete numbers less 'random' overall.

In practice the most relevant difference is probably that generating each digit separately is a lot more work.

Both of these effects, however, point in the same direction:
You should generate your whole random number at once.


问:
Should you mock key-value stores/aggregators
I've always been taught that your tests should run in isolation, and other than a db  for ActiveRecord etc., your tests should not rely on any external software/services. e.g. I always mock my other web-service responses but what about responses from web based key-value stores and aggregators like elasticsearch, redis, and statsd? My intuition would be to mock those as well as I am not testing if they work I am testing my apps IO with them. However I see examples around where people uses namespace strategies to test their elasticsearch and statsd related code.

How much should I mock when it comes to web based services?

thanks!

答:
Mocking serves a number of purposes:

make the test run fast

make sure all services required by the test are always available

make sure defects don't make debugging your module too complicated

If none of these is a substantial problem in your case, don't mock.

Why not?
Because

creating mocks costs time

mocks induce additional effort when changing your own module's
internals (white box testing)

your mock may lie to you about what the real module actually returns


问:
Specific empty children classes
Is having empty children classes, just to specify, a bad practice?

Lets suppose I've a generic Product class. There are electrical, electronic and mechanical products. I need to represent all them and must be able to differ each one in some operations.
I've two options:

add a member named "type" in Product class to specify whether it's a electrical, electronic or mechanical product.

create children classes of Product, one for each kind of product: ElectricalProduct, ElectronicProduct and MechanicalProduct. As these classes don't differ from each other in methods or members - already defined in parent class - they would be all empty.

I consider second option a better approach, but is it the right way? I've never seen such thing on any application or library.

What do you thing? Any suggestion?

Edit:

Here is an example of doing it using second option: https://gist.github.com/tsouzar/c2ffebbcdad30920bf06

I can't see a better way of doing it, but I don't know if it is a good practice to have that empty classes.

答:
If indeed you "must be able to differ each one in some operations"
as you say, introducing three such intermediate classes is a perfectly
sensible idea.

Have a look at the idea of Abstract Class (e.g. in Java).
An abstract class cannot be instantiated itself (most OO languages
have a respective construct that will technically
forbid instantiation) but it provides variables and methods
to its children -- and each child class still needs to add some bit
to make a complete, instantiable class.

In your case, ElectricalProduct, ElectronicProduct and MechanicalProduct
would be abstract classes.

Title: Specific empty children classes


问:
What argument passing mechanism does python use, and where is this officially documented?
As far as I am aware, python is generally referred to as 'call-by-sharing', but does it implement this with call-by-value (like Java) or call-by-reference? or something else? I would appreciate if this could be answered with official python documentation (in which I can't seem to find the answer) as opposed to anything subjective.</p>

答:
Forget call-by-whathaveyou for Python

Pythons scoping and parameter passing are quite simple
if you do not try to understand them by these
semi-well-defined call-by-X terms.

A formal parameter (of a function) is a name.

What you pass as the argument to the function represents an object.

In the body of the function, that object will be bound to
that name during function execution.

Some objects are mutable (e.g. lists, class objects),
others are not (e.g. strings, integers).

That's all there is.

Binding an object to a name (or a name to an object; it's the same)
is exactly what assignment does:
a = 1 will bind the object 1 to the name a and
a = b will bind the object-that-the-name-b-is-currently-bound-to to the name a.

After def myfunc(a), the call myfunc(b) will bind the name a just like
a = b would, except that the object that b refers to is also brought
to the right scope: the body of myfunc.

See the Python language reference Section
4.1 Naming and Binding.


问:
Generating test data for a search application
I have a general question about testing search applications, and what I'm looking for is pointers to resources on the topic that I can go and research on my own. I've tried semi-informed, semi-undirected googling, but that's yielding a lot of distractions and blind alleys (or maybe my search skills aren't all that sharp).

A bit of setup first. When I say "a search application", what I mean this:

you have some data sources which you can put together in a search index

your application has an API that takes as input a search query (keywords and optionally other stuff), and its output is a relevance-ranked list of results from the search index.

there is a whole bunch of business logic on top of just retrieving the results from the index - the final result set in the output could have a large edit distance from the original result set from search.

assume that in real life, the index is large and takes a while to build

The task is to write tests for the application. The basic structure of a test is "given search request X, I expect response Y consisting of relevance-ordered results". The problem, therefore, is: what is a good strategy for generating the underlying data for the tests?

Here are some approaches I'm aware of (and have used in practice):

Don't generate test data. Start with a real index, and apply targeted modifications to it to "introduce" edge cases for your tests as needed. Upside: close to real life. Downsides: large test index; has to be rebuilt every time some change is made to the indexing scheme; most of it is unused by existing test cases.

Generate fake data such that for each request X there is a well-defined, intentionally constructed set of results Y that will be returned. Upsides: full control over the search data; only as much data as needed for the tests, quicker and easier to change. Downsides: still have to rebuild everything in the indexing scheme changes; not necessarily realistic data, which may leave aspects of the system untested or under-tested; too much flexibility and test-specific domain knowledge separate from the real-life domain of the application.

Actually that's where my current knowledge ends. Something tells me there is either a nice middle ground which allows for testing flexibility without deviating from how the application works in real life, or a completely different testing approach that does away with these concerns. What approaches might you consider?

答:
What is a good strategy for generating the underlying data for the tests?

I would use a modified version of the second approach:

Generate fake data such that for each request X there is a well-defined, intentionally constructed set of results Y that will be returned.

But instead of querying the database directly your searchengine should be implemented against datasource-specific repository-interfaces

Each repository-interface has one implementation that uses the database and one fake implementation that can read the result from a human-readable textfile. This way your testdata is less dependant to database-schema-changes.

While testing the searchengine uses the fake-repository.

For each test there is test-specific answer-file that can be maintained with a text editor.