问:
Why does automated testing keep failing in my company?
We have tried to introduce developer automated testing several times at my company. Our QA team uses Selenium to automate UI tests, but I always wanted to introduce unit tests and integration tests. In the past, each time we tried it, everyone got excited for the first month or two. Then, several months in, people simply stop doing it.

A few observations and questions:

Does automated testing actually work? Most of of my colleagues who used to work at other companies have tried and failed to implement an automated testing strategy. I still haven't seen a real-life software company that actually uses it and doesn't just talk about it. So many developers see automated testing as something that is great in theory but doesn't work in reality. Our business team would love developers to do it even at a cost of 30% extra time (at least they say so). But developers are skeptics.

No one really knows how to properly do automated testing. Yes we have all read the unit testing examples on the internet, but using them for a big project is something else altogether. The main culprit is mocking/stubbing the database or anything else that is non-trivial. You end up spending more time mocking than writing actual tests. Then when it starts taking longer to write tests than code, that's when you give up.

Are there any good examples of unit tests/system integration tests used in a complex data centric web applications? Any open source projects? Our application is data centric but also has plenty of domain logic. I tried the repository approach at some point and found it pretty good for unit testing, but it came at the price of being able to optimize data access easily and it added another layer of complexity.

We have a big project undertaken by 20 experienced developers. This would seem to be an ideal environment to introduce unit testing/integration testing.

Why doesn't it work for us? How did you make it work at your company?

答:
One thing that I haven't seen clearly addressed in the answers above is that unit testing is essentially a public good and a private cost. I've written a blog post about it here.

What it comes down to is that while a suite of tests benefits the team or an individual developer, writing the test is a cost to the one doing it, most of the time.

In short, unless writing the test is enforced somehow -- and the answers above list a number of different ways to do that -- there's no reason for an individual developer to do this.

In one company that I've worked at, writing unit tests was a required part of delivering a feature. New code was not accepted unless a unit test was part of the commit or new feature -- there were brief code reviews for every "task" that a developer was given. It may be worthwhile to implement a similar policy at your workplace.


问:
How important is it to reduce the number of lines in code?
I am a Software developer who works on J2SE (core java).

Often during our code reviews we are asked to reduce the number of lines in our code.

It's not about removing redundant code, it's about following a style that focuses on doing the same things with fewer lines in the code, while I believe in having clarity in code even if it means increasing the number of lines.

What do you think is the right way of doing things?

If LOC (lines of code) is a small number, how does it affect the code?
If LOC is a larger number, how does it affect the code?

example from the website : "javaranch"  -

```<code>public static void happyBirthday(int age)
{
    if ((age == 16) || (age == 21) || ((age > 21) && (((age % 10) == 0) || ((age % 25) == 0))))
    {
        System.out.println("Super special party, this year!");
    }
    else
    {
        System.out.println("One year older. Again.");
    }
}
</code>```

VS

```<code>public static void happyBirthday(int age)
{

    boolean sweet_sixteen = (age == 16);
    boolean majority = (age == 21);
    boolean adult = (age > 21);
    boolean decade = (age % 10) == 0;
    boolean quarter = (age % 25) == 0;

    if (sweet_sixteen || majority || (adult && (decade || quarter)))
    {
        System.out.println("Super special party, this year!");
    }
    else
    {
        System.out.println("One year older. Again.");
    }
}
</code>```

答:
IMO, it's a bad idea to measure code effectiveness by a trivial LOC count. There's a lot more to what makes properly engineered and effective code.

In my experience, effective code:

doesn't break any of the SOLID principles

is readable and self-explanatory

passes Albert Einstein's test of simplicity: "Everything should be made as simple as possible, but not simpler."

ContentLicense: CC BY-SA 3.0


答: How important is it to reduce the number of lines in code? In most cases, not at all. In fact, in most cases, it is more important to increase the number of lines of code by breaking out complex operations into more readable steps, and commenting each step as well as adding comments in general to make the code more readable and extensible by other developers who come after you.</p>

Body:
I am a Software developer who works on J2SE (core java).

Often during our code reviews we are asked to reduce the number of lines in our code.

It's not about removing redundant code, it's about following a style that focuses on doing the same things with fewer lines in the code, while I believe in having clarity in code even if it means increasing the number of lines.

What do you think is the right way of doing things?

If LOC (lines of code) is a small number, how does it affect the code?
If LOC is a larger number, how does it affect the code?

example from the website : "javaranch"  -

```<code>public static void happyBirthday(int age)
{
    if ((age == 16) || (age == 21) || ((age > 21) && (((age % 10) == 0) || ((age % 25) == 0))))
    {
        System.out.println("Super special party, this year!");
    }
    else
    {
        System.out.println("One year older. Again.");
    }
}
</code>```

VS

```<code>public static void happyBirthday(int age)
{

    boolean sweet_sixteen = (age == 16);
    boolean majority = (age == 21);
    boolean adult = (age > 21);
    boolean decade = (age % 10) == 0;
    boolean quarter = (age % 25) == 0;

    if (sweet_sixteen || majority || (adult && (decade || quarter)))
    {
        System.out.println("Super special party, this year!");
    }
    else
    {
        System.out.println("One year older. Again.");
    }
}
</code>```


问:
What is the fitness landscape for minimal + viable solutions?
Let's say I'm trying to find a number from 1 .. 100. All numbers in this range are "valid", in that they could be interpreted as potential solutions.
Let's say the ideal number is 50. And all numbers >= 50 are "feasible" in that they actually solve the problem. And all numbers < 50 are "not feasible" (but still valid).
How would you code a scenario like this with a fitness function (assuming that the landscape is similar to but more complex than this contrived example)? Do you give "bonuses" to valid solutions? Do you measure how far an unfeasible solution has left to go before becoming optimal? And do you penalize excessive solutions?

```<code>if (solution < 50) {
  maximalFitness = f(solution)
} else if (solution >= 50) {
  maximalFitness = 1_000 - f(solution)
}
</code>```

The curve wouldn't be continuous if all feasible solutions are strictly better than all unfeasible solutions, despite having a similar distance from optimal.

答:
What you're referring to is called "constrained optimization". This is a really well-studied branch of evolutionary computation, and you can find dozens of resources describing common techniques for solving these problems. Carlos Coello-Coello does a tutorial on the subject each year at GECCO, the primary EC conference in the field. I found slides from one such session available here (ftp://ftp.cs.bham.ac.uk/.snapshot/nightly.1/pub/authors/W.B.Langdon/biblio/gecco2008/docs/p2445.pdf). You might find that a useful jumping-off point for further study.

In short, there are two high-level approaches that are by far the most common: penalization and repair. In penalization, you decrease the fitness of solutions by some function of how badly they violate the constraints. In repair, you actually modify infeasible solutions directly to attempt to move them into the feasible region.

Penalization works well if you can get the penalty scheme right, but that can be a tricky problem to solve, and it's essentially always problem dependent. One option is to just mess around with your fitness function and figure out the right amount to penalize so that "good" infeasible solutions get just enough fitness to not die immediately but not so much that they become preferred over feasible solutions. Another method is to just replace the selection criteria to account for infeasible solutions directly. That is, instead of

```<code>if fitness(p1) > fitness(p2):
    # p1 is better
else:
    # p2 is better
</code>```

you have something like

```<code>if (is_feasible(p1) and is_feasible(p2)) or (not is_feasible(p1) and not is_feasible(p2)):
    if fitness(p1) > fitness(p2):
        # p1 is better
    else:
        # p2 is better
else:
    if is_feasible(p1):
        # p1 is better
    else:
        # p2 is better
</code>```

This explicitly handles all the ways of comparing feasible and infeasible solutions, and uses their fitness values only to break ties. Now it doesn't matter so much if the raw fitness value of an infeasible solution is higher than a feasible solution due to weirdness in your penalization scheme.

Repair is often preferable if you can do it (it isn't always possible to build an effective repair operator). By repair, we mean you take the infeasible solution and actually modify it so that it falls within your feasible region. This could be as simple as just thresholding the value within the allowed range, but more often will require some sort of domain specific or heuristic search. For example, if you have a knapsack problem and a solution has too much weight in the knapsack, a repair operator might randomly start throwing items out of the knapsack until the total weight was less than the capacity. A better operator might bias the removal of items toward those with the highest weight/value ratio. It's usually beneficial for the repair to have some randomness involved -- you may want to bias things slightly, but you don't want to always do a greedy repair.

There are other approaches as well. One recent innovation has been to treat each constraint as a separate objective function and employ multi-objective optimization algorithms to find a range of trade-off solutions. For some problems, this has been shown to be effective, but I think the general rule of thumb should probably be to first see what can be done with the simpler methods like penalization and repair.

Body:
What you're referring to is called "constrained optimization". This is a really well-studied branch of evolutionary computation, and you can find dozens of resources describing common techniques for solving these problems. Carlos Coello-Coello does a tutorial on the subject each year at GECCO, the primary EC conference in the field. I found slides from one such session available here (ftp://ftp.cs.bham.ac.uk/.snapshot/nightly.1/pub/authors/W.B.Langdon/biblio/gecco2008/docs/p2445.pdf). You might find that a useful jumping-off point for further study.

In short, there are two high-level approaches that are by far the most common: penalization and repair. In penalization, you decrease the fitness of solutions by some function of how badly they violate the constraints. In repair, you actually modify infeasible solutions directly to attempt to move them into the feasible region.

Penalization works well if you can get the penalty scheme right, but that can be a tricky problem to solve, and it's essentially always problem dependent. One option is to just mess around with your fitness function and figure out the right amount to penalize so that "good" infeasible solutions get just enough fitness to not die immediately but not so much that they become preferred over feasible solutions. Another method is to just replace the selection criteria to account for infeasible solutions directly. That is, instead of

if fitness(p1) > fitness(p2):
    # p1 is better
else:
    # p2 is better

you have something like

if (is_feasible(p1) and is_feasible(p2)) or (not is_feasible(p1) and not is_feasible(p2)):
    if fitness(p1) > fitness(p2):
        # p1 is better
    else:
        # p2 is better
else:
    if is_feasible(p1):
        # p1 is better
    else:
        # p2 is better

This explicitly handles all the ways of comparing feasible and infeasible solutions, and uses their fitness values only to break ties. Now it doesn't matter so much if the raw fitness value of an infeasible solution is higher than a feasible solution due to weirdness in your penalization scheme.

Repair is often preferable if you can do it (it isn't always possible to build an effective repair operator). By repair, we mean you take the infeasible solution and actually modify it so that it falls within your feasible region. This could be as simple as just thresholding the value within the allowed range, but more often will require some sort of domain specific or heuristic search. For example, if you have a knapsack problem and a solution has too much weight in the knapsack, a repair operator might randomly start throwing items out of the knapsack until the total weight was less than the capacity. A better operator might bias the removal of items toward those with the highest weight/value ratio. It's usually beneficial for the repair to have some randomness involved -- you may want to bias things slightly, but you don't want to always do a greedy repair.

There are other approaches as well. One recent innovation has been to treat each constraint as a separate objective function and employ multi-objective optimization algorithms to find a range of trade-off solutions. For some problems, this has been shown to be effective, but I think the general rule of thumb should probably be to first see what can be done with the simpler methods like penalization and repair.


问:
How can I break my vendor lock-in situation?
I'm developing software for about 10 years. I began on QBasic and now I'm developing .NET using C# language. I'm always trying to follow the best design patterns and principals like SOLID. In short I'm very happy with my work while as a tiny ISV I'm free to choose between technologies. Of course we almost develop mid-size web applications for start-ups.

Although I'm moving to MS open stack more and more (Ex. MVC, EF, etc.) and I've also tried Mono, I feel tied to MS. I seriously want to break this vendor lock-in situation.

I want to be able to develop Windows and Web application using a strongly-type open and free (at least more free) language using the best software practices which is more vendor independent. I think I have no choice except Java. Then;

What is the best way to start breaking this situation?

Should I migrate to the Java platform? (I know it cost me a lot)

Then how can I start?

C# is an amazing language. Some of its features do not exist in Java (LINQ, real generics, async/await, etc.)! How can I get to that?

答:
Ok, this is partly not so serious. Please find the serious part below.

What is the best way to start breaking this situation?

Not breaking it at all because you don't have an obvious reason. You just feel ideologically obliged to do it. Well, that's your thing and I can't stop you from it. Break it by choosing any other platform of choice.

Should I migrate to the Java platform? (I know it cost me a lot)

Ok, I might get a lot of downvotes for that, and it's surely subjective opinion, but: Java Web Development is a major pain in the ass. Even James Gosling (creator of Java) said, and I cite: "JSF sucks". ASP.NET is much friendlier in my opinion. If you absolutely must use Java, don't use the major frameworks. Use Apache Wicket or something. But for you more important is: Java is property of Oracle now. It's as free as the .NET platform. Sure, there's OpenJDK. But for .NET, there's also Mono.

You should rather ask: Should I switch to develop for the JVM because I want to learn new things, in which case I'd say:
Sure, because it offers a wider variety of languages that run on it, for example the elegant Scala.

Then how can I start?

Grab platform of choice that is to your liking and just start hacking (some people even use LISP for web development, some even use C, why not take Ruby or Python or PHP?)

C# is an amazing language. Some of its features do not exist in Java (LINQ, real generics, async/await, etc.)! How can I get to that?

Java will get Lambdas and Closures with the next version. However, if you want to write programs for the JVM, I'd advice you to use Scala and skip Java. Scala is an amazing language!

Serious part

If you want to make money with your startup:

Whether you should change platforms or not depends on your customers. Take the platform that you can sell best in your environment. Probably something based on a standard Apache installation or, for corporate environments, something based on Java Application Servers;

If you want to make money by offering your skills in case the startup fails:

Java is never wrong, neither is .NET. Java will be beneficial to know as the Android market grows. Objective C can be sold very well these days, but I think it will decline again (personal opinion).

If you're doing it for because you love technology:

Take something that is closer to the ivory tower, like Scala or Ruby.


问:
When should pointers be checked for NULL in C?
Summary:

Should a function in C always check to make sure it is not dereferencing a NULL pointer?  If not when is it appropriate to skip these checks?

Details:

I've been reading some books about programming interviews and I'm wondering what the appropriate degree of input validation for function arguments in C is?  Obviously any function that takes input from a user needs to perform validation, including checking for a NULL pointer before dereferencing it.  But what about in the case of a function within the same file that you don't expect to expose through your API?

For example the following appears in the source code of git:

```<code>static unsigned short graph_get_current_column_color(const struct git_graph *graph)
{
    if (!want_color(graph->revs->diffopt.use_color))
        return column_colors_max;
    return graph->default_column_color;
}
</code>```

If *graph is NULL then a null pointer will be dereferenced, probably crashing the program, but possibly resulting in some other unpredictable behavior.  On the other hand the function is static and so maybe the programmer already validated the input.  I don't know, I just selected it at random because it was a short example in an application program written in C.  I've seen many other places where pointers are used without checking for NULL. My question is general not specific to this code segment.

I saw a similar question asked within the context of exception handing.  However, for a unsafe language such as C or C++ there is no automatic error propagation of unhandled exceptions.

On the other hand I have seen lots of code in open source projects (such as the example above) that does not do any checks of pointers before using them. I'm wondering if anyone has thoughts on guidelines for when to put checks in a function vs. assuming that the function was called with correct arguments.

I'm interested in this question in general for writing production code.  But I'm also interested within the context of programming interviews.  For instance many algorithm textbooks (such as CLR) tend to present the algorithms in pseudocode without any error checking.  However, while this is good for understanding the core of an algorithm it's obviously not a good programming practice.  So I would not want to tell an interviewer that I was skipping error checking to simplify my code examples (as a textbook might). But I also would not want to appear to produce inefficient code with excessive error checking.  For instance the graph_get_current_column_color could have been modified to check *graph for null but its not clear what it would do if *graph was null, other than it should not dereference it.

答:
You can skip the check when you can convince yourself somehow that the pointer cannot possibly be null.

Usually, null pointer checks are implemented in code in which null is expected to appear as an indicator that an object is currently not available. Null is used as a sentinel value, for instance to terminate linked lists, or even arrays of pointers. The argv vector of strings passed into main is required to be null-terminated by a pointer, similarly to how a string is terminated by a null character: argv[argc] is a null pointer, and you can rely on this when parsing the command line.

```<code>while (*argv) {
   /* process argument string *argv */
   argv++; /* increment to next one */
}
</code>```

So, the situations for checking for null are those in which a it is an expected value. The null checks implement the meaning of the null pointer, such as stopping the search of a linked list. They prevent the code from dereferencing the pointer.

In a situation in which a null pointer value is not expected by design, there is no point in checking for it. If an invalid pointer value arises, it will quite likely appear non-null,  which cannot be distinguished from valid values in any portable way. For instance, a pointer value obtained from reading uninitialized storage interpreted as a pointer type, a pointer obtained via some shady conversion, or a pointer incremented out of bounds.

About a data type such as graph *: this could be designed so that a null value is a valid graph: something with no edges and no nodes. In this case, all the functions that take a graph * pointer will have to deal with that value, since it is a correct domain value in the representation of graphs. On the other hand,  a graph * could be a pointer to a container-like object which is never null if we hold a graph; a null pointer then might tell us that "the graph object is not present; we did not allocate it yet, or we freed it; or this currently has no associated graph". This latter use of pointers is a combined boolean/satellite: the pointer being non-null indicates "I have this sister object", and it provides that object.

We might set a pointer to null even if we are not freeing a object, simply to dissociate one object from another:

```<code>tty_driver->tty = NULL; /* detach low level driver from the tty device */
</code>```

Body:
You can skip the check when you can convince yourself somehow that the pointer cannot possibly be null.

Usually, null pointer checks are implemented in code in which null is expected to appear as an indicator that an object is currently not available. Null is used as a sentinel value, for instance to terminate linked lists, or even arrays of pointers. The argv vector of strings passed into main is required to be null-terminated by a pointer, similarly to how a string is terminated by a null character: argv[argc] is a null pointer, and you can rely on this when parsing the command line.

while (*argv) {
   /* process argument string *argv */
   argv++; /* increment to next one */
}

So, the situations for checking for null are those in which a it is an expected value. The null checks implement the meaning of the null pointer, such as stopping the search of a linked list. They prevent the code from dereferencing the pointer.

In a situation in which a null pointer value is not expected by design, there is no point in checking for it. If an invalid pointer value arises, it will quite likely appear non-null,  which cannot be distinguished from valid values in any portable way. For instance, a pointer value obtained from reading uninitialized storage interpreted as a pointer type, a pointer obtained via some shady conversion, or a pointer incremented out of bounds.

About a data type such as graph *: this could be designed so that a null value is a valid graph: something with no edges and no nodes. In this case, all the functions that take a graph * pointer will have to deal with that value, since it is a correct domain value in the representation of graphs. On the other hand,  a graph * could be a pointer to a container-like object which is never null if we hold a graph; a null pointer then might tell us that "the graph object is not present; we did not allocate it yet, or we freed it; or this currently has no associated graph". This latter use of pointers is a combined boolean/satellite: the pointer being non-null indicates "I have this sister object", and it provides that object.

We might set a pointer to null even if we are not freeing a object, simply to dissociate one object from another:

tty_driver->tty = NULL; /* detach low level driver from the tty device */


问:
Who first coined the term Higher Order Function and/or First Class Citizen?
I've come to understand that long before Haskell, O'Caml or LISP, higher order functions were an academic research subject and in mathematics, Schönfinkel (in 1967) and Haskell Curry (in 1968) already applied techniques such as currying, but that was before it was available in any programming language.

Scheme, according to Wikipedia, was the first language to introduce proper higher-order functions as first-class citizens, but is there anybody we can attribute the original idea to? Maybe Alonzo Church, who invented lambda calculus in the 1930's? More specifically, who coined the following definition, which I saw in various paraphrases around in several books and online resources?

A function is considered to be of higher order when it takes another
  function as an argument or when it returns a function.  Any function
  not taking functions as arguments or as return types is called a first
  order function.

答:
The wikipedia article already attributes the "first-class citizens" terminology to Strachey.

"Higher order function" I would imagine dates back to the distinction between higher order and first order logic.

See https://en.wikipedia.org/wiki/Higher-order_logic and http://plato.stanford.edu/entries/logic-higher-order/


问:
How should I go about "overhauling" a large legacy application?

Possible Duplicate:

  I’ve inherited 200K lines of spaghetti code — what now?

For my next project, I've been tasked with "overhauling" a large legacy web application with many parts.  It is a JSP application written in 2004 and it is used heavily by my company.

This application was designed badly in that there is no separation of concerns; no service layer, no DAO layer, no MVC structure.  Just a bunch of JSP files loaded with scriptlets containing logic and database queries mixed in with HTML.  It's a mess.

My boss has defined "overhaul" as basically rewriting the entire thing using the technologies we use for our more recent applications, which are:

Maven

Spring

JPA (w/Hibernate)

JSF

AND add a bunch of new features.

My question is: How do I go about this?  What should be my strategy for completely redoing a large application that is currently in use AND add a bunch of features?

Should I rewrite the existing application first and get it working with the updated technologies and THEN worry about adding new features afterward?  Or should I go about it like I'm creating a whole new application and implement the new features WITH the old features?

Has anyone done this successfully?  If so, what strategies were instrumental to your success?

Edit: More info from comments:

How bad does the design have to be to justify a rewrite? This design is pretty bad. There is tons of business logic hiding in the application. No specs, no logs of requirements changes, no list of bug fixes, no list of open bugs, no test suite, no documentation. And the guy who originally wrote it is now in upper management.

答:
Has anyone done this successfully?

Here is the success story of our company. For a pretty long time we had a messy website:

PHP/MySQL, mostly Drupal 6.

Many custom Drupal modules of medium quality.

Some really old legacy code parts written in plain PHP+SQL+HTML+JS+CSS (there were files actually containing all of those at once). Those were actually very important parts which came from older version of the application.

Two internal MySQL databases (one for Drupal and one for older stuff), two external SQL Server databases which our app was integrated with (all with just direct queries).

No spec, no tests, even no repository (we made it later).

Cannot tell the size of codebase now, but there were about 12 custom forms and about 10 custom Drupal modules. Many features were implemented just with modules from official Drupal repository, so it shouldn't be counted as a part of code we wrote/maintained, but rewrite was done with Kohana framework, so pretty mush everything was done from scratch.

The good news were:

Before we redone it, I had over 1 year of maintenance/extension experience of this app, two of my colleagues had about twice less time of such experience. One last guy had no experience.

Through the maintance period we gradually improved some parts and found most of pitfalls and undocumented features.

Before rewrite we managed to make a specification of some sort.

In the last few months we added some new parts which we were going to keep in the new version. Some of this code actually was transplanted.

How it happened:

It was done mostly by two guys, one of them was the guy with no maintenance experience. I helped with spec and just a little bit with code. One guy maintianed the old version through the process.

It took slightly more than half a year, but partially because managers stuck with getting a new  interface concept and graphical design. Just coding could be done in 4-5 months, I think.

Pretty much all old features are now implemented, some of them even slightly improved.

Some rudimentary unit testing was done (but not TDD/BDD), I've not looked into it yet.

After all, new version is online now and we have no major problems.

It was not a 100% rewrite because:

We took some (but little) code to the new version

Old version was actually undergoing the process of gradual improvement, so it was not a huge "paradigm shift" for us as developers

Without that much of maintenance it probably could not be done. I tend to agree with other answers in that full rewrite is generally a bad idea, but as our story shows, it is possible given those conditions:

app is not very big

team is not very big (4 in our case)

most of team members have extensive knowledge and experience dealing with legacy code

For your specific situation I can suggest you:

Try to improve what you have. This will at least give you the deep knowledge of your app.

If improvement will work, consider just continuing on improvement even without switching to your current technology stack. A bird in the hand is worth two in the bush, you know.

If improvement will fail, rewrite will be easier and faster because of the clause 1 in the list.


答:
Frege says in "Funktion und Begriff" (1891):

Wie nun Funktionen von Gegenständen grundverschieden sind, so sind
  auch Funktionen, deren Argumente Funktionen sind und sein müssen,
  grundverschieden von Funktionen, deren Argumente Gegenstände sind und
  nichts anderes sein können. Diese nenne ich Funktionen erster, jene
  Funktionen zweiter Stufe.

In english (my translation):

Like things and functions are different, so are functions, whose arguments are functions radically different from functions whose arguments must be things. I call the latter functions of first order, the former functions of second order.

Title: Who first coined the term Higher Order Function and/or First Class Citizen?

Body:
I've come to understand that long before Haskell, O'Caml or LISP, higher order functions were an academic research subject and in mathematics, Schönfinkel (in 1967) and Haskell Curry (in 1968) already applied techniques such as currying, but that was before it was available in any programming language.

Scheme, according to Wikipedia, was the first language to introduce proper higher-order functions as first-class citizens, but is there anybody we can attribute the original idea to? Maybe Alonzo Church, who invented lambda calculus in the 1930's? More specifically, who coined the following definition, which I saw in various paraphrases around in several books and online resources?

A function is considered to be of higher order when it takes another
  function as an argument or when it returns a function.  Any function
  not taking functions as arguments or as return types is called a first
  order function.


答:
I generally only check when a pointer is assigned, which is generally the only time I can actually do something about it and possibly recover if it is invalid.

If I get a handle to a window for example, I'll check for it being null right and then and there, and do something about the null condition, but I'm not going to check for it being null each and every single time I use the pointer, in each and every function the pointer is passed to, otherwise I'd have mountains of duplicate error handling code.

Functions like graph_get_current_column_color is probably completely unable to do anything useful to your situation if it does encounter a bad pointer, so I'd leave checking for NULL to its callers.

Title: When should pointers be checked for NULL in C?

Body:
Summary:

Should a function in C always check to make sure it is not dereferencing a NULL pointer?  If not when is it appropriate to skip these checks?

Details:

I've been reading some books about programming interviews and I'm wondering what the appropriate degree of input validation for function arguments in C is?  Obviously any function that takes input from a user needs to perform validation, including checking for a NULL pointer before dereferencing it.  But what about in the case of a function within the same file that you don't expect to expose through your API?

For example the following appears in the source code of git:

```<code>static unsigned short graph_get_current_column_color(const struct git_graph *graph)
{
    if (!want_color(graph->revs->diffopt.use_color))
        return column_colors_max;
    return graph->default_column_color;
}
</code>```

If *graph is NULL then a null pointer will be dereferenced, probably crashing the program, but possibly resulting in some other unpredictable behavior.  On the other hand the function is static and so maybe the programmer already validated the input.  I don't know, I just selected it at random because it was a short example in an application program written in C.  I've seen many other places where pointers are used without checking for NULL. My question is general not specific to this code segment.

I saw a similar question asked within the context of exception handing.  However, for a unsafe language such as C or C++ there is no automatic error propagation of unhandled exceptions.

On the other hand I have seen lots of code in open source projects (such as the example above) that does not do any checks of pointers before using them. I'm wondering if anyone has thoughts on guidelines for when to put checks in a function vs. assuming that the function was called with correct arguments.

I'm interested in this question in general for writing production code.  But I'm also interested within the context of programming interviews.  For instance many algorithm textbooks (such as CLR) tend to present the algorithms in pseudocode without any error checking.  However, while this is good for understanding the core of an algorithm it's obviously not a good programming practice.  So I would not want to tell an interviewer that I was skipping error checking to simplify my code examples (as a textbook might). But I also would not want to appear to produce inefficient code with excessive error checking.  For instance the graph_get_current_column_color could have been modified to check *graph for null but its not clear what it would do if *graph was null, other than it should not dereference it.


答: One practice is to always perform the null check unless you have already checked it; so if input is being passed from function A() to B(), and A() has already validated the pointer and you are certain B() isn't called anywhere else, then B() can trust A() to have sanitized the data.


问:
Multithreading synchronization interview question: Find n words given m threads

Is there a way this problem could benefit from a solution with multiple threads, rather than a single thread?

In an interview, I was asked to solve a problem using multiple threads. It appears to me that the multiple threads lends no benefit.

Here's the problem:

You are given a paragraph , which contain n number of words, you are
  given m threads. What you need to do is , each thread should print one
  word and give the control to next thread, this way each thread will
  keep on printing one word , in case last thread come, it should invoke
  the first thread. Printing will repeat until all the words are printed
  in paragraph. Finally all threads should exit gracefully. What kind of
  synchronization will use?

I strongly feel we cannot take any advantage of threads here, but believe the interviewer is trying to measure my synchronization skills. Am I missing something in this problem that would make multiple threads have value?

No need of code, just put some thoughts. I will implement by myself.

答:
Interesting question. Here is my solution in Java using SynchronousQueue to create a rendezvous channel between threads:</p>

```<code>import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import java.util.Stack;
import java.util.concurrent.SynchronousQueue;

public class FindNWordsGivenMThreads {

    private static final int NUMBER_OF_WORDS = 100;
    private static final int NUMBER_OF_THREADS = 5;
    private static final Stack<String> POISON_PILL = new Stack<String>();

    public static void main(String[] args) throws Exception {
        new FindNWordsGivenMThreads().run();
    }

    private void run() throws Exception {
        final Stack<String> words = loadWords();
        SynchronousQueue<Stack<String>> init = new SynchronousQueue<Stack<String>>();
        createProcessors(init);
        init.put(words);
    }

    private void createProcessors(SynchronousQueue<Stack<String>> init) {
        List<Processor> processors = new ArrayList<Processor>();

        for (int i = 0; i < NUMBER_OF_THREADS; i++) {

            SynchronousQueue in;
            SynchronousQueue out;

            if (i == 0) {
                in = init;
            } else {
                in = processors.get(i - 1).getOut();
            }

            if (i == (NUMBER_OF_THREADS - 1)) {
                out = init;
            } else {
                out = new SynchronousQueue();
            }

            Processor processor = new Processor("Thread-" + i, in, out);
            processors.add(processor);
            processor.start();

        }

    }

    class Processor extends Thread {

        private SynchronousQueue<Stack<String>> in;
        private SynchronousQueue<Stack<String>> out;

        Processor(String name, SynchronousQueue in, SynchronousQueue out) {
            super(name);
            this.in = in;
            this.out = out;
        }

        @Override
        public void run() {

            while (true) {

                Stack<String> stack = null;
                try {
                    stack = in.take();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }

                if (stack.empty() || stack == POISON_PILL) {
                    System.out.println(Thread.currentThread().getName() + " Done!");
                    out.offer(POISON_PILL);
                    break;
                }

                System.out.println(Thread.currentThread().getName() + " " + stack.pop());
                out.offer(stack);
            }
        }

        public SynchronousQueue getOut() {
            return out;
        }
    }

    private Stack<String> loadWords() throws Exception {

        Stack<String> words = new Stack<String>();

        BufferedReader reader = new BufferedReader(new FileReader(new File("/usr/share/dict/words")));
        String line;
        while ((line = reader.readLine()) != null) {
            words.push(line);
            if (words.size() == NUMBER_OF_WORDS) {
                break;
            }
        }
        return words;
    }
}
</code>```

Body:
Interesting question. Here is my solution in Java using SynchronousQueue to create a rendezvous channel between threads:

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import java.util.Stack;
import java.util.concurrent.SynchronousQueue;

public class FindNWordsGivenMThreads {

    private static final int NUMBER_OF_WORDS = 100;
    private static final int NUMBER_OF_THREADS = 5;
    private static final Stack<String> POISON_PILL = new Stack<String>();

    public static void main(String[] args) throws Exception {
        new FindNWordsGivenMThreads().run();
    }

    private void run() throws Exception {
        final Stack<String> words = loadWords();
        SynchronousQueue<Stack<String>> init = new SynchronousQueue<Stack<String>>();
        createProcessors(init);
        init.put(words);
    }

    private void createProcessors(SynchronousQueue<Stack<String>> init) {
        List<Processor> processors = new ArrayList<Processor>();

        for (int i = 0; i < NUMBER_OF_THREADS; i++) {

            SynchronousQueue in;
            SynchronousQueue out;

            if (i == 0) {
                in = init;
            } else {
                in = processors.get(i - 1).getOut();
            }

            if (i == (NUMBER_OF_THREADS - 1)) {
                out = init;
            } else {
                out = new SynchronousQueue();
            }

            Processor processor = new Processor("Thread-" + i, in, out);
            processors.add(processor);
            processor.start();

        }

    }

    class Processor extends Thread {

        private SynchronousQueue<Stack<String>> in;
        private SynchronousQueue<Stack<String>> out;

        Processor(String name, SynchronousQueue in, SynchronousQueue out) {
            super(name);
            this.in = in;
            this.out = out;
        }

        @Override
        public void run() {

            while (true) {

                Stack<String> stack = null;
                try {
                    stack = in.take();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }

                if (stack.empty() || stack == POISON_PILL) {
                    System.out.println(Thread.currentThread().getName() + " Done!");
                    out.offer(POISON_PILL);
                    break;
                }

                System.out.println(Thread.currentThread().getName() + " " + stack.pop());
                out.offer(stack);
            }
        }

        public SynchronousQueue getOut() {
            return out;
        }
    }

    private Stack<String> loadWords() throws Exception {

        Stack<String> words = new Stack<String>();

        BufferedReader reader = new BufferedReader(new FileReader(new File("/usr/share/dict/words")));
        String line;
        while ((line = reader.readLine()) != null) {
            words.push(line);
            if (words.size() == NUMBER_OF_WORDS) {
                break;
            }
        }
        return words;
    }
}


答:
Invalid null pointers can either be caused by programmer error or by runtime error.  Runtime errors are something a programmer can't fix, like a malloc failing due to low memory or the network dropping a packet or the user entering something stupid.  Programmer errors are caused by a programmer using the function incorrectly.

The general rule of thumb I've seen is that runtime errors should always be checked, but programmer errors don't have to be checked every time.  Let's say some idiot programmer directly called graph_get_current_column_color(0).  It will segfault the first time it's called, but once you fix it, the fix is compiled in permanently.  No need to check every single time it's run.

Sometimes, especially in third party libraries, you'll see an assert to check for the programmer errors instead of an if statement.  That allows you to compile in the checks during development, and leave them out in production code.  I've also occasionally seen gratuitous checks where the source of the potential programmer error is far removed from the symptom.

Obviously, you can always find someone more pedantic, but most C programmers I know favor less cluttered code over code that is marginally safer.  And "safer" is a subjective term.  A blatant segfault during development is preferable to a subtle corruption error in the field.


问:
How can I deal with a team member who dislikes making comments in code?
One of my team members consistently avoids making comments in his code.

His code is not self-documenting, and other programmers have a difficult time understanding his code.

I have asked him several times to comment his code, however he just gives excuses or claims that he will do it later. His concern is that adding comments will take too much time and delay the projects.

What argument can I present to him to convince him to properly document his code?

On that note, am I wrong to focus on the code comments or is this indicative of a larger problem which should be addressed?

答:
First of all, I would suggest you re-address your approach about the comments.

If they are documentation comments at the API level (exposed later to public), then this developer simply isn't doing his job. But for all other comments, be careful.

In most of the cases I encounter them, comments are evil. I would recommend reading the code comments chapter of "Clean code" by Robert Martin to get a good understanding why.

Comments hurt your code in several ways:

1) They are hard to maintain. You'll have to put in extra work when refactoring; if you change the logic described in the comment, you need to edit the comment too.

2) They often lie. You cannot trust comments and must read the code instead. Which raises the question: why would you need the comments at all?

```<code>// this method returns the sum of 'a' and 'b'
public int GetHash(int a, int b)
{
    //the calculation of the hash
    int result = a*b;
}
</code>```

(The hash is not the sum but the product.)

3) Comments clutter the code, and very rarely add any value.

My solution: Instead of adding more comments, try to get rid of them at all!

Body:
First of all, I would suggest you re-address your approach about the comments.

If they are documentation comments at the API level (exposed later to public), then this developer simply isn't doing his job. But for all other comments, be careful.

In most of the cases I encounter them, comments are evil. I would recommend reading the code comments chapter of "Clean code" by Robert Martin to get a good understanding why.

Comments hurt your code in several ways:

1) They are hard to maintain. You'll have to put in extra work when refactoring; if you change the logic described in the comment, you need to edit the comment too.

2) They often lie. You cannot trust comments and must read the code instead. Which raises the question: why would you need the comments at all?

// this method returns the sum of 'a' and 'b'
public int GetHash(int a, int b)
{
    //the calculation of the hash
    int result = a*b;
}

(The hash is not the sum but the product.)

3) Comments clutter the code, and very rarely add any value.

My solution: Instead of adding more comments, try to get rid of them at all!


问:
Effective ways to make your OSS project popular?
After writing a few (relatively obscure) OSS frameworks, I've learned the hard way that writing a good framework isn't enough--there has to be some time spent marketing your framework as much as the time you spend coding it. So here's my question: What are some effective ways for making your OSS project become well-known? Aside from blogging, how do I gain the most market attention for my code with the least amount of effort as humanly possible?

(EDIT: I'm a lazy programmer and I hate documentation, but I'm biting the bullet. I want to be famous)

答:
build something interesting

show the world what's interesting about it. Make screenshots, write tutorials and documentation (more than only the API reference)

be friendly and open

ContentLicense: CC BY-SA 2.5


问:
Good practices to implement mappers in a multi-tier application
When you are working with a multi-tier application very often you run into task of converting objects in one layer to objects in another layer. This could be converting database objects into domain objects, service objects into domain objects, domain object into view models, etc.

I've seen it being done in multiple ways, those include:

In-line conversion

Creating a translator class with static helper methods

Using extension methods to simplify conversion

Overriding cast operators

Creating sepearate mapper classes for every possible translation

Utilizing solutions such as automapper

What would be the "proper" way to handle it. The application I currently work with utilizes separate mapper class for every possible translation, and although it has benefits of testability we are using dependency injection and we have to pass all those mappers as dependencies in the constructor.

Has anybody figured out a better way to do it?

UPDATE

I am using .NET stack here (EF +Business Layer + MVC)

答: I'm not sure about the best way to implement object-object mapping, but you may prefer to rely on a library specially created to solve that. My project is also based in the .NET stack and Automapper (http://automapper.org) works great. It's available via NuGet.</p>


答:
Kernighan & Plauger, in "Software Tools", wrote that they would check everything, and, for conditions that they believed could in fact never happen, they would abort with an error message "Can't happen".

They report being rapidly humbled by the number of times they saw "Can't happen" come out on their terminals.

You should ALWAYS check the pointer for NULL before you (attempt to) dereference it.  ALWAYS.  The amount of code you duplicate checking for NULLs that don't happen, and the processor cycles you "waste", will be more than paid for by the number of crashes you don't have to debug from nothing more than a crash dump - if you're that lucky.

If the pointer is invariant inside a loop, it suffices to check it outside the loop, but you should then "copy" it into a scope-limited local variable, for use by the loop, that adds the appropriate const decorations.  In this case, you MUST ensure that every function called from the loop body includes the necessary const decorations on the prototypes, ALL THE WAY DOWN.  If you don't, or can't (because of e.g. a vendor package or an obstinate coworker), then you must check it for NULL EVERY TIME IT COULD BE MODIFIED, because sure as COL Murphy was an incurable optimist, someone IS going to zap it when you aren't looking.

If you are inside a function, and the pointer is supposed to be non-NULL coming in, you should verify it.

If you are receiving it from a function, and it is supposed to be non-NULL coming out, you should verify it.  malloc() is particularly notorious for this.  (Nortel Networks, now defunct, had a hard-and-fast written coding standard about this.  I got to debug a crash at one point, that I traced back to malloc() returning a NULL pointer and the idiot coder not bothering to check it before he wrote to it, because he just KNEW he had plenty of memory...  I said some very nasty things when I finally found it.)


问:
How is Sencha Touch performing on Android in practice?
I'm just about to start a project using Sencha Touch, and just done some minor testing on my HTC desire device. All tutorial videos at Vimeo seems to be using an iPhone emulator running on a Mac. Im not sure how fast this emulator is compared to a real iPhone device or even an real Android device, but from what i have experienced, it seems that my HTC desire is not performing that nicly as this emulator.

All animations (sliding, fading, etc) seems abit laggy. You can easily notice that the FPS is much less than on the Vimeo videos.

HTC desire is a relativly new and modern Android 2.2 phone, running with decent hardware, so im wondering if Sencha Touch is "ready" for the Android platform.

Anybody with practical experience with Android and Sencha Touch ?

答:
We are doing a web app for iOS/Android using the Sencha 2 framework and the performance is just horrible on Android compared to IOS.
Unfortunately, I am now telling my clients that if they want an app on Android, either go native or Titanium. Hopefully they can make improvements, but I will never use Sencha again if the target platform is Android.

FYI ( target device was Galaxy S3) .


问:
Is this modified BSD license compatible with GPL?
I am working on open source project, that uses the source code from FlatAssembler project, which uses following license, based on BSD 2-clause with one additional clause added at the end of the text.

So, the questions are:

What this additional clause changes?

Is it legal to use this code and to license my project under GNU GPL? (The original BSD 2-clause license is compatible with GPL, but this one seems to be not.)

flat assembler  version 1.71 Copyright (c) 1999-2012, Tomasz Grysztar.
  All rights reserved.

This program is free for commercial and non-commercial use as long as
  the following conditions are adhered to.

Copyright remains Tomasz Grysztar, and as such any Copyright notices
  in the code are not to be removed.

Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are
  met:

Redistributions of source code must retain the above copyright notice,    this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright    notice, this list of conditions and the following disclaimer in the

  documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR
  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The licence and distribution terms for any publically available
  version or derivative of this code cannot be changed. i.e. this code
  cannot simply be copied and put under another distribution licence
  (including the GNU Public Licence).

答:
Disclaimer: IANAL!

The extra clause does nothing which condition 1 does not already do, so it is pointless:

"1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer."

This already says that basically you cannot change the license. Because "the above copyright notice", "this list of conditions" and "the following disclaimer" constitute the entire license. You can't change a thing.

Also, note how the addition of the useless extra clause wrecks the organization of the license. Clauses 1 and 2 ("this list of conditions") refer to the "above copyright notice" and the "following disclaimer". This means that everything which follows the list of conditions is a disclaimer. So any additional conditions must be added to the list, before the disclaimer. Additional conditions should not be added after the disclaimer.

The license is compatible with the GPL because you can use that source file in a GPL-ed program.  By including BSD licensed modules in a program, you are not changing the source code license of those modules to GPL.


问:
How to better performance
In one of my interviews I was asked a vague question, I'm still not sure of the answer. We have a client and a server they are placed apart. The network that connects them has high latency, which is dominating the performance. How can we improve the performance. Note that we can't change the network's topology. I was thinking of caching, breaking the request to multiple smaller requests and opening multiple connections with the server. Any ideas?

Please note that the question description is vague and I wasn't supplied with more information about the situation.

Clarified question: How should the client-server communication be designed in order to get the best performance on a network that has big latency?

答:
A few ideas that come to mind:

Compression - is the data being sent back and forth compressed to make it as small as possible?

Caching - could there be caches set up on the client to try to make things appear faster?  Could the system be set up so the client can operate disconnected from the server for a time?

Network connectivity - While you can't change the topology, it may be worth identifying if there is something else here that could be changed?

While it is an open question, there is something to be said for what is in this client/server system. Are there database calls?  Is there synchronization to be set up?  Are there alternative architectures that may work better?

The question then becomes how much can compression affect things.  If the compression can mean that 100th or 1000th as much data has to be sent on the wire, I'd think that could be a factor as sending so much less may change things.  The issue with the question is that there are many unknowns.

Title: How to better performance


答:
If high latency is screwing up performance, I'd do the exact opposite of what you're suggesting: find ways to combine multiple requests into a single request.

Let's say latency is 1 second, and you need to process 100 items, and the actual processing time is 0.01 seconds per item.

```<code>100 requests
============
Processing time = 0.01 * 100 =   1 second
Latency         = 1 * 100    = 100 seconds
Total time                   = 101 seconds
</code>```

But if you can find a way to send two items in a single request:

```<code>50 requests
============
Processing time = 0.01 * 100 =  1 second
Latency         = 1 * 50     = 50 seconds
Total time                   = 51 seconds
</code>```

Congratulations! You just cut the runtime for this batch in half.  Or, if you can find a way to send all 100 items in a single request:

```<code>1 request
============
Processing time = 0.01 * 100 = 1 second
Latency         = 1 * 1      = 1 second
Total time                   = 2 seconds
</code>```

As a few commenters have noted, this only makes sense if the problem actually is latency, and not other network-related issues.  But since that's what you were asked about, this is the right way to handle it.

Title: How to better performance

Body:
In one of my interviews I was asked a vague question, I'm still not sure of the answer. We have a client and a server they are placed apart. The network that connects them has high latency, which is dominating the performance. How can we improve the performance. Note that we can't change the network's topology. I was thinking of caching, breaking the request to multiple smaller requests and opening multiple connections with the server. Any ideas?

Please note that the question description is vague and I wasn't supplied with more information about the situation.

Clarified question: How should the client-server communication be designed in order to get the best performance on a network that has big latency?

Body:
If high latency is screwing up performance, I'd do the exact opposite of what you're suggesting: find ways to combine multiple requests into a single request.

Let's say latency is 1 second, and you need to process 100 items, and the actual processing time is 0.01 seconds per item.

100 requests
============
Processing time = 0.01 * 100 =   1 second
Latency         = 1 * 100    = 100 seconds
Total time                   = 101 seconds

But if you can find a way to send two items in a single request:

50 requests
============
Processing time = 0.01 * 100 =  1 second
Latency         = 1 * 50     = 50 seconds
Total time                   = 51 seconds

Congratulations! You just cut the runtime for this batch in half.  Or, if you can find a way to send all 100 items in a single request:

1 request
============
Processing time = 0.01 * 100 = 1 second
Latency         = 1 * 1      = 1 second
Total time                   = 2 seconds

As a few commenters have noted, this only makes sense if the problem actually is latency, and not other network-related issues.  But since that's what you were asked about, this is the right way to handle it.


答:
Either load more data up front so you dont need to go back again until there's an update.

Use a local data store and sync when appropriate if it's mostly local data

Make lots of targeting small requests, small data going across the wire

and compression.

I probably would've thought of a funny answer... personality goes a long way.

```<code>"I'll just use my iPhone, i've got unlimited data and these 4g speeds are GREAT!"
</code>```

Title: How to better performance

Body:
Either load more data up front so you dont need to go back again until there's an update.

Use a local data store and sync when appropriate if it's mostly local data

Make lots of targeting small requests, small data going across the wire

and compression.

I probably would've thought of a funny answer... personality goes a long way.

"I'll just use my iPhone, i've got unlimited data and these 4g speeds are GREAT!"


答:
On the network with big latency, at first place you have to organize you protocols asynchronously. That means that you have to try to work without waiting for the response of the server.

You have to send the requests and to not wait for response. Then you have to organize separate thread or something that to process the responses when they come.

This way you will use the whole speed of the network and will get the respective responses only delayer in time with double the latency time.

Of course, it is possible only if the requests does not depends on each other and this is your task to design such workflow of the application.

Title: How to better performance


问:
R vs Python for data analysis
I have been programming for about a year and I am really interested in data analysis and machine learning. I am taking part in a couple of online courses and am reading a couple of books.

Everything I am doing uses either R or Python and I am looking for suggestions on whether or not I should concentrate on one language (and if so which) or carry on with both; do they complement each other?

-- I should mention that I use C# in school but am familiar with Python through self-study.

答:
As an old school (over 50) scientist who has and continues to use a number of these tools I will add my two cents.  I have worked with colleagues who still write every piece of code in Fortran, from trivial one-off data analysis jobs to code that dominates some of the worlds supercomputers.   Recent Fortran dialects (F90, F95, F2003, F2008) are IMHO, some of the best designed languages in existence.   Decades of experience with high performance computing has led to a quite impressive language development.

I have only used Python at times, and will revisit it (mostly because of Sage) but I use a time tested suite of languages that work well for me.   Fortran, C, Perl, R, and Scheme (with tcl for scripting VMD).   I find the combination of R and Fortran and C to be very comfortable.   In contrast to other comments made about the object model in R, it is a good object model for interactive work, based upon the CLOS concept of generic functions and method dispatch.   When working interactively with a new package you can often rely upon generic functions like “print” and “plot” to do something productive.

The API to Fortran and C is very easy to use.  If your used to working in Fortran and C for modeling and data analysis this is a big plus.   The ability to dynamically generate R code and evaluate it, while not nearly as clean as the macro systems in Lisp and C, is very useful when working up dynamic data sets.

Some limitations of R for real data include the call by value approach.  While there are CS reasons for call by value, real world programming with big numeric data requires some form of call by reference (note the importance of Fortran common blocks in older code, or module data in newer code).  The approach adopted by PDL (Perl Data Language) is especially elegant in this regard.  (Pdls are essentially call by reference unless you request a copy.    Sub-pdl’s reference a sub section of a parent pdl, in a far cleaner syntax than Fortran or C provide).

It is good to learn many languages.   Python is undoubtably an important language, but R is as well in it’s domain.   But when the rubber really needs to meet the road in science Fortran and C (and C++ for some) will be hard to displace.

Title: R vs Python for data analysis


问:
Why do operating systems do low level stuff in C and C++? Why not just C++?
On the Wikipedia page for Windows, it states the Windows is written in Assembly for the bootloader and task switcher, and C and C++ for kernel routines.

IIRC, you can call C++ functions from an extern "C"'d block. I can get using C for the kernel functions so pure C apps can use them (like printf and such), but if they can just be wrapped in an extern "C " block, then why code in C?

答:
It could be that with C you spend most of your time thinking about the problem at hand and how  to code the solution. In C++ you end up thinking about C++ and its myriad of features, functions and obscure syntax.

Also in many C++ shops your code get monitored by the "fashion police" who are enthralled by the latest set of design patterns, or, the latest unintelligible pronouncements of the great god Stroustrup. Pretty code becomes more valued than working code, finding a use for the latest set of Boost templates is admired more than finding a working solution for the business.

My experience is that for all the clever features and OO purity of C++, coding in plain C gets the job done quicker and more effectively.


答:
Latency and bandwidth are two different things; high latency means there's a time overhead in sending messages back and forth, while low bandwidth means that the pipeline has a limit restricting synchronous data transfer. Both of these can mean it will take a while to get data, but for different reasons.

The solution for high latency if bandwidth is not an issue is to use larger requests for data; as much as you know you'll need, to minimize the additive impact that the overhead delay will have. The solution for low bandwidth if latency is not an issue is to use smaller requests for data, as little as you can get away with at one time, to reduce the delay that completely retrieving a large dataset can add. In addition, for both cases, you should work as asynchronously as possible, so that you can give the user feedback while the data pull is occurring. Caching is an option if the data is relatively static and is not sensitive (i.e. anyone else who could get into the system could see exactly the same data; stock ticker = cache, bank account balances = don't cache).

Title: How to better performance


问:
Help to simplify/model complex decision making (if, then, else scenarios)
I find my self in a situation, where I need to program a service, that can determine if two object in a huge database are the same. For simplicity, let's say it's used cars, and I want to determine if they are the exact same, based on various attributes e.g. brand, model name, year, milage, price, color etc. The problem is, that some of the attributes are obversely more significant/critical/important than others e.g. brand/model name HAS to be the same, where as the seller might wrongly have typed in the milage a slight bit differently. Additionally some times information such as the color might even have been left out, an hence I must fall back to less "unique identifying" information such as for an example the engine size. Apart from the direct weight/importance of each attribute there is another dimension of complexity involved - the attributes' importance vary with each other. So let's say e.g. that the car I'm comparing against the rest is very old, then suddenly the importance of the milage it self increases since it's way more likely to be unique among similar cars, than compared to a younger car. This is obviously due to the fact, that the dispersion of milage on older cars are much wider than the one on younger cars.

As you might imagine, modeling this as a lot of nested if-then terms easily ends up in a complete mess, and completely unreadable afterwards, and even hard to model in the first place.

So my questions are:

What might this kind of problem be called in programming, and how do I deal with it?
What solutions/algorithms might be good solutions? References, links?
Good reading on the topic?

答:
Basically, you're defining a framework for a "heuristic" that will be specified by the customer to power a "search engine".

Try using a numeric "relevance" metric, and weigh various attributes to determine how much a match or miss affects relevance. Some of these, like brand/model, will be static. Others, like milage range based on year, could be dynamic (the older the car is, the further outside the user's defined range the mileage could fall before it's not a good match). Some or all of these could be user-specified as to importance (some quirky user may want a car, any make/model/year/color/price, with exactly 111,111 miles on it, and will not consider a car with a mile more or less; if you want to humor this kind of weirdo, you can allow the user to specify that this specific mileage is of "critical" importance).

You can then plug this data in as conditions of a SQL WHERE clause, using whether the data matches the criteria and the relative importance of each criterion to calculate the "relevance" of the record to the query, and show only cars that meet a minimum relevance threshold (say 75% or better; again this can be user-configurable, so if the user thinks he's not getting enough results he can show more "outliers").


答:
I would say that it depends on the following:

Is CPU utilization critical?  Every check for NULL takes some amount of time.

What are the odds that the pointer is NULL?  Was it just used in a previous function.  Could the value of the pointer have been changed.

Is the system preemptive?  Meaning could a task change happen and change the value? Could an ISR come in and change the value?

How tightly coupled is the code?

Is there some sort of automatic mechanism that will check for NULL pointers automatically?

CPU Utilization/Odds Pointer is NULL
Every time you check for NULL it takes time.  For this reason I try to limit my checks to where the pointer could have had its value changed.

Preemptive System
If your code is running and another task could interrupt it and potentially change the value a check would be good to have.

Tightly Coupled Modules
If the system is tightly coupled then it would make sense that you have more checks.  What I mean by this is if there are data structures that are shared between multiple modules one module might change something out from under another module.  In these situations it makes sense to check more often.

Automatic Checks/Hardware Assist
The last thing to take into account is if the hardware that you are running on has some sort of mechanism that can check for NULL.  Specifically I am referring to Page Fault detection.  If you system has page fault detection the CPU itself can check for NULL accesses.  Personally I find this to be the best mechanism since it always runs and does not rely on the programmer to put in explicit checks. It also has the benefit of practically zero overhead.  If this is available I recommend it, debugging is a little harder but no overly so.

To test if it is available create a program with a pointer.  Set the pointer to 0 and then try to read/write it.


答:
Let me add one more voice to the fugue.

Like many of the other answers, I say--don't bother checking at this point; it's the caller's responsibility.  But I have a foundation to build on rather than simple expediency (and C programming arrogance).

I try to follow Donald Knuth's principal of making programs as fragile as possible.  If anything goes wrong, have it crash big, and referencing a null pointer is usually a good way to do that.  The general idea is a crash or an infinite loop is far better than creating wrong data.  And it gets programmers' attention!

But referencing null pointers (especially for large data structures) does not always cause a crash.  Sigh.  That's true.  And that's where Asserts fall in.  They are simple, can instantly crash your program (which answers the question, "What should the method do if it encounters a null?"), and can be turned on/off for various situations (I recommend NOT turning them off, as it's better for customers to have a crash and see a cryptic message than have bad data).

That's my two cents.


答:
If a team member is having trouble understanding another team member's code (for any reason); then that team member should be able to find out who wrote the original code (any sane revision control system) and should be encouraged to ask the code's author to explain it directly (e.g. walk over to their desk, sit down and discuss it).

In this case, if the lack of comments is a problem then the person who isn't adequately commenting their code will spend a large amount of their time explaining what they've done; and (if they're smart) will start adding comments to avoid spending time on all those explanations.

Note that encouraging team members to ask each other for explanations is valuable for other reasons. For example, maybe a team member is less experienced and can learn things from the more experienced team members.

Mostly, by encouraging team members to ask each other for explanations you create a self-balancing system; where different team members "auto-adjust" to each other.

Title: How can I deal with a team member who dislikes making comments in code?

Body:
One of my team members consistently avoids making comments in his code.

His code is not self-documenting, and other programmers have a difficult time understanding his code.

I have asked him several times to comment his code, however he just gives excuses or claims that he will do it later. His concern is that adding comments will take too much time and delay the projects.

What argument can I present to him to convince him to properly document his code?

On that note, am I wrong to focus on the code comments or is this indicative of a larger problem which should be addressed?


问:
Continuous Build server (cc.net, hudson, bamboo, etc...) remote build experience?
We currently use once cc.net server for our build process, which builds both .net (using msbuild & nant) and java (using maven and ant).

CC.net monitors source control, and triggers a remote build running on a separate server.  CC.net then collates the results.

When we run the remote build, it typically:

runs nunit or junit or similar using mocked data

optionally runs a DB script to create a fresh database instance or restore a database from a known position.

runs selenium or similar to test UI

runs emma or ncover for code coverage

builds the system for various deployment environments (test, acceptance, production)

We may have several builds running at a time, some .net and some java (from different project teams).

It's quite time consuming to get the remote builds working when we set up a new project and we feel that there must be something more suited to remote builds than cc.net.

Does anyone have any experience with remote builds with continuous integration systems?

I don't really want feature lists of CI servers, I'd more appreciate hearing about how you've used them in a multi language, multi server environment.

答:
good question. We are currently also trying to find out which tool fits best for us. So I will only be able to tell you a little experience.
But we would be very interested in, which CI system you chose now and for which reasons. So please keep us informed.

I'm very impressed how high the level of your CI is. I have to admit that we have less requirements because we don't run UI tests yet and don't create database instances or similar, we just using mocks for our unit tests.

Now to our experiences up to now:

For Java projects we are using Bamboo which works fine using JUnit and Emma. And there is not so much effort to set up a new project.

For .NET projects we are still searching for the best solution

Cruise Control: We could not get it running yet due to problems with connection to our repository

TFS :

a) There are a few setup steps necessary to be able to execute the first build.

b) There are some pitfalls where you have to get over regarding access rights. There are many roles you can define and you have to know exactly which rights has your build process and which has your personal login account. But if you have enough time to manage, you can define every particular granularity you need.

c) Regarding referenced libs there are also some things to manage if you want to share libs for many projects and don't want to handle them at every single project

d) Running NUnit test is not so easy as we thought. It's only easy if you are using test execution provided by Visual Studio, but this is not NUnit

e) We didn't try to run NCover yet (first things first :-))

Hudson : Next tool we'll try. Seems to have a really good and easy plugin for .NET, I will let you know how it worked

Bamboo : First prediction we got: "Too Java specific". But maybe we'll try the .NET plugin nevertheless, I will let you know

Hope, we can continue this discussion and exchange experiences.

Andy

OwnerDisplayName: Mayoares


问:
NoSQL use cases
I am developing an application for my universities student research symposium around the idea of polyglot persistence. I have read Martin Fowlers book and done some other research online about the verious different types of NoSQL db's. I am curious as to what the different use cases are for the different NoSQL databases?

Currently I have what Fowler has pointed out (other research was mostly done into API's).

Key/Value:

session data

 user profile

shopping cart

basically anything that has a single unique key that can be easily generated and replicated

Document:

event logging

CMS/blogging

e-commerce (after the transaction has been completed)

web analytics

Column Family:

event logging

CMS/blogging

counters

expiring usage

Graph:

connected data

location based services

recommendation engine

So what other use cases are there? More specifically what are the use cases for each type of NoSQL db that are better than a relational model?

答:
A Very Large List of NoSQL Use Cases

That list comes in handy now and then, and also points out when a certain NoSQL solution is specialized for that specific use case. The original HackerNews thread (linked at the bottom of that page) is useful for extended commentary as well.

Title: NoSQL use cases


问:
What's the point of those technique detailed interview question for Senior dev?
I've had an internal promotion interview for gain a higher level programmer title, something like Senior plus.
And I've been interviewed by around 7 people using different technology, and people using the same technology with me(.Net) tend to ask very technique detailed question which can actually easily been found via Google, like what is JIT, how GC works, difference between List and Array, abstract class and interface, delegate and event, even what is the class name when you process Upload file, etc. And I've got only one question asked about one of my design idea in my project and just simple discussion. But for most other my design choice they seems just not interested.

I didn't get the result yet and it would be about late this week, but here is my concern:

I personally think when I play as a senior programmer, I mainly solving problems, and I just need to know there are some certain way can make it happen but I may not remember every detailed thing, and that's should be why we have those detailed reference documentation like MSDN.

I feel it fine if you found I've not too much experiences around this area like Junior to Intermediate level, but when tend to interview higher level guy, isn't you should more focus to see how this guy logical thinking is and how good is he/she at solving problems? Does everybody just think if you know every detailed small point tech stuff then you are a Senior+. And by checking those interview question books I found there are more those kind of questions.

If I spend 5 days go through those interview question list book I can easily make those guys feel a wow, but does that really mean anything? This kind of interview can easily let those guys good at remember things gain higher salary even they have no idea how to solve difficulty problem.

So why is this happening on the world, is that just cause problem solving skills, design skills are hard to measure? I've serve this company for years and only had few interview with other companies so I wonder is that every company doing the same thing?

Or is this actually just my own problem that I should try harder to remember everything in MSDN in my mind so that I can work even without it and internet?

EDIT

For better explain my situation regarding Frank's concern about job tasks.
Sorry it's my bad didn't clarify those background.

There actually will be no specifically job's task changes, I personally think what I've done already played as a senior, like code review, mentor members, review BA's doc and give technique opinions, design architecture of new projects, it's just my title still stayed without senior and I asked to get one to reflect pay slip and lead to such an interview. This is a Saas company so people stay in one project as long as that product is still alive, this leads they need people be able to design more new features based on current product, fixing technique difficulty on live servers, design/code review/mentoring members And higher level title based on technique would be Architect and we do not have any job similar with Technical Expert. And I agree if you want to play Technical Expert role you should know more detail about the tech you use. Sorry it's my bad didn't clarify those background.

答:
As a senior engineer and a person who hires engineers, it sounds like the interview process missed a whole swath of conceptual information.

I want my senior engineers to be able to visualize an end-result, choose the right technology, design the right database and processes, and be able to delegate the detail to other engineers when needed.  Google is always at their side, so I focus on their ability to deliver results, rather than how big of an SD card they plugged into their ear.  (lol).

Hope it works out well.


答:
The usual IANAL applies etc. etc.

It looks to me as if the author tried to turn the BSD license into a viral license by demanding that the original license terms be unchanged for this and any derived work.

While this is a perfectly viable idea, I doubt it would stand up in court under all circumstances - the various versions of GPL are quite long, and there is a good reason why they have to be. Viral licensing is delicate matter, because it walks a thin line - you have to define carefully what is "source" and what is a "binary distribution", what constitutes a "derived work", etc., all the while avoiding clashes with existing law that might render the license partially void. More permissive licenses like BSD-2 can be much shorter for the simple reason that they do not have to make these distinctions: as long as you retain the copyright notice and author credits, you are free to copy and modify as you please.

What the consequence of this license would be is a bit unclear: it could be that the last clause is void, which would make it equivalent to plain BSD-2; or it could be that it overrides most of the freedom that regular BSD-2 would give you, which would make it even more restrictive than GPL.

The problem is that the author tries to release under BSD-2, but at the same time forbidding redistribution under a viral license, which is contradictory, because forbidding relicensing would basically make the license viral itself.

All that theoretical and legal mumbo-jumbo aside, the obvious solution is to contact the author and ask for permission; if he's fine with you including his code in your project, then there is no problem.

Title: Is this modified BSD license compatible with GPL?

Body:
I am working on open source project, that uses the source code from FlatAssembler project, which uses following license, based on BSD 2-clause with one additional clause added at the end of the text.

So, the questions are:

What this additional clause changes?

Is it legal to use this code and to license my project under GNU GPL? (The original BSD 2-clause license is compatible with GPL, but this one seems to be not.)

flat assembler  version 1.71 Copyright (c) 1999-2012, Tomasz Grysztar.
  All rights reserved.

This program is free for commercial and non-commercial use as long as
  the following conditions are adhered to.

Copyright remains Tomasz Grysztar, and as such any Copyright notices
  in the code are not to be removed.

Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are
  met:

Redistributions of source code must retain the above copyright notice,    this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright    notice, this list of conditions and the following disclaimer in the

  documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR
  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The licence and distribution terms for any publically available
  version or derivative of this code cannot be changed. i.e. this code
  cannot simply be copied and put under another distribution licence
  (including the GNU Public Licence).


答:
Interesting, that readability as applied to natural language is measured by speed of reading and comprehension. I guess a simple rule can be indeed adopted, if a particular code comment does not improve this property, it can be avoided.

Why comments?

Although, code comment is a form of embedded documentation, there are multiple ways in high-end programming languages to avoid superfluous "over-documented" programming (of meaningful code) by using elements of the language itself. It is also a bad idea to turn code into listings from programming text book, where individual statements are literally explained in almost tautological fashion (mind the "/* increment i by 1 */" example in already provided answers), making such comments relevant only to programmers inexperienced with the language.

Nonetheless, it is the intention of trying to comment the "under-documented" (but meaningless) code that is truly "the source of all evil". The very existence of "under-documented" code is bad signal - either it is an unstructured mess, or wacky hack of mystical lost purpose. Obviously, the value of such code is questionable at least. Unfortunately there are always examples, when it is indeed better to introduce a comment into a section of (visually grouped) formatted lines of code than wrap it into new subroutine (mind the  "foolish consistency" which "is the hobgoblin of little minds").

Code readability != code comments

Readable code does not require annotations by comments. In each particular place in the code there is always a context of a task that this particular code is supposed to achieve. If purpose is missing and/or code does something mysterious = avoid it at all costs. Do not allow weird hacks to populate your code - it is a direct result of combining buggy technologies with lack of time/interest to understand the foundations. Avoid mystical code in your project!

On the other hand, Readable program = code + documentation can contain multiple legitimate sections of comments, e.g. to facilitate generation of "comments to API" documentation.

Follow code style standards

Funny enough the question is not about why to comment code, it is about team work - how to produce code in highly synchronized style (that everyone else can read/understand). Are you following any code style standards in your company? It's main purpose is to avoid writing code that requires refactoring, is too "personal" and "subjectively" ambiguous. So I guess, if one sees the necessity in using code style, there is a whole serious of tools how to implement it properly - starting with educating people and ending with automation for quality control of the code (numerous lints, etc) and (revision control integrated) code review systems.

Become a code readability evangelist

If you agree that code is read more often than it is written. If clear expression of ideas and thinking clearly is important to you, no matter what language is used to communicate (math, machine code or old-english).. If your mission is to eradicate dull and ugly way of alternative thinking.. (sorry, the last one is from another "manifest").. ask questions, initiate discussions, start spreading thought provoking books on code cleaning (probably not only something similar to Beck's design patterns, but more like already mentioned by R.C. Martin) which address ambiguity in programming. Further goes a bullet-point passage of key ideas (quoted from O'Reilly book on readability)

Simplify naming, commenting, and formatting with tips that apply to
every line of code

Refine your program’s loops, logic, and variables to reduce complexity and confusion

Attack problems at the function level, such as reorganizing blocks of code to do one task at a time

Write effective test code that is thorough and concise—as well as readable

Cutting "commenting" out, one is still left with a lot (I guess writing code that does not need comments is one piece of great exercise!). Naming semantically meaningful identifiers is a good start. Next, structuring your code by grouping logically connected operations into functions and classes. And so on. A better programmer is a better writer (of course, assuming other technical skills given).


问:
Workflow: Using binary document formats in Git without locks (moving from subversion)
We're a software consultancy with a multitude of projects for different customers. We traditionally use Subversion, but are currently considering moving to Git.

A significant portion of the documents we produce are shared with our customers (requirements, global designs, test specs, etc), and we use MS Office to produce these. In Subversion, we could use its "Lock" feature to ensure that no one was editing the same document at the same time. In Git, you can't do that since by its distributed nature, git doesn't have locks.

Locks are really little more than a communication mechanism, but they're a very effective one.

Currently, our code and customer-facing documents are typically in different subfolders of a different svn repository. When moving to git, what would you recommend we do? I see a set of options:

We move the svn repositories to git 1-on-1. Instead of using locks on the Office files, we do what the git people suggest and somehow try to change our workflow to fix it. This could be working in a branch on any document edit, and merging that over review. This approach breaks over e.g. Excel sheets that contain project management info; they're easily edited by team members (and we encourage that this is done), but not subject to any formal review process

We use git for code and svn for docs and project management. This has the disadvantage that certain more design-ish documents won't be "nearby" the code it specs, increasing the chance that people forget to update them. Additionally, everybody has to use and understand two sets of tools. That said, maybe this is a great opportunity to move to text-based doc tools (latex, markdown, HTML, whatever) for non-customer-facing design docs.

Like 1, but we hack up a git lock command that does what svn lock does for us (toggle the read-only flag appropriately and sync with a server through some means).

I don't buy the argument that locks don't work in a DVCS because the system should even work when you're entirely offline. Svn locks can be overridden as well; they're a communication mechanism. Without some sort of network connection, you won't get your computer to communicate a lot.

We can't be the only shop who're very happy with how svn lock fits in our workflow, right?

Any ideas or tips?

I found https://stackoverflow.com/questions/119444/locking-binary-files-using-git-version-control-system but the discussion is rather technical; i'm looking for ways to solve or avoid the practical problem of two team members editing the same binary file at the same time.

答:
I would advise you to stay with SVN for the MS Office documents for two reasons:

It is already there and it is (in my opinion) better for keeping
Office documents (look here). Has much more third party tools for doing this.

The lock, though can be achieved in Git, is not "the Git kind of way
of doing things". If you need these features, stick with the tool
that gives you the best solution.

There is a saying that I like that says something like this: "When You're Holding a Hammer, Everything Looks Like a Nail". Just because you are moving to Git to hold you code, it doesn't mean that you should use it to hold your documents.