问:
Data structures in functional programming
I'm currently playing with LISP (particularly Scheme and Clojure) and I'm wondering how typical data structures are dealt with in functional programming languages.

For example, let's say I would like to solve a problem using a graph pathfinding algorithm.  How would one typically go about representing that graph in a functional programming language (primarily interested in pure functional style that can be applied to LISP)?  Would I just forget about graphs altogether and solve the problem some other way?

答:
In Common Lisp, if I needed to represent a tree, I'd use either a list (if it was for just a quick hack) or define a tree class (or struct, but classes interact well with generic functions, so why not).

```<code>(defclass tree ()
  ((node :accessor node :initarg :node)
   (children :accessor children :initarg :children)))
</code>```

If I need literal trees in the code, I'd probably also define a make-tree function that takes a list representation of the tree I want and turn it into a tree of tree-objects.

Body:
In Common Lisp, if I needed to represent a tree, I'd use either a list (if it was for just a quick hack) or define a tree class (or struct, but classes interact well with generic functions, so why not).

(defclass tree ()
  ((node :accessor node :initarg :node)
   (children :accessor children :initarg :children)))

If I need literal trees in the code, I'd probably also define a make-tree function that takes a list representation of the tree I want and turn it into a tree of tree-objects.


问:
Bug reopen vs. new
A bug was opened, fixed, verified and closed. A month later, it showed up again in a subsequent version after several iterations without any regression.

Provided the bug characteristics are the same, would you reopen the existing bug ID or open a new one with a link to the closed bug?

答: If it was verified and closed, and worked for a while, and then appeared again after something was changed, then it's not the same bug. It may manifest itself similarly as the old bug did, but its cause may well be different. So it's not the same bug. So I would open a new one, with a link to the closed bug.</p>

ContentLicense: CC BY-SA 3.0

Title: Bug reopen vs. new


问:
Facing quality issues
A workforce management software has complex GUI (for example values in a page depends on the status (closed or open) of other pages). Only latest and near past development has test coverage. During our last release, we received lots of bugs from customer in-spite of 2 weeks of testing Sprint. We don't have dedicated test team. The developers does the unit test and User acceptance test. Every day triggers automated regression test.

I am afraid the developers are not testing the entire workflow because its time consuming also not able to automate it because of its complexity. Any suggestions? The legacy code (15 years of development) has less code coverage. How can I improve quality?

Note: Now not possible to hire testers to have independent test team.

答:
You are having( or started having) Unit tests, which is really good to start with.

As you are saying this is a complex project, a separate Quality Assurance team is must. A developer testing their code does not look at the application with the users perspective and they will be able to design and execute the test cases which gives higher level of confidence.

Until the QA team is established , Ask the developers to follow  steps to increase the test coverage for your application and increase the confidence on the application.

1- Keep adding unit tests for all new codes and bug fixes.

2- For Functional testing:

a) Write down the test cases and get them reviewed with users. You don't have to write detailed documents but bullet points are enough ( like , Open the timesheet of an employee , approve the timesheet . Change the schedule to crossing midnight, adjust and approve the timesheet etc).

b) execute them and if you find bugs, don't stop and fix the bugs. But document the bugs and fix them at the end and release. Test them in Non dev environments.

c) I have also seen issues coming in prod for some users/ or some specific scenarios.. That is a clear indicator of some of the data is incorrect and other data is fine or the design of the application has issues.

so If you still see many bugs are creeping up, the problem must be at the design level or the data level. Make sure the data is clean and db has enough constraints. For design issues, you will have to refactor the application slowly.

Good luck.

Title: Facing quality issues


问:
Simple issue tracker for 1-2 developers
I'm currently working mostly alone on a project (in Java). I'm mostly alone as I have an advisor that gives me high level instructions on what to do, and will seldom make any code contribution. She will code in a couple of acceptance tests from time to time, though.

I've never used an issue tracker before, and was thinking about starting to use one now, as I'd like to have a place where I can log possible bugs I find and keep track of them in a centralized manner. Would it be possible to integrate the issue tracker with Eclipse, better yet.

So here are the constraints:

It's NOT a open-source project. Our code is not to be shared with anyone!

we are and will be using Subversion;

we have our own Subversion server and we will keep using this same Subversion server;

it must be free;

it must allow at least 2 users.

What is your advice on what to pick? I'm looking for the simplest solution available.

答:
Trackie is extremely simple but flexible.

It is aimed at small tech or semi-tech/semi-creative teams with a need to keep track of issues in a simple, one-view way. It supports custom statuses (with custom colors) of issues, as well as priority and assignees.

While the UI is already very simple and clean, an extra simplified UI is presented to users who are added to a project as a Client.

Lastly, it accepts issues by email. Not only directly, but also if you forward a client's problem to Trackie, any correspondence with your client will from then on run through Trackie without your client even knowing it. Keeping everything in a single place.

It is free while it is in private beta. Whether it will remain free is unclear at the time of writing.

Disclaimer: I am the developer of this issue tracker. But I think this answer is relevant to the OP nonetheless.

Title: Simple issue tracker for 1-2 developers


答:
One of the reasons why developers can't effectively test (the way a dedicated QA does, not unit testing which all devs should do) is because they know what they did in the code. They therefore won't push the edges of the system. They won't do the things a user might do because they know it wouldn't work. Therefore they don't find the bugs, the unit tests don't catch. Further, I have never seen application developers effectively do integration testing or regression testing. All they care about is their one small piece not how it fits together with everything else. You need this kind of testing and you clearly aren't getting it.

You really need professional qa to catch this stuff before going to production. It saves the company money and time and embarassment where the users find the bugs.

Since you don't have professional qa, you need to at least have different developer do the testing than the one who wrote the code. Someone whose goal is to break the other person's code. If you can borrow some users to do user testing, that is even better.

You also might try assigning one developer at a time to be the QA for a Sprint (you can change each sprint if you have to) that person will develop the other kinds of tests beyond unit tests and will write test cases for user testers to perform. You might find someone who is really good at it and who enjoys it and suddenly you have the start of your QA team.

Title: Facing quality issues

Body:
A workforce management software has complex GUI (for example values in a page depends on the status (closed or open) of other pages). Only latest and near past development has test coverage. During our last release, we received lots of bugs from customer in-spite of 2 weeks of testing Sprint. We don't have dedicated test team. The developers does the unit test and User acceptance test. Every day triggers automated regression test.

I am afraid the developers are not testing the entire workflow because its time consuming also not able to automate it because of its complexity. Any suggestions? The legacy code (15 years of development) has less code coverage. How can I improve quality?

Note: Now not possible to hire testers to have independent test team.


问:
How do you track bugs in your personal projects?
I'm trying to decide if I need to reassess my defect-tracking process for my home-grown projects.  For the last several years, I really just track defects using TODO tags in the code, and keeping track of them in a specific view (I use Eclipse, which has a decent tagging system).

Unfortunately, I'm starting to wonder if this system is unsustainable.  The defects I find are typically associated with a snippet of code I'm working on; bugs which are not immediately understood tend to be forgotten, or ignored.  I wrote an application for my wife which has had a severe defect for almost 9 months, and I keep forgetting to fix it.

What mechanism do you use to track defects in your personal projects?  Do you have a specific system, or a process for prioritizing and managing them?

答: I usually use a web based revision control system(Github, Bitbucket, Redmine, Google Code, ...) to store my source code and track bugs. If you think there's a bug in a specific code, you can create an issue with the revision number/changelist/changeset and specify which file and the range of lines you suspect.</p>

ContentLicense: CC BY-SA 2.5


问:
What difference with Android when iOS is not open?
My question may seem silly, but, I've read in many Android books that the better part of Android is Android is open, and iOS is closed.

But, from a programmer viewpoint, who cares about this point? Programmers just care about the platform: is it easy to program in, easy to have help in the network, etc. They don't care about whether the platform is open or closed.

One book had additional information, but I don't understand it much. Since iOS is not open, I’d be able to share my work with others only if Apple allowed it.

So, if you write an app, and need to share, you must have Apple's permission, but not Android's?

Please help me understand this point. I'm preparing for a presentation on Android. And I don't know if I should put this point into my presentation.

答: There is a direct implication: If you use GPLv3 code in your app, you cannot legally make an iOS app out of it (IANAL, opinions vary), while with Android there is no problem, as long as you provide the source code somewhere.</p>


问:
What do you use macros in your editor for?
My editor of choice is Notepad++. It has macro capabilities, but as much as I think, I don't see how can I use it for anything.

Do you use macros in your editor that have made your work easier? (for general use I mean)

答:
Since you asked about the usage related to programming, and two other answers are related to data manipulation, one example which comes in mind is to change the coding style.

For example, you often see compact switch statements like this in C# code:

```<code>switch (value)
{
    case "Hello":           ret =   5;    break;
    case "World":           ret =   4.9;  break;
    case "Something other": ret = -19.99; break;

    // Fifteen other cases.

    default: ret = string.Empty; break;
}
</code>```

While being, IMO, more readable, they don't comply to the default StyleCop ruleset. To make them compliant, you have to put one statement per line and, for example, add this if ret is not a variable, but a private field. Without automation, it would be cumbersome to do it by hand.

```<code>switch (value)
{
    case "Hello":
        this.ret = 5;
        break;

    case "World":
        this.ret = 4.9;
        break;

    case "Something other":
        this.ret = -19.99;
        break;

    // Fifteen other cases.

    default:
        this.ret = string.Empty;
        break;
}
</code>```

Body:
Since you asked about the usage related to programming, and two other answers are related to data manipulation, one example which comes in mind is to change the coding style.

For example, you often see compact switch statements like this in C# code:

switch (value)
{
    case "Hello":           ret =   5;    break;
    case "World":           ret =   4.9;  break;
    case "Something other": ret = -19.99; break;

    // Fifteen other cases.

    default: ret = string.Empty; break;
}

While being, IMO, more readable, they don't comply to the default StyleCop ruleset. To make them compliant, you have to put one statement per line and, for example, add this if ret is not a variable, but a private field. Without automation, it would be cumbersome to do it by hand.

switch (value)
{
    case "Hello":
        this.ret = 5;
        break;

    case "World":
        this.ret = 4.9;
        break;

    case "Something other":
        this.ret = -19.99;
        break;

    // Fifteen other cases.

    default:
        this.ret = string.Empty;
        break;
}


问:
What is meant by Scope of a variable?
I think of the scope of a variable as -

"The scope of a particular variable is the range within a program's source code in which that variable is recognized by the compiler".

That statement is from "Scope and Lifetime of Variables in C++", which I read many months ago.

Recently I came across this in LeMoyne-Owen College courses:

What exactly is the difference between the scope of variables in C# and (C99, C++, Java) when

However a variable still must be declared before it can be used

答:
What is meant by the "scope" of a variable?  think of the scope of a variable as "The scope of a particular variable is the range within a program's source code in which that variable is recognized by the compiler".

That definition matches the definition of scope in the C# specification; it says:

The scope of a name is the region of program text within which it is possible to refer to the entity declared by the name without qualification of the name

Notice that the C# specification calls out that it is the name which has a scope. Some variables do not have names, and therefore have no scope. And some things have names but are not variables; class names, namespace names, function names, and so on, all have a scope.

The scope of a local variable is frequently confused with its lifetime. The lifetime of a local variable is sometimes related to its scope; when control leaves the scope of a local (either through normal means or via an exception) then the local can be destroyed without anyone noticing. However, the runtime is permitted to make the lifetime of a local variable shorter if it can do so without anyone noticing. The runtime is also permitted to make a local variable live longer than its scope. And there are some situations in which a local variable is required to live beyond when control leaves its scope. (For instance, when the local is a closed-over outer variable of a lambda.)

Therefore it is unwise to use "scope" as a synonym for "lifetime". They are often related but can be quite different.  Use "scope" to mean "the region of text in which a name is valid", not "the period of time in which a variable is alive". Scope is fundamentally a compile-time concept.

What exactly is the difference between the scope of variables in C# and (C99, C++, Java)

I am not sufficiently familiar with the exact scoping rules of C, C++ or Java to give a short list of the differences. I can however point out some oddities of C#.

The first is that in C#, a local variable is in scope throughout its block.

```<code>class C
{
    int x;
    void M()
    {
        int y = x;
        int x = y;
    }
}
</code>```

In C++, the first "x" means "this.x" because the local variable x is not in scope until its declaration. In C#, the local variable x is in scope throughout the block. Using a local that is in scope before its declaration is an error in C#.

The second is that in C#, a simple name must mean the same thing throughout the scope in which it is first used:

```<code>class C
{
    int x;
    void M()
    { // M starts
        int y = x;
        if (y > 100)
        {
            int x = y;
        }
    }
}
</code>```

This is illegal because inside the block labelled "M starts" the simple name "x" is used to mean two different things. Now the first "x" means "this.x" because the local is not in scope. But it is illegal to use the same name to mean two different things, because that is a source of bugs.

If this subject interests you, I have written a number of articles on it. See

Link

http://blogs.msdn.com/b/ericlippert/archive/tags/declaration+spaces/

UPDATE: The rule that a simple name must have a unique meaning throughout the block which encloses it has been relaxed in C# 6. The language design team decided that the cost of user confusion was too high for the benefit in catching bugs.

Title: What is meant by Scope of a variable?

Body:
What is meant by the "scope" of a variable?  think of the scope of a variable as "The scope of a particular variable is the range within a program's source code in which that variable is recognized by the compiler".

That definition matches the definition of scope in the C# specification; it says:

The scope of a name is the region of program text within which it is possible to refer to the entity declared by the name without qualification of the name

Notice that the C# specification calls out that it is the name which has a scope. Some variables do not have names, and therefore have no scope. And some things have names but are not variables; class names, namespace names, function names, and so on, all have a scope.

The scope of a local variable is frequently confused with its lifetime. The lifetime of a local variable is sometimes related to its scope; when control leaves the scope of a local (either through normal means or via an exception) then the local can be destroyed without anyone noticing. However, the runtime is permitted to make the lifetime of a local variable shorter if it can do so without anyone noticing. The runtime is also permitted to make a local variable live longer than its scope. And there are some situations in which a local variable is required to live beyond when control leaves its scope. (For instance, when the local is a closed-over outer variable of a lambda.)

Therefore it is unwise to use "scope" as a synonym for "lifetime". They are often related but can be quite different.  Use "scope" to mean "the region of text in which a name is valid", not "the period of time in which a variable is alive". Scope is fundamentally a compile-time concept.

What exactly is the difference between the scope of variables in C# and (C99, C++, Java)

I am not sufficiently familiar with the exact scoping rules of C, C++ or Java to give a short list of the differences. I can however point out some oddities of C#.

The first is that in C#, a local variable is in scope throughout its block.

class C
{
    int x;
    void M()
    {
        int y = x;
        int x = y;
    }
}

In C++, the first "x" means "this.x" because the local variable x is not in scope until its declaration. In C#, the local variable x is in scope throughout the block. Using a local that is in scope before its declaration is an error in C#.

The second is that in C#, a simple name must mean the same thing throughout the scope in which it is first used:

class C
{
    int x;
    void M()
    { // M starts
        int y = x;
        if (y > 100)
        {
            int x = y;
        }
    }
}

This is illegal because inside the block labelled "M starts" the simple name "x" is used to mean two different things. Now the first "x" means "this.x" because the local is not in scope. But it is illegal to use the same name to mean two different things, because that is a source of bugs.

If this subject interests you, I have written a number of articles on it. See

Link

http://blogs.msdn.com/b/ericlippert/archive/tags/declaration+spaces/

UPDATE: The rule that a simple name must have a unique meaning throughout the block which encloses it has been relaxed in C# 6. The language design team decided that the cost of user confusion was too high for the benefit in catching bugs.

ContentLicense: CC BY-SA 4.0


答: Open a new bug, always. Why? Suppose it turns out to be identical to the previous bug, and you've released the fix for the previous bug. Your release notes will document that "Fix Bug XXX." From the point of view of issue tracking and making the release notes clearer, it is preferable to refer to the new bug "Fix Bug XXX+1 (which was similar in cause and effect to Bug XXX)" rather than to say "Fix Bug XXX (Again)" or something similar.</p>

Title: Bug reopen vs. new

Body:
A bug was opened, fixed, verified and closed. A month later, it showed up again in a subsequent version after several iterations without any regression.

Provided the bug characteristics are the same, would you reopen the existing bug ID or open a new one with a link to the closed bug?


问:
C++ Multithreading on Unix
I have two related questions:

Are there any good books for multithreading in C++, especially now that C++11 contains multithreading in the standard library?

I have the Wrox Programming on Unix book (1000 pages fat red one) and within it, it uses the Unix Thread class.

How does this code relate to boost and the C++11 multithreading libraries?

Is it better/worse/just specific to Unix etc?

Is the performance the same?

答:
That's quite a question!

Threading is an OS task and traditionally people used pthreads on Unix and Windows Gui threads on Windows.

Boost::threads basically wraps these operating system services. How the compiler and OS maker implement c++11 threads is upto them, but generally you would not expect a huge performance difference between any of them.

If performance really matters then you probably need to get much finer grained than any of the  general thread libraries and look at openMP, Intel TBB or OpenCL.

Title: C++ Multithreading on Unix


问:
Why does IIS default to Recycling the Application Pool every 1740 minutes?
Why does IIS default to recycle the app pool after a given time? Is there some specific reason other than perhaps most web apps are not managing the memory prudently? Given that you are managing your application's memory properly, is is safe to go ahead and turn this off? What are some potential down sides, up benefits to turning off recycling or keeping it on?</p>

答:
Shut it off and monitor the app pools. Most complex enterprise applications use lots of legacy code and much of that code is kinda leaky. So for most installs having the app pool restart occasionally is not a bad idea.

There are other options like monitoring the time of inactivity that may be a better solution for your situation.

Spinning up an app pool can take some time and make the application less responsive so keeping them up can help performance.


答:
Yes, the reason it defaults to once a day is out of a worry that the web app could have a memory leak. The biggest downside to frequently recycling IIS app pools is that it will cause web.config reading, assembly loading, and a recompilation of asp.net pages and (if you don't believe in pre-compiling them) code behinds. This is a rather heavy process and it doesn't occur until the next request for a page after the app pool has recycled, greatly slowing down that particular request. IIS7 now has a module you can download called Application Warm Up to help "deal" with this issue. </p>

<p>I personally prefer to use memory-based maximums coupled with logging on app start rather than scheduling my recycling. This allows me to assume my app has no memory leak and be proven wrong when the app pool recycles. </p>

Title: Why does IIS default to Recycling the Application Pool every 1740 minutes?

Body: Why does IIS default to recycle the app pool after a given time? Is there some specific reason other than perhaps most web apps are not managing the memory prudently? Given that you are managing your application's memory properly, is is safe to go ahead and turn this off? What are some potential down sides, up benefits to turning off recycling or keeping it on?</p>

Body:
Yes, the reason it defaults to once a day is out of a worry that the web app could have a memory leak. The biggest downside to frequently recycling IIS app pools is that it will cause web.config reading, assembly loading, and a recompilation of asp.net pages and (if you don't believe in pre-compiling them) code behinds. This is a rather heavy process and it doesn't occur until the next request for a page after the app pool has recycled, greatly slowing down that particular request. IIS7 now has a module you can download called Application Warm Up to help "deal" with this issue.

I personally prefer to use memory-based maximums coupled with logging on app start rather than scheduling my recycling. This allows me to assume my app has no memory leak and be proven wrong when the app pool recycles.


问:
Designing a large database with multiple sources
I have been tasked with redesigning, or at worst optimising the structure of a database for a data warehouse.

Currently, the database has 4 other source databases (which is due to expand to X number of others), all of which have their own data structures, naming conventions etc.  At the moment an overnight SSIS package pulls the data from the various source and then for each source coverts the data into a standardised, usable format.  These tables are then appended to each other creating a 60m row, 40 column beast!.

This table is then used in a variety of ways from an OLAP cube to a web front end.

The structure has been in place for a very long time, and the work I have been able to prove the advantages of normalisation, and this is the way I would like to go.  The problem for me is that the overnight process takes so long I don't then want to spend additional time normalising the last table into something usable.

Can anyone offer any insight or ideas into the best way to restructure or optimise the database efficiently?

Edit:

All the databases are MS SQL Server 2008 R2

Thanks in advance

CM

答:
Hard to say without knowing more details, so here's some suggestions:

Does the extract from other sources HAVE to happen overnight? Is it possible that at least some of the data will be consistent enough at various points throughout the day? That might let you do several smaller data pulls 3 or 4 times a day on a schedule, leaving more free time in the overnight to normalize. Or, you could even start the process of building the OLAP cube in little bits, assuming it's possible with this data. Or go even further and make it a near-continuous process throughout the day.

Is the big table over-indexed? Maybe too many indices is slowing down inserts.

Would partial normalization be OK? Maybe only normalize one or two important columns.

Would it be OK if the OLAP cube weren't available until later than it currently is?

Can you get a budget to buy better (for this job) hardware, assuming there's a bottleneck that can be resolved by hardware upgrades?


问:
Is programmatically clearing the browser's history an accepted best practice?
A colleague is pushing clearing the client browser's history (i.e. back-button) when they logout of our web application. Is this considered a good practice? It strikes me as a really intrusive. If only conditionally, what conditions need to be in place for this practice to be useful?</p>

答:
Is it critical for the app's functionality? Remember that it can probably be blocked by the user disabling the script, so keep in mind that your app should still be able to handle situations where the history clearing script does NOT run.

Personally, I'd find this sort of behaviour nasty and excessively invasive and intrusive. Who do you think you are, trying to mess with my browser's history on my local machine? Don't do it.

Thinking more carefully, this might be acceptable if the web application is intended to be accessed from browsers running in "kiosk" mode, like in those computers in malls used for product promotions. I think it would be OK here because these aren't the "user's" personal comptuer, but a kiosk that random members of the public will use. But in that case, I'd think that such a requirement arose because of bad design elsewhere in the application. I am really having a hard time thinking of a valid business reason (that doesn't sound suspicious or malicious) that a web application needs to get rid of the browser history, but I'm open to suggestions...


答:
No. I've never heard of anyone doing that and I would prefer if a web app didn't mess with my browsing history. Your coworker must be using a trick like the asker in this question is using to clear the history, but it is IE only.

It sounds like the accepted best practice is to use window.location.replace to keep a page from going into history. See A, B.

Title: Is programmatically clearing the browser's history an accepted best practice?

Body: A colleague is pushing clearing the client browser's history (i.e. back-button) when they logout of our web application. Is this considered a good practice? It strikes me as a really intrusive. If only conditionally, what conditions need to be in place for this practice to be useful?</p>


答:
Clearing the browser history when the user leaves your application seems to be a pretty good example of security through obscurity.  This form is particularly invasive, as you're counteracting an expected function on the users computer.

You should not rely on this technique to secure your application.  It should not mater if a malicious knows the URLs that a user visited on your application.


问:
Are there any actual case studies on rewrites of software success/failure rates?
I've seen multiple posts about rewrites of applications being bad, people's experiences about it here on Programmers, and an article I've ready by Joel Spolsky on the subject, but no hard evidence or case studies.  Other than the two examples Joel gave and some other posts here, what do you do with a bad codebase and how do you decide what to do with it based on real studies?

For the case in point, there are two clients I know of that both have old legacy code.  They keep limping along with it because as one of them found out, a rewrite was a disaster, it was expensive and didn't really work to improve the code much.  That customer has some very complicated business logic as the rewriters quickly found out.

In both cases, these are mission critical applications that brings in a lot of revenue for the company.  The one that attempted the rewrite felt that they would hit a brick wall if the legacy software didn't get upgraded at some point in the future.  To me, that kind of risk warrants research and analysis to ensure a successful path.

Have there been actual case studies that have investigated this?  I wouldn't want to attempt a major rewrite without knowing some best practices, pitfalls, and successes based on actual studies.

Aftermath:  okay, after more searching, I did find three interesting articles on case studies:

Rewrite or Reuse.  They did a study on a Cobol app that was converted to Java.

The other was on Software Reuse: Developers Experiences and Perceptions.

Reuse or Rewrite Another study on costs of maintenance versus a rewrite.

I recently found another article on the subject: The Great Rewrite. There the author seems to hit on some of the major issues.  Along with this was the idea of prototyping by using the proposed new technology stack and measuring how quick the devs picked it up.  This was all as a prelude to a rewrite, which I thought was a great idea!

答:
I can't take credit for these great comments, but they were never put into an answer by the original author, so I'm marking it community wiki.

I don't know about case studies, but I think the standard answer for an approach is probably "add unit tests and refactor."

It strikes me as quite possibly the lowest risk approach available. Of course, I don't know enough details to be certain it's the right approach, but based on what I know so far, I don't know of much that's particularly likely to be a lot better.

If they do unit tests (properly -- truly capturing all the requirements) it's difficult to come up with a way for refactoring to result in a true disaster. About the worst that can happen is that you don't make as much progress as fast as you'd like. At the same time, if their code base is in as bad of shape as the question implies, chances are pretty good that serious effort will be needed to enable progress, regardless of the route taken.

I would assume that rewrite projects aren't significantly different in failure rates than projects in general, and would refer to latest CHAOS report for best info.


问:
Google Analytics with multiple environments
We are planning to use Google Analytics in our organization and I am in charge for setting it up. I was wondering how to deal with multiple environments. We sure do not want to collect data during development and QA (or maybe collect data to a different analytics account), but we want to when the site goes to production (obviously).

How do you deal with multiple environments and Google Analytics ?

Do you setup multiple accounts for Google Analytics and use either one depending on the environement ?

We're using ASP.NET 2.0, if that matters.

答:
Here is what I finally did (I did not like the idea of embedding .Net code inside javascript) :

I setup two Google Analytics account, one for development/testing and one for production

Create a page which returns the Google Analytics account id (from the web.config) as its body.

On $(document).ready, make an ajax call to the page

On Ajax complete, call the google analytics script with the response (which contains the account id)

So depending on the environment I deploy to, I use either version of the web.config file containing the right account id.

It's a bit more complicated, but it does not involve generating javascript from .NET, and it's been very simple with jQuery.

Here is the final code :

```<code>var setupGoogleAnalytics = function (clientNumber) {
    var gaq = gaq || [];
    gaq.push(['_setAccount', clientNumber]);
    gaq.push(['_setDomainName', 'none']);
    gaq.push(['_trackPageview']);

    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
}

$(document).ready(function(){
    $.ajax({
        url: "GetGAClientNumber.aspx",
        success: function(data) {
            setupGoogleAnalytics(data);
        }
    });
});
</code>```

And it works perfectly !

Body:
Here is what I finally did (I did not like the idea of embedding .Net code inside javascript) :

I setup two Google Analytics account, one for development/testing and one for production

Create a page which returns the Google Analytics account id (from the web.config) as its body.

On $(document).ready, make an ajax call to the page

On Ajax complete, call the google analytics script with the response (which contains the account id)

So depending on the environment I deploy to, I use either version of the web.config file containing the right account id.

It's a bit more complicated, but it does not involve generating javascript from .NET, and it's been very simple with jQuery.

Here is the final code :

var setupGoogleAnalytics = function (clientNumber) {
    var gaq = gaq || [];
    gaq.push(['_setAccount', clientNumber]);
    gaq.push(['_setDomainName', 'none']);
    gaq.push(['_trackPageview']);

    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
}

$(document).ready(function(){
    $.ajax({
        url: "GetGAClientNumber.aspx",
        success: function(data) {
            setupGoogleAnalytics(data);
        }
    });
});

And it works perfectly !


问:
When does pair programming work? When to avoid it?
Rather than slavishly pair program all the time, we use pair programming selectively on our team. I think it works best in the following circumstances:

Ramping up brand new team members on a project (instead of letting them wade through documentation or code on their own).

Having junior and senior people work together (helps to show some of the skills and tricks of the more experienced developers, plus it allows the old dogs to learn new tricks sometimes).

When someone is trying to track down a defect, it often helps to pair with a fresh set of eyes.

When to use pair program and why?

When to avoid pair programming? Why?

答:
Working on anything of non-trivial complexity tends to be a good candidate for pair programming so that multiple people understand the code rather than just one developer knowing a portion of the code base.  Another case is where someone is wanting to transfer some skills.  An example here may be having someone that is really good at unit tests pair with someone not quite as familiar with the concept and thus helps to get an initial habit on something.

As for where to avoid pair programming, grunt work tasks that are straightforward where it would be better to divide the work into two groups and let each developer do some of the work separately to get the work done.  Some tasks can just require a fair bit of typing but aren't so big that it is worthwhile to spend a few hours trying to find a better way to do it as it could be done if each developer takes a brute force approach for a few hours.


答:
Feature is a holdover from classic ASP days when everything leaked memory and you had to do it. Most folks had a scheduled restart on the web server overnight. IIS6 automated this with app pool shutdowns every 1740 minutes (or if app is idle for 20 minutes). IIS7 continued the tradition.

Advice I got from microsoft back in those days was that this was unnecessary unless you had a legacy app with a known memory leak. So if you were running purely managed code you would be OK.


答:
For OLAP databases, normalization is often not the best approach - this is completely different from classical OLTP databases. The structure of your tables should be optimized for the queries you are going to run. I recommend the Wikipedia articles about star schema or snowflake schema, those are patterns for a good OLAP database design.

Here is a book about the topic I can recommend:

http://www.amazon.com/The-Data-Warehouse-Toolkit-Dimensional/dp/0471153370

Something you did not write (but really ask yourself) is why you actually want to restructure the system. Just because it is denormalized and you think this is not "best practice"? Or do you suffer from real performance or storage problems? If it is only the first reason, you should first read something more about good OLAP db design before changing the system.

Title: Designing a large database with multiple sources

Body:
I have been tasked with redesigning, or at worst optimising the structure of a database for a data warehouse.

Currently, the database has 4 other source databases (which is due to expand to X number of others), all of which have their own data structures, naming conventions etc.  At the moment an overnight SSIS package pulls the data from the various source and then for each source coverts the data into a standardised, usable format.  These tables are then appended to each other creating a 60m row, 40 column beast!.

This table is then used in a variety of ways from an OLAP cube to a web front end.

The structure has been in place for a very long time, and the work I have been able to prove the advantages of normalisation, and this is the way I would like to go.  The problem for me is that the overnight process takes so long I don't then want to spend additional time normalising the last table into something usable.

Can anyone offer any insight or ideas into the best way to restructure or optimise the database efficiently?

Edit:

All the databases are MS SQL Server 2008 R2

Thanks in advance

CM


问:
"Invalid operation" status code in a HATEOAS REST API
In a HATEOAS API links are returned which represent possible state transitions. A conforming client should just be retrieving and following those links, but if a non-conforming client is constructing URIs rather than following the supplied links what would be the most appropriate status code/response to return?

400 would work, together with some information in the response body - this is what we're currently doing

403 I guess would be wrong, as it implies that the request could never work - but potentially the link may be available in the future

404 sounds plausible - at this point in time the resource doesn't exist

What do people think? I know that conditional requests can handle requests based on stale responses (resulting in e.g. 412s), but this is a slightly different situation.

Update:

OK, I see now that the correct response for these types of invalid operations would be a 404. How about where the syntax of a request is correct, it's going to a valid resource but it's somehow violating a business rule. Here's a couple of contrived examples:

Let's say the client can provide two numbers which must be within 20% of each other, but otherwise could be any number.

Let's say the client can provide a number which goes through some calculation the result of which indicates that the original number provided was incorrect.

Is 400 is the correct response for these?

答:
These codes follow the standard set by the HTTP rfc.

According to the Status Code Definitions section:

400 Bad Request

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.

403 Forbidden

The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

404 Not Found

The server has not found anything matching the Request-URI. No indication is given of whether the condition is temporary or permanent. The 410 (Gone) status code SHOULD be used if the server knows, through some internally configurable mechanism, that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

In your case, if the URI constructed doesn't lead anywhere I think it would be appropriate to return 404.

And if the URI is valid but the data passed (like a json doc) is broken, then you should return 400.

EDIT:

Answering the OP comment: I think the system should return 400 when the syntax and also the meaning of the data are broken.


答:
If there's a chance that the links will exist in the future, both 400 Bad Request and 403 Forbidden would be incorrect: their relevant sections in RFC 2616 both have instructions that the client SHOULD NOT repeat the request. The difference between the two is that, in the case of 400 Bad Request, the server did not understand what the client was trying to do, whereas with 403 Forbidden the server did understand the request, but is refusing to fulfill it (ever).

If you want to indicate to the client that the request was understood, but you just won't handle it at the time of the request (but make no claim about handling it in the future), 404 Not Found would be the most appropriate response to send.

If you want to indicate to the client that request was understood, but the request does not follow predetermined rules or guidelines (like your example of a client not providing two numbers within 20% of each other), you want to use 409 Conflict, which is intended to indicate to the client that the request needs to be fixed in order to be handled properly.

You should separate good faith non-conformity from bad-faith non-conformity, though. If you're trying to protect your application from requests by a bad actor, they're almost certainly not going to care what response code you send: they'll just keep constructing URLs until they hit something they like.

Title: "Invalid operation" status code in a HATEOAS REST API

Body:
In a HATEOAS API links are returned which represent possible state transitions. A conforming client should just be retrieving and following those links, but if a non-conforming client is constructing URIs rather than following the supplied links what would be the most appropriate status code/response to return?

400 would work, together with some information in the response body - this is what we're currently doing

403 I guess would be wrong, as it implies that the request could never work - but potentially the link may be available in the future

404 sounds plausible - at this point in time the resource doesn't exist

What do people think? I know that conditional requests can handle requests based on stale responses (resulting in e.g. 412s), but this is a slightly different situation.

Update:

OK, I see now that the correct response for these types of invalid operations would be a 404. How about where the syntax of a request is correct, it's going to a valid resource but it's somehow violating a business rule. Here's a couple of contrived examples:

Let's say the client can provide two numbers which must be within 20% of each other, but otherwise could be any number.

Let's say the client can provide a number which goes through some calculation the result of which indicates that the original number provided was incorrect.

Is 400 is the correct response for these?


答:
Since your time restriction is only on connecting to the production databases, pull the data from them into some sort of staging area with as little processing as possible.

Then you can start the process of normalizing the data as needed.

There are some drawbacks here:

Storage space required for the staging area (of course you can dump this when finished).

This may take longer, so consumers of this database have to wait longer.


答: Indeed, this is purely to clean up leaked resources that (might) be present.  The 1740 minutes is not the only triggering event either.  It also happens after a specific maximum number of requests or after exceeding a specific amount of worker process memory.  It is pretty well documented in the MSDN library.  Unfortunately, this policy breaks things like session state and statics such as singletons.  Your app will need to be robust enough to re-authenticate users and/or re-initialize singletons as needed to avoid disturbing your user's experience.  It forced us to manage authenticated sessions in the database rather than in the ASP.NET session.  Otherwise, our users were booted back to our login page whenever the server recycled due to one of these triggers.</p>


答:
Generally speaking, open a new bug.

However, if you are allowed to do some investigation first, I would check your history in source code.

If you work in a team environment, somebody may have old code on their system (i.e., they didn't do a  Get Latest after the original fix was checked in), made changes, and then checked in without doing a diff.  Bad practice, sure, but it happens "all the time."

Looking at the history of the file(s) where the bug was fixed will quickly confirm or eliminate that as a possibility.

Title: Bug reopen vs. new


问:
How to refactor a Python “god class”?

Problem

I’m working on a Python project whose main class is a bit “God Object”.  There are so friggin’ many attributes and methods!

I want to refactor the class.

So Far…

For the first step, I want to do something relatively simple; but when I tried the most straightforward approach, it broke some tests and existing examples.

Basically, the class has a loooong list of attributes—but I can clearly look over them and think, “These 5 attributes are related…These 8 are also related…and then there’s the rest.”

getattr

I basically just wanted to group the related attributes into a dict-like helper class.  I had a feeling __getattr__ would be ideal for the job.  So I moved the attributes to a separate class, and, sure enough, __getattr__ worked its magic perfectly well…

At first.

But then I tried running one of the examples.  The example subclass tries to set one of these attributes directly (at the class level).  But since the attribute was no longer “physically located” in the parent class, I got an error saying that the attribute did not exist.

@property

I then read up about the @property decorator.  But then I also read that it creates problems for subclasses that want to do self.x = blah when x is a property of the parent class.

Desired

Have all client code continue to work using self.whatever, even if the parent’s whatever property is not “physically located” in the class (or instance) itself.

Group related attributes into dict-like containers.

Reduce the extreme noisiness of the code in the main class.

For example, I don’t simply want to change this:

```<code>larry = 2
curly = 'abcd'
moe   = self.doh()
</code>```

Into this:

```<code>larry = something_else('larry')
curly = something_else('curly')
moe   = yet_another_thing.moe()
</code>```

…because that’s still noisy.  Although that successfully makes a simply attribute into something that can manage the data, the original had 3 variables and the tweaked version still has 3 variables.

However, I would be fine with something like this:

```<code>stooges = Stooges()
</code>```

And if a lookup for self.larry fails, something would check stooges and see if larry is there.  (But it must also work if a subclass tries to do larry = 'blah' at the class level.)

Summary

Want to replace related groups of attributes in a parent class with a single attribute that stores all the data elsewhere

Want to work with existing client code that uses (e.g.) larry = 'blah' at the class level

Want to continue to allow subclasses to extend, override, and modify these refactored attributes without knowing anything has changed

Is this possible?  Or am I barking up the wrong tree?

答:
Having written and then refactored a python "God object", I sympathize. What I did is break the original object down into sub sections based upon methods. For instance, the original looked like this pseudo code:

```<code>method A():
    self.bla += 1

method B():
    self.bla += 1

do stuff():
    self.bla = 1
    method A()
    method B()
    print self.bla
</code>```

The stuff method is a self contained "unit" of work. I migrated it out to a new class which the original instantiates. This pulled out the necessary properties as well. Some were used only by the sub class and could move straight across. Others were shared, and got moved into a shared class.

The "God object" creates a new copy of the shared class at start up, and each of the new sub classes accepts a pointer as part of their init method. For example, here's a stripped version of the mailer:

```<code>#!/usr/bin/env python
# -*- coding: ascii -*-
'''Functions for emailing with dirMon.'''

from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
import os
import smtplib
import datetime
import logging

class mailer:
    def __init__(self,SERVER="mail.server.com",FROM="support@server.com"):
        self.server = SERVER
        self.send_from = FROM
        self.logger = logging.getLogger('dirMon.mailer')

    def send_mail(self, send_to, subject, text, files=[]):
        assert type(send_to)==list
        assert type(files)==list
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug(' '.join(("Sending email to:",' '.join(send_to))))
            self.logger.debug(' '.join(("Subject:",subject)))
            self.logger.debug(' '.join(("Text:",text)))
            self.logger.debug(' '.join(("Files:",' '.join(files))))
        msg = MIMEMultipart()
        msg['From'] = self.send_from
        msg['To'] = COMMASPACE.join(send_to)
        msg['Date'] = formatdate(localtime=True)
        msg['Subject'] = subject
        msg.attach( MIMEText(text) )
        for f in files:
            part = MIMEBase('application', "octet-stream")
            part.set_payload( open(f,"rb").read() )
            Encoders.encode_base64(part)
            part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(f))
            msg.attach(part)
        smtp = smtplib.SMTP(self.server)
        mydict = smtp.sendmail(self.send_from, send_to, msg.as_string())
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug("Email Successfully Sent!")
        smtp.close()
        return mydict
</code>```

It is created once and shared between the different classes that need mailing capabilities.

So for you, create a class larry with the properties and methods you need. Everywhere the client says larry = blah replace it with larryObj.larry = blah. This migrates things to sub projects without breaking the current interface.

The only other thing to do is look for "units of work". If you were going to turn part of the "God Object" into it's own method, do so. But, put the method outside it. This forces you to create an interface between the components.

Laying that groundwork allows everything else to follow it. For example, a piece of the helper object demonstrating how it interfaces with the mailer:

```<code>#!/usr/bin/env python
'''This module holds a class to spawn various subprocesses'''
import logging, os, subprocess, time, dateAdditionLib, datetime, re

class spawner:
    def __init__(self, mailer):
        self.logger = logging.getLogger('dirMon.spawner')
        self.myMailer = mailer
</code>```

Concentrate on the smallest individual unit of work possible, and move it out. This is easier to do, and lets you play with the setup quickly. Don't look at properties for moving stuff, they are ancillary to the tasks being done with them in most cases. Whatever is left over after you have dealt with the methods should probably stay in the original object, since it is part of the shared state.

But, the new objects should now accept the properties they need as init variables, not touching the caller objects property. They then return any necessary values, which the caller can use to update the shared properties as necessary. This helps to decouple the objects and makes for a more robust system.

Title: How to refactor a Python “god class”?

Body:
Having written and then refactored a python "God object", I sympathize. What I did is break the original object down into sub sections based upon methods. For instance, the original looked like this pseudo code:

method A():
    self.bla += 1

method B():
    self.bla += 1

do stuff():
    self.bla = 1
    method A()
    method B()
    print self.bla

The stuff method is a self contained "unit" of work. I migrated it out to a new class which the original instantiates. This pulled out the necessary properties as well. Some were used only by the sub class and could move straight across. Others were shared, and got moved into a shared class.

The "God object" creates a new copy of the shared class at start up, and each of the new sub classes accepts a pointer as part of their init method. For example, here's a stripped version of the mailer:

#!/usr/bin/env python
# -*- coding: ascii -*-
'''Functions for emailing with dirMon.'''

from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
import os
import smtplib
import datetime
import logging

class mailer:
    def __init__(self,SERVER="mail.server.com",FROM="support@server.com"):
        self.server = SERVER
        self.send_from = FROM
        self.logger = logging.getLogger('dirMon.mailer')

    def send_mail(self, send_to, subject, text, files=[]):
        assert type(send_to)==list
        assert type(files)==list
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug(' '.join(("Sending email to:",' '.join(send_to))))
            self.logger.debug(' '.join(("Subject:",subject)))
            self.logger.debug(' '.join(("Text:",text)))
            self.logger.debug(' '.join(("Files:",' '.join(files))))
        msg = MIMEMultipart()
        msg['From'] = self.send_from
        msg['To'] = COMMASPACE.join(send_to)
        msg['Date'] = formatdate(localtime=True)
        msg['Subject'] = subject
        msg.attach( MIMEText(text) )
        for f in files:
            part = MIMEBase('application', "octet-stream")
            part.set_payload( open(f,"rb").read() )
            Encoders.encode_base64(part)
            part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(f))
            msg.attach(part)
        smtp = smtplib.SMTP(self.server)
        mydict = smtp.sendmail(self.send_from, send_to, msg.as_string())
        if self.logger.isEnabledFor(logging.DEBUG):
            self.logger.debug("Email Successfully Sent!")
        smtp.close()
        return mydict

It is created once and shared between the different classes that need mailing capabilities.

So for you, create a class larry with the properties and methods you need. Everywhere the client says larry = blah replace it with larryObj.larry = blah. This migrates things to sub projects without breaking the current interface.

The only other thing to do is look for "units of work". If you were going to turn part of the "God Object" into it's own method, do so. But, put the method outside it. This forces you to create an interface between the components.

Laying that groundwork allows everything else to follow it. For example, a piece of the helper object demonstrating how it interfaces with the mailer:

#!/usr/bin/env python
'''This module holds a class to spawn various subprocesses'''
import logging, os, subprocess, time, dateAdditionLib, datetime, re

class spawner:
    def __init__(self, mailer):
        self.logger = logging.getLogger('dirMon.spawner')
        self.myMailer = mailer

Concentrate on the smallest individual unit of work possible, and move it out. This is easier to do, and lets you play with the setup quickly. Don't look at properties for moving stuff, they are ancillary to the tasks being done with them in most cases. Whatever is left over after you have dealt with the methods should probably stay in the original object, since it is part of the shared state.

But, the new objects should now accept the properties they need as init variables, not touching the caller objects property. They then return any necessary values, which the caller can use to update the shared properties as necessary. This helps to decouple the objects and makes for a more robust system.


问:
Design patterns - do you use them?
Being an IT student, I was recently given some overview about design patterns by one of our teachers. I understood what they are for, but some aspects still keep bugging me.

Are they really used by the majority of programmers?

Speaking of experience, I've had some troubles while programming, things I could not solve for a while, but Google and some hours of research solved my problem. If somewhere in the web I find a way to solve my problem, is this a design pattern? Am I using it?

And also, do you (programmers) find yourself looking for patterns (where am I supposed to look, by the way?) when you start the development? If so, this is certainly a habit that I must start to embrace.

UPDATE: I think when I ask if programmers do use them, I'm asking if when you have a problem to solve you think "Oh, I should use that pattern".

答: You will learn patterns as you start working with existing projects. There are so many patterns out there for you to learn, and it is not worth the time to master all of them as it depends on what project you are working on. Whenever you run into one, learn about it to see how it is used.</p>

Title: Design patterns - do you use them?


问:
How do you dive into large code bases?
What tools and techniques do you use for exploring and learning an unknown code base?

I am thinking of tools like grep, ctags, unit-tests, functional test, class-diagram generators, call graphs, code metrics like sloccount, and so on. I'd be interested in your experiences, the helpers you used or wrote yourself and the size of the code base with which you worked.

I realize that becoming acquainted with a code base is a process that happens over time, and familiarity can mean anything from "I'm able to summarize the code" to "I can refactor and shrink it to 30% of the size". But how to even begin?

答:
Here's my short list:

If possible, have someone give a high-level view of the code.  What patterns were considered, what kinds of conventions could I expect to see, etc.  This may have a few rounds to it as in the beginning I'd get one story that as I get more familiar with the code, I may have new questions to ask as I work through the onion of the pre-existing project.

Run the code and see what does the system(s) look like.  Granted it may have more than a few bugs, but this can be useful for getting an idea of what does it do.  This isn't about changing the code, but rather just seeing how does this run.  How do various pieces fit together to be a system overall?

Look for tests and other indicators of basic documentation that may assist in building an internal mental model of the code.  This is where I'd probably suggest at least a few days unless there is extremely little documentation and tests of course.

How well do I know the languages and frameworks used in this project?  The importance here is the difference between looking at some things and going, "Yes, seen that a dozen times before and know it fairly well," and "What in the world is being attempted here?  Who thought this was a good idea?" kind of questions that while I wouldn't say them out loud, I would be thinking them especially if I'm looking at legacy code that may be quite fragile and the people that did write it are either unavailable or just don't remember why things were done the way they were.  For new areas, it may be worthwhile to spend some extra time getting to know what is the structure and what patterns can I find in this code.

Last but not least: Know the expectations of those running the project in terms of what you are supposed to do at each point in time, given the following few ideas of what may be expected:

Are you putting in new features?

Are you fixing bugs?

Are you refactoring code?  Are the standards new to you or are they very familiar?

Are you supposed to be just familiarizing yourself with the code base?

Title: How do you dive into large code bases?


答:
It takes time

Don't feel too rushed while trying to understand a legacy codebase, especially if it's using technologies/languages/frameworks you're not familiar with.  It's just an unavoidable learning curve that takes some time.

One approach is to go back and forth between the code and tutorials on the related technologies.  You read/watch the tutorial, then go look at the code to see how your predecessors did it, noting any similarities and differences, taking notes, and asking questions to any existing developers.

"Why did you do this part this way"

"I noticed most people online are doing it this way, and you all did it another way.  Why is this?"

"What made you all choose technology X over technology Y?"

The answers to these questions will help you understand the history of the project and the reasoning behind design and implementation decisions.

Eventually, you'll feel familiar enough with it that you can start adding/fixing things.  If it all seems confusing or it seems like there's too much "magic" going on, you haven't spent enough time looking it over, digesting it, and diagramming it.  Creating diagrams (sequence diagrams, process flow diagrams, etc.) is a great way for you to understand a complex process, plus they will help the "next guy".

Title: How do you dive into large code bases?

Body:
What tools and techniques do you use for exploring and learning an unknown code base?

I am thinking of tools like grep, ctags, unit-tests, functional test, class-diagram generators, call graphs, code metrics like sloccount, and so on. I'd be interested in your experiences, the helpers you used or wrote yourself and the size of the code base with which you worked.

I realize that becoming acquainted with a code base is a process that happens over time, and familiarity can mean anything from "I'm able to summarize the code" to "I can refactor and shrink it to 30% of the size". But how to even begin?


问:
Does giving a developer a slower development machine result in faster/more efficient code?
Suppose I give my developers a screaming fast machine.  WPF-based VS2010 loads very quickly.  The developer then creates a WPF or WPF/e application that runs fine on his box, but much slower in the real world.

This question has two parts...

1) If I give a developer a slower machine, does that mean that the resulting code may be faster or more efficient?

2) What can I do to give my developers a fast IDE experience, while giving 'typical' runtime experiences?

Update:

For the record, I'm preparing my even-handed response to management. This isn't my idea, and you folks are helping me correct the misguided requests of my client.  Thanks for giving me more ammunition, and references to where and when to approach this.  I've +1'ed valid use cases such as:

- specific server side programming optimizations

- test labs

- the possibly buying a better server instead of top of the line graphics cards

答:
In reading the question, and the answers, I'm kind of stunned by the vehemence of the NO case.

I've worked in software development for 25 years now, and I can say without any hesitation that programmers need a bunch of things to develop good code:

A REASONABLE development environment. Not dinosaur. Neither does it need to be bleeding edge. Good enough not to be frustrating.

A good specification (how much is done with NO written specification?)

Good and supportive management.

A sensible development schedule.

A good understanding of the users AND THE ENVIRONMENT the users will have.

Further, on this last point, developers need to be in the mindset of what the users will use. If the users have supercomputers and are doing atom-splitting simulations or something where performance costs a lot of money, and the calculations run for many hours, then thinking performance counts.

If the users have 286 steam powered laptops then developing and having developers do their development test on the latest 47 GHz Core i9000 is going to lead to some problems.

Those who say "give developers the best and TEST it" are partly right but this has a big MENTAL problem for the developers. They have no appreciation of the user experience until its too late - when testing fails.

When testing fails - architectures have been committed to, management have had promises made, lots of money has been spent, and then it turns into a disaster.

Developers need to think like, understand, and be in the zone of the user experience from day 1.

Those who cry "oh no it does not work like that" are talking out their whatsit. I've seen this happen, many times. The developers usual response is one of "well tell the CUSTOMERS to buy a better computer", which is effectively blaming the customer. Not good enough.

So this means that you have several problems:

Keep the devs happy and piss of the management, increase the chances of the project failing.

Use slower machines for development, with the risk of upsetting the devs, but keeping them focussed on what really matters.

Put 2 machines on the devs desk AND FORCE THEM TO TEST ON THE CLUNKER (which they wont do because it is beneath contempt.... but at least its very clear then if there are performance problems in test).

Remember batch systems and punch cards? People waited an hour or a day for turnaround. Stuff got done.

Remember old unix systems with 5 MHz processors? Things got done.

Techo-geeks love chasing the bleeding edge. This encourages tinkering, not thinking. Something I've had arguments about with more juniour developers over the years.... when I urge them to get fingers away from the keyboard and spend more time reading the code and thinking.

In development of code, there is no substitute for thinking.

In this case, my feeling is - figure out WHAT REALLY MATTERS. Success of the project? Is this a company making / killing exercise? If it is, you can't afford to fail. You can't afford to blow money on things that fail in test. Because test is too late in the development cycle, the impacts of failure are found too late.

[A bug found in test costs about 10x as much to fix as a bug found by a dev during development.

And a bug found in test costs about 100x as much to fix as that bug being designed out during the architectural design phase.]

If this is not a deal breaker, and you have time and money to burn, then use the bleeding edge development environment, and suffer the hell of test failures. Otherwise, find another way. Lower end h/w, or 2 machines on each desk.


答:
The first thing you need to do when learning a new code base is learn about what it is supposed to do, how its used, and how to use it. Then start looking at architectural documentation to learn how the code is laid out, also look at how the database at this point. At the same time you are learning the architecture its a good time to review any process flows or use case documents. then start diving in and reading code after you have an understanding of the big picture, but only code related to any work you may be doing on this code, don't just try to read all the code. Its more important to know where the code is to do X than exactly how X is done, the code is always there to tell you how if you can find it.

I find that just trying to jump in and read code without a goal beyond learning the code is generally unproductive, trying to do small changes yourself or reviewing the code of someone else's changes is a far more productive use of your time.

Title: How do you dive into large code bases?


答:
I think one of the most important things is to take a simple feature, choose the simplest you can think of, and implement it. If there is a maintained wish list use that or else talk to someone familiar with the code base and get them to suggest a feature. Usually I would expect this to be a change with 5~20 LOC. The important point is not that you are adding a very fancy feature but that you are working (or rather grappling :) ) with the code base and going through the whole workflow. You would have to

Read code to understand the component that you are modifying

Change code and understand how that impacts the surrounding system.

Test the change and thus identify how the components interact with each other

Write the test case and hopefully break one or two test cases so you get to fix them and understand the invariants of the system.

Build the thing or see the CI build it and then ship it

The list goes on but point is that a mini project like this takes you through all of the items on your check list for getting acquainted with a system and also results in a productive change being done.

Title: How do you dive into large code bases?


答:
Are they really used by the majority of programmers?

I would guess yes.  In ADO.Net there is a DataAdapter class to give a simple example though depending on what area you want to specialize the patterns may vary.

Speaking of experience, I've had some troubles while programming,
  things I could not solve for a while, but google and some hours of
  research solved my problem. If somewhere in the web I find a way to
  solve my problem, is this a design pattern? Am I using it?

No, that isn't a design pattern to my mind.  A design pattern tends to have some arrangement of classes and methods that define the recipe of a pattern.

I'd prefer to think of what you did there as a common practice.  Beware of copy and paste coding though.

And also, do you (programmers) find yourself looking for patterns
  (where am I supposed to look btw?) when you start the development? If
  so, this is certainly a habit that I must start to embrace.

Sometimes as I see the same code over and over, I may find a way to refactor the code into a pattern or if I remember a solution to a similar problem involving a pattern I'll take it out and use it.  Head First Design Patterns has more than a few patterns in it while Refactoring would be a suggestion for coding practices that may lead one to find various patterns.  If you want another possible starting point, look at Microsoft's Patterns and Practices.

Title: Design patterns - do you use them?

Body:
Being an IT student, I was recently given some overview about design patterns by one of our teachers. I understood what they are for, but some aspects still keep bugging me.

Are they really used by the majority of programmers?

Speaking of experience, I've had some troubles while programming, things I could not solve for a while, but Google and some hours of research solved my problem. If somewhere in the web I find a way to solve my problem, is this a design pattern? Am I using it?

And also, do you (programmers) find yourself looking for patterns (where am I supposed to look, by the way?) when you start the development? If so, this is certainly a habit that I must start to embrace.

UPDATE: I think when I ask if programmers do use them, I'm asking if when you have a problem to solve you think "Oh, I should use that pattern".


答:
If a code base is large, then focus your attention to the parts that are currently working on.  Otherwise you will feel overwhelmed and possibly your head may explode.  I think some high level overview is helpful (if it is available), but chances are you will be spending lots of time in the debugger for following program flow.  It is a good idea to get an overview of the application and seeing it used, so you can understand how/what/why the code is being used for.

I usually run some sort of code complexity tool on the code to tell me where the problem areas are.  Areas that score high are probably very hard to update.  For example, I ran into a function that scored 450 on the cyclomatic scale.  Sure enough, hundreds of IFs.  Very hard to maintain or change that.  So be prepared for the worst.

Also, don't be afraid to ask questions to existing developers, especially if they worked on the system.  Keep your internal thoughts to yourself and focus on solving the problems.  Avoid comments that may make the other developers becoming upset.  After all, it may be thier baby and nobody likes being told thier baby is ugly.

Take small steps, even the smallest code change can have big impact.

I find it is helpful to come up with program code flows so if I am making changes, I can do dependency searches to see which methods/functions call what.  Suppose I am changing method C.

If only 1 method/function calls C, then it is a pretty safe change.  If 100s of methods/functions call C, then it would be of greater impact.

Hopefully your code base is well architected, written, and maintained.  If so, it will take some time to understand it but eventually the tide will be turned.

If it is a big ball of mud, you may never understand (or want to understand) its inner workings.

Title: How do you dive into large code bases?


问:
How do you achieve a numeric versioning scheme with Git?
My organization is considering moving from SVN to Git. One argument against moving is as follows:

How do we do versioning?

We have an SDK distribution based on the NetBeans Platform. As the SVN revisions are simple numbers we can use them to extend the version numbers of our plugins and SDK builds. How do we handle this when we move to Git?

Possible solutions:

Using the build number from Hudson (Problem: you have to check Hudson to correlate that to an actual Git version)

Manually upping the version for nightly and stable (Problem: Learning curve, human error)

If someone else has encountered a similar problem and solved it, we'd love to hear how.

答:
Versions are identified hashing the SHA1 hashes of all the files in the stored directory tree at the time of checkin. This hash is stored alongside the hashes of the parent checkin(s) so that the full history can be read.

Take a look at  the process of using 'git-describe' by way of GIT-VERSION-GEN and how you can add this via your build process when you tag your release.

Here is a nice blog that gives an example of how to get what you want:

http://cd34.com/blog/programming/using-git-to-generate-an-automatic-version-number/


答:
Use tags to mark commits with version numbers:

```<code>git tag -a v2.5 -m 'Version 2.5'
</code>```

Push tags upstream—this is not done by default:

```<code>git push --tags
</code>```

Then use the describe command:

```<code>git describe --tags --long
</code>```

This gives you a string of the format:

```<code>v2.5-0-gdeadbee
^    ^ ^^
|    | ||
|    | |'-- SHA of HEAD (first seven chars)
|    | '-- "g" is for git
|    '---- number of commits since last tag
|
'--------- last tag
</code>```

Title: How do you achieve a numeric versioning scheme with Git?

Body:
My organization is considering moving from SVN to Git. One argument against moving is as follows:

How do we do versioning?

We have an SDK distribution based on the NetBeans Platform. As the SVN revisions are simple numbers we can use them to extend the version numbers of our plugins and SDK builds. How do we handle this when we move to Git?

Possible solutions:

Using the build number from Hudson (Problem: you have to check Hudson to correlate that to an actual Git version)

Manually upping the version for nightly and stable (Problem: Learning curve, human error)

If someone else has encountered a similar problem and solved it, we'd love to hear how.

Body:
Use tags to mark commits with version numbers:

git tag -a v2.5 -m 'Version 2.5'

Push tags upstream—this is not done by default:

git push --tags

Then use the describe command:

git describe --tags --long

This gives you a string of the format:

v2.5-0-gdeadbee
^    ^ ^^
|    | ||
|    | |'-- SHA of HEAD (first seven chars)
|    | '-- "g" is for git
|    '---- number of commits since last tag
|
'--------- last tag


问:
Functional or non-functional requirement?
I'm wondering about functional or non-functional requirements. I have found lot of different definitions for those terms and I can't assign some of my requirement to proper category.

I'm wondering about requirements that aren't connected with some action or have some additional conditions, for example:

On the list of selected devices, device can be repeated.

Database must contain at least 100 items

Currency of some value must be in USD dollar.

Device must have a name and power consumption value in Watts.

are those requirements functional or non-functional ?

答:
Just to add to some existing good answers that non-functional requirements are sometimes called "ilities" - qualities that the system needs to possess in additional to its plain functionality.  "ilities" include availability, usability, security, flexibility - and even more subjective aesthetics.

Some of these are very difficult to specify and assess. Nevertheless, they matter. If you are signing up to them contractually, then you will want to avoid the meaningless hand-wavy versions, e.g. "The system must be secure".  The problem with trying to nail such requirements down is that people tend to gravitate to the things that are easily measurable, rather than the things that matter (and the requirements may well be written by people who don't have any grounding in the relevant specialities). The end result is that you generally end up with systems that are neither secure, usable, nor flexible (availability isn't so difficult to specify and measure, though it still causes plenty of headaches).

There are cultural differences here between folks who deal with contracts and formal stuff, and folks who deal in more general analysis, architecture, research etc.  A vague hand-wavy requirement is still a requirement as far as the latter is concerned, because it expresses things that matter to the customer, even if they sympathize completely with the contractual folks that its not a useful contractual requirement until it has been explored in detail and thoroughly nailed down.

One final point - if you can't (yet) come up with an objective measure of an "ility", that doesn't mean the customer doesn't need it. Vague != unnecessary. However, it may mean we need to develop better ways to measure such things, elicit and refine non-functional requirements incrementally, or contract in ways (Agile etc) that can work without up-front objective measures for everything.


问:
How do you track bugs in your personal projects?
I'm trying to decide if I need to reassess my defect-tracking process for my home-grown projects.  For the last several years, I really just track defects using TODO tags in the code, and keeping track of them in a specific view (I use Eclipse, which has a decent tagging system).

Unfortunately, I'm starting to wonder if this system is unsustainable.  The defects I find are typically associated with a snippet of code I'm working on; bugs which are not immediately understood tend to be forgotten, or ignored.  I wrote an application for my wife which has had a severe defect for almost 9 months, and I keep forgetting to fix it.

What mechanism do you use to track defects in your personal projects?  Do you have a specific system, or a process for prioritizing and managing them?

答:
I've actually installed the Free MANTIS bugtracker system on my hosted web server (that I use for a blog and other things), and put all my defects in that.

In other words I run my stuff as though it were professional and paid for.

I find it helps keep a better mindset (closing off defects, etc) as well as being consistent (ish) with other practices commonly used in industry.

Use TODO notes in code, etc, as well - but only for notes top self like: "one day I gotta make this more efficient, the bubble sort hurts performance". Or for more immediate notes about where you got up to when you were hauled off for dinner :)


问:
Security in a private web service
I am developing a web site and a web service for a small on-line game. Technically, I'll be using Express (node.js) and MongoDB+Redis for the databases. This the structure I came up with:

One Express server that will server as the Web Service. This will connect to  the databases.

One Express server that will provide the web site. It will connect to the Web Service to retrieve and push the information.

iOS and Android application will be able to interact with the WebService.

Taking into account:

It is a small game. The information transferred is not critical.

There will NOT be third party applications. At least for the moment.

My concern is about which level of security I should use in each of the scenarios:

Security of the user playing through web browser

Security of the applications and the Web Server connecting to the WS.

I have take a look at the different options and:

OAuth and/or Https is too much for this scenario, isn't it?

Will be a good option to hash the user and password with MD5(or similar) and some salt?

I would like to get some directions and investigate by my own rather than getting a response like "you should you use this node.js module..."

Thanks in advance,

答:
I am facing a similar issue, what I am planning on is having each request signed after authentication, similar to OAuth, and a layout similar to the following.

```<code>{"header":{...},"content":{...}}</code>```

The header will contain an authentication token, and a signature hash of the content + a shared secret...

The hash for the content will be simple the content section including "{" through to "}" as JSON encoded, and sent... for cases of high-character values, they must be escaped via \u#### so that only 7-bit ascii values are present. (anything < u0032 with \x##, and greater than u0127 will be escaped with a \u####)  The resulting JSON string, with only 7-bit characters will be easy enough to have a consistant signature..

Keeping the header separate from the content, will ensure the content is properly signed, and reduces the encoding issues and complexity usually seen in OAuth implementations... since OAuth is very broad.  YMMV, but I hope this helps.

Title: Security in a private web service

Body:
I am facing a similar issue, what I am planning on is having each request signed after authentication, similar to OAuth, and a layout similar to the following.

{"header":{...},"content":{...}}

The header will contain an authentication token, and a signature hash of the content + a shared secret...

The hash for the content will be simple the content section including "{" through to "}" as JSON encoded, and sent... for cases of high-character values, they must be escaped via \u#### so that only 7-bit ascii values are present. (anything < u0032 with \x##, and greater than u0127 will be escaped with a \u####)  The resulting JSON string, with only 7-bit characters will be easy enough to have a consistant signature..

Keeping the header separate from the content, will ensure the content is properly signed, and reduces the encoding issues and complexity usually seen in OAuth implementations... since OAuth is very broad.  YMMV, but I hope this helps.


问:
As a software engineer, who should I be following on Twitter?
Who in the software engineering and software development fields uses Twitter to tweet about relevant happenings in the field?</p>

答:
I maintain a list of over 2300 StackOverflow (SO) twitter accounts sorted by reputation.

I mined the latest SO data dump for all users with twitter accounts, then calculated each user's top tags based on most votes, and finally sorted the lists by user reputation.

Here is a screenshot of what the SO list looks like:

I also mined the available Stack Exchange data dumps.  The list of members for this site will also appear on the next data dump.

Here is a subset that is applicable to programmers:

StackOverflow

Game Development

Webmasters


问:
How relevant are Brainbench scores when evaluating candidates?
I've seen many companies using certification services such as Brainbench when evaluating candidates. Most times they use it as a secondary screen prior to interview or as a validation to choose between candidates.

What is your experience with Brainbench scores? Did you try the tests yourself, and if so do you feel the score is meaningful enough to be used as part of a hiring process?

Difficult choice.

Consensus seems to be that BB cert are not very good as a certification.  The biggest argument was around the fact that some of the questions are too precise to form a good evaluation.  This view can probably be tempered somewhat but still, to hold someone's future solely on the results of this evaluation would be irresponsible.

That said, I still think it is possible to use them properly to gain additional objective knowledge on a candidate's level of expertise provided the test is done in a controlled environment ensuring that all taking it stand on equal footing.

Thus I went with the answer that best reflected this view keeping in mind that it is still just an hour long 50ish multiple choice question to evaluate skills and knowledge that take years to acquire.  To be taken with a grain of salt !

In short, the tests have value but whether or not they are worth the money is another debate.

Thanks all for your time.

答:
I have also took one Brainbench exam regarding J2EE.
As a first impression it appeared to me not much related to actual programming. This is like knowing everything about J2EE/Java which in reality is impossible and one cannot claim ever in life.

As per my experience a developer merely touches one aspect/side of any programming language like as my example, I have mainly worked on applications that have a front end in the face of HTML forms and using some framework example Struts/Hibernate I manipulate that information from/to some database.

I guess so far I have touched just one dimension/aspect/corner of java and I don't think that I ever would encompass all features of the ever growing Java.

Point is, the core concepts regarding language semantics and in particular OOP constructs should be very clear to any experienced developer.

If he/she knows what to implement where or which is needed where, then he can be a successful guy. No one has the memory to cover everything related to any language. In fact you just need to strengthen your foundation concepts. Once they are clear, rest is research and development (RnD) work.
It all boils down to the fact, how much effort you contribute towards problem resolution using an initial set of tools and then working your way gradually towards the more sophisticated ones.


答: Pro Git in section 7.2 "Git Attributes" in "Keyword" Expansion part contains a nice example of using smudge&clean filters for generating RCS-style keywords. You can use the same technique for embedding some-version-string into code, formatted and calculated according to your rules. You still can use git describe as a staring point, but you have the possibility to transform to any more appropriate form and get from v2.5-14-feebdaed, for example, clean 2.5.14


问:
The need for source control software - Team Foundation Server? or something different?
Recently,

Here at the company, more than one programmer was appointed in charge for a LightSwitch(C#) software development project and immediately there was a need of some sort of source control/sync.

We have never used Team Foundation Server but we'd gladly use it if it's worth it.

Is it the right tool to use for
synchronising code between
programmers?

Does it have the needed features?

Do you have any other tool in
mind? (I have used TortoiseSVN but
it was too simple and 'texty' if you
know what I mean, we need a
professional tool)

What other features does Team
Foundation has that we can use?

(if you did not understand any of the above please ask me to clarify further)

答:
"we need a professional tool"

oh dear. Let me tell you about the time we scrapped our perfectly working SVN system and replaced it with an 'enterprise' tool that came highly recommended by a consultant and the management. .. actually, no I can't bring myself to document the horror that we endured for the year it was in place before we scrapped it and reverted to the perfectly good SVN system we had before.

Martin Fowler did a quick survey amongst the technophiles reading his blog. The results are very interesting.

The good ones are the 'unprofessional' ones. These are the ones the techies like and are much more productive with. The crpa ones are the ones that have a lot of 'administration' and 'lockdown control' associated with them, stuff like management-inspired workflows. (oh, ok, we had workflows and rules in this 'tool', to the point where you had to get a team lead to create/authorise a new design part that you could then use to checkin your code against - you could not just checkin your code or it would get lost in the mass of requirement tasks that were also associated with design parts, the idea was that we could properly control releases... never happened, total disaster, and that was when all the TLs were in the office and available to create the damn tasks!)

You can do a lot better than the features in TFS by looking around, and many of the OSS tools out there are very very good. Pick the ones you like, I recommend SVN with Jenkins build server, and redmine for task tracking. You might like Jira or CruiseControl with Mercurial.


问:
How would you know if you've written readable and easily maintainable code?
How would one know if the code one has created is easily readable, understandable, and maintainable? Of course from the author's point of view, the code is readable and maintainable, because the author wrote it and edited it, to begin with. However, there must be an objective and quantifiable standard by which our profession can measure code.

These goals are met when one may do the following with the code without the expert advice of the original author:

It is possible to read the code and understand at a basic level the flow of logic.

It is possible to understand at a deeper level what the code is doing to include inputs, outputs, and algorithms.

Other developers can make meaningful changes to the original code such as bug fixes or refactoring.

One can write new code such as a class or module that leverages the original code.

How do we quantify or measure code quality so that we know it readable, understandable, and maintainable?

答:
One point I'd share is if the code is built in "modules," and when I say that I mean that you can change one thing in a module and easily have it working with the whole.  It eliminates effects between unrelated things.
Also:

Code is easy to reuse

Your code is flexible (this ties in with building in modules)

DRY - Don't Repeat Yourself

I highly recommend reading, The Pragmatic Programmer.


问:
Scaling Scrum within a group of 100s of programmers
Most Scrum teams lean toward 7-15 people ******, though it's not clear how to scale Scrum among 100s of people, or how the effectiveness of a given team might be compared to another team within the group; meaning beyond just breaking the group into Scrum teams of 7-15 people, it's unclear how efforts between the teams are managed, compared, etc.

Any suggestions related to either of these topics, or additional related topics that might be of more importance to account for in planning a large scale SCRUM grouping?

****** In reviewing research related to the suggested size of software development teams, which appears to be the basis for the suggested Scrum team size, I found what appears to be an error in the research which oddly appears to show that bigger teams (15+ ppl), not smaller teams (7 ppl) are better.

UPDATE, "Re: Scrum doesn't scale": Made huge amounts of progress personally researching the topic, but thought I'd respond to the general belief of some that Scrum doesn't scale by citing a quote from Succeeding with Agile by Mike Cohn :

Scrum Does Scale: You have to admire the intellectual honesty of the earliest agile authors. They were all very careful to say that
  agile methodolgies like Scrum were for small projects.  This
  conservatism wasn’t because agile or Scrum turned out to be unsuited
  for large projects but because they hadn’t used these processes on
  large projects and so were reluctant to advise their readers to do so.
  But, in the years since the Agile Manifesto and the books that came
  shortly before and after it, we have learned that the principles and
  practices of agile development can be scaled up and applied on large
  projects, albeit it with a considerable amount of overhead.
  Fortunately, if large organizations use the techniques described
  regarding the role of the product owner, working with a shared product
  backlog, being mindful of dependencies, coordinating work among teams,
  and cultivating communities of practice, they can successfully scale a
  Scrum project.

SOURCE: (ran across the book thanks to Ladislav Mrnka answer)

答:
Nearly all Agile methodologies are designed for small teams, they always talk about communication as a primary aspect of the method, and the larger a team gets the more difficult effective communication becomes.

Alistair Cockburn (one of the founding fathers of Agile) has a methodology called Crystal that has different aspects depending on team size. These guidelines differ slightly but still keep to Agile principles.

An alternative is AUP, which is 'part Agile' and more suited for large teams too.