答:
I don't have all the answers.  Hopefully I can shed some light on it.

To simplify my previous statements about .NET's threading models, just know that Parallel Library uses Tasks, and the default TaskScheduler for Tasks, uses the ThreadPool.  The higher you go in the hierarchy (ThreadPool is at the bottom), the more overhead you have when creating the items. That extra overhead certainly doesn't mean it's slower, but it's good to know that it's there.  Ultimately the performance of your algorithm in a multi-threaded environment comes down to its design.  What performs well sequentially may not perform as well in parallel.  There are too many factors involved to give you hard and fast rules, they change depending on what you're trying to do.  Since you're dealing with network requests, I'll try and give a small example.

Let me state that I am no expert with sockets, and I know next to nothing about Zeroc-Ice.  I do know about bit about asynchronous operations, and this is where it will really help you.  If you send a synchronous request via a socket, when you call Socket.Receive(), your thread will block until a request is received.  This isn't good.  Your thread can't make any more requests since it's blocked.  Using Socket.Beginxxxxxx(), the I/O request will be made and put in the IRP queue for the socket, and your thread will keep going.  This means, that your thread could actually make thousands of requests in a loop without any blocking at all!

If I'm understanding you correctly, you are using calls via Zeroc-Ice in your testing code, not actually trying to reach an http endpoint.  If that's the case, I can admit that I don't know how Zeroc-Ice works.  I would, however, suggest following the advice listed here, particularly the part: Consider Asynchronous Method Invocation (AMI).  The page shows this:

By using AMI, the client regains the thread of control as soon as the invocation has been sent (or, if it cannot be sent immediately, has been queued), allowing the client to use that thread to perform other useful work in the mean time.

Which seems to be the equivalent of what I described above using .NET sockets.  There may be other ways to improve the performance when trying to do a lot of sends, but I would start here or with any other suggestion listed on that page.  You've been very vague about the design of your application, so I can be more specific than I have been above.  Just remember, do not use more threads than absolutely necessary to get what you need done, otherwise you'll likely find your application running far slower than you want.

Some examples in pseudocode (tried to make it as close to ice as possible without me actually having to learn it):

```<code>var iterations = 100000;
for (int i = 0; i < iterations; i++)
{
    // The thread blocks here waiting for the response.
    // That slows down your loop and you're just wasting
    // CPU cycles that could instead be sending/receiving more objects
    MyObjectPrx obj = iceComm.stringToProxy("whateverissupposedtogohere");
    obj.DoStuff();
}
</code>```

A better way:

```<code>public interface MyObjectPrx : Ice.ObjectPrx
{
    Ice.AsyncResult GetObject(int obj, Ice.AsyncCallback cb, object cookie);
    // other functions
}

public static void Finished(Ice.AsyncResult result)
{
    MyObjectPrx obj = (MyObjectPrx)result.GetProxy();
    obj.DoStuff();
}

static void Main(string[] args)
{
    // threaded code...
    var iterations = 100000;
    for (int i = 0; i < iterations; i++)
    {
        int num = //whatever
        MyObjectPrx prx = //whatever
        Ice.AsyncCallback cb = new Ice.AsyncCallback(Finished);
        // This function immediately gets called, and the loop continues
        // it doesn't wait for a response, it just continually sends out socket
        // requests as fast as your CPU can handle them.  The response from the
        // server will be handled in the callback function when the request
        // completes.  Hopefully you can see how this is much faster when
        // sending sockets.  If your server does not use an Async model
        // like this, however, it's quite possible that your server won't
        // be able to handle the requests
        prx.GetObject(num, cb, null);
    }
}
</code>```

Keep in mind that more threads != better performance when trying to send sockets (or really doing anything).  Threads are not magic in that they will automatically solve whatever problem you're working on.  Ideally, you want 1 thread per core, unless a thread is spending much of its time waiting, then you can justify having more.  Running each request in its own thread is a bad idea, since context switches will occur and resource waste.  (If you want to see everything I wrote about that, click edit and look at the past revisions of this post.  I removed it since it only seemed to cloud the main issue at hand.)

You can definitely make these request in threads, if you want to make a large number of requests per second.  However, don't go overboard with the thread creation.  Find a balance and stick with it.  You'll get better performance if you use an asynchronous model vs a synchronous one.

I hope that helps.

Title: Load testing : how to generate per second requests?

Body:
I have a server component which runs over Zeroc-ICE. When I wanted to load test it, I thought that using parallel library to create multiple requests would do it. But it dint end up that way. Using Parallel(Parallel.For) library from C# apparently was easier but it doesn't seem to be exactly generating everything parallel in the same instant. So it cannot be the definition for creating N requests per second. How should I do it ? I guess anyone who wants to do load testing first would actually think about this .

What's the efficient way to actually create N requests in really per second ?

Another myth is about the parallel programming. Please enlighten us , if you have used parallel programming patterns in C# or .Net in general . Imagine I have 5 processes. How will start all the five processes at the same time. What does it mean to my consumption of resources ? I have tried reading into many of the material available over the net but I get more and more questions than them being the answer to my questions.

I used Parallel.For and created N threads and measured time. Then I tried the same thing using Task.Factory.start for enumeration of tasks. The time measured was different. So What exactly is the different between using these ? When I should use the corresponding classes and for what purposes exactly ? we often have lots of riches but its just we exactly don't know how to differentiate one from the another. This is one such case for me, not being able to find why I should not use one from the another.

I used the stopwatch class to measure these times which claims to be the best. In the scenario of me load testing a component, what would be the way to measure the time of response. Stopwatch seems to be the best solution for me . Any opinions are welcome.

ps: There are many load testing tools for web applications. Mine is a customised case of server components. And my question is more pertaining to creating N threads per second.

All opinions are welcome. Just don't think its so much not a programming question. It ofcourse is. It should ring bells for any programmer who wants to QE stuff by himself to know the performance of his product , first hand by himself.I have tried many options and then had to fall back on how should I actually do it ?

Body:
I don't have all the answers.  Hopefully I can shed some light on it.

To simplify my previous statements about .NET's threading models, just know that Parallel Library uses Tasks, and the default TaskScheduler for Tasks, uses the ThreadPool.  The higher you go in the hierarchy (ThreadPool is at the bottom), the more overhead you have when creating the items. That extra overhead certainly doesn't mean it's slower, but it's good to know that it's there.  Ultimately the performance of your algorithm in a multi-threaded environment comes down to its design.  What performs well sequentially may not perform as well in parallel.  There are too many factors involved to give you hard and fast rules, they change depending on what you're trying to do.  Since you're dealing with network requests, I'll try and give a small example.

Let me state that I am no expert with sockets, and I know next to nothing about Zeroc-Ice.  I do know about bit about asynchronous operations, and this is where it will really help you.  If you send a synchronous request via a socket, when you call Socket.Receive(), your thread will block until a request is received.  This isn't good.  Your thread can't make any more requests since it's blocked.  Using Socket.Beginxxxxxx(), the I/O request will be made and put in the IRP queue for the socket, and your thread will keep going.  This means, that your thread could actually make thousands of requests in a loop without any blocking at all!

If I'm understanding you correctly, you are using calls via Zeroc-Ice in your testing code, not actually trying to reach an http endpoint.  If that's the case, I can admit that I don't know how Zeroc-Ice works.  I would, however, suggest following the advice listed here, particularly the part: Consider Asynchronous Method Invocation (AMI).  The page shows this:

By using AMI, the client regains the thread of control as soon as the invocation has been sent (or, if it cannot be sent immediately, has been queued), allowing the client to use that thread to perform other useful work in the mean time.

Which seems to be the equivalent of what I described above using .NET sockets.  There may be other ways to improve the performance when trying to do a lot of sends, but I would start here or with any other suggestion listed on that page.  You've been very vague about the design of your application, so I can be more specific than I have been above.  Just remember, do not use more threads than absolutely necessary to get what you need done, otherwise you'll likely find your application running far slower than you want.

Some examples in pseudocode (tried to make it as close to ice as possible without me actually having to learn it):

var iterations = 100000;
for (int i = 0; i < iterations; i++)
{
    // The thread blocks here waiting for the response.
    // That slows down your loop and you're just wasting
    // CPU cycles that could instead be sending/receiving more objects
    MyObjectPrx obj = iceComm.stringToProxy("whateverissupposedtogohere");
    obj.DoStuff();
}

A better way:

public interface MyObjectPrx : Ice.ObjectPrx
{
    Ice.AsyncResult GetObject(int obj, Ice.AsyncCallback cb, object cookie);
    // other functions
}

public static void Finished(Ice.AsyncResult result)
{
    MyObjectPrx obj = (MyObjectPrx)result.GetProxy();
    obj.DoStuff();
}

static void Main(string[] args)
{
    // threaded code...
    var iterations = 100000;
    for (int i = 0; i < iterations; i++)
    {
        int num = //whatever
        MyObjectPrx prx = //whatever
        Ice.AsyncCallback cb = new Ice.AsyncCallback(Finished);
        // This function immediately gets called, and the loop continues
        // it doesn't wait for a response, it just continually sends out socket
        // requests as fast as your CPU can handle them.  The response from the
        // server will be handled in the callback function when the request
        // completes.  Hopefully you can see how this is much faster when
        // sending sockets.  If your server does not use an Async model
        // like this, however, it's quite possible that your server won't
        // be able to handle the requests
        prx.GetObject(num, cb, null);
    }
}

Keep in mind that more threads != better performance when trying to send sockets (or really doing anything).  Threads are not magic in that they will automatically solve whatever problem you're working on.  Ideally, you want 1 thread per core, unless a thread is spending much of its time waiting, then you can justify having more.  Running each request in its own thread is a bad idea, since context switches will occur and resource waste.  (If you want to see everything I wrote about that, click edit and look at the past revisions of this post.  I removed it since it only seemed to cloud the main issue at hand.)

You can definitely make these request in threads, if you want to make a large number of requests per second.  However, don't go overboard with the thread creation.  Find a balance and stick with it.  You'll get better performance if you use an asynchronous model vs a synchronous one.

I hope that helps.

OwnerDisplayName: Christopher Currens


问:
Refactor or Concentrate on Completing App
Would you refactor your app as you go or focus on completing app first? Refactoring will mean progress of app app will slow down.

Completing app will mean you get a possibly very hard to maintain app later on?

The app is a personal project. I don't really know how to answer "What drives the functionality and design", but I guess it's to solve inefficiencies in current software out there. I like minimal easy to use software too. So I am removing some features and add some that I feel will help.

答:
Refactoring will be very important as you work on completing your application.

+VES

Clear Flow: Refactoring will give code clarity because sometimes if the code is little unstructured, understand the flow of the code after a period of time would become difficult and refactoring code could become an uphill task and bugs could get introduced.

Improve performance: Proper refactoring of the application will definitely help improve application performance.

Maintenance: Last but not the least, it would be easier to maintain on the long run.

-VE

Time Consuming: It would be a lot more time consuming at every stage of your progress. So there could be a delay in completion.

Bottom Line

Depending on the project type, you can prioritize on code quality or completion whichever is demanding for the current situation.

ContentLicense: CC BY-SA 3.0


答:
Based from my personal experience, I would usually go to finish the apps first with the condition that there is a deadline to reach. once you completed you can refactor it.

Finish it and refactor it. But if there's no deadline to be rush or time frame, I suggest refactoring it would be good.

Title: Refactor or Concentrate on Completing App

Body:
Would you refactor your app as you go or focus on completing app first? Refactoring will mean progress of app app will slow down.

Completing app will mean you get a possibly very hard to maintain app later on?

The app is a personal project. I don't really know how to answer "What drives the functionality and design", but I guess it's to solve inefficiencies in current software out there. I like minimal easy to use software too. So I am removing some features and add some that I feel will help.


答:
Refactoring is like picking up your room.

If you keep things tidy, you have a linear overhead, proportional to the amount of productive work you're doing on the code, O(n) in algorithmologist terms. Assuming you spend 10% of your time refactoring (or keeping your room tidy), that 10% is a given, and it will remain constant over time.

If, however, you toss your dirty clothes in a corner, and keep doing it, the amount of time you are going to spend picking up your room grows as the mess becomes more complex. Assuming that each individual piece of dirty laundry contributes exponentially to the required cleanup time, you are now in an O(en) situation.

Anyone who has ever digged into the concept of algorithmic complexity will observe that there is a break-even point somewhere, that is, there is an optimal amount of dirty laundry to accumulate; how much that is depends on the constant factors that are discarded in big-O notation. Another factor is the value of your work over time: if your work is worth a lot now, but cheap next week (i.e., there is a deadline this friday for this project and three more, but after that, you'll be mostly idle), the equation might turn out in favor of not refactoring.

And then there's the complexity critical mass. At some point, the mess ('critical mess', if you will) gets so bad that it seems easier to just burn down the entire room and buy new clothes. In reality it usually isn't, but it appears that way, and psychological effects will make it ten times harder to tackle the thing.

And obviously, if you step into a project that is a giant multiply-redundant mess already, you have limited choice.

TL;DR: If in doubt, refactor. You should have really good evidence before deciding not to.


问:
Meaningful concise method naming guidelines
Recently I started releasing an open source project, while I was the only user of the library I did not care about the names, but know I want to assign clever names to each methods to make it easier to learn, but I also need to use concise names so they are easy to write as well.

I was thinking about some guidelines about the naming, I am aware of lots of guidelines that only care about letters casing or some simple notes. Here, I am looking after guidelines for meaningful but yet concise naming.

For example, this could be part of the guidelines I am looking after:

Use Add when an existing item is going to be added to a target, Use
Create when a new item is being created and added to a target.

Use Remove when an existing item is going to be removed from a target,
Use delete when an item is going to be removed permanently.

Pair AddXXX methods with RemoveXXX and Pair CreateXXX methods with
DeleteXXX methods, but do not mix them.

As the above samples show,I would like to find some online material helping me with naming methods and other item complying with English grammar and word meanings.

The above guidance may be intuitive for native English speakers, but for me that English is my second language I need to be told about things like this.

答:
For me, finding a good name for something always comes back to thinking of it as an object that has to justify its existence. Ask yourself:

What does the class/method/variable do, i.e. what is its wider purpose and what is it for?

What specifically about its purpose does it need to communicate, i.e. what is the essential part that the name needs to have in it?

Most developers would agree that readability is always of paramount importance when it comes to naming. Don't just write code so that you know what you mean while you're writing it, write it so that someone looking at the code for the first time at some point in the future knows what you meant without having to think too much. You'll write the code just once, but during its lifetime it will most likely have to be edited many times and read even more times.

Code should be self-documenting, that is, your naming should make it obvious what something does. If you need to explain what a line of code does in a comment, and renaming things doesn't improve it enough, you should seriously consider refactoring that line into a new method with an appropriately descriptive name, so that reading the original method, the new method call describes what's happening. Don't be afraid to have long names; of course you shouldn't write novels in class/method/variable names, but I'd rather have a name be too long and descriptive than too short and I need to figure out what it does by looking under the hood. Except for some obvious exceptions like x/y coordinates and commonly used acronyms, avoid single-character names and abbreviations. Calling something "bkBtn" instead of "backButton" may have had a purpose back when names were limited to 8 characters and dinosaurs roamed the earth, but in these days with code-completion there's really no excuse for abbreviating.

As much as your language allows, make your code read like an English sentence. Objects use nouns, methods use verbs. Boolean methods typically start with "is", but there are many other options that convey the meaning even better, depending on the use case, such as "can", "should", or "does". Of course, not all languages can be as good as Smalltalk at this, but some symbols are generally understood to be parts of the sentence. Two Smalltalk conventions I personally like to take into other languages as much as possible are prefixing the name of loop parameters with "each", and prefixing method parameters with the article "a" (or "an", or "some" for collections). This may not be a common standard in Java, and anyone is welcome to ignore this bit, but I find that this greatly enhances the readability of code. For instance (example in Java):

```<code>public boolean shouldConsiderAbbreviating(List<String> someNames) {
  for (String eachName : someNames) {
    if (isTooLong(eachName)) {
      return true;
    }
  }
  return false;
}
</code>```

This should be readable to people with just a bit of knowledge of Java as something like this:

To determine whether you should consider abbreviating a list of some names (which are strings), iterate over some names, and for each name, determine whether it is too long; if so, return true; if none is too long, return false.

Contrast the above code with just naming the argument strings and the loop variable string, especially in a more complex method. You'd have to look closely to see the difference instead of the usage being obvious from a glance at the name.

Body:
For me, finding a good name for something always comes back to thinking of it as an object that has to justify its existence. Ask yourself:

What does the class/method/variable do, i.e. what is its wider purpose and what is it for?

What specifically about its purpose does it need to communicate, i.e. what is the essential part that the name needs to have in it?

Most developers would agree that readability is always of paramount importance when it comes to naming. Don't just write code so that you know what you mean while you're writing it, write it so that someone looking at the code for the first time at some point in the future knows what you meant without having to think too much. You'll write the code just once, but during its lifetime it will most likely have to be edited many times and read even more times.

Code should be self-documenting, that is, your naming should make it obvious what something does. If you need to explain what a line of code does in a comment, and renaming things doesn't improve it enough, you should seriously consider refactoring that line into a new method with an appropriately descriptive name, so that reading the original method, the new method call describes what's happening. Don't be afraid to have long names; of course you shouldn't write novels in class/method/variable names, but I'd rather have a name be too long and descriptive than too short and I need to figure out what it does by looking under the hood. Except for some obvious exceptions like x/y coordinates and commonly used acronyms, avoid single-character names and abbreviations. Calling something "bkBtn" instead of "backButton" may have had a purpose back when names were limited to 8 characters and dinosaurs roamed the earth, but in these days with code-completion there's really no excuse for abbreviating.

As much as your language allows, make your code read like an English sentence. Objects use nouns, methods use verbs. Boolean methods typically start with "is", but there are many other options that convey the meaning even better, depending on the use case, such as "can", "should", or "does". Of course, not all languages can be as good as Smalltalk at this, but some symbols are generally understood to be parts of the sentence. Two Smalltalk conventions I personally like to take into other languages as much as possible are prefixing the name of loop parameters with "each", and prefixing method parameters with the article "a" (or "an", or "some" for collections). This may not be a common standard in Java, and anyone is welcome to ignore this bit, but I find that this greatly enhances the readability of code. For instance (example in Java):

public boolean shouldConsiderAbbreviating(List<String> someNames) {
  for (String eachName : someNames) {
    if (isTooLong(eachName)) {
      return true;
    }
  }
  return false;
}

This should be readable to people with just a bit of knowledge of Java as something like this:

To determine whether you should consider abbreviating a list of some names (which are strings), iterate over some names, and for each name, determine whether it is too long; if so, return true; if none is too long, return false.

Contrast the above code with just naming the argument strings and the loop variable string, especially in a more complex method. You'd have to look closely to see the difference instead of the usage being obvious from a glance at the name.


问:
Are super methods in JavaScript limited to functional inheritance, as per Crockford's book?
In Douglas Crockford's "JavaScript: The Good Parts", he walks through three types of inheritance: classical, prototypal, and functional. In the part on functional inheritance he writes:

"The functional pattern also gives us a way to deal with super methods."

He then goes on to implement a method named "superior" on all Objects. However, in the way he uses the superior method, it just looks like he is copying the method on the super object for later use:

```<code>// crockford's code:
var coolcat = function(spec) {
    var that = cat(spec),
        super_get_name = that.superior('get_name');
    that.get_name = function (n) {
        return 'like ' + super_get_name() + ' baby';
    };
    return that;
};
</code>```

The original get_name method is copied to super_get_name. I don't get what's so special about functional inheritance that makes this possible. Can't you do this with classical or prototypal inheritance? What's the difference between the code above and the code below:

```<code>var CoolCat = function(name) {
    this.name = name;
}

CoolCat.prototype = new Cat();

CoolCat.prototype.super_get_name = CoolCat.prototype.get_name;
CoolCat.prototype.get_name = function (n) {
    return 'like ' + this.super_get_name() + ' baby';
};
</code>```

Doesn't this second example provide access to "super methods" too?

答:
```<code>var coolcat = function(spec) {
    var that = cat(spec),
        super_get_name = that.superior('get_name');

    that.get_name = function (n) {
        return 'like ' + super_get_name() + ' baby';
    };

    return that;
};

var cat = coolcat({ ... });
</code>```

Can just as well be done with prototypes

```<code>var CoolCat = {
    constructor: function () {
        Cat.constructor.apply(this, arguments);
        return this;
    },
    getName: function () {
        return 'like ' + Cat.getName() + ' baby';
    }
};

var cat = Object.create(CoolCat).constructor({ ... });
</code>```

Of course the main difference here is that your not creating two new functions every time you invoke coolcat. coolcat creates a new function for get_name and a new function for super_get_name and basically wastes memory left and right.

One may argue coolcat has privacy but it does not because anyone can call cat({}).superior('get_name'); and get your "private" method.

Of course if one wants privacy and prototypes that's perfectly possible, using the klass macro as shown in the article

```<code>var Cat = klass(function (privates) {
    return {
        constructor: function (name) {
            privates(this).name = name;
        },
        getName: function () {
            return privates(this).name;
        }
    };
});

var CoolCat = klass(Cat, function (privates, $super) {
    return {
        constructor: function () {
            $super.constructor.apply(this, arguments);
        },
        getName: function () {
            return 'like ' + $super.getName.call(this) + ' baby';
        }
    };
});

var cat = new CoolCat("someName");
console.log(cat.getName());
</code>```

Live Example

Here we achieve both privacy and super class inheritance in an "elegant" manner.

In this elegant means not having the closure overhead of multiple functions per object (we still add a single closure per object for the privates function) and also allowing normal prototypical OO to just work.

Body:
var coolcat = function(spec) {
    var that = cat(spec),
        super_get_name = that.superior('get_name');

    that.get_name = function (n) {
        return 'like ' + super_get_name() + ' baby';
    };

    return that;
};

var cat = coolcat({ ... });

Can just as well be done with prototypes

var CoolCat = {
    constructor: function () {
        Cat.constructor.apply(this, arguments);
        return this;
    },
    getName: function () {
        return 'like ' + Cat.getName() + ' baby';
    }
};

var cat = Object.create(CoolCat).constructor({ ... });

Of course the main difference here is that your not creating two new functions every time you invoke coolcat. coolcat creates a new function for get_name and a new function for super_get_name and basically wastes memory left and right.

One may argue coolcat has privacy but it does not because anyone can call cat({}).superior('get_name'); and get your "private" method.

Of course if one wants privacy and prototypes that's perfectly possible, using the klass macro as shown in the article

var Cat = klass(function (privates) {
    return {
        constructor: function (name) {
            privates(this).name = name;
        },
        getName: function () {
            return privates(this).name;
        }
    };
});

var CoolCat = klass(Cat, function (privates, $super) {
    return {
        constructor: function () {
            $super.constructor.apply(this, arguments);
        },
        getName: function () {
            return 'like ' + $super.getName.call(this) + ' baby';
        }
    };
});

var cat = new CoolCat("someName");
console.log(cat.getName());

Live Example

Here we achieve both privacy and super class inheritance in an "elegant" manner.

In this elegant means not having the closure overhead of multiple functions per object (we still add a single closure per object for the privates function) and also allowing normal prototypical OO to just work.


问:
Jump into Android or learn Java basics first?
I am quite proficient in C and know some C++, but never touched Java. Recently I got an idea for an app which I think has potential, and I want to develop it.

I am planning to go Android first, cause it doesn't require a mac/iPhone.

So my question is: can I go straight to learning Android development, picking up the Java syntax as I go along and need it (after all I don't think it's much different from C/C++), or should I take a couple of weeks to learn the basics of Java first and then start with Android development?

答:
should I take a couple of weeks to learn the basics of Java first and then start with Android development?

Given that you target Android, it makes sense to simply pick their official tutorial of Getting Started kind and start studying it. If it is necessary / desirable to study Java along the way, you can expect this to be mentioned in this tutorial.

Per my recollection early versions of Android tutorials were of acceptable quality. I didn't work with it for quite a while though.


问:
How much time do you invest in exploring new technology?
Sometimes, I see potential in a new technology eg. AIR (not new but I haven't tried it much), and I play around with it. After sometime, I feel inspired to try a new personal project out of it. It was during this project that I sometimes find another "tech" eg. Silverlight more suitable. Maybe its easier to get something done in that platform. But there are many alternatives to such things, and after a while, i feel that i have wasted my time just trying the "new toy", spending time learning it to get started etc. only to decide that I made the wrong decision.

How long do you take to explore new technology? Or do you stick to what you already know?

答: I would suggest you to have an in depth knowledge in one platform to meet delivery needs and have a good understanding in the other platforms which you might encounter so that you can use the overall understand of the two approaches to solve any problem you may encounter effectively based on the situation.

ContentLicense: CC BY-SA 2.5


问:
Do I have the wrong idea about software engineering?
I have a question which has been raised by my latest job (rather intern).

Just to put things into context - I'm 21 and I've finished my 2nd year of university before that I've had around 2 years of experience doing sys admin/QA jobs and basically I can say that I've seen how different IT sectors operated. Flash forward to present times and here's me landing an interning job at one of the premier research institution in the UK.

What I have to do is create some internal tools using a mix of technologies - mainly AWS/Java/Bash - you get the picture. Everything is OK, I'm doing my job BUT I'm not happy. Why is that - because I'm expected to work in an ad-hoc matter. That is create things quickly, without spending time on designing. My manager explicitly said that it was expected to "rush" through problems as they arise and we essentially. As a consequence it turned out that  things had to be re-done and re-engineer and they are still not perfect. As far as testing is concerned - keep it to a minimum, as long as it looks working then it is OK.

Am I at fault to disagree with this way of conducting work? Is it wrong to want to think over the system as a whole, then focus on different components and see how they might inter-operate, to zero in on different "key points" which might turn out to be problematic in the future? Is it a crime to want to do a good job and not a "quick job"? Is it a mistake or wrong attitude to want to research the data structures applicable to a problem so that you can choose the best depending on a particular problem set? To the best of my understanding the "Engineering" bit in "Software Engineering" has got to do exactly with this - research your problem domain and come up with a informed solution then refine as necessary?

I've been to an interview at an Arm's office in the UK and they showed me their SCRUM room and it looked they had pretty good idea as to how to manage their project - they had a backlog, they had metrics as to how long each issue might take to resolve - the usual things for SCRUM - completely different than the way things are run "here"

Have I built a wrong idea about the software industry in general? I'd like to hear your input on that. I mean I "entered" software development purely because I want to create things - plain and simple, but I want to create quality things. I want to see my software used in various scenarios, I want to see it bullet proof - isn't that the driving force for all software engineers? I think everyone can be a programmer/coder by just learning the syntax but for me where the real fun begins is when you actually have to come up with a design which is doable in real world.

I used to do my university assignments by just looking at them and directly start coding and could easily get marks above 75% and never really appreciated the "software development lifecycle" module. But now when I saw in the real world how bad it is to work without any formal process and the frustration that is inherent in situation where you don't know if the requirements are going to change tomorrow (oh, did I say that we don't have clearly defined requirement analysis?)

I really like to believe I just landed a position where some people just needed a code monkey to do their dirty work and this is not the case how the software world operates at large.

答:
What an excellent question! Sometimes you can make something valuable by being fast. This is typically the case in a research lab where the faster we can learn what we don't know, the better off we will be. The software you produce exists only to answer questions. It is "throw away code". This is also the case with startups that don't know what the customers really want. Also, the first time you make something, it will be crappy. Read The Mythical Man-Month.

Sometimes you can make something valuable by being good. This is typically the case with shrink-wraped software like Adobe Photoshop. The research has already been done years ago and now the question is how to add the list of features that customers want in a way that doesn't introduce bugs. It is a matter of architecture, design, and testing, testing, testing. The code itself is what is valuable, not what you learn from it.

If you are not happy with research (making the first of something, learning new things that no one knew before) then give shrink-wrapped software a try. In fact, at your age you should try as many things as possible. Go take risks! You will learn a lot and you will be better off in the long run.


问:
Is it possible to write software that doesn't need to be continually modified?
I have written a lot of software in many different languages, and I've also "written" hardware for use with FPGAs using Verilog and VHDL.

I tend to enjoy writing hardware more than software, and I think one of the primary reasons is that it is possible to write hardware that is "done" and never needs to be modified: you define the interfaces and the functionality, write a test bench, implement the hardware module, then test the heck out of it using a simulator. Then you can rely on that hardware module as a building block to create something bigger and better: if you need to add a feature to that module, you create a second module and add the functionality there. You never throw away the original module since it is working just fine and can still be useful.

One of my main frustrations with software is that it is never "done". There is always another feature to add. Often when adding a feature it introduces a bug somewhere else that was working just fine before. This doesn't happen in hardware as long as the interfaces are not violated.

To be clear, I'm not advocating building one version of something with a list of features and that's it forever: I'm in favor of iterations and multiple releases over time to add new features. I just don't want to poke the code on the left and find a bug on the right, and this seems to happen after adding a new feature.

Is it possible to write software in a similar way in which hardware is "written"?  Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

答:
The difference is just how much easier and cheaper it is to modify software compared to hardware. If hardware was as easy and cheap to modify and ship to customers, they would.

I think if I could sum up my poorly-expressed question it would be something like, "how can I have more fun writing software by not introducing bugs into working code and always making forward progress?" Maybe it has something to do with well-defined interfaces and testing, like hardware?

You should definitely check test-driven development.


答:
Maybe it has something to do with well-defined interfaces and testing, like hardware?

Exactly my thoughts!

Well-designed modules with clear interfaces tend to be essentially perfect. Think of something like String class of Java. It is a computer program, but it has a crystal-clear interface. There are no known bugs in it. It does what it is supposed to do, perfectly. Sure, it has been extensively tested during the last 15 years, and since virtually all programs use Strings as basic building blocks, any bug in it would be quickly noticed. Any "quirks" - not strictly bugs, but design details worth being aware of - such as those described here http://www.jwz.org/doc/java.html are well-known by now, and thus can be taken into account.

Buggy software is partly a cultural issue: people are used to buggy software, and unlike hardware, software can usually be easily fixed afterwards, so it doesn't have to perfect initially (or ever, because hey, we must ship now, let's fix it in the next version). But for a large part, it is a real complexity issue: software complexity has been steadily increasing for the last 50 years or so, but the human brain is the same. When the growing difficulty of achieving perfection and the growing ease of fixing things later on (fast, automatic builds, internet distribution) combine with schedule pressure and lack of discipline, the result is what it is.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

No silver bullet probably, but a good combination of best practices, including but not limited to:

Simple, autonomous modules. In other words, low coupling and high cohesion.

Immutability. Particularly important with growing concurrency.

It's worth noting that both points aim at reducing complexity. That's the key point. The entropy always tends to increase, and unless we fight that, we'll soon drown in complexity. It's also interesting to see that during the last few years, programming languages have been evolving toward encouraging or even enforcing the above mentioned practices. Particularly, the rise of functional languages is just that: pure functions always return the same value for the same input, there's no state in them. Then you just compose pure functions that take and return immutable values, and limit the inevitable mutability to small well-defined places instead of spreading it all around. Check this out: http://clojure.org/state

Title: Is it possible to write software that doesn't need to be continually modified?

Body:
I have written a lot of software in many different languages, and I've also "written" hardware for use with FPGAs using Verilog and VHDL.

I tend to enjoy writing hardware more than software, and I think one of the primary reasons is that it is possible to write hardware that is "done" and never needs to be modified: you define the interfaces and the functionality, write a test bench, implement the hardware module, then test the heck out of it using a simulator. Then you can rely on that hardware module as a building block to create something bigger and better: if you need to add a feature to that module, you create a second module and add the functionality there. You never throw away the original module since it is working just fine and can still be useful.

One of my main frustrations with software is that it is never "done". There is always another feature to add. Often when adding a feature it introduces a bug somewhere else that was working just fine before. This doesn't happen in hardware as long as the interfaces are not violated.

To be clear, I'm not advocating building one version of something with a list of features and that's it forever: I'm in favor of iterations and multiple releases over time to add new features. I just don't want to poke the code on the left and find a bug on the right, and this seems to happen after adding a new feature.

Is it possible to write software in a similar way in which hardware is "written"?  Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?


问:
Should one use pseudocode before actual coding?
Pseudocode helps us understand tasks in a language-agnostic manner. Is it best practice or suggested approach to have pseudocode creation as part of the development lifecycle? For instance:

Identify and split the coding tasks

Write pseudocode

Get it approved [by PL or TL]

Start Coding based on Pseudocode

Is this a suggested approach? Is it practiced in the industry?

答:
The only times I've used pseudocode in the last 10 years are on interview when we're working on a whiteboard and the particular language doesn't matter.

It's certainly helpful for talking through a process/algorithm in loose terms without getting bogged down with syntax. E.g. writing a C++ class which has C# properties, just to illustrate the point.

It has its' place, but should only be used where needed (it's work that you'll throw away) and certainly not part of a formal process, unless you have ISO-type standards that insist on it.


答:
I will comment on some of your remarks, hoping you get the answer from these comments.

One of my main frustrations with software is that it is never "done".

This is because the solution specification is incomplete or because the plan to deliver enhancements is not accurate. This can happen to any project software, hardware or any other.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Of course, creating independent modules should greatly reduce dependency. This has to be considered when you are designing the software. You need to consider, separation of concerns, layers, tiers, controller objects, interfaces, etc.

"how can I have more fun writing software by not introducing bugs into working code and always making forward progress?"

1-Understand requirements carefully. May be you need to consider closing the requirements before design. If you do iterative development, there is no chance of doing this. Some methodologies encourage this, but personally, I think this is not good for all project types. Building software on solid requirements allows for better design.

2-Make your user understand this long-tern philosophy.

3-Plan implementation carefully

4-Design before code.

5-Use generic design when appropriate.

6-Use prototypes as a design confirmation tool.


问:
Date as software version number
Software developers don't typically use date as version number, though YYYYMMDD format (or its variances) looks solid enough to use. Is there anything wrong with that scheme? Or does it apply to limited 'types' of software only (like in-house productions)?</p>

答:
Date As a Version

If your product is not sold then just using the date is fine and probably useful. If you sell it and upgrades to customers they want to buy a model with a specific set of features. It is much easier to sell them on an upgrade if this model has a clear name, it does not really matter what it is but for example no one really buys the 2012 Jan 20th Ford Car they want the Kia Model whatever. The same is true of software. Giving a hard model name and version means it has features different from an older one. For me just a date does not do that, it says I made it then, not it might have new features.

Scheme We Use

I made no claim that our system, creates a clear brand as above.
We now use a four part scheme. A version number, state, the date and a checksum.

```<code>1200_GLD_120120_F0D1
</code>```

This may seam over the top but it allows for us to have several versions of the build for version 1200 that our engineers can use and we differentiate between them by date. The state tell people if it is a internal test version, customer patch build or gold release. The checksum is new, it allows an engineer or customer to verify that they actually have downloaded the correct one from our distribution.

So we might have a sequence of

```<code>1200_TST_120110_EF45
1200_GLD_120120_F0D1
1201_FIX_120125_123E
1201_TST_120130_31A5
1201_TST_120131_FDFD
</code>```

We only normally refer to the major version numbers to customer ie "12" but a customer will get the latest gold version of 12 ie 1200_GLD_120120_F0D1 unless they need the Fix.

Title: Date as software version number

Body:
Date As a Version

If your product is not sold then just using the date is fine and probably useful. If you sell it and upgrades to customers they want to buy a model with a specific set of features. It is much easier to sell them on an upgrade if this model has a clear name, it does not really matter what it is but for example no one really buys the 2012 Jan 20th Ford Car they want the Kia Model whatever. The same is true of software. Giving a hard model name and version means it has features different from an older one. For me just a date does not do that, it says I made it then, not it might have new features.

Scheme We Use

I made no claim that our system, creates a clear brand as above.
We now use a four part scheme. A version number, state, the date and a checksum.

1200_GLD_120120_F0D1

This may seam over the top but it allows for us to have several versions of the build for version 1200 that our engineers can use and we differentiate between them by date. The state tell people if it is a internal test version, customer patch build or gold release. The checksum is new, it allows an engineer or customer to verify that they actually have downloaded the correct one from our distribution.

So we might have a sequence of

1200_TST_120110_EF45
1200_GLD_120120_F0D1
1201_FIX_120125_123E
1201_TST_120130_31A5
1201_TST_120131_FDFD

We only normally refer to the major version numbers to customer ie "12" but a customer will get the latest gold version of 12 ie 1200_GLD_120120_F0D1 unless they need the Fix.


答:
how can I have more fun writing software by not introducing bugs into working code and always making forward progress?

I would love to find a final answer to your question. But the reality is that there is no easy way to do it, thats why extreme programming and TDD technics are becoming so popular. You need to embrace change, because is going to happen. I dont know if its funnier this way, but a lot less stressful for sure ;-)

http://en.wikipedia.org/wiki/Extreme_Programming

When you interact with hardware, the hardware needs x value and thats all (in theory), but when you interact with people today they need x, and tomorrow they can need y, etc. It is how it is, bussines and people requirements change. Because Persons != machines, so code that NEVER changes most of the times is not possible.

Also as i said on my previous/deleted answer, try to avoid changes that are not important by making people think before start coding. Involve users more on decision making, etc. Clarify costs of changes, plan more, etc. Those are not "ways of coding", are ways of "not coding" because with more though about requirements there will be less changes and more fun.


答:
As people are usually very quick to point out, one of the benefits of software is that it is supposed to be easy and relatively cheap to change compared to hardware.  This is especially important when your realize late that you got something fundamentally wrong.  Do the same with hardware, and you lose a million dollars, so as you've said, you use your simulators etc, and you test the bazinga out of it. This I think is where the paradigm fails somewhat when you shift to software.

Get into the average software developer's head, and what you have is a very busy person with an incredibly tight deadline.  His manager is saying that it's ok to leave a few bugs because you can always fix it later. Tests are often an afterthought, but even in a test-driven scenario, the tests are kept minimal and the code written to the minimum of the tests, and often shortcuts are taken so that many of the boundary cases might be missed. The system may be thoroughly unit tested yet is rarely tested rigorously as a whole, and just as rarely stress tested to any great degree. Add to this that you write software from scratch, and there is little opportunity to simulate the software before you commit to writing it, primarily because we rarely write software from the same sort of fine-grained building blocks that you would find in hardware. Taken from this perspective - which I don't believe is too far fetched an example - the software is not tested to the same high degree that hardware would be, because realistically it can't be, and to attempt to do so would not be seent to be very cost effective.

Back to the OP's question. Could you define a system of building blocks from which to derive all of your software?  Possibly.  Would it be very cost effective? Probably not, because by the time you get around to developing a robust enough system of components, tests, and other paraphernalia to support this ideal system of programming, you would find that your competition would have already beaten you to the market, and even worse, from the average programmer's perspective you would probably find a "cookie-cutter" style of programming system to be very limiting and more likely very boring. I personally work on an API, where the bulk of the module code has been refined and standardized so completely, that all I do now is generate a code template and fill in the blanks.  Most of my time can be spent writing simple connector code, and getting the modules out the door as fast as possible.  It's seriously mind-numbing.  There is very little opportunity to do more than just code the same sorts of things over and over, so when another project opportunity comes along, I jump at the chance to be able to do ANYTHING else.

So how can you get to deliver high quality and well factored software, and yet actually enjoy yourself doing it?  I believe this comes down to your choice of tools and methodology.  For me the answer has been to employ the use of a good BDD API, because it has allowed me to create very easy to read, yet highly granular code.  I can build a suite of tests out of a minimal number of reusable methods, and describe my tests in the language of the specifications. In this way, I come close to a more component-ized development approach, except for the fact that I am responsible for designing and checking the building blocks. Additionally, the test output pinpoints the exact part of the test where the failure occurs, so that I don't have to guess whether the failures are in the setup or the assertion.  This means I spend more of my time focused on solving problems instead of chasing them, my time is used more efficiently, I get more work done more quickly, and I can move on to other interesting tasks more quickly.

Tuning your methodology also helps. I'm a big advocate for applying lean development principals, and combining them with many of the other techniques and principals that the Agile movement has been banging-on about for many years now. Having eliminated most of the wasteful practices that I used to find so frustrating has helped a great deal to make development a more enjoyable activity. I'm still left with the issue that sometimes - but hopefully not too often - bugs will appear in my code, however I now find myself with more time, and even more incentive to spend more time writing more robust tests, and aiming for 100% test coverage. Even better, it feels really great to see all of those green lights appear at the end of my day, and be able to provide a complete - automatically generated - test report to give to my boss at the end of the day.


问:
Website Country Detection
I have a web crawler, and I'm looking for hints that will help me automatically detect a website country of origin.

And by country of origin I generally mean the country the website is targeting. For example:

http://www.spiegel.de/ -> Germany

http://www.lemonde.fr/ -> France

http://publico.pt/ -> Portugal

http://www.elpais.es/ -> Spain

I know there's not a foolproof way of doing it, so I will likely rely on a scoring system.

The domain name;

The content language;

The server's IP address;

Whois information;

What additional parameters would you use?

For the previous examples a combination of domain and content language will do, but many websites have a .com domain and a language spoken in more than one country...

答: Country selection information from the content and headers.</p>

Title: Website Country Detection


答:
One of my main frustrations with software is that it is never "done".
  There is always another feature to add.

If that frustrates you, consider a different career.  Seriously.

The point of software is to be able to add features.  The entire reason for inventing "software" in the first place was so that we could add features.

Often when adding a feature it introduces a bug somewhere else that was working just fine before.

That's a QA problem.

This doesn't happen in hardware as long as the interfaces are not violated.

That's also true in software.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Yes.  You have to actually practice Quality Assurance.


问:
What "version naming convention" do you use?
Are different version naming conventions suited to different projects? What do you use and why?

Personally, I prefer a build number in hexadecimal (e.g 11BCF), this should be incremented very regularly. And then for customers a simple 3 digit version number, i.e. 1.1.3.

```<code>1.2.3 (11BCF) <- Build number, should correspond with a revision in source control
^ ^ ^
| | |
| | +--- Minor bugs, spelling mistakes, etc.
| +----- Minor features, major bug fixes, etc.
+------- Major version, UX changes, file format changes, etc.
</code>```

答:
Version numbers should have enough information that you avoid conflicts and fixing a bug in the wrong release type problems, but shouldn't convey additional information that isn't relevant.

For instance if you use the date customers can tell that they have an older version, and patches against old versions can have confusing versions.

I personally like semantic versioning:

Releases are Major.Minor.Patch

Patch increments every time you release a build.

Minor increments every time backwards compatible functionality is added.

Major increments when new functionality is not backwards compatible.

When Major == 0 you're in alpha/pre-release. Major >= 1 are your public releases.

Lower numbers reset to 0 every time you increment, so

1.5.3 -> 1.5.4 (bug fix) -> 1.6.0 (minor feature) -> 2.0.0 (breaking change)

  This way if someone is using, say, version 1.5.3 they could tell at a glance that they could upgrade to 1.5.4 to get the patches, that 1.6.0 would add functionality and that they shouldn't upgrade to 2.0.0 (at least without handling the change).


答:
Here is my advice based on my experiences, I’m 20 and currently on placement working for major financial institution in the UK and had the same feelings you had a few months ago, what I noticed is that this maybe be due to the nature of the work you are doing.

What I mean by that is you said:

“What I have to do is create some internal tools using a mix of technologies - mainly AWS/Java/Bash”

I also had to create internal tools to help manage and automate certain processes and the fact is that in a fast paced environment “small” things are required to be implemented quickly. I didn’t have the luxury of applying most of the software engineering or algorithms and data structure principles I was taught in my 2nd year because a working version of the tool was required in a few weeks.I was very frustrated with this however it was not all bad as I did learn how to write better more readable code.

I had to be patient and I recently rotated to a new team that is working on a new iteration of in-house built system used by 10K+ users and I can assure you the software engineering aspect of it is taken very seriously. I have been told that I will gain exposure to the complete software life cycle from requirements capture/analysis all the way to implementation and testing. I believe I’m going to gain this experience because I’m not working on internal tools but that I’m working on a full scale system with a large user base.

What I recommend is that you be patient, finish creating the tools and make a very good job at it so that your supervisors gain more trust in you and assign you more challenging tasks that will require the use of software engineering principles.
Gain extra knowledge from doing some extra reading and apply that knowledge to what you are currently doing, I remember ransacking the entire e-book library at the company just to further my knowledge muhahah, from all the books I read I felt that effective java was really good book that helped me a lot.

Just be patient, invest heavily in you own  knowledge and apply that knowledge where possible. If you are doing a very good job someone will soon notice.

Title: Do I have the wrong idea about software engineering?

Body:
I have a question which has been raised by my latest job (rather intern).

Just to put things into context - I'm 21 and I've finished my 2nd year of university before that I've had around 2 years of experience doing sys admin/QA jobs and basically I can say that I've seen how different IT sectors operated. Flash forward to present times and here's me landing an interning job at one of the premier research institution in the UK.

What I have to do is create some internal tools using a mix of technologies - mainly AWS/Java/Bash - you get the picture. Everything is OK, I'm doing my job BUT I'm not happy. Why is that - because I'm expected to work in an ad-hoc matter. That is create things quickly, without spending time on designing. My manager explicitly said that it was expected to "rush" through problems as they arise and we essentially. As a consequence it turned out that  things had to be re-done and re-engineer and they are still not perfect. As far as testing is concerned - keep it to a minimum, as long as it looks working then it is OK.

Am I at fault to disagree with this way of conducting work? Is it wrong to want to think over the system as a whole, then focus on different components and see how they might inter-operate, to zero in on different "key points" which might turn out to be problematic in the future? Is it a crime to want to do a good job and not a "quick job"? Is it a mistake or wrong attitude to want to research the data structures applicable to a problem so that you can choose the best depending on a particular problem set? To the best of my understanding the "Engineering" bit in "Software Engineering" has got to do exactly with this - research your problem domain and come up with a informed solution then refine as necessary?

I've been to an interview at an Arm's office in the UK and they showed me their SCRUM room and it looked they had pretty good idea as to how to manage their project - they had a backlog, they had metrics as to how long each issue might take to resolve - the usual things for SCRUM - completely different than the way things are run "here"

Have I built a wrong idea about the software industry in general? I'd like to hear your input on that. I mean I "entered" software development purely because I want to create things - plain and simple, but I want to create quality things. I want to see my software used in various scenarios, I want to see it bullet proof - isn't that the driving force for all software engineers? I think everyone can be a programmer/coder by just learning the syntax but for me where the real fun begins is when you actually have to come up with a design which is doable in real world.

I used to do my university assignments by just looking at them and directly start coding and could easily get marks above 75% and never really appreciated the "software development lifecycle" module. But now when I saw in the real world how bad it is to work without any formal process and the frustration that is inherent in situation where you don't know if the requirements are going to change tomorrow (oh, did I say that we don't have clearly defined requirement analysis?)

I really like to believe I just landed a position where some people just needed a code monkey to do their dirty work and this is not the case how the software world operates at large.


问:
Should I intentionally break the build when a bug is found in production?
It seems reasonable to me that if a serious bug is found in production by end-users, a failing unit test should be added to cover that bug, thus intentionally breaking the build until the bug is fixed. My rationale for this is that the build should have been failing all along, but wasn't due to inadequate automated test coverage.

Several of my colleagues have disagreed saying that a failing unit test shouldn't be checked in. I agree with this viewpoint in terms of normal TDD practices, but I think that production bugs should be handled differently - after all why would you want to allow a build to succeed with known defects?

Does anyone else have proven strategies for handling this scenario? I understand intentionally breaking the build could be disruptive to other team members, but that entirely depends on how you're using branches.

答:
Why would you want to allow a build to succeed with known defects?

Because sometimes, you have time constraints. Or the bug is so minor that it isn't really worth delaying the shipment of the product for a few days needed to unit test and fix it.

Also, what's the point in breaking the build intentionally every time you find a bug? If you found it, fix it (or assign it to the person who will fix it), without disturbing your whole team. If you want to remember to fix a bug, then you need to mark it as very important in your  bug tracking system.


答:
Our strategy is:

Check in a failing test, but annotate it with @Ignore("fails because of Bug #1234").

That way, the test is there, but the build does not break.

Of course you note the ignored test in the bug db, so the @Ignore is removed once the test is fixed. This also serves as an easy check for the bug fix.

The point of breaking the build on failing tests is not to somehow put the team under pressure - it's to alert them to a problem. Once the problem is identified and filed in the bug DB, there's no point in having the test run for every build - you know that it will fail.

Of course, the bug should still be fixed. But scheduling the fix is a business decision, and thus not really the dev's concern... To me, once a bug is filed in the bug DB, it's no longer my problem, until the customer/product owner tells me they want it fixed.

Title: Should I intentionally break the build when a bug is found in production?

Body:
It seems reasonable to me that if a serious bug is found in production by end-users, a failing unit test should be added to cover that bug, thus intentionally breaking the build until the bug is fixed. My rationale for this is that the build should have been failing all along, but wasn't due to inadequate automated test coverage.

Several of my colleagues have disagreed saying that a failing unit test shouldn't be checked in. I agree with this viewpoint in terms of normal TDD practices, but I think that production bugs should be handled differently - after all why would you want to allow a build to succeed with known defects?

Does anyone else have proven strategies for handling this scenario? I understand intentionally breaking the build could be disruptive to other team members, but that entirely depends on how you're using branches.


问:
Load testing : how to generate per second requests?
I have a server component which runs over Zeroc-ICE. When I wanted to load test it, I thought that using parallel library to create multiple requests would do it. But it dint end up that way. Using Parallel(Parallel.For) library from C# apparently was easier but it doesn't seem to be exactly generating everything parallel in the same instant. So it cannot be the definition for creating N requests per second. How should I do it ? I guess anyone who wants to do load testing first would actually think about this .

What's the efficient way to actually create N requests in really per second ?

Another myth is about the parallel programming. Please enlighten us , if you have used parallel programming patterns in C# or .Net in general . Imagine I have 5 processes. How will start all the five processes at the same time. What does it mean to my consumption of resources ? I have tried reading into many of the material available over the net but I get more and more questions than them being the answer to my questions.

I used Parallel.For and created N threads and measured time. Then I tried the same thing using Task.Factory.start for enumeration of tasks. The time measured was different. So What exactly is the different between using these ? When I should use the corresponding classes and for what purposes exactly ? we often have lots of riches but its just we exactly don't know how to differentiate one from the another. This is one such case for me, not being able to find why I should not use one from the another.

I used the stopwatch class to measure these times which claims to be the best. In the scenario of me load testing a component, what would be the way to measure the time of response. Stopwatch seems to be the best solution for me . Any opinions are welcome.

ps: There are many load testing tools for web applications. Mine is a customised case of server components. And my question is more pertaining to creating N threads per second.

All opinions are welcome. Just don't think its so much not a programming question. It ofcourse is. It should ring bells for any programmer who wants to QE stuff by himself to know the performance of his product , first hand by himself.I have tried many options and then had to fall back on how should I actually do it ?

答:
Don't bother with threads, if N is reasonably small. To generate N requests per second, use the wall clock time (DateTime.Now). Take the time both before and after the request, then add a Sleep to delay the next request.

For instance, with N=5 (200 ms):

```<code>Before request: 12:33:05.014
After request: 12:33:05.077
Sleep(137)
Before request: 12:33:05.214
After request: 12:33:05.271
Sleep(131)
</code>```

This is not perfect; you may find that Sleep isn't exact. You can keep a running count of deviations (before the X'th requests, the time should be X-1/N later) and adjust the Sleep period accordingly.

Once N becomes too large, you simply create M threads and let each thread generate N/M requests in the same fashion.

OwnerDisplayName: King

Body:
Don't bother with threads, if N is reasonably small. To generate N requests per second, use the wall clock time (DateTime.Now). Take the time both before and after the request, then add a Sleep to delay the next request.

For instance, with N=5 (200 ms):

Before request: 12:33:05.014
After request: 12:33:05.077
Sleep(137)
Before request: 12:33:05.214
After request: 12:33:05.271
Sleep(131)

This is not perfect; you may find that Sleep isn't exact. You can keep a running count of deviations (before the X'th requests, the time should be X-1/N later) and adjust the Sleep period accordingly.

Once N becomes too large, you simply create M threads and let each thread generate N/M requests in the same fashion.


答: Tests are there to ensure that you don't (re-)introduce problems. The list of failing tests isn't a substitute for a bug tracking system.  There is some validity in the POV that failing tests aren't only indication of bugs, they are also an indication of development process failure (from carelessness to badly identified dependency).</p>


答:
Is it possible to write software in a similar way?

Yes, it is. Just be as careful as if you are developing hardware, test everything you can, and your software will be of similar quality.

by the way, haven't you heard of HW bugs? Much nastier then any SW bug and harder to fix (not just upgrading the software)


答:
The four elements you have identified are also the only ones I can think of.

The domain name, or rather, the TLD which is probably what you mean, can be used for a positive identification, but not for a negative one, because it may be .com, in which case it tells you nothing.

The content language (as specified in the headers) is the best indicator. That's also what your crawler probably wants to know: what language is the content that it crawls, not what country the operator of the web site happens to reside in.

The server's IP address is probably next to useless, but if all else fails, you could use it as an additional means for positive only identification.  Never use it for negative identification though, because lots of people host their web sites in the USA, regardless of where they come from or what their audience is.

Whois information is also next to useless, because it will usually tell you where the registrar is located, which really means nothing. And in any case, is probably going to be quite hard to get and parse any meaningful data out of it.

Title: Website Country Detection

Body:
I have a web crawler, and I'm looking for hints that will help me automatically detect a website country of origin.

And by country of origin I generally mean the country the website is targeting. For example:

http://www.spiegel.de/ -> Germany

http://www.lemonde.fr/ -> France

http://publico.pt/ -> Portugal

http://www.elpais.es/ -> Spain

I know there's not a foolproof way of doing it, so I will likely rely on a scoring system.

The domain name;

The content language;

The server's IP address;

Whois information;

What additional parameters would you use?

For the previous examples a combination of domain and content language will do, but many websites have a .com domain and a language spoken in more than one country...


问:
When and why should one create classes at runtime?
I have read many articles on the creation of dynamic types and classes at run time. For example, the TypeBuilder class in C# lets one create dynamic types. Python has this type function by which one can create dynamic classes. I have certainly good information on how I should create a class at run time, but I don't find enough information on why I should do this.

What is a scenario where one would be forced to create types at runtime?

答:
Mock frameworks like NMock (for generating mock classes in unit test scenarios) do this.

Another application I can think of may be a generic database tool generating classes at run time for each table in the DB (of course, there may be better solutions for this purpose).

In C#, every time when you have the requirement to construct a function at runtime (because, for example, the user is able to enter a function defined by himself in your application), you will need a dynamic class for that purpose since in C# functions cannot live "on their own".

And another application I know of are the dynamically created COM wrappers in C#4.0 for every COM component. Up to C# 3.5, you needed first to generate some type library wrapper which had to be compiled first before you could use it. With C#4.0 and the dynamic keyword, this is not necessary any more, the wrapper can be constructed at run time by the framework and you can write code which calls methods of those classes.

ContentLicense: CC BY-SA 4.0


问:
Are enforced Professional Development Plans useful?
Many companies, especially larger ones have mandatory professional development plans for their employees. Employees and managers set out a personalized professional development plan and follow up on the progress every so often.

As a developer, do you find such PDPs useful, do you follow through with your commitments?

As a manager, do you think such PDPs bring value to the company and improve the overall quality of the employees?

It seems that good developers will continue to educate themselves and strive to be better regardless of companies' procedures while bad ones won't.

Are there benefits of having PDPs or is it just something managers feel they need to do?

答:
Can't say I've ever been a fan of PDPs - and I've always worked in places that have some form of PDP-based process.

For a start, they are often tied to extrinsic motivators (e.g. bonuses), which will motivate some people for a while, but in the long run don't really work. This also leads to "gaming" of the PDP process, where people will set goals deliberately low in order to maximize financial gain with minimal effort. You personally might have the integrity to resist this - you might approach the process with the best intentions - but it sure is hard to motivate yourself to take the process seriously when everybody around you is just going through the motions.

Secondly, having your professional development tied into the goals of the company make it more difficult to explore areas that you want to explore but aren't core to the goals of the company. If you're lucky enough to have the same goals, this maybe isn't an issue - tho' bear in mind the company also has financial goals, which you may or may not share (or at least, may not prioritize the same way).

Finally, you mentioned it above, so +1 - it's so much better to take responsibility for your own professional development. Be in the driver's seat instead of waiting for the bus to come :p


答:
TypeBuilder and other entities of System.Reflection.Emit namespace are used to create Intermediary Language (IL) code. The first usage which comes to mind is for the compilers, but  I've seen other scenarios where simple Reflection wasn't enough, and you had to use Emit in your code (sadly, I don't remember where).

Note that it's not the same thing as dynamic word in .NET Framework 4, which is useful when you interact with dynamic languages like JavaScript.

Title: When and why should one create classes at runtime?

Body:
I have read many articles on the creation of dynamic types and classes at run time. For example, the TypeBuilder class in C# lets one create dynamic types. Python has this type function by which one can create dynamic classes. I have certainly good information on how I should create a class at run time, but I don't find enough information on why I should do this.

What is a scenario where one would be forced to create types at runtime?


答:
You should first define what you mean by "country of origin", and then base your detection on that. It could mean any of:

country (or countries) where the physical server is located

country from where the content is uploaded and maintained (physical location of authors)

country where the domain name is registered

country at which the top-level domain hints (e.g., .fr sites are supposedly "from France")

country that the website targets (e.g., lots of .nu sites target a Dutch audience)

country associated with the primary language or languages used in the site's content

Each meaning calls for different detection methods, some accurate, others more a matter of guesswork.

Title: Website Country Detection


答:
Possible scenario:

some kind of dynamic cache for storing long heavily interlinked dataset which is for some reason represented as individual objects per entry?


答:
Try this approach. Start with your initial list of parameters, then start collecting the data. After you have hundreds of sites in your database look for ones that don't seem to make sense. Then start looking for other clues to solve those outliers.

In other words, use the scientific method.

Title: Website Country Detection


问:
Handling invoices with timestamps
I'm currently going to write something to automatically create invoices with cronjobs by using PHP and timestamps.

I have a, for me, well-considered idea of how to solve it, but I want to ask the community if someone may see errors in reasoning and possible problems with my solution.

I'm trying to describe my idea as detailed as possible so everyone can follow my way of thinking:

In General there are 4 types of invoices:

Paid yearly

Paid semiyearly

Paid quarterly

Paid monthly

Purchased products are saved in a SQL database with the billing cycle:

ID of User

Product ID

Billing Cycle

Last Due Date

Now there is a cronjob that runs once a day to check if it should create a new invoice for each purchased product. In the row Last Due Date I save the timestamp of the first date to pay when it's created.

A code I already wrote calculates the time that has gone by since the Last Due Date timestamp and outputs something like this:

Timestamp is in past or in future

Month gone by

Days gone by

Now my rules for creating a new invoice are:

Paid yearly

if ( Timestamp is in past = true AND Month gone by = 11 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid semiyearly

if ( Timestamp is in past = true AND Month gone by = 5 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid quarterly

if ( Timestamp is in past = true AND Month gone by = 3 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid monthly

if ( Timestamp is in past = true AND Month gone by = 0 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

As you can see a new invoice would be created ~10 days before date of payment and the timestamp in Last Due Date is set to the current time, so when the cronjob checks back the next day no invoice will be created for this purchased product.

My question is if this is an appropriate way of solving this and if you can see any errors in reasoning or problems that may occur?

答:
The biggest thing that I see is missing is keeping track of what the user has actually paid. Some people pay an amount more or less than the amount due, so some may have a balance due from the last invoice while others may have pre-paid for several periods ahead.

EDIT: Based upon your comment, I see this is handled seperately, great!

The other thing I see is the way you are handling the TIMESTAMP field. For example:

Paid monthly

if ( Timestamp is in past = true AND Month gone by = 0 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Suppose I initially signed up on 1/1/2012, then Timestamp starts off with that date. Assuming I pay monthly, this will mean you generate an invoice on 1/20/2012 and set the Timestamp to 1/20/2012. Does this mean you generate an invoice every 20 days rather than once per month? In other words will you generate an invoice on 2/9/2012 (20 days after 1/20/2012)?

The point is that the next invoice should be generated based upon the end date of the current billing period, not the date the invoice was generated.

Currently you "set "Last Due Date" to time()", perhaps you want to set Last Due Date to the first day of the next billing period? So for example when generating monthly invoices on 1/20/2012 you would change 1/1/2012 (current value in DB) to 2/1/2012. You don't state which database you are using, but many have built-in functions to add 1 month, quarter, year etc.

Title: Handling invoices with timestamps


问:
Distributed transactions and queues, ruby, erlang
I have a problem that involves several machines, message queues, and transactions.  So for example a user clicks on a web page, the click sends a message to another machine which adds a payment to the user's account. There may be many thousands of clicks per second. All aspects of the transaction should be fault tolerant.

I've never had to deal with anything like this before, but a bit of reading suggests this is a well known problem.

So to my questions. Am I correct in assuming that secure way of doing this is with a two phase commit, but the protocol is blocking and so I won't get the required performance?  It appears that DBs like redis and message queuing system like Rescue, RabbitMQ etc don't really help me a lot - even if I implement some sort of two phase commit, the data will be lost if redis crashes because it is essentially memory-only.

All of this has led me to look at erlang - but before I wade in and start learning a new language, I would really like to understand better if this is worth the effort.  Specifically, am I right in thinking that because of its parallel processing capabilities, erlang is a better choice for implementing a blocking protocol like two phase commit, or am I confused?

答: I've used Redis with message queues for a small 24-node distributed system. It is very nice, high performance, and easy to use, with lots of language bindings. I use it with Python and PHP. It also has a feature that serializes every operation to disk, and in case of a power outage you can replay the log. Maybe you could try prototyping a solution to your problem and see if it does the job. The learning curve is not as steep as Erlang. :-)</p>


问:
What are the things that you consider in reviewing an iOS source code?
We're going to do some peer reviews for our iOS projects. We're kind of inexperienced at reviewing each other's codes. The iOS developers attending the reviews will be from other projects, since generally one person handles one client project in the company.

I know there are some questions on SO about the things you're looking for when you're reviewing someone else's code. I want to ask this question specifically for iOS projects. I mean, besides the general points for code quality, what are the things that you consider when reviewing an iOS code?

答:
What happens in a code review doesn't depend on the type of project. The general things you look for are the same in iOS projects as Android projects as the embedded systems code for avionics. However, the difference is in the structure. Mission and life critical code typically has more structured reviews involving more people and more time. Non-critical code might be reviewed using something as simple as an over-the-shoulder check or another person just skimming the code and providing comments.

The first thing to address is what you are going to review, which is addressed in another question here on Programmers. In theory, you want to review everything. In practice, you need to prioritize what gets reviewed based on a number of factors ranging from the likelihood that a module contains defects (based on the complexity of the logic) to the amount of use a code segment has (if it's used often, it's probably more important to review, since a defect there would be exposed in more places).

The things that you are looking for in a code review are those things that can't be found through automated means. Check for code style, logic errors that might not be tested for, suspicious boundary conditions, race conditions, off-by-one-errors. Code reviews can also encompass the test cases, and you can make sure that the tests that have been written are value-adding, meaning that you don't have multiple tests covering the exact same conditions and that tests appropriately cover edge cases.

You might also be interested in two questions about what a code review looks like and effective strategies for performing a code review.


答: There are high level tools with lots finished 'bricks', as you call them, that you can combine into applications. The bricks are finished pieces for you to use, you just have to combine them together. Maybe you think that's easier... until your customer asks you for some weird and unexpected changes.</p>


答:
If your goal is to bill daily, not sure you have a choice. You have 365 billing periods per year. With automation, this isn't a problem.

Seems like your method for creating invoices should work (good catch by @jonnyboats to not use the date the bills are created as the bill date). Since you're saving the Last Due Date, if you ever have to make a correction to an invoice (especially the date), you have to make sure this date is updated after any adjustment. You could query this date based on the last invoice date for this client's item in your invoices, so any adjustment would be considered. Your method improves performance.

Give yourself the ability to set a Run Date parameter (default to time() ). If last night's run failed, you want the ability to run it manually by setting the time to yesterday. Weekends and holidays have their own set of issues.

Title: Handling invoices with timestamps

Body:
I'm currently going to write something to automatically create invoices with cronjobs by using PHP and timestamps.

I have a, for me, well-considered idea of how to solve it, but I want to ask the community if someone may see errors in reasoning and possible problems with my solution.

I'm trying to describe my idea as detailed as possible so everyone can follow my way of thinking:

In General there are 4 types of invoices:

Paid yearly

Paid semiyearly

Paid quarterly

Paid monthly

Purchased products are saved in a SQL database with the billing cycle:

ID of User

Product ID

Billing Cycle

Last Due Date

Now there is a cronjob that runs once a day to check if it should create a new invoice for each purchased product. In the row Last Due Date I save the timestamp of the first date to pay when it's created.

A code I already wrote calculates the time that has gone by since the Last Due Date timestamp and outputs something like this:

Timestamp is in past or in future

Month gone by

Days gone by

Now my rules for creating a new invoice are:

Paid yearly

if ( Timestamp is in past = true AND Month gone by = 11 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid semiyearly

if ( Timestamp is in past = true AND Month gone by = 5 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid quarterly

if ( Timestamp is in past = true AND Month gone by = 3 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

Paid monthly

if ( Timestamp is in past = true AND Month gone by = 0 AND Days gone by >= 20 )

then ( create a new invoice and set "Last Due Date" to time() )

As you can see a new invoice would be created ~10 days before date of payment and the timestamp in Last Due Date is set to the current time, so when the cronjob checks back the next day no invoice will be created for this purchased product.

My question is if this is an appropriate way of solving this and if you can see any errors in reasoning or problems that may occur?


问:
Reengineering the project from scratch

Possible Duplicate:

  When do you rebuild an application or keep on fixing the existing one

I am currently working on a project that has been in development for the last few years used throughout the organization but the way the project has been coded the maintainability of it is completely shot.

Reading the code presents with pages and pages of Anti-Patterns and trying to identify the path of a business workflow takes on occasion days.  At this point I would probably classify the software in its current state as "Working by accident" rather then as intended.  So I am looking for some wisdom as to the following:

At what point would you consider simply dumping the project into an abandonware pile and starting from scratch?

P.S.  I understand that in a lot of cases organization would consider it cheaper to maintain the existing project.

答:
You almost certainly should not rewrite, it is very unlikely to make business sense. Joel Spolsky puts it much better than me though: http://www.joelonsoftware.com/articles/fog0000000069.html

Dealing with this code is a reality of software engineering that you may have to learn to accept, but don't dismay, you aren't the first and there are tools and methods for dealing with this problem, I highly recommend Working Effectively with Legacy Code:

http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052

Title: Reengineering the project from scratch


答:
"Break the build" means to prevent a build from completing successfully.  A failing test doesn't do that.  It is an indication that the build has known defects, which is exactly correct.

Most build servers track the status of tests over time, and will assign a different classification to a test that's been failing since it was added vs a regression (test that used to pass and no longer does), and also detect the revision in which the regression took place.


答:
At what point would you consider simply dumping the project into an abandonware pile and starting from scratch?

When the actual cost of maintenance exceeds the value to the organization.

That is, it's cheaper to do things manually than it is to continue using (and maintaining) the software.

This is actually an easy test to apply.  How  many users?  How much time do they spend using the app?  What's the alternative? How much time do you waste on it? If you've only got a few dozen users, you can list the hours-per-day usage to determine the actual cost.

Value is sometimes harder to discern, but it should do something of value, and someone should be able to say that the software saves (or makes) some amount of money each year.

It's best -- of course -- do make the decision before the software actually becomes a liability instead of an asset.

Title: Reengineering the project from scratch

Body:
Possible Duplicate:

  When do you rebuild an application or keep on fixing the existing one

I am currently working on a project that has been in development for the last few years used throughout the organization but the way the project has been coded the maintainability of it is completely shot.

Reading the code presents with pages and pages of Anti-Patterns and trying to identify the path of a business workflow takes on occasion days.  At this point I would probably classify the software in its current state as "Working by accident" rather then as intended.  So I am looking for some wisdom as to the following:

At what point would you consider simply dumping the project into an abandonware pile and starting from scratch?

P.S.  I understand that in a lot of cases organization would consider it cheaper to maintain the existing project.


答:
You're going to have to figure out what the code does anyway. I doubt there is any clear and up to date specs anywhere. Just fix it.

Unless it is written in a language that you can't find a compiler/interpreter/run-time to run it on the availble hardware, fix it.

You may have the resources to create a new app, but the business does not.

I don't know how many change requests you have, and per change it will take longer along with some refactoring, but that will not exceed the time for devvelopment and business staff resources.

Fix what is broken. Refactor as much as you can in the mean time.

Title: Reengineering the project from scratch


问:
Flat File in a Unit Test
Is it considered bad practice to use a flat file when doing unit tests? If an application reads in flat files and parses them, would this be a better approach than hard coding strings into the unit tests themselves?</p>

答:
The advantage of flat files is that you can put many data there to feed the tests. The Django unit test framework works with fixtures, files with data for that purpose (in this case it use json or xml).

Another advantage is that your test code is written for taking the data from that file and is ready for testing synthetic data generated by hand (by now) or by machine, so you could then drop that file, save the script that generate the fixture and you don't have to change your test.

Title: Flat File in a Unit Test


问:
Why is 'void' not allowed as a generic type in C#
What were the design decisions that argued in favour of void not being constructable and not being allowed as a generic type? After all it is just a special empty struct and would have avoided the total PITA of having distinct Func and Action delegates.

(C++ allows explicit void returns and allows void as a template parameter)

答:
From: http://connect.microsoft.com/VisualStudio/feedback/details/94226/allow-void-to-be-parameter-of-generic-class-if-not-in-c-then-just-in-runtime

This is actually by design - we don't allow any instance of the void
  type (along with many other types in the runtime). You can't
  CreateInstance a void or void[] either. We plugged these type holes
  for security.

I can't for the life of me see how void is a security hole, but... such it says.


问:
Is it a good idea to use an unit test framework for another purpose than test code?
I'm about to write a simple script to test a dataset for certain conditions. I was designing it as a set of functions each one describing the condition to be tested and pass them to the test engine:

```<code># The tester engine:
all(f(dataset) for f in conditions)
</code>```

I realized that my approach was similar to unit testing. So, to avoid repeating myself, I am thinking of using my favorite unit testing framework instead.

What do you think about that idea?

Have any of you used an unit test framework for any other purpose than test code?

答:
How are you going to test the main code if is already wrapped in the unit test framework?

Any xUnit testing frameworks will report pass/fail states and do nothing about the result.  If that is what you want, go for it.  If you are going to use that data for anything, then do not use unit test.  For example, you cannot attempt to "fix" the dataset such as, say, converting two-digits years into four digit years.


问:
What is the best way to structure workshops for starting programmers?
We are planning to organize programming workshops for Java web developers with basic coding but no design experience. The goal of the workshops is to introduce these programmers to clean code.

Focus Areas:

TDD

Four Rules of Simple Design

runs all the tests

contains no duplications

expresses the intent of the programmers

minimizes the number of classes and methods

Structure:

6 full day coached workshops spread over 3 months using pair programming and pair-switching

Each workshop contains

four 15 minute presentations/demos

four 90 minute problem solving exercises

4 minute pecha kucha's by every programmer

Planned Katas for first workshop:

prime numbers

bowling game

tennis scoring

Arabic to Roman numerals conversion

prime factors

Do you think this is a good way to structure a programming workshop? Are there any elements that I could introduce to the workshops to make them more interactive and create more interest among developers?

Edit 1:
Summary of the feedback

Create an interest in reading books.

Have real project examples.

Have contact with the group during off days.

答: You might also try dividing the group into sub-groups, say 4 groups of 4, give everyone the same basic programming task, then shuffle the code to another group and have every group add some additional functionality. It will show them the importance of writing maintainable code better than any long-winded lecture.</p>