问:
Which industries use the most high-quality, measurable, reliable approach to software?
What sections of the software industry (i.e. healthcare, aerospace, games, etc) are most likely to practice software engineering with the engineering rigor + depth and breadth defined in the SWEBOK? </p>

答:
Other than at the highest levels of safety criticality, the incremental costs of developing in accordance with SWEBOK/ISO12207/etc are negligible over not - providing that you embrace the standards as they are intended to be adopted.

Too often, the paperwork is done as an afterthought, therefore becomes pure-and-simple overhead.

But to directly answer the question (as its stands now) any regulated software (therefore aerospace, medical and automotive, as examples) should be standards compliant - and increasingly they are.

Standards compliance is clearly more relevant when the through-life cost (as opposed to short-term outlook) is considered, although there is still a certain amount of seemingly unnecessary box-ticking (for audit and should the air-con get polluted, retrospective enquiry purposes)

So-called "commercial" development appears to be less so rigorous, although the regulatory aspects of (eg) financial services may well see this having to change.

However, as Blrfl commented, standards are only there to provide process/procedural and/or technical guidelines: and the process is only ever as good as the information available, and most importantly the quality/capability of the people actually doing the work - and their understanding of the task in hand.

An important aspect of most standards is that they deal with the Competence of staff - and such aspects as training and qualification.  This is one area that many companies neglect.


问:
In the world of .Net, managed code and the web is there still a place for VBA?
Microsoft has moved away from the COM stack, VB6 is so last century and .Net rules the (MS) roost. Yet I find myself still banging out reams of VBA code am I dinosaur? The need for VBA is a new project automating Excel seeing as you ask.

I've tried to doing the same kind of thing using VSTO and it was just too damn buggy/hard/inefficient with a broken development model. I can't get rid of the feeling that I'm missing something, OTOH I really can't see a better way of solving this problem. What are your thoughts?

答:
I have lived with this problem for the last few years. VBA is pretty much dead, yet the VSTO model for me was just like you said: buggy, inflexible, and slow.  So I saw it as a choice of getting the job done fast and effectively yet letting my professional skills stagnate, or getting the job done very painfully yet developing skills that will be around in the future.

My choice was the latter (VSTO), that is until Microsoft began changing their overall strategy starting about a year ago from proprietary only development (.NET), to embracing the rest of the development community (C++, javascript, html, etc).  At that point the VSTO model of the future didn't seem so concrete.  So I said screw it, went back to fast and easy VBA.

A few months later and my choice paid off: Microsoft announced Office 2013 "apps" or automation can be built in javascript.  By looking at the other trends (Typescript, asp.net embracing js, html) it looks like this might be microsoft's preferred language for office going forward.  It also looks like they won't be investing much in the VSTO platform moving forward.

So tl;dr: if you're primarily an office application developer, stick to vba, learn js when you get time and wait till your company upgrades to Office 2013.

Update:

So Office 2013 shipped and the Javascript api it came with is severely lacking in functionality.  In it's current state, I don't believe it can replace VBA except for the most rudimentary applications.

Update August 2015:

JS Office Apps have been out for a few years.  During this time I've been waiting for the Api to expand to the same level of functionality as what we have with VBA/VSTO.  While I have heard rumors that Office 2016 will bring more api functionality to the Office Apps ecosystem, the api has not really evolved to support your standard LOB automation applications.  From what Microsoft has released so far (including their newly revamped Office API's) it appears they aren't focused on this.  Speculating here, but it appears Microsoft sees Office Apps really like an interface to the web/cloud/online store.  Office Apps are not meant to be an application so much as to provide a window to an application.  The focus is also on user interactivity, as opposed to automation.  This is fundamentally different than the VBA/VSTO model, which treats Office as the application, it's own self contained application. I don't expect Apps to support the use cases of VBA/VSTO anytime soon.  I remember reading Microsoft wants heavy office automation to die out, probably due to the security risks among other things.  I use VSTO for Office automation now.

ContentLicense: CC BY-SA 2.5

ContentLicense: CC BY-SA 3.0


问:
How to write junit test case for the program
I have a method which has external dependencies within the method. Please help me in writing a junit test case for the below program.

```<code>    @Override
    public void methodToTest(String user){

      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      Service service = new Service(url, username, password);

      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }
</code>```

Note:

The method overrides the superclass method so that I cannot change the
arguments to contain the url, username and password.

The method also connects to an external service. which is environment specific. The
environment details is taken from the properties file.

答:
If it appears difficult to test a method, I suggest attempting to test the components of the method instead.

It looks like your code would benefit from a little bit of refactoring.

It appears that your method has too much responsibility. Try breaking the method down into smaller chunks.

I suggest taking a look at what actions are being enacted at each stage of the method, as opposed to testing the method as a whole.
By breaking down the method into smaller chunks you ensure that you are getting good test coverage and can easily drill-down into finding out which parts (if any) break.

Based on the small snippet you have provided and my assumptions of what you are trying to do, responsibilities include:

Retrieval of the login details,

Connecting to a service

Adding a user.

These three different responsibilities should be tested independently and where one fails, it should be clear at which steps (if any of these) it failed.

```<code>public myClass extends myFoo {
    Service service;

    public myClass() {
      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      service = new Service(url, username, password); //Error handling??
    }

    @Override
    public void registerUser(String user){
      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }
}
</code>```

Take a look at the above example. There are a few ways you can play with it and without knowing more I'm not sure which would be best for you, but essentially the method that registers a user doesn't "care" about how the service is created or what the login details are - all it needs is a service to add the given user to.

I question whether the service needs to be a field and/or contained in the constructor - there's no reason why it shouldn't be moved into a method of its own:

```<code>public myClass extends myFoo {
    @Override
    public void registerUser(String user){
      Service service = getService();

      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }

    //Would be contained within the superclass (eg if Config shared between all subclasses), else overridden via the subclass (eg if each subclass had a different Config)
    protected Service getService() { //Error handling? eg throws InvalidCredentialsError? throws NetworkConnectionError? etc....
      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      service = new Service(url, username, password);

      /**
      if(service == null) {
        throw new InvalidCredentialsError("The given url, username and password are invalid.");
      }
      **/

      return service;
    }
}
</code>```

Based my assumptions of the responsibilities of the code, I would then break down your method into three (possibly four) sets of tests as follows:

The first "action" is the retrieval - thoroughly test the Config.getProperty().

If you are confident that this method works correctly, you may assume that the url/username/password values will be assigned correctly. Presumably there is also a Config.setProperty method - for these tests to pass, whatever value you pass into setProperty must be identical to the value you retrieve via Config.getProperty().

The next "action" is to setup the Service().

Test all variations of known correct/known incorrect (and potentially known malicious, see injection flaws)/empty string/null values for url/username/password.
Without knowing more about the application I am not sure what the success/failure criteria are (perhaps an exception is thrown when an invalid url/username/password are passed into the constructor, but this begs the question of why it isn't being thrown/caught in the code currently visible).

The final "action" is to attempt the addition of a user.

Looking at the method, I assume that the core functionality of the method you are attempting to test is the addition of a user. Assuming that set of tests #1 and set of tests #2 have passed, this should be as straightforward as passing in values that you expect to succeed (eg "Jim"), values that you expect to fail or aren't sure about (null, empty string, extremely long strings?? etc) and ensuring that the value you expect to come back does actually come back.

What I notice is that you state the service is an external service.

If you trust the external service to have tested their code thoroughly, you may not need this step (in which case, skip straight to #4).

A potential final test is to test the method itself.

This would probably be identical to set of tests #3, but instead of testing the return value of the registerUser() method, you would be testing to see if the appropriate exception(s) are being thrown as opposed to testing the return value(s).

The next step is for you to write the actual JUnit test classes ;)

Body:
If it appears difficult to test a method, I suggest attempting to test the components of the method instead.

It looks like your code would benefit from a little bit of refactoring.

It appears that your method has too much responsibility. Try breaking the method down into smaller chunks.

I suggest taking a look at what actions are being enacted at each stage of the method, as opposed to testing the method as a whole.
By breaking down the method into smaller chunks you ensure that you are getting good test coverage and can easily drill-down into finding out which parts (if any) break.

Based on the small snippet you have provided and my assumptions of what you are trying to do, responsibilities include:

Retrieval of the login details,

Connecting to a service

Adding a user.

These three different responsibilities should be tested independently and where one fails, it should be clear at which steps (if any of these) it failed.

public myClass extends myFoo {
    Service service;

    public myClass() {
      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      service = new Service(url, username, password); //Error handling??
    }

    @Override
    public void registerUser(String user){
      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }
}

Take a look at the above example. There are a few ways you can play with it and without knowing more I'm not sure which would be best for you, but essentially the method that registers a user doesn't "care" about how the service is created or what the login details are - all it needs is a service to add the given user to.

I question whether the service needs to be a field and/or contained in the constructor - there's no reason why it shouldn't be moved into a method of its own:

public myClass extends myFoo {
    @Override
    public void registerUser(String user){
      Service service = getService();

      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }

    //Would be contained within the superclass (eg if Config shared between all subclasses), else overridden via the subclass (eg if each subclass had a different Config)
    protected Service getService() { //Error handling? eg throws InvalidCredentialsError? throws NetworkConnectionError? etc....
      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      service = new Service(url, username, password);

      /**
      if(service == null) {
        throw new InvalidCredentialsError("The given url, username and password are invalid.");
      }
      **/

      return service;
    }
}

Based my assumptions of the responsibilities of the code, I would then break down your method into three (possibly four) sets of tests as follows:

The first "action" is the retrieval - thoroughly test the Config.getProperty().

If you are confident that this method works correctly, you may assume that the url/username/password values will be assigned correctly. Presumably there is also a Config.setProperty method - for these tests to pass, whatever value you pass into setProperty must be identical to the value you retrieve via Config.getProperty().

The next "action" is to setup the Service().

Test all variations of known correct/known incorrect (and potentially known malicious, see injection flaws)/empty string/null values for url/username/password.
Without knowing more about the application I am not sure what the success/failure criteria are (perhaps an exception is thrown when an invalid url/username/password are passed into the constructor, but this begs the question of why it isn't being thrown/caught in the code currently visible).

The final "action" is to attempt the addition of a user.

Looking at the method, I assume that the core functionality of the method you are attempting to test is the addition of a user. Assuming that set of tests #1 and set of tests #2 have passed, this should be as straightforward as passing in values that you expect to succeed (eg "Jim"), values that you expect to fail or aren't sure about (null, empty string, extremely long strings?? etc) and ensuring that the value you expect to come back does actually come back.

What I notice is that you state the service is an external service.

If you trust the external service to have tested their code thoroughly, you may not need this step (in which case, skip straight to #4).

A potential final test is to test the method itself.

This would probably be identical to set of tests #3, but instead of testing the return value of the registerUser() method, you would be testing to see if the appropriate exception(s) are being thrown as opposed to testing the return value(s).

The next step is for you to write the actual JUnit test classes ;)


问:
How do you explain refactoring to a non-technical person?
How do you go about explaining refactoring (and technical debt) to a non-technical person (typically a PHB or customer)? ("What, it's going to cost me a month of your work with no visible difference?!")

UPDATE Thanks for all the answers so far, I think this list will provide several useful analogies to which we can point the appropriate people (though editing out references to PHBs may be wise!)

答:
All the analogies to things in the physical world -- like building a theater -- are, IMO, terrible.

You need to explain that refactoring code is like... refactoring code. Software is malleable in ways that physical analogs are not. As things get more and more complex, one must reactor (or redo, as you wish) massive or small parts of a codebase so that we can continue to increase the complexity without going insane.

Why do we refactor? Because code that is never refactored costs more per minute to maintain and change, and ultimately becomes more problematic.

What's so interesting about refactoring is that we redo the codebase but, at least at the outset, the functionality remains the same.


问:
What does it mean that StAX parsers aren't complete or correct?
I've been browsing the net and found theses two slides on site of XOM (XML object model):

slide1

slide2

Where they refer to SAX(push) and StAX(pull) parsers. So I'm wondering, what does it mean that a StAX parser isn't complete or correct in practice?

答:
I believe the comment was aimed more generally at the pull-based XML APIs and parsers of the time, not just StAX and its implementations. At the time the slide deck was written (over ten years ago!), there were known deficiencies with XMLPULL API design regarding the handling of namespaces and ill-formed documents – in fact, it was Rusty Harold himself who reported them! Also, StAX was in its infancy (JSR 173 did not hit final release until 2004), and other implementations were apparently not yet considered mature.

Ten years is a long time in the software world, and things may well have changed in the intervening time. If you want to know what the current state of the art is for a particular parser or set of parsers, you should ask about it. See for example this much more recent exchange, and questions with the stax tag, on Stack Overflow.


问:
tips, guidelines, points to remember for rendering professional code?
I'm talking about giving clients professional looking code. The whole nine yards, everything you hardcore professional highly experienced programmers here probably do when coding freelance or for the company you work in.

I'm fresh out of college and I'm going into freelance. I just want to be sure that my first few projects leave a good after-taste of professionalism imprinted on the clients' minds.

When I Googled what I'm asking here, I was given pages that showed various websites and tools that let you make flashy websites and templates etc. The $N package and such stuff.

I can't recall the word experts use for it. Standard, framework [I know that's not it]. English isn't my first language so I'm sorry I don't really don't know the exact phrase for it.

That abstract way of writing code so that you don't come across as a sloppy programmer.

That above mentioned way for programming websites and desktop software [in python/C/C++/Java].

EDIT: I can work on the accruing vast knowledge and know-how and logic building etc. what I'm asking for is the programming standard/guidelines you guys follow so that the client on seeing code feels that its a professional solution. Like comment blocks, a particular indentation style something like that. Is there any book on it or specific list of points for enterprise type coding by them? Especially here as in my case, for building websites [php for now..], and desktop software [c/c++/java/python]

答:
Read Code Complete by Steve McConnell. Apply it's principles on every line of code, literally.

My best "ah-ha" moments in coding excellence have been when I reworked particularly bad code and had a striking before and after picture, so to speak.

You will learn that applying coding principles is a synergistic balancing act, that the most trivial guidelines matter, and that code crumbles one poor, lazy edit at a time.


问:
Why would I use Control.Exception in Haskell?
I'm trying to really master Haskell error handling, and I've gotten to the point where I don't understand why I would use Control.Exception instead of Control.Monad.Error.

The way I can see it, I can catch Control.Monad.Error exceptions in both pure and impure contexts.  By contrast, exceptions defined in Control.Exception can be caught only in impure/IO contexts.

Given that they provide two separate interfaces and semantics that I would need to memorize, under what circumstances would I need Control.Exception instead of Control.Monad.Error?

答:
Control.Exception is designed for use:

in otherwise pure code where the addition of an error-related monad or other error reporting structure would be too cumbersome

in cases where one thread or process needs to raise an asynchronous exception in another process, outside of the type system's constraints.

For an example of the first, take division by 0 in the Int type (e.g. div 3 0). You can't give a reasonable answer in that case, but you don't want your divide operator to return a Maybe Int – it could make programmers miserable! So you could use an exception to stop the computation.

In this situation, you could also use error for a similar effect. The difference is that Control.Exception gives you more flexibility for catching and reporting what happened. But such exceptions can only be caught in the IO monad, which could place a burden on your client. Therefore, I'd suggest you save these for truly exceptional, perhaps catastrophic situations.

The stuff in Control.Monad.Error works via a monad, and uses the monadic binding/sequencing machinery to pass the error along. It doesn't subvert the type system like throw does, so it has the virtues of purity and relative transparency going for it. It also gives you more flexibility to handle and recover from errors, and can also make it easier to pass errors back to the client if you're writing a library (see Parsec and polyparse, which both transform some monadic error + result into an Either value when they're done, though polyparse uses something a bit different from Control.Monad.Error). But of course you have to work within the error-handling monad, or wrap an error-handling monad transformer around whatever existing monads you might be working in. Both options can make your code considerably more complex.

If you haven't already, see Real World Haskell, Chapter 19 for a rundown of these and other error-handling options, and see the end of Chapter 18 for some of the more general drawbacks of using monads and monad transformers.


问:
Handling unexpected features during sprint
Our team is going to adopt scrum and agile technics. We've got a product, which we develop for multiple customers. This customers supplied us with necessary requirements, so everithing is fine to adopt agile techinics.

But at some moment (for example during sprint), new customer apears, and he wants to get a demo of product slightly different from what we have now. This may be some new features or minor differences in behavior. And he wants to get this demo for example during the week. It is very important to show that our product supports this features (because otherwise he will address to our competitors), so we have to develop this features (may be partly) during the week.

How we must handle this sort of features with agile? Move them to current sprint backlog? Or split one team to two and create another sprint? Or may be there is another way?

答:
If you are doing scrum, you should have a Product Owner, no?  It's his or her job to prioritize the stuff that comes along and manage the relationship with the customers.  That said, the product owner should not be adding items to a sprint unless the team agrees to it and that there are items that have not been started yet and can be pushed out of the sprint to compensate.

If the new customer results in a second product owner, that makes for a second project.  In that case you should form a second team and create a prioritized backlog for the new project.

If you are running into a lot of this, you might look at kanban as an alternative to scrum.


答: If you want to give your codebase a "professional" look, a good idea is to make it follow a particular set of published guidelines/conventions/standards. For example, a Java-based project might follow Oracle's Java Code Conventions. Use an IDE that has an automated code style formatter (e.g. Eclipse) to ensure no code ever gets committed that does not follow the chosen style of the project.</p>

Title: tips, guidelines, points to remember for rendering professional code?

Body:
I'm talking about giving clients professional looking code. The whole nine yards, everything you hardcore professional highly experienced programmers here probably do when coding freelance or for the company you work in.

I'm fresh out of college and I'm going into freelance. I just want to be sure that my first few projects leave a good after-taste of professionalism imprinted on the clients' minds.

When I Googled what I'm asking here, I was given pages that showed various websites and tools that let you make flashy websites and templates etc. The $N package and such stuff.

I can't recall the word experts use for it. Standard, framework [I know that's not it]. English isn't my first language so I'm sorry I don't really don't know the exact phrase for it.

That abstract way of writing code so that you don't come across as a sloppy programmer.

That above mentioned way for programming websites and desktop software [in python/C/C++/Java].

EDIT: I can work on the accruing vast knowledge and know-how and logic building etc. what I'm asking for is the programming standard/guidelines you guys follow so that the client on seeing code feels that its a professional solution. Like comment blocks, a particular indentation style something like that. Is there any book on it or specific list of points for enterprise type coding by them? Especially here as in my case, for building websites [php for now..], and desktop software [c/c++/java/python]


答:
There are times when the marketing promise a potential customer having a feature that is not implemented yet :-(

From mangement point of view the "new feature or customer will not buy" is similar to a "critical production bug that must be fixed and deliverd as soon as possible" .

For this emergency situation i suggest

stop the sprint and

stop working on the current development-branch and

create a new emergencybranch that originates from the last released-stable-fulltested-production-branch and

fix the bug (or in your case create the "new customer" feature)

do the testes, deployment, create patches,... (for bugfix only)

if every thing works ok this bugfix/feature is merged into the stopped development branch and the sprint is either resumed or canceld_with_new_sprint_planning.

Your management should be aware this workflow is costly and should only be used in real emergency situations.

Title: Handling unexpected features during sprint

Body:
Our team is going to adopt scrum and agile technics. We've got a product, which we develop for multiple customers. This customers supplied us with necessary requirements, so everithing is fine to adopt agile techinics.

But at some moment (for example during sprint), new customer apears, and he wants to get a demo of product slightly different from what we have now. This may be some new features or minor differences in behavior. And he wants to get this demo for example during the week. It is very important to show that our product supports this features (because otherwise he will address to our competitors), so we have to develop this features (may be partly) during the week.

How we must handle this sort of features with agile? Move them to current sprint backlog? Or split one team to two and create another sprint? Or may be there is another way?


答:
Spring cleaning.

You aren't doing any modifications to the house. You're just shifting stuff around and getting rid of some of the dust. Maybe throwing some things out that you don't use or need anymore. but you're not adding anything.

Title: How do you explain refactoring to a non-technical person?

Body:
How do you go about explaining refactoring (and technical debt) to a non-technical person (typically a PHB or customer)? ("What, it's going to cost me a month of your work with no visible difference?!")

UPDATE Thanks for all the answers so far, I think this list will provide several useful analogies to which we can point the appropriate people (though editing out references to PHBs may be wise!)


答:
Since the resources are external, in your case the Config and Service, it makes it a bit difficult to test. You basically have two options:

Go with the flow and set up the external resources under your control if possible

Refactor the code so that it is easier to mock the resources

Since the first one you clearly can't do, we'll go through the second one. Refactoring the code with each step fixing each dependency so that they are replaceable for the test:

Fixing Config dependency

I'm assuming that Config is a static resource, which means you need to be able to replace it somehow. Easiest is to add a new replaceable resource:

public interface IConfigResource {
    String getProperty(String propertyName);
}
```

<p>Together with a handy dandy default static version of it:</p>

<pre class="lang-java prettyprint-override"><code>public class DefaultConfigResource : IConfigResource {
    public String getProperty(String propertyName) {
        return Config.getProperty(propertyName);
    }
}
</code>```

In your class you need to add/change the following so that the dependency can be injected for test:

// The default is set at construction
private IConfigResource configResource = new DefaultConfigResource();

// Method used by the test to inject the config resource
public void setConfigResource(IConfigResource configResource) {
    this.configResource = configResource;
}
```

<p>Now we can either use Mockito or manually handroll a mocked version of <code>IConfigResource</code> to test that your method is calling the config resource correctly, in the test below we use Mockito:</p>

<pre class="lang-java prettyprint-override"><code>ClassUnderTest cut;
IConfigResource mockedResource;

@Before
public void setup() {
    cut = ... ; // setup for class under test

    // We mock with mockito and inject the resource
    mockedResource = mock(IConfigResource.class);
    cut.setConfigResource(mockedConfigResource);
}

@Test
public void callingMethod_ShouldGetValuesFromResource() {
    // Arrange
    // might change the return values to reflect the correct ones
    when(mockedResource.getProperty("serviceurl"))
        .thenReturn("correct service url");
    when(mockedResource.getProperty("serviceusername"))
        .thenReturn("correct service username");
    when(mockedResource.getProperty("servicepassword"))
        .thenReturn("correct service password");

    // Act
    cut.methodToTest("username");

    // Assert, with mockito
    verify(mockedResource).getProperty("serviceurl");
    verify(mockedResource).getProperty("serviceusername");
    verify(mockedResource).getProperty("servicepassword");
}
</code>```

There, the Config part of the method should now be tested

Fixing Service dependency (and removing Config dependency as a bonus)

So we need something that needs to handle this part of the code:

//Connect to an external service.
Service service = new Service(url, username, password);

//invoke the method on the service.
String returnValue = service.registerUser(user);
```

<p>First the service needs to have a replaceable interface in order for us to be able to test that your method is calling <code>service.registerUser(String)</code>. This should be easy:</p>

<pre class="lang-java prettyprint-override"><code>public interface IService {
   String registerUser(String userName);
}
</code>```

Your current service can implement this interface with ease, just add it to the class signature (since it should have the method already):

public class Service implements IService {
...
```

<p>We also need something (a factory method of sorts) that can connect to the service and return something that looks like an <code>IService</code>. We can also add the config resource we've made above to make it a bit easier:</p>

<pre class="lang-java prettyprint-override"><code>public interface IServiceConnector {
    void setConfigResource(IConfigResource configResource);

    // no need to have parameters, the config is injected above
    IService getService();
}
</code>```

The default implementation is easy, we can use the ConfigResource we used before:

public class DefaultServiceConnector {
    IConfigResource configResource = new DefaultConfigResource();

    public void setConfigResource(IConfigResource configResource) {
        this.configResource = configResource;
    }

    public IService getService() {
        String url = Config.getProperty("serviceurl");
        String username = Config.getProperty("serviceusername");
        String password = Config.getProperty("servicepassword");

        return new Service(url, username, password);
    }
}
```

<p><strong>You can now move the test written above to be used as test for this connector class where the method under test is <code>connectService()</code></strong>. Furthermore your original method under test should be easier now:</p>

<pre class="lang-java prettyprint-override"><code>IServiceConnector serviceConnector = new DefaultServiceConnector();

public void setServiceConnector(IServiceConnector serviceConnector) {
    this.serviceConnector = serviceConnector;
}

@Override
public void methodToTest(String user) {
  // ... Removed some code ...

  //Connect to an external service.
  Service service = serviceConnector.getService();

  //invoke the method on the service.
  String returnValue = service.registerUser(user);

    if (returnValue.equals("failure")){
      throw new Exception("User could not be registered");
    }
}
</code>```

The test should look something like this now:

ClassUnderTest cut;
IServiceConnector mockedConnector;
IService mockedService;

@Before
public void setup() {
    cut = ... ; // setup for class under test

    // We mock with mockito and inject the resource
    mockedConnector = mock(IServiceConnector.class);
    cut.setServiceConnector(mockedConnector);

    // We mock the service to be used
    mockedService = mock(IService.class);
}

@Test
public void registerUser_ShouldRegisterToService() {
    // Arrange
    // might change the return values to reflect the correct ones
    when(mockedConnector.getService())
        .thenReturn(mockedService);
    String newUsername = "newusername";

    // Act
    cut.methodToTest(newUsername);

    // Assert, with mockito
    // we only need to assert that the service registers the
    // new user
    verify(mockedService).registerUser(username);
}
```

<p>You can also test the exception throwing:</p>

<pre class="lang-java prettyprint-override"><code>// test the exception
@Test(expected = Exception.class)
public void registerUser_ThrowExceptionOnError() {
    // Arrange
    when(mockedConnector.getService())
        .thenReturn(mockedService);
    // return failure on any string
    when(mockedService.registerUser(anyString))
        .thenReturn("failure");
    String newUsername = "newusername";

    // Act
    cut.methodToTest(newUsername);

    // Assert
    // Should throw exception and be checked with the
    // expected parameter in test
}
</code>```

Hope this helps you out.

Title: How to write junit test case for the program

Body:
I have a method which has external dependencies within the method. Please help me in writing a junit test case for the below program.

```<code>    @Override
    public void methodToTest(String user){

      //fetch the values from the properties file. Config class has a static method called
      //getProperty which reads the values from the environment specific property file.
      String url = Config.getProperty("serviceurl");
      String username = Config.getProperty("serviceusername");
      String password = Config.getProperty("servicepassword");

      //Connect to an external service.
      Service service = new Service(url, username, password);

      //invoke the method on the service.
      String returnValue = service.registerUser(user);

      if (returnValue.equals("failure")){
        throw new Exception("User could not be registered");
      }
    }
</code>```

Note:

The method overrides the superclass method so that I cannot change the
arguments to contain the url, username and password.

The method also connects to an external service. which is environment specific. The
environment details is taken from the properties file.

Body:
Since the resources are external, in your case the Config and Service, it makes it a bit difficult to test. You basically have two options:

Go with the flow and set up the external resources under your control if possible

Refactor the code so that it is easier to mock the resources

Since the first one you clearly can't do, we'll go through the second one. Refactoring the code with each step fixing each dependency so that they are replaceable for the test:

Fixing Config dependency

I'm assuming that Config is a static resource, which means you need to be able to replace it somehow. Easiest is to add a new replaceable resource:

public interface IConfigResource {
    String getProperty(String propertyName);
}

Together with a handy dandy default static version of it:

public class DefaultConfigResource : IConfigResource {
    public String getProperty(String propertyName) {
        return Config.getProperty(propertyName);
    }
}

In your class you need to add/change the following so that the dependency can be injected for test:

// The default is set at construction
private IConfigResource configResource = new DefaultConfigResource();

// Method used by the test to inject the config resource
public void setConfigResource(IConfigResource configResource) {
    this.configResource = configResource;
}

Now we can either use Mockito or manually handroll a mocked version of IConfigResource to test that your method is calling the config resource correctly, in the test below we use Mockito:

ClassUnderTest cut;
IConfigResource mockedResource;

@Before
public void setup() {
    cut = ... ; // setup for class under test

    // We mock with mockito and inject the resource
    mockedResource = mock(IConfigResource.class);
    cut.setConfigResource(mockedConfigResource);
}

@Test
public void callingMethod_ShouldGetValuesFromResource() {
    // Arrange
    // might change the return values to reflect the correct ones
    when(mockedResource.getProperty("serviceurl"))
        .thenReturn("correct service url");
    when(mockedResource.getProperty("serviceusername"))
        .thenReturn("correct service username");
    when(mockedResource.getProperty("servicepassword"))
        .thenReturn("correct service password");

    // Act
    cut.methodToTest("username");

    // Assert, with mockito
    verify(mockedResource).getProperty("serviceurl");
    verify(mockedResource).getProperty("serviceusername");
    verify(mockedResource).getProperty("servicepassword");
}

There, the Config part of the method should now be tested

Fixing Service dependency (and removing Config dependency as a bonus)

So we need something that needs to handle this part of the code:

//Connect to an external service.
Service service = new Service(url, username, password);

//invoke the method on the service.
String returnValue = service.registerUser(user);

First the service needs to have a replaceable interface in order for us to be able to test that your method is calling service.registerUser(String). This should be easy:

public interface IService {
   String registerUser(String userName);
}

Your current service can implement this interface with ease, just add it to the class signature (since it should have the method already):

public class Service implements IService {
...

We also need something (a factory method of sorts) that can connect to the service and return something that looks like an IService. We can also add the config resource we've made above to make it a bit easier:

public interface IServiceConnector {
    void setConfigResource(IConfigResource configResource);

    // no need to have parameters, the config is injected above
    IService getService();
}

The default implementation is easy, we can use the ConfigResource we used before:

public class DefaultServiceConnector {
    IConfigResource configResource = new DefaultConfigResource();

    public void setConfigResource(IConfigResource configResource) {
        this.configResource = configResource;
    }

    public IService getService() {
        String url = Config.getProperty("serviceurl");
        String username = Config.getProperty("serviceusername");
        String password = Config.getProperty("servicepassword");

        return new Service(url, username, password);
    }
}

You can now move the test written above to be used as test for this connector class where the method under test is connectService(). Furthermore your original method under test should be easier now:

IServiceConnector serviceConnector = new DefaultServiceConnector();

public void setServiceConnector(IServiceConnector serviceConnector) {
    this.serviceConnector = serviceConnector;
}

@Override
public void methodToTest(String user) {
  // ... Removed some code ...

  //Connect to an external service.
  Service service = serviceConnector.getService();

  //invoke the method on the service.
  String returnValue = service.registerUser(user);

    if (returnValue.equals("failure")){
      throw new Exception("User could not be registered");
    }
}

The test should look something like this now:

ClassUnderTest cut;
IServiceConnector mockedConnector;
IService mockedService;

@Before
public void setup() {
    cut = ... ; // setup for class under test

    // We mock with mockito and inject the resource
    mockedConnector = mock(IServiceConnector.class);
    cut.setServiceConnector(mockedConnector);

    // We mock the service to be used
    mockedService = mock(IService.class);
}

@Test
public void registerUser_ShouldRegisterToService() {
    // Arrange
    // might change the return values to reflect the correct ones
    when(mockedConnector.getService())
        .thenReturn(mockedService);
    String newUsername = "newusername";

    // Act
    cut.methodToTest(newUsername);

    // Assert, with mockito
    // we only need to assert that the service registers the
    // new user
    verify(mockedService).registerUser(username);
}

You can also test the exception throwing:

// test the exception
@Test(expected = Exception.class)
public void registerUser_ThrowExceptionOnError() {
    // Arrange
    when(mockedConnector.getService())
        .thenReturn(mockedService);
    // return failure on any string
    when(mockedService.registerUser(anyString))
        .thenReturn("failure");
    String newUsername = "newusername";

    // Act
    cut.methodToTest(newUsername);

    // Assert
    // Should throw exception and be checked with the
    // expected parameter in test
}

Hope this helps you out.


问:
How is Python used in the real world?
I'm looking to get a job as a Python programmer. I know the basics of the language and have created a few games with it using pygame. I've also started to experiment with Django.

However, looking at the job market, it doesn't seem very many Python jobs are web-related. On the desktop side of things, it doesn't seem like very many companies use the popular GUI libraries like pyQt or wxPython.

How are companies actually using Python? What areas should one focus on to land a job as a Python programmer?

答: I implement and maintain a fair number of Python (well technically Jython) scripts which are utilized in aspects of processing real time online stock/option transactions.  Python is most definitely used in some mission critical applications.  However, it appears to me that python knowledge doesn't translate well as a marketable skill in the job market.</p>

Title: How is Python used in the real world?


答:
In addition to @kwah excelent (+1) answer you should encapsulate the external service into an interface that can be more easily mocked/simulated than the real service:</p>

```<code>  //Connect to an external service.
  IService service = getService(url, username, password);

  //invoke the method on the service.
  String returnValue = service.registerUser(user);
</code>```

Body:
In addition to @kwah excelent (+1) answer you should encapsulate the external service into an interface that can be more easily mocked/simulated than the real service:

  //Connect to an external service.
  IService service = getService(url, username, password);

  //invoke the method on the service.
  String returnValue = service.registerUser(user);


问:
How to share common methods if objects have different roles?
If 2 classes have the following in common:

part of how their state is represented (both have a linear container)

multiple identical methods (identical code, not just signature)

But are not substitutable, ie: they play different roles.

Then how do I avoid repeating those methods?

The obvious solution would be inheritance, but I have read: "inheritance is not for code reuse". (Does this apply here?, Is this really a case of just code reuse?)

I am deliberately not telling you which language, because I am hoping for a language agnostic answer.

The methods are couple lines long, all of them modify the container, one of them transforms/extracts a sort of "final result" from the container. By roles I mean: substituting one for the other would be valid runnable code (in the places where only the common methods are used), but would result in logic/semantic error. By inheritance I meant that introducing a 3rd class as base/super with the common methods, but that would be virtual. So by substitution I meant substituting a "sibling" type and not a subtype.

答:
Sometimes, inheritance can be used for code re-use. Especially private inheritance in languages that support it.

And if that is not possible for you, you can always factor out the common stuff into a separate class and use composition to include it in your original classes.

In any case, before you refactor this, you should consider if the similarity is structural or accidental. Ask yourself these questions:

Do the container and associated functions represent the same concept in both places where they are used?

Is it possible the container type gets changed in one place without it having to be changed in the other? Same goes for the implementation of the functions.

If the answer to the first is 'No' or to the second is 'Yes', then the similarity is accidental. Thus you are not repeating yourself and don't need to refactor.


问:
Should a complex EL expression be replaced by a single javabean getter?
In JSF if I have a component that conditionally renders based on a number of variables what is the optimal way to handle the render statement... should the logic live in the component declaration or in some form of helper class?

The text woof is only displayed when an animal is an elephant or dog and the animal is not mute.

Option 1:

Implementation in the view:

```<code><h:outputText id="text" value="woof"
  rendered="#{animal.type == 'elephant'
                  or animal.type == 'dog' and not(animal.mute)}"/>
</code>```

or Option 2:

Encapsulation:

```<code><h:outputText id="text" value="woof" rendered="#{animal.barkingAnimal}"/>
</code>```

with implementation:

```<code>public class Animal {
     public boolean isBarkingAnimal() {
         return ("dog".equals(getType()) || "elephant".equals(getType())) && !isMute();
     }
...
</code>```

So both work... but which is the right way to handle the scenario?

答: As a rule of thumb, try to keep the .xhtml files as logic-free as possible, so they only deal with presentation. So, clearly go for option 2.</p>

OwnerDisplayName: TEL


答:
In most cases, it is a simply a matter of taste. If your concern is simply reusing the condition, you could as well do:

```<code><ui:param name="animalCanBark" value="#{animal.type == 'elephant'
              or animal.type == 'dog' and not(animal.mute)}" />
...
<h:outputText id="text" value="woof" rendered="#{animalCanBark}"/>
</code>```

This is often useful when the expression you need to write involves more than just one object, say:

```<code><ui:param name="showBarking" value="#{animal.type == 'elephant'
              and not facesContext.validationFailed" />
</code>```

I think many people prefer to write this type of logic in Java instead of in EL because the Java compiler can do type checking in the Java code (also, most IDEs have better autocompletion for Java then for .XHTML files), which is a concern for some.

Having said that, if it's an information about the model that someday may be used in another place (not just another Facelets/JSF page), that would make a good reason to have it done in your Java code. In the example you gave, the method isBarkingAnimal() gives some information about the model (Animal), which someday might end up being useful when another object (not just another Facelets page) needs to know if a given animal barks. So, I would probably go with that.

Title: Should a complex EL expression be replaced by a single javabean getter?

Body:
In JSF if I have a component that conditionally renders based on a number of variables what is the optimal way to handle the render statement... should the logic live in the component declaration or in some form of helper class?

The text woof is only displayed when an animal is an elephant or dog and the animal is not mute.

Option 1:

Implementation in the view:

```<code><h:outputText id="text" value="woof"
  rendered="#{animal.type == 'elephant'
                  or animal.type == 'dog' and not(animal.mute)}"/>
</code>```

or Option 2:

Encapsulation:

```<code><h:outputText id="text" value="woof" rendered="#{animal.barkingAnimal}"/>
</code>```

with implementation:

```<code>public class Animal {
     public boolean isBarkingAnimal() {
         return ("dog".equals(getType()) || "elephant".equals(getType())) && !isMute();
     }
...
</code>```

So both work... but which is the right way to handle the scenario?

Body:
In most cases, it is a simply a matter of taste. If your concern is simply reusing the condition, you could as well do:

<ui:param name="animalCanBark" value="#{animal.type == 'elephant'
              or animal.type == 'dog' and not(animal.mute)}" />
...
<h:outputText id="text" value="woof" rendered="#{animalCanBark}"/>

This is often useful when the expression you need to write involves more than just one object, say:

<ui:param name="showBarking" value="#{animal.type == 'elephant'
              and not facesContext.validationFailed" />

I think many people prefer to write this type of logic in Java instead of in EL because the Java compiler can do type checking in the Java code (also, most IDEs have better autocompletion for Java then for .XHTML files), which is a concern for some.

Having said that, if it's an information about the model that someday may be used in another place (not just another Facelets/JSF page), that would make a good reason to have it done in your Java code. In the example you gave, the method isBarkingAnimal() gives some information about the model (Animal), which someday might end up being useful when another object (not just another Facelets page) needs to know if a given animal barks. So, I would probably go with that.

OwnerDisplayName: elias


答:
Personally, I would use Option #2.  While I know it's very possible to solve the problem using EL and gain some bit of reuse across xhtml documents using functions or ui:params it really seems to lack the portability, maintainability and testability of the Java bean implementation.

If a developer is fluent in both EL and Java and owns both the xhtml and Java beans, it doesn't seem to make much sense to use EL to do ANY conditional evaluation with a size > 1.

There just seem to be too many benefits to implementing on the Java side:

Ability to lean on the IDE + compiler

Use constants or enums (for "dog" and "bark"), chances are they are being use elsewhere in the code for comparisons as well... if the String value changes it's real fun having to manually replace every occurance of it across a code base

Instead of having to navigate to the page in question with appropriate data I can exercise logic using unit tests

One of the main arguments I've heard (outside of Stack) in favor of Option 1 is:

"It's a lot easier to see when a component renders if you keep this logic in the view."

I've found that this might be the case for an application in the initial stage of it's life where it's lighter weight and less complicated.  However applying this practice on a larger scale and as a smaller application matures it can cause a rats nest of conditionals and become a nightmare to maintain.  Here are a couple of examples similar to what I've seen out in the wild:

```<code><h:outputText value="grrr"
    render="#{animal.type == 'dog' or animal.type == 'cat' or animal.type == 'horse'
        or animal.type == 'pony' or animal.type == 'mule' or animal.type == 'lion'
        or animal.type == 'tiger' or (animal.type == 'bird'
        and animal.subType == 'macaw') or .... continue this for another line or two}"
/>
</code>```

Or my favorite, using multiple components with render conditions that are exclusive of one another to represent the different values that could be displayed:

```<code><h:outputText value="grr" render="#{theMonstrosityFromPreviousExample} />
<h:outputText value="cry"
    render="#{animal.type == 'human' and animal.subType == 'baby'}" />
<h:outputText value="yo"
    render="#{animal.type == 'human' and animal.subType == 'teenager'}" />
<h:outputText value="hello"
    render="#{animal.type == 'human' and animal.subType == 'adult'}"/>
</code>```

Can up to 4 texts be displayed at once?  At first glance you can't tell, a check of each condition will be necessary.  As a side note I realize this example is also poor design, as these could be put in a c:choose... but I've seen this before.

At the end of the day this is still theoretically 'view' logic since it determines what actually gets displayed so there's a conceptual argument it should live in the xhtml.  The problem that I've found is that including logic like this in the view template can make the layout much harder to understand in the long run and I have yet to see that this method of solving the problem holds any real benefit over using the Java bean implementation.

Body:
Personally, I would use Option #2.  While I know it's very possible to solve the problem using EL and gain some bit of reuse across xhtml documents using functions or ui:params it really seems to lack the portability, maintainability and testability of the Java bean implementation.

If a developer is fluent in both EL and Java and owns both the xhtml and Java beans, it doesn't seem to make much sense to use EL to do ANY conditional evaluation with a size > 1.

There just seem to be too many benefits to implementing on the Java side:

Ability to lean on the IDE + compiler

Use constants or enums (for "dog" and "bark"), chances are they are being use elsewhere in the code for comparisons as well... if the String value changes it's real fun having to manually replace every occurance of it across a code base

Instead of having to navigate to the page in question with appropriate data I can exercise logic using unit tests

One of the main arguments I've heard (outside of Stack) in favor of Option 1 is:

"It's a lot easier to see when a component renders if you keep this logic in the view."

I've found that this might be the case for an application in the initial stage of it's life where it's lighter weight and less complicated.  However applying this practice on a larger scale and as a smaller application matures it can cause a rats nest of conditionals and become a nightmare to maintain.  Here are a couple of examples similar to what I've seen out in the wild:

<h:outputText value="grrr"
    render="#{animal.type == 'dog' or animal.type == 'cat' or animal.type == 'horse'
        or animal.type == 'pony' or animal.type == 'mule' or animal.type == 'lion'
        or animal.type == 'tiger' or (animal.type == 'bird'
        and animal.subType == 'macaw') or .... continue this for another line or two}"
/>

Or my favorite, using multiple components with render conditions that are exclusive of one another to represent the different values that could be displayed:

<h:outputText value="grr" render="#{theMonstrosityFromPreviousExample} />
<h:outputText value="cry"
    render="#{animal.type == 'human' and animal.subType == 'baby'}" />
<h:outputText value="yo"
    render="#{animal.type == 'human' and animal.subType == 'teenager'}" />
<h:outputText value="hello"
    render="#{animal.type == 'human' and animal.subType == 'adult'}"/>

Can up to 4 texts be displayed at once?  At first glance you can't tell, a check of each condition will be necessary.  As a side note I realize this example is also poor design, as these could be put in a c:choose... but I've seen this before.

At the end of the day this is still theoretically 'view' logic since it determines what actually gets displayed so there's a conceptual argument it should live in the xhtml.  The problem that I've found is that including logic like this in the view template can make the layout much harder to understand in the long run and I have yet to see that this method of solving the problem holds any real benefit over using the Java bean implementation.


答:
You might consider reading a book like "The Art of Readable Code", by Dustin Boswell and Trevor Foucher.

Also, when you learn a new programming language, do not invent you own style. Follow the established style (identifier naming: underscores, camelCase; indentation, etc.) used by more experienced people. Above all, be consistent.

Have your line comments answer the questions "Why?" and "How?", not so much "What?". For example, if you use an unusual algorithm, cite its wikipedia page. If you traverse an array from last to first instead of from first to last, explain your reason. You will thank yourself 6 months later when you read your own code.

Use well-established programming patterns like those listed in "Design Patterns", by the "Gang of Four", and, in complicated cases, cite which pattern(s) you use.


问:
We need a DVS. They need an SVN repo. What should we do?
We are a team of two people who need a versioning system for their university project. Due to administration issues, the graders need us to register on a site that happens to only support SVN; however, we cannot commit to SVN from inside the university as the required protocols are filtered out by the proxy (including the HTML extensions that would make SVN checkins over HTTP possible). Sigh.

Is it possible to have a hg repo for ourselves that can then be converted to SVN for purposes of hosting on this site?

Please note that practically all students commute to university, so the "we can't commit from university so we can't do this" is not as strong of an argument as you'd think.

答: As someone who's been on the grader/ta end of the world, the appropriate action is to raise a stink with the professor and the grades. He needs to know. Also raise a stink with the IT department. Stink all around. This is unreasonable, and something needs to give, and it's not you - in this case.


问:
UML modeling tool for Linux
I am starting a new project using Eclipse under Linux and I am looking for a UML 2.x modeling tool not just a diagramming tool.

Do anyone has experience with UML plugin for Eclipse? Does it support all UML diagrams?

Do you recommend another modeling tool?

If you have been in a similar situation, what did you end up doing anyway?

答:
I use Papyrus in Eclipse 4.2.

It can be used as integrated tool in eclipse or it can be downloaded from their website and used as a standalone tool.

It supports UML 2

It supports almost all UML diagrams.

It can be used with other eclipse modeling tools to generate and reverse engineer code (Java code in my case).

Title: UML modeling tool for Linux


问:
Python in Big Data?
Can python be efficiently implemented in big data field? To be precise I am building an web app that analyses really big data in medical health care field consisting of medical history and huge personal information. I need some advice on how to handle very big data in python efficiently and with high performance. Also are their some open source packages in python available which have high performance and efficiency in big data handling?

About users and data:
Each user has about 3gb of data. Users are grouped based on their family and friend circles and the data is then analysed to predict important information and co-relations out of that. Currently I am talking about 10,000 users and will be rapidly increasing number of users.

答:
Python is used extensively in the big data field. There are a couple of packages that tend to get used quite a bit and they are probably the main reason Python has made inroads so deeply into big data:

NumPy - The Fundamental package for scientific computing in Python

SciPy - Mathematics, Scientific and Engineering package

Given that they are both open source and the popularity and ease of learning Python has pretty much catapulted it's use in Academia. This in turn has caused it to be used more and more outside academia and in larger companies or when students move into work roles they bring these packages with them.

These are very good packages and I had dabbled with them in a few projects. I have not used Python enough in Big Data projects to answer your ancillary question on how to handle Big Data efficiently with Python.

Title: Python in Big Data?


问:
Why is reverse debugging rarely used?
gdb implemented support for reverse debugging in 2009 (with gdb 7.0). I never heard about it until 2012. Now I find it extremely useful for certain types of debugging problems. I wished that I heard of it before.

Correct me if I'm wrong but my impression is that the technique is still rarely used and most people don't know that it exists. Why?

Do you know of any programming communities where the use of reverse debugging is common?

Background information:

Stackoverflow: How does reverse debugging work?

gdb uses the term "reverse debugging" but other vendors use other terms for identical or similar techniques:

Microsoft calls it IntelliTrace or "Historical Debugging"

There's a Java reverse debugger called Omniscient Debugger, though it probably no longer works in Java 6

There are other Java reverse debuggers

OCaml's debugger (ocamldebug) calls it time travel

答: As mentioned already, performance is key e.g. with gdb's reversible debugging, running something like gzip sees a slowdown of 50,000x compared to running natively. There are commercial alternatives however: I work for Undo undo.io, and our UndoDB product does the same but with a slowdown of less than 2x. There are other commercial reversible debuggers available too.</p>

Title: Why is reverse debugging rarely used?

ContentLicense: CC BY-SA 4.0


答:
It depends on what you want from your handling of big data.  This concept is relatively vague.  For example, if you're talking about MapReduce jobs across disparate data sources, then you may be interested in using Hadoop Streaming with the Dumbo library.  If you're talking about statistical analysis, then NumPy and SciPy (as mentioned by Akira71) are interesting, as well as pandas (a data analysis toolkit).  If you want graphing, look into matplotlib.

However, if you're talking about the storage and querying of big data, Python is not your best bet.  You will want something like the Hadoop ecosystem to make this perform well, perhaps with layers on top for querying and building intermediate data sets.  One project that really interests me is Spark; you may want to look at it as well.  Unfortunately, this type of application framework does not play to Python's strengths.

Title: Python in Big Data?

Body:
Can python be efficiently implemented in big data field? To be precise I am building an web app that analyses really big data in medical health care field consisting of medical history and huge personal information. I need some advice on how to handle very big data in python efficiently and with high performance. Also are their some open source packages in python available which have high performance and efficiency in big data handling?

About users and data:
Each user has about 3gb of data. Users are grouped based on their family and friend circles and the data is then analysed to predict important information and co-relations out of that. Currently I am talking about 10,000 users and will be rapidly increasing number of users.


答:
My experience, knowledge and what I've heard is:</p>

```<code>  most
   /|\  Aerospace   # Usually NASA gets it right.  Fixes are hard in space.
    |   Finance     # Money and bean counters are picky.
    |   Open Source # Passion helps leads to quality.
    |   Healthcare  # In the US it's a huge mess in many areas.
    |   ECommerce   # Lack of standards. Marketing pressure.
    |   Consulting  # Make the sale, move on.
    |   Education   # Lack of standards.
   \|/  VideoGames  # Sell today, gone tomorrow. Publisher pressure to release prematurely.
  least
</code>```

Title: Which industries use the most high-quality, measurable, reliable approach to software?

Body: What sections of the software industry (i.e. healthcare, aerospace, games, etc) are most likely to practice software engineering with the engineering rigor + depth and breadth defined in the SWEBOK? </p>

Body:
My experience, knowledge and what I've heard is:

  most
   /|\  Aerospace   # Usually NASA gets it right.  Fixes are hard in space.
    |   Finance     # Money and bean counters are picky.
    |   Open Source # Passion helps leads to quality.
    |   Healthcare  # In the US it's a huge mess in many areas.
    |   ECommerce   # Lack of standards. Marketing pressure.
    |   Consulting  # Make the sale, move on.
    |   Education   # Lack of standards.
   \|/  VideoGames  # Sell today, gone tomorrow. Publisher pressure to release prematurely.
  least


问:
Are there any algorithms for resource scheduling of courses-teachers-classes?
I'm developing an application to help school principals in dedicating teachers to classes and courses over the hours of a week (scheduling). The scenario is roughly something like this:

User enters the list of teachers and their free times into the system

User enters the list of courses for this semester

User enters the list of available classes into the system

Well, up to here, there is no big deal. Just simple CRUD operations and nothing extraordinary. However, now what makes this system useful is that the application should automatically and based on an algorithm create the semester scheduling.

I think you've got the main idea here. For example application should suggest that teacher A should go to class 1 for mathematics, and at the same time teacher B should go to class 2 for physics. This way all of the classes would be dedicated to lessons and teacher times won't overlap each other. Piece a cake for school principal.

However, I can't find a good algorithm for this resource dedication. I mean it seems hard to me.

Searching Google resulted in articles from different websites, but they are of no help and use to me. For example:

http://en.wikipedia.org/wiki/Resource_allocation or
http://en.wikipedia.org/wiki/Scheduling_(production_processes)

Is there any algorithm out there, or any application or engine which can help me here? Does this requirements have a known name, like for example time scheduling engine?

Any help would be appreciated.

答:
I have seen a custom solution for this.  Some of the major points:

There are different approaches to organizing the week, for example the order to actually start 'packing' events can vary.  Approaches can include 'Monday first', 'AMs first', '2 day schedule', '6 day schedule', 'A/B schedule'.

Class size is not an absolute.  It's best to have an 'ideal' size plus a minimum and maximum.

Teachers work different schedules.  Some are part time.  Some Districts count any hours worked in a given day as having worked a full day.  Some count half-days this way and some count quarter days.  So the solution tends to be specific to the district.

It's important to allow and build in some flexibility and 'wiggle-room'.  Scheduling resources too 'tightly' leads to issues when the inevitable 'stuff' comes up such as teachers sick, kids sick, weather event days, etc.

Students enter and leave the district through the year.  For some districts that can easily be 20% of their student body.  Scheduling algorithms needs to allow for this factor also.

Special Education has a whole set of requirements including IEP, instruction alone, etc.  Also the instructor travel between schools and so distance and travel time has to be incorporated for these services.

There are different roles to be considered including Teachers, Teacher Assistants, Aides, Translators etc.

People eat lunch :)

Teachers need time for Lesson Preparation, Meetings, and Discussions with Parents.

It is good to distinguish direct Service (actually seeing students) activities as compared to indirect activities.

At the higest level, given your requirements, I would imagine an algorithm for you would be:

```<code>For each Course
    For each available Teacher
        For each available Classroom
           Select the first Classroom available and book the resource.
           Update the course and teachers availability.
        end Classrooms
    end Teachers
end Courses
</code>```

Body:
I have seen a custom solution for this.  Some of the major points:

There are different approaches to organizing the week, for example the order to actually start 'packing' events can vary.  Approaches can include 'Monday first', 'AMs first', '2 day schedule', '6 day schedule', 'A/B schedule'.

Class size is not an absolute.  It's best to have an 'ideal' size plus a minimum and maximum.

Teachers work different schedules.  Some are part time.  Some Districts count any hours worked in a given day as having worked a full day.  Some count half-days this way and some count quarter days.  So the solution tends to be specific to the district.

It's important to allow and build in some flexibility and 'wiggle-room'.  Scheduling resources too 'tightly' leads to issues when the inevitable 'stuff' comes up such as teachers sick, kids sick, weather event days, etc.

Students enter and leave the district through the year.  For some districts that can easily be 20% of their student body.  Scheduling algorithms needs to allow for this factor also.

Special Education has a whole set of requirements including IEP, instruction alone, etc.  Also the instructor travel between schools and so distance and travel time has to be incorporated for these services.

There are different roles to be considered including Teachers, Teacher Assistants, Aides, Translators etc.

People eat lunch :)

Teachers need time for Lesson Preparation, Meetings, and Discussions with Parents.

It is good to distinguish direct Service (actually seeing students) activities as compared to indirect activities.

At the higest level, given your requirements, I would imagine an algorithm for you would be:

For each Course
    For each available Teacher
        For each available Classroom
           Select the first Classroom available and book the resource.
           Update the course and teachers availability.
        end Classrooms
    end Teachers
end Courses


问:
Is a degree needed for low-level/embedded programming jobs?
I know that it is possible to get into software development without a degree in computer science, but is it possible (or rather, common) to be able to get an embedded programming job without the computer science degree (or any engineering degree as well)?</p>

答:
For low-level programming, you need to know some electronics and circuits, and some of the math behind it. Not a ton per se, but enough.

A degree is always nice.


问:
Default vs Impl when implementing interfaces in Java
After reading Should package names be singular or plural? it occurred to me that I've never seen a proper debate covering one of my pet peeves: naming implementations of interfaces.

Let's assume that you have a interface Order that is intended to be implemented in a variety of ways but there is only the initial implementation when the project is first created. Do you go for DefaultOrder or OrderImpl or some other variant to avoid the false dichotomy? And what do you do when more implementations come along?

And most important... why?

答:
I decide the naming by the use case of the interface.

If the interface is used for decoupling, then I choose Impl for implementations.

If the purpose of the interface is behavioral abstraction, then the implementations are named according to what they are concretely doing. I often append the interface name to that. So if the interface is called Validator, I use FooValidator.

I find that Default is a very bad choice. First it pollutes code completion features, because the names always start with it. The other thing is that a default is subject to change over time. So what first might be a default can some time after be a deprecated feature. So either you always start renaming your classes as soon as defaults change or you live with misleading names.


问:
How does the Law of Demeter apply to object-oriented systems regarding coupling and cohesion?
How does the Law of Demeter applied to object-oriented systems with coupling and cohesion?

I was reading a book "Software development and professional practice" and came across the chapter about LoD, and was curious about how that principle is applied into object oriented systems.

答:
Study (V. Basili, L. Briand, and W. L. Melo. A validation of object-oriented design metrics as quality indicators) has shown that classes with larger response sets have tendencies to create more errors than classes with smaller response set because more response set means the chance of higher coupling.

The value of Law of Demeter is that it reduces response set by definition. An object's method can only call a method of itself, any parameter that were passed onto the method, method of any object it created, and method of any directly held object. Since the response set is smaller, there is less chance of high coupling. Since the module/method is only using immediate available references there is higher cohesion.


答:
According to Emerson Macedo the Law of Demeter states the following:

Each unit should have only limited knowledge about other units: only units "closely" related to the current unit.

Each unit should only talk to its friends; don't talk to strangers.

Only talk to your immediate friends.

This corresponds directly to the principle of low coupling as the units (or objects) are supposed to, much like above:

Not be tightly coupled to each other. Only the closest ones are.

Each one should only talk to its collaborators and not to the collaborators of the collaborator

Only talk to immediate collaborating object

One of the lowest forms of coupling is message passing, i.e. the data is shared between objects through method calls with parameters.

Furthermore, from what I can see, the Law of Demeter does not directly correspond to the principle of high cohesion as that only states the objects themselves should know what data they possess themselves. An object with low cohesion has data members that it is not using frequently in its own methods. It is more about the contents of an object rather than its relationships and collaborating objects.

Title: How does the Law of Demeter apply to object-oriented systems regarding coupling and cohesion?

Body:
How does the Law of Demeter applied to object-oriented systems with coupling and cohesion?

I was reading a book "Software development and professional practice" and came across the chapter about LoD, and was curious about how that principle is applied into object oriented systems.


答:
That is a very vague question, there is no canon definition for what constitutes big data. From a development point of view the only thing that truly changes how you need to handle data is if you have so much that you can't fit it all in memory at once.

How much of a problem that is depend greatly on what you need to do with the data, for most jobs you can do a single pass scheme where you load a block of data, do whatever needs to done with it, unload it, and go on to the next.

Sometimes issues can be solved by doing an organization pass, first going through data organizing it into chunks of data that need to be handled together, then going through each chunk.

If that strategy doesn't fit your task you can still get a long way with OS handled disk swapping, handle the data in blocks as far as possible, but if you need a little arbitrary access here and there it is still going to work.

And of course an always excellent strategy when dealing with a lot of data is to dwarf it by hardware. You can get 64 GB of memory in 16 GB blocks for $500, if you are working with that much data it is an easily justified investment. Some good SSDs is a nobrainer.

Specific case:

A big part of this job is definitely to reduce those 3GB data per person. It is often a bit of an art in its own right to find out what can be thrown away, but given the amount I must presume that you have got a fair amount of bulk measurements, in general you should first find patterns and aggregations for those data, and then use those results for comparing persons to one another. The majority of your raw data is either noise, repetition or irrelevant, you have got to cut that away.

This reduction process is easily suitable for a cluster as you can just give each process its own pile of persons.

The processing thereafter is a bit more tricky, what is optimal depend on a lot of factors, and you will probably have to do some trial and error. If you can make it fit the job try to load select pieces of data from all persons on the same computer and compare those, do the same with other pieces of data on other computers. Use those results as new data sets etc.

Title: Python in Big Data?


问:
Static variable - Usage and Implications on Threading
I have some confusion regarding the use of static variables/references in a class. It feels like I may not have entirely figured out the implications of keeping something static.

When I say a variable is public static, is it one instance of the variable per class or one instance per assembly?

For example, I have a user control (say, for a WPF app). This user control has a public static variable that coordinates various things within the control. If there is a form in a client application that uses this user control, with multiple instances of this control per form, how many instances of the public static variable are created? (My guess is only one,  and it is modified by every control).

Now, I actually need each instance of the control to have one such variable, not one variable across all the instances of the control in that single form.

Does using a singleton pattern, with a "lock" keyword solve this issue i.e will it create one variable per instance of the user control?
If not, what issue does the singleton pattern solve?
What are the options in solving this design problem?

(Note: I have resorted to creating an internal non static global variable and passing it around carefully, instead of going the static route and its working fine. I am looking to see if there is another option.)

(Please bear with me if the question sounds very naive or unresearched, I did read a bit before posting, but seem stuck on finding an answer that explains it very clearly to me.)

答:
I think you're doing this too convoluted for yourself. I'd use some kind of a presenter/controller class in the application that the user control will have a reference to. That way a state is maintained between all user controls if they're using the same presenter object.

There is no need for singletons here, the application itself can hold the state for each presenter/controller, unless you have a more compelling reason to why you'd use singleton.

EDIT:

So from the comments I've gathered the OP wants a collaborator object to exist once per instance. This means that for each instance there is one object and is not what singleton is about (or anything static really).

You can just as well create a collaborator like this:

```<code>public class Collaborator {
    // members and methods
}
</code>```

And for your custom User control you can create it in the constructor, though you also need to call the base class's constructor as well like this:

```<code>private Collaborator _collaborator;

// Constructor in your CustomUserControl
public CustomUserControl() : base() {
    _collaborator = new Collaborator;
}
</code>```

Whenever the code is creating a child user control you can let it pass the collaborator through the constructor to the child user control. Say there is an event that creates child user controls:

```<code>void OnButtonMakeSubClick(object sender, RoutedEventArgs e)
{
    _childControl = new ChildControl(_collaborator);
}
</code>```

And in your child control you add the following constructor:

```<code>private Collaborator _collaborator;

public ChildControl(Collaborator collaborator) : base() {
    _collaborator = collaborator;
}
</code>```

Hope this hints out what you want.

Body:
I think you're doing this too convoluted for yourself. I'd use some kind of a presenter/controller class in the application that the user control will have a reference to. That way a state is maintained between all user controls if they're using the same presenter object.

There is no need for singletons here, the application itself can hold the state for each presenter/controller, unless you have a more compelling reason to why you'd use singleton.

EDIT:

So from the comments I've gathered the OP wants a collaborator object to exist once per instance. This means that for each instance there is one object and is not what singleton is about (or anything static really).

You can just as well create a collaborator like this:

public class Collaborator {
    // members and methods
}

And for your custom User control you can create it in the constructor, though you also need to call the base class's constructor as well like this:

private Collaborator _collaborator;

// Constructor in your CustomUserControl
public CustomUserControl() : base() {
    _collaborator = new Collaborator;
}

Whenever the code is creating a child user control you can let it pass the collaborator through the constructor to the child user control. Say there is an event that creates child user controls:

void OnButtonMakeSubClick(object sender, RoutedEventArgs e)
{
    _childControl = new ChildControl(_collaborator);
}

And in your child control you add the following constructor:

private Collaborator _collaborator;

public ChildControl(Collaborator collaborator) : base() {
    _collaborator = collaborator;
}

Hope this hints out what you want.


问:
Is it a good idea to do TDD on low level components?
I'm considering writing a low level driver or OS components/kernels.

The osdev.org folks seem to think that the important bits are not meaningfully testable this way, but I have read some discussions where people thought differently. I've looked around, but have failed to find any real life examples of TDD on low-level components.

Is this something people actually do, or just something that people talk about in theory because there is not a good way to do it in practice?

答: If you're interacting with or controlling hardware, then it's difficult to test without it.  You can try emulating the hardware, but that's often more difficult than writing the driver in the first place, so you wind up not knowing whether the bug's in your driver or your emulator.</p>


问:
Is there a correlation between the type of a company/industry and the software engineering rigor?
I would like the answer to explain what impact, if any, does the type of company/industry have on the rigor, depth and breadth with which software engineering is practiced. The best would be some links to support the answer with references.

As a control point, let's declare an assumption that a company which follows the SWEBOK or CMMI (any level) is doing 100% (the best) whereas one which does not follow anything at all is 0% (worst).

Would it be possible to find out what companies or what industries score highest, average, and lowest?

EDIT:
CMMI (any level) refers to any organization which is CMMI certified.

EDIT 2: The middle paragraph specifies that a company which follows some rigorous software engineering standard should score higher than a company which doesn't.

EDIT 3: The SWEBOK is not only organizing content. It does much more, i.e. the SWEBOK characterizes the contents of software engineering body of knowledge.

The SWEBOK promotes a consistent view of software engineering worldwide which is the main point. It is aimed at both practitioners and academics, individuals and organizations. For example, see the CSDP and consider companies who hire them. These are the companies that follow the SWEBOK.

EDIT 4: Note that the answer should tell us about industries or companies that follow some rigorous software engineering standard in the prescribed depth and breath, and those who don't. What is the correlation between the type of company/industry and their software engineering rigor?

Throwing in a bunch of links for reference would be great.

答:
It is obvious that an industry that can't afford failure is more rigorous than another one that can.

The most illustrative example of this is aerospace vs. web design.

The software of a spacecraft can't fail or it falls and hundred million dollars are thrown away. If it doesn't fall, nobody will go up there and fix it (patching is possible to some extend, as long as the bug is not in the patch handler and the antennas and solar panels are correctly oriented).

The software of an aircraft can't fail or hundreds of passengers die. But if an error is found   and handled before a failure, software can be fixed and the patch can be sent to all companies operating that plane.

If the software of a web site fails, nobody is hurt, few money is lost and a fix occurs within a day, if not within an hour.

In aerospace, every task from the system requirements to the final product is verified over and over, software validation of a few thousand lines needs several years. Despite all of that, systems are duplicated or triplicated from basic hardware components to whole satellites (e.g., see Iridium satellite constellation, in-orbit spares). In that process, CMMi, ISO9000, DO-178 and other quality stuff are just a by-product; the rigor is in the process itself, not in how it is controlled.


问:
Why aren't user-defined operators more common?
One feature I miss in from functional languages is the idea that operators are just functions, so adding a custom operator is often as simple as adding a function. Many procedural languages allow operator overloads, so in some sense operators are still functions (this is very true in D where the operator is passed as a string in a template parameter).

It seems that where operator overloading is allowed, it is often trivial to add additional, custom operators. I found this blog post, which argues that custom operators don't work nicely with infix notation because of precedence rules, but the author gives several solutions to this problem.

I looked around and couldn't find any procedural languages that support custom operators in the language. There are hacks (such as macros in C++), but that's hardly the same as language support.

Since this feature is pretty trivial to implement, why isn't it more common?

I understand that it can lead to some ugly code, but that hasn't stopped language designers in the past from adding useful features that can be easily abused (macros, ternary operator, unsafe pointers).

Actual use cases:

Implement missing operators (e.g. Lua doesn't have bitwise operators)

Mimic D's ~ (array concatenation)

DSLs

Use | as Unix pipe-style syntax sugar (using coroutines/generators)

I'm also interested in languages that do allow custom operators, but I'm more interested in why it has been excluded. I thought about forking a scripting language to add user-defined operators, but stopped myself when I realized that I haven't seen it anywhere, so there's probably a good reason why language designers smarter than me haven't allowed it.

答:
Another factor for it is that it's not always straight forward to define an operation with the available operators. I mean, yes, for any kind of number, the '*' operator can make sense, and are usually implemented in the language or in existing modules. But in the case of the typical complex classes that you need to define (things like ShipingAddress, WindowManager, ObjectDimensions, PlayerCharacter, etc) that behaviour is not clear... What does mean to add or subtract a number to an Address? Multiply two Addresses?

Sure, you can define that adding a string to an ShippingAddress class means a custom operation like "replace line 1 in address"(instead of "setLine1" function) and adding a number is "replace zip code" (instead of "setZipCode"), but then the code is not very readable and confusing. We typically think that operator are used in basic types/classes, as their behaviour is intuitive, clear and consistent (once you are familiar with the language, at least). Think in types like Integer, String, ComplexNumbers, etc.

So, even if defining operators can be very useful in some specific cases, their real-world implementation is quite limited, as the 99% of the cases where that will be a clear win are already implemented in the basic language package.


答:
For an overview of the technology choices and products, see a series of blog posts I wrote about a year ago (and some follow-ups since):

http://jakob.engbloms.se/archives/1547 - the tech

http://jakob.engbloms.se/archives/1554 - research

http://jakob.engbloms.se/archives/1564 - products

http://jakob.engbloms.se/archives/1768 - updates to the previous post

My feeling for why it is used so little is that it requires special hardware, or using a special debugger, or setting up your system right.  Most people do not invest the time to get maximum value from their debug tools, unfortunately.

And the fact that the "cheap default" of gdb is almost unusably slow and has quite a few stability problems for everything but the most common target systems.

Title: Why is reverse debugging rarely used?

Body:
gdb implemented support for reverse debugging in 2009 (with gdb 7.0). I never heard about it until 2012. Now I find it extremely useful for certain types of debugging problems. I wished that I heard of it before.

Correct me if I'm wrong but my impression is that the technique is still rarely used and most people don't know that it exists. Why?

Do you know of any programming communities where the use of reverse debugging is common?

Background information:

Stackoverflow: How does reverse debugging work?

gdb uses the term "reverse debugging" but other vendors use other terms for identical or similar techniques:

Microsoft calls it IntelliTrace or "Historical Debugging"

There's a Java reverse debugger called Omniscient Debugger, though it probably no longer works in Java 6

There are other Java reverse debuggers

OCaml's debugger (ocamldebug) calls it time travel


问:
Do You Still Need Indexing After Database Normalization
After you have done a good normalization, do you still need to index the table? How is this going to affect the performance? Will it even affect the performance in some way after a good normalization?

Which columns are usually indexed if you already have the primary key and the foreign key?

It's seems like it's already effective to normalize a database. But, I might have skipped how indexing affects the database. Is this only effective when queries are used? How does this work/perform and make a database better?

答:
I think you misunderstood what indexing does for database performance.

An index helps the database find rows. Indexes are specialized data-structures that, in exchange for extra disk-space and some performance when inserting and updating, help the database engine home in on matching rows.

Because they take extra space and cost (a modicum of) performance to keep them up-to-date, you as a database designer must create indexes explicitly to suit your application and query patterns.

Indexes are orthogonal to database normalization.


问:
Tasks incorrectly categorized with online text classifiers
Context: Finding company to do the job

The use case of an application I want to write is that a user will describe a task to be performed by some company, e.g.,

I want to move 10 boxes (30x40x50cm) of books and clothes from London
  to Bristol next week. How much would that cost?

and the application will find candidate companies that could do the task. In this case it would need to categorize the task as for removal/transportation companies.

Problem: incorrect categorization

I tried a number of online services for text categorization but results are discouraging. uClassify classified my text as "Home", "Games", and "Arts"; Textimate.me --- as "Science & Environment"; Textwise returned "Business/Consumer_Goods", "Services/Clothing", and "Arts/Design/Fashion".

The possible reasons for incorrect categorization:

task description is very short and there is no evident characteristics in the text to differentiate what is more important for the user: boxes, clothes or maybe moving them,

classifiers trained on other types of texts,

classifiers trained with respect to a different taxonomy.

How would you tackle such a problem?

Potential solution: harvest domain-specific corpus

Since the problem is quite specific to my domain I think about using supervised machine learning algorithm, that would learn with respect to already categorized corpus of such tasks. The corpus can be harvested from sites with tasks already categorized by customers (variation of e-bay). I could also pre-process text before machine learning: extract only verbs ("move") and nouns originating from verbs ("translation"), as they denote what has to be done.

This requires quite a lof of work, so I'm curious whether this is the right direction.

答:
I think your problem entails a mini-schlep:

There are great startup ideas lying around unexploited right under our noses. One reason we don't see them is a phenomenon I call schlep blindness. Schlep was originally a Yiddish word but has passed into general use in the US. It means a tedious, unpleasant task.

No one likes schleps, but hackers especially dislike them. Most hackers who start startups wish they could do it by just writing some clever software, putting it on a server somewhere, and watching the money roll in—without ever having to talk to users, or negotiate with other companies, or deal with other people's broken code. Maybe that's possible, but I haven't seen it.

You've identified one element in your business plan that will be difficult to solve with only software. This is a great sign!

One of the many things we do at Y Combinator is teach hackers about the inevitability of schleps. No, you can't start a startup by just writing code.

That scariness makes ambitious ideas doubly valuable. In addition to their intrinsic value, they're like undervalued stocks in the sense that there's less demand for them among founders. If you pick an ambitious idea, you'll have less competition, because everyone else will have been frightened off by the challenges involved.

So here's my advice:

Write code that does a reasonable job of scanning the user's input for keywords. When implementing this piece, be mindful of the Pareto Principle (or 80-20 rule).

In your keyword matching code, allow for a default case that does not match on any keywords. On the screen, apologize to the user that you could not find a match, but encourage them to return soon because you are constantly improving the application.

In your keyword matching code, log the incoming keywords and the match that it found in a data store.

Review this data store frequently. Perhaps thrice daily on working days, once daily on non-working days.

The data should give you clues about how you can improve your algorithm, so pursue a strategy of continuous improvement.

In your spare time, learn more about computational linguistics and how its concepts can help you better solve your business problem.