问:
Where do you find the time?
I find myself lagging behind on new skills, techniques, language features, etc, and I am finding the time to do so is lacking. Between work, professional, personal and family obligations, I'm lucky to find a few stray hours here and there to focus on any new technologies or reading that I want or need to do. I do make it out to relevant local user groups, but even those are sometimes hard to get to?

So I thought I would ask the community, where, or when, do you find the time to focus on honing your skills or learning new ones? Do you schedule time? Forgo some sleep during the week or weekend? Insomnia? Something else?

EDIT I want to thank all of you who took the time to answer and offer up your advice. There were some things I knew or had thought about, and others I hadn't considered as an option until you shed new light on it.

EDIT 2: In trying to find which answer among the many great ones to accept, I went with @Paddyslacker's since it is the one I feel is best suited for my current situation, although everyone had some very good nuggets of wisdom, such as @Elisha @Martin or @dash-tom-bang.

答:
Here is what I realized -

"I have tons of time"

Don't waste time taking lunch with people who don't add any value to you, read a blog or something you want to study while taking lunch. = +30mins

If you want to study something, buy an audiobook for it and listen while you commute = +30mins.

I know this will sound silly, but - Time your shower & getting ready routine. I learnt I saved 15 minutes in the morning, if I consciously got ready, without letting my head wander. = +15mins

If you sleep at 10:30, start reading a book at 10:00 sharp, it'll not only help you sleep, but will end your day peacefully.

Keep one weekend day(Sat or Sunday, I prefer Saturday) to not meet anyone till 19:00. Then have fun as much as you can.

Without the regular pressure of the day, you can get the work of 12 hours completed in 6 on a saturday.

Hope this helps you 8-)

Title: Where do you find the time?


问:
Are There Realistic/Useful Solutions for Source Control for Ladder Logic Programs
Version control for ladder logic (LL) programs for programmable logic controllers (PLCs) seems to be virtually non-existent. It may be because LL is a visual language and tends to be stored in binary files, or it may be because source code control hasn't "caught on" in process control engineering circles - or perhaps my Google-Fu is weak tonight.

Do you know of any realistic and useful solutions for version control for such systems?

Definitions:

realistic = changes to the programs are tracked by user and subject to reversion and merges

useful = the system integrates with visual LL designers, is not limited to LL from a single PLC manufacturer, and does not cost a ridiculous amount of money?

Note: I have heard of people using SVN or Mercurial et al to track the binary files, but I don't think the diff/merge capabilities would display readable differences.

ADDENDUM:

At first we only had to support Allen-Bradley PLCs. Now we also have Siemens and MicroLogix PLCs. Still searching for a viable solution...

答:
Unfortunately the answer is going to depend on the PLC vendor you're using.  Most of them store their code in proprietary file formats, so it makes it difficult to use regular source control.

Rockwell offers FactoryTalk AssetCenter if you're using Allen-Bradley.  I haven't priced it, but it's likely expensive.  It does more than source control though.

I have used regular (Mercurial) source control with Beckhoff TwinCAT PLC files.  That seems to work OK, but I never had to merge with anyone.  Their new version of TwinCAT (3) coming out later this year is supposed to be built on Visual Studio 2010, and I'm assuming will have much better out-of-the-box offerings for version control integration.  Fingers crossed.

Start Edit

I just wanted to add that I've now used the new TwinCAT 3 product, and I am using Mercurial (TortoiseHg and the VisualHg add-in for Visual Studio).  It is working pretty well.  First of all VisualHg makes it feel very integrated into the Visual Studio 2010 IDE that TwinCAT 3 uses.  However, the source code for TwinCAT 3 programs are generally stored in XML files.  This is a vast improvement over other vendor's proprietary binary formats that I've used, but it still doesn't merge well.  Some of the files don't have any line breaks in the XML (I've written to Beckhoff about this) which means a line-by-line source control system doesn't do much.  Also, since it's XML, the ordering of the nodes in the XML file seems to change randomly, even when you don't make any changes.  Also, I think it sometimes generates new IDs for some nodes when it doesn't have to, which makes for superfluous changes that Hg picks up on.  This effectively makes it impossible to make changes to a TwinCAT 3 program by 2 programmers at the same time, and then merge the changes.  It's an unfortunate oversight by the TwinCAT 3 developers, who undoubtedly use source control regularly in their own work, and didn't see the advantage for us lowly automation programmers to have access to similarly powerful tools.  :(

End Edit

Start Edit #2

I would like to point out that TwinCAT 3.1 now has file formats that are better suited to source control, particularly the structured text language files.  In fact, the product is now built to support integration to Team Foundation Server, I believe.

End Edit #2

The other alternative is that most PLC programs can be exported to text files.  RSLogix 5000, for instance, exports its projects to an L5K file, which is just text.  I've run scripts against those files before - they're fairly easy to parse.  They would work well with source control.  Of course that means exporting every time, which sucks.

If you do go with any standard version control, I highly suggest a distributed VCS, like Git or Mercurial, because with PLCs, half the time you're onsite and can't connect to your home server, so the ability to do local commits is a real bonus.

The other thing you have to realize is that some PLC programming environments, like RSLogix, already include a diff tool, so you can run diffs against two versions of your projects.  This, combined with saving a new file with today's date every day, is what most automation shops seem to get by with.

ContentLicense: CC BY-SA 3.0


问:
backup visual studio folder and sql database
I like to backup things every night or say every week. I'm sure there's a more efficient way than emailing the solution folder and database files to myself! I don't have team foundation or any commercial products like that. I'm looking for a freeware. </p>

答:
I do the same thing quite often actually.  I have a two step process:

1) Backup the database to a child directory in the project folder

2) Backup the project folder to disk

It really is a very, very quick process that isn't even worth a double-take.  If you don't have version control, and especially if you are a sole-developer on the project, I don't see any issue with this.  I recommend keeping a history of about a week of progress (maybe more or less depending on how fast the project is getting developed).


问:
Are software developers who ignore quality/standards better for the company?
Do software developers who choose not to put code optimization, standards and best practices as a top priority, create more useful code than those developers who want to worry about optimization, implementation of coding standards and practices above completing tasks on time?

How do these differing methodologies compare when it comes to individual performance reviews?

How do these styles compare in peer reviews?

What is the best way to influence your team to implement more best practices during the SDLC?

答: No, and I would find the quickest route away from said company that appreciates those vices.


问:
c# Web Content Filter
Can anyone point me to any GOOD open source .NET parental web content filters? I would like to gain an understanding of how I can filter/block web traffic based on Url and/or keyword.

I've looked on google but can't seem to find any developed c#

答:
Usually the content filters use black and white listing. Keywords are pretty unreliable, and unless implemented properly, you will be blocking a lot of sites of distinguished members of society and organizations where the words magna cum laude happen to reside.

There were some cool articles credit to Jeff where he was discussing the things about Bayesian Filters.

http://www.codinghorror.com/blog/2004/09/some-plans-for-spam.html

I think that this is a good start.

Title: c# Web Content Filter


问:
Should a developer also act as a tester?
We're a scrum team of 3 developers, 1 designer, the scrum master, and the product owner. However, we don't have official tester in our team. A problem that is always with us, is that, testing the application and passing those tests and removing bugs has been defined as one of the criteria to consider a PBI (Product Backlog Item) as done.

But the problem is that, no matter how much we (3 developers and 1 designer) try to test the application and implemented use cases, still some bugs are not seen and ruin our presentation (Murphy's law) to stakeholders.

As a remedy, we recommended that the company hire a new tester. Someone who's job would be testing, and testing only. An official professional tester.

However, the problem is that, scrum master and stakeholders believe that a developer (or a designer) should also be a tester.

Are they right? Should we developers (also designers) be testers too?

答:
Ex ante: There seems to be a lot of confusion on what is regarded as testing what is not. Sure, every developer needs to test his code as he creates it, he/she needs to verify it works. She/he can't hand it to a tester before he/she thinks it's done and good enough. But developers don't see everything. They might not recognize bugs. These bugs can only be found later in the development cycle when thorough testing is conducted. The question is whether developers should conduct that kind of testing or not, and in my humble opinion this needs to be looked at from a project manager's point of view:

Developers can be testers, but they shouldn't be testers. Developers tend to unintentionally/unconciously avoid to use the application in a way that might break it. That's because they wrote it and mostly test it in the way it should be used.

A good tester on the other hand, tries to torture the application. His/her primary intention is to break it. They often use the application in ways developers wouldn't have imagined. They're closer to the users than the developer and often times have a different approach to test a workflow.

Also, using developers as testers increases development costs and does not benefit the quality of the product as much as having a dedicated tester. I wouldn't let developers cross-test their works when I can have it done better by a tester for cheap. Only if the feedback loop between developers and testers became too expensive, I'd have developers crosstest each other's code, but in my experience that is rarely the case and it highly depends on the process.

That does not mean a developer should be sloppy and leave everything to the tester. The software should be backed up by unit tests and technical errors should be reduced to a minimum before handing the software to the tester. Still, sometimes you have fix here, break there problems or other bugs that no developer could forsee, that's ok. Also, integration testing should be done mostly by the developers. The tester's main objective is to verify that the requirements are met.

In such a small team (and also depending on the size of the application), I can also see the tester in a hybrid role, writing unit tests and UI tests. You should definitely hire one.

But more important than the tester are regular freezes/branches. Don't present anything that hasn't been properly tested. When you've added a feature or changed something, everything surrounding it has to be verified again. You'll only get a bad reputation if your company doesn't. Don't release something unstable. When the customer wants to have the software by a certain date, then stop developing early enough and test it properly, so you have enough time for bug fixes. Often it's better to decline last-minute feature-requests than to implement them poorly or release without proper testing.


答:
Developers can be testers - of the code of other developers.

But testing your own code is not a good move - developers tend to have mental blocks about their own code, and so have difficulty in designing comprehensive or appropriate tests.

There will always be developers who think they do this well, but usually they don't (and I sure know I have many blind spots).

If you REALLY CANT hire a tester, then get developers to cross test each others work - that is, if A writes the code and does unit tests, then get B to look over those unit tests and see if there are other things could be added. And get B to try and test the code (as a user) and report defects.

This is not perfect but it is better than a single developer trying to do everything.

Sometimes your colleagues can be really good at breaking your software, because they get enjoyment from that and don't care so much - because it is not THEIR code.

Title: Should a developer also act as a tester?

Body:
We're a scrum team of 3 developers, 1 designer, the scrum master, and the product owner. However, we don't have official tester in our team. A problem that is always with us, is that, testing the application and passing those tests and removing bugs has been defined as one of the criteria to consider a PBI (Product Backlog Item) as done.

But the problem is that, no matter how much we (3 developers and 1 designer) try to test the application and implemented use cases, still some bugs are not seen and ruin our presentation (Murphy's law) to stakeholders.

As a remedy, we recommended that the company hire a new tester. Someone who's job would be testing, and testing only. An official professional tester.

However, the problem is that, scrum master and stakeholders believe that a developer (or a designer) should also be a tester.

Are they right? Should we developers (also designers) be testers too?


问:
How to retain student/intern programmers?
We have a small team in our department and typically employ 1-3 students/interns.  The problem we have is retention.  We like to hire a student that will stick around as part of the team for more than just a summer quarter.  The problem is though the last 4 students we hired, up front we discussed this with them and they all ended up leaving earlier than we would have liked.

The relationships all ended in a good fashion, the students either obtained teaching assistant or research positions within the university, or went on to bigger and better things.  This is all understandable and we do not hold it against them for doing what is best for them.

But my perspective is different as I worked within our department for 2 years during undergraduate at which point I was brought on full time upon graduation.

How can we retain students?  We give them hardware they need/want, we provide them with problems to solve in anyway they see fit (within reason).  All in all to me this type of job for a student programmer is so flexible and awesome I cannot see how anyone would want to leave.

Thoughts ?

答:
Maybe you don't have enough great developers to inspire/motivate them to stay

I've had 2 internships at 2 different companies. I just finished the 2nd one a week ago and I realized that I prefer the 1st over the 2nd. The reason is because even though I'm practically doing the same thing at the 2nd company (Web development), most of the developers there are almost as young as I am. Which leads me to think that they don't have enough experience as programmers yet. The best way I can think of for my career to grow and to learn more as a developer is to surround myself with the best programmers that's in my reach. They have those kind of developers over the 1st company that I've been in. That's why I prefer the 1st one over the 2nd one.

Maybe that's one of the reasons why they don't stick around at your company.

ContentLicense: CC BY-SA 2.5


答:
Better for the company? It depends.

For a startup with limited funds, get it done and get it out there - doesn't matter if it's messy, that can be fixed later when there are more resources.

For a mature product where there are many users relying on the quality of the software, everything should be done "properly".

Better for the individual developer? Actually it's irrelevant. Just do the same as what your colleagues are doing, and let management deal with the fallout - that isn't your job. If you're fixing other people's crap all the time you WILL be seen as slow, and you'll be the one who's still there while the others have been promoted above you. Get with the programme, or get out while you can if it's that bad.

Title: Are software developers who ignore quality/standards better for the company?

Body:
Do software developers who choose not to put code optimization, standards and best practices as a top priority, create more useful code than those developers who want to worry about optimization, implementation of coding standards and practices above completing tasks on time?

How do these differing methodologies compare when it comes to individual performance reviews?

How do these styles compare in peer reviews?

What is the best way to influence your team to implement more best practices during the SDLC?


问:
Using GPL libraries without modification on a commercial website, do I need to make my source code available?
If I write a public-facing commercial website, and use a GPL javascript library and/or binaries without modifying them (such as plupload), would I be required to make the source for my website available? Is that considered distribution?

That component is just an example. I know a commercial license is available, but would I need a commercial license if I wasn't modifying the source code of the component?

答:
No, and it doesn't matter what the license says (in the United States). As a matter of law, you are not copying the covered work and you are not distributing the covered work. So you don't need a license. In the United States, the ordinary use of a work does not require any sort of license.

The GPL only applies to people who agree to it. Generally, you agree to the GPL because you want to do something that copyright law says you may not do without a license such as distributing or modifying a work. If you aren't doing anything that copyright law reserves to the copyright holder, you have no need for any license.


问:
Alternatives to Java Applet for interactive Websites
In short I am searching for an alternative to Java applets for making some interactive parts like minigames etc. on my website

Facts:

I´m developing an e-learning website for children.
This site has following main functions:

practice part where children can do grammar test and receive points for good answers.

tamagotchi like pet which moves around, has to be fed and so on

shop in which gotchi stuff can be bought

As a framework I'm using Ruby on Rails.
At the moment I am using Java applets as a "quick and dirty solution" for nearly everything  on the site as I totally freaked out over Flash after some years not using it and learning java in the meanwhile...

(yes I also use applets for the practices which are not much more than some textfields an dropboxes in some text..)

Now I'm searching for a cleaner solution as applets seam to me to be really slow. I  already read some things in the internet but I'm searching for some opinions on my special problem.

My findings until now:

JavaScript seems somehow nice, but many users deactivate it so
would it be wise to use it?

Flash really drove me crazy, but if anyone can give me some reasons for it I will take it in mind again.

Here is what happened when I tried to use it: I learned Flash some years ago and I liked it very much but now I am somehow lost as it seems to me to be not really good programming and I always mix it up with Java commands and I was missing code completion and this stuff.

My main problem when I totally dropped the idea of Flash was that I wanted to read the practices out of some xml file or maybe a database (having in mind creating some programm in which teachers can produce their own practices in some later version) and it seemed somehow near to unsolveable to create some individual buttons in some sort of loop out of this extern data. also the community seemed to be more designer like than programmer like and wasn´t too helpfull on my questions (100 times yes I know I can drag it from the library but thats not at runtime possible)

Java applets are pretty slow I don't think I should use them in next versions

I welcome all hints and ideas how to solve my task.

答:
Flash can be a reasonably comfortable development experience (positively luxurious compared to JS) if you stick to open source tools and avoid the bloated commercial stuff.  Flashdevelop is good, lightweight and has most of what you'd expect in a modern IDE. You'll need the (free) Flex SDK too but I think the installer handles that for you these days.

The thing you have to remember is that all your code in Flash runs on the UI thread, so when you're loading data etc it's all asynchronous. Create a URLLoader, handle the complete event then do what you want with the data. E4X is built in, if you're using XML, and as3corelib (OSS too) has tools for JSON.

"Flash Builder" (Adobe, commercial) has a debugger and a decent profiler, but for simple stuff they're not necessary.

Silverlight is actually less flexible with a steeper learning curve, in my experience.


答:
Should the journalist tend to write correctly? I mean, it is correctors and editors job to find all grammatical errors.

Nevertheless, journalists provide some spellcheck by themselves.
Nevertheless, corrector is a separate and important profession.

The same about developers and testers, except the fact that QA is even more important part of the development. Even if you are a good developer you just
have no time to test thoroughly all test cases, to cover all the environments, browsers, OSes your product is supporting.

If one, besides developing, constantly doing that job also, it just means a simple fact - he is a part-time tester.


问:
How do you manage cross-class dependencies on destruction/design (more of a C++ question)
So if I understand correctly, from SOLID design principles, every class should keep a single responsibility.

So there should be one class that creates and manages a resources, a second class that does resource processing/update, the third class that connects the resource to some other resource and so on.

The question is, how do you cleanly express that in code?

Because, in C++, the class destruction order plays an important role.

So you basically come to a situation where you get

```<code>ResourceClassInstanceManager cf;//does the ResourceClass creation, accounting and destruction?
ResourceClass &rcInstance = cf.GetResourceClass();//creates the instance of resource
ResourceClassUpdater rcu(&rcInstance);//updater that works on the class instance
ResourceConnector rcc(&rcInstance, &instanceOfAnotherClass);
</code>```

The problem here is that if cf leaves the scope first, or rcInstance is deleted before rcu or rcc. The other classes will still reference the nonexisting instance of ResourceClass.

This seems somewhat messy, as the whole creation/destruction chain has to be followed very carefully, and becomes really troubling as soon as you have pointers to objects and deeper nesting levels.

Is there a clean solution for this design problem?

答:
I think that you are using a definition of "responsibility" that is too narrow. Your defintion of "reponsibility" seems to be more like "activity" or "action".

I would expect a resource class' responsibility to be able to be stated in a sentence something like: "Manage the lifetime and access to X, allowing updates and connections to X."

A class is a larger entity than, say, a function so you should expect its responsibility to be broader. A member function would normally have a narrower responsibility but it can also have a single responsibility. For example the Update member function might have the responsiblity of "validiating and executing the requested update to X".


问:
What are the benefits and disadvantages in the approaches of C#, Java and Scala to Closures/Lambdas/...?
I wonder what the technical implementation differences between C# and Scala are and how both solutions compare to the implementation ideas and concerns voiced in the email Peek Past lambda by Brian Goetz, sent to the mailing list of Project Lambda (JSR 335)?

From the email:

We explored the road of "maybe lambdas should just be  inner class
  instances, that would be really simple", but eventually came  to the
  position of "functions are a better direction for the future of  the
  language".

and further:

The lambdas-are-objects view of the world conflicts with this possible
  future.  The lambdas-are-functions view of the world does not, and
  preserving this flexibility is one of the points in favor of not
  burdening lambdas with even the appearance of object-ness.

Conclusion:

Lambdas-are-functions opens doors.  Lambdas-are-objects closes them.

  We prefer to see those doors left open.

And some comment from a person on the Reddit thread says:

I actually e-mailed Neal Gafter about this and to my limited
  understanding of his explaination C# and the current Java design are
  quite similar in that Delegates are actually objects and not function
  types. It seems like he believes that Java should learn from the
  disadvantages of C#'s lambdas and avoid them (much like C# learned
  from Java's disadvantages and avoided them in the beginning).

Why does the "Lambdas-are-functions" approach enable more opportunities in the future than "Lambdas-are-objects"? Can someone explain what differences exist and how they would influence how code would be written?

Seeing that things in Scala "just work", I keep thinking that I'm missing something about the approaches taken/proposed in C#/Java (8), probably it is related to concerns about backward-compatibility?

答:
From what I understand, it's more about how the lambdas are considered at the primary language level.  As the email says,

You might think that the current design is tightly tied to an object box
  for lambdas -- SAM types -- making them effectively objects anyway.  But
  this has been carefully hidden from the surface area so as to allow us
  to consider 'naked' lambdas in the future, or consider other conversion
  contexts for lambdas, or integrate lambdas more tightly into control
  constructs.  We're not doing that now, and we don't even have a concrete
  plan for doing so, but the ability to possibly do that in the future is
  a critical part of the design.

I think that sums up your question quite nicely.  If you declare that "lambdas are objects", then they're just an object of a particular class and you're stuck with that.  On the other hand, if you declare "lambdas are functions", then semantically you have a much richer playing field.  Java 1.7 might compile them down to objects, so they're practically identical at that point.

But Java 1.8, or 1.9, might bring in changes to the language (such as reified structural types) than enable functions to be used in much more flexible ways.  If "lambdas were objects", these changes wouldn't be backwards compatible and you'd have to introduce a new concept altogther, so as not to break people's existing code.  But if javac was just quietly converting lambdas to objects behind the scenes, the new javac can convert them to whatever it wants, so long as the semantics still hold.

OwnerDisplayName: soc

OwnerDisplayName: Andrzej Doyle


答:
I think the discussion regarding objects vs. functions is a red herring.  If the question is, "Is a lambda a function or an object?" the answer should be yes.

That's the point of first-class functions: they aren't treated differently than any other type.  Java already manages to mostly ignore the differences between Object and primitive types (and Scala does even better), so whether a lambda is a subclass of Object or is a new primitive type or something else isn't really important for the language.  What is important is that you can put your lambdas in collections, pass them around to be called, call them, and do anything else you might want to do with either an object or a method.

Scala accomplishes this by using a wrapper object that has a method called apply which can be invoked with just (), making it look just like a method call.  This works splendidly; much of the time you don't need to even care whether you have a method or are calling the apply of a function object.

Title: What are the benefits and disadvantages in the approaches of C#, Java and Scala to Closures/Lambdas/...?

Body:
I wonder what the technical implementation differences between C# and Scala are and how both solutions compare to the implementation ideas and concerns voiced in the email Peek Past lambda by Brian Goetz, sent to the mailing list of Project Lambda (JSR 335)?

From the email:

We explored the road of "maybe lambdas should just be  inner class
  instances, that would be really simple", but eventually came  to the
  position of "functions are a better direction for the future of  the
  language".

and further:

The lambdas-are-objects view of the world conflicts with this possible
  future.  The lambdas-are-functions view of the world does not, and
  preserving this flexibility is one of the points in favor of not
  burdening lambdas with even the appearance of object-ness.

Conclusion:

Lambdas-are-functions opens doors.  Lambdas-are-objects closes them.

  We prefer to see those doors left open.

And some comment from a person on the Reddit thread says:

I actually e-mailed Neal Gafter about this and to my limited
  understanding of his explaination C# and the current Java design are
  quite similar in that Delegates are actually objects and not function
  types. It seems like he believes that Java should learn from the
  disadvantages of C#'s lambdas and avoid them (much like C# learned
  from Java's disadvantages and avoided them in the beginning).

Why does the "Lambdas-are-functions" approach enable more opportunities in the future than "Lambdas-are-objects"? Can someone explain what differences exist and how they would influence how code would be written?

Seeing that things in Scala "just work", I keep thinking that I'm missing something about the approaches taken/proposed in C#/Java (8), probably it is related to concerns about backward-compatibility?

OwnerDisplayName: Rex Kerr


答:
Mostly, he just doesn't want to commit too soon to something. I don't see any reason beyond the erasure one he presented, but that is a very strong reason indeed.

I don't like function types -- I love function types -- but that
  function types fought badly with an existing aspect of the Java type
  system, erasure.  Erased function types are the worst of both worlds.

Consider these methods in Scala's Regex:

```<code>def replaceAllIn (target: CharSequence, replacer: (Match) ⇒ String): String
def replaceSomeIn (target: CharSequence, replacer: (Match) ⇒ Option[String]): String
</code>```

It would be simpler if they were simply this:

```<code>def replace (target: CharSequence, replacer: (Match) ⇒ String): String
def replace (target: CharSequence, replacer: (Match) ⇒ Option[String]): String
</code>```

Unfortunately, that is not possible, because those two functions are identical under erasure. But, fine, these function do somewhat different things, so a different name might is fair enough.

However, a Match is something very useful, but most of the time you want either the matched String or the list of subgroups. We would like to have this:

```<code>def replaceAllIn (target: CharSequence, replacer: (String) ⇒ String): String
def replaceAllIn (target: CharSequence, replacer: (Seq[String]) ⇒ String): String
def replaceAllIn (target: CharSequence, replacer: (Match) ⇒ String): String
</code>```

Not possible, because of erasure. I chose this particular example because I was directly involved with it, but I have seen at least half a dozen questions on Stack Overflow, where people ask why an overload is illegal, caused by this exact issue.

And, then, consider that Scala's pattern matching and Java's instanceof are effectively useless because of erasure.

I think it is fair to delay function types until this issue is dealt with. After all, there are plenty languages that have it on the JVM, and it is not like Java is a quickly evolving language.

Body:
Mostly, he just doesn't want to commit too soon to something. I don't see any reason beyond the erasure one he presented, but that is a very strong reason indeed.

I don't like function types -- I love function types -- but that
  function types fought badly with an existing aspect of the Java type
  system, erasure.  Erased function types are the worst of both worlds.

Consider these methods in Scala's Regex:

def replaceAllIn (target: CharSequence, replacer: (Match) ⇒ String): String
def replaceSomeIn (target: CharSequence, replacer: (Match) ⇒ Option[String]): String

It would be simpler if they were simply this:

def replace (target: CharSequence, replacer: (Match) ⇒ String): String
def replace (target: CharSequence, replacer: (Match) ⇒ Option[String]): String

Unfortunately, that is not possible, because those two functions are identical under erasure. But, fine, these function do somewhat different things, so a different name might is fair enough.

However, a Match is something very useful, but most of the time you want either the matched String or the list of subgroups. We would like to have this:

def replaceAllIn (target: CharSequence, replacer: (String) ⇒ String): String
def replaceAllIn (target: CharSequence, replacer: (Seq[String]) ⇒ String): String
def replaceAllIn (target: CharSequence, replacer: (Match) ⇒ String): String

Not possible, because of erasure. I chose this particular example because I was directly involved with it, but I have seen at least half a dozen questions on Stack Overflow, where people ask why an overload is illegal, caused by this exact issue.

And, then, consider that Scala's pattern matching and Java's instanceof are effectively useless because of erasure.

I think it is fair to delay function types until this issue is dealt with. After all, there are plenty languages that have it on the JVM, and it is not like Java is a quickly evolving language.

OwnerDisplayName: Daniel C. Sobral


问:
How can I comply with Open Source licenses, and yet protect my own work + App Store
I recently developed an application for an organization (and I'm not charging them).
I want to maintain total ownership of the source code and there are various Open Source Project that I've used. I want/need to comply by the rules of the licenses that they use and still keep the project within a few guidelines:

If I ever give the source code out to the organization, It cannot
  be modified, redistributed, or shared in any form without my written
  permission (which needs to be re-obtained every version).

It should always attribute credits to the developer.

I retain all rights to redistribute/reuse the source code, since
  well, I made it

It needs to comply with the App Store's
  rules (which aren't too open-source friendly)

The organization has retained the copyright for the compiled application (if that matters)

The licenses include:

liberal BSD

Apache License, Version 2.0

royalty free

public domain (same as 3)

Small amount of artwork from an MIT licensed project

So my question is how can I comply with these licenses but also create a license for my own project that allows me to retain ownership of the source code.

答:
There is nothing special you need to do. If you don't provide them any license, all they can do is use it. They cannot modify it or redistribute it. Just to make sure there is no confusion, add a notation in the file that it is copyright by you as of some particular date and all rights are reserved.

You cannot give them less than nothing. And with no license, you are not giving them anything.


问:
does correctly applying an open-source license require designating a copyright holder?
Let's say I want to initiate publication of some source code under an open-source license like Apache or MIT, but I know that I'm only the Creator of the source code in its initial form, and I'm unlikely to be the one who maintains it long term. I just want to get it out there and make sure it's free for others to use per one of these licenses.

What do I need to do as far as copyright statement? Do I need to put my name or company down in order for the open-source license to apply? Is there a way that source code can be published anonymously, or without a designated "owner"?

Alternatively, if I put my name or company down, can this ownership be reassigned later?

答: There is no requirement that a copyright holder be designated anymore in pretty much any major country. It is entirely sufficient to put: "All code is copyrighted by its author(s)" (which it is whether you say that or not) or even to put no notice whatsoever.</p>

OwnerDisplayName: Jason S


问:
PBI vs User Story
Recently an item has been added to the Product Backlog by product owner which says "When I go to login page from x page, I see an error. I want that error to be removed".

It seems to me that this is not a use case, and shouldn't be a PBI (Product Backlog Item). However, when I discussed it, scrum master told me that user stories are not PBIs and, a PBI could be a bug report, a task, a user story, anything, and literally any item that should be addressed first.

I'm not sure about this. Also I can't find a good definition of PBI on the web. So, my question is, what sort of things can get into the Product Backlog as items? Does a product backlog item maps to a user story? Are they the same?

答:
Does a product backlog item maps to a user story? Are they the same?

Not necessarily, but in general, they do. Like your scrum master said, other things can be product backlog items, too. However, it depends on how your SCRUM works. Some teams have a separate bug backlog which is taken into consideration for sprints as well, while others keep such things in the product backlog.

Two separate logs make it more difficult for the product owner to prioritize tasks, as now two logs have to be taken into consideration for the next sprint. But they do offer a better oversight and both can be prioritized separately.

So, my question is, what sort of things can get into the Product
  Backlog as items?

This can be anything which is part of the product vision and the journey to the product you want to create. It mostly contains requirements (user stories) but can also contain actions or technical things that do not directly belong to the product (e.g. "Buy a new server for dev team", "Create advertisement for product"). The backlog should avoid uncessary details and should not try to micromanage technical things. The product backlog can contain anything that delivers value to the product.

There's not the one true Scrum. Sometimes separate backlogs are a better way to manage the product, sometimes they are just in the way. Find out what works for you best.

Title: PBI vs User Story


问:
What is more standard shortening for "regular expression", "regex" or "regexp"?
I want to know what is more standard way to call regular expression in short way. I have seen both, regex and regexp.

Google says regex has almost 2 as much hits that regexp, but it depends on way I search it (when I add "regular expression" after the word, "regex" becomes "regex regular expression", the result is differend). And on stackexchange there is regex tag. :)

What is more proper way to shorten it?

Perhaps it does not matter. :) But thanks for answer.

答: There is no standard that dictates this. As to what is more common, Google says regex vs. regexp by some 13.8 vs 9.8 million finds.


答:
@Falcon has explained it well. One page that has a formal definition is:
http://en.wikipedia.org/wiki/Scrum_(development)#Product_backlog
What you have described is not to be placed in product backlog according to that discription at least.</p>

Title: PBI vs User Story

Body:
Recently an item has been added to the Product Backlog by product owner which says "When I go to login page from x page, I see an error. I want that error to be removed".

It seems to me that this is not a use case, and shouldn't be a PBI (Product Backlog Item). However, when I discussed it, scrum master told me that user stories are not PBIs and, a PBI could be a bug report, a task, a user story, anything, and literally any item that should be addressed first.

I'm not sure about this. Also I can't find a good definition of PBI on the web. So, my question is, what sort of things can get into the Product Backlog as items? Does a product backlog item maps to a user story? Are they the same?


答: As some indication - there's a Regex class in .NET and regex package in Java; not Regexp. Two fairly large 'ecosystems'. This by itself means more to me than as if some committee, somewhere, decided it this way or that way.

Title: What is more standard shortening for "regular expression", "regex" or "regexp"?

Body:
I want to know what is more standard way to call regular expression in short way. I have seen both, regex and regexp.

Google says regex has almost 2 as much hits that regexp, but it depends on way I search it (when I add "regular expression" after the word, "regex" becomes "regex regular expression", the result is differend). And on stackexchange there is regex tag. :)

What is more proper way to shorten it?

Perhaps it does not matter. :) But thanks for answer.


问:
Who does test-driven development?
I've been working in the enterprise space for the past 4½ years and have noticed that generally speaking, enterprises are not conducive environments for the test-first style of development. Projects are usually fixed-cost, fixed-timeline and waterfall style. Any unit testing, if done at all, usually comes after development in the QA phase and done by another team.

Prior to working for an enterprise, I consulted for many small to medium sized companies, and none of them were willing to pay for a test-first style of development project. They usually wanted development started immediately, or after a short design stint: i.e., something more akin to Agile, though some clients wanted everything mapped out similar to waterfall.

With what types of shops, companies, and clients does test-driven development work
best? What types of projects tend to be conducive to TDD?

答:
I wanted to ask this very question, to see how many companies were actually practicing TDD.

In the 11 years I've been programming professionally only the last two organisations were even aware of TDD (that spans almost 5 years mind, before which time TDD wasn't as popular as it is today). I'll cut to the chase and answer your question before digressing into my sales pitch for TDD :)

At the last company I worked for (online academic publisher of humanities and science collections), we knew we needed to practice TDD but we never quite got there. In our defence we had a 250k code base, so adding tests to an untestable code base of that size felt insurmountable (I feel guilty typing that now!). Even the best of us make mistakes.

Anyone who's done even a small amount of TDD knows how painful retrofitting tests to brown field untestable code can be ... The primary causes are implicit dependencies (you can't pull all the levers to assert results from code - you can't mock scenarios) and violation of the single responsibility principle (tests are complicated, contrived, require too much setup and are hard to understand).

We temporarily grew our QA team (from one, maybe two people to half a dozen or more) to test the platform before any release. It was hugely expensive time wise and financially, some releases would take three months to complete 'testing'. Even then we knew we were shipping with issues, they just weren't 'blockers' or 'critical', just 'high-priority'.

If you've a years commercial experience you'll appreciate that every company asserts critical tasks, and then invents a higher priority level above that, and most likely one above that too - especially when someone from above is pushing a feature/bug fix. I digress ...

I'm happy to report I'm practicing TDD in my current company (telecommunications, web and mobile app development house), coupled with Jenkins CI to give other static analysis reports (code coverage being the most useful after asserting the test suite passes). The projects I've used TDD on are a payment system and grid computing system.

The sales pitch ...

It can often be an uphill struggle justifying automated testing to non-technical team members. Writing tests does add more work to the development process but ... the time you invest in testing now, you'll save in maintenance effort later. You're really just borrowing time. The longer the product is in use, the greater saving you'll make - and it'll help you avoid the big rewrite.

Test first means you're coding your intent first, and then confirming your code fulfills that intent. This provides focus and distills your code to do only what is intended and no more (read no bloat). It's an executable specification and documentation at the same time (if your test is well written, and tests should be as readable/clean as your system code, if not more!).

Non-programmers will (often) not have this insight and so TDD doesn't hold much value for them, and is seen as disposable shortcut to an earlier release date.

Programming is our domain, and in my mind this makes it our responsibility, as professionals, to advise on best practice like TDD. Not for project managers to decide if it's done to reduce development time, it's out of their jurisdiction. In the same way they don't tell you what framework, caching solution or search algorithm to use, they shouldn't tell you if you should be employing automated testing.

In my opinion the software development industry (on the whole) is broken at present, the fact that having tests for your software is NOT the norm.

Picture this in other industries: medical, aviation, automobile, cosmetics, soft toys, alcoholic beverages etc. I asked my fiancee to name an industry where they don't test the product and she couldn't!

Perhaps it's unfair to say no testing occurs because it does ... but in companies without automated testing, it's very manual/human (read clunky and often error prone) process.

One point I would contend in your question ...

They usually wanted development started immediately, or after a short design stint. More akin to Agile.

Being "Agile" doesn't prescribe proceeding without tests, the first member listed on agilemanifesto.org is Kent Beck, the creator of XP and TDD!

Two books I would highly recommend if you're interested in TDD, or just haven't read them and are a keen programmer (that's everyone reading this right? ;)

Growing Objected Oriented Software Guided by Tests

Clean Code - Robert C Martin ("Uncle Bob") Series

These two books compliment one another and condense a lot of sense into few pages.

Thanks for asking this question :)

Title: Who does test-driven development?


答: Either get a free SVN server for code & related stuff or get a generic backup software that can store files where you want (I user GFI Backup and an external HDD for pictures & stuff)</p>

Title: backup visual studio folder and sql database

Body: I like to backup things every night or say every week. I'm sure there's a more efficient way than emailing the solution folder and database files to myself! I don't have team foundation or any commercial products like that. I'm looking for a freeware. </p>


答: Go to Assembla.com. Set up a free private space. Select a version control system of your preference. Each your commit can be treated as a backup.</p>


问:
What is understood under "unit" in unit testing
As I understand in theory under "unit" people mean method (in OOP). But in practice tests which verify some method in isolation are very fragile behaviour tests (verifying not the result but the fact that some dependency method was called). So I see a lot of people who by unit understand a small set of closely related classes. In this case only outer dependencies are mocked/stubbed and for dependencies which are inside unit real implementations are used. In this case there are more state, meaningful (according to specification) and not so fragile tests. So the question is how do you feel about these approaches and is it valid to call the second approach unit testing or may be it is some kind of low-level integration testing?

If you see some specific considerations about applying TDD by one of these ways of testing, I would be thankful for your thoughts.

答:
Originally TDD came from the agile movement, where testing was written up front as a way to ensure what you coded remained correct given the specification that was now well-defined in the test code. It also appeared as a very important aspect of refactoring, in that when you modified your code, you could rely on the tests to prove that you hadn't changed the code's behaviour.

Then the tools people came along and thought they knew information about your code and could then generate test stubs to assist you in writing your unit tests, and I think this is where it all went wrong.

The test stubs are generated by a computer that has no clue what it is you're doing, it just mindlessly produces a stub for every method because that's what it's told to do. This means you have a test case for each method regardless of the complexity of that method or whether its suitable for testing in isolation.

This is coming at testing from the wrong end of the TDD methodology. In TDD you're supposed to figure out what the code is to do, and then produce code that achieves this. This is self-fulfilling in that you end up writing tests that prove the code does what the code does, not what its supposed to do. Combined with the auto-generation of method-based test stubs, you pretty much waste your time proving each tiny aspect of your code that can so easily prove to be wrong when all the little pieces are put together.

When Fowler described testing in his book, he referred to testing each class with its own main method. He improved this, but the concept is still the same - you test the entire class so it works as a whole, all your tests are bundled together to prove the interaction of all those methods so the class can be reused with defined expectations.

I think the test toolkits have done us a disservice, led us down the path of thinking that the toolkit is the only way to do things when really, you need to think more for yourself to get the best result from your code. Blindly putting test code in test stubs for tiny pieces just means you have to repeat your work in an integration test anyway (and if you're going to do that, why not skip the now-redundant unit test stage completely). It also means people waste a lot of time trying to get 100% test coverage, and a lot of time creating large amounts of mocking code and data that would have been better spent making the code easier to integration test (ie if you have that much data dependencies, unit test might not be the best option)

Lastly, the fragility of method-based unit tests just shows up the problem. Refactoring is designed to be used with unit tests, if your tests break all the time because you're refactoring then something has gone seriously wrong with the whole approach. Refactoring likes to create and delete methods, so obviously the blind per-method based test approach is not what was originally intended.

I have no doubts that many methods will get tests written for them, all of the public methods of a class should be tested, but you cannot get away from the concept of testing them together as part of a single test case. For example, if I have a set and a get method, I can write tests that put the data in and check the internal members are set ok, or I can use each to put some data and then get it out again to see if it's still the same and not garbled. This is testing the class, not each method in isolation. If the setter relies on a helper private method, then that's fine - you don't need to mock the private method to ensure the setter is working, not if you test the entire class.

I think the religion is getting into this topic, hence you see the schism into what's now known as 'behaviour-driven' and 'test-driven' development - the original concept of unit testing was for behaviour driven development.


答: As long as none of the licenses is 'viral' (such as GPL), you can use, modify and redistribute the code under any license you like. Some of those licenses require you to put their copyright notices into your code and documentation, but that's about the only practical consequence.</p>

Title: How can I comply with Open Source licenses, and yet protect my own work + App Store

Body:
I recently developed an application for an organization (and I'm not charging them).
I want to maintain total ownership of the source code and there are various Open Source Project that I've used. I want/need to comply by the rules of the licenses that they use and still keep the project within a few guidelines:

If I ever give the source code out to the organization, It cannot
  be modified, redistributed, or shared in any form without my written
  permission (which needs to be re-obtained every version).

It should always attribute credits to the developer.

I retain all rights to redistribute/reuse the source code, since
  well, I made it

It needs to comply with the App Store's
  rules (which aren't too open-source friendly)

The organization has retained the copyright for the compiled application (if that matters)

The licenses include:

liberal BSD

Apache License, Version 2.0

royalty free

public domain (same as 3)

Small amount of artwork from an MIT licensed project

So my question is how can I comply with these licenses but also create a license for my own project that allows me to retain ownership of the source code.


问:
Software Design Idea for multi tier architecture
I am currently investigating multi tier architecture design for a web based application in MVC3. I already have an architecture but not sure if its the best I can do in terms of extendability and performance.

The current architecure has following components

DataTier (Contains EF POCO objects)

DomainModel (Contains Domain related objects)

Global (Among other common things it contains Repository objects for CRUD to DB)

Business Layer (Business Logic and Interaction between Data and Client and CRUD using repository)

Web(Client) (which talks to DomainModel and Business but also have its own ViewModels for Create and Edit Views for e.g.)

Note: I am using ValueInjector for convering one type of entity to another. (which is proving an overhead in this desing. I really dont like over doing this.)

My question is am I having too many tiers in the above architecure?
Do I really need domain model? (I think I do when I exposes my Business Logic via WCF to external clients).

What is happening is that for a simple database insert it

(1) create ViewModel

(2) Convert ViewModel to DomainModel for Business to understand

(3) Business Convert it to DataModel for Repository and then data comes back in the same order.

Few things to consider,

I am not looking for a perfect architecure solution as it does not exits.
I am looking for something that is scalable.
It should resuable (for e.g. using design patterns ,interfaces, inheritance etc.)
Each Layers should be easily testable.

Any suggestions or comments is much appriciated.

Thanks,

答:
I'll be very general so sorry if it is nothing new for you, I know that's boring:

adding a layer adds flexibility but costs some performance

adding a layer does not improve scalability
(it has to be addressed separately at every layer that creates bottleneck)

I do not know your stack, but similar applications usually consists of database, middleware (domain object with their mapping to database, business logic) and frontend (web interface for humans).

Depending on situation some layers can be merged (embedded database, business logic and frontend as single web app) - that is OK as long as you keep the dependencies clean (no cycles) and you already know where to cut to divide the siamese twins when it becomes necessary.

OwnerDisplayName: activebiz


答: Regex is much more common, and you might get an odd glance or two if you say Regexp.


答:
Lets start with the code you write and ignore the other licenses for a moment. You retain copyright of any code you write unless you expressly give that copyright up (sell it, doing work for hire, etc). You say that the organization has retained copyright for the compiled application. This is confusing. I don't see how you can have copyright of your code and they can have copyright of the compiled application as either:

You could compile your code again and have an application they have copyright over, or

You retain copyright of the code but can never build it as the org has copyright over the resulting executable

This sounds like a legal minefield, and its fairly clear you need to consult a lawyer if you want to leave it as it is but sort it out at a later date.

Since you are doing the work for free, I would recommend you sit down with the org and work out a license between the two of you that is acceptable for both parties. Which probably means that the org has no copyright claims to the application or source code, but express permission to use without fee the application (and possibly permission to use the source and modify it in the future without fee, but I guess no license to sell or distribute the source code). Other wise you could turn around in 6 months and say, any changes to the app will cost this much and I'm not giving you the source code. It is unclear whether the work you are doing is covered by a contract. This is where you would put such points.

So think about that very carefully, and protect your rights, but be professional and protect your client as well.

Now for the mixed licenses.

Public domain code you don't need to worry about.

Royalty free is a loaded term. It literally means you don't need to pay royalties for it. I assume the author means freely open source, with no requirement to pay royalties, advertise, include a copyright notice etc, but I can only amuse that, you have access to the actual license.

BSD licenses require that you include a copyright notice, you will be obliged to do this, but there is no requirement to provide source code or relinquish copyright.

Apache is like BSD, it requires copyright and disclaimer notices to be distributed with code/application but is permissive and doesn't require you to release source code etc.

Once again the MIT license is permissive requiring only that a statement be included with code and application.

If you  write code using other peoples code with he mentioned licenses you will retain copyright of it and you will have no requirement to provide that code to any one else. you will be able to sell that code, or license that code as you see fit. you will however have to reprint the associated copyright disclaimers. None of these licenses will jeopardize your ownership.

However you still need to figure out how you license your application and or code to the organization you are dealing with. And the usual warning applies, consult a lawyer if any of this is important to you. They have the experience and knowledge to make better sense of this then us.


答:
Well, we had two developers cross test after the first one made some changes to an entry screen. This was when our regular tester was off on maternity leave.

He basically changed an Invoice Listing screen that the users used to select invoices before zooming in to do some editing via an "Edit" button. The original list was thrown out and a new gridview inserted, with filtering, grouping, sorting and all sorts of cool functionality.

Testing went great and they uploaded the changes to the customer the next day. Two weeks later, the customer calls up and says "We really like the new thingy you put in, we can see all sorts of information now. But... er..... where do we go to edit the invoices now???"

Turns out the developer took out the check box (for selection) and edit button and since the developers always double clicked to select an item anyway, none of them found anything wrong with it......

Developers and users live in different worlds, having cross testing is better than having the developer test his own work but its still not quite the same thing.


答: what's more standard depends on the predominant language of the folks you're talking to, I would argue. Correct above about Regex in .NET and Java, but Ruby uses the Regexp class for regular expressions.


问:
Are custom DSLs still viable?
Are custom DSLs in JavaScript still viable, Specifically ones written with Antler, Boo, or the former Microsoft Oslo?

With the proliferation of open source JavaScript/ECMAScript engines (V8, IronJS, etc.), does it make more sense to write the functionality needed into one of those instead of writing an entirely new language?

答:
If you mean DSL in strict terms, as in a programming language spec restricted to only one domain, then yes. The proliferation of the JavaScript/ECMAScript engines actually increases their viability - given a common virtual machine which is as flexible as the ECMAScript ones tend to be, DSL are both much easier to create, and likelier to be encountered.

What you won't see as much of, though, are languages with their own dedicated environments. At the high end, you'll see compilers to the existing languages (ala the Java to Javascript compiler that Google put out). Much more commonly, you'll see a library of supporting functions coupled with a programming language/style enforced by api restrictions and convention (the api to the jQuery library could be called a DSL).

Title: Are custom DSLs still viable?


答:
When working on bugs, we add them to the backlog and call them bug stories.  By adding bug fixes te the backlog in this way, it's clear it's not just the bug fix. We can add other tasks to make sure automated tests are written and verification is done. It also makes it more explicit that the DoD should be followed.

We've never used the term PBI (even though our backlog tool calls them that), it's always user stories, bug stories or simply just stories.

It's mainly just your team's choice of terminology and as long as you're all clear what is what it doesn't really matter.

Title: PBI vs User Story


问:
Should I stick with or abandon Python to deal with concurrency?
I have a 10K LOC project written in Django with quite a deal of Celery (RabbitMQ) for asynchronicity and background jobs where needed, and have come to the conclusion that parts of the system would benefit from being rewritten in something other than Django for better concurrency. Reasons include:

Signals handling and mutable objects. Especially when one signal triggers another, handling them in Django using the ORM can be surprising when instances change or disappear. I want to use some messaging approach where the data passed along doesn't change in a handler (Clojure's copy-on-write approach seems nice, if I got it right).

Parts of the system are not web-based, and need better support for performing tasks concurrently. For example, the system reads NFC tags, and when one is read an LED goes on for a few seconds (Celery task), a sound is played (other Celery task), and the database is queried (other task). This is implemented as a Django management command, but Django and its ORM being synchronous by nature and sharing memory is limiting (we are thinking of adding more NFC readers, and I don't think that the Django+Celery approach will work any longer, I'd like to see better message-passing capabilities).

What are the pros and cons of using something like Twisted or Tornado compared with going for a language such as Erlang or Clojure? I am interested in practical benefits and detriments.

How did you come to the conclusion that some parts of the system would fare better in another language? Are you suffering performance problems? How severe are those problems? If it can be faster, is it essential that it is faster?

Example 1: Django at work outside an HTTP request:

An NFC tag is read.

The database (and possibly LDAP) is queried, and we want to do something when data becomes available (red or green light, play a sound). This blocks using the Django ORM, but as long as there are Celery workers available it doesn't matter. May be a problem with more stations.

Example 2: “message-passing” using Django signals:

A post_delete event is  handled, other objects may be altered or deleted because of this.

At the end, notifications should be sent to users. Here, it would be nice if arguments passed to the notification handler were copies of deleted or to-be-deleted objects and guaranteed not to change in the handler. (It could be done manually simply by not passing objects managed by the ORM to handlers, of course.)

答:
Opening Thoughts

How did you come to the conclusion that some parts of the system would fare better in another language? Are you suffering performance problems? How severe are those problems? If it can be faster, is it essential that it is faster?

Single-Thread Asynchrony

There are several questions and other web resources that already deal with the differences, pros, and cons of single-thread asynchrony vs. multi-thread concurrency. It’s interesting to read about how Node.js's single-thread asynchronous model performs when I/O is the major bottleneck, and there are many many requests being serviced at once.

Twisted, Tornado, and other asynchronous models make excellent use of a single-thread. Since a lot of web programming has lots of I/O (network, database, etc.), the time spent waiting on remote calls adds up significantly. That is time that could be spent doing other things—like kicking off other database calls, rendering pages, and generating data. The utilisation of that single-thread is extremely high.

One of the greatest benefits of single-thread asynchrony is that it uses much less memory. In multi-thread execution, each thread requires a certain amount of reserved memory. As the number of threads increases, so does the amount of memory required just for the threads to exist. Since memory is finite, it means there are bounds on the number of threads that can be created at any one time.

Example

In the case of a web server, pretend each request is given its very own thread. Say 1MB of memory is required for each thread, and the web server has 2GB of RAM. This web server would be capable of processing (approximately) 2000 requests at any point in time before there just isn't enough memory to process any more.

If your load is significantly higher than this, requests are going to take a very long time (when waiting for older requests to complete), or you’re going to have to throw more servers into the cluster to expand the number of concurrent requests possible.

Multi-thread Concurrency

Multi-thread concurrency instead relies on executing several tasks at the same time. That means that if a thread is blocked waiting on a database call to return, other requests can be processed at the same time. Thread utilisation is lower, but the number of threads executing is much larger.

Multi-thread code is also much harder to reason about. There are issues with locking, synchronisation, and other fun concurrency problems. Single-thread asynchrony doesn't suffer from the same problems.

Multi-thread code is much more performant for CPU intensive tasks, however. If there exists no opportunities for a thread to “yield”—such as a network call that would normally block—a single-thread model just isn't going to have any concurrency whatsoever.

Both May Coexist

There's of course overlap between the two; they are not mutually exclusive. For instance, multi-thread code can be written in a non-blocking way, to better utilise each thread.

The Bottom Line

There are many other issues to consider, but I like to think about the two like this:

If your program is I/O bound, then single-thread asynchrony is probably going to work quite well.

If your program is CPU bound, then a multi-thread system will probably be best.

In your particular case, you need to determine what kind of asynchronous work is being completed, and how often those tasks arise.

Do they occur on every request? If so, memory is probably going to become an issue as the number of requests rise.

Are these tasks ordered? If so, you’re going to have to consider synchronisation if using multiple threads.

Are these tasks CPU intensive? If so, is a single-thread able to keep up with the load?

There's no simple answer. You must consider what your use cases are, and design accordingly. Sometimes an asynchronous single-thread model is better. Other times, using a number of threads to achieve massive parallel processing is required.

Other Considerations

There are other issues you need to consider also, rather than just the concurrency model you choose. Do you know Erlang or Clojure? Do you think you'd be capable of writing safe multi-thread code in one of these languages such that you improve the performance of your application? Is it going to take a long time to get up to speed in one of these languages, and will the language you learn benefit you in the future?

How about the difficulties associated with communication between these two systems? Will it be overly complex maintaining two separate systems in parallel? How will the Erlang system receive tasks from Django? How will Erlang communicate those results back to Django? Is performance significant enough a problem that the added complexity is worth it?

Final Thoughts

I've always found Django to be quick enough, and it is used by some very heavily trafficked sites. There are several performance optimisations you can make to increase the number of concurrent requests and response time. Admittedly, I haven’t done anything with Celery thus far, so the usual performance optimisations are probably not going to solve any issues you may be having with these asynchronous tasks.

Of course, there’s always the suggestion of throwing more hardware at the problem. Is the cost of provisioning a new server cheaper than the development and maintenance cost of an entirely new subsystem?

I've asked far too many questions at this point, but that was my intention. The answer is not going to be easy without analysis and further detail. Being able to analyse the problems comes down to knowing the questions to ask, though…so hopefully I've helped on that front.

My gut feeling says that a rewrite in another language is unnecessary. The complexity and cost will probably be too great.

Edit

Response to Follow-Up

Your follow-up presents some very interesting use cases.

1. Django working outside HTTP requests

Your first example involved reading NFC tags, then querying the database. I don’t think that writing this part in another language will be that useful to you, simply because querying the database or an LDAP server is going to be bound by network I/O (and potentially database performance). On the other hand, the number of concurrent requests will be bound by the server itself, since each management command will be run as its own process. There will be setup and teardown time that impacts performance, since you aren't sending messages to an already running process. You will, however, be able to send multiple requests at the same time, since each will be an isolated process.

For this case, I see two avenues you can investigate:

Ensure that your database is capable of handling multiple queries at once with connection pooling. (Oracle, for example, requires that you configure Django accordingly 'OPTIONS': {'threaded':True}.) There may be similar configuration options at the database level or Django level that you can tweak for your own database. No matter what language you write your database queries in, you will have to wait for this data to return before you can light up the LEDs. The performance of the querying code can make a difference though, and the Django ORM isn't lightning fast (but, usually fast enough).

Minimise setup/teardown time. Have a constantly running process, and send messages to it. (Correct me if I’m wrong, but this is what your original question is actually focusing on.) Whether this process is written in Python/Django or another language/framework is covered above. I don’t like the idea of using management commands so frequently. Is it possible to have a small piece of code running constantly, that pushes messages from the NFC readers onto the message queue, which Celery then reads and forwards to Django? The setup and teardown of a small program, even if it’s written in Python (but not Django!), should be better than starting and stopping a Django program (with all of its subsystems).

I’m unsure what web server you’re using for Django. mod_wsgi for Apache lets you configure the number of processes and threads within the processes that service requests. Be sure to tweak the relevant configuration of your web server to optimise the number of serviceable requests.

2. “Message-passing” with Django signals

Your second use case is also fairly interesting; I’m not sure if I have the answers for that. If you’re deleting model instances, and wish to operate on them later, it might be possible to serialize them JSON.dumps and then deserialize JSON.loads. It’ll be impossible to fully recreate the object graph later on (querying related models), since related fields are lazy loaded from the database, and that link will no longer exist.

The other option would be to somehow mark an object for deletion, and only delete it at the end of the request/response cycle (after all signals have been serviced). It might require a custom signal to implement this, rather than relying on post_delete.

OwnerDisplayName: Simon Pantzare


问:
What's the difference between ASP.NET Web Applications and Websites?
Currently, I am reading Pro ASP.NET 4 in C# and in it, two ways of creating a web application are described:

Website

Web Application Project

In which scenarios should I choose the former over the latter, and vice versa?

答:
Microsoft made a table listing the differences between the two models, and when to use one instead of the other, check out http://msdn.microsoft.com/en-us/library/aa730880%28VS.80%29.aspx#wapp_topic5
This is exactly what you are looking for, and the explanations are very easy to understand.

In short: "web application projects" may be formed of multiple Visual Studio projects and are also to be used for websites designed around the MVC pattern. Whereas a "Web site" project is just that, one "web" project and all files must be contained inside (you may however link to other assemblies); it is better suited for old-school ASP.Net websites.

That article I linked to above gives a lot more information, check it out!


答: There is a HUGE list of reasons to use one over the other, but usually when I'm making a decision I base it on a few parameters.  For instance, do you want your code files to be out in the open (well, still behind server security) or would you rather they are compiled into code libraries?</p>

Title: What's the difference between ASP.NET Web Applications and Websites?

Body:
Currently, I am reading Pro ASP.NET 4 in C# and in it, two ways of creating a web application are described:

Website

Web Application Project

In which scenarios should I choose the former over the latter, and vice versa?


答:
Practical problems with Twisted (which I love and have used for about five years):

The documentation leaves something to be desired, and the model is quite complex to learn anyway. I find it hard to get other Python programmers to work on Twisted code.

I ended up using blocking file I/O and database access for lack of good blocking APIs. This can really hurt performance.

There does not seem to be a huge community and healthy community using Twisted; for instance Node.js has a lot more active development especially for web back-end programming.

It is still Python, and at least CPython isn't the fastest thing around.

I have done a bit of work using Node.js with CoffeeScript and if concurrent performance is your concern then that may be worth the leap.

Have you considered running multiple instances of Django with some arrangement to spread clients between instances?

Title: Should I stick with or abandon Python to deal with concurrency?

Body:
I have a 10K LOC project written in Django with quite a deal of Celery (RabbitMQ) for asynchronicity and background jobs where needed, and have come to the conclusion that parts of the system would benefit from being rewritten in something other than Django for better concurrency. Reasons include:

Signals handling and mutable objects. Especially when one signal triggers another, handling them in Django using the ORM can be surprising when instances change or disappear. I want to use some messaging approach where the data passed along doesn't change in a handler (Clojure's copy-on-write approach seems nice, if I got it right).

Parts of the system are not web-based, and need better support for performing tasks concurrently. For example, the system reads NFC tags, and when one is read an LED goes on for a few seconds (Celery task), a sound is played (other Celery task), and the database is queried (other task). This is implemented as a Django management command, but Django and its ORM being synchronous by nature and sharing memory is limiting (we are thinking of adding more NFC readers, and I don't think that the Django+Celery approach will work any longer, I'd like to see better message-passing capabilities).

What are the pros and cons of using something like Twisted or Tornado compared with going for a language such as Erlang or Clojure? I am interested in practical benefits and detriments.

How did you come to the conclusion that some parts of the system would fare better in another language? Are you suffering performance problems? How severe are those problems? If it can be faster, is it essential that it is faster?

Example 1: Django at work outside an HTTP request:

An NFC tag is read.

The database (and possibly LDAP) is queried, and we want to do something when data becomes available (red or green light, play a sound). This blocks using the Django ORM, but as long as there are Celery workers available it doesn't matter. May be a problem with more stations.

Example 2: “message-passing” using Django signals:

A post_delete event is  handled, other objects may be altered or deleted because of this.

At the end, notifications should be sent to users. Here, it would be nice if arguments passed to the notification handler were copies of deleted or to-be-deleted objects and guaranteed not to change in the handler. (It could be done manually simply by not passing objects managed by the ORM to handlers, of course.)


答: I have never before this thread seen it shortened to regexp and always seen it as regex. Just to add to the list, the new C++ Standard has <regex>, not <regexp>.


答:
There's no right or wrong abbreviation here. It'll mostly be influenced by what APIs you're working with:

It's Regex!

java.util.regex in Java

Regex in .NET

regex in GLib

It's Regexp!

Regexp in Ruby

RegExp in JavaScript

QRegExp in Qt

It's something else entirely!

re in Python

preg in PHP

NSRegularExpression in iOS/Mac OS

// in Perl ;)

The fact that Jeffrey Friedl, author of the excellent Mastering Regular Expressions, lives at regex.info is good enough for me: I tend to call them regexes. But I keep this close at hand just in case I change my mind later...

```<code>s/\bregex\b/$&p/gi;
</code>```

Body:
There's no right or wrong abbreviation here. It'll mostly be influenced by what APIs you're working with:

It's Regex!

java.util.regex in Java

Regex in .NET

regex in GLib

It's Regexp!

Regexp in Ruby

RegExp in JavaScript

QRegExp in Qt

It's something else entirely!

re in Python

preg in PHP

NSRegularExpression in iOS/Mac OS

// in Perl ;)

The fact that Jeffrey Friedl, author of the excellent Mastering Regular Expressions, lives at regex.info is good enough for me: I tend to call them regexes. But I keep this close at hand just in case I change my mind later...

s/\bregex\b/$&p/gi;


答:
Custom DSL's are still viable, but you have other options which could be more or less complex depending on the problem you are trying to solve. For example, you can implement DSLs using dynamic languages. (DSLs using Groovy is one example that comes to mind) Which can then be compiled or translated it yet other languages. This can have a lower upfront cost, but may you might find that it is not the right fit in your case.

Perhaps you should start with defining the domain in question, and put together a list of things you would need from a DSL.

Title: Are custom DSLs still viable?

Body:
Are custom DSLs in JavaScript still viable, Specifically ones written with Antler, Boo, or the former Microsoft Oslo?

With the proliferation of open source JavaScript/ECMAScript engines (V8, IronJS, etc.), does it make more sense to write the functionality needed into one of those instead of writing an entirely new language?

ContentLicense: CC BY-SA 4.0


答:
My answer is more morphological. Consider yourself as the first person who wants to provide an abbreviation for this concept (Definitely regex or regexp both come from somewhere and haven't been with us since Adam), and you know that it should come from regular and expression.

My dictionary tells me that reg. is an abbreviated form of regularly and exp. is a shorthand of express. Thus if you want to follow the rules of abbreviations, REGEXP seems more structural. However, forming new words (neologism as they call it) is not always straight and often is done by people who are not linguists. Considering these issues, to make a name which with a more aesthetically appealing pronunciation, you can create REGEX.

Also as a note, StackExchange editor considers regex as wrong spell, while regexp is fine :)


问:
Languages to complement C++
I primarily work in C++.  My question is, for a C++ developer, what languages are beneficial to learn, from a job-profile point of view.  For example, I see a lot of work combining C++ and Php these days.  Conversely, I don't see a lot of work combining Ruby and C++ (unfortunately).

Of course, learning another language is beneficial regardless of whether it compliments a language.

答:
I'd vote for Python or Ruby.  C++ is The Language for doing as much as possible statically at compile time.  Learning Python or Ruby will teach you how to think in terms of doing things dynamically at runtime.  I'm in a similar situation, using D and Python as my primary languages.  D's template metaprogramming is much more powerful than C++'s, but it's still all at compile time.  Python's metaprogramming and reflection is all at runtime.  It's a very useful combination because often there are idioms that are useful in both languages, but are more obvious in one than the other.  Therefore, my D experience makes me a better Python programmer and vice-versa.

On another note, Python and C++ complement each other in that C++ is good at performance and bad at programmer productivity, where Python is just the opposite.  If you learn both, you'll have mastered both ends of the spectrum on this tradeoff.

Title: Languages to complement C++


问:
What does it mean to drop support for Internet Explorer 6?

When Facebook, Youtube, and even Microsoft themselves say that they've "dropped support for IE6", what do they really mean? Youtube still functionally works fine in IE6. If they mean that they've dropped support for visual appearance and fancy features, then their point is moot. Web developers have degraded IE6's appearance and fancy feature set since the dawn of time.

So what does all this IE6 bashing from top companies mean for us web developers today? There's no big website that has dropped dropped functionality support for IE6 because they're scared that they would lose the 5% or so customers that are still using it. Until the day Google itself blocks IE6 from searching, thereby lowering IE6's market share to near 0%, I don't see how this "support dropping" accomplishes anything.

答: It means that they no longer test the site in IE6 and, when it inevitably breaks, they will not make fixing it a priority.</p>


问:
Is it possible to get a job in a high-demand company without having hobby projects?
I was curious if the recruitment team at high-demand companies such as Google takes a lack of hobby projects into consideration when evaluating candidates.

I'm a straight 40 hour/week programmer, who is lucky to spend an hour or two a month outside of work on anything programming related. I love hanging out on SO/SE during my breaks at work, and love answering questions, but after work I leave the programming world and go back to my life.

I already understand that you don't need hobby projects to be a good programmer, but does this lack of hobby projects affect my chances getting a job at a company that always has a long list of candidates trying to get in?

答:
For the company that is, as you noted, high-demand the hiring market is a buyers market - they get to choose who they want, and they have ample supply of candidates. The natural motivation for them is to choose a candidate that would be most profitable for the company. Now who is better for them - somebody who doesn't have a life outside of code and would obsessively invest his time into cool projects the company throws at them - or somebody who does his part but clocks out every day at 5pm sharp (I am exaggerating but you get my point)? Now, it may be that the latter person is way more productive than the former - but the reality is that you rarely love to do something that you are bad at, so usually they can hope that person that codes a lot for his personal enjoyment is reasonable good at it. It's not always true, but it's a good heuristic and a big part of hiring is about finding good heuristics that would predict your future usefulness for the company. So yes, I think it is inevitable that you will be getting lower marks on whatever scale they are using in Google or likes than somebody with a lot of hobby projects. That doesn't mean you won't be hired at the end - but it must reduce the chances compared to other candidates.

But this is the time you need to ask yourself if you really want to play by these rules? There are a lot of companies that are positively starved for good developers, and many of them are ready to offer you very good conditions and be completely content to get you only for 40 hours a week - provided that you are a good developer. They don't have Google's glamour and fame, so they can't be as picky with talent. Sure, you are taking a bet - some of them might be next Twitter or Linkedin, some of them would forever be a footnote on the dusty pages of history, some of them would be neither - just old boring regular business. But a lot of them desperately need good people to work for them, and it's sellers market there.

So the decision is - which rules are more fit to your lifestyle?


答:
I'm trying to get into TDD myself. I think as long as you follow routes you allready know (the day to day work) it's rather simple. But I just can't wrap my head around tests for the UI parts or when you have to come up with a new solution to some problem you have not encountered before. Or using a framework you don't know.

So I guess you have to do some kind of LearningTests, seperate proof-of-concepts and rewriting it afterwards, etc. or am I wrong?

And (I know it's an old one but I just seen no good answer yet): how do you code algorithms using TDD (when the results might be to complex to really "Assert" on with ease)?

I can really see the positive sides of TDD and im on the boat but it's very hard for beginners when the code you write takes you allmost double the time (and you've got peers that don't see the pros at all and mock you with RAD)

Title: Who does test-driven development?

Body:
I've been working in the enterprise space for the past 4½ years and have noticed that generally speaking, enterprises are not conducive environments for the test-first style of development. Projects are usually fixed-cost, fixed-timeline and waterfall style. Any unit testing, if done at all, usually comes after development in the QA phase and done by another team.

Prior to working for an enterprise, I consulted for many small to medium sized companies, and none of them were willing to pay for a test-first style of development project. They usually wanted development started immediately, or after a short design stint: i.e., something more akin to Agile, though some clients wanted everything mapped out similar to waterfall.

With what types of shops, companies, and clients does test-driven development work
best? What types of projects tend to be conducive to TDD?


问:
how to advance into higher level functional programming
first my apologies if this isn't the right place for this question but I think that th.CS is not the right place for sure neither is StackOverflow.

I really like functional programming and I think I'm a medium level guy in F# and understand the basic concepts of Haskell rather well - and yes I think I know/fell like I know what a Monad is.

Now I really want to take this to the next level and want to understand what all the talk about functors and stuff is all about.
I just don't know where to start.

Read something about Lambda Calculus? Category Theory? Both I know a bit and I don't think that this is the right/quickest way to enlightment.

Maybe you can give me some hints on that.

PS: I'm not afraid of math and I don't get frustrated on excercises easily but I would prefere some intros/text where you can put some code down because this way you get the feedback at once.

答:
Read the Lambda Papers,

Lurk around Lamba the Ultimate,

Pick up a functional language (purely functional or not), like:

Haskell,

Scheme,

OCaml.

F# is good start, you should keep going with Haskell, and definitely read up on lambda calculus and read the Lambda Papers mentioned above.

Using an hybrid language (Clojure, Scala, OCaml, ...) might help to ease it onto if you come from an OO background, but I think that if you really want to understand the functional progamming mindset, you should rather go for the real thing (Haskell, Lisp, ...).


问:
Is there a standard pseudocode for parallel algorithms?
The common styles of pseudocode are largely intelligible, and it is more or less clear how to write pseudocode for sequential programs.

But if parallelism is not hidden behind a full library and is regarded as a regular part of programming, then it should be treated as the same way in regards to pseudo-code.

Is there a consistent and widely used style of pseudocode for parallel algorithms? Are there good, practical examples of this?

答: It depends on what level you need to reason about the code. For low-level code you may need to describe shared resources, readers, writers, and how they cooperate to prevent starvation (see the readers-writers problems). On a higher level it may be sufficient to describe system states, tasks, and how workers are synchronized.</p>


答:
I recommend you read those three books if time is a luxury you have:

Pearls of Functional Algorithm Design

Purely Functional Data Structures

Type Theory and Functional Programming

Those three books will enrich your understanding of Functional Programming in general.

Title: how to advance into higher level functional programming

Body:
first my apologies if this isn't the right place for this question but I think that th.CS is not the right place for sure neither is StackOverflow.

I really like functional programming and I think I'm a medium level guy in F# and understand the basic concepts of Haskell rather well - and yes I think I know/fell like I know what a Monad is.

Now I really want to take this to the next level and want to understand what all the talk about functors and stuff is all about.
I just don't know where to start.

Read something about Lambda Calculus? Category Theory? Both I know a bit and I don't think that this is the right/quickest way to enlightment.

Maybe you can give me some hints on that.

PS: I'm not afraid of math and I don't get frustrated on excercises easily but I would prefere some intros/text where you can put some code down because this way you get the feedback at once.


问:
Do good project managers need a programming background?
Sometimes I can't stand it when project managers ask me to estimate time to complete for various tasks.  An estimate is a guess, and guesses can be wrong.  Generally, bad requirements and documentation will lead to bad guesses.

So I often wonder if the project managers were ever in my shoes trying to guess at how long task X and Y will take, and how difficult it is to assign a number to it based on what little is known and collected from the client.

My question then is: Do good project managers need to have a programming background?

Or maybe the question should be, do good project managers need to have been a good programmer before?  Is there any correlation?

答:
In my experience, management is about effective communication and decision-making. With that in mind, it makes sense that someone who understands the crafts (at least the core concepts and terminology) utilized by the people they manage, is better suited to be a manager than someone who has less of an understanding, but there is definitely no correlation. I've seen managers with programming experience succeed and fail, just as often as managers without programming experience.

Either extreme is bad, in my opinion; People with too little programming experience can blindly trust their programmers (Shepard following the sheep); People with too much experience can continually question their team's efforts (micro managing).

Personally, I think someone who has a good grasp of the core programming concepts, but realizes that they aren't a "hot shot," is the ideal kind of manager.


问:
Getting management to hire a new developer?
I've been working here for about 3 months now, and this company has been around since 2003. I've the only full time developer, and I don't know what happened to the part time guy who was doing work before I got hired, I haven't seen him since a month after I started, aside from a couple emails.

Essentially, that makes me the only developer, trying to balance adding new features to our main product (subscription website service), developing an Android-based version of the product, handling client issues, working on migrating clients from old version to new version (and the headaches that will result from this), fixing issues with a signup service for out ISP side of things, and helping with tech support for the ISP side. Not to mention the massive overhaul I'm fairly certain the database needs, that will mean at least a few months' work.

My bosses hired an intern to help me with the Android version, but the one boss randomly went to England until December, taking the intern with him.

Any tips I can use on convincing management to hire somebody else? Even if just a second full time support guy to help with the phone calls?

答: As Fishtoaster mentioned, you have to build a case for it. I'd go a little further in saying you need to build a FINANCIAL case for it. If you had another programmer the following features could be delieverd this much sooner resulting in quicker product launch. Another programmer would reduce the backlog of support request raising customer satisfaction resulting in new upgrades - etc.</p>


答:
I'll suggest the following before you consider switching to another language.

Use LTTng for recording system events like page faults, context switches, and system call waits.

Convert whereever it's taking too much time to use the C library, and use any design pattern you like (multi-threading, signal event based, call back async, or Unix traditional select) which is good for I/O in there.

I would not use threading in Python once the application has priority in performance. I would take the above option, which can solve many problems like software reuse, connectivity with Django, performance, ease of development, etc.