答:
because git is some sort of becoming the "standard" and there are a lot of "pro git" people among the web users we tend to using git.

This is poor foundation for making choice in SCM-world. Git is not "standard", but rather "fashion" with a lot of noise around it. "Pro Git" is more fanboyism, than than sensible and reasoned decision in a lot of cases

work togheter on the same project at the same time without overwriting our codes and simply gives us the advantage of version-control

Branches in any SCM will allow you to get it, even with Git (keyword: "Git-flow")

Title: Manage version control with a central development server (LAMP)

Body:
I'm working in a small team with up to 5 (web)developers. Because our team is growing frequently and we ran into problems with multiple people working on the same code we decided to set up a VCS.

Current Situation

Currently we are working with a central development server (LAMP). So every developer works on the same code base and if the code is tested and ready for our live server we just copy it via ftp. I know this is some sort of an anno 1600 workflow but yeah - it is what it is and also the reason for this question.

On the development server our directory structure looks like this:

```<code>/var/www
      /Project1
      /Project2
      /Project3
      ...
</code>```

Additionally there are some small non-webapplications - Android/iPhone/Windows 8 etc. apps and some C# tools which also should be included in the VCS.

Goal and problems

Our goal is to get a clean setup for a VCS, which works together with an issue tracking software, enables us to work togheter on the same project at the same time without overwriting our codes and simply gives us the advantage of version-control.

I think the first question for us is, which technology should we use. Some of us have allready experienced subversion. But because git is some sort of becoming the "standard" and there are a lot of "pro git" arguments among the web users we tend to using git.

There starts our uncertainty. For using git - a decentralised VCS - it looks like we have to start using seperate development servers on each developer's computer. The problems with that are:

Some times we work on different computers, so when we forget to push our code we have a problem.

We would have to work with virtual machines because the dev servers should be the same as our live server (This would simply not be enforceable in our environment, believe me it's not possible).

The development server usually also served as a "tryout" or "presentation" server where non-developers had a look at what's going on.

Is there another possible setup with git so that we are able to benefit from the system while still using a single(!) development server? Maybe with different directories for each developer. Or can we still work on the same code base with maybe locking the files we are working on and then commit them to a repository. It's maybe important to say that while it became a factor for us it is still uncommon that multiple developers work on the same part of an application at the same time.


问:
I can write code... but can't design well. Any suggestions?
I feel that I am good at writing code in bits and pieces, but my designs really suck. The question is, how do I improve my designs - and in turn become a better designer?

I think schools and colleges do a good job of teaching people how to become good at mathematical problem solving, but let's admit the fact that most applications created at school are generally around 1000 - 2000 lines long, which means that it is mostly an academic exercise which doesn't reflect the complexity of real world software - on the order of a few hundred thousand to millions of lines of code.

This is where I believe that even projects like topcoder / project euler also won't be of much help, they might sharpen your mathematical problem solving ability - but you might become an academic programmer; someone who is more interested in the nice, clean stuff, who is utterly un-interested in the day to day mundane and hairy stuff that most application programmers deal with.

So my question is how do I improve my design skills? That is, the ability to design small/medium scale applications that will go into a few thousand of lines of code? How can I learn design skills that will help me build a better html editor kit, or some graphics program like gimp?

答:
It's tough, what we're really talking about is an ability to abstract rather than create better code, but two things will make you better and one thing will make you happier:

"Better"

A) Find the best designer you can and pair program / do a design together.  Ask them to explain what they're thinking as they address the problem, don't settle for"it just feels right" and keep digging.  That process will help the "mentoring" party too

B) Imagine everything as individual actors and conversations between them.  Each of the actors should have a single role / responsibility and groups of them handle different systems.  If that conversation works and each actor feels coherent and cohesive then you're on your way.

And "Happier"

C) If you've tried your best and it still isn't happening then there's nothing wrong with accepting that some people can't do some stuff.  You could write tight, brilliant code, but never be able to design or architect.  So what?  I can't play physical sports for toffee, I'm not good looking and my car driving will never be better than average.  Revel and utilise what you're good at.

ContentLicense: CC BY-SA 3.0


问:
Manage version control with a central development server (LAMP)
I'm working in a small team with up to 5 (web)developers. Because our team is growing frequently and we ran into problems with multiple people working on the same code we decided to set up a VCS.

Current Situation

Currently we are working with a central development server (LAMP). So every developer works on the same code base and if the code is tested and ready for our live server we just copy it via ftp. I know this is some sort of an anno 1600 workflow but yeah - it is what it is and also the reason for this question.

On the development server our directory structure looks like this:

```<code>/var/www
      /Project1
      /Project2
      /Project3
      ...
</code>```

Additionally there are some small non-webapplications - Android/iPhone/Windows 8 etc. apps and some C# tools which also should be included in the VCS.

Goal and problems

Our goal is to get a clean setup for a VCS, which works together with an issue tracking software, enables us to work togheter on the same project at the same time without overwriting our codes and simply gives us the advantage of version-control.

I think the first question for us is, which technology should we use. Some of us have allready experienced subversion. But because git is some sort of becoming the "standard" and there are a lot of "pro git" arguments among the web users we tend to using git.

There starts our uncertainty. For using git - a decentralised VCS - it looks like we have to start using seperate development servers on each developer's computer. The problems with that are:

Some times we work on different computers, so when we forget to push our code we have a problem.

We would have to work with virtual machines because the dev servers should be the same as our live server (This would simply not be enforceable in our environment, believe me it's not possible).

The development server usually also served as a "tryout" or "presentation" server where non-developers had a look at what's going on.

Is there another possible setup with git so that we are able to benefit from the system while still using a single(!) development server? Maybe with different directories for each developer. Or can we still work on the same code base with maybe locking the files we are working on and then commit them to a repository. It's maybe important to say that while it became a factor for us it is still uncommon that multiple developers work on the same part of an application at the same time.

答:
```<code> It's maybe important to say that while it became a factor for us it is still uncommon that multiple developers work on the same part of an application at the same time.
</code>```

If few people work on same machine and same codebase things have to go wrong. Because of overwriting and because if one breaks something others have to wait untill fix to test their stuff.

You need to distinguish few environments. Looks like you need three (very common):

Production - where the project is actually used by end users.

Staging - place where developer merges his work with other developers, tests it and sometimes clients can peek into this area.

Development - where each developer works alone, before merging with others.

Usualy development environment means machine developer works on. If you want to keep it on shared machine you can do that too. Environments and machines is something decoupled. You just need to have a different codebase for each project, each developer and each environment.

So in case of Git and 5 developers in company there would be 5 repositories for development environment. For staging environment (merging code) because it's not a good idea to push to non-bare repositories you would need one repo for merging and another for testing code there and showing client current state. If production server has Git another one there. If not you would just transfer files to production via FTP from staging environment. So few repositories for each project.

And the workflow is like so:
You work on your development environment, when you added a feature or fixed a bug you push your changes to staging environment bare repo and pull this out on the non bare (a Git hook could do that at once), and when the project is ready to release a new version you push (or transfer via FTP) it to the production.

Some times we work on different computers, so when we forget to push
  our code we have a problem.

Well if you have to do development environment on the very same shared server you can synchronize your files from local machine to the server via something like rsync, or map a network drive or whatever but this always will be a little slower then working on local machine. Working localy however has the problem you mention. Even without VCS. Like when using FTP - if you change something on your machine and forget to transfer it - same thing happens. But you wouldn't want to merge your dissasembled code to the staging environment for sure. It has to stay in development until it reaches a resonable state.

```<code>We would have to work with virtual machines because the dev servers should be the same as our live server (This would simply not be enforceable in our environment, believe me it's not possible).
</code>```

GUI-less wev servers don't use much resources. I think you guys could work on the code and test it everyone on his own machine. So development environment localy, and only staging on the shared server. You could check out something like Vagrant to help managing virtual machine images. So you would make sure that every project gets right settings. This would ensure that you guys test in same way, be safer (now if shared server would break nobody can continue work, and you need to have a good backup there) and faster (no transfer needed until merging). Git is better ten SVN in this matter because it keeps full repo on each machine with history and everything so if one pc breaks there are still good copies somewhere else.

Also things like Gitosis can help you manage the staging environment repo settings, and Redmine or other project managing software often has ways to integrate with VCS.

Body:
 It's maybe important to say that while it became a factor for us it is still uncommon that multiple developers work on the same part of an application at the same time.

If few people work on same machine and same codebase things have to go wrong. Because of overwriting and because if one breaks something others have to wait untill fix to test their stuff.

You need to distinguish few environments. Looks like you need three (very common):

Production - where the project is actually used by end users.

Staging - place where developer merges his work with other developers, tests it and sometimes clients can peek into this area.

Development - where each developer works alone, before merging with others.

Usualy development environment means machine developer works on. If you want to keep it on shared machine you can do that too. Environments and machines is something decoupled. You just need to have a different codebase for each project, each developer and each environment.

So in case of Git and 5 developers in company there would be 5 repositories for development environment. For staging environment (merging code) because it's not a good idea to push to non-bare repositories you would need one repo for merging and another for testing code there and showing client current state. If production server has Git another one there. If not you would just transfer files to production via FTP from staging environment. So few repositories for each project.

And the workflow is like so:
You work on your development environment, when you added a feature or fixed a bug you push your changes to staging environment bare repo and pull this out on the non bare (a Git hook could do that at once), and when the project is ready to release a new version you push (or transfer via FTP) it to the production.

Some times we work on different computers, so when we forget to push
  our code we have a problem.

Well if you have to do development environment on the very same shared server you can synchronize your files from local machine to the server via something like rsync, or map a network drive or whatever but this always will be a little slower then working on local machine. Working localy however has the problem you mention. Even without VCS. Like when using FTP - if you change something on your machine and forget to transfer it - same thing happens. But you wouldn't want to merge your dissasembled code to the staging environment for sure. It has to stay in development until it reaches a resonable state.

We would have to work with virtual machines because the dev servers should be the same as our live server (This would simply not be enforceable in our environment, believe me it's not possible).

GUI-less wev servers don't use much resources. I think you guys could work on the code and test it everyone on his own machine. So development environment localy, and only staging on the shared server. You could check out something like Vagrant to help managing virtual machine images. So you would make sure that every project gets right settings. This would ensure that you guys test in same way, be safer (now if shared server would break nobody can continue work, and you need to have a good backup there) and faster (no transfer needed until merging). Git is better ten SVN in this matter because it keeps full repo on each machine with history and everything so if one pc breaks there are still good copies somewhere else.

Also things like Gitosis can help you manage the staging environment repo settings, and Redmine or other project managing software often has ways to integrate with VCS.


问:
RESTful URI's, Creating a new noun/uri or add a queryparameter
I have a Person class with a one to many relationship to Employment. A Person in the CRM can have multiple Employments (historic data).

My current REST URI for this class looks like this:

```<code>http://domain/persons GET [collection]
http://domain/persons/{id} GET [resource]
http://domain/persons/{id}/employments GET [collection]
</code>```

To get the CURRENT active employment (so the employment where the endDate is NULL) should I create a URI or should this be done using a QueryParam?

Basically should I do this:

```<code>http://domain/persons/{id}/employments/current GET [resource]
</code>```

or

```<code>http://domain/persons/{id}/employments?current GET [collection with query param filter]
</code>```

I'm thinking the second one (with the queryparam) because as far as I understand REST a resource should be identified with a non changable URI. Since the current employment position can change it should be done with a QueryParam.

What are your suggestions?

答: The "current" employment can be identified with a "non-changeable" url: it's always at employments/current. It is allowed to point to a different employment object just as the download link for the "current" version of a library can point at different actual files as the library is updated while retaining the same URI. I would use /employments/current. /employments?current isn't correct as the resource has changed (from a collection of employments to a single employment) but the URI path has not.


问:
What is "Semantics visibility"?
I'm reading 97 Things Every Programmer Should Know, now I'm positioned in "Apply Functional Programming Principles", and there is a paragraph that says:

...A leading cause of defects in imperative code is attributable to
  mutable variables. Everyone reading this will have investigated why
  some value is not as expected in a particular situation. Visibility
  semantics can help to mitigate these insidious defects, or at least to
  drastically narrow down their location, but their true culprit may in
  fact be the providence of designs that employ inordinate mutability...

What is the objective of Semantics Visibility in this context? How to apply this approach to solve mutable variables' insidious defects?

答: Not 100% sure, but it seems they talk about how a language handles the logical scope of variables. The worst case would be a language with only global variables. Mutability allows you to "accidently" change a variable. So if you can limit the variables visibility (or scope) to the level of a function, a class, namespace, module or similar smaller units the risk to change something that's used in different places too would be far lower.</p>

Title: What is "Semantics visibility"?


问:
Assuring Quality and Bug Fix speed in Open Source Python Project
I'm maintaining an open source framework (in Python on *nix platforms if that matters) for the first time in my life. It is pretty much pre-alpha, not much more then a scientific proof of concept, yet. But it is also already used in production by another department because it is the only framework globally that approaches their needs. Now there are two anti polar goals: quality and development speed.

Of course I want quality in form of documentation, unit tests, code reviews and some kind of "beta" usage, before I am confident to let a change be used in production. But the development team has dead lines and when they find a bug, which happens rather often in this prototype, then they need the bug fix to be in production very fast. I have currently no working solutions and our ideas diverge.

I think this project can't be the only one with that problem. How do other projects solve this?

I'm going to post my idea and the dev teams idea as answers for further discussions, both ways are not a solution though, because my idea basically only focusses on quality and their solution only focusses on speed.

答:
My solution:

There are two branches:

master contains only stable, tested builds

dev contains the most bleeding edge development state

f-<xyz> feature branches, which contain features and bug fixes that are currently under development and not acknowledged for the dev branch yet; that's possible because most people working on the project have writing permission to the repository, which is company rule

Each feature request or bug should go through the following process:

Write an Issue on the project page, which contains the problem description, the suggested solution and further notes like the feature branch name.

Check in and Push your changes to the feature branch and reference the Issue.

Find 2 people to review and acknowledge your change (looking for mistakes, unclear text, that the documentation is up to date and that there are unit tests for the change)

After getting 2 ACKs find a maintainer to merge your change into the dev branch.

This process depends on at least 2 other people (if one of them is a maintainer) and even afterwards it is not released yet, it's just in the dev branch.

For a release to happen:
 1. Someone needs to prepare the release
  * find a meaningful release candidate commit
  * check again for stability, quality and documentation
  * mark the release candidate
  * get the okay at a weekly project meeting through votes

All users should use the last release or must be able to adapt their code to structural changes and unexpected bugs themselves, if they are using the dev branch directly.

As said in the question this mostly concerns about quality and longevity of the project, which is my main goal. The development team in a production environment of course needs speed as well.


答:
Dev Team's solution:

Just one branch is needed: master. Everybody can push to that branch and reviews aren't necessary. Every body is responsible that this branch's HEAD is working correctly and is not breaking the projects that rely on it. Solutions should be added when needed and quality assurance as documentation and unit tests can be added later on.

I don't think in this approach there is anything that will keep quality alive. The production team will never have the time to write documentation and unit tests and nobody will review the code.

Title: Assuring Quality and Bug Fix speed in Open Source Python Project

Body:
I'm maintaining an open source framework (in Python on *nix platforms if that matters) for the first time in my life. It is pretty much pre-alpha, not much more then a scientific proof of concept, yet. But it is also already used in production by another department because it is the only framework globally that approaches their needs. Now there are two anti polar goals: quality and development speed.

Of course I want quality in form of documentation, unit tests, code reviews and some kind of "beta" usage, before I am confident to let a change be used in production. But the development team has dead lines and when they find a bug, which happens rather often in this prototype, then they need the bug fix to be in production very fast. I have currently no working solutions and our ideas diverge.

I think this project can't be the only one with that problem. How do other projects solve this?

I'm going to post my idea and the dev teams idea as answers for further discussions, both ways are not a solution though, because my idea basically only focusses on quality and their solution only focusses on speed.


问:
When to separate a project in multiple subprojects
I'd like to know if it makes sense to divide the project I'm working on in two repositories instead of one.

From what I can say:

Frontend will be written in html+js

Backend in .net

The backend doesn't depend on frontend and the frontend doesn't depend on the backend

The frontend will use a restful api implemented in the backend.

The frontend could be hosted on any static http server.

As of now, the repository has this structure:

root:

frontend/*

backend/*

I think it's a mistake to keep both project in the same repository. Since both project do not have dependencies between each others, they should belong in individual repositories and if needed a parent repository that has submodules.

I've been told that it's pointless and that we won't get any benefit from doing that.

Here are some of my arguments:

We have two modules that don't depend between each others.

Having source history of both projects in the long term may complicate things (try searching in the history for something in the frontend while you have half of the commits that are completely unrelated to the bug you're looking for)

Conflict and merging (This shouldn't happen but having someone pushing to the backend will force other developer to pull backend changes to push frontend changes.)

One developer might work only on the backend but will always have to pull the frontend or the other way around.

In the long run, when it will be time to deploy. In some way, the frontend could be deployed to multiple static server while having one backend server. In every case, people will be forced to either clone the whole backend with it or to make custom script to push to all servers the frontend only or to remove the backend. Easier to just  push/pull only the frontend or backend than both if only one is needed.

Counter argument (One person might work on both projects), Create a third repo with submodule and develop with it. History is kept seperated in individual modules and you can always create tags where version of backend/frontend do really work together in sync. Having both frontend/backend together in one repo doesn't mean that they will work together. It's just merging both history into one big repo.

Having frontend/backend as submodules will make things easier if you want to add a freelancer to the project. In some case, you don't really want to give full access to the codebase. Having one big module will make things harder if you want to restrict what the "outsiders" can see/edit.

Bug introduction and fixing bug, I inserted a new bug in the frontend. Then someone fix a bug in the backend. With one repository, rolling back before the new bug will also rollback the backend which could make it difficult to fix. I'd have to clone the backend in a different folder to have the backend working while fixing the bug in the frontend... then trying to remerge things up... Having two repository will be painless because moving the HEAD of one repo won't change the other. And testing against different version of backend will be painless.

Can someone give me more arguments to convince them or at least tell me why it is pointless (more complicated) to divide the project in two submodules. The project is new and the codebase is a couple of days old so it's not too soon to fix.

答:
At my company, we use a separate SVN repository for every component of the system.  I can tell you that it gets extremely frustrating.  Our build process has so many layers of abstraction.

We do this with Java, so we have a heavy build process with javac compilation, JibX binding compilation, XML validation, etc.

For your site, it may not be a big deal if you don't really "build it" (such as vanilla PHP).

Downsides to splitting a product into multiple repositories

Build management - I can't just checkout code, run a self-contained build script and have a runnable / installable / deployable product.  I need an external build system that goes out to multiple repos, runs multiple inner build scripts, then assembles the artifacts.

Change tracking - Seeing who changed what, when, and why.  If a bug fix in the frontend requires a backend change, there are now 2 divergent paths for me to refer back to later.

Administration - do you really want to double the number of user accounts, password policies, etc. that need to be managed?

Merging - New features are likely to change a lot of code.  By splitting your project into multiple repositories, you are multiplying the number of merges needed.

Branch creation - Same deal with branching, to create a branch, you now have to create a branch in each repository.

Tagging - after a successful test of your code, you want to tag a version for release.  Now you have multiple tags to create, one in each repository.

Hard to find something - Maybe frontend/backend is straightforward, but it becomes a slippery slope.  If you split into enough modules, developers may have to investigate where some piece of code lives in source control.

My case is a bit extreme as our product is split across 14 different repos and each repo is then divided into 4-8 modules.  If I remember, we have somewhere around 80 or some "packages" which all need to be checked out individually and then assembled.

Your case with just backend/frontend may be less complicated, but I still advise against it.

Extreme examples can be compelling arguments for or against pretty much anything :)

Criteria I would use to decide

I would consider splitting a product into multiple source code repositories after considering the following factors:

Build - Do the results of building each component merge together to form a product?  Like combining .class files from a bunch of components into a series of .jar or .war files.

Deployment - Do you end up with components that get deployed together as one unit or different units that go to different servers?  For example, database scripts go to your DB server, while javascript goes to your web server.

Co-change - Do they tend to change frequently or together?  In your case, they may change separately, but still frequently.

Frequency of branching/merging - if everybody checks into trunk and branches are rare, you may be able to get away with it.  If you frequently branch and merge, this may turn into a nightmare.

Agility - if you need to develop, test, release and deploy a change on a moment's notice (likely with SaaS), can you do it without spending precious time juggling branches and repos?

Your arguments

I also don't agree with most of your arguments for this splitting.  I won't dispute them all because this long answer will get even longer, but a few that stand out:

We have two modules that don't depend between each others.

Non-sense.  If you take your backend away, will your frontend work?  That's what I thought.

Having source history of both projects in the long term may complicate
  things (try searching in the history for something in the frontend
  while you have half of the commits that are completely unrelated to
  the bug you're looking for)

If your project root is broken into frontend/ and backend/, then you can look at the history of those hierarchies independently.

Conflict and merging (This shouldn't happen but having someone pushing
  to the backend will force other developer to pull backend changes to
  push frontend changes.) One developer might work only on the backend
  but will always have to pull the backend or the other way around.

Splitting your project into different repos doesn't solve this.  A frontend conflict and a backend conflict still leaves you with 2 conflicts, whether it's 1 repository times 2 conflicts or 2 repositories times 1 conflict.  Somebody still needs to resolve them.

If the concern is that 2 repos means a frontend dev can merge frontend code while a backend dev merges backend code, you can still do that with a single repository using SVN.  SVN can merge at any level.  Maybe that is a git or mercurial limitation (you tagged both, so not sure what SCM you use)?

On the other hand

With all this said, I have seen cases where splitting a project into multiple modules or repositories works.  I even advocated for it once for a particular project where we integrated Solr into our product.  Solr of course runs on separate servers, only changes when a changeset is related to search (our product does much more than search), has a separate build process and there are no code artifacts or build artifacts shared.

ContentLicense: CC BY-SA 4.0


问:
Should I retrieve one large set of data to filter or smaller individual sets of data?
I have a company intranet that utilizes a similar set of data across many different pages and many different methods.

Speaking in generalities, is it closer to "best practice" to retrieve a large dataset and filter it for each specific page or method or retrieve many smaller specific sets of data to use?  I cannot really test both approaches, as I would like to pick a design approach from the beginning and use it throughout.  This is for a redesign.  Thanks.

答:
I think this requires more context to answer with a yes or no, you'll probably get more/better answers if you provide more detail.

With that said, I tend to go down the route of retrieving only the data that's required, when its required. Since you're talking about an intranet it seems reasonable to assume that there will be no problems with fetching data (as you're db server will be on the same network as your intranet server), which means there's not much reason to fetch it all at once.

Keeping unneeded data out of the way saves resources (memory on the system requesting it, disk IO on the system serving it) and if you pull all the data at the start of a session, it may well be out of date in five minutes time when The user is still browsing.

In most cases, pulling it all at once is the lazy way. You would probably know if there was some reason which made it more efficient.


问:
Why validate XML?
I am looking for an explanation on just why XML needs to be validated. I have been testing DTD's and Schema's for the past month and recently tried to find out why I am doing this. I know it makes the XML better and more Semantic but what are the overall benefits of validating XML?</p>

答: Well, you don't need to. Validating XML against a schema (DTD is outdated garbage) is equivalent to using a statically typed language. It ensures a higher level of correctness at the expense of more upfront labor. I generally find that validating XML is very useful when you're dealing with remote services. It's self-documenting. If you're just transferring data from a client to server where you control both sides, then it's less useful.</p>

Title: Why validate XML?


问:
Where did the html/css 'float' concept come from?
Pretty straightforward, was wondering what the inspiration or precedent for the CSS 'float' concept. I'm not familiar with any other graphic APIs or other layout DSLs that use this concept, but it's such a huge part of HTML layouts. I'm curious if a previous language had a similar concept, or if 'floats' originated from the land of HTML/CSS. Thanks.</p>

答: LaTeX has them. Tables and figures are typically floats. They are automatically placed in a good place rather than just wherever in your document you put them. The idea there presumably comes from how figures in books and articles are typically placed where there is room as opposed to just in the text.</p>


答: You might find that at the beginning, if there are a lot of bugs needing fixes, then something like the dev team's preference is more appropriate.  Once the code is a little more stable, then you can shift more to what you have in mind.</p>


答:
Speaking in terms of using XML for web services: A well-formed schema lets the consumer of the web service know what the server expects and then what the server will give you back. It really helps knowing what the inputs and outputs for the web service are.

Then there are the badly-formed web services: they tell you what they expect from you, but then return some nonsense. I am currently dealing with such a service: they tell you how to form the input, but then they return the response as a string. A string containing complex XML. Said complex XML has no schema to validate against, so I have no idea what I am guaranteed to get back from the server, what is optional, or what types are possibly returned. It is effectively useless, as it tells me nothing useful.

Title: Why validate XML?

Body: I am looking for an explanation on just why XML needs to be validated. I have been testing DTD's and Schema's for the past month and recently tried to find out why I am doing this. I know it makes the XML better and more Semantic but what are the overall benefits of validating XML?</p>


问:
Where can I get free public raw data?
In web development, a mashup is a web page or application that uses and combines data, presentation or functionality from two or more sources to create new services.

There are many sources for getting data (raw information). Governments are good examples.

However many of those cost and have complicated licensing.
For instance SMHI in Sweden sells temperature data meanwhile our neighbor country Norway gives it away for free.

One would say what kind of application are you building and what data do you need?  I would say give me some interesting data and I will do a good mashup of it. I could just plot it on Google maps for instance.

So the question is: Where can I get free data?

Note: I'm not interested in data mining using a webcrawler!

Please share your sources with me.

答:
The Open Data Protocol site lists a number of live oData services on its Producers page.

edit> StackExchange has an oData service listed there.

ContentLicense: CC BY-SA 2.5

Title: Where can I get free public raw data?


问:
finds all possible permutations of a certain numbers of distinct digits
I am stuck on the following problem. I want to find an algorithm that finds all possible permutations of a certain numbers of distinct digits and then put them in a large matrix for instance. For instance for the digits 0, 1 and 2 I want to get the matrix:

2 1 2 0 1 0

1 2 0 2 0 1

0 0 1 1 2 2

Here they are in order but this is not required per se. Does anyone know a nice algorithm to achieve this? Thanks in advance.

答:
Building on MichaelT's comment, here's an example solution in Python:

from itertools import permutations
for p in permutations(['0', '1', '2'], 3):
    print p
```

<p>Output looks like this:</p>

```('0', '1', '2')
('0', '2', '1')
('1', '0', '2')
('1', '2', '0')
('2', '0', '1')
('2', '1', '0')
```

You'll have to arrange the output to match your desired format.

Body:
Building on MichaelT's comment, here's an example solution in Python:

from itertools import permutations
for p in permutations(['0', '1', '2'], 3):
    print p

Output looks like this:

('0', '1', '2')
('0', '2', '1')
('1', '0', '2')
('1', '2', '0')
('2', '0', '1')
('2', '1', '0')

You'll have to arrange the output to match your desired format.


问:
Looking for companies with strong R&D departments, specifically for A.I., but I can't seem to find many. Are there only a handful?
I plan to get a PhD in CS, concentrating in some subfield of artificial intelligence, not sure which one at the moment.  I'm looking into R&D departments in industry, but there don't seem to be too many.

For example, I have literally found hundreds of labs for the basic sciences and other engineering fields, but for CS I've found: Google, IBM Research, Microsoft Research, Yahoo Research, AT&T Research and GE Research.  Are there not tons of companies like there are in other disciplines?

答:
It depends on what you mean by R&D. Your research background will give you a breadth of knowledge and demonstrated ability to pursue problems deeply. Most open-minded companies will give you a chance to innovate to tangibly improve their products, their product offering, and increase profits. This makes sense, but the work has to be practical.

Few companies will give you a check for a year to theorize and try random off-the-wall things, even if that may lead to fantastic ideas in the end. That kind of risk is too high for most small to medium sized businesses.

The companies you listed are large and have deep pockets -- so much that it is possible for them to make long-term investments and fund more basic CS research with hopes that it leads to huge payoffs in the end. These companies have research groups that mimic the setting at universities.


答:
You are correct in believing that such companies are rare.  Pure R&D departments have a significant investment of capital and time before they get a payoff, and as such, are not viable for any but the largest and already-successful companies.

Some of the ones that do offer such a benefit, such as Google or Valve, eschew a "department" in favor of "do whatever you want time" in which you can innovate as you please during your work time (at a rate of 20% and 100% of your time, respectively).  Of course, they do this only because they believe, as it has statistically proved, to be profitable for them in the long run.  It all comes down to money, in the end.

Title: Looking for companies with strong R&D departments, specifically for A.I., but I can't seem to find many. Are there only a handful?

Body:
I plan to get a PhD in CS, concentrating in some subfield of artificial intelligence, not sure which one at the moment.  I'm looking into R&D departments in industry, but there don't seem to be too many.

For example, I have literally found hundreds of labs for the basic sciences and other engineering fields, but for CS I've found: Google, IBM Research, Microsoft Research, Yahoo Research, AT&T Research and GE Research.  Are there not tons of companies like there are in other disciplines?


答:
This is referring to the visibility and scope of variables.  If you are familiar with Java, you may recognize the usage of terms like Public, Private, Protected, and so forth.  Proper usage of these terms is part of encapsulation, and is core to good programming practices of Object Oriented Programming.

Making variables private means that they cannot be accessed from code outside of the code in question.  The benefit is that you can be assured that it is not changed or otherwise disturbed in any way that you are not familiar with.  In other words, you would only have to look at the code within that single file in order to see what could be done to that variable, instead of worrying about a 1,000,000+ line codebase and knowing that it could be changed anywhere.  The latter case is a very real problem for languages such a COBOL, where all variables are global, and any code can effect any variable.  Nightmares are generated this way.

Title: What is "Semantics visibility"?

Body:
I'm reading 97 Things Every Programmer Should Know, now I'm positioned in "Apply Functional Programming Principles", and there is a paragraph that says:

...A leading cause of defects in imperative code is attributable to
  mutable variables. Everyone reading this will have investigated why
  some value is not as expected in a particular situation. Visibility
  semantics can help to mitigate these insidious defects, or at least to
  drastically narrow down their location, but their true culprit may in
  fact be the providence of designs that employ inordinate mutability...

What is the objective of Semantics Visibility in this context? How to apply this approach to solve mutable variables' insidious defects?


问:
Coping with test code becoming too complex
Our test automation team writes tons of automation tests for our WEB application, but it seems to me that their code is becoming way too complex and warrants that someone will test it as well.

Is there a best practice for writing these automation tests?
I'll appreciate any answer with concrete inputs from someone who experienced this problem.

If it matters - we are using jSystem for automating our WEB application tests - if there are frameworks which simplify life - I'd like to hear about them.

答:
You should give an example of what "too complex" is to you.

In general, tests should exercise a small part of a system while keeping everything else constant. For example, if your website reads documents from a database, you can test the web site against a prepared database of known documents. Having said that, testing features on websites is hard because websites are large and complex systems, while your features may be fairly simple.

If your tests require a fully running system, then they are truly "integration tests", not "unit tests". To make sure that the page "/user/RonK/profile" shows "ronkitay.wordpress.com", you will implicitly also test that this page returns an HTTP status code 200, and that the response body is text/html. If your tests are running inside a browser, then you'll also be testing that browser's handling of the HTML DOM and any client-side scripts. Finally, you'll test that "ronkitay.wordpress.com" is on the page. If it isn't, then any of the previously implicit conditions may have failed -- the component that renders RonK's website URL may be coded correctly, but simply hasn't executed. A tool like Selenium will let you control an instance of a Firefox browser and test expectations/behaviors by inspecting the DOM. This is usually okay, but it doesn't test Safari or Chrome or IE.

One thing that's often overlooked in automated website testing is the page layout -- the DOM may be correctly organized so the things you are testing for will exist, but they may be laid out very poorly (or not at all, if the CSS file isn't referenced properly.) This is overlooked because it's hard to test layout conditions: how do you express this in code: "the Community Bulletin page element must not spill over to the Related element, the text must wrap nicely, and links shouldn't be underlined"?

What I've seen is that parts of the website content generation can be isolated and tested without the web server. If you use an MVC framework (like SpringMVC), then models and views and controllers can be tested independently as pure Java code. (The jUnit framework will let you do generic tests pretty easily.)

By testing the M, V and C parts of an MVC-structured website, you focus the testing on the business rules in your code, which is usually where the bugs will be. For example, you can build (or mock) a fake user object that is given to the UserProfile model. You will expect that this UserProfile model's websiteURL field will be correctly set. You can then couple the UserProfile model with its view (the SpringMVC framework does this normally, but you can do this yourself as well) and confirm that the result is correct HTML for that portion of the page.

As for your comment about testing tests ... if there is complete coverage of your code, then the tests have exercised everything there is and you are probably okay. (They may be difficult to extend, or modify, but those are design choices.) If that isn't enough, you could have two people independently write tests for the same thing and look for consensus. ;-)


问:
What is the real point of an “oracle” in the SW testing?
I know the definition of an testing oracle but to me, it seems to be overrated. I have a specification, why should I bother with thinking about oracles? I just compare what the spec wants and what the program does. Or I have the use cases, whatever. When I read about oracles in books, it looks as if it a science on its own. Is there anything more behind this concept? Maybe I just did not understand its point.

An oracle consists of a generator and a comparator. What is the meaning of the comparator if you just check whether an actual output equals to an expected output? How else could it be checked?

答:
I think its a nice way of saying Gospel truth. For some programs the exact output is difficult to get for different data sets (algorithms, computations, accountancy, taxes, insurance ...). The tester/ test suite needs the correct answer to compare actual software output. The specs will have the method & some samples but not the answer for current data set & state.

The oracle in this case is parallel software or person or method to get the answer. For you might be your neighbor, for a software suite might be the text file where you have entered the expected value.

Oracle is the one with the answers.

Example :  When you have one data set, say for a tax application, a person earning X, with Y deductions, and other environmental factors like number of children, etc The comparator would take this data and come up with the tax due, then compare it what the program outputs; to see if it got it right. If they do not match, one of them has it wrong. Hopefully the app or the oracle is not a good oracle.


问:
Multiprocess RPC Architecture Design
I am currently working on a project that has client applications communicating with a server process. The client applications could be local to the same machine as the server process, on the same network, or over the internet. I had written a single-threaded asynchronous RPC module for the server process using Boost ASIO while Google Protobuf was used for RPC message serialization. The architecture was a pretty simple client-server model where a client would pack a request into protobuf message and send it to the server which would send back a response. The server process is written in C++ and the client applications were in varying languages. The main use case for this was to present a GUI to allow users to monitor, and interact with, the server process.

In the future, the server process is going to be separated into functionally separate processes instead of one monolithic process. This presents an issue wherein I now have multiple processes for the client applications to connect to. I don't think this would be a problem in the case of the local or lan client applications but, over the internet, this could become unwieldy if the number of processes grow. Furthermore, I may want to enable IPC between server processes by sending each other RPC messages.

I'm not sure what distributed application architecture would best fit this system going forwards. I don't want to reinvent the wheel if it's not necessary so I've taken cursory glances at D-bus and ZeroMQ, but seeing as I don't really have my architecture settled I thought I'd ask you fellow programmers.

Would it be wise to have a single server process handling communication between client applications and server processes or is there a better architecture style I should be considering? If I want to enable IPC between server processes should I have them directly send messages to each other or should I use the single server process mentioned previously? Should I be looking at something like a publish/subscribe model?

答:
I think you should have one main process as the gateway. To put less load on it, you can have other processes that do part of work. To scale the main process you can have it listen to an IP and port but advertise the IP as a URL. Then you can use DNS providers which give more than 1 IP on look up (they change the order so both IPs get traffic)

That is what small and big websites do. They serve web pages and you would have your app listening on its own ports (could be 80 if over HTTP or any other that you have opened in your firewall)

The sub processes can have sync or async methods. In Java we have containers and ejbs that do this, will have to do more research for such systems in c++

Title: Multiprocess RPC Architecture Design


问:
Python, magic and objects that add attributes to its owner
Let me start with a disclaimer: I'm not the best programmer out there. I do however study I.T. and learnt a bit of Java and C.

I'm getting stuck into Python and Django + Mongoengine, I'm not going to explain it into detail as it will deviate from the original question.

While trying to understand what is happening at this bit of Django code I was shocked by some of concepts I seemed to understand.

Mostly I was completely confused by how one object could add accessible attributes to a parent object and how Python defines __set__() and __get__() in such a way as to be so confusing as that.

Could someone tell me how orthodox this design style is and whether I find it bizarre because I'm an amateur programmer.

答:
You are looking at a descriptor. Descriptors are special types of objects that can alter their behaviour when bound to a Python class or instance. They give you a hook into class and instance attribute interactions (setting, getting and deleting).

Python functions and the property decorator are descriptors. Functions have a __get__ method that returns a method wrapper when accessed; this is how Python methods can pass in the self parameter when you call a method on an instance. property objects have __get__, __set__ and __del__ methods to handle attribute interactions and translate those interactions into method calls.

The FileDescriptor class you linked to, translates attribute access into a lookup for the named field on the instance they are bound to, guaranteeing that a FieldFile object is returned. It acts as a proxy for the field, and it is therefor completely natural that they change the named field on the instance they are bound to.

See the Descriptor HOWTO Guide  and the Implementing Descriptors section of the Python Datamodel documentation for more details.


问:
algorithm to create all possible column vectors of size N containing only ones and zeroes
I am a mathematics student and am just starting to learn about programming. I am stuck on a particular program I am trying to write for fun. I want to find an algorithm to create all possible column vectors of size N containing only ones and zeroes and combine them in one large matrix by putting all the columns after each other. The order of the columns is of no importance. For instance if N=3 I want to get the matrix:

0 1 1 1 1 0 0 0

0 0 1 0 1 1 1 0

0 0 0 1 1 0 1 1

which vertically contains all possible combinations of ones and zeroes of length 3. Does anyone have any idea how to do this? I am not necessarily looking for a very efficient or clever way to do this because the value I will be using for N is 14, so my matrix will be 14 x 2^14 which is 14 x 16384 which is not too big. Also, sorry for the formatting of my question, I am used to the math stackexchange where you can use LaTeX text to make your matrices and equations look nice. Thanks to anyone who can help me along!

答:
I wrote you an quick example of a shell script that would do such a thing in Python. Of course you could make it way faster using a c (or a different programming approach) but that is not my point.

```<code># Size of n
n = 3
# Total possibilities
p = 2**n
# This creates a set of binary numbers
x=0
bnum=[]
bitem=1
while x < n:
  bnum.insert(0,bitem)
  bitem = bitem*2
  x = x +1

# Print the set of binary number to the screen
print 'bin set', bnum
pset = range(p)
print 'total possibilities = %i' %p

# Function to covert to binary numbers
def convbin(num):
  modc = num
  x=0
  binset = []
  while x < n:
    mod = modc%bnum[x]
    if modc == mod :
      binset.append(0)
    elif mod == 0:
      binset.append(1)
    else:
      binset.append(1)
    modc = mod
    x = x+1
  print binset

# Call function
for item in pset:
  convbin(item)
</code>```

Output would be:

```<code>bin set [4, 2, 1]
total possibilities = 8
[0, 0, 0]
[0, 0, 1]
[0, 1, 0]
[0, 1, 1]
[1, 0, 0]
[1, 0, 1]
[1, 1, 0]
[1, 1, 1]
</code>```

Body:
I wrote you an quick example of a shell script that would do such a thing in Python. Of course you could make it way faster using a c (or a different programming approach) but that is not my point.

# Size of n
n = 3
# Total possibilities
p = 2**n
# This creates a set of binary numbers
x=0
bnum=[]
bitem=1
while x < n:
  bnum.insert(0,bitem)
  bitem = bitem*2
  x = x +1

# Print the set of binary number to the screen
print 'bin set', bnum
pset = range(p)
print 'total possibilities = %i' %p

# Function to covert to binary numbers
def convbin(num):
  modc = num
  x=0
  binset = []
  while x < n:
    mod = modc%bnum[x]
    if modc == mod :
      binset.append(0)
    elif mod == 0:
      binset.append(1)
    else:
      binset.append(1)
    modc = mod
    x = x+1
  print binset

# Call function
for item in pset:
  convbin(item)

Output would be:

bin set [4, 2, 1]
total possibilities = 8
[0, 0, 0]
[0, 0, 1]
[0, 1, 0]
[0, 1, 1]
[1, 0, 0]
[1, 0, 1]
[1, 1, 0]
[1, 1, 1]


问:
Should I create a use case diagram for each screen?
In projects with many use cases, it's valid to create different use case diagrams, one for each screen? Or I shouldn't guide myself on the GUI of the software to make this, why?

As a clarification, the software does already exist, and I'm just documenting it for my undergraduation final project.

答:
I think you'll run into trouble as you design an application by thinking there is a strong relationship between the tasks requirements of an application and the number of screens (forms, web pages, etc). Even simple CRUD apps have that one requirement with too much complexity to handle this way.

Take the example of a simple data entry grid with the ability to edit the data inline. Easy enough to have some update button, the controls are editable and there are save and cancel buttons. For whatever reason, this is confusing to the users, so you decide to have a separate pop-out screen of the updated record. I don't see why you would have to go back and change the use case diagram since the user is accomplishing the same task.

What if you start to use an outside data provider (stock pricing site) that will update these data through a service? That requires you to create another use case, but there are no additional screens.

Many developers can get into this trap with data base design as well. You don't automatically create a new data entry form for every table and every form doesn't require a database table.


答:
The two other answers (so far) are good ones, but I think it's interesting to point out the recursive nature of the problem. If you want to generate all the binary values up to 2n-1 for some n, you can start by writing the values for n=1, i.e. just write down the possible digits:

```<code>0 1                                     n=1
</code>```

Next, copy those values, and prepend each of the possible digits to each copy:

```<code>0 0 1 1                                 n=2
0 1 0 1
</code>```

Repeat as necessary:

```<code>0 0 0 0 1 1 1 1                         n=3
0 0 1 1 0 0 1 1
0 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1         n=4
0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1
0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
</code>```

You can see that an easy way to generate all the combinations is to write the first line by printing 2n-1 0's followed by 2n-1 1's, repeated 20 times. The next line gets 2n-2 0's followed by 2n-2 1's, repeated 21 times, and so on.

Body:
I am a mathematics student and am just starting to learn about programming. I am stuck on a particular program I am trying to write for fun. I want to find an algorithm to create all possible column vectors of size N containing only ones and zeroes and combine them in one large matrix by putting all the columns after each other. The order of the columns is of no importance. For instance if N=3 I want to get the matrix:

0 1 1 1 1 0 0 0

0 0 1 0 1 1 1 0

0 0 0 1 1 0 1 1

which vertically contains all possible combinations of ones and zeroes of length 3. Does anyone have any idea how to do this? I am not necessarily looking for a very efficient or clever way to do this because the value I will be using for N is 14, so my matrix will be 14 x 2^14 which is 14 x 16384 which is not too big. Also, sorry for the formatting of my question, I am used to the math stackexchange where you can use LaTeX text to make your matrices and equations look nice. Thanks to anyone who can help me along!

Body:
The two other answers (so far) are good ones, but I think it's interesting to point out the recursive nature of the problem. If you want to generate all the binary values up to 2n-1 for some n, you can start by writing the values for n=1, i.e. just write down the possible digits:

0 1                                     n=1

Next, copy those values, and prepend each of the possible digits to each copy:

0 0 1 1                                 n=2
0 1 0 1

Repeat as necessary:

0 0 0 0 1 1 1 1                         n=3
0 0 1 1 0 0 1 1
0 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1         n=4
0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1
0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

You can see that an easy way to generate all the combinations is to write the first line by printing 2n-1 0's followed by 2n-1 1's, repeated 20 times. The next line gets 2n-2 0's followed by 2n-2 1's, repeated 21 times, and so on.


问:
How do you store "fuzzy dates" into a database?
This is a problem I've run into a few times. Imagine you have a record that you want to store into a database table. This table has a DateTime column called "date_created". This one particular record was created a long time ago, and you're not really sure about the exact date, but you know the year and month. Other records you know just the year. Other records you know the day, month and year.

You can't use a DateTime field, because "May 1978" isn't a valid date. If you split it up into multiple columns, you lose the ability to query. Has anyone else run into this, if so how did you handle it?

To clarify the system I'm building, it is a system that tracks archives. Some content was produced a long time ago, and all that we know is "May 1978". I could store it as May 1 1978, but only with some way to denote that this date is only accurate to the month. That way some years later when I'm retrieving that archive, I'm not confused when the dates don't match up.

For my purposes, it is important to differentiate "unknown day in May 1978" with "May 1st, 1978". Also, I would not want to store the unknowns as 0, like "May 0, 1978" because most database systems will reject that as an invalid date value.

答:
ISO 8601 also specifies a syntax for "fuzzy dates". February 12th, 2012 at 3pm would be "2012-02-12T15" and February 2012 could be simply "2012-02". This extends nicely using standard lexicographic sorting:</p>

```<code>$ (echo "2013-03"; echo "2013-03"; echo "2012-02-12T15"; echo "2012-02"; echo "2011") | sort
2011
2012
2012-02
2012-02-12T15
2013-03
</code>```

Body:
ISO 8601 also specifies a syntax for "fuzzy dates". February 12th, 2012 at 3pm would be "2012-02-12T15" and February 2012 could be simply "2012-02". This extends nicely using standard lexicographic sorting:

$ (echo "2013-03"; echo "2013-03"; echo "2012-02-12T15"; echo "2012-02"; echo "2011") | sort
2011
2012
2012-02
2012-02-12T15
2013-03


答:
I work for a publishing company that deals in lots of old books where we often cannot get the exact dates for things. We typically have two fields for a given date entry, the date and a circa boolean:

```<code>date date
dateCirca enum('Y', 'N')
</code>```

We use the date field to indicate the date of some event, or a date that is "close enough" in the case where we don't know the true date. In the event that we do not know the true date, we mark the dateCirca field as Y and give a close enough date, that is marked as the "1st", such as

```<code>1st March, 2013  // We don't know the day of the month
1st January, 2013  // We don't know the month/day of the year
1st January, 2000  // We don't know the month/day/year, we only know the century
</code>```

Title: How do you store "fuzzy dates" into a database?

Body:
This is a problem I've run into a few times. Imagine you have a record that you want to store into a database table. This table has a DateTime column called "date_created". This one particular record was created a long time ago, and you're not really sure about the exact date, but you know the year and month. Other records you know just the year. Other records you know the day, month and year.

You can't use a DateTime field, because "May 1978" isn't a valid date. If you split it up into multiple columns, you lose the ability to query. Has anyone else run into this, if so how did you handle it?

To clarify the system I'm building, it is a system that tracks archives. Some content was produced a long time ago, and all that we know is "May 1978". I could store it as May 1 1978, but only with some way to denote that this date is only accurate to the month. That way some years later when I'm retrieving that archive, I'm not confused when the dates don't match up.

For my purposes, it is important to differentiate "unknown day in May 1978" with "May 1st, 1978". Also, I would not want to store the unknowns as 0, like "May 0, 1978" because most database systems will reject that as an invalid date value.

Body:
I work for a publishing company that deals in lots of old books where we often cannot get the exact dates for things. We typically have two fields for a given date entry, the date and a circa boolean:

date date
dateCirca enum('Y', 'N')

We use the date field to indicate the date of some event, or a date that is "close enough" in the case where we don't know the true date. In the event that we do not know the true date, we mark the dateCirca field as Y and give a close enough date, that is marked as the "1st", such as

1st March, 2013  // We don't know the day of the month
1st January, 2013  // We don't know the month/day of the year
1st January, 2000  // We don't know the month/day/year, we only know the century


问:
Are there any statically-typed Web scripting languages?
There seems to be two major ways to set up a web server backend.  You can use a LAMP stack, with the server-side code in something like PHP or Python (or Ruby or JavaScript, which don't start with a P.)  These scripting languages tend to be dynamically typed, with all of the disadvantages in performance, correctness and maintainabilty that dynamically typed languages bring, but it's easy to update the server because all the page generation logic is stored in external scripts.

The other major architecture style in wide use is ASP.NET, where the page generation logic is written in compiled code.  This tends to use statically typed languages, so you get code that's had the benefit of a compiler, but all of the page generation logic is compiled into the server, which means if you need to change something, you have to take the server down and replace it with a new build.

So that makes me wonder.  Are there any server-side scripting languages that use static typing, for the best of both worlds?

答:
UPDATE: More current solutions include AtScript

You should try Google Dart.

It has optional typing, which means that you can add type annotations that will be checked in 'checked' aka developer mode at runtime, but they are not required and the language provides many of the advantages of other dynamic languages like python. However, the editor will do type inference to try to help you find errors ahead of time. The dart VM is fast and can produce snapshots, which can accelerate startup time up to 10x. It also compiles down to javascript and it's fast.

The main downside here is that is a somewhat new technology, so there aren't as many libraries for it, and honestly, interoperability with javascript is not as simple as it should be. Also, the language is still in development, so there are breaking changes from time to time.


答:
As there is no inherent relation between a use-case and a GUI screen, it does not really make sense to divide use-case diagrams the GUI screen they relate to.

To elaborate, a use-case describes how an external actor (either a machine or a human) interacts with a system to achieve a particular goal for that external actor.
If the use-case does not involve an interaction with a human, there are no GUI screens involved either. And if the use-case does involve a human actor, then the GUI could be anything from a single button (e.g. to print the current document to the default printer) to multiple screens (e.g. a class creation wizard in an IDE).

Good use-cases don't even mention the existence of a GUI. It just specifies that actor X provides A, B and, optionally C pieces of information and leaves the how for the design team.

If there are many use-cases, then it can be good to split them into multiple diagrams, but that should not be based on a design choice (how the GUI is organised). Better lines of division are

having use-cases that deal with similar functionality on the same diagram

having use-cases that interact with the same actor on the same diagram

Title: Should I create a use case diagram for each screen?

Body:
In projects with many use cases, it's valid to create different use case diagrams, one for each screen? Or I shouldn't guide myself on the GUI of the software to make this, why?

As a clarification, the software does already exist, and I'm just documenting it for my undergraduation final project.


答:
Your question is based on several incorrect assumptions.

Python, Ruby are both compiled to bytecode before execution.  The process is automatic, but application changes require a full restart.  Node.JS behaves the same way but compiles to machine code.

ASP.NET code can be precompiled and copied to the server or dynamically compiled on demand from source code on the server.  IIS handles both forms of updates without zero down time (the first requests after the update will be slow.)  Zero downtime updates are possible with Python and Ruby, but with so many deployment options, are not something to be taken for granted.

Based on that, I would think that you would be happy with any .NET language.

Title: Are there any statically-typed Web scripting languages?

Body:
There seems to be two major ways to set up a web server backend.  You can use a LAMP stack, with the server-side code in something like PHP or Python (or Ruby or JavaScript, which don't start with a P.)  These scripting languages tend to be dynamically typed, with all of the disadvantages in performance, correctness and maintainabilty that dynamically typed languages bring, but it's easy to update the server because all the page generation logic is stored in external scripts.

The other major architecture style in wide use is ASP.NET, where the page generation logic is written in compiled code.  This tends to use statically typed languages, so you get code that's had the benefit of a compiler, but all of the page generation logic is compiled into the server, which means if you need to change something, you have to take the server down and replace it with a new build.

So that makes me wonder.  Are there any server-side scripting languages that use static typing, for the best of both worlds?


问:
Design pattern for data privileges
I'm designing a financial system that should grant access to data based on roles and privileges. For example, a manager can see the financial transaction of users under his domain but not information regarding other transaction. Although it's possible to implement ad-hoc I was wondering if there are patterns or guidelines for implementing such instrumentation.</p>

答:
The Proxy Pattern has a few uses, one is specifically designed for access control. The C2 Wiki also has a discussion of the Proxy Pattern and its variants and more specifically the Protection Proxy.

When the client makes a request, it is forwarded through the Proxy, which would contain your permission checks and access control. If the request is valid, the results would be returned from the RealSubject through the Proxy.

Title: Design pattern for data privileges


问:
Do Windows mail clients actually care about the MIME type of binary attachments?
...or, in other words: When sending a mail programmatically, do I really need to go through the hassle of setting the correct MIME type or can I just use application/octet-stream?

The mail clients I tested all do "the right thing" based on the file extension, even if the MIME type is just the generic application/octet-stream, so why bother?

I know that it is considered "good practice". I'm asking out of scientific curiosity, because I want to know if it makes a practical difference or not.

答:
The robustness principle suggests that should send as correct of data as you can.  The mail clients are also following it by accepting things that are not always properly formatted.  That isn't an excuse not to format your data correctly.

This is most famously described in RFC 791 which defines the IP in TCP/IP.

In general, an implementation must be conservative in its sending behavior, and liberal in its receiving behavior

RFC 1122 (Requirements for Internet Hosts -- Communication Layers) further clarifies this in section 1.2.2.

RFC 3117 (On the Design of Application Protocols) describes some issues when programmers don't follow a robust protocol practice.  It concludes with:

Accordingly, explicit consistency checks in a protocol are very useful, even if they impose implementation overhead.

Write your code following the standard as closely as possible.  Don't take shortcuts because they work on the implementations you have tested on, because someday you will find an implementation that it doesn't work on.


答:
I have a specification, why should I bother with thinking about oracles? I just compare what the spec wants and what the program does.

The thing is: what if you don't have a specification, or the specification is more complex than just a list of inputs and expected outputs? Some examples:

You have a domain expert instead of a complete written specification

The specification says: the system must behave exactly like the legacy system except for these here listed changes

Some of the system's behavior is random, the specification lists constraints like statistical tests

You're implementing a complex numerical algorithm to solve differential equations. The output will contain numerical errors, in some cases more than in others

"Oracle" is simply a more general term than specification.

What is the meaning of the comparator if you just check whether an actual output equals to an expected output?

The last two points above are examples where the check is much more complex than a simple equality test.

Title: What is the real point of an “oracle” in the SW testing?

Body:
I know the definition of an testing oracle but to me, it seems to be overrated. I have a specification, why should I bother with thinking about oracles? I just compare what the spec wants and what the program does. Or I have the use cases, whatever. When I read about oracles in books, it looks as if it a science on its own. Is there anything more behind this concept? Maybe I just did not understand its point.

An oracle consists of a generator and a comparator. What is the meaning of the comparator if you just check whether an actual output equals to an expected output? How else could it be checked?


问:
I can write code... but can't design well. Any suggestions?
I feel that I am good at writing code in bits and pieces, but my designs really suck. The question is, how do I improve my designs - and in turn become a better designer?

I think schools and colleges do a good job of teaching people how to become good at mathematical problem solving, but let's admit the fact that most applications created at school are generally around 1000 - 2000 lines long, which means that it is mostly an academic exercise which doesn't reflect the complexity of real world software - on the order of a few hundred thousand to millions of lines of code.

This is where I believe that even projects like topcoder / project euler also won't be of much help, they might sharpen your mathematical problem solving ability - but you might become an academic programmer; someone who is more interested in the nice, clean stuff, who is utterly un-interested in the day to day mundane and hairy stuff that most application programmers deal with.

So my question is how do I improve my design skills? That is, the ability to design small/medium scale applications that will go into a few thousand of lines of code? How can I learn design skills that will help me build a better html editor kit, or some graphics program like gimp?

答:
Don't live in fear

Strive for simplicity

Listen to your users

Try lots of ideas

Create something, then make it better

Work on things that add value, abandon things that don't


问:
Number of sites to build before trying to become a web developer?
Someone has asked me this question and I couldn't answer as I am not on the web part of development so I'm asking here.

If one has just finished college and is trying to break through into web development/web-design a portfolio of already developed projects/sites seems to be essential to me for going to interviews.

What would you say is a good number of individual sites to create for your portfolio before sending your CV and going to interviews, for a web developer position?

Also what would be a good number of sites to design if you were aiming for a web designer position?

答: Focus on quality, not quantity. If you've built a couple of kick-ass sites, and you aren't a total douchebag, you shouldn't have any trouble getting a web development job.</p>


答:
there are 3 kind of developers i have seen in 13 years of work. some who should be testers and a few have made the switch. others who are mediocore, get by copy-paste, add few lines, set up systems made by others adding a few features here and there, do not design but get the work done. and last developers who take the long view, take time to research, think of maintainability, time to develop etc ...

For (3) The answer is not a number of sites but a number of features. do 2 sites, maybe only 5 pages each but can be rich, with enough background jobs and UI to make it big.

get a good java 6 (or 7 if you can afford it) and java EE book. read it cover to cover.

try to get work on elance/ freelancer

download a few open src (code and binaries), see how the framework works (struts & apache commons, http components ...)

know there are many and sometimes quicker ways to do things - example to get a page you can use http components but for some simple pages URL & java.io is enough ... but they have limitations ... so use an interface where you can.

go thru intervew questions/ pdfs what not

write a few standalone apps [as a servers; utils (sort + file manipulation) ; maybe one UI - web start so you can host it and show it, design the file manipulation so it can be reused in the UI as a component]

go thru what should every programmer know about web development twice (ty gnat)

after reading the books and seeing the code, go back to the 2 websites that you did, see  if you can edit anything - refactor. try to refactor and see the results compare to earlier

try to get a mentor to review and give criticism

Title: Number of sites to build before trying to become a web developer?

Body:
Someone has asked me this question and I couldn't answer as I am not on the web part of development so I'm asking here.

If one has just finished college and is trying to break through into web development/web-design a portfolio of already developed projects/sites seems to be essential to me for going to interviews.

What would you say is a good number of individual sites to create for your portfolio before sending your CV and going to interviews, for a web developer position?

Also what would be a good number of sites to design if you were aiming for a web designer position?


问:
How to optimise an sentiment analysis algorithm for larger data sets?
I am a noob to sentiment analysis and found a good resource for Bayesian Opinion Mining and a way to make it self improving. I was wondering though, if the optimum analysis is dependent upon the supplied data set and since self improvement would mean adding known patterns to the data set (my understanding), wouldn't the application become too overloaded with huge data set over due course of time with more and more patterns getting added to the data set every day ? What should be the proper approach to make the application scalable (if I am using the right term at the right place) ?</p>

答:
It sounds to me that you are building a text classifier with a supervised
training stage at the beginning, where you assign labels manually. Your model
is performing well (high precision and recall), so you want to supplement the
initial training model with an unsupervised training process over new input
strings.

These new inputs will have some known signals (words you've seen before) so your
model can do its job well, but they will also have unknown signals (words you
haven't seen before). You want your unsupervised training process to associate those new words with the known ones, to "learn". This way, you are trusting that the association between the new word and the known word is correct. Because
language processing is so difficult, you will probably generate false positive
associations automatically which would have been excluded/corrected in a
supervised environment. Thus, by doing the unsupervised learning you are
risking lowering your precision.

Your question is about being "overloaded" with lots of data. This is a fair
concern, and depends very much on your data size, implementation choice, and
system behavior expectations. While responsiveness and the tractability of
dealing with large quantities of data is one thing, I feel that the precision
and recall of your sentiment labeling algorithm is probably of greatest
importance.

In the article you linked the author has a confidence score that causes unsupervised associations to be considered only if there's a "high confidence". This is good, but there's still a risk that over time your overall precision will drop. Your system would have to be periodically evaluated for precision and recall, and re-trained. The "Bad Santa" example in the comments is a good example. I suggest you read about semi-supervised training and get this labeling right on small data sets before trusting it to work well on much larger data sets. Language processing is hard!


答: I agree with Bart - use cases are not broken down by screen, it's about what you want to accomplish. For example, you may have a use case that describes what data a customer supplies to create a customer profile or what data the user would supply to make a payment. </p>


问:
How do I make my ASP.NET application take an action based on time?
I have an ASP.NET MVC4 application. Suppose that I wanted to execute some code (such as emailing a user) 2 days after a user has signed up.

One way is to have code executed on every request which scans the database for users and their sign-up time and execute code based on that. This seems messy and it depends on people visiting the application frequently.

Another way is to write a separate application that runs on the server 24/7 and performs the checking itself, perhaps scanning the database at 12:00am every day and performing its own code.

What's the good way to go about this?

答:
A third way is to set up a "task server," which could be its own server, or an independent thread on your server.

Create a class that contains two members: a timestamp, and a "task," which is not a .NET Task, but a representation of all of the information the server will need to do whatever it is that needs to be done.

Create a threadsafe priority queue of objects of this class, that will sort based on timestamp.  The first object to come off the queue should always be the one with the lowest timestamp.

The server thread runs like this:

```<code>while true
   var time = queue.peek.timestamp
   if time > Now
      sleep(Now - time)
   queue.pop.task.execute
   if queue.empty
      block until there's an item in the queue
end
</code>```

That's the basic concept.  Scheduling repetitive tasks (every day at midnight do X) and ensuring that scheduled tasks are preserved even in the event of a server failure are left as an exercise for the reader.

Body:
A third way is to set up a "task server," which could be its own server, or an independent thread on your server.

Create a class that contains two members: a timestamp, and a "task," which is not a .NET Task, but a representation of all of the information the server will need to do whatever it is that needs to be done.

Create a threadsafe priority queue of objects of this class, that will sort based on timestamp.  The first object to come off the queue should always be the one with the lowest timestamp.

The server thread runs like this:

while true
   var time = queue.peek.timestamp
   if time > Now
      sleep(Now - time)
   queue.pop.task.execute
   if queue.empty
      block until there's an item in the queue
end

That's the basic concept.  Scheduling repetitive tasks (every day at midnight do X) and ensuring that scheduled tasks are preserved even in the event of a server failure are left as an exercise for the reader.


问:
RESTful URI's, Creating a new noun/uri or add a queryparameter
I have a Person class with a one to many relationship to Employment. A Person in the CRM can have multiple Employments (historic data).

My current REST URI for this class looks like this:

```<code>http://domain/persons GET [collection]
http://domain/persons/{id} GET [resource]
http://domain/persons/{id}/employments GET [collection]
</code>```

To get the CURRENT active employment (so the employment where the endDate is NULL) should I create a URI or should this be done using a QueryParam?

Basically should I do this:

```<code>http://domain/persons/{id}/employments/current GET [resource]
</code>```

or

```<code>http://domain/persons/{id}/employments?current GET [collection with query param filter]
</code>```

I'm thinking the second one (with the queryparam) because as far as I understand REST a resource should be identified with a non changable URI. Since the current employment position can change it should be done with a QueryParam.

What are your suggestions?

答:
If you code looks something like this or more complicated

```<code> import javax.ws.rs.GET;
 import javax.ws.rs.Path;
 import javax.ws.rs.QueryParam;
 import javax.ws.rs.core.Response;
 @Path("/domain/persons")
 public class UserRestService {

@GET
@Path("{id}/employments/current")
public Response getPersonById(@QueryParam("id") int id) {

   return Response.status(200).entity("getPseronById is called, id : " + id).build();

}
}
</code>```

All this means is that your first assumption is correct

Use domain/persons/{id}/employments/current

Body:
If you code looks something like this or more complicated

 import javax.ws.rs.GET;
 import javax.ws.rs.Path;
 import javax.ws.rs.QueryParam;
 import javax.ws.rs.core.Response;
 @Path("/domain/persons")
 public class UserRestService {

@GET
@Path("{id}/employments/current")
public Response getPersonById(@QueryParam("id") int id) {

   return Response.status(200).entity("getPseronById is called, id : " + id).build();

}
}

All this means is that your first assumption is correct

Use domain/persons/{id}/employments/current


问:
Is it reasonable to use Javascript MVC Frameworks for closed source paid web applications?
I am wondering if it is reasonable to write closed source, paid web apps in Javascript (JS MVC Frameworks like AngularJS, Backbone, Knockout, ...)? I'm concerned because in this type of frameworks you use typically a REST backend for CRUD operations and the majority of business and application logic happens in Javascript which can be looked up by anyone using my app. He can see how i do things.

When I use for example PHP or Java (Wicket) most of the logic is happening on the server and so a lot less of my source code is exposed. This seems to me a lot safer if I want to have an edge over my competitors, so potentially I earn more money.

So is it reasonable to use JavaScript MVC Frameworks for paid applications? Does it depend on something and if yes on what?

答:
If you are only using REST as your entire backend to communicate with the database, that maybe your problem. Using EJB and JPA, the business logic is abstracted away from the user, and handled by the server and its database.

If your application logic is happening in your java code, well then I would suggest that it is not clean code. A servers job is to respond to the questions asked by the client. If the business application logic happens in javascript, then the client is essentially answering their own questions. Essentially, your questions makes no sense.


答:
There is no reason that a compiled language has to be strongly typed or hard to update.

You may want to look at some of the erlang web setups like Cowboy, Webmachine and nitrogen. With Dialyzer you can get some very nice type checking for correctness and you can reload modules on the fly at run time as erlang was designed to be able to upgrade software in place without interrupting existing users.

In addition webmachine (the one I have used most) can be quite fast. I have seen webmachine have average responses on API endpoints of less than 10ms on an amazon EC2 small node!

If you want static typing try the Yesod framework which is for the Haskell language. Haskell is strongly typed with a very rich type system.

I have not used it personally, but the folks I know who use it love it.


答:
There are free, easy, professional, and robust solutions like hangfire designed to do exactly this, with retries and logging and stuff. I recommend it.

(If that's impossible for some reason: In the old days we just created a .NET "console" application and used Scheduled Tasks (which is built into Windows) to run it on a schedule (e.g.: once every hour). Another popular option was to create your own .NET Windows Service. A bit more "serious" and possibly more robust. Trickier to set up and a bit harder to debug, though).

Title: How do I make my ASP.NET application take an action based on time?

Body:
I have an ASP.NET MVC4 application. Suppose that I wanted to execute some code (such as emailing a user) 2 days after a user has signed up.

One way is to have code executed on every request which scans the database for users and their sign-up time and execute code based on that. This seems messy and it depends on people visiting the application frequently.

Another way is to write a separate application that runs on the server 24/7 and performs the checking itself, perhaps scanning the database at 12:00am every day and performing its own code.

What's the good way to go about this?


问:
Improve Bad testing
We have a large team of developers and testers. The ratio is one tester for every one developer.

We have full bug tracking and reporting systems in place.

We have test plans in place.

Every change to the product, the testing team is involved in the design of the feature and are included in the development process as much as possible.

We build in small iterative blocks, using scrum methodology and every scrum they are included in, including the grooming sessions etc.

But every release of the product, they miss even the most simple and obvious defects.

How can we improve this?

答: The first thing you need to do, before this question can be answered any further, is to do a root cause analysis - keep asking "why?". Then you'll hopefully know what needs to be improved/fixed.</p>

Title: Improve Bad testing


答:
As a lone developer, there is not much you can do to improve the number/quality of the defects found by the testers.

As a team, you can try to encourage the testers to try to break the system in as many and as creative ways as they can. That is the role of the tester, to break the system, not to prove that it works.

To get the testers to think in terms of breaking the system, you might try to put out a small reward for the tester that, during the last sprint, broke the system in the most creative way. It does not even have to have monetary value or physical form.

Title: Improve Bad testing

Body:
We have a large team of developers and testers. The ratio is one tester for every one developer.

We have full bug tracking and reporting systems in place.

We have test plans in place.

Every change to the product, the testing team is involved in the design of the feature and are included in the development process as much as possible.

We build in small iterative blocks, using scrum methodology and every scrum they are included in, including the grooming sessions etc.

But every release of the product, they miss even the most simple and obvious defects.

How can we improve this?


问:
How to implement session state in a backend web application?
When using a non-MVC service-oriented/Interactor pattern approach to decoupled system architecture, how is session state implemented?

I've been thinking of building the frontend of an application using MVC (PHP/Ruby etc.), and send messages to a command-line backend which is the actual application. Since I want the offline application to be completely in control of functionality I would imagine using an offline session management system to work best, which at login issues an ID that is thereafter sent to the application on every request. The ID would be stored in a cookie on the client-side, just like a regular session cookie. Since the frontend works through HTTP requests anyway it makes little sense to do it any other way to me.

Are there any better alternatives to this method that I should consider? I would prefer to keep the actual system completely decoupled from the web UI and also plan on implementing a desktop UI aswell, so the Interactor pattern seems to be the best way to achieve this to me. What I'm explicitly trying to avoid is the "do everything in our MVC framework! (and be stuck forever)" that seems to be popular nowadays.

答:
I get what you're asking but the implementation you're describing is remote authentication, not session state. Session state doesn't care whether the user is authenticated or not. You don't want sessions to require authentication generally.

Regardless, back to the question:

What you are interested in is Distributed Session State. Traditionally, this is usually achieved by 1 of 2 main approaches:

A distributed cache cluster that the callers can access remotely. Particular softwares that do this really depends on what languages you are working with.

or

Sessions get persisted to a back-end database and interactions are handled by a service or other proxy-type interface.

The caching implementation is the fastest, as is the nature of reading from memory (even when distributed) but sessions are not persisted through restarts, shutdowns or failures.

The database implementation is a bit slower but is fairly fault-tolerant since they are persisted to the database. To compensate for the speed difference, database implementations usually use in-memory tables within the database, as it speeds up reads significantly and careful planning of indexing must be used (to many unneeded index changes causes the indexes to actually be counter-productive).

Another approach you could consider would be handling session state management like a web-based service. This would allow you to completely decouple sessions from the application and adhere to the mvc principle of maintaining statelessness. The service could be called via anything that can make http requests (even jQuery or ajax if you choose) and all responsibility could be pushed to the client (so you can have your cake and eat it too:))

I'm actually planning on trying the latter approach just to see how large it could be expanded (I've got my sights set on a HPC backend using Casandra db and aiming for a cloud-like structuring of session services).

Either way, any of these approaches will allow you to have sessions for any channels your application offers, including desktop :)


答:
Definitely do NOT use your first idea! lol That would lead to massive problems in the backend when things start to get bigger.

If your backend is MS Sql Server (guessing by the tags in your post), sql server has the ability to schedule tasks, elimination the need to "ping" your app to cause the task to execute (which is very unreliable if important things depend on it). If this is an option for you, then it is probably the best route, unless your apps business logic needs access to the task as well for some reason. If that is the case, the options mentions in other answers will serve well. Another option to note is if you go with scheduled jobs in sql server and have several jobs you would like to be done sequentially or asynchronously, you could couple this with triggers in appropriate locations that start the next (or parallel) task(s).

One note on the task server idea though:
The idea of a task server would be fine but the idea of creating a dedicated thread on the server could be very unreliable due to app pool restarts and, more importantly, thread timeouts (for say, tasks that run once a day or every few days). Just a suggestion.


答:
If you are designing a new system (as opposed to revamping a dinosaur) you may want to look into claims-based authentication systems. It's very versatile and is the next big movement in authentication. It's implementation is now spreading across many platforms and depending on what technologies you choose to work with, may even have support from the application, right through to the back-end.

EDIT: A little more info.

To my knowledge, Microsoft is the only family to integrate claims-based identity support directly into the .NET framework via Windows Identity Foundation (WIF). What I mean by its implementation spreading across platforms is that the Claims-Based Identity model as a design pattern is being implemented across platforms. AT this time it is mostly via open source projects or custom implementations but the pattern is even present in the OAuth2 protocol.

The idea behind claims-based identities is that is supports single sign-on design and does not force the implementation to rely on any one single user store. I think the idea matured originally from Java SAML design. The idea is that rather than the application relying on a user maintaining credentials strictly with the host app, it allows for credentials of users to be supplied from "trusted" sources. A claim is just that. It is a claim of something about the user from a trusted source. A claims aware identity can hold numerous claims, all of different nature and a claim can be anything deemed to be identifiable or an attribute describing the user (email, address, country, phone number, or many other possibilities). These claims are managed via a Security Token Service, which acts as the trusted source.

I am skipping over some points here for the sake of keeping it short and sweet but the reason I suggested the idea to you is that the pattern really can be applied to anything that can work with http requests, so it can be worked into any framework capable of http.

Below are 2 solid articles about claims based identity models.
The first is a bit dated (2009 I believe) but it has a very gentle and easy to understand explanation to the concept.
The second is the microsoft description of their implementation, the pattern its self, and discussions on how to use this cross-realm. (It's long... lol but it's well worth the read)

http://muhammad-ahsan.blogspot.ca/2009/03/claims-based-identity-model.html

http://msdn.microsoft.com/en-us/library/ff359108.aspx

Hope this helps and sparks some fresh ideas :)

Title: Design pattern for data privileges

Body: I'm designing a financial system that should grant access to data based on roles and privileges. For example, a manager can see the financial transaction of users under his domain but not information regarding other transaction. Although it's possible to implement ad-hoc I was wondering if there are patterns or guidelines for implementing such instrumentation.</p>