问:
How do developers find the time to stay on top of latest technologies?
I was a freelance web developer until circa 2004 when I started going down the management route but have decided to try to get back into development again (specifically JavaScript and HTML5 web/mobile web apps) and I really get the impression to be truly good at these and similar fast moving technologies a constant amount of time is required to be set aside to invest in getting better at existing skills in addition to learning new skills.

I understand right now since I am getting back into things there is a pretty steep learning curve, but seeing how good many guys are out there - the only way I see of getting up there is putting in a serious amount of time.

For those working as fulltime developers, what I am trying to understand is this - on most days, how much time in the office is spent actually grinding out code compared to learning/research. I could easily spend 2-4 hours daily getting on top of the best ways to go about doing things.

Do most good developers who are employed full time invest significant hours outside of work sharpening their skills?

Or maybe I'm looking at all of this completely wrong?

答:
There's a few things that I do to try to stay on top of new tech:

I probably spend an average 8-16 hours per week learning something new, or honing some idea, or just trying out a new technology.  For example, I wanted to learn more about ASP.Net MVC 4, so I put together a quick app for work to let me manage some metadata in an old database.  The app almost never gets used, and this wasn't a sanctioned project, but it's around if I need to add new fields, etc, and I can twiddle with it as needed.

I will revisit something I wrote or worked on in the past, and try to redo it in said new technology.  I had written a simple script in Perl a while ago to let me scrape a public webpage every 15 minutes watching for changes, and decided I would see what it would look like in Python, and then I also rebuilt in Clojure.

I pay for and use a Safari account from O'Reilly and I constantly swap in new books on something that interests me.

I read through the different StackExchange boards to see what's popular, what's trending, etc.  Even if I don't know the answer to a question, I will sometimes try to figure out the answer on my own.

These are just a few of the things that I do, but the thing that's clear is if you want to learn and stay up-to-date on something new, you'll just have to be willing to invest the time.  It's like any other skill you want to hone.


问:
What build tools do not depend on java (or Ruby)?
I'm wondering what generic build tools out there include their binary run-times and do not depend on another environment not shipped with them.

For example, ANT requires Java, Rake requires Ruby, etc..

would be great if talking about also target-platform-agnostic tools, where I'd just give whatever command for building, whatever command for testing, etc.. and can then define my artifacts in CI or so.

Would see something like that useful for building .NET projects (say, on both Windows .NET and  Mono), and Node JS projects especially. I do not want to install Java and / or Ruby if what I want is a .NET build or a Node JS build.

This is a bit of general awareness question not an exact problem I'm facing, that's why it's here not on StackOverflow.

Update:

To explain a bit more,  what I'm after is the build script that would run MSBuild for compiling for example ( in .NET, and then maybe several Node/NPM commands in Node, etc..), and then have the rest build/test steps, instead of setting these all in MSBuild (again, in .NET case, also, wondering if there is equivalent story in Node).

Update2:

My target is being able to have a developer on a vanilla machine to just clone the code without installing any dependencies except the platform the code targets be it .NET/Mono or Node (not even Java, Ruby, or Python) and still be able to build and run tests.

答: Sounds like a job for make(1).  It's compiled to a native executable, so you don't need to install anything, and can launch any other build tools you might need.  If your builds are straightforward (no complicated dependency management or fragmented configuration specifications) then it shouldn't be too hard to set this up.  The only thing simpler would be to have a pair of build scripts (one for Windows and one for Linux) you check out with the source.

ContentLicense: CC BY-SA 3.0


问:
Looking for enterprise web application design inspiration
I've checked many websites to be inspired about what the look and feel of a serious enterprise web-application should look like. But the whole I saw were designed for being used by single users and not serious corporate users. The designs I saw were mostly happy-colored, and looked like being developed by a small team of eager young passionate developers.

But what I'm looking for, are showcases of serious web apps' designs (e.g. web apps developed by large corporations) that are developed for being used by a large number of corporate uses.

Can you suggest me sources for this kind of inspiration?

答: Go to Google image search and enter  "${product} screenshots" where "${product}" is something like "peoplesoft" or "sap financials" or "baan".  Not sure how inspiring they'll be (enterprise-class systems are rarely sexy) but they'll be real-world examples.</p>


问:
Software Manager who makes developers do Project Management
I'm a software developer working in an embedded systems company.
We have a Project Manager, who takes care of the overall project schedule (including electrical, quality, software and manufacturing) hence his software schedule is very brief.

We also have a Software Manager, who's my boss.
He makes me write and maintain the software schedule, design documents (high and low level design), SRS, change management, verification plans and reports, release management, reviews, and of course the software.

We only have one Test Engineer for the whole software team (10 members), and at any given time, there are a couple of projects going on.

I'm spending 80% of my time making these documents. My boss comes from a Process background, and believes what we need is better documentation to improve software:

He considers the design to be paramount, coding is "just writing the design down", it shouldn't take too long, and "all the code should be written before the hardware is ready".

Doesn't understand the difference between a Central & Distributed Version control, even after we told him its easier to collaborate with a distributed model.

Doesn't understand code, and wants to understand every bug and its proposed solution.

Believes verification should be done by developer, and validation by the Tester. Thing is though, our verification only checks if implementation is correct (we don't write unit tests, it's never considered in the schedule), and validation is black box testing, so the units tests are missing.

I'm really confused.

Am I responsible for maintaining all these documents? It makes me feel like I'm doing the Software Project Management, in essence. I'm ok with technical documentation, but I believe scheduling/planning should not be done by the developer.

I don't really like creating documents, I want to solve problems and write code. In my experience, creating design documents only helps to an extent, its never the solution to better or faster code.

I feel the boss doesn't really care about making better products, but only about being a good manager in the eyes of the management.

What can I do? This whole year I've done 3 months of actual coding, the rest just spent in making documents and waiting for bug reports from clients.

答:
The CMMI implementations I've seen or directly heard about have all been process and documentation heavy; your bosses belief that the documentation should be detailed enough to make the implementation trivial strongly implies that his expectation is similar.

If you're receiving a disproportionate share of the documentation writing/maintenance, asking that it be more evenly distributed is reasonable.  If the other developers have similar documentation to coding ratios and you want to spend most of your time writing code; it may be time to consider finding a new employer.


问:
How to deal with well-meaning open-source contributors?
If a programmer contacts you and asks to contribute to your project, how do you handle it?

You don't know if this guy is any good. Perhaps he'll be more trouble than he's worth.

He might be trying to attach his name to a successful project just for the kudos.

He might be trying to take the project in a direction you don't really want, adding features you think aren't worth the extra complexity.

Or, he might be a very useful contributor. You just don't know.

How do you handle such requests from people you don't know (On GitHub, specifically, if that makes any difference)? What's the etiquette here?

答:
Membership in an OSS project is not the same as a funded, corporate team where people are interviewed and chosen. The source is already out there (it isn't open source otherwise). Tell them to send in some patches. If they are good patches (and you must review them first), commit them. Once the prospect builds up trust and a  a history of making valuable contributions, give him write access.

OSS teams grow organically. Make yourself available to ask for questions and let them build up some street cred over time.


问:
How to sell logistical procedures that require less time to perform but more finesse?
I am working with a group where part of the responsibilities is managing a certain set of configuration files which, of course, have the same skeleton/structure across different environments but different values (like server, user, this setting, that setting etc.).  Pretty classic scenario...

The problem is that everyone just goes and modifies final, environment-specific files and basically repeats the work for every environment.  Personally, I am offended to have to peform repeatable, mundane tasks in this day and age when we have technologies to automate it all.

So I devised a very simple procedure of abstracting the files into templates, stubbing env-specific values with parameters and then wrote a simple Perl script that, given a template and an environment matrix with env-specific values for each param, produces the final file. So this is nothing special, cutting-edge or revolutionary -- I am pretty sure that 20 years ago efficient places did their CM like that.

However, that requires that changes are made at the template level and then distributed across different environments using the script and not making changes in the final environment-specific files.  This is where I am encountering resentment as they feel "comfortable" doing it their old, manual, repeated labor way.  Personally, I don't have a problem with them working hard rather than smart but the problem is when I have to build on top of someone else's changes, I have to merge their changes into my template from a specific file, which takes time and is grueling.

So my question is how to go about selling my method, which makes it so much faster in an environment that is resentful to change and where most things have to be done at the level of the least competent team member?

答:
Some people feel any time spent in analysis is time wasted that could be better used doing "actual job". Disregardling the fact that that analysis time will pay off later.

They need to feel they are "doing, not thinking".

Chances are their boss just see them "busy", and that makes him happy.

The only way to convince them is to show them.

Prepare a demo, call a meeting, get a projector and show them.

Make it so good they cannot afford not to use it.

Show them numbers showing the time saved with your method.

Offer them support to implement it.

Show them an scenario when you can do a masive change in matter of minutes instead of days, and then roll it back.


问:
How to get lookahead symbol when constructing LR(1) NFA for parser?
I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial NFA. What I don't understand is how to get/compute a lookahead symbol.

Here is the example from the book, the grammar:

```<code>S -> E
E -> E - T
E -> T
T -> ( E )
T -> n
</code>```

n is terminal. The "weird" transitions for me are is the sequence:

```<code>1)   S -> . E        eof
2)   E -> . E - T    eof
3)   E -> . E - T    -
4)   E -> E . - T    -
5)   E -> E - . T    -
</code>```

(Note: In the above table, the state numbers are in front and the lookahead symbol is at the end.)

What puzzles me is that transition from (4) to (5) means reading - token, right? So how is it that - is still a lookahead symbol and even more important why is it that eof is no longer a lookahead symbol? After all in an input such as n - n eof there is only one - symbol.

My naive thinking tells me (5) should be written as:

```<code>5)   E -> E - . T    - eof
</code>```

And another thing -- n is terminal. Why it is not used at all as a lookahead symbol? I mean -- we expect to see - or (, it is ok, but lack of n means we are sure it won't appear in input?

Update: after more reading I am only more confused ;-) I.e. what is really a lookahead? Because I see such state as (p.292, 2nd column, 2nd row):

```<code>E -> E . - T      eof
</code>```

Lookahead says eof but the incoming input says -. Isn't it a contradiction? And it is not only in this book.

答: A lookahead token is a character (or sequence of characters, it's a token after all) defined as either one of the terminals or those tokens which are in the FOLLOW set. Look at the possible transitions from Follow to the next FIRST and consider that the next token is possibly the eof token because of the nature of LR parsing. (it considers the whole next token and its inner unfoldings. hence bottom-up parsing.)</p>


问:
Which library/framework did you ditch as being too complex for the problem it is solving?
... and coded the functionality "manually"?

As a rather metaphorical example, you'd hardly need a library for solving quadratic equations, although such libraries do exist and someone takes them seriously.

As a more questionable case, depending on the circumstances I may ditch jQuery (for example when I don't need to support some stone age browsers): it does simplify some things but it adds another layer of complexity and unreliability to DOM. And overusing jQuery leads to absurd problems, such as one seen recently on SO: how do I assign an empty href to an a tag with jQuery? Turned out it was a HTML question, not even JavaScript.

Another absurd case and yet non-obvious for many is using some templating engine/language built on top of another templating system: PHP. Third level of templating anyone?

And another one: sometimes just spitting out XML with (figurally) printf is far easier than doing it with some monstrous XML engine.

Any other cases from your experience?

答: Ohhh, so many. I've worked on quite a few agile projects using open source apis. Great when they work, but often we have suffered developers with a fetish for bringing in all sorts of 3rd party apis, some obscure, some not, just because they want to use one or two classes in them. The end result being a mish-mash of code and hacked together systems. They hand it over claiming it's the best code ever, leave and the poor slobs who pick it up find an unintelligible, undocumented mess full of dependency issues and hacks.</p>

ContentLicense: CC BY-SA 2.5


问:
AGPL License - does it apply in this scenario?
There is an AGPLv3 based software (Client) that makes web service calls (using SOAP) to another software (Server - commercial, cloud based). There is no common code or any connection whatsoever between these two except for the web service calls being made.

My questions -

Does the Server need to be AGPL too?  I guess not - but would like to confirm.

Let us say the end point URL for the Server can be configured on the Client side (by editing an XML file) to connect it to different Servers (again, there is no connection other than the webservice calls being made) does it require any of these Servers being AGPL?

Are there any issues in running the Client as a DLL that is loaded by other commercial applications on users' desktops? Does it require these other applications also to be AGPL?

答:
Two months late, but still...

First of all, IANAL, ask a real professional, and all the usual stuff.

I'll start with 3.

You're touching something very big. According to the FSF, not just AGPL, but even GPL forces you to open-source anything that is dynamically linked to a (A)GPL binary.

According to some other people, you're not really forced to do it. There hasn't been any major lawsuit that can confirm either view of the problem, but the spirit of the GPL is that you should open-source the client application in its entirety. Needless to say, this goes to AGPL too, as AGPL is more restrictive than the GPL. Important note: you can license the rest of the client as GPL, not AGPL, since AGPL allows for this.

1 and 2 are virtually the same, to a certain extent. In both cases I believe you're allowed to do what you want under AGPL. Let's look at the relevant text in AGPL:

Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.

So, if you modify the client library that is AGPL licensed, you must provide the modified source code to all parties, that will communicate with it, that is to all servers. As far as I understand, you're not modifying the source code of the client, so it's not your problem at all.


答:
Consider the expression

A - B - C

You would have E - B .- in your parse state.  At that point, you have to reduce B to a T, then reduce E - T to E, leaving E .- C as the parse state.  (Yes, that's sloppy.)

Title: How to get lookahead symbol when constructing LR(1) NFA for parser?

Body:
I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial NFA. What I don't understand is how to get/compute a lookahead symbol.

Here is the example from the book, the grammar:

```<code>S -> E
E -> E - T
E -> T
T -> ( E )
T -> n
</code>```

n is terminal. The "weird" transitions for me are is the sequence:

```<code>1)   S -> . E        eof
2)   E -> . E - T    eof
3)   E -> . E - T    -
4)   E -> E . - T    -
5)   E -> E - . T    -
</code>```

(Note: In the above table, the state numbers are in front and the lookahead symbol is at the end.)

What puzzles me is that transition from (4) to (5) means reading - token, right? So how is it that - is still a lookahead symbol and even more important why is it that eof is no longer a lookahead symbol? After all in an input such as n - n eof there is only one - symbol.

My naive thinking tells me (5) should be written as:

```<code>5)   E -> E - . T    - eof
</code>```

And another thing -- n is terminal. Why it is not used at all as a lookahead symbol? I mean -- we expect to see - or (, it is ok, but lack of n means we are sure it won't appear in input?

Update: after more reading I am only more confused ;-) I.e. what is really a lookahead? Because I see such state as (p.292, 2nd column, 2nd row):

```<code>E -> E . - T      eof
</code>```

Lookahead says eof but the incoming input says -. Isn't it a contradiction? And it is not only in this book.


问:
What to do when a project is too difficult to continue developing?
As a developer, can you tell your project manager that an application is unworkable? Or, if you're a project manager, how would you need this presented to you in order to be compelled? This isn't about "how to work on a poor project", it's assuming you cannot.

I can provide an example of the situation if anyone thinks it's important, but I'm trying to avoid proposed solutions to "plodding through".

EDIT, 8 years later:

I still think this is valid question and generally agree with @dagnellies, but I understand what everyone else was drawing at. As @mattnz pointed out, nothing is impossible, but I think it may have been impossible for me to succeed, by myself, on any reasonable timeline.

A team of 2-3 engineers, some more senior, armed with a solid plan for acceptance testing, refactoring, and unit testing probably could have succeeded.

I couldn't fathom a solution because I was working alone, not very experienced, and had no support from my manager. In the end, those problems were just as important as the tech debt.

答:
Professional pride (the bad kind, not the good kind) often makes coders feel like they have to plod through. In practice, I find it better to get together with the project manager (doing tougher discussions like this during lunch often takes the pressure off, IMHO) and discuss your concerns. Let them know the problems, perhaps insurmountable, you have found. Propose some sort of compromise, if at all humanly possible. For example, "We cannot add Foobar feature in the time allotted, but implementing FizzBuzz will take a quarter of the time and give us 75% of the functionality."

Any project manager worth his salt wants to know these things and will be open to discuss concerns especially if you have some sort of solution.

ContentLicense: CC BY-SA 4.0


答:
Whenever someone assigns a task for you in that project, make an honest and careful estimation about the effort. If the thing is really "unworkable", as you say, those estimations fewer or later will be adding up to a sum so high that your project manager will consider to stop the project. </p>

<p>(Or he will assign the task to someone else who does not think the same way you do about the thing).</p>

Title: What to do when a project is too difficult to continue developing?

Body:
As a developer, can you tell your project manager that an application is unworkable? Or, if you're a project manager, how would you need this presented to you in order to be compelled? This isn't about "how to work on a poor project", it's assuming you cannot.

I can provide an example of the situation if anyone thinks it's important, but I'm trying to avoid proposed solutions to "plodding through".

EDIT, 8 years later:

I still think this is valid question and generally agree with @dagnellies, but I understand what everyone else was drawing at. As @mattnz pointed out, nothing is impossible, but I think it may have been impossible for me to succeed, by myself, on any reasonable timeline.

A team of 2-3 engineers, some more senior, armed with a solid plan for acceptance testing, refactoring, and unit testing probably could have succeeded.

I couldn't fathom a solution because I was working alone, not very experienced, and had no support from my manager. In the end, those problems were just as important as the tech debt.

Body:
Whenever someone assigns a task for you in that project, make an honest and careful estimation about the effort. If the thing is really "unworkable", as you say, those estimations fewer or later will be adding up to a sum so high that your project manager will consider to stop the project.

(Or he will assign the task to someone else who does not think the same way you do about the thing).


答:
Well, I think that basically everything actually can be done, it's just a matter of how much time it takes. On a bad codebase everything naturally takes a lot of time. So when you are saying something is unworkable, I take it you mean that it would take way too much time to implement a certain feature for it to be practically viable. I know a manager might think it's just a small issue, you know, "we already have other features like this, how can it be so hard?".

If your manager is the kind of person who does not understand when you explain them about the anti-patterns and the smells in the code, they should at least understand time and money. So create a good, solid plan on how you would implement the requested feature if you indeed are to do that. Take your time to analyze the code and split the project into manageable sub-tasks of no more than about 8 hours long. Draw a Gantt chart of the project stages. When you cannot provide an accurate time estimate, give at least some range: 10-30 hours is better than nothing. Explain the optimal case when it'd take 10 hours, and let your manager know of the conditions of the worst-case scenario of 30 hours. Then perhaps say something like "Given the general code quality, I'm pretty sure we'll end up closer to 30 than 10 hours. Also notice that this task is on the critical path of the Gantt chart, so when this one is delayed, the whole project is delayed."

You'll end up with your project A taking, say, 100 hours. Then devise another plan, writing the requested feature from scratch. This might be difficult to include in your existing project if/when your interfaces are bad too, so remember to include the time of integrating the two systems together. This way you might end up with a project B that takes, say, 90 hours instead. That's probably too close to the original plan for the manager to buy it, but then you should explain them how the new codebase would be way easier to modify further: if you implemented Feature 2 on top of Project A, it'd take 30 hours, but if you did the same to Project B, it'd only take 5 hours. At this point I'm pretty sure your manager starts seeing the things in perspective.


答:
There is really no way to answer this without specifics. How important is this project for the company?  How much time and money has already been spent?  Are you proposing to abandon the project completely, or is it possible to reduce its scope, increase the time frame, or add more people?

If this is the main product of the company, and abandoning it means that the company goes belly up, then all you can do is resign.  On the other hand, if you can suggest something constructive, such as dropping some features or allocating more time and people, then it is perfectly reasonable to take it up with the manager.


问:
Can fixed scope + fixed deadline + fixed price contract ever be made to work with "agile"?
Some projects we run internally using are Scrum, while still being "fixed everything" to the customer.  We're experiencing mixed success on our part (the customer likes the visibility of the burndown chart). Can the types of projects we work be successfully executed using the agile methods?</p>

答:
Fixed price / fixed deadline / fixed scope can be done at least as well agile as it can be in waterfall.

In waterfall, time estimations are inaccurate, and details end up being implemented differently than the original specifications.  In other words, the deadline/scope can't be known exactly in advance.

In agile, you can do sprint zero to generate a backlog of user stories, and make some estimations.  Then agree to meet the value stories for a fixed price, by a fixed deadline.  The scope is fixed in terms of the value stories that you intend to fulfill, and no promises are made about user stories.

In other words, you promise to deliver what matters, and avoid making promises about specific design decisions that are unrelated to the revenue/savings/etc. that the project is supposed to deliver.


答:
One trick I use when learning something new is to go to StackOverflow (or the tech's dedicated forums), find a question I don't know the answer to, and work out the answer. Also, I try to implement something using the new skill/tech. For example, in order to learn how to program for windows phone, I asked my wife what kind of app she wants. The process of writing that app forced me to learn more about the platform and how to do things.

You can read all you want, but you don't get experience from reading.

Title: How do developers find the time to stay on top of latest technologies?

Body:
I was a freelance web developer until circa 2004 when I started going down the management route but have decided to try to get back into development again (specifically JavaScript and HTML5 web/mobile web apps) and I really get the impression to be truly good at these and similar fast moving technologies a constant amount of time is required to be set aside to invest in getting better at existing skills in addition to learning new skills.

I understand right now since I am getting back into things there is a pretty steep learning curve, but seeing how good many guys are out there - the only way I see of getting up there is putting in a serious amount of time.

For those working as fulltime developers, what I am trying to understand is this - on most days, how much time in the office is spent actually grinding out code compared to learning/research. I could easily spend 2-4 hours daily getting on top of the best ways to go about doing things.

Do most good developers who are employed full time invest significant hours outside of work sharpening their skills?

Or maybe I'm looking at all of this completely wrong?


答:
As Michael just suggested, the crucial point is: why this project is unworkable?

A project manager just wants to know this. Give me at least one really convincing reason to give up and I will do. I will try to limit the damage at its very least and I will stop the development.

As you can imagine, it is very, very hard to convince a project manager that a running project has to be stopped because of technical considerations. Even a total mess of code can be improved or rewritten.

Different is the case of business considerations. A project can lose its reason to exist during development. Maybe a new competitor has already occupied the intended niche, maybe the investor hasn't any more money. There can be business reasons to stop a project.

My very personal suggestion is to identify with your best care the issues with the project, explain to your project manager why they make the project unworkable and what you can do to fix them.

Be constructive. Bring to the table every solution you can find. Do your best to save the project even if you think it should be killed.

The last decision must be left to the investor and to the project manager. If your explanation is convincing, they will stop the project.


答:
Very rarely there is a case where there is an unworkable requirement, usually it's a "I don't want to", being phrased "I can't".

Never use absolutes like "Can't" and "Unworkable" unless you can prove it and are absolutely certain you have exhausted all options. You look like an idiot or a lair when it does get done. A number of times I have been told "That can't be done ...." and seen it working a few days or weeks later, leaving the developers looking bad. It puts them and the rest of the team in a really difficult position next time, as they say "Can't"  and are not believed.

If you really can't, provide alternates for consideration, with the benefits and drawbacks of each. Follow this with open and honest discussion. Focus on finding a solution, not defining the problem.

Your manager did not hire you to create problems. He hired you to create solutions. Never take a problem to you manager that you have not got (or at least tried to get) a solution for.


答:
You want to start by doing some sort of a cost-benefit analysis. You aren't truly going to understand how important it is to simply start over without being able to prove that this is the best approach to take. This is not to say that your gut instinct is necessarily wrong, however you really need to determine for yourself whether it is a case of reluctance or professional pride holding you back from working on an existing product.

As bad as an existing code base may seem, you need to remember that there is likely already a sizable investment in it that your employer won't want you to throw away without really good cause.  The reality is that even if you have a very good reason for wanting to throw out the existing product, you are going to need to justify your reasoning, and to make this work for both your team and the company, you are going to need to be able to show not only how much the product is costing and could save if done another way, you are also going to need to show that there is a pathway to your "golden ideal" that won't be prohibitively expensive, and you can be sure that this extra pathway cost is going to be counted against any potential savings you might think you can show for your rewrite.

What I am saying is that this is going to be a lot of work.  You need to show what the product is costing to maintain as is, what it would cost to improve it to make it of a better standard, what it would cast to maintain while preparing to move over to a new product, what it will cost to develop a new product, and what the new product is likely to cost to maintain.

The reality though is that no product is actually TOO difficult to maintain.  All products may be improved with a careful approach to refactoring and gradual improvement over time.  This is also going to be a lot of work, but less likely to be as costly or effort intensive than attempting to justify throwing in the towel simply because the present code base is hard to maintain.  When I hear someone say that code is unmaintainable, what I hear is that it is likely the code is unsupported by unit tests, likely unsupported by test/build automation, likely a lot of code duplication, and likely to be a lot of classes and methods that are all trying to do too much.  These are all problems which are fixable to a very large degree if a little effort is applied to address them.

In a few very rare cases, you may have code that has dependencies over which you have no control.  I'm talking old legacy 3rd party products that are being used by your code base, but which have been forgotten about because they were added years ago, and all of the experts have left without documenting why these products were in use. In such cases you might be justified in starting something new, however depending on the size of your product it is rare that this would justify rewriting an entire product from scratch.

I don't think you can really find a definitive answer to your question.  Much of this sort of thing really comes down to you, your team, company culture, time, available funding, and a whole mess of other things.  This is the sort of stuff that you really need to sit down with your manager to discuss, and to work out as a team to determine the best path forward.


答:
This kind of problem is usually a disagreement over the time and resources required, and there are many ways to either negotiate for more time, more resources, a different development path, or different objectives.

If you can't negotiate a satisfactory work plan, then your fallback position can always be "this project can't be done by me (with these resources)".  If that doesn't either cancel the project or move it to someone else's plate, then it's hard for management to hold you responsible for failure.

In any case, the important thing is that you do not accept a task you can't complete just to placate your manager.


问:
Caching by in-memory dictionaries. Are we doing it all wrong?
This approach is pretty much the accepted way to do anything in our company.

A simple example

When a piece of data for a customer is requested from a service, we fetch all the data for that customer(relevant part to the service) and save it in a in-memory dictionary then serve it from there on following requests(we run singleton services).

Any update goes to DB, then updates the in memory dictionary.

It seems all simple and harmless but as we implement more complicated business rules the cache gets out of sync and we have to deal with hard to find bugs.

Sometimes we defer writing to database, keeping new data in cache until then. There are cases when we store millions of rows in memory because the table has many relations to other tables and we need to show aggregate data quickly.

All this cache handling is a big part of our codebase and I sense this is not the right way to do it. All of this juggling adds too much noise to the code and it makes it hard to understand the actual business logic. However I don't think we can serve data in a reasonable amount of time if we have to hit the database every time.

I am unhappy about the current situation but I don't have a better alternative.

My only solution would be to use NHibernate 2nd level cache but I have nearly no experience with it.

I know many companies use Redis or MemCached heavily to gain performance but I have no idea how I would integrate them into our system.

I also don't know if they can perform better than in-memory data structures and queries.

Are there any alternative approaches that I should look into?

答:
First you last question:  Why Redis/memcached?

No, they're not (usually) faster than simple in-process dictionaries.  The advantage comes when you have several worker processes, or even many app-layer machines.  In that case, instead of each process having its own small cache, they all share a single big (distributed) cache.  With bigger caches, you get better hit ratios.

As you can see, the cache layer becomes a shared resource, much like the database, but (hopefully) faster.

Now, about the big part: how to avoid the mess?

It seems that your problem is keeping cache consistent while at the same time decoupling it from the database.  I see three pain points there:

cache invalidation.  This is just hard.  Sometimes the easiest solution is to add a generation ID to every record and use it as part of the cache key.  When the data is updated, you get a new generation ID, and the next cache query won't hit so you go to the database and refresh the cache.  Of course, the (now unused) entry must have a sensible expiry time so it gets eventually purged from cache.

writeback. You say you work on the cache and update the database later.  This is dangerous; most architectures avoid that idea.  A step on the right direction would be to mark every new or modified entry in the cache as 'dirty' so can be flushed to database by a decoupled process.  A better idea might be to add to a message queue as soon as it's modified, effectively making the writing to the database 'inline but async'.  In the end, I think you should realize that this is not a valid use for a cache, this is a "staging area" that should be treated with a different architecture than a cache layer.

interprocess synchronization: since your in-process cache is private to each process, any modification there isn't propagated to other processes until they're flushed to database.  This might be correct under your app design (kind of poor man's transaction isolation), but might have unintended results.  A much more manageable architecture is a cache layer that is just a faster API to the database, with the same shared properties as the database, and just as 'authoritative' as it.  For that you need out-of-process caches, like memcached or Redis.


答:
If you want to see what corporate look and feel that's been designed by a committee and tweaked over a number of years, I can think of no better example than Microsoft's online version of Outlook.

Failing that, take a look at Salesforce.com or QuickBooks Online.

Use TMN's advice to see these if you don't have access.

These are all example of well-designed, but very boring and to the point interfaces, which is what you seem to be after.

Title: Looking for enterprise web application design inspiration

Body:
I've checked many websites to be inspired about what the look and feel of a serious enterprise web-application should look like. But the whole I saw were designed for being used by single users and not serious corporate users. The designs I saw were mostly happy-colored, and looked like being developed by a small team of eager young passionate developers.

But what I'm looking for, are showcases of serious web apps' designs (e.g. web apps developed by large corporations) that are developed for being used by a large number of corporate uses.

Can you suggest me sources for this kind of inspiration?


问:
As a programmer, are you required to do timesheets?
Timesheets are something that I've never been fond of, but none the less something that is a requirement within my company. They don't bother me so much, but they seem to really grind some other people's gears. I suppose I have a few questions, and feedback would be great.

Are you required to do timesheets, assuming you aren't a contractor? (That is understandable to me).

What is the granularity of timesheets that you would be comfortable with or that you use? (ex: all entries must be under two hours).

Would timesheets ever factor into your reasons for not accepting a job or leaving a current one?

How has management within your organization justified timesheets if you aren't billing to a client?

答:
Our timesheet application is also used to track vacation and expense reports.

Entering time is done to a granularity of 1/2 hour. This is done for reporting purposes to senior managers. Some of the developers refuse to do it and thus end up getting tasked about quarterly to stop working and get their time up to date. I try to remind the guys if they spend 12 hour days, and they write 8 hours, the idiots up top will get the idea we don't need any new developers at all - we can handle the load. We had a death march that involved 7-day workweeks for many months over last winter. Half the devs wrote 40hr/week in the reporting app, while several of us reported actual numbers. By federal law, programmers are "exempt" (which means exempt from overtime in the absence of a union contract to the contrary), so reporting overtime won't get us paid overtime, but the numbers will still show up in reports.

Time spent working on bugs and new code is tracked (sort of) in Team Foundation Server and we only track it with a granularity of 1 whole day. This we're trying to do in order to get better at estimating how much time it will take to do things, as our estimation process is off by -25% to +1000%. At the moment, throwing darts at a calendar across the room is about as accurate as our estimating processes.

The previous place used timesheets to bill clients, so if you spent 45 minutes working on a client's bug, then the client got billed for 45min.


问:
Abstract Factory Method and Polymorphism
Being a PHP programmer for the last couple of years, I'm just starting to get into advanced programming styles and using polymorphic patterns. I was watching a video on polymorphism the other day, and the guy giving the lecture said that if at all possible, you should get rid of if statements in your code, and that a switch is almost always a sign that polymorphism is needed. At this point I was quite inspired and immediately went off to try out these new concepts, so I decided to make a small caching module using a factory method. Of course the very first thing I have to do is create a switch to decide what file encoding to choose. DANG!

```<code>class Main {
    public static function methodA($parameter='')
    {
        switch ($parameter)
        {
            case 'a':
                $object = new \name\space\object1();
                break;

            case 'b':
                $object = new \name\space\object2();
                break;

            case 'c':
                $object = new \name\space\object3();
                break;

            default:
                $object = new \name\space\object1();
        }

        return (sekretInterface $object);
    }
}
</code>```

At this point I'm not really sure what to do. As far as I can tell, I either have to use a different pattern and have separate methods for each object instance, or accept that a switch is necessary to "switch" between them. What do you guys think?

答:
switch and if are generally required for primitives (which $parameter is), although PHP allows you to create class names and call methods from variables, so you can get away with a little magic.

For example you could do this:

```<code>public static function methodA($parameter = '') {
   $class = '\name\space\object' . $parameter;
   return new $class;
}
</code>```

This of course requires that $parameter and the existing class names are all named appropriately, and it does not account for the default case.

The other alternative requires that $parameter be an object.  You can remove the switch, but it doesn't cut down on verbosity (although it is polymorphic).  One possible way:

```<code>public static function methodA(EncodingType $parameter = null) {
   return new $parameter->getObject();
}
</code>```

...which would return the required object string.

I should note that the principle of avoiding conditionals in contemplation of using polymorphism is great, especially in theory -- but if you try to eliminate all conditionals in your code you can half kill yourself and end up writing unnecessarily complicated code.  It may also be a signal to rethink your design.

I will say that one common spot where you find conditionals is in factory methods, which is what your example seems to be.

Body:
switch and if are generally required for primitives (which $parameter is), although PHP allows you to create class names and call methods from variables, so you can get away with a little magic.

For example you could do this:

public static function methodA($parameter = '') {
   $class = '\name\space\object' . $parameter;
   return new $class;
}

This of course requires that $parameter and the existing class names are all named appropriately, and it does not account for the default case.

The other alternative requires that $parameter be an object.  You can remove the switch, but it doesn't cut down on verbosity (although it is polymorphic).  One possible way:

public static function methodA(EncodingType $parameter = null) {
   return new $parameter->getObject();
}

...which would return the required object string.

I should note that the principle of avoiding conditionals in contemplation of using polymorphism is great, especially in theory -- but if you try to eliminate all conditionals in your code you can half kill yourself and end up writing unnecessarily complicated code.  It may also be a signal to rethink your design.

I will say that one common spot where you find conditionals is in factory methods, which is what your example seems to be.


问:
How do web-developers do web-design when freelancing?
So I got my first job recently as junior web-developer.

My company creates small/medium sites for wide variety of customers: autobusiness companies, weddign agencies, some sauna websites, etcetc, hope you get my point. They don't do big serious stuff like bank systems or really big systems, it's mostly small/medium-sized websites for startups/medium sized business.
My main skills are PHP/MySQL, I also know HTML and a bit of CSS/JS/AJAX.
I know that good web-developer must know some backend language (like PHP/Ruby/Python) AND HTML+CSS+JS+AJAX+JQuery combo.

However, I was always wondering. In my company we have web-designer. In other serious organisations I often see the same stuff: web-developers who create business-logic and web-designers, who create design.
As far as I know, after designers paint design of website they give it to developers either in PSD or sliced way, and developers put it together with logic, but design is NOT created by developers. Such separation seems very good for full-time job, but I am concerned with question how do freelance web-developers do websites?
Do most of them just pay freelance designers to create design for them? Or do some people do both?

Reason why I ask - I plan to start some freelancing in my free time after I get good at web-development. But I don't want to create websites with great business-logic but poor design. Neither I want to let someone else create a design for me. I like web-development very much and I am doing quite good, I like design aswell, even though I am a bit lost how to study it and get better at it.
But I am scared that going in both directions won't let me become expert, it seems like two totally different jobs and getting really good in both seems very hard. But I really want to do both. What should I do? Thank you!

答:
I started back in the late 90s and have done a bunch of "Lone Gunman" projects right up to last year. When it came to design, I just did the best I could. As time went by I got better and better at it. I've had many compliments regarding the sites I've built from scratch, design and development. It didn't come easy, but eventually you get a sense of what is needed for a particular project. Also, expect to be crushed from time to time. :(

Either you have a "design bone" or you don't. You'll soon find out which.

If you have some skill, it's easy to get inspiration from other great looking sites. Look at the way they use (but not overuse) color. Try to stay within your abilities. Learn how to use fonts, and which fonts work together and which don't.

Overall, consistency is king. Horrible sites are usually horrible because they are a mish-mash of too many conflicting ideas.

However, if you try and fail... then it's time to hire a good designer.


问:
Access functions from user control without events?
On some generic functions, it seems that accessing a function on mainform directly from the usercontrol is easier than raising an event. For example: A function on main form that displays one desired usercontrol centered and tweaked.

Making this function static, or access this function by reference from the user control to its parent, rather then raising an event, seems very convenient. Like this:

```<code>usercontrol_A uc_a = new usercontrol_A();
MainForm mainform_functions = (MainForm)Parent;
mainform_functions.DisplayControl(uc_a);
</code>```

Raising an event seems more complex in such cases - I need to declare all new userControl_B's events (which also includes initiating usercontrol_C and displaying it) inside the event that shows (and initiates) userControl_B:

```<code>  userControl_A.ShowUserControl_B+= (s,e) =>
 {
 usercontrol_B uc_b = new usercontrol_B();   DisplayControl(uc_b);
    uc_b.show_uc_c += (s2,e2) => {usercontrol_C uc_c = new usercontrol_C(); DisplayControl(uc_c); }
 }
</code>```

too much clutter here.

That's hard to code and to read later. (In addition, passing parameters to another usercontrol is a nightmare like this - I need to save the parameter artificially on the sender usercontrol class and then cast it back later inside the event)

What is your opinion? Is it a completely "prohibited" to code like this, at least on generic functions like I mentioned? Isn't clearer code that needs some small tweaking for reuse preferable?

答:
Could you please just state conceptually what you are trying to achieve? The limited explanation and code make it hard to judge about the grander picture which is going to be important here.

However, to answer your initial question more generally:

On some generic functions, it seems that accessing a function on
  mainform directly from the usercontrol is easier than raising an
  event.

Depending on your scenario:

It is not prohibited if your user control always has to be located within that specific parent. In that case I would pass the parent (or an interface which the parent implements) to the constructor of the user control (constructor injection). This makes it impossible for the parent to be anything else than the correct expected type.

If it's at all possible to reuse that user control in different parents, casting to the specific parent would make that impossible. That's where the events would come in handy. This allows for a lot more loosely coupled code.

However, judging from your given code I would say probably the events are the way to go. It seems you are hard coding a lot stuff like userControl_A.ShowUserControl_B. Perhaps you could handle this conceptually in your parent by linking them together there instead? I'm thinking of creating an abstract user control class which has something along a Closed event which the parent class handles after which it shows the next user control.


问:
Writing a Software Requirement Specification
I have a few questions about writing a specification and they are:

When we write a software specification, under the topic "User requirements definition" we have to specify the "Functions" and "Constraints" only ?

Does "User Interface" fall into "functions" or "constraints" ?

What are the major key areas (requirements) a software can be broken into (e.g. UI ) ?

答:
When we write a software
  specification, under the topic "User
  requirements definition" we have to
  specify the "Functions" and
  "Constraints" only ?

A requirement is a combination of two things...

What the thing does. Functional requirement.

How well it does it. Non Functional requirement or "Constraint"

Does "User Interface" fall into
  "functions" or "constraints" ?

I would say "User Interface" would be category of requirements as you have identified in your last question.

What are the major key areas
  (requirements) a software can be
  broken into (e.g. UI ) ?

It depends on the software. You can group requirements based on parts of the system or you can group them based on use case or the business requirement that the functions are fulfilling.

Of course all of this is secondary to your actual goal which is to determine a clear, unambiguous and testable description of the software system.


答:
Quick answer:  yes, you learn to do both.

I'm one that 'started' way back when doing good ol' C programming, with a little bit of self-taught HTML (hey, it was the 90s), then started with 'back end' programming.  Eventually, I gained enough experience along the way where now seeing people doing text on images, loads of text on pages, poor quality images, etc., etc. drive me crazy. I've gotten a few compliments myself on some basic designs, though I'm still obviously not a professional at web design.  A couple of pointers:

No matter what, do your own website - even if it is just customizing Wordpress templates.  Tinkering with a personal site is a great bit of experience.  Make it your own little playground.

Expect your first 'designs' (or even the designs you continue to create) to suck.  A lot.  The thing about being around the web is that you've been exposed to a lot of really neat sites that makes the ones that you create look horrible.  This is good - it's a sign that you might actually have taste.  Eventually you'll learn the tech and tips and tricks to make your product start to look pretty decent to you.

Ira Glass (the NPR guy) had a great take on this problem:

“Nobody tells this to people who are beginners, I wish someone told
  me. All of us who do creative work, we get into it because we have
  good taste. But there is this gap. For the first couple years you make
  stuff, it’s just not that good. It’s trying to be good, it has
  potential, but it’s not. But your taste, the thing that got you into
  the game, is still killer. And your taste is why your work disappoints
  you. A lot of people never get past this phase, they quit. Most people
  I know who do interesting, creative work went through years of this.
  We know our work doesn’t have this special thing that we want it to
  have. We all go through this. And if you are just starting out or you
  are still in this phase, you gotta know its normal and the most
  important thing you can do is do a lot of work. Put yourself on a
  deadline so that every week you will finish one story. It is only by
  going through a volume of work that you will close that gap, and your
  work will be as good as your ambitions. And I took longer to figure
  out how to do this than anyone I’ve ever met. It’s gonna take awhile.
  It’s normal to take awhile. You’ve just gotta fight your way through.”

http://www.goodreads.com/author/quotes/113989.Ira_Glass

The same actually applies for programming in some aspects - sometimes it's just not very good, the hardest part is to keep opening the editor day after day - just keep at it in the hopes that your talent (and experience) will eventually match your taste.

Title: How do web-developers do web-design when freelancing?

Body:
So I got my first job recently as junior web-developer.

My company creates small/medium sites for wide variety of customers: autobusiness companies, weddign agencies, some sauna websites, etcetc, hope you get my point. They don't do big serious stuff like bank systems or really big systems, it's mostly small/medium-sized websites for startups/medium sized business.
My main skills are PHP/MySQL, I also know HTML and a bit of CSS/JS/AJAX.
I know that good web-developer must know some backend language (like PHP/Ruby/Python) AND HTML+CSS+JS+AJAX+JQuery combo.

However, I was always wondering. In my company we have web-designer. In other serious organisations I often see the same stuff: web-developers who create business-logic and web-designers, who create design.
As far as I know, after designers paint design of website they give it to developers either in PSD or sliced way, and developers put it together with logic, but design is NOT created by developers. Such separation seems very good for full-time job, but I am concerned with question how do freelance web-developers do websites?
Do most of them just pay freelance designers to create design for them? Or do some people do both?

Reason why I ask - I plan to start some freelancing in my free time after I get good at web-development. But I don't want to create websites with great business-logic but poor design. Neither I want to let someone else create a design for me. I like web-development very much and I am doing quite good, I like design aswell, even though I am a bit lost how to study it and get better at it.
But I am scared that going in both directions won't let me become expert, it seems like two totally different jobs and getting really good in both seems very hard. But I really want to do both. What should I do? Thank you!


答:
There's really never a project that is completely unworkable if it can be designed or envisioned - the unworkable part may come from the choices that are made to implement the product.

For example, it's perfectly okay to say that it is unworkable to develop an iPad application with only a Windows machine and a copy of Visual Studio.  Other than situations like that, it's better to think about things in terms of cost and opportunity.

Think about the solutions to the potential problem, and be open with your product manager about what is difficult, why, how you might solve it, and how you might need help in solving it.  (Sometimes it can be a matter of not knowing a particular technology well enough to know what it's really good at and what it isn't good at.)


答:
I don't know if I'd use the phrase "serious web apps" to describe the typical enterprise app - most consumer facing apps/sites these days are built to be more scalable than the typical multi-tier "enterprise" app.  Different apps, different challenges.

That being said, an enterprise application is usually less focused around messaging/content (etc.) and more focused on the ability to view and/or manipulate data.  It's usually the ability to view and manipulate data that good enterprise design allows users to do.

A good example (and possibly a solution for you) in terms of enterprise design is Oracle's "Application Development Framework" (ADF), specifically the "ADF Rich Faces" components.

(There's a version you can freely experiment with here:  http://www.oracle.com/technetwork/developer-tools/adf/overview/adfessentials-1719844.html  You might also want to grab a copy of JDeveloper for simple exploration.)

Compare the design of the data-driven components (tables, components designed for data display) versus the design of other JSF libraries like PrimeFaces; while both can be comparable, the ADF Rich Faces libraries are designed with mass data views and manipulations in mind.


问:
How do developers find the time to stay on top of latest technologies?
I was a freelance web developer until circa 2004 when I started going down the management route but have decided to try to get back into development again (specifically JavaScript and HTML5 web/mobile web apps) and I really get the impression to be truly good at these and similar fast moving technologies a constant amount of time is required to be set aside to invest in getting better at existing skills in addition to learning new skills.

I understand right now since I am getting back into things there is a pretty steep learning curve, but seeing how good many guys are out there - the only way I see of getting up there is putting in a serious amount of time.

For those working as fulltime developers, what I am trying to understand is this - on most days, how much time in the office is spent actually grinding out code compared to learning/research. I could easily spend 2-4 hours daily getting on top of the best ways to go about doing things.

Do most good developers who are employed full time invest significant hours outside of work sharpening their skills?

Or maybe I'm looking at all of this completely wrong?

答:
Just ask yourself what drives you not what others are driven by?

For example - Do I want to create mobile apps or continue working on web apps since I have past experience on it or do I want to do some technical consulting? I think once you know what you want to be doing for the next few months or years, you'll know exactly where to begin and what you need to be learning.

This isn't a question what just IT people ask.


问:
Strategy for versioning on a public repo
Suppose I'm developing a (javascript) library which is hosted on a public repo (e.g. github). My aim in terms of how version numbers are assigned and incremented is to follow the guidelines of semantic versioning.

Now, there's a number of files in my project which compose the actual lib and a number of files that 'support it', the latter being docs, a test suite, etc. My perspective this far has been that version numbers should only apply to the actual lib - not the project as a whole - since the lib alone is 'the unit' that defines the public API.

However I'm not satisfied with this approach as, for example, a fix in the test suite constitutes an 'improvement' in my project, which will not be reflected in the version number (or the docs which contain a reference to it). On a more practical level, various tools, such as package managers, may (understandably) not play along with this strategy. For example, when trying to publish a change which is not reflected in the version number, npm publish fails with the suggestion "Bump the 'version' field set the --force flag, or npm unpublish".

Am I doing it wrong?

答:
My perspective this far has been that version numbers should only apply to the actual lib - not the project as a whole - since the lib alone is 'the unit' that defines the public API.

But to the World your final Product is something more: A combination of

Code (the lib per se)

Doc

Test-sute

If you change anything in this triad you get new version of product which can't (logically) share the old one's ID.

If you prefer to strictly follow SV-rules and use only 3-octets, you can

Use p.7 for all non-library changes

Patch version Z (x.y.Z | x > 0) MUST be incremented if only backwards compatible bug fixes are introduced

docs|tests changes may be considered as bug-fixes and they are backwards compatible. Thus, from version 1.1.0, docs changesets may increment it towards 1.1.9999, but not 1.2.0

Operate with p.11 for non-core code changes

A build version MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch version or pre-release version.

and from 1.1.0, the version will be incremented towards 1.1.0+9999.


问:
Should I encrypt data in database?
I have a client, for which I'm going to do an Web application about patient care, managing patients, consults, history, calendars, everything about that basically.

The problem is that this is sensitive data, patient history and such.

The client insists on encrypting the data at the database level, but I think this is going to deteriorate the performance of the web app. ( But maybe I shouldn't be worried about this )

I've read the laws about data protection on health issues ( Portugal ), but isn't very specific about this ( I just questioned them about this, I'm waiting for their response ).

I've read the following link, but my question is different, should I encrypt the data in the database, or not.

One problem that I foresee in encrypting data, is that I'm going to need a key, this could be the user password, but we all know how user passwords are ( 12345 etc etc ), and generating a key I would have to store it somewhere, this means that the programmer, dba, whatever could have access to it, any thoughts on this?

Even adding an random salt to the user password isn't going to solve the problem since I can always access it, and therefore decrypt the data.

答:
Ignoring for a moment what the client is asking for, and whatever the laws are...

No, you probably should not encrypt the data.  If you do, you will not be able to easily search it.  For example, how would you search for a last name like 'Smith%' if every name entry is encrypted?  How would you graph a patient's blood pressure over time if you cant  select .... from.... where patient_id = N?

You should, obviously, make sure the server is properly secured, the network connection is secured, and that the user interface is properly secured (including session timeouts so users cant walk away leaving access to anyone who uses their computer).  You also may want to encrypt database backups.  And physically secure the room the server is located in.  But I would not encrypt the live data.

Clarification: This is assuming what the OP was asking about is actually encrypting the data in the database.  Not the file system the database resides on.

Title: Should I encrypt data in database?


问:
Client-Server connection response timeout issues
User creates a folder in client and in the client-side code I hit an API to the server to make this persistent for that user. But in some cases, my server is so busy that the request timesout. The server has executed my request but timedout before sending a response back to client. The timeout set is 10 seconds in client. At this point the client thinks that server has not executed its request (of creating a folder) and ends up sending it again. Now I have 2 folders on the server but the user has created only 1 folder in the client. How to prevent this?

One of the ways to solve this is to use a unique ID with each new request. So the ID acts as a distinguisher between old and new requests from client. But this leads to storing these IDs on my server and do a lookup for each API call which I want to avoid.

Other way is to increase the timeout duration. But I dont want to change this from 10 seconds.

Something tells me that there are better solutions.

I have posted this question in stackoverflow but I think its better suited here.

UPDATE: I will make my problem even more explicit. The client is a webbrowser and the server is running nginx+django+mysql (standard stack). The user creates a folder in webbrowser. As a result I need to hit a server API. The API call responds back, thereby client knows API call was success. This is normal scenario. Sometimes though, server successfully completes the API request but the client-side (webbrowser) connection timesout before server can respond back. The client has no clue at this point. The user thinks the request was a fail & clicks again. This time it was a success but when the UI refreshes he sees 2 folders. I want to remedy this situation.

答: Sounds like you are working in a distributed network environment, have transactions that one node expects another node to carry out reliably and you are specifically dealing/experiencing with timeouts.  This is a relatively generic problem that many people had to solve at one time or another.  Take a look at the three-phase commit protocol and see if that would be suitable for your application.  Or possibly two-phase commit is good enough for what you are doing.</p>


答:
Yes, you should encrypt the database.

Basic encryption for stored data ("data at rest") is a Generally Accepted Security Principle, and is probably mandated by law if your country has laws that protect personal or health information.

We're using SQL Server 2008, so we use Microsoft's TDE; there may be some third-party solution for MySQL or perhaps just a general volume encryption approach (such as TrueCrypt) would work (although I'd want to have something that has been certified for use with a database).

If done properly, the performance hit should be small.

By the way, the link you mentioned (regarding the separation of sensitive information) is something that you should consider on top of the basic database encryption.

EDIT: The encryption mentioned above would encrypt the volume.  If someone were to steal the hard drive, they'd find the data encrypted.  However, if someone were to run queries on the database, they'd see the unencrypted data (that's why I mentioned the separation of information, even though the OP didn't want to discuss this).

Note that this recommendation is meant as the minimum that you should do.  If you want legal advice, then of course you'll need to look elsewhere.  If you want a more thorough discussion of writing secure code, I would starting with the book Writing Secure Code.

Title: Should I encrypt data in database?

Body:
I have a client, for which I'm going to do an Web application about patient care, managing patients, consults, history, calendars, everything about that basically.

The problem is that this is sensitive data, patient history and such.

The client insists on encrypting the data at the database level, but I think this is going to deteriorate the performance of the web app. ( But maybe I shouldn't be worried about this )

I've read the laws about data protection on health issues ( Portugal ), but isn't very specific about this ( I just questioned them about this, I'm waiting for their response ).

I've read the following link, but my question is different, should I encrypt the data in the database, or not.

One problem that I foresee in encrypting data, is that I'm going to need a key, this could be the user password, but we all know how user passwords are ( 12345 etc etc ), and generating a key I would have to store it somewhere, this means that the programmer, dba, whatever could have access to it, any thoughts on this?

Even adding an random salt to the user password isn't going to solve the problem since I can always access it, and therefore decrypt the data.


问:
How do you keep track of a requirements document on an agile team?
I understand that User Stories dominate the agile world, but how are these artifacts stored, so that new developers who join the team can come up to speed with the requirements?

What if the User Story changes later, how is it updated and kept as an artifact? I've seen many teams just open a new ticket/feature request/bug report instead of keeping track of the original story.

答:
[update #1] As @MatthewFlynn pointed out, his experience with agile as well as many others (including my own) is very different from the answer that I'm providing here.  The answer here is based on my observations of what did and didn't work on my own team in the past, combined with many books and blogs that I read on the subject...

most of the drive towards agile development is specifically targeted at eliminating requirements documents.

Agile tries to do away with most documentation and I agree with their ideas but of all documents, requirements have by far the largest bull's eye painted on them. The reason for that (IMO) is that requirements docs are furthest away from actual working code and of all the documents, that makes them

least accurate

hardest to maintain

least useful

To guide the team as to what should be developed next, Agile replaces requirements documents with a backlog of stories that identify what you should working on next and highest priority items with largest bang for the buck (both present and future buck) are typically placed first in that list.

However, a backlog should not be confused with a requirements doc:

In a backlog, only N number of stories should have detail filled in. The further out a story is, the less detail you should put in it (i.e. don't waste your time trying to define something that will not happen for half a year).

Beyond a certain point, "requirements" items should not even be placed in a backlog if they are out more than 2 release cycles (aka potential shippable increments (PSI) ).  Even if you know the requirement is important and must get done at some point, resist temptation of adding it to the backlog.  If it is really important, someone will remember it in the next release. If no one remembers it, most likely it's because it wasn't that important after all.

Once a story is completed, that story gets removed from the backlog and is CHUCKED(1). Again, stories are not requirements. They ONLY tell the team what to work on next; they are not for historical record.

However, in proper agile process, whenever you deliver work, part of that delivery should be unit/integration/acceptance tests. These tests are very valuable because they have many purposes. I won't get into the full list, but one of those purposes is documentation for your current production software.

A test documents how the software should behave given a certain set of inputs and preconditions. It also documents how to use public (and internal) APIs of your code.  It also serves as a safety net so that when a new developer enters a team and inadvertently breaks something, that error will be caught as soon as it is checked in.

Agile process obviously promotes taking advantage of automated unit tests as much as possible but we all know that not every single thing can be automated. Your software suite will always have a set of tests that have to be executed manually. However, a) your developers should be actively working on automating as much as feasible and b) manual set of tests should be regularly executed by your QA team so that any break in functionality is discovered as soon as possible.

(1) - Since I got several responses for the "chucked" part. In 5 years since moving to agile, my team never threw away a single story, even 30% of those that got scheduled, then deferred and then forgotten. My boss wanted to keep them "for reference" and yet no one has ever looked at any of those stories.

People are generally attached to their data and I know it's hard to fathom throwing something out once you already have it, but keeping inventory (whether physical or electornic) on hand isn't free and the more I think about it, the more I agree with the  "chucking".  This is from "Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise" (p.190) - "User stories can be safely thrown away after implementation. That keeps them light-weight, keeps them team friendly, and fosters negotiation, but acceptance tests persist for the life of the application..."


问:
Decision for Unchecked Exceptions in Scala
As a java programmer, I have always been critical of Unchecked Exceptions. Mostly programmers use it as an en-route to coding easiness only to create trouble later. Also the programs (though untidy) with checked exceptions are much robust compared to unchecked counterparts.

Surprisingly in Scala, there is nothing called Checked Exceptions. All the Java checked and unchecked are unchecked in Scala.

What is the motivation behind this decision? For me it opens wide range of problems when using any external code. And if by chance the documentation is poor, it results in KILL.

答:
If you want to gain efficiency, you have to give up.. precision/control <-- I need a better word for that.

Scala is located towards the top as far as abstraction goes.  If one of Scala's goals is getting rid of annoying boilerplate code, then an obvious place to look at is Java's exception handling.  If you want to write fast code in java, just keep throwing your checked exceptions up until they hit main() and effectively become unchecked.

I don't know if I'm getting at exactly what you're asking but this is the most obvious reason in my opinion.

Well, I did a little looking and someone has written about the tragedy of check exceptions.


答:
Checked exceptions are mostly considered a failure. Note that no languages created after Java adopted them. See http://www.artima.com/intv/handcuffs2.html, http://googletesting.blogspot.ru/2009/09/checked-exceptions-i-love-you-but-you.html, http://www.mindview.net/Etc/Discussions/CheckedExceptions, etc.

In particular, they are uncomposable (except by reverting to throws Exception).

In Scala you have a better option: using algebraic types for return values such as Option[T], Either[Exception, T], your own type when you want the user to handle specific cases (e.g. instead of

```<code>def foo: Int // throws FileNotFoundException, IllegalStateException
</code>```

you have

```<code>sealed trait FooResult
case class Success(value: Int) extends FooResult
case class FileNotFound(file: File) extends FooResult
case object IllegalState extends FooResult

def foo: FooResult
</code>```

and the consumer now is required to handle all results)

For dealing with external code which does throw exceptions, you have scala.util.control.exception or scala.util.Try (starting with Scala 2.10).

Title: Decision for Unchecked Exceptions in Scala

Body:
As a java programmer, I have always been critical of Unchecked Exceptions. Mostly programmers use it as an en-route to coding easiness only to create trouble later. Also the programs (though untidy) with checked exceptions are much robust compared to unchecked counterparts.

Surprisingly in Scala, there is nothing called Checked Exceptions. All the Java checked and unchecked are unchecked in Scala.

What is the motivation behind this decision? For me it opens wide range of problems when using any external code. And if by chance the documentation is poor, it results in KILL.

Body:
Checked exceptions are mostly considered a failure. Note that no languages created after Java adopted them. See http://www.artima.com/intv/handcuffs2.html, http://googletesting.blogspot.ru/2009/09/checked-exceptions-i-love-you-but-you.html, http://www.mindview.net/Etc/Discussions/CheckedExceptions, etc.

In particular, they are uncomposable (except by reverting to throws Exception).

In Scala you have a better option: using algebraic types for return values such as Option[T], Either[Exception, T], your own type when you want the user to handle specific cases (e.g. instead of

def foo: Int // throws FileNotFoundException, IllegalStateException

you have

sealed trait FooResult
case class Success(value: Int) extends FooResult
case class FileNotFound(file: File) extends FooResult
case object IllegalState extends FooResult

def foo: FooResult

and the consumer now is required to handle all results)

For dealing with external code which does throw exceptions, you have scala.util.control.exception or scala.util.Try (starting with Scala 2.10).


答:
You can use freemind to gather the list of features. How it is done, take a look into this tutorial (somewhere in the middle).

When you have list of features, you go with writing user stories. This can be done by using a simple text file, or word document, or something as complex as a agile management tool.

When you finish with user stories, they are prioritized. Later, from the user stories, people produce tasks, people take tasks and implement it in code.

All this can be seen how a c# project is managed from the start in the autumn of agile video cast series.

Title: How do you keep track of a requirements document on an agile team?

Body:
I understand that User Stories dominate the agile world, but how are these artifacts stored, so that new developers who join the team can come up to speed with the requirements?

What if the User Story changes later, how is it updated and kept as an artifact? I've seen many teams just open a new ticket/feature request/bug report instead of keeping track of the original story.


问:
When do I bite the bullet and hire a developer?
I have an awesome URL, I've had it since the mid 90's, and up until around 2002 I was having an awesome time writing music reviews and features into static pages and adding their URL to static index pages and everything was just great.

Then things got complicated really fast, and for one reason or another I handed the site over to a very talented friend who turned it into a db based site run by cf.

Now, here I am around twelve years later putting it all back together using mysql, php and css on a need-to-learn basis and even though I'm sooo close to where I want it to be I realize, erm, it should have been written using mysqli etc etc ad infinitum.

So I'm wondering at what point do I just give in and hire a developer to take over, how much does it even cost, and how do I know I'm working with someone who is better than I am?

答:
You should probably hire one now.

Its a site you've had for a while, so you're obviously committed to it.  You're trying to do it yourself, but it sounds like you're overwhelmed.  Get someone to set it up for you the way you want it set up so that you can focus on improving the site content & advertising (if you do that).

My recommendation would be to have a professional look over your site, and perhaps recommend & install one of the many Content Management Systems (CMS).  This would let you add/remove pages & features without programming (though the option would still exist to customize the code as well).  This way once set up, you wont need continual programming resources to do simple things like adding a review.

As for rates and such, thats really a regional thing.  But in general you get what you pay for.  Look around at local sites and if you find one you like, often there's a link to the developer/developing company at the bottom of the page.


问:
Strategy for clients to retrieve real-time log from HTTP server
I have an HTTP Server Service application which has its own logging mechanism. It's written in Delphi. I would like to provide a way for multiple clients to connect to this service and get a real-time update of the log. The log in the service moves rather fast, there's a lot of things to log. There may be up to 50 messages within 1 second at times. The existing log which is already implemented is not saved, it's only kept in the memory of the server service - where I will need to distribute it to any client which needs it. Once all clients have a log message, it should be deleted.

I intend to use HTTP to "ask" the server for the log, and respond with an XML packet. The connections are not keep-alive. The only problem is, the server should only send the client those log records which it needs, not everything. I have no way of the server pushing the log to the clients in real-time, so each client needs to repeatedly ask the server for the latest log records. This HTTP Server is very lightweight, and there is no session management. There isn't even any type of authentication.

The only way I see is for a client to register its self on the server, and whenever a log is issued on the server, it creates a copy of the log for each client, where each client has a log queue (string list). However, suppose there are 100 clients connected and expecting to receive this log. That means the server must create 100 copies of each log, add this log to the end of each client log queue, and wait for the client to request it. At that point, when the server replies with the XML log, it should flush (delete) whatever's in the queue. I'm worried however that this could cause memory issues. Each client log queue might get 100 log messages before the client requests the latest logs.

How should I go about doing this in the fastest way possible without hindering the performance of the server? I'm trying to avoid having to create a copy of each log for each client.

答:
Why not give each client a cookie that has the index of the last line sent to the client.  When the client requests a new portion of the log, simply read the index form the cookie and return something like

```<code>$tail -n <current_number_of_lines_in_the_log - index_from_cookie> the_log.txt
</code>```

to the client while at the same time

```<code>Set-cookie: lines=current_numer_of_lines_in_the_log
</code>```

in the response header.

Body:
Why not give each client a cookie that has the index of the last line sent to the client.  When the client requests a new portion of the log, simply read the index form the cookie and return something like

$tail -n <current_number_of_lines_in_the_log - index_from_cookie> the_log.txt

to the client while at the same time

Set-cookie: lines=current_numer_of_lines_in_the_log

in the response header.


答:
Depends how much your own knowledge is worth to you. You can learn this all yourself. It will cost you time but it will save you developer costs. Which about breaks it even, plus-minus how much your time is worth.

The added value is the knowledge you gain. You can use it for other things, it can very well start earning you money. But if you have a solid, good job that pays well, and no interest in programming, that knowledge will be useless in the long run.

Or you can hire a smart freelancer part-time as a co-programmer, a guide and someone to teach you all you need as you both write it together. (just make sure to make these terms and goals clear from moment one. I can tell for sure a guy hired to write a webpage will quit the moment he discovers he actually signed up for a teaching gig.)

Title: When do I bite the bullet and hire a developer?

Body:
I have an awesome URL, I've had it since the mid 90's, and up until around 2002 I was having an awesome time writing music reviews and features into static pages and adding their URL to static index pages and everything was just great.

Then things got complicated really fast, and for one reason or another I handed the site over to a very talented friend who turned it into a db based site run by cf.

Now, here I am around twelve years later putting it all back together using mysql, php and css on a need-to-learn basis and even though I'm sooo close to where I want it to be I realize, erm, it should have been written using mysqli etc etc ad infinitum.

So I'm wondering at what point do I just give in and hire a developer to take over, how much does it even cost, and how do I know I'm working with someone who is better than I am?


答: Look into Wordpress, it's very easy to setup and manage, if you need something more complicated than that and your struggling to do things yourself you should look into the services of a qualified developer.</p>


问:
Tips/Process for web-development using Django in a small team
We're developing a web app using Django and we're a small team of 3-4 programmers — some doing the UI stuff and some doing the Backend stuff. I'd love some tips and suggestions from the people here. This is our current setup:

We are using Git as as our SCM tool and following this branching model.

We are following the PEP8 for your style guide.

Agile is our software development methodology and we're using Jira for that.

We are using the Confluence plugin for Jira for documentation and I'm going to be writing a script that also dumps the PyDocs into Confluence.

We are using virtualenv for sandboxing

We are using zc.buildout for building

This is whatever I can think of off the top of my head. Any other suggestions/tips would be welcome. I feel that we have a pretty good set up but I'm also confident that we could do more.

答:
This presentation shows a good collection of tools/practices for python / django team:

Automated tests (Django built-in)

Continuous integration (Jenkins, django-jenkins)

Measure all the things (sentry, statsd + graphite, new relic)

Scripted database (South)

Scripted deployment (fabric)

Configuration management (puppet/chef)

Virtualised development (Vagrant)

Test separation (test decorator, customise test runner)

To this I would add:

Use pip for your dependencies.

Have a gatekeeper for your main branch so no code can be merge if it doesn't passes all the tests.

You want to run your tests as often as possible while developing. So make them as fast as possible (i.e. use sqlite for tests) or at least split them between slow/fast.


答:
When to give it to a professional

To be honest, every average chimpanzee can produce a web application that gives you 70 - 80% of the required functionalities. It is the remaining 20 to 30% that is hard to develop. When you arrive at 70% and feel stuck, it is time to get help. If you already have this feel, find a professional and ask his/her help.

How much would it cost

If you just hire a lone programmer and ask him to help you structuring the application and writing some code (that is: you will keep on writing code and administering the server) you can expect to spend just few thousands of US$ globally (most likely in a few months). If you give the responsability to design, code and style a new web site to a team of professional, you can expect to spend few tens of thousands dollars. Keep in mind that, in this second case, you will get a graphic designer that will take care of the site style and a few other professionals that will give you much more than just code.

How long it would take

Not less than 2 or 3 months, in any case. Most likely 6 - 8 months if you work with a single programmer. Most likely 4 - 8 months if you hire a team of professionals (what it is usually called a "web agency" in Italy and a "web software house" in other countries).

Best way to run alone or with a programming partner

Please, take into account Ruby-on-Rails and Refinery CMS. Take them into account even if they would require you to start from scratch. It is easier to create from scratch a document-oriented web site (like yours) with a Ruby-based CMS like Refinery or Radiant than "improving" any existing home-grown, PHP based one. Have a look here for a choice of Ruby CMSs: https://www.ruby-toolbox.com/categories/content_management_systems . If possible, do not write code. Just configure and tweak an existing CMS. Refinery is fantastic for this ( http://refinerycms.com/ ).

Please, stay away from "naked" PHP, MySQL and LAMPS platforms. Do not try to home-grew your web site in this way. There are much better (open source, free) solutions on the market nowadays. Stay away from Java if you work alone or with a single programming partner. Java (Spring, Wicket, Tapestry, etc.) can be heavy to manage, in particular for what regards the server administration and the application deployment. Java is mainly intended to be used and managed by a small-medium size team of professionals.

Best way to go with a team

Just tell them what you want (not how you want it done) and wait.

Bets way to choose a partner programmer

Please, look for a seasoned web application programmer. Look for someone who already build at least 4 or 5 web applications in the last 4 or 5 years using the best tools on the market (Ruby-on-Rails, Django, Pylons, etc.). Look for someone who can demostrate you he/she still read books and magazines and he/she still lives on the web, reading articles and partecipating in forums. You need an intelligent, well-informed, web-addicted guy, anything else. Look for someone who speak his mind and can demonstrate he/she can suggest you the best road to follow. Writing code is quite easy nowadays. The hard part is deciding which code to write.

Stay away from people who use old-school tools (like Perl, naked PHP, old LAMP solutions and the like) or who use their own, home-grown tools. You need a modern, standard, elegant tool that other programmers can maintain for you in the near future.

Best way to choose a team

Look for a small, web-oriented software house that already did something similar to what you want to get. Look for a team of 3 to 5 people including at least:

One proven, good programmer (Ruby, if you will follow my suggestion)

A graphical designer (someone with your same aesthetical sense)

A good UI specialist (HTML, Javascript, CSS)

A good project manager. Someone with good manners and a clear vision.

Best way to choose a partner programmer

Look for someone on the Net and talk with him/her. Ask to see his/her portfolio. Ask for his/her opinions and suggestions. You will quickly understand who is the guy for you. Unfortunately, in this case you have to look for someone who can physically work with you. It must be a guy who can reach your office every morning in some way.

Better a team or a single programming partner?

A team. Working with modern, best-of-the-breed tools would probably require you to start learning almost from scratch. You would be better entrust a small team with the task to develop your site.


问:
Collaboration using github and testing the code
The procedure in my team is that we all commit our code to the same development branch. We have a test server that runs updated code from this branch so that we can test our code on the servers. This test server is exposed on the internet so that we can test callbacks from third-party services like sendgrid. (where you specify a url for sendgrid to update you on the status of emails sent out)

The problem is that if we want to merge the development branch to the master branch in order to publish new features to our production servers, some features that may not have been ready will be applied to the production servers.

So we're considering having each developer work on a feature/topic branch where each of them work on their own features and when it's ready, merge it into the development branch for testing, and then into the master branch.

However, because our test server only pulls changes from the development branch, the developers are unable to test their features. While this is not a huge issue as they can test it on their local machine, the only problem I foresee is if we want to test callbacks from third-party services using our test server during the course of developing a feature.

How can we handle this problem?

Note: We're not advanced git users. We use the Github app for MacOSX and Windows to commit our work.

Setup: It is a PHP project. We do not have any form of CI setup and do not have the knowledge to do so. Eventually we want to use Jenkins but right now we're just focusing on getting our minimum viable product out.

答:
One solution is to let the test server also build the branches when commits are made. Jenkins for example can be told to monitor some or all branches in a repo, and whenever it detects a commit to a branch, it will build that branch. See e.g. Git, Feature Branches, and Jenkins – or how I learned to stop worrying about broken builds

You could have some naming convention for "scratch" branches that are not supposed to be tested, and only let Jenkins build the "real" feature branches.

Another common solution is to have some kind of integration branch between your feature branches and master (or have branches "master" and "release"). Then everyone merges to master once a feature is ready, everything is tested on master, but merges to "release" only happen once a feature is sufficiently tested to be deployed. See e.g. A successful Git branching model for one popular approach.


答:
Before deciding on such security matters, you should assess the threat model. Without an idea what you're defending against, any measures you take are likely to be of little value.

Now, there are a few things you could worry about in this context:

Attackers gaining physical access to your data (e.g., breaking into the datacenter, stealing backup tapes, etc.)

Attackers gaining read access to your raw database

Attackers compromising your application, e.g. through SQL injection, buffer overruns, etc.

For the first scenario, storing the database and all backups on encrypted volumes should work, provided the server is headless - stealing the server or the tapes would then require breaking the disk-level encryption.

For the second scenario, encrypting database data does help, but only if you don't store the required keys or passphrases anywhere.

In the third scenario, everything depends on the context in which the attack happens: if it is, for example, an XSS or CSRF attack, then an attacker can do anything the legitimate user can, and encrypting your data doesn't help at all.

You threat model, thus, is an attacker who gains read access to the raw database, either by finding the login credentials and managing to log into the database server from outside, or by gaining root access to the database server. A typical path is to first gain shell access on the web server; from there, an attacker can then read the access credentials from the configuration file and connect to the database.

An additional consideration is where you keep the keys and passphrases, especially if you're using a platform with a stateless execution model such as PHP. Ideally, you have the customer type their passphrase when required, and keep it only in memory, or even better, do the decryption client-side (but this is not often feasible). On a stateless platform, state is usually carried along using sessions, memcache, databases, or flat files; but all these are far more vulnerable than keeping state in a stateful web application's own memory. Avoiding this is a chicken-and-egg problem, because if you encrypt the state before persisting it, you just created another secret that you have to remember safely. Remembering the passphrase on the client and sending it along with each request that needs it may be the least horrible solution then; the client is still vulnerable if you have any XSS leaks, and you need to take the usual precautions (serve only https, set all cookies to httponly and secure, etc.), but those are issues you have anyway.

Title: Should I encrypt data in database?


答: I've never met a client who could afford a developer after referring to the decision to hire one as to "bite the bullet".</p>