问:
Cross border data obfuscation and how to deal with it?
I am currently facing a situation where I will need to obfuscate data for legal reasons (these are countrywide rules and not specific to me)

I will still need to work with teams outside this area who will still need to access applications/databases for testing/deployment and so on, however the data must be fully obfuscated.

What are your experiences of this scenario?

e.g. Who owns what stage of the process, what access do the various parties have, how strong is the encryption, how do updates occur

答:
First thing, as you say, this isn't encryption or possibly even obfuscation.  There needs to be no possible way to retrieve the data  - the legal restriction isn't on exporting the data securely, it's on exporting it at all.

Also worth noting that that will include someone outside the country accessing a database within the country (exporting a single record is still exporting data) so they can have no access to your systems via any means.

The issue with anonymising the data is it's very hard to do.  You've got to scrub every personally identifiable piece of information (don't just scramble them) including names, addresses, e-mails, dates of birth, social security numbers.  Anything that might be usable even if only as part of a composite needs to be removed.  You also need to check comments fields, notes fields, audit fields and anything else for these same values.

When you're scrubbing the data, don't swap things.  Replace each one from a random list of possible values.  If you swap them the data is still there and there are risks they'll be swap and swapped back or in some other way remain readable.  Remember, the restriction is on exporting the data, not making it hard to read.

Beyond this you need to understand what might be implied by data.  Say it's a film rental database and the guys outside the country know from a support mail that someone has rented one particular a few times.  That's a straight forward search which will likely give you a very short list.  Cross reference it with another few things (it's a guy so he probably wouldn't rent that and so on) and without his name being anywhere there you've got his whole history.

Read this: http://arstechnica.com/tech-policy/news/2009/09/your-secrets-live-online-in-databases-of-ruin.ars for a better example.

You imply that the reasons the people outside the country need the database are for technical reasons so the actual data isn't important to them - just that they have something they can run up, check it works and so on.

If that's the case then for my money the best way is don't bother.  Send them a test database instead with no real data in.  Need more data than your test database has?  Write scripts to generate it.

The scripts then need to be maintained along with the rest of the database and application, updated with each new release and so on.

If you have a skeleton database (that is with structure but no data) you can just give the other guys access to the scripts to create databases as they want (which is what I'd recommend).  If you do go with a process of scrubbing the live data you probably need to maintain the scripts for doing it in the same way but have to run the process yourself - I'd personally create a test database with each release.


问:
Methods to identify and resolve memory leaks in the JVM
As the data is migrated via the object level, there has been a slow upwards trend in memory usage. I can see the periodic gc activity, but the memory trend is still going up slowly.  I have currently for my application set the starting and max heap at 14GB in production, however if we might need a large data set, my concern is the we will eventually hit the limit and then the application will slow down (which was what happened).

What are some effective ways and proving strategies to help identify and remedy a potential memory leak in a running Java process?

答:
I used to use BEA's JRockit enter link description here to find out the memory leak points in Java program.

Now BEA has been bought by Oracle,I do not know whether JRockit is live or dead.

ContentLicense: CC BY-SA 3.0


问:
Is it *ever* okay to catch StackOverflowError in Java?
I used to think that it's not, but yesterday I had to do it. It's an application that uses Akka (an actor system implementation for the JVM) to process asynchronous jobs. One of the actors performs some PDF manipulation, and because the library is buggy, it dies with a StackOverflowError every now and then.

The second aspect is that Akka is configured to shutdown its whole actor system if any JVM fatal error (e.g. StackOverflowError) is caught.

The third aspect is that this actor system is embedded inside a web app (for WTF-ish, legacy, reasons), so when the actor system is shut down, the web app is not. The net effect is that on a StackOverflowError our job processing application becomes just an empty web app.

As a quick fix I had to catch the StackOverflowError being thrown, so that the thread pool of the actor system isn't torn down. This lead me to think that maybe it's sometimes okay to catch such errors especially in contexts like this? When there's a thread pool processing arbitrary tasks? Unlike an OutOfMemoryError I can't imagine how a StackOverflowError can leave an application in an inconsistent state. The stack is cleared after such an error, so computation can go on normally. But maybe I'm missing something important.

Also, let it be noted that I'm all for fixing the error in the first place (as a matter of fact I have already fixed an SOE in this same app a few days ago), but I really don't know when this kind of situation might arise.

Why would it be better to restart the JVM process instead of catching the StackOverflowError, mark that job as failed, and continue with my business?

Is there any compelling reason to never catch SOEs? Except "best practices", which is a vague term that tells me nothing.

答:
As a general rule, if it were absolutely never, ever acceptable to do something, and there was agreement about that, the language implementers would not have allowed it. There are almost no such unanimously clear-cut maxims. (Luckily, because that's what keeps us human programmers in jobs!)

It looks very much as if you've found a situation where catching this error is the best option for you: it lets your application work, while all other alternatives don't, and that's what counts in the end. All "best practices" are simply summations of long experiences with many cases that can usually be used in place of a detailed analysis of a specific case to save time; in your case, you've already done the specific analysis and got a different result. Congratulations, you're capable of independent thought!

(That said, surely there are situations where a stack overflow might leave an application inconsistent just like a memory exhaustion. Just imagine that some object is constructed and then initialized with the help of nested internal method calls - if one of them throws, the object may very well be in a state not supposed to be possible, just as if an allocation had failed. But that doesn't mean that your solution couldn't still be the best one.)


问:
How do you dive into large code bases?
What tools and techniques do you use for exploring and learning an unknown code base?

I am thinking of tools like grep, ctags, unit-tests, functional test, class-diagram generators, call graphs, code metrics like sloccount, and so on. I'd be interested in your experiences, the helpers you used or wrote yourself and the size of the code base with which you worked.

I realize that becoming acquainted with a code base is a process that happens over time, and familiarity can mean anything from "I'm able to summarize the code" to "I can refactor and shrink it to 30% of the size". But how to even begin?

答:
There are few small tips I can share.

For an existing product, I start testing them intensively. If pick/get assigned a task, I will focusing on the particular feature more.

The next step would be finding the code where I can break in and start exploring
On the way I will find the dependent modules, libraries, frameworks etc.

Next step would be creating simple class diagram with it's responsibilities (Like CRC cards)

Start making minor changes or take up minor bugs to fix and commit. So we can learn the work flow of the project; not just the code. Often large products will have some sort of book keeping for the sake of authorization and audits (e.g. healthcare projects)

Talk to the people who are already working on the project. Express your ideas, thoughts and in return get their experience and views on working with this project for long time. This is quite important because that also helps you get along with the team well.

Title: How do you dive into large code bases?


问:
Algorithm to pick user
I need an algorithm for picking a user.

Users are identified by letter {A, B, C, ...} and are ranked by number {1, 2, 3, ...}.  Rank is the degree of likelihood to be picked, so a rank 2 user is twice as likely to be picked over a rank 1 user, and rank 4 is four times as likely, etc.

So let's say four users are {A, B, C, D}, with rank {1, 1, 5, 2} respectively.  A user table might store rank:

```<code>   USER   RANK
      A   1
      B   1
      C   5
      D   2
</code>```

How do I pick a user based on rank?

My first idea for an algorithm is to add up all the ranks 1 + 1 + 5 + 2 to obtain a sum of 9.  Then assign subranges from 1 to 9 to each user, where subrange size is user rank.  So A has range [1, 1], B has range [2, 2], C has range [3, 7] (5 possibilities due to rank being 5), and D has range [8, 9].  The table becomes this:

```<code>   USER   RANK  RANGE
      A   1     [1, 1]
      B   1     [2, 2]
      C   5     [3, 7]
      D   2     [8, 9]
</code>```

Then calculate a random value from 1 to 9, and find the user whose range contains that value.  So if the random value is 4, user C is picked because 4 falls within range [3, 7].

This achieves fairness based on rank.  If you repeat picking randomly this way, on the average, users will be chosen fairly, and no user will be starved.

However, it feels sloppy, because to add or remove users, I have to recalculate all the ranges.  Here is removing user C (recalculate range of user D):

```<code>   USER   RANK  RANGE
      A   1     [1, 1]
      B   1     [2, 2]
      D   2     [3, 4]
</code>```

And here is changing user A to have rank 2 (recalculate range of B and D):

```<code>   USER   RANK  RANGE
      A   2     [1, 2]
      B   1     [3, 3]
      D   2     [4, 5]
</code>```

What is the best algorithm to pick a user?  Surely recalculating ranges for every user add, delete, or update is suboptimal.  Is there a well-known function or algorithm that can help?

UPDATE: @rwong got me thinking about grouping people by rank, then picking rank instead of picking user.  With one person in each rank, you turn the random-shaped ranges into a very linearly-growing ranges (one dot is one chance to be picked):

```<code>Rank 1:  *
Rank 2:  * *
Rank 3:  * * *
Rank 4:  * * * *
Rank 5:  * * * * *
</code>```

Works great for exactly one person per rank, but it falls apart when you have lots of people at rank 1, where their chances should increase by the number of people within that rank, possibly  overtaking the chances of higher ranks.

Now we're back to an irregular shape, and must pick from it.  A friend pointed me to Rejection sampling.  This seems to be a great solution where you pick a random x and y coordinate, see if it falls within the shape, and if so, we now picked rank.  Then pick a random person within that rank.

To show this solution, here is the shape with multiple people inside each rank.  For rank 2, there are 5 people, so that's 2 x 5 = 10 chances:

```<code>User count   Rank   Chances(graph)       Chances(number)
5            1      * * * * *            5
5            2      * * * * * * * * * *  10
1            3      * * *                3
0            4                           0
1            5      * * * * *            5
</code>```

The algorithm is to pick a random row (like picking an x coordinate), and pick a random value from 1 to max(chances) (like picking a y coordinate).  See if that is less than or equal to the number of chances at that rank.  If not repeat until you have a valid rank and chance.  This takes care of distribution probability.  Now that you have chosen a valid rank, at that point you simply pick a random person within that rank.  Done!

The beauty is, to add or remove someone, you increment or decrement the user count column, and assign a new Changes(number).  Two small very simple updates.  The memory complexity and time complexity of add/remove/update user and also complexity to pick a user is nothing, like O(1).

We grouped user by rank, and captured user count in each rank, then calculate chances for that rank, and used a simple looping algorithm to pick random rows and chance values until we get a hit, and finally pick a random person at that rank.

答:
You don't have to pre-calculate the ranges if you know the rank.

Add up the total in ranks

create your random number between 1 and that sum

To select the user, start a count at zero, loop through the users to create a running total of ranks, and when that total equals or exceeds your random number, thats your user.

Title: Algorithm to pick user


问:
Is a well written documentation a good enough reason for learning a programming language?
I am learning Python currently which wasn't part of my college curriculum. I was asked in an interview why I chose Python and I replied that it is easy to learn and the documentation is very well written. The interviewer didn't reply whether it was a good enough reason. He looked convinced but I cannot be sure.

Is a well written documentation along with ease of learning a good enough reason for choosing a scripting language? Or should I have elaborated more about the availability of Python libraries and bigger user base of Python?

Just a note. Python wasn't required for the job. The company worked on Ruby-on-rails. Python was in my resume and I think the interviewer just wanted to know what considerations I made as a fresher while choosing a programming language.

答:
No, well-written documentation is not a good enough reason for learning a language.

The interviewer asked this question for that you tell him some  goodness of python.You should compare python and ruby.


问:
How do I find a qualified web designer in my area?
I just sent out emails to five local web design companies to my area asking to take drawings to HTML/CSS/jQuery. None of the ones who accepted the deal seem suitable to myself. Others rejected the offer because they wanted to 'provide an end-to-end solution' or are 'booked till June'.

The local companies did not seem suitable to myself because my review process is this:
goto their website, do a view-source. I'll see really weird things (contact us forms that go nowhere), really old things (mm_menu.js), and portfolios that are non-existent (aren't on the site, don't link anywhere, or otherwise).

The company would like to hire as locally as they can rather than out-source to another country.

Answers I'm looking for

Processes you use when searching for someone

How you qualify their aptitude for the project

Anything that you think I'm doing wrong, or should be doing also.

Answers I'm not looking for:

"Hello sir please contact me we do everything for 10 dolla."

My bud's great at this stuff, call him.

example.com is the best for this.

答:
Why are you going to web design studios if you need web development? Web design studios usually design and develop websites from scratch. What you need is a web development company.

Tips for evaluating web development companies:

Look at their previous work, view source.

Contact them and see if they respond reasonably fast and that the manager is adequate, helpful and concise. This is very important, as it tells a lot about the company culture.

Contact their past clients. This can be easily done by contacting owners of the websites they've made. People are surprisingly eager to recommend, or -- even more so -- to criticize their contractors. These references are very useful.

Ask for an estimate and evaluate their offer. Things to look out for: are the price and time constraints reasonable? Did they forget an important stage (e.g. testing)? Is there any noticeable BS?

The most important things are their previous work and their communication and management skills. Keep in mind: when things go wrong (as they inevitably will), you'll have to rely on their communication and management skills, as much as their ability to produce quality code, to fix things.

ContentLicense: CC BY-SA 2.5


问:
C++ class with only pure virtual functions: what's that called?
So i'm looking for some input/consensus on what terminology we should be using to describe something that looks like this:

```<code>class Printable
{
  public:
    virtual void printTo(Printer *) = 0;
    virtual double getWidthInPoints() = 0;
    virtual double getHeightInPoints() = 0;
};
</code>```

So this is a class that only has pure virtual methods. Any class that inherits it needs to implement the whole thing, i.e. there is no partial implementation.

So what is this called?

I have mostly seen two different names:

1) Interface

Used in Windows COM programming (perhaps it came from the IDL), it's a keyword in both C# and Java and also a compiler directive/keyword in Objective-C (but isn't the same thing). I've heard engineers using the phrase 'interface' synonymously with 'API', but, at least in this scenario, it's a single thing whereas an API could comprises more than one.

2) Protocol

I've seen this used in various open-source libraries such as Bloomberg BSL and it's a compiler directive/keyword in Objective-C, but does not necessarily need to be in the inheritance hierarchy. In the broader computer science sense, protocol is a formal definition of information exchange between two parties, like TCP and HTTP, but this definition kind of holds for the above class too.

Is there any clear distinction between these in C++ or are they pretty much interchangeable?

答:
You can call  C++ class with only pure virtual functions as interface or protocol.

Interface and protocol both are the same concept.

If you were a java/C# programmer,you call the class with only purt virtual functions as an interface.
If you were a Objective-C programmer,you call the class with only purt virtual functions as a protocol!


答:
Put each user rank times into a suitable data structure (bag, multiset, list, array). Such a data structure should already be available for your language, you probably don't need to implement it yourself.

Sample a random element from that data structure. Such an operation should already be available, you probably don't need to implement it yourself.

For example in Ruby:

```<code>users = %i[A B C D]
ranks = [1, 1, 5, 2]

user_pool = users.map.with_index {|u, i| [u] * ranks[i] }.flatten
# => [:A, :B, :C, :C, :C, :C, :C, :D, :D]

user_pool.sample
# => :C
</code>```

No loops, no counting, no computing totals. Just two lines making use of the collections framework.

Title: Algorithm to pick user

Body:
I need an algorithm for picking a user.

Users are identified by letter {A, B, C, ...} and are ranked by number {1, 2, 3, ...}.  Rank is the degree of likelihood to be picked, so a rank 2 user is twice as likely to be picked over a rank 1 user, and rank 4 is four times as likely, etc.

So let's say four users are {A, B, C, D}, with rank {1, 1, 5, 2} respectively.  A user table might store rank:

```<code>   USER   RANK
      A   1
      B   1
      C   5
      D   2
</code>```

How do I pick a user based on rank?

My first idea for an algorithm is to add up all the ranks 1 + 1 + 5 + 2 to obtain a sum of 9.  Then assign subranges from 1 to 9 to each user, where subrange size is user rank.  So A has range [1, 1], B has range [2, 2], C has range [3, 7] (5 possibilities due to rank being 5), and D has range [8, 9].  The table becomes this:

```<code>   USER   RANK  RANGE
      A   1     [1, 1]
      B   1     [2, 2]
      C   5     [3, 7]
      D   2     [8, 9]
</code>```

Then calculate a random value from 1 to 9, and find the user whose range contains that value.  So if the random value is 4, user C is picked because 4 falls within range [3, 7].

This achieves fairness based on rank.  If you repeat picking randomly this way, on the average, users will be chosen fairly, and no user will be starved.

However, it feels sloppy, because to add or remove users, I have to recalculate all the ranges.  Here is removing user C (recalculate range of user D):

```<code>   USER   RANK  RANGE
      A   1     [1, 1]
      B   1     [2, 2]
      D   2     [3, 4]
</code>```

And here is changing user A to have rank 2 (recalculate range of B and D):

```<code>   USER   RANK  RANGE
      A   2     [1, 2]
      B   1     [3, 3]
      D   2     [4, 5]
</code>```

What is the best algorithm to pick a user?  Surely recalculating ranges for every user add, delete, or update is suboptimal.  Is there a well-known function or algorithm that can help?

UPDATE: @rwong got me thinking about grouping people by rank, then picking rank instead of picking user.  With one person in each rank, you turn the random-shaped ranges into a very linearly-growing ranges (one dot is one chance to be picked):

```<code>Rank 1:  *
Rank 2:  * *
Rank 3:  * * *
Rank 4:  * * * *
Rank 5:  * * * * *
</code>```

Works great for exactly one person per rank, but it falls apart when you have lots of people at rank 1, where their chances should increase by the number of people within that rank, possibly  overtaking the chances of higher ranks.

Now we're back to an irregular shape, and must pick from it.  A friend pointed me to Rejection sampling.  This seems to be a great solution where you pick a random x and y coordinate, see if it falls within the shape, and if so, we now picked rank.  Then pick a random person within that rank.

To show this solution, here is the shape with multiple people inside each rank.  For rank 2, there are 5 people, so that's 2 x 5 = 10 chances:

```<code>User count   Rank   Chances(graph)       Chances(number)
5            1      * * * * *            5
5            2      * * * * * * * * * *  10
1            3      * * *                3
0            4                           0
1            5      * * * * *            5
</code>```

The algorithm is to pick a random row (like picking an x coordinate), and pick a random value from 1 to max(chances) (like picking a y coordinate).  See if that is less than or equal to the number of chances at that rank.  If not repeat until you have a valid rank and chance.  This takes care of distribution probability.  Now that you have chosen a valid rank, at that point you simply pick a random person within that rank.  Done!

The beauty is, to add or remove someone, you increment or decrement the user count column, and assign a new Changes(number).  Two small very simple updates.  The memory complexity and time complexity of add/remove/update user and also complexity to pick a user is nothing, like O(1).

We grouped user by rank, and captured user count in each rank, then calculate chances for that rank, and used a simple looping algorithm to pick random rows and chance values until we get a hit, and finally pick a random person at that rank.

Body:
Put each user rank times into a suitable data structure (bag, multiset, list, array). Such a data structure should already be available for your language, you probably don't need to implement it yourself.

Sample a random element from that data structure. Such an operation should already be available, you probably don't need to implement it yourself.

For example in Ruby:

users = %i[A B C D]
ranks = [1, 1, 5, 2]

user_pool = users.map.with_index {|u, i| [u] * ranks[i] }.flatten
# => [:A, :B, :C, :C, :C, :C, :C, :D, :D]

user_pool.sample
# => :C

No loops, no counting, no computing totals. Just two lines making use of the collections framework.


答:
If you reverse your RANK value so that lower values are more likely than it's a lot easier, or you can define a cap RANK value of say 10 and then just go (10-RANK).

To implement this in SQL it's easy.

```<code>SELECT *,((10-rank) * RAND()) AS odds
    FROM users
    ORDER BY odds
    LIMIT 1;
</code>```

That will generate a random value per user from 0 to their rank. Users with a higher rank are more likely to sort above others, then just find the first one.

Title: Algorithm to pick user

Body:
If you reverse your RANK value so that lower values are more likely than it's a lot easier, or you can define a cap RANK value of say 10 and then just go (10-RANK).

To implement this in SQL it's easy.

SELECT *,((10-rank) * RAND()) AS odds
    FROM users
    ORDER BY odds
    LIMIT 1;

That will generate a random value per user from 0 to their rank. Users with a higher rank are more likely to sort above others, then just find the first one.


问:
What is bootstrap listener in the context of Spring framework?
I am studying Spring framework, in web.xml I added following which is a bootstrap listener. Can anyone give me a proper idea of what is a bootstrap listener?

```<code><listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
</code>```

You can see the doc here: ContextLoadListener

Bootstrap listener to start up and shut down Spring's root WebApplicationContext. Simply delegates to ContextLoader as well as to ContextCleanupListener.

This listener should be registered after Log4jConfigListener in web.xml, if the latter is used.

As of Spring 3.1, ContextLoaderListener supports injecting the root web application context via the ContextLoaderListener(WebApplicationContext) constructor, allowing for programmatic configuration in Servlet 3.0+ environments. See WebApplicationInitializer for usage examples...

答:
It's an event listener as defined in the servlet standard. Basically, it's a way to execute code when the servlet container has started the web app.

Spring's ContextLoaderListener uses this mechanism to create the WebApplicationContext, which involves reading and parsing all bean configuration XML files, as well as (depending on the configuration) instantiating the singleton beans.


问:
Method vs Function vs Procedure
Simple question, but I often hear these three terms defined with such ferocity, but which have been known to me to mean different things over the years.

What are the "correct" definitions of "Procedures", "Methods", "Function", "Subroutines", etc?

答:
A function returns a value, but a procedure does not.

A method is similar to a function, but is internal to part of a class. The term method is used almost exclusively in object-oriented programming.

Title: Method vs Function vs Procedure


问:
First step in analysing a proposed system
A client asked us to automate his offline paper-based processes. He sent us a copy of the physical forms that he uses.

Our team met and discussed how the navigation should be, what major components would exist in the UI. That's it.

Problem is, we don't have actual "written" requirements. With a lack of an actual business/system analyst in our company, what are the steps that we should take to reach the point where we are ready to start developing the system?

答:
Find out the business processes that he uses with the material.

Ask questions like:

Where does the data come from that he enters on the forms?

What happens to the forms after being filled out?

What does the typical data entered on them look like?

What fields are required?

What validations would be required?

Who uses the information once entered?

How long should the information be retained?

How should records be identified?

What would users want to search on to find records?

What additional functionality is already expected from automating the process?

What is the timeframe or time limits?

What external changes are expected/possible in the next year that would affect a solution?


答:
Even if you don't have a business analyst, you'll need to have requirements in written form in order to:

have a clear picture of what you are doing. No matter how many times you've discussed the application and how familiar you are with the details, you cannot guarantee to remember everything, and eventually you'll either miss a feature or deliver something else than the client needs.

use the requirements in negotiations with the client. The client should see what they will eventually have in an early stage. If they want to change anything, the written requirements will help to see where, what and possibly why the change was made.

protect yourselves. People tend to forget a lot. There is no chance that the client will remember every single detail of what you've agreed upon. If this happens, you'll have the document to prove otherwise.

It isn't necessary that you write the requirements as a business analyst would. Depending on the complexity of the system and on how obvious it is, even simple diagrams of high-level processes and UI can suffice.

If you still cannot have the requirements written, I suggest you build a simple prototype, discuss it with the client and continue working on that (though requirements and prototype don't exclude one another).

Title: First step in analysing a proposed system

Body:
A client asked us to automate his offline paper-based processes. He sent us a copy of the physical forms that he uses.

Our team met and discussed how the navigation should be, what major components would exist in the UI. That's it.

Problem is, we don't have actual "written" requirements. With a lack of an actual business/system analyst in our company, what are the steps that we should take to reach the point where we are ready to start developing the system?


问:
Java Heap Allocation Faster than C++
I already posted this question on SO and it did ok. It was unfortunately closed though(only needs one vote to reopen) but someone suggested I post it on here as it is a better fit so the following is literally a copy paste of the question

I was reading the comments on this answer and I saw this quote.

Object instantiation and object-oriented features are blazing fast to use (faster than C++ in many cases) because they're designed in from the beginning. and Collections are fast. Standard Java beats standard C/C++ in this area, even for most optimized C code.

One user (with really high rep I might add) boldly defended this claim, stating that

heap allocation in java is better than C++'s

and added this statement defending the collections in java

And Java collections are fast compared to C++ collections due largely to the different memory subsystem.

So my question is can any of this really be true, and if so why is java's heap allocation so much faster.

答:
The main reason is that, when you ask Java for a new lump of memory,it goes straight to the end of the heap and gives you a block. In these way, memory allocation is as fast as allocating on the stack (which is how you do it most of the time in C/C++, but apart from that..)

So allocations are fast as anything but... that doesn't count the cost of freeing the memory. Just because you don't free anything until much later doesn't mean it doesn't cost quite a lot, and in the case of GC system, the cost is quite a lot more than 'normal' heap allocations - not only does the GC have to run through all objects to see if they're alive or not, it also then has to free them, and (the big cost) copy the memory around to compact the heap - so you can have the fast allocation at the end mechanism (or you'd run out of memory, C/C++ for example will walk the heap on every allocation looking for the next block of free space that can fit the object).

This is one reason why Java/.NET benchmarks show such good performance, yet real-world applications show such bad performance. I only have to look at the apps on my phone - the really fast, responsive ones are all written using the NDK, so much so even I was surprised.

Collections nowadays can be fast if all the objects are locally allocated, eg in a single contiguous block. Now, in Java, you simply don't get contiguous blocks as objects are allocated one at a time from the free end of the heap. You can end up with them happily contiguous, but only by luck (ie down to the whim of the GC compaction routines and how it copies objects). C/C++ on the other hand explicitly supports contiguous allocations (via the stack, obviously). Generally heap objects in C/C++ is no different from Java's BTW.

Now with C/C++ you can get better than the default allocators which were designed to save memory and use it efficiently. You can replace the allocator with a set of fixed-block pools, so you can always find a block that is exactly the right size for the object you're allocating. Walking the heap just becomes a matter of a bitmap lookup to see where a free block is, and de-allocation is simply re-setting a bit in that bitmap. The cost is that you use more memory as you allocate in fixed-size blocks, so you have a heap of 4 byte blocks, another for 16 byte blocks, etc.

Title: Java Heap Allocation Faster than C++


问:
When should use of database be preferred over parsing data from a text file?
I was making a Python program to measure the growth of codereview.SE. My approach was to get the "Site stats" shown on the front page and store them on my hard drive. I plan to do this once every day. So far I have made enough to get the stats and append them to a text file. The python script can be viewed on github. The format I am using is the following

```<code>22-08-2013

questions 9073
answers 15326
answered 88
users 26102
visitors/day 7407

22-08-2013

questions 9073
answers 15326
answered 88
users 26102
visitors/day 7407
</code>```

I just ran the script twice to get the format I would be using in the file. Initially this seemed good to me because I would be storing it myself and the format would be the same so it would be easily parsed but not I am not sure. It seems that using a database should be a better here because that way retrieving data should be easier. Just a note, I have never used any database and have no knowledge of SQL, MySQL or any other variants of RDBMS.

So this brings me to the question. When should a database be preferred for storing the data over storing the data in a text file? Are there some pointers that I can look for when making decisions about whether I need a database or simple text files?

PS: If better tags can be added please do so. I had some doubts about the tags which could be added.

答:
Data bases have many advantages, but making access easier isn't one of them. Faster, more standardized, interpretable as an embedded command sublanguage, safer, yes - but not easier. No matter how much syntactic sugar your language and standard library provide, you have to have a data base in the first place, open a connection to it and route data from your program something completely different and back. As long as there are no problems with what you do, and ease of programming is your priority, never switch to a database just because you think it's "good practice".

My take on when to make the switch is to follow the historical development. After all, people stored data in files for a long time before the relational DB was invented, and in fact a whole bunch of inferior database models (hierarchical DB, network DB...) were invented before that. They started writing data bases and used them when it became clear that this would save major processing effort, increase reliability etc. overall and in the long run. As long as that's not the case for you, and you don't foresee it becoming the case any time soon, switching would be over-engineering.


答:
Databases are used for not just storing but manipulating and querying data, therefore you'd have to make an educated decision:

A big factor is the benefit you get from installing a database on the machine vs the functionality it brings

Obviously if you need to query and manipulate the data, and you want access to be speedy - and additionally you might be thinking about using a database for other functions then it might be a good idea. Databases storage models allow data to be looked up by key values very quickly, and I can imagine parsing a file could be slow (depending on how you are doing it)

If you want to have a play with SQL and what it can do, SQLFiddle.com has a few different RDBMS models which you can toy around with (run queries, create schema etc)

Title: When should use of database be preferred over parsing data from a text file?

Body:
I was making a Python program to measure the growth of codereview.SE. My approach was to get the "Site stats" shown on the front page and store them on my hard drive. I plan to do this once every day. So far I have made enough to get the stats and append them to a text file. The python script can be viewed on github. The format I am using is the following

```<code>22-08-2013

questions 9073
answers 15326
answered 88
users 26102
visitors/day 7407

22-08-2013

questions 9073
answers 15326
answered 88
users 26102
visitors/day 7407
</code>```

I just ran the script twice to get the format I would be using in the file. Initially this seemed good to me because I would be storing it myself and the format would be the same so it would be easily parsed but not I am not sure. It seems that using a database should be a better here because that way retrieving data should be easier. Just a note, I have never used any database and have no knowledge of SQL, MySQL or any other variants of RDBMS.

So this brings me to the question. When should a database be preferred for storing the data over storing the data in a text file? Are there some pointers that I can look for when making decisions about whether I need a database or simple text files?

PS: If better tags can be added please do so. I had some doubts about the tags which could be added.


问:
How to improve in code after you know many languages?
Over the past years I learned about 14 programming languages
  (assembler phyton delphi qbasic c++ c# ruby perl   - etc etc etc). However this was all out of my own interest, I didn't went to school for it.

Now without real education I am into a team with other programmers.
We are into robotics with our work. In contrast to me they know about 2 languages and many frameworks I never had heard of before.

I don't really feel behind them, but they can surprise me and other times I can surprise them with solutions. The largest difference is that they have worked for years in the IT, while to me the programming part, is new and I enjoy it. But after the many languages I find it a bit hard to improve. Sometimes I see code (or tools) that are new to me.

I never used tools as anksvn, or did versioning or never had seen USB driver made in code, I got used to them, learned from them on projects and my colleagues. I started making money with code (without a real degree) the last 2 years.

Given that I haven't found any good YouTube channel or a book, that might improve my skills, how can I improve further?

My interests these days are C++ and C#. The point I loose interests if books start with the basics of "if then while for types objects" after 14 languages you get that.

答:
Knowing programming languages is one thing. Knowing how to make successful software is another.

What your colleagues may have learnt more than you is:

How to work in a team. This is an essential skill for most developers.

How to communicate. This is an essential skill for any developer.

Workflows and project management methods like Scrum. Just like you don't start building a house by putting bricks side by side, you don't start building a software product by writing code.

Methodologies, like TDD.

Tools, like version control. Every developer is expected to use one, and refuse to work without it.

etc.

There is no a single book or YouTube channel which will teach you all that. In order to move from "I can code" to "We achieved this software product for the deadline with all requirements finished and practically no bugs left", there is only way: practice, practice, practice.


问:
Zero-knowledge code hosting?
In light of recent revelations about widespread government monitoring of data stored by online service providers, zero-knowledge services are all the rage now.

A zero-knowledge service is one where all data is stored encrypted with a key that is not stored on the server. Encryption and decryption happens entirely on the client side, and the server never sees either plaintext data or the key. As a result, the service provider is unable to decrypt and provide the data to a third party, even if it wanted to.

To give an example: SpiderOak can be viewed as a zero-knowledge version of Dropbox.

As programmers, we rely heavily on, and trust some of our most sensitive data - our code - to a particular class of online service providers: code hosting providers (like Bitbucket, Assembla, and so on). I am of course talking about private repositories here - the concept of zero-knowledge does not make sense for public repositories.

My questions are:

Are there any technological barriers to creating a zero-knowledge code hosting service? For example, is there something about the network protocols used by popular version control systems like SVN, Mercurial, or Git that would make it difficult (or impossible) to implement a scheme where the data being communicated between the client and the server is encrypted with a key the server does not know?

Are there any zero-knowledge code hosting services in existence today?

答:
You can encrypt each line seperately. If you can afford to leak your file names and approximate line lengths and the line numbers on which lines changes occur, you can use something like this:

https://github.com/ysangkok/line-encryptor

As each line is encrypted seperately (but with the same key), the uploaded changes will (like usually) only involve the relevant lines.

If it is presently not convenient enough, you could make two Git repositories, one with plaintext and one with ciphertext. When you commit in the plaintext repository (which is local), a commit hook could take the diff and run it through the line encryptor referenced above, which would apply it to the ciphertext repository. The ciphertext repository changes would be committed and uploaded.

The line encryptor above is SCM agnostic, but can read unified diff files (of plaintext) and encrypt the changes and apply them to the ciphertext. This makes it usable on any SCM that will generate you a unified diff (like Git).

Title: Zero-knowledge code hosting?


问:
Without C++-like destructors, how do we return resources that aren't managed by garbage collector in Java?
I'm in a job hunting. And in my CV I placed a skill list like:

```<code> Skills: C/C++/Java/...
</code>```

The most common question I got is: "hem, since you are familiar with both C++ and Java, can you tell some similarities or difference between the two languages."

And I just don't know how to answer, what I said is basically some language level details like  they have some different keywords like Interface,abstract and so on. I want to see some comparison in high level like the difference in generics, the garbage collector and so on.

At least I want to go deep into one side, that is the resource management. Java has no lifetime for an object, this is managed by the garbage collector, and in C++ you need to carefully manage your resource especially for the heap. In C++ we can greatly reduce the memory leak by introducing RAII, using object to manage the heap memory, and so is for the other resources like connection,lock and so on. I am not sure what to do in Java, because the garbage collector can only be a nice tools for the management of heap memory (AFAIK ).

Question: How can we manage other resources in an situation that we do not have a destructor to do all these automatically? Do we need to manually guarantee that the resources be returned in a right place. And how?

答:
Actually the question make some sense, because all the three languages somewhat tightened by a same overall history, from VonNeuman machines to C as a "high level assembler", to the need to industrialize code re-usability and the introduction of the idea of "object".

In C this is just "function and data and opaque types". After a formalization of object oriented methodologies, C++ added to C the idea of function overload and runtime-based polymorphism, with the drawback to grant object life across functions, and the consequence of object lifetime management.

Java comes soon after, embracing the OOP paradigm and being designed around it and around the idea of "platform independence" (thus creating the n+1 platform: the java machine). By constraining all object to be dynamic, it solved the problem of C++ pointer, introducing garbage collection and unifying pointer and object into a common syntax (the . notation).

Similarly C++ evolved towards a more generalized language, not tightening into OOP, but introducing templates and lambdas, opening that way towards other programming paradigm (like generic programming, and functional programming) and resource management methodologies (like RAII, reference counting and the like) embedding the memory management problems into library and purposed defined classes.

For what nowadays concern, Java gorws where resource control is not "the issue" but where having a large multiplatfom OOP codebase is straightforward. C++ is more appreciated where resource control and high optimization is a must (a compiler can hack down to the asembler instruction set of a machine, Java has to remain at a java machine level, thus breaking the optimization process in two distinct pieces) and where algorithm and hardware are required to better cooperate.

To answer to the final question, in absence of deterministic destruction, deterministic resource management have to be handled explicitly with finalization methods (like "on_final" or similars) eventually exposed by interfaces (like "finalizable") to be eventualli chained in collection, to be explcitly called where appropriate.


问:
Breaking down a complex story at project start
I'm trying to get to grips with agile project management (with Pivotal Tracker) but keep finding myself running into walls when trying to define the first few stories of a project. Take for example this very simple story:

"A user should be able to tag a product"

Assuming I've already defined "product" elsewhere, this story would possibly involve writing a polymorphic tagging system under the hood, on completion of that system id be able to finally add the ability to tag a product.

My problem is with the amount of work hidden in this story. I can define tasks to get the story done but stories as a whole are supposed to be 1-2 days work? I don't feel right about the story just revealing the tip of the iceberg but that's the only part that relates to the User.

How do you overcome this kind of thing? What are the best practices?

UPDATE 25/08 Thanks to everyone for your guidance, I've taken all advice onboard on how to define stories. I am now switching to Jira Grasshopper which has better support for tasks within stories (assignment, estimates etc) and tracking of development tasks once the sprint has began.

答:
It seems that you are confusing stories and tasks.

User Story

A user story is a complete "feature", something that when added to the product, provides more value to the product.

A user story should not be larger than it can be implemented during a sprint. During the first part of sprint planning, you decide which user stories you want to work on during the sprint. The goal of the sprint is to complete these user stories, thus adding more value to the product.

Task

During second part of the planning phase of the sprint, developers divide the story into tasks. The tasks are development tasks. They could be stuff like "Add column to database", "Extend service x", etc. A task should not be larger than it can be completed in one day.

During the daily scrum you evaluate the progress of these tasks. If a task has been in progress for more than one daily scrum, it is taking too long, and you, as a team, has the responsibility of resolving that situation.

Remember, user stories represent business value for the stake holders. The stake holders should be interested in the completion of the user stories, not tasks.

The task division is a tool for the development team to manage the sprint, to monitor progress of the user stories during a sprint, and to visualize potential problems.

The stake holders should not concern themselves with these development tasks. Unfortunately, it is my experience that they often do, particularly for organizations new to agile development. Dealing with this situation is a different matter though.

Epic

If a user story is bigger than you think you can complete it in one sprint, it is called an epic. It needs to be divided into several smaller user stories before you as a team can start working on it.

Remember that a user story adds value to the end user, so splitting an epic into a "front-end" and a "back-end" story is not the right way. Adding the back-end for a new feature does not in itself provide value to the end users.

Dividing an epic into user stories manageable within the time frame of a sprint is not always easy when you are not experienced with doing so.

Using Pivotal Tracker

I think Pivotal Tracker is a great tool for tracking user stories. But it is not a scrum tool as such, and the way that scrum teaches to divide stories into tasks is not easily handled by pivotal tracker. You can enable the ability to add tasks to user stories. But if you are running a project using scrum, I would suggest using a white board and sticky notes to track the progress of tasks during a sprint.

OwnerDisplayName: matthewrk


答: Java 7 has try-with-resources to handle non-memory resources.  You can implement java.lang.AutoCloseable to use it with your own objects.  RAII wasn't an original design intent for destructors, it was "discovered" later and only came into vogue in the late 90s, early 2000s, after Java was created.  That's why Java 1-6 didn't have any RAII-like construct.

Title: Without C++-like destructors, how do we return resources that aren't managed by garbage collector in Java?

Body:
I'm in a job hunting. And in my CV I placed a skill list like:

```<code> Skills: C/C++/Java/...
</code>```

The most common question I got is: "hem, since you are familiar with both C++ and Java, can you tell some similarities or difference between the two languages."

And I just don't know how to answer, what I said is basically some language level details like  they have some different keywords like Interface,abstract and so on. I want to see some comparison in high level like the difference in generics, the garbage collector and so on.

At least I want to go deep into one side, that is the resource management. Java has no lifetime for an object, this is managed by the garbage collector, and in C++ you need to carefully manage your resource especially for the heap. In C++ we can greatly reduce the memory leak by introducing RAII, using object to manage the heap memory, and so is for the other resources like connection,lock and so on. I am not sure what to do in Java, because the garbage collector can only be a nice tools for the management of heap memory (AFAIK ).

Question: How can we manage other resources in an situation that we do not have a destructor to do all these automatically? Do we need to manually guarantee that the resources be returned in a right place. And how?


问:
What do I call people who extend my class in base class documentation?
Been calling them "implementers", but it seems weird to me.

```<code>/// <summary>
/// Implementers should implement this.  Derp
/// </summary>
protected abstract void InternalExecute();
</code>```

A point of clarification, I'm interested in what to call the people who create child classes, not the child classes themselves.

"Hey, you there" not "that thing there."

答: I typically say child class. When talking about inheritance, implementers would be a little confusing to me because it makes me think that the class is implementing an Interface, which is different from class inheritance.


答:
It has been a long time since I have had to dive into a large code base myself. But in the recent couple of years I tried many times to get new developers into teams where we had an existing, rather large, code base.

And the method that we have used successfully, and I would say is the most effective way without question IMHO, is pair programming.

In the last 12 months, we have had 4 new members to the team, and every time, the new member would pair with another member well acquainted with the code base. In the beginning, the older team member would have the keyboard. After about 30 minutes we would pass the keyboard to the new member, who would work under the guidance of the older team member.

This process has proven to be quite successful.

Title: How do you dive into large code bases?

Body:
What tools and techniques do you use for exploring and learning an unknown code base?

I am thinking of tools like grep, ctags, unit-tests, functional test, class-diagram generators, call graphs, code metrics like sloccount, and so on. I'd be interested in your experiences, the helpers you used or wrote yourself and the size of the code base with which you worked.

I realize that becoming acquainted with a code base is a process that happens over time, and familiarity can mean anything from "I'm able to summarize the code" to "I can refactor and shrink it to 30% of the size". But how to even begin?


问:
What is the exact syntax of inline?

CASE 1 (Definition and declaration in same source file)

Suppose both my prototype and definition of global function is in  .cpp file. Where should I write inline keyword to make compiler know?

In the prototype

In the definition

Both

CASE 2  (Definition and declaration in same header file)

If I have declared prototypes in .h file I want to make that function inline in more than one .cpp file using it, then should I define the function (to be inlined) in that .h  file and include it to individual .cpp files? If yes, then in this case  where  inline keyword should  be used?

CASE 3  (Definition in Header file and declaration in Source file)

If the prototype is in a header file, definition in a source file (in which other functions use the inline function), where must the inline keyword be used?

In the declaration in header file

In the definition in source file

In both

答:
You asked about the syntax of the inline keyword:

Case 1

There's only one definition here, so no possible ODR violation, and no need for inline at all.

Case 2

Just write the prototype and definition in your header file as

```<code>inline int foo() { return 42; }
</code>```

or whatever

Case 3

Like case 1, it isn't clear why you think this function is inline in the first place. Don't use the keyword, there is no risk of ODR violation here.

I now suspect you meant to ask how to force the compiler to generate inlined calls to your global function.

The inline keyword does not force the compiler to do this. It was only ever a hint, and now its only deterministic effect is on the ODR (one definition rule).

The general answer is to turn up the optimization level and, if your compiler doesn't support cross-module inlining, to make the function body visible in-line. The compiler may still decide not to inline the function call even if it's visible, but the optimizer will have the option.

Title: What is the exact syntax of inline?

Body:
You asked about the syntax of the inline keyword:

Case 1

There's only one definition here, so no possible ODR violation, and no need for inline at all.

Case 2

Just write the prototype and definition in your header file as

inline int foo() { return 42; }

or whatever

Case 3

Like case 1, it isn't clear why you think this function is inline in the first place. Don't use the keyword, there is no risk of ODR violation here.

I now suspect you meant to ask how to force the compiler to generate inlined calls to your global function.

The inline keyword does not force the compiler to do this. It was only ever a hint, and now its only deterministic effect is on the ODR (one definition rule).

The general answer is to turn up the optimization level and, if your compiler doesn't support cross-module inlining, to make the function body visible in-line. The compiler may still decide not to inline the function call even if it's visible, but the optimizer will have the option.


答:
inline is NOT part of the interface, and client users do not care if a function is inline or not. Basically you can place a inline key word both in the declaration and definition. But it is preferred to place it in the implementation that is in the definition.

So, the rest 3 cases is trivial. You can choose anyone you like. But with the keyword in definition is my prefer.

Title: What is the exact syntax of inline?

Body:
CASE 1 (Definition and declaration in same source file)

Suppose both my prototype and definition of global function is in  .cpp file. Where should I write inline keyword to make compiler know?

In the prototype

In the definition

Both

CASE 2  (Definition and declaration in same header file)

If I have declared prototypes in .h file I want to make that function inline in more than one .cpp file using it, then should I define the function (to be inlined) in that .h  file and include it to individual .cpp files? If yes, then in this case  where  inline keyword should  be used?

CASE 3  (Definition in Header file and declaration in Source file)

If the prototype is in a header file, definition in a source file (in which other functions use the inline function), where must the inline keyword be used?

In the declaration in header file

In the definition in source file

In both


问:
Encapsulation in Domain Driven Design models?
I am using EF Code First and I had a model like below.

```<code>public class Account
{
    [Required]
    public string AccountNo { get; set; }
    [Required]
    public decimal Balance { get; set; }
}
</code>```

I used a Service class to withdraw and deposit amount to Account. Then I came across this and realised that Account is Anemic. So I happily added Withdraw and Deposit methods to Account.

```<code>public class Account
{
    [Required]
    public string AccountNo { get; set; }
    [Required]
    public decimal Balance { get; set; }
    public void Withdraw(decimal amount)
    {
        // TODO: Check if enough balance is available
        Balance -= amount;
    }
    public void Deposit(decimal amount)
    {
        Balance += amount;
    }
</code>```

}

I was very happy with this until I realised that this does not implement encapsulation correctly. You should not be able to modify the Balance directly - you should use Withdraw and Deposit methods. But in my solution you can modify the Balance directly.

But I cannot make Balance a readonly property as I am using EF code first approach and readonly properties won't result in the column being created in the database table.

I thought of leaving Account as anemic and create a higher level Account class which implements encapsulation and Withdraw/Deposit methods. But in that case the higher level class would not have validation data annotations and I won't get the free functionality provided by the framework.

What is the solution?

答:
There is a fundamental difference between your Data Access Object (DAO) and your Business Object (BO). Your DAO is a one-to-one mapping of what your data store looks like. In this case, for your AccountDAO object, the AccountNo and Balance properties will have getters and setters. Your Account object - the business layer objects - will not look the same as your DAO.

```<code>// This is a DAO
public class AccountDAO {
   public string AccountNo { get; set; }
   public decimal Balance { get; set; }
}

// This is a business object
public class Account {
   private readonly string accountNo;
   private readonly decimal openingBalance;
   private readonly List<decimal> adjustments; // This should probably be a list of transactions.

   public class Account(string accountNo, decimal openingBalance) {
      this.accountNo = accountNo;
      this.openingBalance = openingBalance;
      this.adjustments = new List<decimal>();
   }

   public string AccountNo {
      get { return accountNo; }
   }

   public decimal Balance {
      get { return openingBalance + SumOfAdjustments(); }
   }

   public void Deposit(decimal amount) {
      adjustments.Add(amount);
   }

   public void Withdraw(decimal amount) {
      decimal testBalance = Balance - amount;
      if (testBalance < 0.0m)
         throw new AccountBalanceException("Account Balance for " + accountNo + " is less than $0.00.");

      adjustments.Add(-1.0m * amount);
   }

   private decimal SumOfAdjustments() {
      decimal sum = 0.0m;
      foreach (var adjustment in adjustments) {
         sum += adjustment;
      }
      return sum;
   }
}

// Let's come up with a good way of performing actions on an account.
public class CashDepositActivity {
   private readonly Account account;
   private readonly decimal amount;

   public CashDepositAction(Account account, decimal amount) {
      this.account = account;
      this.amount = amount;
   }

   public void ExecuteAction() {
      // You should probably have some logging here.
      account.Deposit(amount);
   }
}

public class BalanceTransferActivity {
   private readonly Account sourceAccount;
   private readonly Account targetAccount;
   private readonly decimal amount;

   public BalanceTransferActivity(Account sourceAccount, Account targetAccount, decimal amount) {
      this.sourceAccount = sourceAccount;
      this.targetAccount = targetAccount;
      this.amount = amount;
   }

   public void ExecuteAction() {
      sourceAccount.Withdraw(amount);
      targetAccount.Deposit(amount);
   }
}
</code>```

This is just an example, of course, and it is by no means an end-all solution. I know tools like EF have myriad examples where the data object, business object, and view/edit object are all the same object, passed from the database to view. In my experience, that works great on small projects where there is not a lot of business logic. However, it breaks down as your business rules and access restrictions grow in complexity.

Body:
There is a fundamental difference between your Data Access Object (DAO) and your Business Object (BO). Your DAO is a one-to-one mapping of what your data store looks like. In this case, for your AccountDAO object, the AccountNo and Balance properties will have getters and setters. Your Account object - the business layer objects - will not look the same as your DAO.

// This is a DAO
public class AccountDAO {
   public string AccountNo { get; set; }
   public decimal Balance { get; set; }
}

// This is a business object
public class Account {
   private readonly string accountNo;
   private readonly decimal openingBalance;
   private readonly List<decimal> adjustments; // This should probably be a list of transactions.

   public class Account(string accountNo, decimal openingBalance) {
      this.accountNo = accountNo;
      this.openingBalance = openingBalance;
      this.adjustments = new List<decimal>();
   }

   public string AccountNo {
      get { return accountNo; }
   }

   public decimal Balance {
      get { return openingBalance + SumOfAdjustments(); }
   }

   public void Deposit(decimal amount) {
      adjustments.Add(amount);
   }

   public void Withdraw(decimal amount) {
      decimal testBalance = Balance - amount;
      if (testBalance < 0.0m)
         throw new AccountBalanceException("Account Balance for " + accountNo + " is less than $0.00.");

      adjustments.Add(-1.0m * amount);
   }

   private decimal SumOfAdjustments() {
      decimal sum = 0.0m;
      foreach (var adjustment in adjustments) {
         sum += adjustment;
      }
      return sum;
   }
}

// Let's come up with a good way of performing actions on an account.
public class CashDepositActivity {
   private readonly Account account;
   private readonly decimal amount;

   public CashDepositAction(Account account, decimal amount) {
      this.account = account;
      this.amount = amount;
   }

   public void ExecuteAction() {
      // You should probably have some logging here.
      account.Deposit(amount);
   }
}

public class BalanceTransferActivity {
   private readonly Account sourceAccount;
   private readonly Account targetAccount;
   private readonly decimal amount;

   public BalanceTransferActivity(Account sourceAccount, Account targetAccount, decimal amount) {
      this.sourceAccount = sourceAccount;
      this.targetAccount = targetAccount;
      this.amount = amount;
   }

   public void ExecuteAction() {
      sourceAccount.Withdraw(amount);
      targetAccount.Deposit(amount);
   }
}

This is just an example, of course, and it is by no means an end-all solution. I know tools like EF have myriad examples where the data object, business object, and view/edit object are all the same object, passed from the database to view. In my experience, that works great on small projects where there is not a lot of business logic. However, it breaks down as your business rules and access restrictions grow in complexity.


答:
Having a design goal of implementing a polymorphic tagging system is fine, but you should still focus on implementing features the customer wants. This might mean that, fine-grained-User-Story by fine-grained-User-Story, your system evolves into having a polymorphic tagging system over time. At any point on that journey however you should have system made up of lots of small and testable features, described by a collection of User Stories you've implemented.

In this case it sounds like "Tagging Products" in your system might be an Epic, i.e. something which is far bigger than a single User Story and can be broken down into several smaller stories with a little effort. Taking a fair amount of artistic licence, I can think of the following User Stories which might be roughly applicable to your requirements:

As a system administrator I want to seed the system with some valid tags so that users have some values to choose from when first using the tagging feature.

As a user of the system I want to be able to search for a product by name so that I can tag it later.

As a user of the system I want to be able to read the description of a product so that I can decide what tags it should have.

As a user of the system I want to be able to see a picture of the product so that I can decide what tags it should have.

As a user of the system I want to be able to attach a single tag to a single product.

As a user of the system I want to be able to remove a single tag from a single product.

As a user of the system I want to be able to attach a single tag to multiple products.

As a user of the system I want to be able to attach multiple tags to a single product.

As a system administrator I want to see a distinct list of tags in use in the system so that I can decide if any of them are duplicates.

As a system administrator I want to consolidate duplicated tags.

...and I could go on.

I doubt any of these will be so lifelike that you will use them, but hopefully they illustrate the sort of detail you can go to with your User Stories.

If a User Story really cannot be sub-divided into any smaller stories and you still deem it too large to attempt to implement in one go, then you can split it into vertical slices. This is a technique which means delivering only part of the feature under each User Story, but each part being a testable slice through all the relevant layers of the technology, e.g. for a website this might mean changing the database, application and UI layers. Avoid having one User Story for the database work, another for application and another for UI, as these won't be individually testable.

Taking even more artistic licence, I might split "As a user of the system I want to be able to attach a single tag to a single product." into the following vertical slices:

As a user of the system looking at a single product I want to be able to look up a list of tags so that I can decide which one to apply.

As a user of the system looking at a single product, having decided on a tag to apply to that product, I want to be able to apply it.

As a user of the system looking at a single product, having applied a tag to that product, I want a confirmation message on screen telling me that it has saved successfully.

Each of those is testable - if not particularly valuable in their own right.

Title: Breaking down a complex story at project start

Body:
I'm trying to get to grips with agile project management (with Pivotal Tracker) but keep finding myself running into walls when trying to define the first few stories of a project. Take for example this very simple story:

"A user should be able to tag a product"

Assuming I've already defined "product" elsewhere, this story would possibly involve writing a polymorphic tagging system under the hood, on completion of that system id be able to finally add the ability to tag a product.

My problem is with the amount of work hidden in this story. I can define tasks to get the story done but stories as a whole are supposed to be 1-2 days work? I don't feel right about the story just revealing the tip of the iceberg but that's the only part that relates to the User.

How do you overcome this kind of thing? What are the best practices?

UPDATE 25/08 Thanks to everyone for your guidance, I've taken all advice onboard on how to define stories. I am now switching to Jira Grasshopper which has better support for tasks within stories (assignment, estimates etc) and tracking of development tasks once the sprint has began.


问:
Cross border data obfuscation and how to deal with it?
I am currently facing a situation where I will need to obfuscate data for legal reasons (these are countrywide rules and not specific to me)

I will still need to work with teams outside this area who will still need to access applications/databases for testing/deployment and so on, however the data must be fully obfuscated.

What are your experiences of this scenario?

e.g. Who owns what stage of the process, what access do the various parties have, how strong is the encryption, how do updates occur

答:
One large financial client we do business with has a standardized automated process for obfuscating data. We don't, so I have a few scripts where I do this by hand. The point is to leave reasonably realistic data (lengths of names, postal codes) while rendering the personally identifiable data irretrievably scrambled. Their system is far more complicated than this, but basically when production data gets copied to development and QA environments, it will be scrambled automatically. This way there is no potential for "forgetting" to do some of the scrambling.

Passwords:

Set them all to something test accounts use: like Password1 or 1234567.

Tax ID numbers, Social Insurance Numbers, Social Security Numbers:

Take the first 3 digits and generate random numbers for the remainder. In the US, the first 3 digits are generally assigned based on where you lived when the SSN was issued, so not all combinations of first 3 digits are valid. For EINs, take the first 2 digits, as not all combinations of first 2 digits are valid. Adjust which digits get left alone if your country uses different rules.

Names:

Hash and base64 the first and last names separately. Take the first letter of unhashed name append the hash afterwards and truncate the result to original name's length

Example:
Name = "John Doe"  (I am using SHA384)

So John Doe gets turned into Jnbn Dnh. It helps to keep the names the same length as that may help to point out usability issues.

If you have rules such as "names cannot have digits" then you need to remove out the base 64 values that aren't valid, also lowercasing the subsequent letters (done in sample code below).

Addresses:
Street names and city names get hashed as names above do. Numbers stay the same. State and zip stays the same.

So 1313 Mockingbird Lane becomes 1313 Mvtqiwtuqrd Lzzx

Phone numbers:

Leave area code the same, generate random digits for the remaining digits.

Credit Card Numbers:

You should not be storing these at all.

Here is some sample & crude C# code for hashing and truncating (simple to display the concept)

```<code>    using System.Security.Cryptography;
    using System.Text.RegularExpressions;

    public string ScrambleInput(string sInput)
    {
        string sReturn = sInput.Substring(0,1);
        string sTemp = string.Empty;
        System.Security.Cryptography.SHA384Managed Hasher = new SHA384Managed();
        System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
        byte[] buff = new byte[sInput.Length];
        buff = enc.GetBytes(sInput);
        Hasher.ComputeHash(buff);
        sTemp = Convert.ToBase64String(Hasher.Hash, 0, Hasher.Hash.Length, System.Base64FormattingOptions.None);
        sTemp = sTemp.ToLower().Replace("+", "").Replace("/", "");
        sReturn += Regex.Replace(sTemp, @"\d", "");
        sReturn = sReturn.Substring(0, sInput.Length );
        return sReturn;
    }
</code>```

Body:
One large financial client we do business with has a standardized automated process for obfuscating data. We don't, so I have a few scripts where I do this by hand. The point is to leave reasonably realistic data (lengths of names, postal codes) while rendering the personally identifiable data irretrievably scrambled. Their system is far more complicated than this, but basically when production data gets copied to development and QA environments, it will be scrambled automatically. This way there is no potential for "forgetting" to do some of the scrambling.

Passwords:

Set them all to something test accounts use: like Password1 or 1234567.

Tax ID numbers, Social Insurance Numbers, Social Security Numbers:

Take the first 3 digits and generate random numbers for the remainder. In the US, the first 3 digits are generally assigned based on where you lived when the SSN was issued, so not all combinations of first 3 digits are valid. For EINs, take the first 2 digits, as not all combinations of first 2 digits are valid. Adjust which digits get left alone if your country uses different rules.

Names:

Hash and base64 the first and last names separately. Take the first letter of unhashed name append the hash afterwards and truncate the result to original name's length

Example:
Name = "John Doe"  (I am using SHA384)

So John Doe gets turned into Jnbn Dnh. It helps to keep the names the same length as that may help to point out usability issues.

If you have rules such as "names cannot have digits" then you need to remove out the base 64 values that aren't valid, also lowercasing the subsequent letters (done in sample code below).

Addresses:
Street names and city names get hashed as names above do. Numbers stay the same. State and zip stays the same.

So 1313 Mockingbird Lane becomes 1313 Mvtqiwtuqrd Lzzx

Phone numbers:

Leave area code the same, generate random digits for the remaining digits.

Credit Card Numbers:

You should not be storing these at all.

Here is some sample & crude C# code for hashing and truncating (simple to display the concept)

    using System.Security.Cryptography;
    using System.Text.RegularExpressions;

    public string ScrambleInput(string sInput)
    {
        string sReturn = sInput.Substring(0,1);
        string sTemp = string.Empty;
        System.Security.Cryptography.SHA384Managed Hasher = new SHA384Managed();
        System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
        byte[] buff = new byte[sInput.Length];
        buff = enc.GetBytes(sInput);
        Hasher.ComputeHash(buff);
        sTemp = Convert.ToBase64String(Hasher.Hash, 0, Hasher.Hash.Length, System.Base64FormattingOptions.None);
        sTemp = sTemp.ToLower().Replace("+", "").Replace("/", "");
        sReturn += Regex.Replace(sTemp, @"\d", "");
        sReturn = sReturn.Substring(0, sInput.Length );
        return sReturn;
    }


答:
As always using a database or not depends on what you need to do.
If you have a huge amount of data and you need to perform many different queries on it, probably a database could help you.

In your case I would keep the storage in a test file until the performance are acceptable. Usually reading a text file (even big) doesn't take that long.
If you need more you can always add the database later.

For my experience, if you are completely new to databases you may find easier using something like couchdb: http://couchdb.apache.org/ which is no-sql and you can use directly javascript or python, etc. for queries.


答:
To elaborate on @MainMa's answer:
Sounds like you're at the point that you should cultivate a much broader perspective on software development.

Learn a complete development "stack".

Will give you a point of reference as you run into other unfamiliar tools, frameworks, etc.

Development IDE

Unit Testing

i.e. a testing framework like NUnit (C#), JUnit (Java), etc.

software version control

learn a distributed model tool - Git, Mercurial

Continuous Integration

i.e. checking-in code automatically triggers ALL the tests.

The application is built every time, as part of the above

a tool like CruiseControl

scripting to make it "continuous" - Ant

Understand Functional programming paradigm

The formalisms of functional are becoming very main stream today.

C# evolution is clearly supporting functional programming

It is different from conventional OO thinking but is not mutually exclusive

IMHO, can help OO code:

reducing side effects - bugs

Adds flexibility to your software design

Develop at least a "conversational understanding" of

Agile, Test Driven Development, Behavior Driven Development, Functional Programming, Software as a service, Unit Testing, continuous integration, specific design patterns, ...

Focus study on architecture, design, design patterns

Learn to recognize these patterns in code. Practice, practice ...

Design needs drives pattern use. Don't force your code/design to fit into the pattern du jour

Object Oriented Design

Think OO everywhere in your code. Most code I've seen starts out OO and then falls apart quickly in the details. IMHO too many dev'ers really do not grok OO. Excel at this and stand out from the crowd.

Theory to Practical

No magic bullets to bridge the gaps, but I emphasize actual books over youtube. You need complete, cohesive, coherent content in depth.

You will have to read a lot to glean the gems.

2 Book every software developer should have

Code Complete. First and last. Get this book. Read it.

Also has an appendix of recommended books

Refactoring, Improving the design of existing code

Some of the Books that particularly helped me

Code Complete. Lots of very good, practical, detailed advice. For example, everyone says "write good code comments", but you will not until you read this book.

Head First Design Patterns

Head First Object Oriented Analysis & Design

Functional Programming in C#

XUnit Design Patterns

Brownfield Application Development in .NET

Title: How to improve in code after you know many languages?

Body:
Over the past years I learned about 14 programming languages
  (assembler phyton delphi qbasic c++ c# ruby perl   - etc etc etc). However this was all out of my own interest, I didn't went to school for it.

Now without real education I am into a team with other programmers.
We are into robotics with our work. In contrast to me they know about 2 languages and many frameworks I never had heard of before.

I don't really feel behind them, but they can surprise me and other times I can surprise them with solutions. The largest difference is that they have worked for years in the IT, while to me the programming part, is new and I enjoy it. But after the many languages I find it a bit hard to improve. Sometimes I see code (or tools) that are new to me.

I never used tools as anksvn, or did versioning or never had seen USB driver made in code, I got used to them, learned from them on projects and my colleagues. I started making money with code (without a real degree) the last 2 years.

Given that I haven't found any good YouTube channel or a book, that might improve my skills, how can I improve further?

My interests these days are C++ and C#. The point I loose interests if books start with the basics of "if then while for types objects" after 14 languages you get that.


答:
Bruce has a good answer.  I would add, semantically:

A procedure should "do something" to the arguments or cause some other side effect (e.g. printf)

A function should (a) answer a question about the arguments, or (b) compute a new value based on the arguments

A function method should answer a question about the state of the object

A procedure method should change the state of the object

Title: Method vs Function vs Procedure

Body:
Simple question, but I often hear these three terms defined with such ferocity, but which have been known to me to mean different things over the years.

What are the "correct" definitions of "Procedures", "Methods", "Function", "Subroutines", etc?


问:
We need a custom strategy for collecting unhandled application exceptions. What are our options?

"Unhandled exception" term

In .NET Framework, unhandled exceptions are the exceptions which were not handled by the application itself, and result in a crash. In a case of a desktop application, it means that a window similar to this one is displayed:

For a web application, it mostly means an HTTP 500.

Under unhandled exceptions, I also include ones which are "handled globally", i.e., in a case of a desktop applications, the global handling which consists of displaying a custom window instead of the Windows default one.

Context

When I work as a freelancer, I use my own in-house solution to gather unhandled exceptions from different sources (web apps and desktop apps). The gathered results are then displayed on a monitoring panel in real time as well as collected for future analysis (to be linked with a bug tracking software, etc.)

Currently, I work in a company where I wouldn't be able to use my in-house solution to collect the exceptions (one of the reasons being that they won't accept to send all the exception messages to my servers).

This company doesn't have any precise strategy for collecting unhandled exceptions. The only solution which was used before is both rudimentary and out of question: it consists of sending every exception by e-mail.

This means that for the new product I'm working on, we should develop a custom strategy for collecting unhandled exceptions.

Question

I can always do by hand the part which will save the exceptions to the database or a log and the part which will load them from the database, a log or Windows Events.

I would like to avoid reinventing the wheel and use something which is already commonly used.

What are my choices? How are unhandled exceptions usually collected and processed later?

By the way:

Are there any libraries which help collecting those exceptions?

Are there any software products which help analyzing those exceptions?

答:
In short: It will depend on .NET Framework version, as it might be handled differently in .NET 4.0 than in 2.0.

A general rule of thumb is, when a runtime error occurs ( Runtime Error Yellow Screen of Death - YSOD) on a web application in production it is important to notify a developer and to log the error so that it may be diagnosed at a later point in time.

There are several resources on this topic. For .NET 2.0 applications you may look at overview of a post how ASP.NET processes runtime errors and looks at one way to have custom code execute whenever an unhandled exception bubbles up to the ASP.NET runtime.

Starting with .NET 4.0 developers get more flexibility by Asynchronous Programming Model (APM pattern). More details are posted here.

Prior to the .NET Framework 2.0, unhandled exceptions were largely ignored by the runtime.  For example, if a work item queued to the ThreadPool threw an exception that went unhandled by that work item, the ThreadPool would eat that exception and continue on its merry way.  Similarly, if a finalizer running on the finalizer thread threw an exception, the system would eat the exception and continue on executing other finalizers.

There are different flavor of User friendly Exception Handling  dialog strategies that you might look for windows forms applications.

In conclusion, there are some options depending on the Framework version that .NET application currently runs on.

ContentLicense: CC BY-SA 4.0


答:
In previous dev shop we used Microsoft's Exception Application Block - a .NET extension - to capture exceptions into a database and notify the developer. We had a home-made web front end to access (no pun intended) that data.

The Application Block is designed to be highly configurable.

A dated blurb from 4GuysFromRolla.com

Title: We need a custom strategy for collecting unhandled application exceptions. What are our options?

Body:
"Unhandled exception" term

In .NET Framework, unhandled exceptions are the exceptions which were not handled by the application itself, and result in a crash. In a case of a desktop application, it means that a window similar to this one is displayed:

For a web application, it mostly means an HTTP 500.

Under unhandled exceptions, I also include ones which are "handled globally", i.e., in a case of a desktop applications, the global handling which consists of displaying a custom window instead of the Windows default one.

Context

When I work as a freelancer, I use my own in-house solution to gather unhandled exceptions from different sources (web apps and desktop apps). The gathered results are then displayed on a monitoring panel in real time as well as collected for future analysis (to be linked with a bug tracking software, etc.)

Currently, I work in a company where I wouldn't be able to use my in-house solution to collect the exceptions (one of the reasons being that they won't accept to send all the exception messages to my servers).

This company doesn't have any precise strategy for collecting unhandled exceptions. The only solution which was used before is both rudimentary and out of question: it consists of sending every exception by e-mail.

This means that for the new product I'm working on, we should develop a custom strategy for collecting unhandled exceptions.

Question

I can always do by hand the part which will save the exceptions to the database or a log and the part which will load them from the database, a log or Windows Events.

I would like to avoid reinventing the wheel and use something which is already commonly used.

What are my choices? How are unhandled exceptions usually collected and processed later?

By the way:

Are there any libraries which help collecting those exceptions?

Are there any software products which help analyzing those exceptions?


答:
When should a database be preferred for storing the data over storing the data in a text file?

Wikipedia tells us that a database is an organized collection of data. By that measure, your text file is a database. It goes on to say:

The data are typically organized to model relevant aspects of reality in a way that supports processes requiring this information. For
  example, modeling the availability of rooms in hotels in a way that
  supports finding a hotel with vacancies.

That part is subjective -- it doesn't tell us specifically how the data should be modeled or what operations need to be optimized. Your text file consists of a number of distinct records, one for each day, so you're modeling an aspect of reality in a way that's relevant to your problem.

I realize that when you say "database" you're probably thinking of some sort of relational database management system, but thinking of your text file as a database changes your question from "when should I use a database?" to "what kind of database should I use?" Seeing things in that light makes the answer easier to see: use a better database when the one you've got no longer meets your requirements.

If your Python script and simple text file work well enough, there's no need to change. With only one new record per day and computers getting faster each year, I suspect that your current solution could be viable for a long time. A decade's worth of data would give you only 3650 records that, once parsed, would probably require less than 75 kilobytes.

Imagine that instead of one small record per day, you decided to record every question asked on CodeReview, who asked it, and when. Furthermore, you also collect all the answers and the relevant metadata. You could store all that in a text file, but a flat file would make it difficult to find information when you needed it. There'd be too much data to read the whole thing into memory, so whenever you wanted to find a question or answer, you'd have to scan through the file until you found what you were looking for. When you wanted to find all the questions asked by a given user, you'd have to scan through the entire file. If you wanted to find all the questions that have "bugs" as a tag, you'd have to scan through the file.

That'd be horribly slow, so you might decide to speed things up by building some indexes that tell you where to look in the file to find a given record. You could have an index for questions, another for users, a third for answers, and so on. When you wanted to find a question you'd search the (much smaller) question index, get the position of the question in the main data file, and jump quickly to the right spot in the file. That'd be a big performance improvement. Indeed, that's pretty much what a database management system is.

So, use a DBMS when it's what you need. Use it when you have a lot of data, when you need to be able to access that data quickly and perhaps in ways that you can't entirely predict at the outset. If you have different kinds of data -- different types of records -- that are connected to each other, use a RDBMS so that you can relate the various records appropriately.


答: This will of course be a judgement call, but the three main criteria I would consider are: does it need to be ACID compliant,  how complex the data is and finally, how many things need to read/write it. As long as you are simply reading and writing one line per and your app is the only app doing either reading or writing, you can probably skip the database.  Once you start having multiple apps either reading or writing or your data structure becomes complex (particularly if it has relationships between seperate lines) then a DB starts looking really attractive.</p>


问:
Instantiating Null Objects with Null-Coalescing Operator
Consider the following typical scenario:

```<code>if(myObject == null) {
    myObject = new myClass();
}
</code>```

I'm wondering what is thought of the following replacement using the null-coalescing operator:

```<code>myObject = myObject ?? new myClass();
</code>```

I'm not sure whether I should be using the second form. It seems like a nice shorthand, but the myObject = myObject construct at the beginning seems like it could be a bit of a code-smell.

Is this a reasonable thing to do, or is there a better shorthand that I am missing?  Or maybe, "It's three lines, get over it!"?

Edit:
As has been mentioned, perhaps calling this a typical scenario is something of an overstatement. I usually find that I encounter this situation when I'm retrieving an entity from a database that has a child reference type property that may or may not be populated yet:

```<code>myClass myObject = myClassService.getById(id);
myObject.myChildObject = myObject.myChildObject ?? new myChildClass();
</code>```

答:
I use the null coalescing operator all of the time.  I like the conciseness of it.

I find this operator to be similar in nature to the ternary operator (A ? B : C).  It takes a little practice before the reading of it is second nature, but once you're used to it I feel readability improves over the longhand versions.

Also, the situation you describe is only one scenario where the operator is useful.  It's also handy to replace constructs like this:

```<code>if (value != null)
{
    return value;
}
else
{
    return otherValue;
}
</code>```

or

```<code>return value != null ? value : otherValue;
</code>```

with

```<code>return value ?? otherValue;
</code>```

Body:
I use the null coalescing operator all of the time.  I like the conciseness of it.

I find this operator to be similar in nature to the ternary operator (A ? B : C).  It takes a little practice before the reading of it is second nature, but once you're used to it I feel readability improves over the longhand versions.

Also, the situation you describe is only one scenario where the operator is useful.  It's also handy to replace constructs like this:

if (value != null)
{
    return value;
}
else
{
    return otherValue;
}

or

return value != null ? value : otherValue;

with

return value ?? otherValue;


问:
Client Server System in .Net
I am not asking for code but rather design ideas. I am trying to develop a new system to learn more about client/server development.

I would have 3 systems:

Client 1 : Client

Server 1 : Server

DB Server 1: DB Server

I would develop a server application as well as a client application

What happens is that my idea would be to use DB authentication where the clients actually have different access rights which are stored in a DB table on the DB server.

However, different access rights would get access to different features in the application. For example, a superuser would log in with his account by sending data from the client to the Server application running on Server 1, which would then check the superuser table in the DB to check if the account exists.

My initial idea would be to send the account object back to the client in which the client application would check what type of account it belongs to and actually displays the corresponding form/User Interface to the user.

However, this approach does not seem very feasible as it seems too flimsy and incorporates a lot of bad design.

Perhaps someone could give me a better idea on how do i approach this problem.

Summary : It would be just like a client server application but there would be different kinds of users being allowed different rights and different features.

答:
Your initial idea seems OK to me. One way or another, the client needs to respond to the role membership, and sending back information about the role membership is one solution to the problem. In fact it probably is a simple and effective one.

Another approach would be to implement some inversion of control patterns and simply let the server instruct the client what needs to be displayed as a matter of course. Under this model, the client doesn't know anything about the details of the UI in the first place, and always just displays whatever GUI the server instructs it to. The changes in the GUI could be on the basis of role membership, then, or any other situation the server detects (say, global policy changes, service status, etc.) This is more complicated, however, and may not be right for your situation.

Either way, though, one thing is very important: always make sure that the access check is performed on the server even if it has theoretically already been checked on the client. Do not simply trust input or actions from the client because you assume such checks were already validly and correctly performed-- they could be spoofed by malicious actors or simply reflect out-of-date permissions information. This applies to either model.

OwnerDisplayName: aceminer

Title: Client Server System in .Net