问:
Phrasing a requirement about filename encodings
I am in the process of writing a requirements spec, and I have a dilemma in phrasing a piece of the requirements.

Scenario: We download files from a website and the downloaded files need to be attached to an item in the CM tool we have. The downloaded files contain names which can be ASCII, ISO-8859-1, Japanese, etc.

In the phrasing below, does "non-ASCII" cover all situations?

The downloaded file name may contain non-ASCII characters and processing of this shall not crash the application

答:
The requirement that you've written doesn't have the characteristics of a good requirement. Specifically, it's not cohesive, it's not atomic, and it's not unambiguous. Because of the lack of these characteristics, it's also not easily verifiable.

Your initial state requirement is:

The downloaded file name may contain non-ASCII characters and processing of this shall not crash the application

I would recommend removing the "...and processing of this shall not crash the application". If you have a requirement that a piece of software needs to do something, I think it's OK to make the assumption that it should do it without crashing the software.

This transforms the requirement into:

The downloaded file name may contain non-ASCII characters

Now, you have a cohesive and atomic requirement. However, I'm not sure that it's unambiguous. In your question, you mention a number of different formats. There are a few options.

Some would recommend a separate and unique requirement for each file name encoding that must be supported. This would best support cohesive, atomic, traceable, unambiguous, and verifiable requirements. It would also make it easier to specify importance of each requirement - perhaps support for some encodings are more important or needed sooner.

Others may recommend a table of supported formats and this requirement would link to a table. It would be less complete (you have a textual sentence and a table to be maintained), but they would be in the same document or database. However, if you were going to perform linking in a requirements management tool, they could be linked together so that changes to one would highlight the linked requirement. It would also allow the text to flow to other software packages as is, but with a different table for different encodings.

How you document the requirements does depend on your specific needs, though.


答:
There are a couple of issues with your wording that weaken the requirement:

1) You should express the requirement in positive terms, rather than in terms of what it should not do. How does one test for "not crashing".

2) The phrase "The downloaded file name may contain..." is vague.

A suggested alternative wording (purely subjective, of course) might be:

The application shall support downloaded file names containing non-ASCII characters.

(The word "support" is still a little vague and could be changed to be more concrete when taken in concert with other requirements for your application.)

ContentLicense: CC BY-SA 3.0

Title: Phrasing a requirement about filename encodings

Body:
I am in the process of writing a requirements spec, and I have a dilemma in phrasing a piece of the requirements.

Scenario: We download files from a website and the downloaded files need to be attached to an item in the CM tool we have. The downloaded files contain names which can be ASCII, ISO-8859-1, Japanese, etc.

In the phrasing below, does "non-ASCII" cover all situations?

The downloaded file name may contain non-ASCII characters and processing of this shall not crash the application


问:
'import module' vs. 'from module import function'
I have always been using this method:

```<code>from sys import argv
</code>```

and use argv with just argv. But there is a convention of using this:

```<code>import sys
</code>```

and using the argv by sys.argv

The second method makes the code self documented and I (really) adhere to it. But the reason I prefer first method is it is fast because we are importing only the function that is needed rather than import the whole module (which contains more useless functions which python will waste time importing them). Note that I need just argv and all other functions from sys are useless to me.

So my questions are. Does the first method really makes the script fast? Which method is preferred most? Why?

答:
In my opinion using regular import improves readability. When reviewing Python code I like seeing where the given function or class comes from right where it is used. It saves me from scrolling to the top of the module to get that info.

As for the long module names I just use the as keyword and give them short aliases:

```<code>import collections as col
import foomodule as foo
import twisted.internet.protocol as twip

my_dict = col.defaultdict()
foo.FooBar()
twip_fac = twip.Factory()
</code>```

As an exception I always use the from module import something notation when I deal with the __future__ module. You just can't do it another way when you want all strings to be unicode by default in Python 2, e.g.

```<code>from __future__ import unicode_literals
from __future__ import print_function
</code>```

Body:
In my opinion using regular import improves readability. When reviewing Python code I like seeing where the given function or class comes from right where it is used. It saves me from scrolling to the top of the module to get that info.

As for the long module names I just use the as keyword and give them short aliases:

import collections as col
import foomodule as foo
import twisted.internet.protocol as twip

my_dict = col.defaultdict()
foo.FooBar()
twip_fac = twip.Factory()

As an exception I always use the from module import something notation when I deal with the __future__ module. You just can't do it another way when you want all strings to be unicode by default in Python 2, e.g.

from __future__ import unicode_literals
from __future__ import print_function


问:
In ifs inside for loops, prefer checking for true, or for false and continue?
I'm discussing this with a work colleague.
Say we want to sum the numbers from 0 to 9 skipping 5.
He prefers this:

```<code>int sum = 0;
for(int i = 0; i < 10; ++i)
{
    if(i == 5)
    {
        continue;
    }

    sum += i;
}
</code>```

I prefer this:

```<code>int sum = 0;
for(int i = 0; i < 10; ++i)
{
    if(i != 5)
    {
        sum += i;
    }
}
</code>```

Any reason to prefer one over the other?
Cyclomatic complexity is the same in both cases, right?

答:
A for loops natural tendency is to "continue" and as others have pointed out, it's an extra line of code.

In 15 years of writing code I have MAYBE used continue (in any language) 3 times; and none of that was recent. It's simply not a necessary construct as you quickly realize.


问:
Writing Unit Tests in the Middle
Is unit testing a 100% or not at all kind of deal?

I was browsing through my old projects and started adding features, this time with unit testing.  However, is this ultimately worthless if I'm going to be re-using past components that do not have unit tests?

Do I need to write unit tests for all the previous classes and not bother at all, or is it OK to only write unit tests for the new stuff I'm adding?

答:
I have worked on a very large code base which initially had no unit tests. By following a few practices we now (after several years) have most of the code base covered by tests.

All new code must have unit tests.

All changed code must have unit tests added to it.

The way that we safely added tests to old code without breaking it is primarily to use the following basic approach:

Choose a small section of code that you need to change the functionality of.

Try to create system level integration tests to surround the code. Because of the combinatorial complexity of testing at this level, these tests will only form a "smoke" test to pick up major mistakes.

Introduce the interfaces you need in order to be able to test the code you are changing.  Use Refactoring techniques consisting of sequences of very small changes which you have high confidence are correct. Try to use tool support where possible. You might do this by, for example, moving/extracting the method you are changing onto its own object. Check in your changes regularly so you can revert. Regularly peer-review how you made the changes by going through the revision control history.

Try to make the minimum about of changes that are required in order to break the dependancies that are preventing you from adding tests.

Write tests to as far as possible cover the functionality of the code that you are going to change. Check in regularly and peer review all changes.

Write tests for the new functionality/functionality change.

Implement the functionality (this is your normal TDD cycle)

Make sure to refactor the areas you have covered by the tests (red-green-refactor).

We found that the more we did this, the easier it got. As every time you go back to the code base, it is a little bit better.

We have seen a massive drop in the number of bugs getting through to our QA testers. With functionality regressions being now almost unheard of, so I think it was worth the effort for us.

ContentLicense: CC BY-SA 2.5

Title: Writing Unit Tests in the Middle


答:
As it stands right now (a single condition) I don't see much reason to prefer one over the other, but I'd probably prefer the second as slightly simpler.

When, however, your set of conditions are more complex, some variant of the first can be preferable, often by a wide margin. Consider something like:

```<code>for (...) {
    if (i != 5 && (i%2!=0) && i>99 && i<1000 && (i%17 !=3))
        sum += i;
}
</code>```

vs:

```<code>for (...) {
    if (i==5)
        continue;
    if (i%2 == 0)
        continue;
    if (i<100 || i > 999)
        continue;
    if (i % 17 == 3)
        continue;
    process(i);
}
</code>```

In this case, the latter seems (to me) more understandable and maintainable. It lets us group the items that are logically connected (the range, 100...1000) together, but keep those that aren't so closely related separate so each can be analyzed on its own more easily. It also fits closely with a pattern I use in quite a few functions that basically works out to something like:

```<code>function(x, y, z) {
    if (!preconditions(x, y, z))
        // return an error or (preferably) throw an exception

    // now do processing on x, y, z
}
</code>```

Although in your case it's a loop body instead of a function per se, you still end up with the same basic pattern: assure that some set of preconditions is met, and then if (and only if) it s, process the data.

Title: In ifs inside for loops, prefer checking for true, or for false and continue?

Body:
I'm discussing this with a work colleague.
Say we want to sum the numbers from 0 to 9 skipping 5.
He prefers this:

```<code>int sum = 0;
for(int i = 0; i < 10; ++i)
{
    if(i == 5)
    {
        continue;
    }

    sum += i;
}
</code>```

I prefer this:

```<code>int sum = 0;
for(int i = 0; i < 10; ++i)
{
    if(i != 5)
    {
        sum += i;
    }
}
</code>```

Any reason to prefer one over the other?
Cyclomatic complexity is the same in both cases, right?

Body:
As it stands right now (a single condition) I don't see much reason to prefer one over the other, but I'd probably prefer the second as slightly simpler.

When, however, your set of conditions are more complex, some variant of the first can be preferable, often by a wide margin. Consider something like:

for (...) {
    if (i != 5 && (i%2!=0) && i>99 && i<1000 && (i%17 !=3))
        sum += i;
}

vs:

for (...) {
    if (i==5)
        continue;
    if (i%2 == 0)
        continue;
    if (i<100 || i > 999)
        continue;
    if (i % 17 == 3)
        continue;
    process(i);
}

In this case, the latter seems (to me) more understandable and maintainable. It lets us group the items that are logically connected (the range, 100...1000) together, but keep those that aren't so closely related separate so each can be analyzed on its own more easily. It also fits closely with a pattern I use in quite a few functions that basically works out to something like:

function(x, y, z) {
    if (!preconditions(x, y, z))
        // return an error or (preferably) throw an exception

    // now do processing on x, y, z
}

Although in your case it's a loop body instead of a function per se, you still end up with the same basic pattern: assure that some set of preconditions is met, and then if (and only if) it s, process the data.


问:
Is it premature optimization to add database indices?
A colleague of mine today suggested that we go through all of the queries in our application and to add indices accordingly.

I feel this is premature optimisation because our application is not even released yet. I suggested to monitor for slow queries once we go live and then add indices accordingly.

What is the general consensus when designing your database, should you add a matching index every time you write a new query? Or is it better to just monitor and see how it goes?

答:
Although I don't think every query should be optimized, indexes are so much a part of RDBMS that they need to be given consideration before releasing. When you execute a query, unlike other forms of programming you're not telling the system how to execute it. They develop own plans and almost always base it on the availability of an index. The makeup and volume of data will be considered as well at later times.

Here are some things I would consider:

There are some queries that you should identify in your early development that you just know will be used frequently. Focus on them.

There will be slow queries. By indexing them first, you can then determine if the performance still isn't fast enough and then consider a redesign (Denormalizing may be premature). I'd rather do this before a release. Nobody wants a system where it takes 10 minutes to find something in the inventory.

Indexes may improve query performance but they're not to hinder data modification.

Many systems have tools to analyze your queries, so don't be afraid to use them.

After your initial review, you should follow it up with some considerations for when you should review this again and how you're going to be able to collect the information to do this (monitor usage, get copies of client data , etc.).

I realize you don't want to prematurely optimize, but it's almost certain you will have poor performance without indexing your database. By getting this out of the way, you can determine if there are other areas causing performance issues.


问:
Releasing open source software too soon
What is the moral responsibility of releasing open source software too soon?  For instance, a close-to-complete product that hasn't been fully tested.

What is the expectation of the programmer? Wait until it is fully tested, or release to open source and then continue further development, testing, and advancements?

The fear is that software is open sourced and could potentially lead to issues for consumers.

Is this an unfounded fear?

答:
My experience is that there's a balance to be achieved.

Right now, I'm working (in the sense of answering questions and providing development suggestions, without seeing any code) with a developer who's producing what looks to be a very exciting FOSS project that utilizes code I've written. The public release has been repeatedly delayed by realizations of design changes that will make the project much better in the long term, but which require significant rewrites of code that's already been written and that was already "working". My view is that, had a working but imperfect release been made as soon as there was something working to show, ideas for changes (and actual patches) could have come from the broader community that's interested in this project an accelerated it forward rather than having the issues surface slowly, one-at-a-time, as the developer thinks of them and asks for design feedback from myself and other interested members of the community. So from this standpoint, I'm very much an advocate for "release early, release often".

On the other hand, low-quality releases can make a new project look bad before it even gets off the ground. Some pitfalls I've seen include:

Skeleton trees with interface definitions but no code.

Code that doesn't successfully compile for anyone but the developer.

No instructions for how to build/run the program.

No documentation of what aspects can be expected to work.

Unclear description of what the program even does or will do.

Lack of any demonstration of usefulness.

For the last point, I'm thinking of things like:

Compiler/interpreter that can't even compile/run a hello-world type program.

Emulator that can't at least run a sample program of some sort or otherwise demonstrate that it's doing something.

Image processing tool that can't do anything but load and resave the unmodified image.

Game with nothing but a title screen.

These kinds of problems lend to a "vaporware" image that can be hard to shake unless you're very open about the lack of working code to begin with.

Finally, make your version numbers make sense. Don't call your project "1.0" until it does what users expect it to do without crashing. I've always had luck with using version numbers around "0.5" for initial public release and going from there, but I've also seen things like "0.1" or "0.10" that make sense.

Title: Releasing open source software too soon


问:
Team member questioning moving from VBA to C#
Background

Last year, I was asked to create a tool to be used for business planning for around 10 users. This was done on behalf of another IT team who "sub-contracted" the work to me, and due to the project deadlines being a little unplanned on their side, I had to implement it in a bit of a rush.

At the time, we decided that the quickest way would be to create an Excel workbook with VBA and then have the users download this VBA-enhanced workbook from an Intranet to use on their PCs. Excel was a constraint in this case because the planning system (i.e. database) we use can only interact via an Excel add-in which must be loaded at the same time the planning workbook is open. However, VBA was not a constraint at that time.

The workbook I created around 4,000 lines of VBA code and whilst I tried to separate data and presentation layers, I couldn't in all cases due to the project deadlines. To be honest, whilst I am proud of creating this workbook, I am at the same time a little disappointed in that it could've been done better, in both terms of coding and also deployment to the users.

Today

Back to today and the IT team has again come to me to request a similar workbook (so I could reuse parts of the other workbook above), but this time it is a lot more complicated and will be used by a greater number of users (around 200).

However, this time, it is a little better planned and I can see that we have a bit more time to plan things. Based on this, I thought about the solution and infrastructure as programming for 100 users has more of an impact than for 10 users. Therefore, I suggested to the team that perhaps we should consider migrating the existing code to a C# solution so that we could manage the code in a more refined way. I'm still considering it as an add-in written using VSTO / Excel-DNA which can then be deployed to the users.

I discussed this with the IT team two weeks ago and everything seemed to be fine, until yesterday I received a mail from one of the team (who does not know VBA or C#) questioning why we should start this new project in C# versus using the same approach as before. Some of their concerns were:

It is a fairly important project so it has to work - a C# solution would not be as stable or work as well as the existing a VBA-based solution.

We would have to throw away what we [I] had done in the VBA solution and recreate it from scratch in C#.

Someone will have to support two separate solutions, one in VBA and one in C#. [actually, they currently do not have anyone for support, I usually step in].

Now, I can understand some of their concerns to some degree, but I need to come to a decision on next steps and what to go back to them with. Personally, I would like to implement in C# because I feel it would lend itself better to building an "Enterprise" solution like this. Furthermore, I would like to take this opportunity brush up on my C# skills as I am currently not as competent in C# as I am VBA and I'd like a project like this to take me to the "next level".

I prepared a list of points that I could use to try and convince them that a C# solution would be better for this project, this is what I have so far:

Unit testing.

Source control.

Code documentation - for knowledge transfer to other support persons.

Better coding conventions - can use things like ReSharper to enforce better naming and structure.

Better IDE - fewer mistakes due to error highlighting.

More modularity through assemblies - can promote re-use in future tools.

Managed deployment - can control who this tool is used by.

Question: What other points could I add to convince them? Or am I trying to bite off more than I can chew with this project? Should I just keep quiet and do it in VBA anyway?

I am aware that just moving to a new language because its "newer" or seen to be "cooler" should not be a basis for a decision and as such I have resisted to include it as a decision point - this is about facts.

Also, I am not asking for a literal comparison between C# and VBA as languages, as there are plenty of comparisons on SO.

答:
I would like to take this opportunity brush up on my C# skills as I am
  currently not as competent in C# as I am VBA and I'd like a project
  like this to take me to the "next level".

Be honest. Do you plan on sharing this information with the other people involved in making this decision? If not, I would place the entire blame on you if the project fails.  Since a similar project has already been created and put out into the field, everybody has formed what they expect to happen in their own mind. They know how long it took to build the last one and will believe you can create this one in less time (which may or may not be true based on the additional requirements).  Are you ready to take on this responsibility along with learning a new language?

I recommend porting the old app to C# ASAP. Maybe you can learn enough in the process before a decision is made. At least you'll achieve your goal of increasing your skills with the new language and still get the project done on time.

Then again, if you feel that strong about it and building the project is all resting on your shoulders, do it the way you want. It's always easier to get forgiveness than permission.


答:
It is a good practice to identify which columns definitely need an index by some upfront analysis.  There is a real risk of gradual or unexpected performance degradation in production as database size increases if you have absolutely no indices.  The situation you want to avoid is where a commonly run query requires scanning a large number of table rows.  It is not premature optimization to add indices to critical columns since you have much of the necessary information available and the potential performance differences are significant (orders of magnitude).  There are also situations where the benefit of indices is less clear or more dependent on the data - you can probably defer deciding for some of these cases.

Some questions you need to ask are:

What are design limits for the size of each table going to be?

If tables are always going to be small (say < 100 rows), it isn't a disaster if the database has to scan the entire table.  It may be beneficial to add an index, but this requires a little more expertise or measurement to determine.

How often will each query be run, and what is the required response time?

If the query is run infrequently and doesn't have strict response time requirements (e.g. report generation) and the number of rows is not huge, then it is probably fairly safe to defer adding indices.  Again, expertise or measurement can help tell if it's going to be beneficial.

Does the query require looking up the table by something aside from the primary key?  E.g. filtering by date range, joining on a foreign key?

If these queries are run frequently and touch tables with many rows, then you should seriously consider preemptively adding an index.  If you're not sure whether this is the case for a query, you can populate the database with a realistic amount of data, then look at the query plan.

Title: Is it premature optimization to add database indices?

Body:
A colleague of mine today suggested that we go through all of the queries in our application and to add indices accordingly.

I feel this is premature optimisation because our application is not even released yet. I suggested to monitor for slow queries once we go live and then add indices accordingly.

What is the general consensus when designing your database, should you add a matching index every time you write a new query? Or is it better to just monitor and see how it goes?


答:
My suggestion would be to extract that logic out into a method so that if you add more conditions or even change your condition it is easier to maintain in one place + it is more testable.

The reason I am suggesting that is because I am assuming this is not the actual problem but it is a more simplified version of your problem that you have narrowed it down to demo it here.

If, however, my assumptions are wrong then it would still be a good idea to extract it out because it makes it a lot more readable and understandable (mostly because of the method name and the variable names) - so that when you come back to it after a while it is still easy to grasp and know what you meant when you wrote it the first time.

```<code>    private void sumItUp()
    {
        int sum = 0;

        for (int numberToAddToSum = 0, i = 0; i < 10; numberToAddToSum = SkipIfNextOneIsFive(i), i++)
        {
            sum += numberToAddToSum;
        }
    }

    private int SkipIfNextOneIsFive(int i) { return i + 1 == 5 ? 0 : ++i; }
</code>```

Body:
My suggestion would be to extract that logic out into a method so that if you add more conditions or even change your condition it is easier to maintain in one place + it is more testable.

The reason I am suggesting that is because I am assuming this is not the actual problem but it is a more simplified version of your problem that you have narrowed it down to demo it here.

If, however, my assumptions are wrong then it would still be a good idea to extract it out because it makes it a lot more readable and understandable (mostly because of the method name and the variable names) - so that when you come back to it after a while it is still easy to grasp and know what you meant when you wrote it the first time.

    private void sumItUp()
    {
        int sum = 0;

        for (int numberToAddToSum = 0, i = 0; i < 10; numberToAddToSum = SkipIfNextOneIsFive(i), i++)
        {
            sum += numberToAddToSum;
        }
    }

    private int SkipIfNextOneIsFive(int i) { return i + 1 == 5 ? 0 : ++i; }


答:
There is one case when releasing free software can have negative consequences. Some specifications are licensed to the public on the condition that all implementations distributed to the public shall conform to the specification completely when first published. The publisher legally forbids you from distributing a work-in-progress implementation of the spec. Without a specific negotiated license from the spec's publisher, you must share it with nobody until it passes all tests. This forces a "cathedral model" (as Eric S. Raymond called it) on implementations of the spec.

One spec under such a license is the Java Language Specification. This restriction applies to developers of virtual machines compatible with the JVM, but fortunately not to developers of applications that run in a JVM.

The article "4 Shifty Details About Microsoft's 'Open Source' .NET" by Liu Qihao & Ciaran O'Riordan mentions the possibility of interpreting the Microsoft Patent Promise for .NET Libraries and Runtime Components to exclude incomplete implementations of the CLR in a similar manner. But again, this doesn't apply to applications that run in the CLR.

Title: Releasing open source software too soon

Body:
What is the moral responsibility of releasing open source software too soon?  For instance, a close-to-complete product that hasn't been fully tested.

What is the expectation of the programmer? Wait until it is fully tested, or release to open source and then continue further development, testing, and advancements?

The fear is that software is open sourced and could potentially lead to issues for consumers.

Is this an unfounded fear?


答:
The requirement, as stated, is fuzzy to me.

The first question I would have is: how many character encodings need to be supported?  Possible interpretations include:

Every encoding ever devised, including single-byte (e.g. ISO-8859-15), multibyte (e.g. Big5, Shift-JIS, HZ), and rare/weird ones (e.g. UTF-7, Punycode, EBCDIC).

That's obviously extreme.  How about just the minimum support, namely ISO-8859-1?

Just ISO-8859-1 seems weaselly.  How about just supporting modern best practices, namely Unicode as UTF-8?

If you don't specify which encodings you mean, then when an encoding-specific bug occurs, you and the implementor could have a fight and you'd both be right.  That is, by definition, the consequence of a fuzzy spec.

Going further, what does the software need to do with the filename, besides not crashing?  Should it…

Preserve the filename in its original encoding, byte-for-byte?

Normalize everything to Unicode?  If so, does it need to auto-detect the source encoding?  By what mechanism?

Store both the Unicode form and the original, just in case the normalization fails?

A better version of your requirement would be

The downloader must support filenames in various encodings, including at least ASCII, ISO-8859-1, ISO-8859-15, KOI8-R, UTF-8, Shift-JIS, EUC-JP, GB2312, and Big5.  If the web server response specifies an encoding, it must be respected.  (If the encoding is unspecified, ISO-8859-1 may be assumed, or a better guess may be made.)  Filenames shall be normalized to a Unicode representation in the content management system.

The specific examples of required encodings are essential for devising acceptance criteria.  The added sentences state what the software needs to do, beyond not crashing.


问:
Addition or deletion of elements of a dynamic array
Is there any consensus among programmers (or a common convention) on the "right way" to deal with the addition or deletion of one or more elements of a dynamic (mutable) array at runtime while gracefully handling changes to the references to the other elements?

For example within a loop in JavaScript, to remove some elements

does one use delete() to replace the items with 'undefined' while keeping the length of the array unchanged, so that references to the other elements have not changed?

Or do you slice() or splice(x,1) the array and then somehow update all current references?

Or do you make a copy of the array at the beginning of a function call, for example, then use delete() while still in that function, then at the end use filter() to remove the undefined slots and then write that back out to the original object?

What if more than one user or thread is trying to access this array simultaneously? Is the only choice a locking mechanism for writes? Or perhaps the answer is simply, "it just depends on what you want/need it to do".

Thanks in advance for your opinions.

答:
There are multiple ways to deal with this and each has benefits and drawbacks. You need to pick the right approach depending on how the data structure should work.

Use single-threaded access. This removes a large class of problems but is not always feasible. Specifically, if you save array offsets, iterators, etc. they can be invalidated even by a single thread. E.g. save off index 7, then delete index 4, now the element that was at index 7 is now at 6. This can be fixed by refactoring the code to work better and more consistently, such as by not searching for the element until you need it.

Control access to the array using a locking mechanism. Want to access or modify the array? Use a semaphore or other mechanism to ensure only one thread can access it. This goes a long way toward ensuring safety, but not entirely. It has the same potential single-threaded bugs as the previous point.

Wrap the array in a thread-safe data structure. One example is a copy-on-write list. This ensures that writes by one thread (or reference) are only seen by that same object reference. The benefit is two processes modifying the same structure do not step on each others' toes, with the obvious drawback that different threads cannot truly share data because once a thread modifies the list, it receives its own copy that is now out of sync with other threads.

Use a different data structure. For example, a hash table that happens to use integers as keys operates similar to an array, but does not have the "I deleted or inserted and shifted everything" problem. This may or may not be appropriate for a given task, but may be worth considering. This is also a good alternative if you need a sparse array.


问:
Concurrency: How do you approach the design and debug the implementation?
I've been developing concurrent systems for several years now, and I have a pretty good grasp on the subject despite my lack of formal training (i.e. no degree).  There's a few new languages that have become popular to at least talk about lately that are designed to make concurrency easier such as Erlang and Go.  It appears that their approach to concurrency echoes my own experience as to how to make systems scalable and take advantage of multiple cores/processors/machines.

However, I find that there are very few tools to help visualize what you intend to do, and verify that you are at least close to your original vision.  Debugging concurrent code can be a nightmare with languages that are not designed for concurrency (like C/C++, C#, Java, etc.).  In particular, it can be near impossible to recreate conditions that happen readily on one system in your development environment.

So, what are your approaches to designing a system to deal with concurrency and parallel processing?  Examples:

How do you figure out what can be made concurrent vs. what has to be sequential?

How do you reproduce error conditions and view what is happening as the application executes?

How do you visualize the interactions between the different concurrent parts of the application?

I have my own answers for some of these, but I'd also like to learn a bit more.

Edit

So far we have a lot of good input.  Many of the articles linked to are very good, and I've already read some of them.

My personal experience with concurrent programming leads me to believe you need a different mindset than you do with sequential programming.  The mental divide is probably as wide as the difference between object oriented programming and procedural programming.  I'd like this set of questions to focus more on the thought processes necessary (i.e. theory) to systematically approach the answers.  When providing more concrete answers, it helps to provide an example--something you went through personally.

Goal for the Bounty

Don't tell me what I should do.  I already have that under control.  Tell me what you do.  Tell me how you solve these problems.

答:
How do you figure out what can be made concurrent vs. what has to be sequential?

I would question first whether the application (or component) will actually see a benefit from concurrent processing, or in layman's terms -- where is the bottleneck?  Concurrency will obviously not always provide a benefit for the investment it takes to make it work. If it looks like a candidate, then I would work bottom up -- trying to find the largest operation or set of operations that can do its work effectively in isolation -- I don't want to spin up threads for insignificant, cost-ineffective operations -- I'm looking for Actors.

Working with Erlang I've come to absolutely love the concept of using asynchronous message passing and the actor model for concurrency -- it's intuitive, effective, and clean.

Off of Understanding Actor Concurrency

The actor model consists of a few key principles:

No shared state

Lightweight processes

Asynchronous message-passing

Mailboxes to buffer incoming messages

Mailbox processing with pattern matching

An actor is a process that executes a function. Here a process is a lightweight user-space thread (not to be confused with a typical heavyweight operating-system process). Actors never share state and thus never need to compete for locks for access to shared data. Instead, actors share data by sending messages that are immutable. Immutable data cannot be modified, so reads do not require a lock.

The Erlang concurrency model is easier to understand and debug than locking and shared data.  The way in which your logic is isolated makes is easy to do testing of components by passing them messages.

Working with concurrent systems this is pretty much how my design worked anyway in any language -- a queue that multiple threads would pull data from, perform a simple operation and repeat or push back onto the queue.  Erlang is just enforcing immutable data structures to prevent side-effects and reducing the cost and complexity of creating new threads.

This model is not Erlang exclusive, even within the Java and .NET world there exists ways to create this -- I would look at the Concurrency and Coordination Runtime (CCR) and Relang (there is also Jetlang for Java).

How do you reproduce error conditions and view what is happening as the application executes?

In my experience, the only thing I can have done is given a commitment to tracing / logging everything.  Every process / thread needs to have an identifier and each new unit of work needs to have a correlation id.  You need to be able to look through your logs and trace exactly what was being processed and when -- there's no magic I've seen to eliminate this.

How do you visualize the interactions between the different concurrent parts of the application?

See above, it's ugly but it works.  The only other thing I do is to use UML sequence diagrams -- of course this is during design time -- but you can use them to verify that your components are speaking the way that you want them too.


问:
Is Pair programming works good when it come to backend developer and frontend developer working together?
If there's a work that need frontend and backend developers to work together, should they:

Pair programming all the times? When it's backend work, frontend developer is navigator, backend developer is driver and vice-versa. This would most time-consume and I don't really sure it's has benefit enough for all time wasted.

Pair programming with only some part like only API part that need to communicate the data with each other. Frontend may need to know some database structure to be able to pair with backend too.

Just do the API Documentation together and after that, just do their own work base on the API Documentation. (Fastest way but I don't know if there will be any drawback).

P.S. They work at the same place and can ask the others anytime when they do not understand something.

答:
It Depends

I worked in a very small shop where we were all on billable hours. You were assigned the entire task front-end, back-end (which included any database stuff) and the specified hours. After a year or so of doing all the tasks individually we decided to team up and work together. I was much more suited for javascript error trapping and backend stuff especially database work. My buddy was much more suited for front end stuff and configuration.

We decided to team up, pool our work and hours. It worked out great.
I focused on backend stuff building my pieces so I could plop them inside a set of

 tags.

My buddy did the layout work knowing all I needed was as set of

 tags.

I would then jump on all the javascript error tapping and he did all the configuration.

Each of us focused on what we were stronger at. More work got done with less stress.

Body:
It Depends

I worked in a very small shop where we were all on billable hours. You were assigned the entire task front-end, back-end (which included any database stuff) and the specified hours. After a year or so of doing all the tasks individually we decided to team up and work together. I was much more suited for javascript error trapping and backend stuff especially database work. My buddy was much more suited for front end stuff and configuration.

We decided to team up, pool our work and hours. It worked out great.
I focused on backend stuff building my pieces so I could plop them inside a set of <div></div> tags.

My buddy did the layout work knowing all I needed was as set of <div></div> tags.

I would then jump on all the javascript error tapping and he did all the configuration.

Each of us focused on what we were stronger at. More work got done with less stress.


问:
Bottlenecks, nginx as a load balancer
Why nginx can keep itself from being a bottleneck when serving as a load balancer. If it becomes the single point bottleneck in some conditions, then is there solution except hardware load balancer? </p>

答:
When you use a software load balancer or caching proxy, that load balancer or caching proxy is a bottleneck – but it's a much wider bottleneck than before.

Assume we have an application server that can only handle 50 requests per second, since it has to query the database and do all kinds of expensive stuff. How can we scale?

If we add a second application server, that won't help. We need a load balancer that distributes load between these application servers. Because the load balancer itself does nothing except passing through connections, let's say that it can handle 1000 requests per second – that's 20× as much, and my actual capacity is 2·50=100 requests per seconds since I have two application servers. I can now add further application servers if the need arises.

In most cases, this is absolutely sufficient, and it's unlikely that you'll push the load balancer to its limits. If you want to scale beyond this load balancer, you cannot add another software load balancer in front of the existing load balancer, and you will have to beef up the hardware (or switch to a balancer with better performance).

If you get beyond the capacity for a single load balancer (e.g. when you hit the millions of daily users range) and if you need to maintain servers in multiple geographic locations, then you can use a DNS server that resolves your domain name to different IP addresses. Content Delivery Networks use this technique to distribute load through a large number of distributed proxies.
Instead of going that route yourself, it will likely be much easier to put your servers behind an existing commercial CDN that handles all that for you.

Title: Bottlenecks, nginx as a load balancer


问:
Black box or white box testing - which do you do first?
On a very small team, where the black box and white box testing is done by the same person, which should the tester do first?</p>

答:
Neither. I try to write good tests using my Right BICEP, keeping in mind CORRECT boundary conditions no matter what order they come to mind. Those are both acronyms proposed in Pragmatic Unit Testing.

My goal is to focus on writing good tests, and not which colour to write first.


问:
What architecture/design to adopt when consuming a RESTful API? (iOS App)

TL;DR

Part 1)

I want to refactor the biggest part of the app on which I work - and it is pretty much spaghetti. It's a single class that makes the requests to the server, parses the JSON, and saves the data in the database (it also has knowledge of some UI that needs to change when specific business rules are met and it definitely shouldn't know all this stuff). It has somewhere below 5000 LOC :(.

One way of breaking the class in smaller pieces would be to separate the requests by semantics (for example User requests, Search requests etc). Would that be useful or not really?

Also, I think that networking code should be decoupled from the parsing code and also from the DB code. The problem is that I don't really know what approach (architecture/design pattern) should I use to do this the best way. (for example, our JSON parsing library is old and is not maintained anymore - we wanted to change it because it produced some problems and we were stuck because of the tight-coupling).

Part 2)

I also would like to receive some suggestions for the following issue:
We use the DB mostly for offline mode & caching. The user can see data in offline mode, modify it (and requests get serialized) and when he gets back online, the requests will be made to the server. We also use it as a cache to show something on the UI while we do a request in the background to see if there is new/modified data (we use a similar mechanism like ETag to check with the server).

So this is the flow when a user enters a screen:

```<code>enter_screen ->
check_cache + background_request ->
show_UI with cache data ->
update_UI when the background request finishes and new/modified data is available.
</code>```

Can this stuff be abstracted away somehow? I was thinking of some kind of delegate pattern where I get an immediate callback when I hit the cache and then get a later callback when the request finishes (with optional flags to distinguish between cache and fresh data).

A bit of a background story

I currently work for an iOS app that is aprox. 4 years old. I only joined the team a year ago. In my opinion, the code is ugly, full of hacks, and has a very shaky architecture. We are 2 iOS developers working on it. The client doesn't have the technical expertise and may not understand all the implications of some issues that we have. Making a client understand why "refactoring" is needed, is a hard task. I struggle to refactor how much I can, but we still have a lot to go trough.

The app also has a lot of features that increases the complexity of the app: deeplinks, push notifications, offline mode, non blocking UI etc. all combined with a rather large and hairy data model.

Because the client requests features in a constant manner, I feel that we have a lot of technical debt.

The mobile world is changing pretty fast and new technologies appear (Size classes, Auto Layout, Swift, etc.). This is true for all technologies, but I somehow feel that the race is a bit more intense in the mobile world. Currently, I only dream of using Swift at work (although  I play with it at home in my spare time).

答:
As Frank said, you appear to have figured out a lot of this on your own already, so I only have one piece of advice regarding the main question.

Do the refactoring gradually, incrementally, alongside whatever changes you make to add business value. Don't try to break the 5000 LOC monster into five 1000 LOC critters all in one go. As long as you keep in mind all the long-term goals you listed, the networking code, the JSON parsing, the database handling and whatnot will all steadily inch their way into separate self-contained corners of the codebase.

ContentLicense: CC BY-SA 4.0


答:
Both of those code fragments are doing way too much work.  You can replace them with int sum = 40; (in fact, if you gave those code fragments to a compiler, that's basically what the compiler would do).  There's a famous trick for summing up the first n integers I used to do it in my head, but if you didn't know the trick and had no calculator handy, you could work it out with pen and paper in about the same amount of time it would take to type up that code.  So the problem, as stated, isn't something you should be writing a program for at all.

Of course, you aren't asking about only that one example problem.  What problem are you actually trying to solve?  The question overspecifies a bit, so I'll just guess that you're either (a) going for something that sums up all the consecutive integers over a range except for a specified one, or (b) you're going for summing up all the consecutive integers in a range, except for a range in the middle.

If you want (a), then you can just subtract the specified number from the range.  If you want (b), then you can break the large range into a bottom half and a top half and deal with each separately.  So, in either case, we want something that can sum up a range of integers.

If you know the famous formula for summing up the first n integers (n*(n+1)/2), and you know how it works, it isn't hard to come up with a formula for a range ((top-bottom+1)*(top+bottom)/2).  If not, there's the obvious and straightforward way of doing it with for loops.

If we write a function that does that (with either method), we can rewrite the original problem as sum_range(0,9) - 5 or sum_range(0,4) + sum_range(6,9).  It turns out we didn't need any ifs or continues, and, depending on how we wrote it, maybe no for loop either.

If your question is really about which keyword to use when nested inside a construct using a specific keyword which is nested inside another construct using another specific keyword, then I don't think your question really has an answer, and that trying to figure out how to program by looking at which keywords nest inside which other keywords is never going to be useful.

(For those interested, the formula works like this: say we're trying to add up all the integers from 1 to 100.  1+100 = 101, 2+99 = 101, 3+98 = 101, and so on, so we could just multiply 101 by the number of pairs, and the number of pairs is the number of numbers being added / 2.  In this case, top = 100, bottom = 1, top+bottom = 101, and the number of pairs is (top-bottom+1)/2 = n/2 = 50.)


问:
Understanding dependency injection
I'm reading about dependency injection (DI). To me, it is a very complicated thing to do, as I was reading it was referencing inversion of control (IoC) as well and such I felt I was going to be in for a journey.

This is my understanding: Instead of creating a model in the class which also consumes it, you pass (inject) the model (already filled with interesting properties) to where it is needed (to a new class which could take it as a parameter in the constructor).

To me, this is just passing an argument. I must have miss understood the point? Maybe it becomes more obvious with bigger projects?

My understanding is non-DI (using pseudo code):

```<code>public void Start()
{
    MyClass class = new MyClass();
}

...

public MyClass()
{
    this.MyInterface = new MyInterface();
}
</code>```

And DI would be

```<code>public void Start()
{
    MyInterface myInterface = new MyInterface();
    MyClass class = new MyClass(myInterface);
}

...

public MyClass(MyInterface myInterface)
{
    this.MyInterface = myInterface;
}
</code>```

Could some one shed some light as I'm sure I'm in a muddle here.

答:
This is another option in addition to amon's answer

Use Builders:

```
class Foo {
    private Bar bar;
    private Qux qux;

    private Foo() {
    }

    public Foo(Builder builder) {
        this.bar = builder.bar;
        this.qux = builder.qux;
    }

    public static class Builder {
        private Bar bar;
        private Qux qux;
        public Buider setBar(Bar bar) {
            this.bar = bar;
            return this;
        }
        public Builder setQux(Qux qux) {
            this.qux = qux;
            return this;
        }
        public Foo build() {
            return new Foo(this);
        }
    }
}

// in production:
Foo foo = new Foo.Builder()
    .setBar(new Bar())
    .setQux(new Qux())
    .build();

// in test:
Foo testFoo = new Foo.Builder()
    .setBar(new BarMock())
    .setQux(new Qux())
    .build();
```

This is what I use for dependencies. I don't use any DI framework. They get in the way.

Title: Understanding dependency injection

Body:
This is another option in addition to amon's answer

Use Builders:

class Foo {
    private Bar bar;
    private Qux qux;

    private Foo() {
    }

    public Foo(Builder builder) {
        this.bar = builder.bar;
        this.qux = builder.qux;
    }

    public static class Builder {
        private Bar bar;
        private Qux qux;
        public Buider setBar(Bar bar) {
            this.bar = bar;
            return this;
        }
        public Builder setQux(Qux qux) {
            this.qux = qux;
            return this;
        }
        public Foo build() {
            return new Foo(this);
        }
    }
}

// in production:
Foo foo = new Foo.Builder()
    .setBar(new Bar())
    .setQux(new Qux())
    .build();

// in test:
Foo testFoo = new Foo.Builder()
    .setBar(new BarMock())
    .setQux(new Qux())
    .build();

This is what I use for dependencies. I don't use any DI framework. They get in the way.


问:
What is the difference between embedding and composing?
What is the difference between go's type embedding and object composition?

My understanding is that type embedding is object composition except the methods of the embedded type are automatically forwarded as they would be in inheritance.

答:
It's exactly as you described it: Go's embedding is composition with the added benefit of "forwarding" the interface.

In my mind embedding is an attempt to preserve the main benefit of traditional OOP inheritance--code reuse--while avoiding its greatest pitfall--extremely tight coupling. When you "inherit" a method via embedding, your "derived" class is not responsible for or even capable of reimplementing it, so it can't possibly "break the base class" somehow. And, since there are no "protected" or "private" methods that your "derived" class might gain access to, there's no violation of encapsulation. But you still have all the convenience of directly "inheriting" the "base class interface" as opposed to writing a bunch of wrapper functions.


问:
Is it normal for service methods to correlate to your views?
I've been working on learning how to implement domain services and view models in my project and have noticed a suspicious trend:

When obtaining data for views from my controllers, if my domain services are producing view models, I end up with one service method for each view model being requested.

I'm wondering if perhaps there's something that I might have misunderstood in how/where I create view models?  Should I be creating a special set of services in my project that compose my more inner domain services to build view models?

答:
The View model is a different bounded context from your domain model.
So you need a context map to map between them.

Your domain services should produce domain entities and value objects. The mapping to view models is a different responsability. If the domain service would produce view models, it cannot be reused.


答:
The problem with the spec as written is that it doesn't say what the application should do with "interesting" filenames.  I've encountered one program which would replace any filename characters it didn't understand with _, with the effect that when asked to copy a directory which contained two characters whose names were identical except in characters the utility didn't understand, the second file written to the directory would overwrite the first.  Such behavior would qualify as "not crashing", but that shouldn't imply that it's acceptable absent an explicit spec saying so.

I would suggest that a good spec should affirmatively specify what should happen, or else note what courses of action are acceptable, e.g. "If a file name contains unrecognized characters, the system should generate a new GUID for the overall operation, and generate a filename which combines that GUID, an index number, and any portion of the original filename that can readily be accommodated; it should produce a table mapping the old and new filenames" or "If a file name contains unrecognized characters, the system may form a new name by concatenating the characters it recognizes; if two file names end up becoming identical through such transformation, either one may arbitrarily be declared the 'winner'".


问:
Pem certificate in mobile app
I am developing a cross platform app that can handle payments, the online payment service   issued a certificate for my use, thus I have a .pem certificate and within it there is 2 actual certificates and my private key.

The problem is that I can't seem t find how to use it safely, as it has to go in the requests to the web service of the online payment service, therefore I have to include it in my mobile app package, but if anybody opens the package he or she will instantly have access to the private key.

How can I manage this situation? Should I just include it and forget about it as anyway an attacker will have to have valid credit card numbers, or should I create an intermediate web service that can add the certificate?

答:
The cardinal rule is never ever leave your private key in your app.  Only you should have access to your private key.  Web services, or any service, should only need your public key.

If they require access to your private key then I suspect that the setup is screwy or you're dealing with a shady service.

Title: Pem certificate in mobile app


问:
Naming conventions: camelCase versus underscore_case ? what are your thoughts about it?
I've been using underscore_case for about 2 years and I recently switched to camelCase because of the new job (been using the later one for about 2 months and I still think underscore_case is better suited for large projects where there are alot of programmers involved, mainly because the code is easier to read).

Now everybody at work uses camelCase because (so they say) the code looks more elegant .

What are you're thoughts about camelCase or underscore_case

p.s. please excuse my bad english

Edit

Some update first:

platform used is PHP (but I'm not expecting strict PHP platform related answers , anybody can share their thoughts on which would be the best to use , that's why I came here in the first place)

I use camelCase just as everibody else in the team (just as most of you recomend)

we use Zend Framework which also recommends camelCase

Some examples (related to PHP) :

Codeigniter framework recommends underscore_case , and honestly the code is easier to read .

ZF recomends camelCase and I'm not the only one who thinks ZF code is a tad harder to follow through.

So my question would be rephrased:

Let's take a case where you have the platform Foo which doesn't recommend any naming conventions and it's the team leader's choice to pick one. You are that team leader, why would you pick camelCase or why underscore_case?

p.s. thanks everybody for the prompt answers so far

答:
camelCase

This is one of the few places I will always choose 'type-ability' over readability. CamelCase is just easier to type, and being nicer to my fingers wins out over a slight readability gain for me.

assuming of course that the project does not build on an existing codebase with a different standard.


问:
How to make one test depend on the results of another test?
Let's say there is a utility class that provides some common static methods used everywhere in your code by many other classes.

How would you design your unit tests for the consumers of the utility so that their tests fail if any one of the utility tests do not pass? Can you do it or do you have to check it yourself whether the utility class tests are all green?

For example I have a message-splitter utility that is used (or rather its output) by a message-parser. I'd like to be sure that the message-splitter works correctly before the message-parser gets tested.

I have written tests for both of them but is there a way to link them and make one test depend on the result of some other test?

I couldn't find a suitable tag for this but I'm using Visual Studio's unit test engine.

答:
In a summary you don't want to use tools/techniques for doing things other than what they are meant for.

What you are trying to do sounds like a concern that could be solved easily with a CI (continuous integration) practice.

This way you can keep your tests atomic as already suggested and let the CI take care of verifying your tests.

If any tests fail you can set it so that it doesn't let your code to get published.


问:
Executing scripts in parallel based on dependency tree
We have an app that is using a fairly simple stack (Linux, PHP, Oracle, Shell Scripts, etc).

We have a series of scripts that need to be executed:

/scr/app1/start.sh

/scr/app2/start.php

/scr/app3/kickoff.py

/scr/app4/test.sh

The dependency rules are as follows:

app1 must be run first

app2 then must be run next

app3 and app4 can be run in parallel

That being said, I am looking to capture the following from ALL scripts:

exit codes (used to determine success of failure / breakage of chain)

standard out and/or standard error

The solution does not necessarily have to be written in PHP, it just has to work. Node.js / io.js is part of our build stack so we are comfortable building there.

Basically the parallelism is the real challenge that I see. What is the best approach to this problem while satisfying all of the requirements?

答:
So the "PHP" way to do it is via the pcntl (Process Control) extension. This extension is NOT supported on Windows, so it must be used in *nix environments only.

That being said, pcntl_fork is where the magic lies. While there is a third-party extension for threading (vs. forking) in PHP, it is not officially supported, not well tested and not being actively developed. I.e., don't use it.

Forking seems to have it's own set of problems (you can google forking vs. threading), but all around, it is the easiest and most simple / maintainable way to do parallel computing.

I'm writing a library that takes advantage of symfony/process to run processes and get the buffer. Previously, I used a StackOverflow answer to safely asynchronously run a process without any blocking in PHP. But I will be replacing it with symfony/process and editing this answer with a GitHub link when I publish it.

As for the dependency chain, the library I'm working on takes a json object (to keep things as simple as can be), which describes the sequence of processes to execute, including which to run in parallel.


问:
What do design and implementation mean?
The words "design" and "implementation" have been used widely. But I am not sure that I understand them.

What do design and implementation of a computer-related system (e.g.
a computer system, an OS, a programming language, a DBMS, a software
program) mean?

Think "design of a system" as a process, is its input the
requirements of the users of the system, and is its output the user
interface?

Think "implementation of a system" as a process, is its input a
design of the system, and is its output the system?

What are common in the design for a computer system, an OS, a
programming language, and a DBMS?

What are common in the implementation for a computer system, an OS,
a programming language, and a DBMS?

Thanks!

答:
I wouldn't get too hung up on the terminology.  Google has some good definitions:

Design:  (search for 'definition of design')

noun:

a plan or drawing produced to show the look and function or workings of a building, garment, or other object before it is built or made.

verb:

decide upon the look and functioning of (a building, garment, or other object), typically by making a detailed drawing of it.

Actual drawings sometimes exist (think flowcharts), but most designs I'm familiar with are typically written descriptions.

Implementation:

the process of putting a decision or plan into effect; execution.

As for your specific questions:

They mean pretty much what the dictionary says they mean.  "Designing a thing" means figuring out how it's going to work, possibly what it looks like, etc.  "The design" is the output of the process of "Designing a thing".  "Implementing a design" means actually doing the work to convert the idea (the design) into something real.

Short answer, "yes". I would substitute "designing a system" for "design of a system" and "implementing a system" for "implementation of a system", but you have the right idea.

If you asked this question by itself, it would be closed as 'too broad' within minutes.  The process of designing anything will have common steps including collecting requirements, identifying possible solutions, analyzing those solutions, etc.  Likewise, the process of implementing anything will also have certain steps in common including the actual construction, verification that the construction is correct, etc.

Title: What do design and implementation mean?


问:
Grouping and nesting of REST APIs
My question is around best practice of aggregating or grouping REST APIs.
I have a scenario where there are many different vendors, data sources, etc. and I think grouping REST APIs would make a lot of sense to keep the system maintainable.

I have many scenarios where there will be a single API call triggering many other (similar) API calls to create the same entity in another system. For example for an example entity "user":

Front-end calls REST API: PUT .../user

What I envision is that the code listening on the above API will then make multiple REST PUT calls to let's say vendorA/user, vendorB/user, vendorC/user, internalSystemA/user, internalSystemB/user, etc.

Like this:

```<code>                                                            +-------------------+
+--------+                   PUT     +-----------+  CALL    | Vendor A API      |
|        |                 +-------> | user      +--------> |                   |
|        |                 |         +-----------+          +-------------------+
|        |                 |
| Front  | PUT    +--------++  PUT   +-----------+  INSERT  +-------------------+
| End    +------> | user    +------> | user      +--------> | Internal System   |
|        |        +--------++        +-----------+          +-------------------+
|        |                 |
|        |                 |         +-----------+  CALL    +-------------------+
|        |                 +-------> | user      +--------> | Vendor B API      |
+--------+                   PUT     +-----------+          |                   |
                                                            +-------------------+
                 +                                 +
                 +---------------------------------+
                         Internal REST APIs
</code>```

Note that the example entity doesn't have to be "user", there are many entities that are going to have a counterpart on the vendor APIs.

The vendors in this scenario provide different functionality. But there might be multiple vendors who provide the same functionality (and the user will chose the vendor he wishes to use). For simplicity let's say example functionalities are

Procurement,

Human Resources,

Management,

Payment.

What is the best practice to group and nest REST APIs in such a scenario?
Is it a good idea to group them by vendor or should they be grouped functional or by business entity? How would the URL look like?

答:
I'd go for more logical grouping,

Imagine a system where hotel bookings are performed, where there are 2 different hotel reservation API (A and B) and one payment gateway.

In this sort of situation few best practices to follow would be,

Wrap any third party APIs into one of services (whenever third party API changes only the wrapper service is need to be changed), in this example single interface can be used for both A and B services.

Create higher level service facades based on functionality (in this case one facade for searching and one for booking)

This also will help to troubleshoot, because if there is an error it would be easy to track entire process since it happens in a single facade method

OwnerDisplayName: phpPhil

Title: Grouping and nesting of REST APIs


答:
The difference between "design" and "implementation" stem from a foolish idea people perpetuate that the approach to building software should be like building a bridge, where you "design" the bridge, then a bunch of construction workers "implement" it.

In reality, all software development is design.

2), Generally, kinda-sorta, yeah, that is what they are understood to be by the people who belong to the group above. Some people will consider requirements gathering to be part of design, others will not. The user interface is not necessarily the exclusive output of a design phase, even subscribing to the foolish concepts above. Many software platforms don't have UIs per-se, e.g. libraries.

3) The common elements of a "computer system" is that they use do math and move information around. That's really the only thing common. Not all projects use databases, not all have operating systems (e.g. embedded software), and some are programmed in assembly (which I would not lump in with programming languages considering the OPs understanding of the subject) as they are run on platforms that don't have C or other compilers available. Some computer systems even run on analog systems for very obscure and specialized purposes. Not all use microprocessors, as there are systems out there such as older elevator controllers, that are built out of relays but are very much a simple computer system.

Title: What do design and implementation mean?

Body:
The words "design" and "implementation" have been used widely. But I am not sure that I understand them.

What do design and implementation of a computer-related system (e.g.
a computer system, an OS, a programming language, a DBMS, a software
program) mean?

Think "design of a system" as a process, is its input the
requirements of the users of the system, and is its output the user
interface?

Think "implementation of a system" as a process, is its input a
design of the system, and is its output the system?

What are common in the design for a computer system, an OS, a
programming language, and a DBMS?

What are common in the implementation for a computer system, an OS,
a programming language, and a DBMS?

Thanks!


问:
What should every programmer know about programming?
Please, stay on technical issues, avoid behavior, cultural, career or political issues.

答:
Big O notation and its implications.

Some useful references

http://www.perlmonks.org/?node_id=227909

http://www.programmerinterview.com/datastruct/big-o-notation-example.php

http://en.wikipedia.org/wiki/Big_O_notation


问:
Will C compiler attach the whole object code of the header file?
When you compile a C source code does the compiler convert the whole header file to object code or just the functions of the header file you use?

Like for example, in the header file #include stdio.h, there are printf(); and scanf(); but even to use one of them, you have to include the header file "stdio.h", so when you compile a C source code which uses only printf(); but doesn't use scanf();, will the compiler also include the object code of scanf(); and other functions of "stdio.h" in the file?

答:
Indirectly, yes.

To know what happens to the code in the header file one must understand how the compiler handles them.

The #include directive

The #include directive is a preprocessor directive. Even before the compiler starts to analyze and translate the code it will process all #includes by inserting the whole code in the given header file to the position of the #include statement. (You can see what the preprocessor spits out when you use e.g. gcc -E ... on your source files.)

That is why include guards are important.

So the answer to your question is:
Yes, the code in header files is translated to an object file - to be precise to the object file of the first translated source file which includes it in a translation unit.

The definitions in the header files have to be known in every translation unit which uses the definitions. If you translate your programm file by file and link it afterwards, you will have the code in the header file compiled multiple times. That is why you should put the implementation to a separate source file. If you do that you only will declare each function in every translation unit but not translate the implementation. All files will then be linked to the one object file with the implementation.


答:
The compiler will generate code for everything.

The linker may (and usually does) not include code of unused functions in the final executable.

Implementation, of course, varies from linker to linker.

Title: Will C compiler attach the whole object code of the header file?

Body:
When you compile a C source code does the compiler convert the whole header file to object code or just the functions of the header file you use?

Like for example, in the header file #include stdio.h, there are printf(); and scanf(); but even to use one of them, you have to include the header file "stdio.h", so when you compile a C source code which uses only printf(); but doesn't use scanf();, will the compiler also include the object code of scanf(); and other functions of "stdio.h" in the file?


答:
The compiler doesn't do anything with other object files. It merely compiles code to object files one translation unit at a time. It's the linker that takes your new object file, looks for symbols like "printf", and then "attaches" the implementation of printf from stdio.h's object file. In theory the linker could throw in the whole stdio.h object file, but in practice it should leave out all scanf code in the final executable since you don't need it.

So the literal answer to your question is: No, the compiler doesn't do anything like that. If you were using "compiler" to informally denote your entire toolchain, then the answer is: Yes at first, but only until it finishes checking which parts of the object file you do and don't need.

Note that this only describes static linking, where you combine all object files into a single executable before executing anything. Dynamic linking is a very different story.


问:
Should programmers help testers in designing tests?
How much should programmers help testers in designing tests?

I don't think they should help at all.  My worry is that if they help testers in designing tests for their own code, they'll 'infect' the testers with their own prejudices and blind spots about that code.

I feel that the requirements should be sufficient for giving the information needed for testers to create their tests.  If there is some part of the implementation that the programmers find worrisome, then I think it's their duty to implement unit tests to test that part or even run their own informal system tests to test that part.

Not everyone I know agrees with this though (and I understand some of their points to a certain extent).  What do others think about this?  Is this discussed in the literature anywhere?

答: I like to review the test plans and suggest additional test cases that QA might not have thought of.  I'm not sure how that would "infect the testers with my own prejudices".  </p>


问:
What naming convention should you use for data objects solely meant for parameters
This is my pseudocode for DAL:

```<code>CusOrderDTO GetCustomerOrder(DateTime OrderDate, string customerCode)
    CusOrderDTO orderSet = new CusOrderDTO()
    * query data from database, populate CusOrderDTO
    return orderSet
</code>```

Now the parameters for the DAL have grown so large that I should pass a custom data object for parameter

```<code>CusOrderDTO GetCustomerOrder(SomeParameterClassName orderParameters)
</code>```

What naming convention is generally used for SomeParameterClassName? Naming it DTO seems weird since it's solely used for parameter purposes only and service layer is only going to work on CusOrderDTO object.

For the above, please assume that CusOrderDTO and orderParameters have nothing in common.

答:
If SomeParameterClassName regroups logically connected objects, then name it accordingly.

For instance, if it regroups coordinate X, Y and Z, name it 3DPoint. If it regroups the product name, product price and product description, name it Product.

```<code>void Demo(int x, int y, int z) { }
            ――― ↧ ―――
void Demo(3DPoint coordinates) { }
</code>```

and:

```<code>void Demo(string productName, int productPrice, string productDescription) { }
            ――― ↧ ―――
void Demo(ProductInfo product) { }
</code>```

If SomeParameterClassName regroups objects which have nothing in common except being passed to the same method, then you are doing it wrong. There is no need to create a type for unrelated entities: just pass them as-is to the method.

For instance, if it regroups coordinate X and product description, remove the object and pass two parameters.

```<code>void Demo(int x, string productDescription) { }
            ――― ↺ ―――
</code>```

It may be that you have many parameters passed to a method, but not all can be grouped. In this case, search for the ones you can group, and leave the others.

For instance, with parameters such as product name, product price, product description and logger, then group first three into a product class and leave the last one:

```<code>void Demo(string productName, int productPrice,
          string productDescription, AppLogger logger) { }
            ――― ↧ ―――
void Demo(ProductInfo product, AppLogger logger) { }
</code>```

In a presence of a method which accepts many disparate parameters, additional refactoring may be required. For example, in the previous piece of code, one may wonder why is AppLogger passed as a parameter, instead of being injected when calling a constructor:

```<code>var something = new Something();
...
something.Demo(product, logger);
          ――― ↧ ―――
var something = new Something(logger);
...
something.Demo(product);
</code>```

Since other method of Something are probably logging their activity as well, injecting the logger in the constructor makes sense.

Another refactoring technique is to split the method. The large number of parameters is a good indication that the method does too much. There are sometimes obvious cases:

```<code>void Demo(a, b, c, d, e, f, g, h)
{
    if (a)
    {
        // A few hundred lines which use exclusively the parameters b, c and d.
    }
    else
    {
        // A few hundred lines which use exclusively the parameters e, f, g and h.
    }
}
</code>```

but most are less obvious: some parameters may be used through the entire method, may be interconnected, etc. Still, if you feel like you have too many parameters, look at other indications (such as LOC, ILLOC in C# and the number of branches) and check if your method is doing one and one only thing.

Body:
If SomeParameterClassName regroups logically connected objects, then name it accordingly.

For instance, if it regroups coordinate X, Y and Z, name it 3DPoint. If it regroups the product name, product price and product description, name it Product.

void Demo(int x, int y, int z) { }
            ――― ↧ ―――
void Demo(3DPoint coordinates) { }

and:

void Demo(string productName, int productPrice, string productDescription) { }
            ――― ↧ ―――
void Demo(ProductInfo product) { }

If SomeParameterClassName regroups objects which have nothing in common except being passed to the same method, then you are doing it wrong. There is no need to create a type for unrelated entities: just pass them as-is to the method.

For instance, if it regroups coordinate X and product description, remove the object and pass two parameters.

void Demo(int x, string productDescription) { }
            ――― ↺ ―――

It may be that you have many parameters passed to a method, but not all can be grouped. In this case, search for the ones you can group, and leave the others.

For instance, with parameters such as product name, product price, product description and logger, then group first three into a product class and leave the last one:

void Demo(string productName, int productPrice,
          string productDescription, AppLogger logger) { }
            ――― ↧ ―――
void Demo(ProductInfo product, AppLogger logger) { }

In a presence of a method which accepts many disparate parameters, additional refactoring may be required. For example, in the previous piece of code, one may wonder why is AppLogger passed as a parameter, instead of being injected when calling a constructor:

var something = new Something();
...
something.Demo(product, logger);
          ――― ↧ ―――
var something = new Something(logger);
...
something.Demo(product);

Since other method of Something are probably logging their activity as well, injecting the logger in the constructor makes sense.

Another refactoring technique is to split the method. The large number of parameters is a good indication that the method does too much. There are sometimes obvious cases:

void Demo(a, b, c, d, e, f, g, h)
{
    if (a)
    {
        // A few hundred lines which use exclusively the parameters b, c and d.
    }
    else
    {
        // A few hundred lines which use exclusively the parameters e, f, g and h.
    }
}

but most are less obvious: some parameters may be used through the entire method, may be interconnected, etc. Still, if you feel like you have too many parameters, look at other indications (such as LOC, ILLOC in C# and the number of branches) and check if your method is doing one and one only thing.


问:
SOC and self-initializing configuration
Disclaimer: I'm using C# for code examples. But I guess they should be readable/understandable for everyone. If they are not: please leave a comment.

Okay I have a Client-Server application. The server can be extended with Agents. An agent can be used to distribute loads.

An Agent should have some static variables like ID etc. All of these values will be self-generated (ex.: ID = Guid.NewGuid(). The ID should be once set and always be the same. Even if you restart the Agent.

My first thought was I want to have some kind of Config class which contains these variables.

```<code>class Config
{
    public Guid ID { get; private set; }
    // ...
}
</code>```

Okay when an Agent starts it should check IsConfigurationInitialized. If its not, initialize it.

I thought it would be an neat idea not to care about the initalization of the configuration. So the configuration itself check in the constructor if it needs to be initalized. So my main path of the application just do something like Config config = new Config();

But for some reason I feel like this is a bad idea since the class is just a variable holder.

To my question:

Does the above given example break the SOC or any other design principle?

答:
That design seem to violate SRP since your Config (which is a config, just a configuration holder) will know how to generate configuration for each specific agent).

I think an Agent should create Config instance, passing to it it's name (which is used to deduce configuration file path from it) and the callback, which should returns Agent's initial configuration.

Config checks for configuration file existence when created / initialized (somehow I do not like to perform heavy tasks in constructors and tend to create initialize() methods for those). If it exists, it just loads config from it. If it does not, it executes callback, receives initial configuration from Agent and saves it to the file for later use.

This way your Agent and Config are more or less separated from each other.

Title: SOC and self-initializing configuration


问:
Why isn't the addremove recommended by default in Mercurial?
This is perhaps a question of strategy more than anything else. I am new to version control and I guess I should count myself lucky to have relatively more mature systems like Mercurial and Git at my disposal as compared to the older systems.

I started with Hg because I read somewhere that it's more Pythonlike in its approach. The few tutorials that I went through (examples and videos) consistently use the command hg add *. After using that command for some time now on my project, I have a long list of files which have ! in front of the files that have been deleted. I am speaking of the output of the command hg st of course.

I guess using hg addremove * will solve my problem. If that's the case, why isn't it the default command to teach Hg to a beginner? Is there a pitfall I am missing? Wouldn't it be reasonable to assume that typically a person working on a folder would want the files which are deleted to visibly disappear from the output of hg st (esp. since they are stored in the repository by Hg anyways)?

答:
hg add * is a terribly idea. The tutorials you've read/watched probably weren't very good. Most repositories contain many files that you don't want to put under version control: Generated artifacts, junk from experiments you did, and files that should be part of the repository but only later (i.e., are not ready to be committed). You don't want to version control those. Unless they are only temporary, they should be listed in .hgignore so they won't pollute the output of hg status with ?s and won't be added accidentally. hg add and hg addremove already consider all files, except ignored ones, and hg add * and hg addremove * override this sane default. So it's not only redundant, but also breaks a very useful feature.

You should only add the files that make sense of have under version control. Then files "suddenly disappearing" is not a problem, because the only files Mercurial tracks are the important ones, which you rarely remove, and if so, only manually after careful consideration. In that case, use hg rm file1 file2 to remove those files both from the file system and from mercurial — no need for addremove.

hg addremove is a useful shorthand if you screwed up and made significant changes to the project layout via some tool other than Mercurial, especially since it properly handles copies/renames. Note that you can restrict it to some files by giving, for example, hg addremove src/. But it should never override good judgement of what belongs under source control and what doesn't. If you accidentally added a file that shouldn't be under source control, use hg forget the-file before committing.


答:
Of course, I never saw your actual code, but from what you've said, I would propose something close to the following architecture. I'm sorry for pseudo-code, I just wanted it to be fast & clear ;)

UI layer:

```<code>dataService = new DataService();

dataService->makeDataRequest(requestName, requestParams[], cachedDataArrivedCallback, freshDataArrivedCallback);

showWaitMessageToUser();
</code>```

Service layer (DataService fictional class):

```<code>processor = RequestProcessorFactory.getProcessor(requestName);

if(processor.cachedDataAvailable(requestParams)) {
    cachedDataCallback(processor.getCachedData());
}

freshData = processor.getFreshDataAsync(requestParams, freshDataArrivedCallback);
</code>```

RequestProcessor::getFreshDataAsync:

```<code>networkTransport = NetworkRequestFactory.getTransport(requestName);
networkTransport.send(requestParams[]);
rawResult = networkTransport.result();

deserializer = DeserializerFactory.getDeserializer(requestName); //JSON deserializers
result = deserializer.deserialize(rawResult);

storageHandler = StorageHandlerFactory.getStorageHandler(requestName); //Database layer
storageHandler.storeData(requestName, result);

cacheHandler = CacheHandlerFactory.getCacheHandler(requestName);
cacheHandler.store(requestParams, result);

freshDataArrivedCallback(result);
</code>```

I don't like diagrams, but I think this should be pretty clear.

Your UI calls your service layer with a request to provide data. It passes there callbacks to execute when data would be available. And then it shows message to user and waits for user interactions.

Service layer first gets RequestProcessor, which is responsible for handling this particular request (User requests, Search requests etc). It checks with processor if it has cached data for this particular request parameters. If so, it gets it and triggers one of UI callbacks.
It then asks requestProcessor to go for new data.

Request processor calls network layer to relay request to the server. It deserializes reply using some Deserializer, which is currently your old&ugly JSON library, but you will be able to switch to another pretty easy with what I propose.

Deserialized results go to database handler and cache handler (but may be, they are the same in your app, I am not sure). Well, clean results may go to whatever handlers you want now ;)

Well, at last request processor calls freshDataArrivedCallback to announce that new data is available for UI to display.

This is just an example of how this can be split by. There may be various improvements to the scheme. For example, it is better not to pass freshDataArrivedCallback around, keep it at the DataService and have DataService expose it's own callback to reduce coupling.

This example uses factories every now and again, but you can skip them if you have only one handler for something to avoid unnecessary complications.

I just tried to make my thinking 5-min short to give you an idea ;) I hope you find this useful.

That was an interesting question to answer and if this answer help, I will gladly dive deeper ;) BTW, I never tried to do iOS development ;))

Title: What architecture/design to adopt when consuming a RESTful API? (iOS App)

Body:
TL;DR

Part 1)

I want to refactor the biggest part of the app on which I work - and it is pretty much spaghetti. It's a single class that makes the requests to the server, parses the JSON, and saves the data in the database (it also has knowledge of some UI that needs to change when specific business rules are met and it definitely shouldn't know all this stuff). It has somewhere below 5000 LOC :(.

One way of breaking the class in smaller pieces would be to separate the requests by semantics (for example User requests, Search requests etc). Would that be useful or not really?

Also, I think that networking code should be decoupled from the parsing code and also from the DB code. The problem is that I don't really know what approach (architecture/design pattern) should I use to do this the best way. (for example, our JSON parsing library is old and is not maintained anymore - we wanted to change it because it produced some problems and we were stuck because of the tight-coupling).

Part 2)

I also would like to receive some suggestions for the following issue:
We use the DB mostly for offline mode & caching. The user can see data in offline mode, modify it (and requests get serialized) and when he gets back online, the requests will be made to the server. We also use it as a cache to show something on the UI while we do a request in the background to see if there is new/modified data (we use a similar mechanism like ETag to check with the server).

So this is the flow when a user enters a screen:

```<code>enter_screen ->
check_cache + background_request ->
show_UI with cache data ->
update_UI when the background request finishes and new/modified data is available.
</code>```

Can this stuff be abstracted away somehow? I was thinking of some kind of delegate pattern where I get an immediate callback when I hit the cache and then get a later callback when the request finishes (with optional flags to distinguish between cache and fresh data).

A bit of a background story

I currently work for an iOS app that is aprox. 4 years old. I only joined the team a year ago. In my opinion, the code is ugly, full of hacks, and has a very shaky architecture. We are 2 iOS developers working on it. The client doesn't have the technical expertise and may not understand all the implications of some issues that we have. Making a client understand why "refactoring" is needed, is a hard task. I struggle to refactor how much I can, but we still have a lot to go trough.

The app also has a lot of features that increases the complexity of the app: deeplinks, push notifications, offline mode, non blocking UI etc. all combined with a rather large and hairy data model.

Because the client requests features in a constant manner, I feel that we have a lot of technical debt.

The mobile world is changing pretty fast and new technologies appear (Size classes, Auto Layout, Swift, etc.). This is true for all technologies, but I somehow feel that the race is a bit more intense in the mobile world. Currently, I only dream of using Swift at work (although  I play with it at home in my spare time).

Body:
Of course, I never saw your actual code, but from what you've said, I would propose something close to the following architecture. I'm sorry for pseudo-code, I just wanted it to be fast & clear ;)

UI layer:

dataService = new DataService();

dataService->makeDataRequest(requestName, requestParams[], cachedDataArrivedCallback, freshDataArrivedCallback);

showWaitMessageToUser();

Service layer (DataService fictional class):

processor = RequestProcessorFactory.getProcessor(requestName);

if(processor.cachedDataAvailable(requestParams)) {
    cachedDataCallback(processor.getCachedData());
}

freshData = processor.getFreshDataAsync(requestParams, freshDataArrivedCallback);

RequestProcessor::getFreshDataAsync:

networkTransport = NetworkRequestFactory.getTransport(requestName);
networkTransport.send(requestParams[]);
rawResult = networkTransport.result();

deserializer = DeserializerFactory.getDeserializer(requestName); //JSON deserializers
result = deserializer.deserialize(rawResult);

storageHandler = StorageHandlerFactory.getStorageHandler(requestName); //Database layer
storageHandler.storeData(requestName, result);

cacheHandler = CacheHandlerFactory.getCacheHandler(requestName);
cacheHandler.store(requestParams, result);

freshDataArrivedCallback(result);

I don't like diagrams, but I think this should be pretty clear.

Your UI calls your service layer with a request to provide data. It passes there callbacks to execute when data would be available. And then it shows message to user and waits for user interactions.

Service layer first gets RequestProcessor, which is responsible for handling this particular request (User requests, Search requests etc). It checks with processor if it has cached data for this particular request parameters. If so, it gets it and triggers one of UI callbacks.
It then asks requestProcessor to go for new data.

Request processor calls network layer to relay request to the server. It deserializes reply using some Deserializer, which is currently your old&ugly JSON library, but you will be able to switch to another pretty easy with what I propose.

Deserialized results go to database handler and cache handler (but may be, they are the same in your app, I am not sure). Well, clean results may go to whatever handlers you want now ;)

Well, at last request processor calls freshDataArrivedCallback to announce that new data is available for UI to display.

This is just an example of how this can be split by. There may be various improvements to the scheme. For example, it is better not to pass freshDataArrivedCallback around, keep it at the DataService and have DataService expose it's own callback to reduce coupling.

This example uses factories every now and again, but you can skip them if you have only one handler for something to avoid unnecessary complications.

I just tried to make my thinking 5-min short to give you an idea ;) I hope you find this useful.

That was an interesting question to answer and if this answer help, I will gladly dive deeper ;) BTW, I never tried to do iOS development ;))


问:
How to organize code which converts from one type to another
If I have a DateRange class, and I want to translate a DateRange into SQL (e.g. some_col >= '2015-3-5' AND some_col <= '2015-3-5'), where should I put the method that does the translating? Nothing I've come up so far with seems very good.

Option #1: Put a method in the DateRange class. (Downside: close coupling of the DateRange class and SQL?)

```<code>class DateRange {
    public function toSQL ($colName) {...}
}
</code>```

Opton #2: Make a class that only has one method in it?

```<code>class DateRangeTranslator {...}
</code>```

Option #3: a standalone function

```<code>function dateRangeToSQLWhereCondition (DateRange $dr, $colName) {...}
</code>```

I'm leaning towards option #2. But is there a better way?

EDIT:

It's a bit tedious but here's what I've got now:

```<code>class DateRangeSQLBuilder {
    public $dateRange;
    public function buildWhereCondition ($colName) {...}
}
</code>```

And you would use it like so:

```<code>$dateRange = new DateRange($fromDate, $toDate);
$builder = new DateRangeSQLBuilder($dateRange);
$whereConditions[] = $builder->buildWhereCondition('runDate');
</code>```

答:
I'd go with the function. Like you said, the first option introduces too much coupling. Option two looks okay but since your class will probably don't have to maintain state I'd go with option three.

If in doubt, go with the easy/dumb solution ;).