问:
Class structure/pattern for validating several different Excel templates?
I'm trying to validate the contents of Excel sheets that follow several different templates.  For each one, there are three possible validation actions for various cells:

Regex (e.g. "XYZ-123" fails because it doesn't fit pattern "[A-Z]{4}-\d{3}")

Date/Time format (e.g. "Jan 24, 2013" fails because it's not mm/dd/yyyy)

Data type (e.g. "xyz" fails because it's not an integer)

My first thought was to use the following class:

```<code>public class Validator
{
 Excel.Worksheet vSheet;
 List<Tuple<string,string,string>> cellActions=new List<Tuple<string,string,string>>();

 //populates the list of cellActions based on the template type
 public Validator(Excel.Worksheet sht)
 {
  this.vSheet=sht;
  string templateType=templateCheck(sht);
  switch (templateType)
  {
   case "type1":
    cellActions.Add(new Tuple<string,string,string>("C5","regex","[A-Z\d]{6}");
    cellActions.Add(new Tuple<string,string,string>("D3","datatype","long");
    //and so on for another 30 list items
    break;
   case "type2"
    cellActions.Add(new Tuple<string,string,string>("A3","date_time","yyyy-MM-dd HH:mm");
    cellActions.Add(new Tuple<string,string,string>("A6","regex",".+\s[ACGT]{3}");
    //etc.
  }
 }

 private string templateCheck(Excel.Worksheet sht)
 {
  //return template type as string based on contents of worksheet
 }

 public void validate()
 {
  foreach(Tuple<string,string,string> cellAction in cellActions)
  {
   //check the contents of the cell in the vSheet property according to cellAction's rules
  }
 }
}
</code>```

}

Is this a case where Strategy Pattern would be appropriate?  There are several different types of sheets, each with a different list of cells and rules for those cells, so maybe it would be right to think about each template as a strategy to be implemented as a class.  Or is this not a situation where the pattern applies, given that each of the 3 validation action types is always the same regardless of which template it's applied in?

答:
I would go with specific typing for your cells. C5 is an... identifier type cell? Go specific, as specific as your domain requires.

Then, describe each sheet in terms of what cells with what types it contains. Encapsulate that, put it in a class that represents the sheet.

Now you have a nice object structure representing your allowable Excel spreadsheets. Validation gets encapsulated in the type for each cell. Note how you achieve maximum reuse and capture the structure of your sheets.

As to whether this is strategy, the lines are blurry, but yes, I consider the validators thus embedded in the types as an incarnation of the Strategy design pattern.

Validation now becomes a simple matter of going through each sheet, and each cell of interest, and invoking the validator on the cell, which will delegate to the type validator.

```<code>class CellType {
    bool IsValid(string value);
    string DisplayName { get; }
}

class Cell {
    string Value { get; set }
    CellType Type { get; set; }
    boolean IsValid() { return Type.IsValid(Value); }
}

class CellPosition {
    ...
}

class Sheet {
    Cell CellAt(CellPosition position);
    IEnumerable<Cell> CellsOfInterest { get; }
}

void Validate() {
    foreach (Sheet sheet in sheets) {
        foreach (Cell cell in sheet.CellsOfInterest) {
            if (!cell.IsValid()) {
                // Whoops!
            }
        }
    }
}
</code>```

Note: it is overkill to employ Visitor to validate the degenerate "composite" structure induced by Sheet and Cell, as the structure always predictably has exactly two layers. For that reason, I have not mentioned it. :)

Body:
I would go with specific typing for your cells. C5 is an... identifier type cell? Go specific, as specific as your domain requires.

Then, describe each sheet in terms of what cells with what types it contains. Encapsulate that, put it in a class that represents the sheet.

Now you have a nice object structure representing your allowable Excel spreadsheets. Validation gets encapsulated in the type for each cell. Note how you achieve maximum reuse and capture the structure of your sheets.

As to whether this is strategy, the lines are blurry, but yes, I consider the validators thus embedded in the types as an incarnation of the Strategy design pattern.

Validation now becomes a simple matter of going through each sheet, and each cell of interest, and invoking the validator on the cell, which will delegate to the type validator.

class CellType {
    bool IsValid(string value);
    string DisplayName { get; }
}

class Cell {
    string Value { get; set }
    CellType Type { get; set; }
    boolean IsValid() { return Type.IsValid(Value); }
}

class CellPosition {
    ...
}

class Sheet {
    Cell CellAt(CellPosition position);
    IEnumerable<Cell> CellsOfInterest { get; }
}

void Validate() {
    foreach (Sheet sheet in sheets) {
        foreach (Cell cell in sheet.CellsOfInterest) {
            if (!cell.IsValid()) {
                // Whoops!
            }
        }
    }
}

Note: it is overkill to employ Visitor to validate the degenerate "composite" structure induced by Sheet and Cell, as the structure always predictably has exactly two layers. For that reason, I have not mentioned it. :)


问:
Champaign Fountain Puzzle
Empty glasses of water are arranged in the following order:

When you pour liquid into the 1st glass if it's full, then the extra liquid would be flown into the glasses 2 and 3 in equal quantities. When glass 2 is full, the extra liquid would be flown into 4 and 5 and so on.

Given an N liters of liquid and maximum capacity of each glass is 1 liter, give the amount of liquid  present in any glass if you empty N liters of liquid by pouring into glass by filling out the function getWaterInBucket(int N, int X) where X is the glass number. So for example if I want have 4 liters at the beginning and I want to find the water in glass 3 the function is getWaterInBucket(4, 3)

How do I solve this programmatically? I tried to find a math solution using Pascal's triangle. This did not work. I considered it to be a tree so I can add a parameter like this getWaterInBucket(BTree root, int N, int X) and then try some recursive solution for each level, but parameters are not allowed in this problem. Is there something obvious, some trick?

答:
Here's how I would answer this question in an interview situation (I haven't seen this question before, and I didn't look at the other answers until I had my solution):

First, I tried to just figure it out (which you called the "math solution") and when I got to glass 8 I realized that it would be tougher than it seemed because glass 5 starts to overflow before glass 4. At that point I decided to go down the recursion route (just an FYI, a lot of programming interview questions require recursion or induction to solve).

Thinking recursively, the problem becomes much easier: How much water is in glass 8? Half of the amount that has spilled out of glasses 4 and 5 (until it is full). Of course, that means we have to answer how much has spilled out of glasses 4 and 5, but it turns out that's not too hard either. How much has spilled out of glass 5? Half of however much spilled out of glasses 2 and 3, minus the liter that stayed in glass 5.

Solving this completely (and messily) gives:

```<code>#include <iostream>
#include <cmath>
using namespace std;

double howMuchSpilledOutOf(int liters, int bucketId) {
    double spilledInto = 0.0;
    switch (bucketId) {
        case 1:
            spilledInto = liters; break;
        case 2:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 3:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 4:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 2); break;
        case 5:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 2) + 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 6:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 3); break;
        default:
            cerr << "Invalid spill bucket ID " << bucketId << endl;
    }
    return max(0.0, spilledInto - 1.0);
}

double getWaterInBucket(int liters, int bucketId) {
    double contents = 0.0;
    switch (bucketId) {
        case 1:
            contents = liters; break;
        case 2:
            contents = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 3:
            contents = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 4:
            contents = 0.5 * howMuchSpilledOutOf(liters, 2); break;
        case 5:
            contents = 0.5 * howMuchSpilledOutOf(liters, 2) + 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 6:
            contents = 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 7:
            contents = 0.5 * howMuchSpilledOutOf(liters, 4); break;
        case 8:
            contents = 0.5 * howMuchSpilledOutOf(liters, 4) + 0.5 * howMuchSpilledOutOf(liters, 5); break;
        case 9:
            contents = 0.5 * howMuchSpilledOutOf(liters, 5) + 0.5 * howMuchSpilledOutOf(liters, 6); break;
        case 10:
            contents = 0.5 * howMuchSpilledOutOf(liters, 6); break;
        default:
            cerr << "Invalid contents bucket ID" << bucketId << endl;
    }
    return min(1.0, contents);
}

int main(int argc, char** argv)
{
    if (argc == 3) {
        int liters = atoi(argv[1]);
        int bucket = atoi(argv[2]);
        cout << getWaterInBucket(liters, bucket) << endl;
    }
    return 0;
}
</code>```

At this point (or as I was writing this out), I would tell the interviewer that this isn't the ideal solution in production: there's duplicate code between howMuchSpilledOutOf() and getWaterInBucket(); there should be a central location that maps buckets to their "feeders". But in an interview, where speed and accuracy of implementation are more important that speed of execution and maintainability (unless otherwise indicated), this solution is preferable. Then I would offer to refactor the code to be closer to what I consider production-quality, and let the interviewer decide.

Final note: I'm sure my code has a typo in it somewhere; I would mention that to the interviewer too and say that I would feel more confident in it after either refactoring it or unit-testing it.

ContentLicense: CC BY-SA 3.0

Title: Champaign Fountain Puzzle

Body:
Here's how I would answer this question in an interview situation (I haven't seen this question before, and I didn't look at the other answers until I had my solution):

First, I tried to just figure it out (which you called the "math solution") and when I got to glass 8 I realized that it would be tougher than it seemed because glass 5 starts to overflow before glass 4. At that point I decided to go down the recursion route (just an FYI, a lot of programming interview questions require recursion or induction to solve).

Thinking recursively, the problem becomes much easier: How much water is in glass 8? Half of the amount that has spilled out of glasses 4 and 5 (until it is full). Of course, that means we have to answer how much has spilled out of glasses 4 and 5, but it turns out that's not too hard either. How much has spilled out of glass 5? Half of however much spilled out of glasses 2 and 3, minus the liter that stayed in glass 5.

Solving this completely (and messily) gives:

#include <iostream>
#include <cmath>
using namespace std;

double howMuchSpilledOutOf(int liters, int bucketId) {
    double spilledInto = 0.0;
    switch (bucketId) {
        case 1:
            spilledInto = liters; break;
        case 2:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 3:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 4:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 2); break;
        case 5:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 2) + 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 6:
            spilledInto = 0.5 * howMuchSpilledOutOf(liters, 3); break;
        default:
            cerr << "Invalid spill bucket ID " << bucketId << endl;
    }
    return max(0.0, spilledInto - 1.0);
}

double getWaterInBucket(int liters, int bucketId) {
    double contents = 0.0;
    switch (bucketId) {
        case 1:
            contents = liters; break;
        case 2:
            contents = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 3:
            contents = 0.5 * howMuchSpilledOutOf(liters, 1); break;
        case 4:
            contents = 0.5 * howMuchSpilledOutOf(liters, 2); break;
        case 5:
            contents = 0.5 * howMuchSpilledOutOf(liters, 2) + 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 6:
            contents = 0.5 * howMuchSpilledOutOf(liters, 3); break;
        case 7:
            contents = 0.5 * howMuchSpilledOutOf(liters, 4); break;
        case 8:
            contents = 0.5 * howMuchSpilledOutOf(liters, 4) + 0.5 * howMuchSpilledOutOf(liters, 5); break;
        case 9:
            contents = 0.5 * howMuchSpilledOutOf(liters, 5) + 0.5 * howMuchSpilledOutOf(liters, 6); break;
        case 10:
            contents = 0.5 * howMuchSpilledOutOf(liters, 6); break;
        default:
            cerr << "Invalid contents bucket ID" << bucketId << endl;
    }
    return min(1.0, contents);
}

int main(int argc, char** argv)
{
    if (argc == 3) {
        int liters = atoi(argv[1]);
        int bucket = atoi(argv[2]);
        cout << getWaterInBucket(liters, bucket) << endl;
    }
    return 0;
}

At this point (or as I was writing this out), I would tell the interviewer that this isn't the ideal solution in production: there's duplicate code between howMuchSpilledOutOf() and getWaterInBucket(); there should be a central location that maps buckets to their "feeders". But in an interview, where speed and accuracy of implementation are more important that speed of execution and maintainability (unless otherwise indicated), this solution is preferable. Then I would offer to refactor the code to be closer to what I consider production-quality, and let the interviewer decide.

Final note: I'm sure my code has a typo in it somewhere; I would mention that to the interviewer too and say that I would feel more confident in it after either refactoring it or unit-testing it.


问:
Switch interface implementation using configuration
We want to allow the same core service to be either fully implemented or, as other option, to be a proxy toward a client legacy system (via a WSDL for example).

In that way, we have both implementation (proxy & full) and we switch which one to use in the configuration of the app.

So in a nutshell, Some desired features:

Two different implementation (proxy, full) instead of one implementation with a switch inside

Switch implementation using configuration: dependency injection? reflection?

Nice-to-have: the packaged delivered to the client doesn’t have to change depending on the choice between proxy or full

Nice-to-have: Client can develop their custom implementation of the Core Interface and configure the applciation to use that one

With this background, the question is: What alternatives we have to choose one implementation or other of an interface just changing configuration?

Thanks

答:
We use this model at work:

```<code>Production config.properties:
serviceClass=com.mycompany.LiveService

Development config.properties:
serviceClass=com.mycompany.MockService

spring.xml:
<bean id="service" class="${serviceClass}"/>
(with the appropriate property configurer)
</code>```

Try this one too. I like it better, because you don't end up putting class names in the configuration, which I think should speak at a different level of abstraction.

```<code>Production config.properties:
environment=production

Dev config.properties:
environment=development

<bean id="service" class="com.mycompany.ServiceSelector">
    <property name="environment" value="${environment}"/>
    <property name="liveService" ref="liveService"/>
    <property name="mockService" ref="mockService"/>
</bean>

// a proxy that decides on the class to "let through" based on the environment
class ServiceSelector {
    public void setLiveService(Service liveService) {
        this.liveService = liveService;
        updateCurrentService();
    }

    public void setMockService(Service mockService) {
        this.mockService = mockService;
        updateCurrentService();
    }

    public void setEnvironment(Environment environment) {
        this.environment = environment;
        updateCurrentService();
    }

    private void updateCurrentService() {
        if (environment == null) {
            currentService = null;
            return;
        }
        switch (environment) {
            case production:
                currentService = liveService;
                break;
            case development:
                currentService = mockService;
                break;
            default:
                throw new IllegalStateException();
        }
    }
}
</code>```

If you have a large enough number of services in this situation, you can put all such services under a facade and implement one facade selector instead of many service selectors.

Another variation is to throw away the configuration file and provide the environment flag in the environment itself. It could be a JVM property defined with -Dname=value when the JVM is invoked, etc.

Body:
We use this model at work:

Production config.properties:
serviceClass=com.mycompany.LiveService

Development config.properties:
serviceClass=com.mycompany.MockService

spring.xml:
<bean id="service" class="${serviceClass}"/>
(with the appropriate property configurer)

Try this one too. I like it better, because you don't end up putting class names in the configuration, which I think should speak at a different level of abstraction.

Production config.properties:
environment=production

Dev config.properties:
environment=development

<bean id="service" class="com.mycompany.ServiceSelector">
    <property name="environment" value="${environment}"/>
    <property name="liveService" ref="liveService"/>
    <property name="mockService" ref="mockService"/>
</bean>

// a proxy that decides on the class to "let through" based on the environment
class ServiceSelector {
    public void setLiveService(Service liveService) {
        this.liveService = liveService;
        updateCurrentService();
    }

    public void setMockService(Service mockService) {
        this.mockService = mockService;
        updateCurrentService();
    }

    public void setEnvironment(Environment environment) {
        this.environment = environment;
        updateCurrentService();
    }

    private void updateCurrentService() {
        if (environment == null) {
            currentService = null;
            return;
        }
        switch (environment) {
            case production:
                currentService = liveService;
                break;
            case development:
                currentService = mockService;
                break;
            default:
                throw new IllegalStateException();
        }
    }
}

If you have a large enough number of services in this situation, you can put all such services under a facade and implement one facade selector instead of many service selectors.

Another variation is to throw away the configuration file and provide the environment flag in the environment itself. It could be a JVM property defined with -Dname=value when the JVM is invoked, etc.


问:
MVC exposes database primary keys?
I'm going through a MVC tutorial, and I notice that convention seems to be to expose a tables primary key on detail pages/urls (ie. /Movies/Details/5 as an example from the tutorial).

It's obviously not a problem for things like a movie record or a SO post, but it might be a bit different for an invoice or transaction with confidential information on it if the key was sequential or guessable.

So, is it just this tutorial or do MVC apps typically expose a tables primary key?  Is there a common library or pattern for hiding the key if you don't want to expose it?

答:
an invoice or transaction with confidential information on it if the key was sequential or guessable.

Even if it's guessable, it shouldn't be accessible.  Show something like an "Access Denied" error message.  Sequential keys are used because they're simple and don't hold information about their contents.

Security by obscurity (neener neener neener, you can't guess the ID!) is no security.

Title: MVC exposes database primary keys?


答:
This is just the standard routing aspect of MVC. Another example would be:

Movies/Details/{parentKey}/{thisKey}

You shouldn't expect MVC to provide any level of security. That is your job. Just because I can guess that there is a key of 5, doesn't mean I should have access to it. Any ActionResult that you feel will reveal sensitive data should have some type of security involved - either in the ActionResult itself or by using an AuthorizeAttribute on the ActionResult (or even on the entire controller class).

Title: MVC exposes database primary keys?

Body:
I'm going through a MVC tutorial, and I notice that convention seems to be to expose a tables primary key on detail pages/urls (ie. /Movies/Details/5 as an example from the tutorial).

It's obviously not a problem for things like a movie record or a SO post, but it might be a bit different for an invoice or transaction with confidential information on it if the key was sequential or guessable.

So, is it just this tutorial or do MVC apps typically expose a tables primary key?  Is there a common library or pattern for hiding the key if you don't want to expose it?


答:
This is common in tutorials, and even simple apps where record keys aren't sensitive. But MVC doesn't force you to do it that way... If you don't want to use easily guessable sequential IDs, you don't have to.

And to be fair, this doesn't have anything to do with MVC. I can't count how many websites I've seen where the convention is something like /movies/details.aspx?id=5 or /movies/details.php?id=5

Title: MVC exposes database primary keys?


答:
Displaying the key is totally optional, but it isn't a security issue even if you do. You need to check if the user has access, always. That is your job, not the framework's.

But you don't have to use a numeric id. I like to do something like the content type and title in the url:

```<code>/news/title-of-article
/reservations/description-of-hotel-and-reservation
/properties/address-of-property
etc.
</code>```

I then use that information to find the piece of content. Not for security reasons, the title is only superficially more private than an id, but for usability and/or SEO.

I also like to allow the user to get rid of the slug (title, whatever you want to call it) in the URL and receive exactly what they expect. Again, just usability, not security:

```<code>/properties/address-of-property (single property)
/properties (list of properties)
</code>```

MVC apps often have a default convention of controller/function/id. That is just a default. There is no reason to use it if the URL matters to you. Tutorials will usually teach the defaults, but after the tutorial you can (and should!) learn more. Then change things to match your needs.

In every framework I have worked in, there has been a way to map urls to functions and pass in parameters (whatever part of the url you want). They have, without exception, been called "routes."

Use well thought out routes if you want a more user/search engine frindly url. Or use the defualt if you don't need better urls, but just want to quickly build the app. Either way, check if a user has permission to see something. Always.

Knowing the URL must not equal able to gain access and it is your job to enforce that, no matter what your URL looks like.

Title: MVC exposes database primary keys?

Body:
Displaying the key is totally optional, but it isn't a security issue even if you do. You need to check if the user has access, always. That is your job, not the framework's.

But you don't have to use a numeric id. I like to do something like the content type and title in the url:

/news/title-of-article
/reservations/description-of-hotel-and-reservation
/properties/address-of-property
etc.

I then use that information to find the piece of content. Not for security reasons, the title is only superficially more private than an id, but for usability and/or SEO.

I also like to allow the user to get rid of the slug (title, whatever you want to call it) in the URL and receive exactly what they expect. Again, just usability, not security:

/properties/address-of-property (single property)
/properties (list of properties)

MVC apps often have a default convention of controller/function/id. That is just a default. There is no reason to use it if the URL matters to you. Tutorials will usually teach the defaults, but after the tutorial you can (and should!) learn more. Then change things to match your needs.

In every framework I have worked in, there has been a way to map urls to functions and pass in parameters (whatever part of the url you want). They have, without exception, been called "routes."

Use well thought out routes if you want a more user/search engine frindly url. Or use the defualt if you don't need better urls, but just want to quickly build the app. Either way, check if a user has permission to see something. Always.

Knowing the URL must not equal able to gain access and it is your job to enforce that, no matter what your URL looks like.


问:
Hacking Python "Requests" library.. How to start as an intermediate pythonista?
My question is conceptual, yet extremely vital for me.

I'm an intermediate Python developer. I know fair enough about it, and actually use it on a daily basis. Now, I would like to transform my capabilities from a library/API user to "designer" one.

I found that "Python Requests" library by Kenneth Reitz is an extremely good place to explore Python patterns and idioms. I need to learn to split a big problem into its essential Modules, define required classes/functions, and use them to work together.

I lack such a discipline, and I'm starving to gain this knowledge.

How should I dive into this library? It is big, with many modules (with logical names, of course), classes, and methods. I started to read the code, but I don't know where I am and what this portion of code is designed for.

答:
As @Martijn Pieters has pointed out, you need to know what problems that library solves, which in practical terms means, that you have to understand the problem domain.

then you will be able to have an idea on the choices the author made, appreciate some, criticize others.

if the library is too big to get an immediate overview, I would advice to start in some place where you have an (at least approximate) idea what the code is doing, like e.g. requests/api.py and

understand the objects you find there, and

understand how the code you are reading modifies those objects.

see if (and which) objects are created and

what those objects are used for.

if you do this from several starting points, you will get an overview of the whole. it is somewhat like exploring a unknown territory.


问:
Why doesn't CSS natively support variables and hierarchy?
I am new to UI development, but I feel very uncomfortable with how CSS works.

My use case is that I wanted to apply some specific styles inside a particular div on a page.

CSS attempt:

```<code>div.class1 {
    font: normal 12px arial, helvetica, sans-serif;
    font-color: #f30;
}
div.class1 div.class2 {
    border: 1px solid #f30;
}
</code>```

LESS attempt:

```<code>@red: #f30;
@font-family: arial, helvetica, sans-serif;
div.class1 {
    font: normal 12px @font-family;
    font-color: @red;
    div.class2 {
        border: 1px solid @red;
    }
</code>```

The CSS version can induce bugs, since it forces you to repeat #f30 & div.class2 every time you try to achieve hierarchy and variable re-use.

My questions:

Why is that CSS makes things difficult?

LESS does nothing special - just makes obvious improvements and translates to CSS?

What is that CSS wants to motivate in users because of which it promotes such a redundant coding style?

I strongly believe CSS should be what LESS is. Definitely I am overlooking some obvious advantage of why things are done that way in CSS. I thought it was a legacy problem, but I was surprised when I saw no attempts to solve this with CSS3.

Please help me understand how should I approach CSS?

答:
CSS isn't trying to make things more difficult on purpose, it was designed with a far simpler goal in mind, variables and hierarchies are hardly its only shortcomings. LESS and Sass exist specifically to address these shortcomings, and until either capability is natively supported, you should stick with them.

That said, W3C's CSS Working Group is working on drafts for both CSS variables and CSS hierarchies:

CSS Variables Module Level 1

CSS Hierarchies Module Level 3

There's absolutely no way to tell when either draft will be ready for implementation, or how soon major browsers will adopt them. All I can say is that the CSS Variables draft is closer to being adopted, you can already test drive CSS variables in Chrome, Safari and Firefox, but keep in mind that support in all three browsers is considered experimental and subject to change. Read Using CSS variables on MDN for more details.


问:
Algorithms for data structures in distributed system
The hash table data structure can be easily spread across multiple machines with a simple algorithm to distribute the keys:

```<code>machine_to_query = item_key % machine_count
</code>```

When you want to read and write key value pairs, you use the key to work out which machine stores the data, then you talk to that machine.  If you want a count of the total number of items, you'd need to request the count from each server and add them up.

What algorithms exist for efficiently managing data structures where the data is partitioned across multiple machines?  Distributed algorithms, not parallel algorithms.

How might something like a sorted array work in a distributed fashion?  Efficiently.

答:
Here's a link to a similar question on SO:

https://stackoverflow.com/questions/3927537/how-do-you-implement-sorting-and-paging-on-distributed-data

Also the last article from T-79.4001 Seminar on Theoretical Computer Science looks useful:

http://www.tcs.hut.fi/Studies/T-79.4001/2007SPR/

Some books on the subject:

Distributed Algorithms - Kaufmann

Design and Analysis of Distributed Algorithms - Wiley


问:
Service layer design
I am developing an MVC website in PHP, and for the first time, I would like to implement a service layer. I have some design considerations I would like to get some advice on. The backend will by no means be a large enterprise system, so I am looking to keep things relatively simple while still leveraging the benefits of a service layer.

The idea is that my controllers will interact with services, which in turn will interact with DAOs. My main question is how granular my services should be. I am familiar with the concept of service granularity in SOA, but have never implemented this in practice. For instance, to handle user registration, should I have a RegistrationService class, or should I have a UserService which contains a register method along with other user-related services? The former approach means that I will have a service layer full of small or fine-grained services, but I guess that is a good thing. The latter seems to hinder service reuse and result in large classes with many methods.

Instead, perhaps I should combine the two approaches to ensure that my services can be reused? For instance, have a RegistrationService which interacts with a DAO class. Then, I could have a UserService class with a register method. This method interacts with the fine-grained RegistrationService. This way, the RegistrationService can easily be reused to compose more coarse-grained services. Surely there are examples where the reuse would be more likely; for instance, placing an order would need to use/orchestrate many different services. I guess you could call this a "layered" approach, which I have seen discussed before. Could I structure this in a way such that my service layer will not become a mess?

I would love to hear your thoughts on this, whether you have other suggestions, recommendations and experiences or comments on the ideas I presented above. Thank you in advance.

答:
Today's PHP frameworks implementations do not use service layer, all database computations are stored in the the Model layer (ie Doctrine & Co). If you want to insert a service layer between your model and the controllers, you should think as "transactions".

This layer will allow you to put only atomic operations in your database layer (DAO) and chain them inside transactions in the service layer. This makes even more sense If your rdms support partial transaction rollback.

This layer will therefor become the point where technical exceptions (database primary key exception) become business exceptions (could not place order exception).

Don't know how to cleanly implement that with ORMs thought...

Title: Service layer design


问:
How do we structure our JavaScript so that it is easily editable, testable, and can make calls to render a view without knowing specifics about it?
I am working on a web application that displays some data and uses JavaScript.

Right now, we are serving up parts of our JavaScript code (to display certain types of information, that is, "Render a View") via our server which dynamically writes the JavaScript code with the needed data embedded in it using JSPs.

Three main points/ requirements we need to consider are:

We need to be able to easily edit our JavaScript code (spread through multiple files right now with global variables). We can't edit anything without maybe (no testing framework) breaking everything.

We want to be able to make calls on the client side to render a view without knowing specifics about the View (in case we add/ change a View).

We need to test our JavaScript code (for obvious reasons)

Currently we have our JavaScript code very spread around and in large files which makes it hard to make edits. I've been looking into the module pattern to address this concern, but many of the posts I've been reading have been:

(a) Older (most recent from 2010)

(b) Had some disagreement about how to make modules that are also testable

Any ideas about ways we should change/ restructure our application to accomplish all three goals? Do these goals have too much inherent conflict, or is there hope? Also, any helpful blog posts are appreciated.

答:
So as a follow up, doing almost exactly as Benjamin Gruenbaum said worked in this instance.

"templating" the code, or having one piece of code write another one was bad for many reasons, including performance (no caching of JavaScript code), maintainability and test-ability. I combed apart the system into a server part that encoded the needed data into JSON and then sent it to our JavaScript code on the client side, as well as started to organize the code into modules, which helped tidy things up a lot so far, and I foresee leading to better (again) maintainability and test-ability.

I can also endorse http://addyosmani.com/resources/essentialjsdesignpatterns/book/ as a great, current resource on how to write bigger pieces of JavaScript code in a well-organized and generally right way.


答:
Interesting.

This takes the simulation approach.

```<code>private void test() {
  double litres = 6;
  for ( int i = 1; i < 19; i++ ) {
    System.out.println("Water in glass "+i+" = "+getWater(litres, i));
  }
}

private double getWater(double litres, int whichGlass) {
  // Don't need more glasses than that.
  /*
   * NB: My glasses are numbered from 0.
   */
  double[] glasses = new double[whichGlass];
  // Pour the water in.
  pour(litres, glasses, 0);
  // Pull out the glass amount.
  return glasses[whichGlass-1];
}

// Simple non-math calculator for which glass to overflow into.
// Each glass overflows into this one and the one after.
// Only covers up to 10 glasses (0 - 9).
int[] overflowsInto =
{1,
 3, 4,
 6, 7, 8,
 10, 11, 12, 13,
 15, 16, 17, 18, 19};

private void pour(double litres, double[] glasses, int which) {
  // Don't care about later glasses.
  if ( which < glasses.length ) {
    // Pour up to 1 litre in this glass.
    glasses[which] += litres;
    // How much overflow.
    double overflow = glasses[which] - 1;
    if ( overflow > 0 ) {
      // Remove the overflow.
      glasses[which] -= overflow;
      // Split between two.
      pour(overflow / 2, glasses, overflowsInto[which]);
      pour(overflow / 2, glasses, overflowsInto[which]+1);
    }
  }
}
</code>```

which prints (for 6 liters):

```<code>Water in glass 1 = 1.0
Water in glass 2 = 1.0
Water in glass 3 = 1.0
Water in glass 4 = 0.75
Water in glass 5 = 1.0
Water in glass 6 = 0.75
Water in glass 7 = 0.0
Water in glass 8 = 0.25
Water in glass 9 = 0.25
Water in glass 10 = 0.0
...
</code>```

which seems to be about right.

Title: Champaign Fountain Puzzle

Body:
Empty glasses of water are arranged in the following order:

When you pour liquid into the 1st glass if it's full, then the extra liquid would be flown into the glasses 2 and 3 in equal quantities. When glass 2 is full, the extra liquid would be flown into 4 and 5 and so on.

Given an N liters of liquid and maximum capacity of each glass is 1 liter, give the amount of liquid  present in any glass if you empty N liters of liquid by pouring into glass by filling out the function getWaterInBucket(int N, int X) where X is the glass number. So for example if I want have 4 liters at the beginning and I want to find the water in glass 3 the function is getWaterInBucket(4, 3)

How do I solve this programmatically? I tried to find a math solution using Pascal's triangle. This did not work. I considered it to be a tree so I can add a parameter like this getWaterInBucket(BTree root, int N, int X) and then try some recursive solution for each level, but parameters are not allowed in this problem. Is there something obvious, some trick?

Body:
Interesting.

This takes the simulation approach.

private void test() {
  double litres = 6;
  for ( int i = 1; i < 19; i++ ) {
    System.out.println("Water in glass "+i+" = "+getWater(litres, i));
  }
}

private double getWater(double litres, int whichGlass) {
  // Don't need more glasses than that.
  /*
   * NB: My glasses are numbered from 0.
   */
  double[] glasses = new double[whichGlass];
  // Pour the water in.
  pour(litres, glasses, 0);
  // Pull out the glass amount.
  return glasses[whichGlass-1];
}

// Simple non-math calculator for which glass to overflow into.
// Each glass overflows into this one and the one after.
// Only covers up to 10 glasses (0 - 9).
int[] overflowsInto =
{1,
 3, 4,
 6, 7, 8,
 10, 11, 12, 13,
 15, 16, 17, 18, 19};

private void pour(double litres, double[] glasses, int which) {
  // Don't care about later glasses.
  if ( which < glasses.length ) {
    // Pour up to 1 litre in this glass.
    glasses[which] += litres;
    // How much overflow.
    double overflow = glasses[which] - 1;
    if ( overflow > 0 ) {
      // Remove the overflow.
      glasses[which] -= overflow;
      // Split between two.
      pour(overflow / 2, glasses, overflowsInto[which]);
      pour(overflow / 2, glasses, overflowsInto[which]+1);
    }
  }
}

which prints (for 6 liters):

Water in glass 1 = 1.0
Water in glass 2 = 1.0
Water in glass 3 = 1.0
Water in glass 4 = 0.75
Water in glass 5 = 1.0
Water in glass 6 = 0.75
Water in glass 7 = 0.0
Water in glass 8 = 0.25
Water in glass 9 = 0.25
Water in glass 10 = 0.0
...

which seems to be about right.


问:
Egoless programming, is it possible or even desirable
Is it possible to have egoless programming or is it even desirable?

As a profession we do seem to want to show off the latest gizmo, technique or say “look at this awesome piece of code I’ve written”. Yet we can get very defensive when asked to submit items of work for code reviews or get negative comments from other programmers (hearing the term WTF, has never been a good sign).

Can we as a profession be able to sit down and analyse a piece of code, data or architecture for its merits or constraints and a calm and respectful manner, without causing offense or antagonising our colleagues, or are we just the archetypal Dilbert character, raging against the system?

答:
Yes, egoless programming is quite possible. I've done "maintenance" programming for more than 1/2 my working career, and the second most important thing [1] is to make sure that the next guy can read my code. We have two programmers here at the office [2] who write in such a distinctive style that one can immediately tell who wrote them. We call the little bundles of code and joy that he leaves for us "robstacles."

Some developers feel that the code they write is the result of their sweat and blood, and they get very defensive when questioned (such as during code reviews). Try not to attack them during code reviews and it will become easier in the future to discuss code as the posturing and defensive behaviors will die out.

Notes:

1. The most important thing is to fix the bug or write the feature correctly.

2. Most of these applications have been shipping for more than  a decade, so it is quite likely that they will be around long after I am gone.

ContentLicense: CC BY-SA 2.5


问:
How to improve efficiency with functional programming?
I've recently been going through the Learn You a Haskell for Great Good guide and as practice I wanted to solve Project Euler Problem 5 with it, which specifies:

What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20?

I decided to first write a function determining whether a given number is divisible by these numbers:

```<code>divisable x = all (\y -> x `mod` y == 0)[1..20]
</code>```

Then I calculated the smallest one using head:

```<code>sm = head [x | x <- [1..], divisable x]
</code>```

And finally wrote the line to display the result:

```<code>main = putStrLn $ show $ sm
</code>```

Unfortunately this took about 30 seconds to finish. Doing the same thing with the numbers 1 to 10 yields a result almost immediately, but then again the result is much smaller than the solution for 1 to 20.

I solved it earlier in C and there the result for 1 to 20 was also calculated almost instantly. This leads me to believe I'm misunderstanding how to interpret this problem for Haskell. I looked through other people's solutions and found this:

```<code>main = putStrLn $ show $ foldl1 lcm [1..20]
</code>```

Fair enough, this uses a built-in function, but why is the end result so much slower when doing it yourself? The tutorials out there tell you how to use Haskell, but I don't see much help with transforming algorithms into fast code.

答:
First you need to make sure you have an optimized binary, before thinking the language is the problem. Read the Profiling and optimization chapter in Real Wolrd Haskell. It is worth noting that in most cases the high-level nature of the language costs you at least some of the performance.

However, note that the other solution is not faster because it uses a built-in function, but simply because it utilizes a much faster algorithm: to find the least common multiple of a set of numbers you need to only find a few GCDs. Compare this with your solution, which cycles through all of the numbers from 1 to foldl lcm [1..20]. If you try with 30, the difference between the runtimes will be even greater.

Take a look at complexities: your algorithm has O(ans*N) runtime, where ans is the answer and N is the number up to which you are checking for divisibility (20 in your case).

The other algorithm executes N times lcm, however lcm(a,b) = a*b/gcd(a,b), and GCD has complexity O(log(max(a,b))). Therefore the second algorithm has complexity O(N*log(ans)). You can judge for yourself which is faster.

So, to summarize:

Your problem is your algorithm, not the language.

Note that there are specialized languages that are both functional and focused on math-heavy programs, like Mathematica, which for math-focused problems is probably faster than almost anything else. It has a very optimized library of functions, and it supports the functional paradigm (admittedly it also supports imperative programming).


问:
Alternative for Access+SQL Server for enterprise projects
We're a small software company, developing projects for manufacturing facilities about analysis, traceability, reporting etc. We're using Access for front end, SQL Server for back end. We've quite big customers too and our company is growing. So far it is working fine but I wonder should we move to more influential technologies, such as web based solutions. What do you think about the future of Access?</p>

答:
Microsoft Access no longer supports direct interaction with SQL Server (Access Data Projects) it's been removed from Access 2013- the only option is to go through a SharePoint enterprise external connector license which is literally $60 grand.

OpenOffice / LibreOffice has some great abilities to connect directly to mySQL. I haven't used them for a couple of years, but I'll go out on a limb and say that they're easier to use than Access / Jet / Linked Tables.  Client-Server is a more efficient algorithm, it always has been.. and it always will be.

Also, Microsoft "Visual Studio Lightswitch" is a great alternative to Access. It allows simple web-based data entry.. I've never integrated Reporting Services with lightswitch, but it seems like a dream come true.


答:
My first thought was that only numbers divisible by all primes <= 20 will be divisible by all numbers less than 20.  So you only need to consider numbers that are multiples of 2*3*5*7*11*13*17*19.  Such a solution checks 1/9,699,690 as many numbers as the brute-force approach.  But your fast-Haskell solution does better than that.

If I understand the "fast Haskell" solution, it uses foldl1 to apply the lcm (least common multiple) function to the list of numbers from 1 to 20.  So it would apply lcm 1 2, yielding 2.  Then lcm 2 3 yielding 6.  Then lcm 6 4 yielding 12, and so on.  In this way, the lcm function is only called 19 times to yield your answer.  In Big O notation, that's O(n-1) operations to arrive at a solution.

Your slow-Haskell solution goes through the numbers 1-20 for every number from 1 to your solution.  If we call solution s, then the slow-Haskell solution performs O(s * n) operations.  We already know that s is over 9 million, so that probably explains the slowness.  Even if all shortcuts and gets an average of half-way through the list of numbers 1-20, that's still only O(s * n/2).

Calling head does not save you from doing these calculations, they have to be done in order to calculate the first solution.

Thanks, this was an interesting question.  It really stretched my Haskell knowledge.  I wouldn't be able to answer it at all if I hadn't studied algorithms last fall.

Title: How to improve efficiency with functional programming?

Body:
I've recently been going through the Learn You a Haskell for Great Good guide and as practice I wanted to solve Project Euler Problem 5 with it, which specifies:

What is the smallest positive number that is evenly divisible by all of the numbers from 1 to 20?

I decided to first write a function determining whether a given number is divisible by these numbers:

```<code>divisable x = all (\y -> x `mod` y == 0)[1..20]
</code>```

Then I calculated the smallest one using head:

```<code>sm = head [x | x <- [1..], divisable x]
</code>```

And finally wrote the line to display the result:

```<code>main = putStrLn $ show $ sm
</code>```

Unfortunately this took about 30 seconds to finish. Doing the same thing with the numbers 1 to 10 yields a result almost immediately, but then again the result is much smaller than the solution for 1 to 20.

I solved it earlier in C and there the result for 1 to 20 was also calculated almost instantly. This leads me to believe I'm misunderstanding how to interpret this problem for Haskell. I looked through other people's solutions and found this:

```<code>main = putStrLn $ show $ foldl1 lcm [1..20]
</code>```

Fair enough, this uses a built-in function, but why is the end result so much slower when doing it yourself? The tutorials out there tell you how to use Haskell, but I don't see much help with transforming algorithms into fast code.


问:
How to remove duplicate exception block code
I have good number of Service and DAO classes which has the same set of 30 line exception code block which gets repeated and it shows up in Code Duplication report.

The approach which I can think of is to

Extract the common exception code blocks to a common method and call

the method in one shot

Re-organize the methods to ensure there is always one entry and exit
point to Services or DAOs which will have just one set of exception
code.

Are there any better ways to approach this scenario?

答:
Consider building an explicit exception handler.

It's a similar approach as your first suggestion but places the methods within an object instead.  It's also similar to older, monolithic applications that had dedicated interrupt handlers.  Either way, the point is the same - write the handling code once and then call as needed.


问:
Is there any reason to use "plain old data" classes?
In legacy code I occasionally see classes that are nothing but wrappers for data. something like:

```<code>class Bottle {
   int height;
   int diameter;
   Cap capType;

   getters/setters, maybe a constructor
}
</code>```

My understanding of OO is that classes are structures for data and the methods of operating on that data. This seems to preclude objects of this type. To me they are nothing more than structs and kind of defeat the purpose of OO. I don't think it's necessarily evil, though it may be a code smell.

Is there a case where such objects would be necessary? If this is used often, does it make the design suspect?

答:
Structs have their place, even in Java. You should only use them if the following two things are true:

You just need to aggregate data that doesn't have any behavior, e.g. to pass as a parameter

It doesn't matter one bit what sort of values that aggregate data has

If this is the case, then you should make the fields public and skip the getters/setters. Getters and setters are clunky anyway, and Java is silly for not having properties like a useful language. Since your struct-like object shouldn't have any methods anyway, public fields make the most sense.

However, if either one of those do not apply, you're dealing with a real class. That means all fields should be private. (If you absolutely need a field at a more accessible scope, use a getter/setter.)

To check if your supposed-struct has behavior, look at when the fields are used. If it seems to violate tell, don't ask, then you need to move that behavior into your class.

If some of your data shouldn't change, then you need to make all those fields final. You might consider making your class immutable. If you need to validate your data, then provide validation in the setters and constructors. (A useful trick is to define a private setter and modify your field within your class using only that setter.)

Your Bottle example would most likely fail both tests. You could have (contrived) code that looks like this:

```<code>public double calculateVolumeAsCylinder(Bottle bottle) {
    return bottle.height * (bottle.diameter / 2.0) * Math.PI);
}
</code>```

Instead it should be

```<code>double volume = bottle.calculateVolumeAsCylinder();
</code>```

If you changed the height and diameter, would it be the same bottle? Probably not. Those should be final. Is a negative value ok for the diameter? Must your Bottle be taller than it is wide? Can the Cap be null? No? How are you validating this? Assume the client is either stupid or evil. (It's impossible to tell the difference.) You need to check these values.

This is what your new Bottle class might look like:

```<code>public class Bottle {

    private final int height, diameter;

    private Cap capType;

    public Bottle(final int height, final int diameter, final Cap capType) {
        if (diameter < 1) throw new IllegalArgumentException("diameter must be positive");
        if (height < diameter) throw new IllegalArgumentException("bottle must be taller than its diameter");

        setCapType(capType);
        this.height = height;
        this.diameter = diameter;
    }

    public double getVolumeAsCylinder() {
        return height * (diameter / 2.0) * Math.PI;
    }

    public void setCapType(final Cap capType) {
        if (capType == null) throw new NullPointerException("capType cannot be null");
        this.capType = capType;
    }

    // potentially more methods...

}
</code>```

Body:
Structs have their place, even in Java. You should only use them if the following two things are true:

You just need to aggregate data that doesn't have any behavior, e.g. to pass as a parameter

It doesn't matter one bit what sort of values that aggregate data has

If this is the case, then you should make the fields public and skip the getters/setters. Getters and setters are clunky anyway, and Java is silly for not having properties like a useful language. Since your struct-like object shouldn't have any methods anyway, public fields make the most sense.

However, if either one of those do not apply, you're dealing with a real class. That means all fields should be private. (If you absolutely need a field at a more accessible scope, use a getter/setter.)

To check if your supposed-struct has behavior, look at when the fields are used. If it seems to violate tell, don't ask, then you need to move that behavior into your class.

If some of your data shouldn't change, then you need to make all those fields final. You might consider making your class immutable. If you need to validate your data, then provide validation in the setters and constructors. (A useful trick is to define a private setter and modify your field within your class using only that setter.)

Your Bottle example would most likely fail both tests. You could have (contrived) code that looks like this:

public double calculateVolumeAsCylinder(Bottle bottle) {
    return bottle.height * (bottle.diameter / 2.0) * Math.PI);
}

Instead it should be

double volume = bottle.calculateVolumeAsCylinder();

If you changed the height and diameter, would it be the same bottle? Probably not. Those should be final. Is a negative value ok for the diameter? Must your Bottle be taller than it is wide? Can the Cap be null? No? How are you validating this? Assume the client is either stupid or evil. (It's impossible to tell the difference.) You need to check these values.

This is what your new Bottle class might look like:

public class Bottle {

    private final int height, diameter;

    private Cap capType;

    public Bottle(final int height, final int diameter, final Cap capType) {
        if (diameter < 1) throw new IllegalArgumentException("diameter must be positive");
        if (height < diameter) throw new IllegalArgumentException("bottle must be taller than its diameter");

        setCapType(capType);
        this.height = height;
        this.diameter = diameter;
    }

    public double getVolumeAsCylinder() {
        return height * (diameter / 2.0) * Math.PI;
    }

    public void setCapType(final Cap capType) {
        if (capType == null) throw new NullPointerException("capType cannot be null");
        this.capType = capType;
    }

    // potentially more methods...

}


问:
Version control: Dealing with incomplete/broken code
It seems to be a generally accepted good practice not to push invalid, broken, or incomplete code. But one of the huge advantages of version control systems is that it gives you a remote place for your code, so you can work on the same codebase from more than one place.

Suppose you are working on a feature with a couple other developers, and a bad winter storm is announced; everybody agrees to go home and keep working rather than get caught in the storm, but nobody is at an ideal stopping place. What do you do?

Create different branches for each developer and commit your changes to that, push those branches to remote and hope you can clean it up later on remote without making life difficult?

Create a different remote just for incomplete code?

Copy your code to your personal flash drive and hope you don't lose it later, or get caught violating company policy?

Something else?

(We use git, but I would hope the answer could be general to any version control system.)

答:
If you're using a DVCS then it's as simple as "4. Create a remote repository for each developer."

For most CVCSes, you really have to use branches. TFS offers shelvesets which can be used for this purpose.


答: In normal usage, I wouldn't commit changes to source control until a feature is complete and tested.  The "bad winter storm" seems like a rare occurrence, which might warrant special rules.  If someone was not at an ideal stopping place, then I would ask them to save those changes to another drive, or perhaps zip them up and email them to the team lead.  If both of those are prohibited by your work environment, then I would break the normal usage and just have them commit the incomplete code to source control.  If it breaks anything, then it is pretty easy to rollback.  And this should be a rare occurrence rather than the norm.</p>

Title: Version control: Dealing with incomplete/broken code

Body:
It seems to be a generally accepted good practice not to push invalid, broken, or incomplete code. But one of the huge advantages of version control systems is that it gives you a remote place for your code, so you can work on the same codebase from more than one place.

Suppose you are working on a feature with a couple other developers, and a bad winter storm is announced; everybody agrees to go home and keep working rather than get caught in the storm, but nobody is at an ideal stopping place. What do you do?

Create different branches for each developer and commit your changes to that, push those branches to remote and hope you can clean it up later on remote without making life difficult?

Create a different remote just for incomplete code?

Copy your code to your personal flash drive and hope you don't lose it later, or get caught violating company policy?

Something else?

(We use git, but I would hope the answer could be general to any version control system.)


问:
Using a stream manipulator (endl) or a newline escape character (\n)?
I don't have a specific context in which I'm asking the question, but while I was reading a beginner book on C++ I noticed the use of both an endl stream manipulator and a newline escape character when dealing with a stream object.

The exmaple is as follows:

```<code>cout << "Hello World" << endl;
cout << "Hello World\n";
</code>```

My questions are:

Is it more appropriate to use the stream manipulator (endl) in a certain
situation and an escape character in a different one?

Are there drawbacks efficiency wise to using one of the two?

Are they completely interchangeable?

I read that an escape sequence is stored in memory as a single character. Does that mean it is more appropriate to use endl if you're going for low memory consumption?

Does the stream manipulator endl use up memory in any way, if so is it more than the escape sequence?

Thanks, StackExchange
Apologies if I posted this in the wrong section, I thought it counted as data structures.

答:
1) For portability, use endl. Windows newlines are \r\n, Linux \n and Mac \r. Edit: As per comments, line system specific endings are handled at a lower level.

2) endl flushes the stream, "\n" does not.

3) Depends on portability.

As to memory usage, you can minimize it by flushing to other storage as often as possible with endl. However, it'll decrease performance.

Edit: Clear up some mistakes.


问:
What are some things you have read that inspired and guided you as a programmer?
For example, being a beginner, I find a lot of inspiration and direction from reading this post by Bryan Woods.</p>

答:
Soul of a new Machine (inspiration when I was in high school)
SICP (probably one of the best books related to software development I have ever read)</p>


答: This is where good branching strategies shine -- nothing incomplete should get into master but it don't mean that incomplete and work in progress code can't hit the shared repository. There are lots of good reasons to do that such as backup / switching machines / switching locations / help & review / gives us seniors a good idea that the juniors are doing more than playing WoW.


答:
o << std::endl is equivalent to the following code:

```<code>o.put(o.widen('\n'));
o.flush();
</code>```

In other words, you should only use std::endl when you need to flush the stream.  For example:

You're about to perform a time-consuming operating and want to make sure you display a message before doing so.

Another process is waiting on your output.

If more than one thread or process is running, you may need to flush output so that the output from each thread or process is displayed correctly.  (Otherwise, you may get seemingly random blocks of output interleaved at awkward intervals.)

If you don't need to flush the stream, then use \n instead of std::endl.  The extra calls to flush can hurt performance (sometimes significantly).

For more details, see cppreference.com.

Regarding memory usage:  Worrying about \n versus std::endl is almost certainly an unnecessary micro-optimization, but in general, I'd expect \n to take less memory.  \n is just one more byte at the end of a string literal, while writing std::endl is translated by the compiler into (probably inlined) function calls to put and flush.

Platform-specific differences in line endings (Windows \r\n versus Linux and OS X \n) are handled at a lower level than std::endl and \n:

If a stream is opened in text mode, then if you write \n, it automatically translates it to the appropriate platform-specific line ending.  If you read a platform-specific line ending, the stream automatically translates it to \n.

If a stream is opened in binary mode, then they pass any line endings through verbatim, and it's up to you.

For std::cout and std::cin in particular, they're treated as if they're text mode.

Title: Using a stream manipulator (endl) or a newline escape character (\n)?

Body:
I don't have a specific context in which I'm asking the question, but while I was reading a beginner book on C++ I noticed the use of both an endl stream manipulator and a newline escape character when dealing with a stream object.

The exmaple is as follows:

```<code>cout << "Hello World" << endl;
cout << "Hello World\n";
</code>```

My questions are:

Is it more appropriate to use the stream manipulator (endl) in a certain
situation and an escape character in a different one?

Are there drawbacks efficiency wise to using one of the two?

Are they completely interchangeable?

I read that an escape sequence is stored in memory as a single character. Does that mean it is more appropriate to use endl if you're going for low memory consumption?

Does the stream manipulator endl use up memory in any way, if so is it more than the escape sequence?

Thanks, StackExchange
Apologies if I posted this in the wrong section, I thought it counted as data structures.

Body:
o << std::endl is equivalent to the following code:

o.put(o.widen('\n'));
o.flush();

In other words, you should only use std::endl when you need to flush the stream.  For example:

You're about to perform a time-consuming operating and want to make sure you display a message before doing so.

Another process is waiting on your output.

If more than one thread or process is running, you may need to flush output so that the output from each thread or process is displayed correctly.  (Otherwise, you may get seemingly random blocks of output interleaved at awkward intervals.)

If you don't need to flush the stream, then use \n instead of std::endl.  The extra calls to flush can hurt performance (sometimes significantly).

For more details, see cppreference.com.

Regarding memory usage:  Worrying about \n versus std::endl is almost certainly an unnecessary micro-optimization, but in general, I'd expect \n to take less memory.  \n is just one more byte at the end of a string literal, while writing std::endl is translated by the compiler into (probably inlined) function calls to put and flush.

Platform-specific differences in line endings (Windows \r\n versus Linux and OS X \n) are handled at a lower level than std::endl and \n:

If a stream is opened in text mode, then if you write \n, it automatically translates it to the appropriate platform-specific line ending.  If you read a platform-specific line ending, the stream automatically translates it to \n.

If a stream is opened in binary mode, then they pass any line endings through verbatim, and it's up to you.

For std::cout and std::cin in particular, they're treated as if they're text mode.


问:
Using a "dead man's switch" to manage time-sensitive code
In our software environment, we often run a/b tests, as is probably good practice. However, our environment is set up such that, in very short order, the code starts to become very crufty with dead tests. The testing registry is little more than a collection of internal wiki pages.

I thought of a "dead man's switch" style of defunct code management. If you're not familiar with the term, it refers to a switch that must be reset periodically in order to keep something from triggering -- in essence, if you don't responde, the switch triggers, and whatever you wanted the switch to trigger is performed.

For example, I would write some code, register it with this system in some way, and when a date of my predetermined choosing rolled around, I would get a notification that this code would be removed (automatically cleaned up) unless I intervened (manually clean up, or snooze).

What are the pros, cons and viability of incorporating such a system? Is it possible or wise? What might be some alternative ways to manage code against rotting?

答:
I'd be worried about a system that automatically removes code. Unless your team is very well diciplined, I can only see this as a road to tears and pain. Things happen: people go on vacation, get sick, leave the company, forget what the code that's about to expire does, ... and having code be automatically cleaned up sounds like it will invite all kinds of trouble. You'd have to ensure that there are no compiler dependencies between these modules, or they could suddenly fail to compile when something they depend on gets expired out of existence. And don't forget what would happen if someone in production support is working on a older trouble ticket and the stack trace in the error message references code that was deleted. And this would not make people who do auditing very happen either, I imagine...

Something that might be safer is a framework where your little modules of code can query for an activation date and a deactivation date. When the piece of code is about to run, it checks in a database for what is dates are and if the current date is between the activation and deactivation date, your framework allows the code to run. And if you store these dates in a database, you could easily write a script that will generate a report of all modules expiring in the next n days, and email it to you so it's the first thing in your inbox on Monday morning.


答:
This system creates a problem of orphaned tests: if someone who wrote a suite of tests and a set of production code associated with it departs the company, the tests are in danger of being removed prematurely by omission of their new owner.

I do not think that "too many tests" is a problem: as long as tests are automated, your waste is limited mostly to CPU hours, which is small change compared to man hours. A test removed automatically through the dead man's switch could catch bugs that would otherwise end up in production, causing severe maintenance problems down the road.

I think you can build a process-based alternative by creating a registry pairing up test suites with modules of your code. Every time a maintenance is performed on a module, the process would require making a decision on keeping/removing/updating the corresponding test suite. Since this is a pure bookkeeping, there is no danger of code being removed from the list automatically.

Title: Using a "dead man's switch" to manage time-sensitive code

Body:
In our software environment, we often run a/b tests, as is probably good practice. However, our environment is set up such that, in very short order, the code starts to become very crufty with dead tests. The testing registry is little more than a collection of internal wiki pages.

I thought of a "dead man's switch" style of defunct code management. If you're not familiar with the term, it refers to a switch that must be reset periodically in order to keep something from triggering -- in essence, if you don't responde, the switch triggers, and whatever you wanted the switch to trigger is performed.

For example, I would write some code, register it with this system in some way, and when a date of my predetermined choosing rolled around, I would get a notification that this code would be removed (automatically cleaned up) unless I intervened (manually clean up, or snooze).

What are the pros, cons and viability of incorporating such a system? Is it possible or wise? What might be some alternative ways to manage code against rotting?


答:
Sounds like a very bad idea, because later code will depend on those changes. Automatic rollback will cause widespread damage.

Instead, write your tests as automated (unit) tests, and run them nightly.


答:
I have long used simple calendar alerts (whatever calendar software your company uses should suffice) where I simply set it to alert me at whatever time and I put all the information I'll need presume I forgot everything by the time the alert goes off.

Install an ssl cert? Look at the expiration date, set a calendar event for myself (and a manager and another engineer in case one or 2 of the 3 of us happen to no longer be with the company, which has happened) 2 months before that to start sounding the alarm to whomever that it need be replaced, put in the event the contact information for the people we received the cert from, the specific places the cert is used and details of systemic ramifications of letting it expire (maybe it's a dead system and the cert should expire when that time comes around).

Automatic code modification is a downright terrible, dangerous idea. All you need to do is judiciously use your calendar to make sure you are notified of time sensitive things when their time becomes relevant. This is specifically what calendars are for. Now if you have a problem of not heeding time sensitive events in a responsible manner, I think you have an altogether different problem that you might want to ask on the workplace about.


答:
Extract the exception handling into a method:

```<code>public interface Action {
    public void execute() throws ...;
}

public static void handleErrors(Action action) throws ... {
   try {
      // setup .....
      action.execute();
   } catch (...) {
      // ...
   } finally {
      // cleanup ...
   }
}
</code>```

Then call it like this:

```<code>handleErrors(new Action(){ public void execute() throws ... {
    // ...
}});
</code>```

The syntax is ugly, but it's still much better than duplicating the error handling dozens of times.  Anytime you are duplicating more than a couple of lines, you should stop and figure out how to avoid that duplication.  In Java, this often means using interfaces and anonymous classes to pass one block of code to run inside some common control logic.  The same technique can be reused to eliminate duplicated looping code.

Title: How to remove duplicate exception block code

Body:
I have good number of Service and DAO classes which has the same set of 30 line exception code block which gets repeated and it shows up in Code Duplication report.

The approach which I can think of is to

Extract the common exception code blocks to a common method and call

the method in one shot

Re-organize the methods to ensure there is always one entry and exit
point to Services or DAOs which will have just one set of exception
code.

Are there any better ways to approach this scenario?

Body:
Extract the exception handling into a method:

public interface Action {
    public void execute() throws ...;
}

public static void handleErrors(Action action) throws ... {
   try {
      // setup .....
      action.execute();
   } catch (...) {
      // ...
   } finally {
      // cleanup ...
   }
}

Then call it like this:

handleErrors(new Action(){ public void execute() throws ... {
    // ...
}});

The syntax is ugly, but it's still much better than duplicating the error handling dozens of times.  Anytime you are duplicating more than a couple of lines, you should stop and figure out how to avoid that duplication.  In Java, this often means using interfaces and anonymous classes to pass one block of code to run inside some common control logic.  The same technique can be reused to eliminate duplicated looping code.


答:
There is a lot of grey area on the confidence - arrogance continuum. I'd rather have someone who takes ownership of their work and strives for excellence based on merit and not self-delusion.

Trust me, you'll write a lot of crap that will fail and you will be to blame, so you may as well take advantage when all goes well.

When I think something is wrong, I say so. Feel free to tell me I'm wrong and be prepared to back it up. It won't be the first or the last time.

If you don't want criticism (Obviously tact and restraint can be exercised.), then write code for your eyes only and run it on your own computer. Anyone who has ever excelled at anything has had to deal with critics.

Title: Egoless programming, is it possible or even desirable

Body:
Is it possible to have egoless programming or is it even desirable?

As a profession we do seem to want to show off the latest gizmo, technique or say “look at this awesome piece of code I’ve written”. Yet we can get very defensive when asked to submit items of work for code reviews or get negative comments from other programmers (hearing the term WTF, has never been a good sign).

Can we as a profession be able to sit down and analyse a piece of code, data or architecture for its merits or constraints and a calm and respectful manner, without causing offense or antagonising our colleagues, or are we just the archetypal Dilbert character, raging against the system?


问:
Can we create a GPL application that has a payware advertisement on it?
We've been thinking of using GPL components to make a GPL application, and freely give it away. We then want to use that as a loss-leader. In that free application, where even the source code is available, we plan to have an advertisement for some payware. The payware would be for functionality that is completely different than the free application. For instance, the freeware would be like a checkbook application, let's say, but the payware would be a virus remover utility that people try out and then can pay for, and is closed-source, binary.

Does the GPL permit one to build an application that has a payware advertisement on it?

答:
The GPL allows that.

However, the GPL requires that you give the source to everyone that gets your GPL application and allows anyone to modify and redistribute your code.  If the GPL application you release is useful, it won't take long for someone to take your source, remove the ads, and redistribute the application.  And, presumably, the vast majority of users will prefer the ad-free version rather than the version that you are distributing with ads.  Unless you have some distribution channel that is substantially better than the distribution channel of whoever decides to fork your app and distribute the ad-free version, it seems unlikely that your GPL application would attract many users.


问:
Online compilers and repls - not one big security hole?
There are plenty of compilers and REPL services on the web. For example: Fay ide.

I find that implementing some similar technology would be very interesting. But it seems like a major security hole. Am I wrong in thinking that exposing a compiler would be risky? An interpreter seems even worse.

update: I can be a bit more specific. The kind service I'm thinking about hosting myself (a thought experiment at this stage) would be kind of close to the case of the Fay ide above. You take input from the user, process it and return it, potentially as runable javascript to the same or other users. Let's look at that example in particular.

The most obvious security concern that I see for this case, as I don't actually evaluate the code server side, would be for the users, as they would run code made by an 'unknown' author. The fact that they are allowed to read the code before running it should be a significant safeguard in this regard.

Although I also wonder if and when translation of code constitute a threat to the server? What kind of exploit vectors could be thought of in this context?

Summary:

Beyond what you can expect from any web server set up, and beyond the most obvious concerns (or perhaps some of those obvious concerns as well), how could one user compromise another?
-. How can the server application be hardened to ensure not being compromised?

What important questions have I neglected? ;)

I would like to try some experiments, and I have nearlyfreespeech in mind as a host. I could imagine that such use might be prohibited by their terms of use? Or perhaps I can trust them to be sure enough that they have sandboxed my account well enough. What should be my "minimum" safeguards, for trying stuff out and what safeguards would be possible with such limited hosting environment?

update2: I believe that compiling to javascript for client side evaluation or evaluating limited edsls would cover for most/all use cases I have in mind. I take it that this means that I really don't put the server side to a more prominent risk than usual. And I don't see how the user is being put into more risk than compared to going to a random link on jsFiddle.

答:
If all you're doing is compiling code (taking the user's code and transforming it without actually running it), it's no different than securing any other online program.  You have a program (a compiler) that takes potentially untrusted user input (the user's code) and does something with it, and you need to make sure that it doesn't do anything it shouldn't even when given malicious input.  For a compiler, many of the standard web security risks (SQL injection, CSRF) don't really apply, but you would need to check for risks like these:

For language features like C's #include, make sure you don't permit including files outside of the standard library files.

Some languages can take arbitrarily large CPU or memory to compile (thanks to features like C++'s Turing-complete templates), so you need to guard against denial of service.  However, operating systems and web servers have long had features to terminate processes that take too long or grow too large.

If you're interpreting and running the user's code, then security requires running the user code in a sandbox of some sort.  For example, if you disable all library calls that involve file or network I/O and similar operating system calls, and if you have the aforementioned limits on CPU and memory, then in theory, the user code can't do anything bad.  In practice, this can be very tricky to get right.  For example, Python lets you easily execute code with restricted globals (so you can restrict the os module), but builtin language features and introspection still provide enough functionality to break out of the sandbox.  (See here.)

Another approach to securing an online interpreter or REPL is to execute everything client-side (using JavaScript and, possibly, tools like Emscripten.  If the user code executes on the user's machine, then malicious user code isn't really your problem.

Update: As @JimmyHoffa points out, another approach is to use the OS or VM software to provide a more complete sandbox than the programming language is designed to do itself.  For examples of this, see Chromium's sandbox for Windows or various Linux options.


答:
Take it a step farther than just creating a branch to solve this specific problem and move to creating a branch for each feature, bugfix, etc... that you decide to implement. Each of these branches should be expected to have broken code in it any any time. Once the code is been completed it can be merged / tested against the main branch of your repository.

This will allow you to solve the current issue of where to store incomplete and broken code and help in organizing your thoughts and ideas. If you want to try a new crazy feature in your spare time you simply create a branch for that feature and begin working on it. You can put it away for two months in a broken state and pick it back up at a later date by merging back in changes that have made it into your main branch.

A good outline of a branching in this manner using continuous integration (CI) can be found here: http://nvie.com/posts/a-successful-git-branching-model/


答:
If you work with CVCS you can try rolling with one trunk. I've seen this done very well a few times. The thing to do is to have continuous integration with unit tests, and contiguous testing on that trunk.

One rule: Don't break the build and don't break functionality.

But, there are some 'incomplete' checkins that are acceptable. For example, if you're going to add a feature and you need to check in some model classes that aren't used anywhere - that's OK, just mention what they're for in the commit logs. If you need to extend an existing class to have more functionality - that is also OK.

Frequent checkins like that have 3 benefits:

1) Everyone will see your 'direction' early. They will be able to coordinate with you on their work, and use your changes if they see value in them. One problem with branching for features is that people can simultaneously invent the same wheel.

2) Your stuff will be tested early. One problem with branching for everything is that there needs to be 2 rounds of testing: one for the feature done in branch, and one for the feature after the merge.

3) If things get broken they will be caught by other developers even before it goes to QA. Bugs are better addressed as early as possible.

One caveat: this will work on small-medium sized teams. At some point people start stepping on each-other's toes too much and you will be forced to branch.


答: Seth Godin, while not totally programming-related, has some inspiring posts for me as a programmer and hopeful business person (in addition to Joel on Software).</p>

Title: What are some things you have read that inspired and guided you as a programmer?

Body: For example, being a beginner, I find a lot of inspiration and direction from reading this post by Bryan Woods.</p>


问:
Should heavy binary files not be stored in database?
I was asked an interesting question: Should a database contains all data? Or heavy binary files should be stored in file system?

Example of heavy binary files : videos or heavy pdf files (+200 MB)

With an old aspx web app (1.1) I tried to open a 200MB pdf file stored as a blob in an Oracle 11g database, and it just run out of memory.

However, same asp.net web application had no problem to open same pdf file stored in file system of a server.  It could be that maybe there is some proper way to open heavy blobs fields with asp.net.

For integrity reasons, I say that all data should be stored in database, but my described case showed me that maybe it's not the way.

I read once that if your web application goes to cloud, then it would be very difficult to keep references of files stored in file system (file paths like: ../MyFolder001/MyFile.mpg), since you don't know where those files will be distributed.

Should heavy binary files not be stored in database?

答:
No, large binary files should generally not be stored in the database.

Generally, your filesystem will provide better caching/buffering than your database will for large files

Databases are generally designed and optimized around smaller bits of data - large binary files are often not optimized for in databases

Database storage is generally more expensive (both in terms of CPU time and disk space) than filesystem storage

You often lose opportunities to optimize sending files to a web client when storing them in a database (e.g. sendfile())

You're putting an unncessary load on the database by having to store/fetch large binary objects from the db


答:
It's a question of advantages and disadvantages. Binary files in the database will be stored with all the other data, but you have to pay the overhead including it in the database.

In your case, it sounds like the disadvantages include limited support in the tools you're using. Does the value of having the binary data in the database justify the effort needed to work around those limitations?

Body:
I was asked an interesting question: Should a database contains all data? Or heavy binary files should be stored in file system?

Example of heavy binary files : videos or heavy pdf files (+200 MB)

With an old aspx web app (1.1) I tried to open a 200MB pdf file stored as a blob in an Oracle 11g database, and it just run out of memory.

However, same asp.net web application had no problem to open same pdf file stored in file system of a server.  It could be that maybe there is some proper way to open heavy blobs fields with asp.net.

For integrity reasons, I say that all data should be stored in database, but my described case showed me that maybe it's not the way.

I read once that if your web application goes to cloud, then it would be very difficult to keep references of files stored in file system (file paths like: ../MyFolder001/MyFile.mpg), since you don't know where those files will be distributed.

Should heavy binary files not be stored in database?


答:
The most important reason not to use large blobs is that you cannot stream them - when you query a database, it's the whole column value or nothing. With files, you can open a handle, then read the data as you go - you never need to have the entire file in memory at the same time.

At the same time, many of the advantages of databases, such as more efficient storage of lots of tiny bits of data and more efficient lookup of such values, ACID guarantees, etc., become less important when large files are concerned.

Title: Should heavy binary files not be stored in database?


答:
SQL Server 2012 now has the FileTables. That combine the strategy of appearing to have the data in the database but utilizes the file structure behind the scene.

Your other concern:

I read once that if your web application goes to cloud, then it would
  be very difficult to keep references of files stored in file system
  (file paths like: ../MyFolder001/MyFile.mpg), since you don't know
  where those files will be distributed.

For an application that handles a large number of large files, you better find a cloud solution that will allow you to store them. Changing the file paths (which are probably stored in a database)  can be handled by:

storing the root folder in a single location in your app and hopefully maintaining the rest of the folder and file naming structure.

updating any other sub folder structurs in the database with a script. This is the piece that databases handle well.


问:
Do I really want a dynamic Enum, or something else? Better caching? A Struct?
This question is asked on SO and this site often it looks like.  "How do I do a dynamic enum?"  Answers range from you shouldn't to possible compile time solutions.  Now my question is do I want a dynamic enum that is generated from look up tables in the DB, or do I want something else that is similar?

I have a solution using T4 at build time to generate enums from DB tables, that works fine.  But now some of the look up tables have an 'active' flag on them.  Does this mean they are not look up tables anymore?  If that is changed I want to somehow regenerate my code without having to redeploy.  These tables will not change often.

Currently the app when it wants to populate a dropdown or something will query the DB.  But if the values mostly remain static, why go to the DB every time, hence the use of enums.  It also has the added benefit of getting rid of magic strings and numbers.

Would static structs be better? Some sort of caching for the look up tables?  Any advise welcome.

答:
Don't go to the DB every time. Go to the DB at process start; load into readonly dictionaries your lookup data, and call it good. Stuff changes underneath, you need to restart the app to re-grab those dictionaries.

Alternatively, look at actual resource cost of looking the data up every time, if your app isn't significant enough for this to matter, just let it lookup the data everytime via join.

Cache expiration is a complex thing but two very simple techniques come to mind:

Lock your lookup tables down to being updated by the app and those updates can signal a reinstantiation of your read-only lookup table dictionaries, this is so long as you don't need to worry about people changing the lookup tables directly through management studio or otherwise.

Using SQL server's message broker triggered on updates to lookup tables to send a message to your running app which signals it to replace it's readonly dictionaries with updated versions. This is presuming the updates will be manual SQL scripts

Alternatively use a proper cache like appfabric or memcached for these where you can have appropriate cache expiration if you need. This seems like overkill for what could really be served just fine by my initial suggestion, most systems have admins turn off their app when they go to update their DB directly anyway, and turn it back on only after DB updates are finished.

In summation:

No, you do not want dynamic enums, this goes against what an enum is even for

Enums are named consts. Remember that, consts. As soon as you want a group of key-value pairs you are starting down the path of hashtable/dictionary/associative-array or whatever word you wish to call them, that is precisely the purpose of such structures. Not the purpose of enums.


答:
Whether something is an enum and whether you know it at compile time are 2 separate issues.  In the case where you do always know the values at compile time (even though it changes over releases), and it happens to be convenient to have it as some kind of explicitly typed object in your object oriented language and as a table in the DB, then I think code generating the object from the DB (or anything DRY - generating a DB script via reflection, generating DB script and object generation from something else...) is O.K.

Whether you should model that as an enum is a separate issue.  Enums aren't very flexible when requirements change, so I would be wary.

You also say you don't want to redeploy.  That seems like the enemy of explicitness.  If the changes to your "enums" (or whatever they turn out to be) happen more often then your releases, then you really don't know the values at compile time.  So you should get it from the DB, with or without caching based on the usual considerations.

Title: Do I really want a dynamic Enum, or something else? Better caching? A Struct?

Body:
This question is asked on SO and this site often it looks like.  "How do I do a dynamic enum?"  Answers range from you shouldn't to possible compile time solutions.  Now my question is do I want a dynamic enum that is generated from look up tables in the DB, or do I want something else that is similar?

I have a solution using T4 at build time to generate enums from DB tables, that works fine.  But now some of the look up tables have an 'active' flag on them.  Does this mean they are not look up tables anymore?  If that is changed I want to somehow regenerate my code without having to redeploy.  These tables will not change often.

Currently the app when it wants to populate a dropdown or something will query the DB.  But if the values mostly remain static, why go to the DB every time, hence the use of enums.  It also has the added benefit of getting rid of magic strings and numbers.

Would static structs be better? Some sort of caching for the look up tables?  Any advise welcome.


问:
What should every programmer know about programming?
Please, stay on technical issues, avoid behavior, cultural, career or political issues.

答: Don't stop learning.</p>

Body: Don't stop learning.