问:
What is the best way to load balance javascript applications?
I write a lot of javascript applications and in many different circumstances the browser will become unresponsive or give a "slow script" error. Even when following best practices, initializing large data sets, complex animation, or when too many event handlers fire at once, I have to include extra setTimeouts or requestAnimationFrames around script blocks to load balance them. Seems like there should be some standard way of managing the browser load for large javascript applications.

Any ideas? It seems like there must be JS application designers thinking about this, but I can find nothing on the web or on stack.

edit

I appreciate all the response. My question is not "How can I write efficient Javascript?" which seems to be the question the current responses are answering. My question is: How can I balance the load of multiple sequential threads of javascripts running in the same browser?

I am well aware of all the information posted in the responses so far and appreciate the posters, but I am looking for a framework or a design pattern that balances script execution in a browser. This question is common in many other languages and for devops, I am simply asking the same question for javascript.

答:
As many others have said already, you can't really "load balance" javascript. Unless you're using workers, javascript does one thing at a time. The best you can hope to do is work on getting tasks out of the queue faster than they're coming in. Here's some things to look at:

Be careful with the DOM

A big part of that is, as the other answers state, avoiding DOM redraws, as they can be extremely expensive. Try to do all your DOM reads in one step, and all your writes in another, as reading from the DOM often causes it to draw all the queued changes to make sure it has up to date information. If you're using the information to make more DOM changes, then it'll redraw again, and the user will never even see that first redraw.

Avoid setInterval

If the callback function takes longer than the interval to run, you'll get multiple copies of the callback function in the queue, and you code will just fall farther and farther behind as it runs. This is why many people suggest using a setTimeout that sets itself when it's done. That way, there's only ever one instance of the callback waiting in the queue.

Buffer "Burst" Events

Some events fire several times at once, but really you should only be reacting the last last occurrence. One example of this I've seen often is a scroll event listener. When the user's scrolling the page, this event can fire multiple times a second, and if the listener does any real work, then you're queue's building up way faster than it's clearing. That's why you often see patterns like this:

```<code>var scrollTimer;

function scrollHandler(e) {
    // Actually handle the scrolling, DOM manipulations, real work...
}

window.addEventListener('scroll', function (e) {
    clearTimeout(scrollTimer);
    scrollTimer = setTimeout(scrollHandler, 100, e);
});
</code>```

This way, no matter how much the user scrolls, the heavy code doesn't actually run until they stop scrolling (in this case for a tenth of a second).

Cache Continuous Events

I've found that in some situations certain events come in almost continuously, and have multiple listeners. If you're reacting to the accelerometer, GPS, Game controllers (or keyboards being used as controllers), then the events will probably be coming in as fast as they possibly can, triggering their listeners almost constantly. Everythings trying to happen as fast as possible, and that makes it slow. Unlike the "burst" events, however, you can't just wait for these events to stop before reacting. The whole point is to react as they change.

If you think about it though, your code only needs to react once per frame. Have a single keydown/keyup listener that sets flags for the keys you want. Have a single accelerometer listener update and array with orientation data, or a GPS listener store the latitude and longitude. These listeners should be fast enough to keep the queue emptying as quickly as it fills. With that in place, you can loop through all your objects once a frame and just have them check the flags and arrays for the information they need.

Body:
As many others have said already, you can't really "load balance" javascript. Unless you're using workers, javascript does one thing at a time. The best you can hope to do is work on getting tasks out of the queue faster than they're coming in. Here's some things to look at:

Be careful with the DOM

A big part of that is, as the other answers state, avoiding DOM redraws, as they can be extremely expensive. Try to do all your DOM reads in one step, and all your writes in another, as reading from the DOM often causes it to draw all the queued changes to make sure it has up to date information. If you're using the information to make more DOM changes, then it'll redraw again, and the user will never even see that first redraw.

Avoid setInterval

If the callback function takes longer than the interval to run, you'll get multiple copies of the callback function in the queue, and you code will just fall farther and farther behind as it runs. This is why many people suggest using a setTimeout that sets itself when it's done. That way, there's only ever one instance of the callback waiting in the queue.

Buffer "Burst" Events

Some events fire several times at once, but really you should only be reacting the last last occurrence. One example of this I've seen often is a scroll event listener. When the user's scrolling the page, this event can fire multiple times a second, and if the listener does any real work, then you're queue's building up way faster than it's clearing. That's why you often see patterns like this:

var scrollTimer;

function scrollHandler(e) {
    // Actually handle the scrolling, DOM manipulations, real work...
}

window.addEventListener('scroll', function (e) {
    clearTimeout(scrollTimer);
    scrollTimer = setTimeout(scrollHandler, 100, e);
});

This way, no matter how much the user scrolls, the heavy code doesn't actually run until they stop scrolling (in this case for a tenth of a second).

Cache Continuous Events

I've found that in some situations certain events come in almost continuously, and have multiple listeners. If you're reacting to the accelerometer, GPS, Game controllers (or keyboards being used as controllers), then the events will probably be coming in as fast as they possibly can, triggering their listeners almost constantly. Everythings trying to happen as fast as possible, and that makes it slow. Unlike the "burst" events, however, you can't just wait for these events to stop before reacting. The whole point is to react as they change.

If you think about it though, your code only needs to react once per frame. Have a single keydown/keyup listener that sets flags for the keys you want. Have a single accelerometer listener update and array with orientation data, or a GPS listener store the latitude and longitude. These listeners should be fast enough to keep the queue emptying as quickly as it fills. With that in place, you can loop through all your objects once a frame and just have them check the flags and arrays for the information they need.


问:
Find the peak of each islands in sparse matrix
I have a sparse matrix that contains several islands of unknown size. I would like to find the highest peak of each islands. Consider this matrix as an example:

```<code>0 0 1 0 0 0 0
0 0 1 2 1 0 0
0 0 0 3 2 1 0
0 1 0 0 0 0 0
0 2 3 4 0 1 0
0 0 1 1 0 1 0
0 0 0 0 0 1 0
</code>```

In this case I would like to get the position of 3 at (3, 2), the position of 4 at  (3, 4) and the position of 1 at (5, 4), (5, 5) or (5, 6). Note: I consider 4-connected neighbours.

Until now I came up with the solution that scans each pixel and if it is not zero it starts a flood fill from that pixel. The flood fill paints the pixels to zero keeping track the maximum value visited. This algorithm works well but I was wondering if there are any faster solution?

In my target platform (iOS) I have access to different vector processing frameworks like BLAS, LAPACK, vDSP that provide some functions for fast finding the maximum value in a vector (or matrix). Maybe one could implement a faster solution with the help of these functions?

答:
If I understand your current algorithm correctly it is:

```<code>loop through all n values in your matrix looking for non 0 ones in O(n)
for each of island, flood fill over the m non zero elements and set them to 0 in O(m)
</code>```

given you matrix is sparse i.e. n >> m then this algorithm is going to be O(n) complexity.

We can get better complexity by using a data structure that doesn't store the 0s. for instance if you added your non 0 data to a dictionary with the key being the index of the data (i,j) then we can still get any data in O(1) and rely on a key not being present meaning the value is 0.

```<code>we can then for each non zero element we haven't looked at yet, run the flood fill
and remove every key we look at from the dictionary in a complexity of O(m).
</code>```

Now your question doesn't ask for a lower algorithmic complexity solution but a faster one, is this new algorithm going to help?

Well, it depends:

If your matrix is big enough and sparse enough then this should outperform your current implementation. For a smaller matrix you original algorithm may well end up being faster as it could well have smaller constant factors.

Can you even change your data structures? (answer here appears to be no)

ContentLicense: CC BY-SA 3.0

Body:
If I understand your current algorithm correctly it is:

loop through all n values in your matrix looking for non 0 ones in O(n)
for each of island, flood fill over the m non zero elements and set them to 0 in O(m)

given you matrix is sparse i.e. n >> m then this algorithm is going to be O(n) complexity.

We can get better complexity by using a data structure that doesn't store the 0s. for instance if you added your non 0 data to a dictionary with the key being the index of the data (i,j) then we can still get any data in O(1) and rely on a key not being present meaning the value is 0.

we can then for each non zero element we haven't looked at yet, run the flood fill
and remove every key we look at from the dictionary in a complexity of O(m).

Now your question doesn't ask for a lower algorithmic complexity solution but a faster one, is this new algorithm going to help?

Well, it depends:

If your matrix is big enough and sparse enough then this should outperform your current implementation. For a smaller matrix you original algorithm may well end up being faster as it could well have smaller constant factors.

Can you even change your data structures? (answer here appears to be no)


问:
What is the best way to construct your CSS documents?
Normally I start a project working on a site, I'd do the basic html skeleton and then start tweaking the styles using CSS. Slowly I'd just add one CSS rule after another with no "organized structure" or groupings in the CSS rules.

And at the end of the project I look at the CSS document and the abomination I have created. Often spending quite some time trying to search for the spot in the document where I added a certain style.

Is there some type of 'rule-of-thumb' that one can use in laying out and getting more organized in your CSS documents? And I'm not just talking about adding comments...

答:
a good advice is to start with an existing web framework like bootstrap or foundation, these well help you get a default well written clean and fast CSS, which you can extend and use as a guide for your style sheet.

classes should be descriptive and with as few actions as possible, also splitting your CSS into multiple ones that are concern separated, i mean one for
buttons and one for common etc, this will help adding some organization to your CSS.

try to organize your CSS with a top-down structure, starting with generics like body and p, afterward #xxxx #yyyy, use multiple classes to style a tag, stuff like, class="pull-left title".

if you used a framework all best practices are utilized, try to follow the same method as much as possible.

Edit - Comment reply-----

i personal use different CSS for my site customization, because if i needed to upgrade the framework in case they released something new i won't have to keep track of changes manually.

and i add a different css to frame work that i call extension, which init i keep everything that i see useful for the future to be carried out to next project, so this would act as my an extension on any framework i use.

Yours,
OQ


问:
What's the best documentation you produce and why?
I think it's safe to assume that for most programmers, producing documentation is not as fun as actually coding. I think it's also safe to assume that most good programmers recognize the need for useful documentation, and the code that they write is not an exception by any stretch of the imagination.

So, I'd like to know: what's the best documentation you produce (and source code doesn't count). Answer can be anything from comments to unit tests.

The bigger question is why is that the best documentation you produce?

答: Any documentation written in natural language is susceptible to ambiguity.  Code that is neatly written and well commented can reduce the ambiguity totally or almost totally.  So, it should be a combination of code with comments and the specs that were used to write the code.</p>

ContentLicense: CC BY-SA 2.5


问:
Statistics collection engine for C++ systems
We have a research project with idea->prototype->statistics development cycle.
Anyway, our final product is a prototype, so the statistics collection suite is
not used persistently. Supposing I have following class:

```<code>class Transform {
    private:
        someData;

    public:
        transformForward (inp_block, outp_block);
        transformBackward (inp_block, outp_block);
};
</code>```

Imagine that it is a part of a big system. I need to periodically collect some statistics on such transform (internal data could be considered as well). It seems that adding statistical routines would be a violation of single-responsibility principle. The second smell is that I do not want to touch code that uses Transform instances to explicitly invoke these routines. I would be best if I could just
trigger some kind of switch so that the statistics for that module will be collected.

I've met that challenge a number of times and I have a feeling that I'm constantly reinventing the wheel. Is there some good practices for configuring and collecting the statistics suite for a compound system without interfering into it's internal code base?

UPDATE:

As I can see from the answers proposed, my question is too non-specific, so I'll provide more concrete example.

Consider an image-compression system composed by two huge blocks: Predictor and Encoder. There are a lot of various prediction and compression algorithms, during our research we need to explore the behavior of the components under various conditions. We should answer questions like "how many times the pixel is processed within each context", "how well does this predictor works", "how does each predictor affects the Encoder's internal state" and many others.

Anyway, our final product is just a Codec with no statistical suite shipped with it; all kind of statistics collection is used internally during our research. Thus the question arises: how could one build flexible statistics engine that knows the very internals of the system? How could one keep the system itself independent of the statistics engine?

答:
If the internal state of the transform implementation is not considered, Decorator seems to be the answer. Otherwise, see Bart van Ingen Schenau's answer.

Before that could happen, two refactorings are needed on the existing code base:

Create an interface for the methods of the Transform classes. This is Abstraction.

Existing code that uses the Transform classes must allow higher-level code to specify which TransformInterface implementation to be used. This is Dependency Injection.

A decorator class responsible for capturing statistics from the data that pass into and out of the Transform interface would:

Implement the Transform interface

Decorate (wrap-around) a concrete implementation of Transform

Have transformForward and transformBackward methods which gathers the data, and then forward the call to the concrete implementation.

```

class TransformInterface
{
public: <br>
    virtual ~TransformInterface() = 0;
    virtual void transformForward ( ... ) = 0;
    virtual void transformBackward ( ... ) = 0;
};

class TransformStatisticsGatherer : public TransformInterface
{
public:
    TransformStatisticsGatherer( TransformInterface * pConcreteTransform )
    {
        myConcreteTransform = pConcreteTransform;
    }
    void transformForward ( void* inputBlock, void* outputBlock )
    {
        ... // do something with inputBlock and outputBlock _before_ the call
        myConcreteTransform -> transformForward ( inputBlock, outputBlock );
        ... // do something with inputBlock and outputBlock after_ the call
    }
    void transformBackward ( void* inputBlock, void* outputBlock )
    {
        ... // do something with inputBlock and outputBlock _before_ the call
        myConcreteTransform -> transformBackward ( inputBlock, outputBlock );
        ... // do something with inputBlock and outputBlock _after_ the call
    }
    void getStatistics ( ... )
    {
        ... // return the gathered statistics
    }
private:
    TransformInterface* myConcreteTransform;
    ... // additional private fields for statistics gathering
};

```

Body:
If the internal state of the transform implementation is not considered, Decorator seems to be the answer. Otherwise, see Bart van Ingen Schenau's answer.

Before that could happen, two refactorings are needed on the existing code base:

Create an interface for the methods of the Transform classes. This is Abstraction.

Existing code that uses the Transform classes must allow higher-level code to specify which TransformInterface implementation to be used. This is Dependency Injection.

A decorator class responsible for capturing statistics from the data that pass into and out of the Transform interface would:

Implement the Transform interface

Decorate (wrap-around) a concrete implementation of Transform

Have transformForward and transformBackward methods which gathers the data, and then forward the call to the concrete implementation.

class TransformInterface
{
public:

    virtual ~TransformInterface() = 0;
    virtual void transformForward ( ... ) = 0;
    virtual void transformBackward ( ... ) = 0;
};

class TransformStatisticsGatherer : public TransformInterface
{
public:
    TransformStatisticsGatherer( TransformInterface * pConcreteTransform )
    {
        myConcreteTransform = pConcreteTransform;
    }
    void transformForward ( void* inputBlock, void* outputBlock )
    {
        ... // do something with inputBlock and outputBlock _before_ the call
        myConcreteTransform -> transformForward ( inputBlock, outputBlock );
        ... // do something with inputBlock and outputBlock after_ the call
    }
    void transformBackward ( void* inputBlock, void* outputBlock )
    {
        ... // do something with inputBlock and outputBlock _before_ the call
        myConcreteTransform -> transformBackward ( inputBlock, outputBlock );
        ... // do something with inputBlock and outputBlock _after_ the call
    }
    void getStatistics ( ... )
    {
        ... // return the gathered statistics
    }
private:
    TransformInterface* myConcreteTransform;
    ... // additional private fields for statistics gathering
};


问:
DDD - Domain Object calling a web service
Is it ok to call a webservice from a Domain object?.

As I write the question I am thinking that you should never do that, as it is poor design, but the situation is the following:

I have a domain object called Postman that works very closely with a Message object. This Message is provided by a web service. Without the Message, the Postman object could not do its business logic.

I understand that this code doesn't smell good but, since the Postman depends on the Message, it seems logical to call the web service in order to get the Message.

There is no other way to get the Message without the web service.

答: Just abstract the web service into a domain service and encapsulate it inside an implementation. That is why services in DDD exist. To abstract and encapsulate the outside world.</p>


问:
Can I use select2 in my website?
I am working on project that deals in customer support. This website actually sells products ( so it is commerical ), and provides customer support.  So in this project/website I was about to use select2 .

Select2 License:

This software is licensed under the Apache License, Version 2.0 (the
  "Apache License") or the GNU General Public License version 2 (the
  "GPL License"). You may choose either license to govern your use of
  this software only upon the condition that you accept all of the terms
  of either the Apache License or the GPL License.

Can I use this in the website?

Ask me if any other info is also required.

答:
Apache license is a non-copyleft open-source license that does not require anything from you. You can use code covered by Apache version 2 license for anything you want.

The GPL alternative is there to permit combining the work with code covered by GPL version 2, because Apache version 2 license by itself is only compatible with GPL version 3.

Note that for server side you could even use GPL code in a website, because the requirements of GPL pertain to distributing the code and you are not distributing that code. You are distributing the JavaScript code though, so you need to rely on the Apache license option in this particular case.

Title: Can I use select2 in my website?


答:
I once had to solve the exact same problem, as part of commercial image processing software. In the end I opted for a scan-line implementation, because it reads the matrix only once.

Basically, you consider a single row of the matrix and keep track of all 'events' on this scan line. Events are left and right edges of islands. Note that the same island might have multiple left and right edges on the same scan line. Also note that what seems to be 2 seperate islands on your scan line, might turn out to be the same island later.

You also keep a record of each known island with its highest value.

You start with the top row, initialize the events. And then go down one row at a time, each time revealing one more row of the matrix and updating the events. Next to updating the position of the existing events, you need consider also the following topology-changes:

Start (top) of new land: a new island record, and new left and right edges are inserted in the scan line.

End (bottom) of a piece of land: a left and right event are removed from the scan line.

Top of a lake/sea: An island that splits into 2 seperate peninsulae of the same island (2 new events)

Bottom of a lake/sea: two peninsulae merge into one. If they didn't already belong to the same island records, then these 2 island records need to be merged.

Title: Find the peak of each islands in sparse matrix

Body:
I have a sparse matrix that contains several islands of unknown size. I would like to find the highest peak of each islands. Consider this matrix as an example:

```<code>0 0 1 0 0 0 0
0 0 1 2 1 0 0
0 0 0 3 2 1 0
0 1 0 0 0 0 0
0 2 3 4 0 1 0
0 0 1 1 0 1 0
0 0 0 0 0 1 0
</code>```

In this case I would like to get the position of 3 at (3, 2), the position of 4 at  (3, 4) and the position of 1 at (5, 4), (5, 5) or (5, 6). Note: I consider 4-connected neighbours.

Until now I came up with the solution that scans each pixel and if it is not zero it starts a flood fill from that pixel. The flood fill paints the pixels to zero keeping track the maximum value visited. This algorithm works well but I was wondering if there are any faster solution?

In my target platform (iOS) I have access to different vector processing frameworks like BLAS, LAPACK, vDSP that provide some functions for fast finding the maximum value in a vector (or matrix). Maybe one could implement a faster solution with the help of these functions?


问:
Partitioning set into subsets with respect to equality of sum among subsets
Let's say I have {3, 1, 1, 2, 2, 1, 5, 2, 7} set of numbers, I need to split the numbers such that sum of subset1 should be equal to sum of subset2 {3,2,7} {1,1,2,1,5,2}. First we should identify whether we can split number(one way might be dividable by 2 without any remainder) and if we can, we should write our algorithm two create s1 and s2 out of s.

How to proceed with this approach? I read partition problem in wiki and even in some articles but I am not able to get anything. Can someone help me to find the right algorithm and its explanation in simple English?

答:
```<code>using System;
using System.Linq;
public class CandidateCode
{
    public static string partition(int[] input1)
    {
        bool[] best_assignment = PartitionValues(input1);

        string result1 = "", result2 = "";
        int total1 = 0, total2 = 0;
        for (int i = 0; i < best_assignment.Length; i++)
        {
            if (best_assignment[i])
            {
                result1 += "\r\n " + input1[i];
                total1 += input1[i];
            }
            else
            {
                result2 += "\r\n " + input1[i];
                total2 += input1[i];
            }
        }
        if (result1.Length > 0) result1 = result1.Substring(2);
        if (result2.Length > 0) result2 = result2.Substring(2);

        return "{"+ result1 + " } {" + result2 + " } total  " + total1.ToString() + " & " + total2.ToString();
    }

    private static bool[] PartitionValues(int[] values)
    {
        bool[] best_assignment = new bool[values.Length];
        bool[] test_assignment = new bool[values.Length];

        int total_value = values.Sum();

        int best_err = total_value;
        PartitionValuesFromIndex(values, 0, total_value, test_assignment, 0, ref best_assignment, ref best_err);

        return best_assignment;
    }

    private static void PartitionValuesFromIndex(int[] values, int start_index, int total_value,
            bool[] test_assignment, int test_value,
            ref bool[] best_assignment, ref int best_err)
    {
        // If start_index is beyond the end of the array,
        // then all entries have been assigned.
        if (start_index >= values.Length)
        {
            // We're done. See if this assignment is better than what we have so far.
            int test_err = Math.Abs(2 * test_value - total_value);
            if (test_err < best_err)
            {
                // This is an improvement. Save it.
                best_err = test_err;
                best_assignment = (bool[])test_assignment.Clone();
            }
        }
        else
        {
            // Try adding values[start_index] to set 1.
            test_assignment[start_index] = true;
            PartitionValuesFromIndex(values, start_index + 1, total_value,
                test_assignment, test_value + values[start_index],
                ref best_assignment, ref best_err);

            // Try adding values[start_index] to set 2.
            test_assignment[start_index] = false;
            PartitionValuesFromIndex(values, start_index + 1, total_value,
                test_assignment, test_value,
                ref best_assignment, ref best_err);
        }
    }
}
</code>```


问:
Is RESTful API appropriate for services which are nearly computational in nature?
Is RESTful API appropriate for the following scenario?

A time-limited token is obtained, which will last for one minute only

Image is uploaded to a server

Server performs some CPU-heavy image processing upon request, within the time limit

The result is available for download, within the time limit

When the token expires:

The image, computation, result are all discarded / terminated, and any open connection will be closed.

All other necessary aspects of web services, such as authentication / service authorization, quota, encryption etc., might still be necessary.

Since the "resource" is only allowed to exist for less than one minute, does it still make sense to make "resource" the first-class citizen in this API?

Update

I have found this link that talks about a REST API for job queueing in RabbitMQ. However it is an "idea page", so I'm not sure whether this is relevant or not.

答:
It seems like your reading too much into this.

RESTful API is just a more lightweight stripped down version of SOAP, most commonly it will use JSON (not required) as its generally used for smartphones because its much more lightweight for an environment where a user may have a bad connection or be charged for using it.

SOAP web services have built in benefits for security, encryption etc. if these are going to be required it sounds like you should use SOAP instead of trying to implement your own on top of REST


答:
I suppose there's no harm to using restful if you're only using get with the id of the download in order to start download of a file.  Just a couple important points to observe:

I would just be careful to make every other standard restful accessor disabled if you're not going to use them (what sense does update and delete have at that point?).  You could still use post to upload the initial file, since that still fits the pattern.

Change the root to distinguish from other more normal restful accesses, even if the file being uploaded involves updating a record which is normally accessed through restful (i.e., if you have /api/files as a root for crud operations on records regarding files, have an /api/files/upload root for the actual file upload).

Restful is meant primarily to simplify server/client interface, so if you run into some sort of trouble, don't wrack your brain trying to make it work.  Simply abandon restful for this type of transaction.

Title: Is RESTful API appropriate for services which are nearly computational in nature?

Body:
Is RESTful API appropriate for the following scenario?

A time-limited token is obtained, which will last for one minute only

Image is uploaded to a server

Server performs some CPU-heavy image processing upon request, within the time limit

The result is available for download, within the time limit

When the token expires:

The image, computation, result are all discarded / terminated, and any open connection will be closed.

All other necessary aspects of web services, such as authentication / service authorization, quota, encryption etc., might still be necessary.

Since the "resource" is only allowed to exist for less than one minute, does it still make sense to make "resource" the first-class citizen in this API?

Update

I have found this link that talks about a REST API for job queueing in RabbitMQ. However it is an "idea page", so I'm not sure whether this is relevant or not.


答:
What would you do if the message was in a database? Do the exact same thing. A web service is just another storage device, in essence.

Your calling code should know nothing of the repository implementation details, if at all possible. Database, text file, web service, it's all GetMessage().

Title: DDD - Domain Object calling a web service

Body:
Is it ok to call a webservice from a Domain object?.

As I write the question I am thinking that you should never do that, as it is poor design, but the situation is the following:

I have a domain object called Postman that works very closely with a Message object. This Message is provided by a web service. Without the Message, the Postman object could not do its business logic.

I understand that this code doesn't smell good but, since the Postman depends on the Message, it seems logical to call the web service in order to get the Message.

There is no other way to get the Message without the web service.


问:
Which design pattern would be best for this case?
I have a class, called PolicyProvider, at present with the following (abridged) interface:

```<code>public interface IPolicyProvider
{
    List<Policy> GetRenewalPolicies(Client client, int financialYear);
}
</code>```

The purpose is quite obvious - every year, the insurance renews for the client, and they will get a number of new policies.

The thing is, the policies change year by year.  Therefore, a concrete implementation of the class needs to be changed every year (or amended with lots of ugly little "if's" evaluating the financial year).  I can see that over time this will become horrendous.

In this case, what design pattern is most appropriate?

答: A standard approach to avoiding "if"s is to use subtyping and dynamic binding. All you need to do is to create subtype policies for each financial year.</p>


问:
How do you unit test private methods?
I am working on a java project. I am new to  unit testing. What is the best way to unit test private methods in java classes?</p>

答:
What I normally do in C# is make my methods protected and not private.
It's a slightly less private access modifier, but it hides the method from all classes that do not inherit from the class under test.

```<code>public class classUnderTest
{
   //this would normally be private
   protected void methodToTest()
   {
     //some code here
   }
}
</code>```

Any class that doesn't inherit directly from classUnderTest has no idea that methodToTest even exists. In my test code, I can create a special testing class that extends and provides access to this method...

```<code>class TestingClass : classUnderTest
{
   public void methodToTest()
   {
     //this returns the method i would like to test
     return base.methodToTest();
   }
}
</code>```

This class only exists in my testing project.
Its sole purpose is to provide access to this single method.
It allows me to access places that most other classes do not have.

Title: How do you unit test private methods?

Body:
What I normally do in C# is make my methods protected and not private.
It's a slightly less private access modifier, but it hides the method from all classes that do not inherit from the class under test.

public class classUnderTest
{
   //this would normally be private
   protected void methodToTest()
   {
     //some code here
   }
}

Any class that doesn't inherit directly from classUnderTest has no idea that methodToTest even exists. In my test code, I can create a special testing class that extends and provides access to this method...

class TestingClass : classUnderTest
{
   public void methodToTest()
   {
     //this returns the method i would like to test
     return base.methodToTest();
   }
}

This class only exists in my testing project.
Its sole purpose is to provide access to this single method.
It allows me to access places that most other classes do not have.


问:
A backlog of "bite-size" tasks in parallel to the "main" feature backlog?
After over two years of working in a highly siloed, "lone-wolf" development department structure, we're adopting Agile SCRUM. Great. I like Agile; as a dev it keeps you focused, busy, and productive without having myriad stakeholders shoving project after project down your throat with the expectation they all be done yesterday.

There is, however, one aspect of moving to SCRUM versus our current "model", that I think people outside Development are not going to like in the slightest. That is their current ability to have us do small changes "while you wait". A large portion of our development is for in-house consumption only, and we're mostly all in the same building. So, it's been common practice for years for a department lead or manager elsewhere to come to the "codebase owner" of a particular application and ask for small stuff (sometimes not so small, but we're pretty good about not taking on three-week projects based on these "drive-bys"). Even our boss sometimes relays things brought up to him in this way. Very often, if we're working in the codebase in question at the time, we can simply pop up the source file, make the change, and run it with them looking over our shoulder to verify the change is what they want, before checking it into Subversion for inclusion in the next build.

With a basic Agile SCRUM methodology, these tweaks would either be logged as defects (we didn't meet a requirement specified in a story previously consumed) or new small stories (we met all stated requirements, but those requirements were incomplete, vague or incorrect, or they changed after delivery once the users saw the new features). Either way, the vast majority would be one-pointers at most if not zeroes, and of relatively low priority (the system is usable in its current state, but it would be so much cooler if...), making them unlikely to be brought into a sprint when working the backlog top-down.

This possibility was raised at a dev meeting as being a source of active opposition to our Agile process by other departments, who would see it as less "agile" than our current ability to make small tweaks on request. It's a valid concern IMO; the stakeholders behind the PO don't always agree on what things are most important, because they don't all have the same point of view, yet it's typically only the managers who make the final decision, and therefore their bias is the one that shows in the product backlog.

A solution was then proposed, which was tentatively called the "candy jar" (another term thrown out was the "gravy boat"). Small tweaks requested by the "little guys" in the various departments, that are not defects in an existing story, that are estimated by consensus or acclamation within the team to take less than one-half of a developer-day, and that would have an immediate, significant, positive impact on the user experience in the opinion of the end user, are put on a list in parallel to the primary backlog. They'd be identified as "stories", but would be kept separate from the primary backlog of "big" stories subject to prioritization. If, at any time during the normal progress of a sprint, we happen to be working in an area of the system in which one of these tweaks can be made, making the tweak trivial, we can bring the tweak into the sprint and code it alongside the larger story. Doing this must not jeopardize the completion of the larger story or any other committed work. The PO would also have access to this list, and if they were working on an upcoming user story touching the basic feature involving the tweak, they could fold it into the story as a requirement and then we'd meet the requirement as we would any other. This, it was thought, would make tweaks more likely to be worked on sooner than later.

This triggered the reaction among those of us with ScrumMaster training of "uh-uh". There is one backlog. Two backlogs introduces the question of which #1 item is really the most important, which list's items determine real velocity, and which of the two backlogs a story actually belongs in (any demarcation of size/complexity is going to have some cases that fall relatively arbitrarily to one side or the other). "Let the process work", we said; if the change really is significant to the end users, they'll make enough noise to get the department heads making time/money decisions on board, and it'll get bumped up into the dev team's consciousness toward the top of the backlog.

I thought I'd pose the question to the floor: In your opinion, would a parallel list of "bite-size" stories have value in getting small, useful but ultimately low-priority changes made faster, or is it overall a better decision to fold them into the main backlog and let the basic process govern their inclusion in a sprint?

答: In my opinion; your problem is the way that tasks are prioritised in the backlog. For example, "priority" could take into account both importance and how quickly it can be completed. Something that isn't as important but can be completed in 10 minutes can have a higher priority than something more important that will take several weeks to implement.</p>


问:
Building a WordPress-like filter system
This question is purely hypothetical. I use WordPress a lot and know the filter structure from an implementation point of view. I'm now wondering what's the best way to implement such a structure (not the way WordPress uses, but the best way).

I will give a short overview of what the WordPress filter structure is. After that, I'll list my requirements and thoughts. At last, I'll ask some questions.

WordPress' filter structure

WordPress sends nearly every data through a filter function before it outputs the data to the browser. The WordPress Codex gives information in the Plugin API and the Filter Reference.

For a plugin developer, it's quite easy to add a filter:

Create the PHP function that filters the data.

Hook to the filter in WordPress, by calling add_filter()

Put your PHP function in a plugin file, and activate it.

Let's look at these steps in some more detail:

A filter function is a PHP function that takes one parameter, the input, and returns the output. It should not echo or print anything.

The add_filter function (reference) looks like:

```<code>add_filter ( 'hook_name', 'your_filter', [priority], [accepted_args] );
</code>```

hook_name is the action hook at which the filter should be called.

your_filter is the name of your filter function.

priority gives the, well, the priority of the filter (default: 10), where lower numbers are more important.

accepted_arguments tells WordPress the filter function will take more parameters, but let's leave that out here, it's not one of my requirements.

I don't have to explain this step, I think.

Requirements

I'd like to know how to implement a filter structure like WordPress has and for the sake of an actual answerable question, I have made up a hypothetical case with some requirements.

My case is a forum, in which I have several cases in which I'd want to use filters for the content, like:

A linkify filter to make bare URLs into nice, working hyperlinks

A smiley filter to replace ASCII smileys like ':D' into an image  (sorry for the imgur abuse)

I think the advantages of using filters for this would be:

Code clarity

Filters can be easily reused

Making it easy to implement that a user can switch a filter on and off

Easy implementation of additional filters in the future

My thoughts

I imagine WordPress keeps a multidimensional array like $filters[$hook][$filter]. Here is $hook the name of the hook when the filter has to be called and $filter an array of the filter settings, basically those that were passed to the add_filter() function. When an action hook point is reached, WordPress can iterate through the $filters[$current_hook] array and execute every filter.

Questions

Is this the best way to include a filter system with the requirements I listed? If so, why? If not, why not; and what would be a better system and why?

As I said I want users to be able to switch filters on and off.

I thought of adding a enabled_filters column to the user table in the database, which would be a bitmask of the enabled filters. That would mean every filter would have a unique identifier, but more important that there aren't very much filters possible. So this wouldn't be the way to go.

I also thought of adding a table enabled_filters with columns userId, filterName and enabled, to set the filters on and off with a new row. With using the filter name in every row, this would cost some data space on much users and filters, so a better but similar idea would be adding a table filters with id and filterName, and changing the filterName column of the enabled_filters table into a filterId column. This would also allow an additional field in the filters table, allow_disable, to disable the disabling of the filter.

Is this second approach a good one, or are there better options? Key requirement is that I don't want to have to modify the base system when adding a filter.

WordPress wants programmers to not print anything in a filter function, but return a new string. That means you'll have a variable $return in your filter function, to which you append new data all the time with $return .= '...';. Using echo or print() could be easier for programmers, also to make it easier to port existing code (which uses print functions) to a filter function. The filter system could use the ob_* functions to capture the printed data instead of sending it to the browser. Would this be a good way to implement filters? Are there any disadvantages, like speed?

The last question is about when to use filters, and when not. It seems clear to me that the listed cases (linkify and smileys) are cases where filters are well-used. For things like signatures or avatars, to stick with the forum, it's different. I've tried to figure out why it feels different, and think it's because that would limit the use of a filter to one place.

For example, avatars and profile overview filters could only be used next to a post. One of the nice things of a filter is that you can add the same filter function to a different action hook, so that you can pass both the post and the signature content through the linkify filter. Am I right here?

Is it true that one shouldn't use filters for avatars, profile overviews and signatures? What can be said about general rules when and when not to use a filter? If I were to write documentation on my system for third-party developers, what should I write down on this topic?

答:
Implementation

Is this the best way to include a filter system with the requirements I listed?
  If so, why? If not, why not; and what would be a better system and why?

WordPress is a grown system with the goal to be backward compatible all the time. Thus, it still uses techniques to mimic things, that can be done more straight with modern PHP. The hook system mimics the Observer pattern. WordPress' hooks are events in this context.

Storage

I want users to be able to switch filters on and off.

Your second approach is reasonable with slightly different semantics. You need a table for the filters and a map for the user's setting. You can even add per-filter configurations (serialized).

```<code>+-----------+
| Filters   |
+-----------+
| +id       | // unique filter id
| +name     | // unique name (ie., [part of] filename)
| +title    | // human readable name
| +enabled  | // whether or not the filter is available to users
| +settings | // default settings managed by admin
+-----------+

+---------------+
| FilterUserMap |
+---------------+
| +user_id      | // the user
| +filter_id    | // the filter
| +enabled      | // whether or not the user has enabled the filter
| +settings     | // the user's preferrences
+---------------+
</code>```

Output

The filter system could use the ob_* functions to capture the printed data
  instead of sending it to the browser. Would this be a good way to implement
  filters? Are there any disadvantages, like speed?

Performance should never be your first concern. Most important is the readability and maintainability of the code. Using the output buffering is a good approach, if you add larger chunks of HTML, because that increases readability very much. Usually a filter will change the content instead of add something, so you mostly end up using preg_* functions.

Nevertheless, I guess, output buffering will be faster than string concatenation, since it is done on a level closer to the machine, but I don't have any figures.

Usage

Is it true that one shouldn't use filters for avatars, profile overviews and
  signatures? What can be said about general rules when and when not to use a
  filter? If I were to write documentation on my system for third-party
  developers, what should I write down on this topic?

No, I don't think so. It is normal that a filter does not fit into all situations. Think of html_specialchars, strip_tags, or mysqli_real_escape_string as some of PHP's native filter functions - they have special use cases and are not suitable everywhere.

You could define a FilterInterface extending PHP's SplObserver interface, and use it to implement special filter classes, so can control, where what kind of filter is usable.

Title: Building a WordPress-like filter system

Body:
Implementation

Is this the best way to include a filter system with the requirements I listed?
  If so, why? If not, why not; and what would be a better system and why?

WordPress is a grown system with the goal to be backward compatible all the time. Thus, it still uses techniques to mimic things, that can be done more straight with modern PHP. The hook system mimics the Observer pattern. WordPress' hooks are events in this context.

Storage

I want users to be able to switch filters on and off.

Your second approach is reasonable with slightly different semantics. You need a table for the filters and a map for the user's setting. You can even add per-filter configurations (serialized).

+-----------+
| Filters   |
+-----------+
| +id       | // unique filter id
| +name     | // unique name (ie., [part of] filename)
| +title    | // human readable name
| +enabled  | // whether or not the filter is available to users
| +settings | // default settings managed by admin
+-----------+

+---------------+
| FilterUserMap |
+---------------+
| +user_id      | // the user
| +filter_id    | // the filter
| +enabled      | // whether or not the user has enabled the filter
| +settings     | // the user's preferrences
+---------------+

Output

The filter system could use the ob_* functions to capture the printed data
  instead of sending it to the browser. Would this be a good way to implement
  filters? Are there any disadvantages, like speed?

Performance should never be your first concern. Most important is the readability and maintainability of the code. Using the output buffering is a good approach, if you add larger chunks of HTML, because that increases readability very much. Usually a filter will change the content instead of add something, so you mostly end up using preg_* functions.

Nevertheless, I guess, output buffering will be faster than string concatenation, since it is done on a level closer to the machine, but I don't have any figures.

Usage

Is it true that one shouldn't use filters for avatars, profile overviews and
  signatures? What can be said about general rules when and when not to use a
  filter? If I were to write documentation on my system for third-party
  developers, what should I write down on this topic?

No, I don't think so. It is normal that a filter does not fit into all situations. Think of html_specialchars, strip_tags, or mysqli_real_escape_string as some of PHP's native filter functions - they have special use cases and are not suitable everywhere.

You could define a FilterInterface extending PHP's SplObserver interface, and use it to implement special filter classes, so can control, where what kind of filter is usable.


答:
I will talk about a few points which, hopefully, will help you find your way:

"SCRUM" is about being agile. Common sense is required. If the change is a few minutes change, I don't think you need a backlog for it. If it's more than 2 hours, I think you should give it a second thought. Not everything that is an "easy-win" should be done. In SCRUM you work by priorities. I think that the PO must get the information about what you gain from the addition and the effort it takes. Only then can the PO decide whether or not it's important. Moving to SCRUM, sometimes comes with a lot of questions and developers will often say "but, that will only takeד a few hours". So what? Few hours is time, not everything that is short needs to be included.

I once worked at a project where we had "engineering backlog". This backlog contained  items suggested by the developers for improving the product. Those items did not have an explicit user value (but did have an implicit user value). For example: refactoring a component in the code. We had described the change, the effort and the gain (in this case, you can't present the user anything. But if such refactoring causes to develop new capabilities more quickly then it's definitely a gain for the user). Our PO decided that during the version we should invest 10% of each sprint (in average) to  such items. Before each sprint, when the PO decided about the upcoming sprint's backlog, he made sure that we had 10% of enginerring backlog items. So 2 version backlog -> 1 sprint backlog.

Buffers - When starting to work in SCRUM people often forget that, as software engineers, we leave in a world of uncertainty. That's ok to count 1 day of work as 6 hours instead of 8. Let's say you have a 15 days sprint, that means you have extra 30 hours which go to meetings, to things that took too long, and yes - also for those little things that are too minor to remember but are a part of our day-2-day job.

Stay focused - Last but not least, in SCRUM it's important to stay focused. Decide how much of your total effort, and what's the priority, to invest in such tasks and remember to invest this time & effort. Don't drift to work on "little things" just because they're little. You have a PO to help you decide and you have your common sense.

Stay Agile - And, at the end, don't forget to give a try to different methods to approach the issue, question yourself if that's indeed the best way. Improve on the way.

Good luck :)

Title: A backlog of "bite-size" tasks in parallel to the "main" feature backlog?

Body:
After over two years of working in a highly siloed, "lone-wolf" development department structure, we're adopting Agile SCRUM. Great. I like Agile; as a dev it keeps you focused, busy, and productive without having myriad stakeholders shoving project after project down your throat with the expectation they all be done yesterday.

There is, however, one aspect of moving to SCRUM versus our current "model", that I think people outside Development are not going to like in the slightest. That is their current ability to have us do small changes "while you wait". A large portion of our development is for in-house consumption only, and we're mostly all in the same building. So, it's been common practice for years for a department lead or manager elsewhere to come to the "codebase owner" of a particular application and ask for small stuff (sometimes not so small, but we're pretty good about not taking on three-week projects based on these "drive-bys"). Even our boss sometimes relays things brought up to him in this way. Very often, if we're working in the codebase in question at the time, we can simply pop up the source file, make the change, and run it with them looking over our shoulder to verify the change is what they want, before checking it into Subversion for inclusion in the next build.

With a basic Agile SCRUM methodology, these tweaks would either be logged as defects (we didn't meet a requirement specified in a story previously consumed) or new small stories (we met all stated requirements, but those requirements were incomplete, vague or incorrect, or they changed after delivery once the users saw the new features). Either way, the vast majority would be one-pointers at most if not zeroes, and of relatively low priority (the system is usable in its current state, but it would be so much cooler if...), making them unlikely to be brought into a sprint when working the backlog top-down.

This possibility was raised at a dev meeting as being a source of active opposition to our Agile process by other departments, who would see it as less "agile" than our current ability to make small tweaks on request. It's a valid concern IMO; the stakeholders behind the PO don't always agree on what things are most important, because they don't all have the same point of view, yet it's typically only the managers who make the final decision, and therefore their bias is the one that shows in the product backlog.

A solution was then proposed, which was tentatively called the "candy jar" (another term thrown out was the "gravy boat"). Small tweaks requested by the "little guys" in the various departments, that are not defects in an existing story, that are estimated by consensus or acclamation within the team to take less than one-half of a developer-day, and that would have an immediate, significant, positive impact on the user experience in the opinion of the end user, are put on a list in parallel to the primary backlog. They'd be identified as "stories", but would be kept separate from the primary backlog of "big" stories subject to prioritization. If, at any time during the normal progress of a sprint, we happen to be working in an area of the system in which one of these tweaks can be made, making the tweak trivial, we can bring the tweak into the sprint and code it alongside the larger story. Doing this must not jeopardize the completion of the larger story or any other committed work. The PO would also have access to this list, and if they were working on an upcoming user story touching the basic feature involving the tweak, they could fold it into the story as a requirement and then we'd meet the requirement as we would any other. This, it was thought, would make tweaks more likely to be worked on sooner than later.

This triggered the reaction among those of us with ScrumMaster training of "uh-uh". There is one backlog. Two backlogs introduces the question of which #1 item is really the most important, which list's items determine real velocity, and which of the two backlogs a story actually belongs in (any demarcation of size/complexity is going to have some cases that fall relatively arbitrarily to one side or the other). "Let the process work", we said; if the change really is significant to the end users, they'll make enough noise to get the department heads making time/money decisions on board, and it'll get bumped up into the dev team's consciousness toward the top of the backlog.

I thought I'd pose the question to the floor: In your opinion, would a parallel list of "bite-size" stories have value in getting small, useful but ultimately low-priority changes made faster, or is it overall a better decision to fold them into the main backlog and let the basic process govern their inclusion in a sprint?


答:
The Observer pattern might be a good fit here. The Transform class defines a set of events that might be of interest to the statistics engine, and the statistics engine registers itself with the relevant Transform instance to gather the statistics on that transformation (or the statistics that include that transformation).

Update

As stated in a comment, the basic problem is how does the statistics engine know that something of interest has happened.
You could execute the codec in a virtual machine to keep track of everything (valgrind uses this approach to check memory access), but then you have the problem of deciding what it means that the codec accessed address 0x12345678.

All other methods of statistics gathering invade into the code-base of the codec in one way or another. The least invasive is probably to add copious amounts of logging and to let the statistics engine analyse that. All logging packages also provide means to disable the generation of the logs with minimal cost, sometimes even compiling those statements to no-ops.

Title: Statistics collection engine for C++ systems

Body:
We have a research project with idea->prototype->statistics development cycle.
Anyway, our final product is a prototype, so the statistics collection suite is
not used persistently. Supposing I have following class:

```<code>class Transform {
    private:
        someData;

    public:
        transformForward (inp_block, outp_block);
        transformBackward (inp_block, outp_block);
};
</code>```

Imagine that it is a part of a big system. I need to periodically collect some statistics on such transform (internal data could be considered as well). It seems that adding statistical routines would be a violation of single-responsibility principle. The second smell is that I do not want to touch code that uses Transform instances to explicitly invoke these routines. I would be best if I could just
trigger some kind of switch so that the statistics for that module will be collected.

I've met that challenge a number of times and I have a feeling that I'm constantly reinventing the wheel. Is there some good practices for configuring and collecting the statistics suite for a compound system without interfering into it's internal code base?

UPDATE:

As I can see from the answers proposed, my question is too non-specific, so I'll provide more concrete example.

Consider an image-compression system composed by two huge blocks: Predictor and Encoder. There are a lot of various prediction and compression algorithms, during our research we need to explore the behavior of the components under various conditions. We should answer questions like "how many times the pixel is processed within each context", "how well does this predictor works", "how does each predictor affects the Encoder's internal state" and many others.

Anyway, our final product is just a Codec with no statistical suite shipped with it; all kind of statistics collection is used internally during our research. Thus the question arises: how could one build flexible statistics engine that knows the very internals of the system? How could one keep the system itself independent of the statistics engine?


问:
Programming task organization
My friend and I decided to create a large programming project together. Before that, I have never worked in a pair (I had quite many projects on my own and at work the boss defined the task for me). But now we have an idea of what kind of app we are going to do, how to plan it...but how can we divide the responsibility for the code in the project so that we don't do each other's work and it is not so closely related that we have to overlap each other's code?

How would you divide the tasks, what is the original way to do that? How would you divide the tasks in modules if they represent one application and are very closely related?

答:
First of all, you need some sort of source code control. My company uses Team Foundation Server, but there is some free software out there (don't know which one is the best, so I can't recommend). Then, you need to have some sort of shared task list, so you can keep track of what's to be done and who's done what. There are some sites for that also, but I haven't tried them out.
Having said that, the organization of modules is entirely up to you. I'd like to stress out that you really must plan the project, that is, to write some sort of technical specification, or at least have user stories. If there are some common methods (for example, converting a DateTime into a string), you have to know who's writing it, and who's editing it...

Really, there is not golden rule to apply, especially if you are not implementing through a methodology (scrum, kanban). Just apply common sense.

Title: Programming task organization


问:
How is referential transparency enforced?
In FP languages, calling a function with the same parameters over and over again returns the same result over and over again (i.e. referential transparency).

But a function like this (pseudo-code):

```<code>function f(a, b) {
    return a + b + currentDateTime.seconds;
}
</code>```

is not going to return the same result for the same parameters.

How are these cases handled in FP?

How is referential transparency enforced? Or is it not and it depends on the programmers to behave themselves?

答: a and b are Numbers, whereas currentDateTime.seconds returns an IO<Number>. Those types are incompatible, you cannot add them together, therefore your function is not well-typed and simply won't compile. At least that's how it's done in pure languages with a static type system, like Haskell. In impure languages like ML, Scala or F#, it is up to the programmer to ensure referential transparency, and of course in dynamically typed languages like Clojure or Scheme, there is no static type system to enforce referential transparency.


答:
I will try to illustrate Haskell's approach (I am not sure my intuition is 100% correct since I am not a Haskell expert, corrections are welcome).

Your code can be written in Haskell as follows:

```<code>import System.CPUTime

f :: Integer -> Integer -> IO Integer
f a b = do
          t <- getCPUTime
          return (a + b + (div t 1000000000000))
</code>```

So, where is referential transparency?
f is a function that, given two integers a and b, will create an action, as you can tell by the return type IO Integer.
This action will always be the same, given the two integers, so the function mapping a pair of integers to IO actions is referentially transparent.

When this action is executed, the integer value it produces will depend on the current CPU time: executing actions is NOT function application.

Summarizing: In Haskell you can use pure functions to construct and combine complex actions (sequencing, composing actions, and so on) in a referentially transparent way. Again, note that in the above example the pure function f does not return an integer: it returns an action.

EDIT

Some more details regarding JohnDoDo question.

What does it means that "executing actions is NOT function application"?

Given sets T1, T2, Tn, T, a function f is a mapping (relation) that associates to each tuple in T1 x T2 x ... x Tn one value in T.
So function application produces an output value given some input values.
Using this mechanism you can construct expressions that evaluate to values e.g. the value 10 is the result of evaluating the expression 4 + 6. Note that, when mapping values to values in this way, you are not performing any kind of input / output.

In Haskell, actions are values of special types which can be constructed by evaluating expressions containing appropriate pure functions that work with actions. In this way, a Haskell program is a composite action that is obtained by evaluating the main function. This main action has type IO ().

Once this composite action has been defined, another mechanism (not function application) is used to invoke / execute the action (see e.g. here). The whole program execution is the result of invoking the main action which can in turn invoke sub-actions.
This invocation mechanism (whose internal details I do not know) takes care of performing all the needed IO calls, possibly accessing the terminal, the disk, the network, and so on.

Going back to the example.
The function f above does not return an integer and you cannot write a function that performs IO and returns an integer at the same time: you have to choose one of the two.

What you can do is embed the action returned by f 2 3 into a more complex action. For example, if you want to print the integer produced by that action, you can write:

```<code>main :: IO ()
main = do
          x <- f 2 3
          putStrLn (show x)
</code>```

The do notation indicates that the action returned by the main function is obtained by a sequential composition of two smaller actions, and the x <- notation indicates that the value produced in the first action must be passed to the second action.

In the second action

```<code>putStrLn (show x)
</code>```

the name x is bound to the integer produced by executing the action

```<code>f 2 3
</code>```

An important point is that the integer that is produced when the first action is invoked can only live inside IO actions: it can be passed from one IO action to the next but it cannot be extracted as a plain integer value.

Compare the main function above with this one:

```<code>main = do
      let y = 2 + 3
      putStrLn (show y)
</code>```

In this case there is only one action, namely putStrLn (show y), and y is bound to the result of applying the pure function +. We could also define
this main action as follows:

```<code>main = putStrLn "5"
</code>```

So, notice the different syntax

```<code>x <- f 2 3    -- Inject the value produced by an action into
              -- the following IO actions.
              -- The value may depend on when the action is
              -- actually executed. What happens when the action is
              -- executed is not known here: it may get user input,
              -- access the disk, the network, the system clock, etc.

let y = 2 + 3 -- Bind y to the result of applying the pure function `+`
              -- to the arguments 2 and 3.
              -- The value depends only on the arguments 2 and 3.
</code>```

Summary

In Haskell pure functions are used to build the actions that constitute a program.

Actions are values of a special type.

Since actions are constructed by applying pure functions, action construction is referentially transparent.

After an action has been constructed, it can be invoked using a separate mechanism.

Body:
In FP languages, calling a function with the same parameters over and over again returns the same result over and over again (i.e. referential transparency).

But a function like this (pseudo-code):

```<code>function f(a, b) {
    return a + b + currentDateTime.seconds;
}
</code>```

is not going to return the same result for the same parameters.

How are these cases handled in FP?

How is referential transparency enforced? Or is it not and it depends on the programmers to behave themselves?

Body:
I will try to illustrate Haskell's approach (I am not sure my intuition is 100% correct since I am not a Haskell expert, corrections are welcome).

Your code can be written in Haskell as follows:

import System.CPUTime

f :: Integer -> Integer -> IO Integer
f a b = do
          t <- getCPUTime
          return (a + b + (div t 1000000000000))

So, where is referential transparency?
f is a function that, given two integers a and b, will create an action, as you can tell by the return type IO Integer.
This action will always be the same, given the two integers, so the function mapping a pair of integers to IO actions is referentially transparent.

When this action is executed, the integer value it produces will depend on the current CPU time: executing actions is NOT function application.

Summarizing: In Haskell you can use pure functions to construct and combine complex actions (sequencing, composing actions, and so on) in a referentially transparent way. Again, note that in the above example the pure function f does not return an integer: it returns an action.

EDIT

Some more details regarding JohnDoDo question.

What does it means that "executing actions is NOT function application"?

Given sets T1, T2, Tn, T, a function f is a mapping (relation) that associates to each tuple in T1 x T2 x ... x Tn one value in T.
So function application produces an output value given some input values.
Using this mechanism you can construct expressions that evaluate to values e.g. the value 10 is the result of evaluating the expression 4 + 6. Note that, when mapping values to values in this way, you are not performing any kind of input / output.

In Haskell, actions are values of special types which can be constructed by evaluating expressions containing appropriate pure functions that work with actions. In this way, a Haskell program is a composite action that is obtained by evaluating the main function. This main action has type IO ().

Once this composite action has been defined, another mechanism (not function application) is used to invoke / execute the action (see e.g. here). The whole program execution is the result of invoking the main action which can in turn invoke sub-actions.
This invocation mechanism (whose internal details I do not know) takes care of performing all the needed IO calls, possibly accessing the terminal, the disk, the network, and so on.

Going back to the example.
The function f above does not return an integer and you cannot write a function that performs IO and returns an integer at the same time: you have to choose one of the two.

What you can do is embed the action returned by f 2 3 into a more complex action. For example, if you want to print the integer produced by that action, you can write:

main :: IO ()
main = do
          x <- f 2 3
          putStrLn (show x)

The do notation indicates that the action returned by the main function is obtained by a sequential composition of two smaller actions, and the x <- notation indicates that the value produced in the first action must be passed to the second action.

In the second action

putStrLn (show x)

the name x is bound to the integer produced by executing the action

f 2 3

An important point is that the integer that is produced when the first action is invoked can only live inside IO actions: it can be passed from one IO action to the next but it cannot be extracted as a plain integer value.

Compare the main function above with this one:

main = do
      let y = 2 + 3
      putStrLn (show y)

In this case there is only one action, namely putStrLn (show y), and y is bound to the result of applying the pure function +. We could also define
this main action as follows:

main = putStrLn "5"

So, notice the different syntax

x <- f 2 3    -- Inject the value produced by an action into
              -- the following IO actions.
              -- The value may depend on when the action is
              -- actually executed. What happens when the action is
              -- executed is not known here: it may get user input,
              -- access the disk, the network, the system clock, etc.

let y = 2 + 3 -- Bind y to the result of applying the pure function `+`
              -- to the arguments 2 and 3.
              -- The value depends only on the arguments 2 and 3.

Summary

In Haskell pure functions are used to build the actions that constitute a program.

Actions are values of a special type.

Since actions are constructed by applying pure functions, action construction is referentially transparent.

After an action has been constructed, it can be invoked using a separate mechanism.


问:
Where should I store the files for python modules I install?
If I download a module for Python (such as alglib) and installing is simply a matter of running

```<code>python setup.py install
</code>```

Where should I be keeping the files I download, or the files that get created after I run setup.py? Is there some neat folder in the python directory that people put these modules before running setup.py? Can I move the files around after running it? Can I delete them... or will that stop them working?

I couldn't find any recommendation here: http://docs.python.org/2/install/

Also I am running windows and installed python through python(x,y)

答:
Good question!

Python has this tool, called pip, which is python's package manager.  It can pull packages from the PyPi repository (think an App Store).

These will handle where packages get installed for you.  Almost every package I've found available on the Internet was also available on PyPi.

Along with this, there is another tool that is used in conjunction with pip: virtualenv.

This is similar to ruby's rvm/bundler (I believe, someone please fact check this part).

It allows you to create a "local" set of dependencies for your application. The install directory for packages - while this virtualenv is activate - would be C:\path_to_virtualenv\lib\python\site-packages\ .  Installing via python setup.py install should drop the package into this directory!

A sample workflow that I follow:

```<code>cd dev/my_new_app
virtualenv ../venvs/my_new_app
source ../venvs/my_new_app/bin/activate
pip install my_new_dependency
pip freeze > requirements.txt
~~~<coding>~~~
</code>```

pip freeze will generate a list of all of the packages that you installed (along with versions).  > says "put everything on the left into the right", which is a file.  Anyone else who wants to start working on your project simply does pip install -r requirements.txt to get all of your dependencies installed!

If for whatever reason that library you are trying to work with (or an equivilant) is not available in the PyPi repository, the default install directory should now be considered your "install directory" for the setup.py .

Here is an awesome answer by Colonel Panic on installing pip on Windows.

If you decide not to use virtualenv, pip on its own will install to the default python package directory, which is somewhere around C:\\site-packages\

Body:
Good question!

Python has this tool, called pip, which is python's package manager.  It can pull packages from the PyPi repository (think an App Store).

These will handle where packages get installed for you.  Almost every package I've found available on the Internet was also available on PyPi.

Along with this, there is another tool that is used in conjunction with pip: virtualenv.

This is similar to ruby's rvm/bundler (I believe, someone please fact check this part).

It allows you to create a "local" set of dependencies for your application. The install directory for packages - while this virtualenv is activate - would be C:\path_to_virtualenv\lib\python\site-packages\ .  Installing via python setup.py install should drop the package into this directory!

A sample workflow that I follow:

cd dev/my_new_app
virtualenv ../venvs/my_new_app
source ../venvs/my_new_app/bin/activate
pip install my_new_dependency
pip freeze > requirements.txt
~~~<coding>~~~

pip freeze will generate a list of all of the packages that you installed (along with versions).  > says "put everything on the left into the right", which is a file.  Anyone else who wants to start working on your project simply does pip install -r requirements.txt to get all of your dependencies installed!

If for whatever reason that library you are trying to work with (or an equivilant) is not available in the PyPi repository, the default install directory should now be considered your "install directory" for the setup.py .

Here is an awesome answer by Colonel Panic on installing pip on Windows.

If you decide not to use virtualenv, pip on its own will install to the default python package directory, which is somewhere around C:\\site-packages\


答:
First of all, use some kind of project management or bug tracking tool for your tasks. (e.g. Trello, Redmine etc.). That way you could see open tasks and the tasks you are working on. If you decide you need to add more people on the project, they can easily see all the tasks.

Dividing the tasks is the hard part. But some logical way is to separate the tasks to e.g. design of user interface, client / server side programming, database etc. So you can work on design / UI, while your friend builds database and back end; or you can build some things together and other modules separately. You can choose to build some part of the system because you know how to do it, or you can choose to build it because it's fun and you want to learn something new.

If that's your personal project you have all the freedom to do what you want and how you want it.

Title: Programming task organization

Body:
My friend and I decided to create a large programming project together. Before that, I have never worked in a pair (I had quite many projects on my own and at work the boss defined the task for me). But now we have an idea of what kind of app we are going to do, how to plan it...but how can we divide the responsibility for the code in the project so that we don't do each other's work and it is not so closely related that we have to overlap each other's code?

How would you divide the tasks, what is the original way to do that? How would you divide the tasks in modules if they represent one application and are very closely related?


答:
I've worked in agile for a while now. All I can say is this - there are situations where insisting on a methodology and all that it incorporates, is absolutely wrong. You have to be AGILE, and tweaking a methodology to specific situations is, IMO, a must.

Like Avi above said - "SCRUM" is about being agile. Common sense is required. If the change is a few minutes change, I don't think you need a backlog for it.

If the change takes time, it means it is not all that "harmless" and it should be well documented.


答:
Programming frameworks like Agile and SCRUM are designed to apply discipline and structure to development.  However, discipline and structure seem to be the antonyms of fun and creativity.  Typically, you need to work harder to establish and maintain discipline.  It is very difficult to find a balance between the these opposing concepts.  Therefore, frameworks tend to avoid the topic.

On my last project, we observed that the developers needed a little time each day to refresh or clear their heads, etc.  They were given budgeted time (.5 hours/day or 2.5 hr/wk) in which they could do anything, within reason (with the possibility of being applied to something at work).  To keep things disciplined, they were asked to document their activities so they could get credit for any accomplishments, etc.  Having a specific budget for "the candy jar" fit within our timelines and prevented things from getting out of hand.

Btw, we called ours "project coolness" and it wound up being the source of many good ideas and improvements at that company.


答:
I would advise against using bootstrap to start off.  It really depends on your needs if you want your site to look exactly like bootstrap then it is fine but once you need to change something that isn't configurable you can waste a lot of time.  Try not having an input focus color of blue for example.

My recommendation would be to use something like: http://necolas.github.io/normalize.css/ to deal with cross browser issues and then to build your styles in Sass/Compass or Less.  I prefer Sass because it has Compass which is a framework built from Sass that has lots of useful mixins and functions.  Compass also allows you to easily separate your files but still build them into one.  I have individual files for core tags, common layouts, a nav bar and footer bar and many others but they will all compile to one css file.  That file will have comments above each rule which shows while file and line they were generated from.

Title: What is the best way to construct your CSS documents?

Body:
Normally I start a project working on a site, I'd do the basic html skeleton and then start tweaking the styles using CSS. Slowly I'd just add one CSS rule after another with no "organized structure" or groupings in the CSS rules.

And at the end of the project I look at the CSS document and the abomination I have created. Often spending quite some time trying to search for the spot in the document where I added a certain style.

Is there some type of 'rule-of-thumb' that one can use in laying out and getting more organized in your CSS documents? And I'm not just talking about adding comments...


答:
In my experience, it's usually no more difficult than making a list of everything that needs to get done, then each team member picking something off the list.  If you both feel that the same task must be done first, then either work together or break it down further.  If you estimate a task to take longer than 20 hours or so, it's probably too big, so break it down.

As other answers have mentioned, you need source control and a central place to keep your task list.  I would recommend putting both in one place.  Services like Bitbucket or GitHub are excellent for that sort of thing.

Title: Programming task organization


答:
You should keep the small stories in the main backlog.

I face similar issues to what you are describing, although I am not using scrum. I see that you face challenges of prioritization and efficiency.

It sounds like under your "old way", anyone was implicitly empowered to make their task the current "top priority" if they visited a developer's office. This has some benefits. The requester feels like they are being responded to, and both requester and developer get a "quick win".

The downside is that frequently inserting these tasks tends to slow down or derail the top priority tasks that were agreed upon with the product owner. (Side note, often everyone underestimates the amount of time needed for these "quick" fixes.)

My experience is that trying to squeeze in a low priority task undermines the benefits of prioritization. As a developer, prioritization validates to me that I am working on what the product owner wants me to work on. The product owner should decide whether she wants to take on the extra work and risk (however slight) of folding in a "nice to have" request.

Often when I ask stakeholders to prioritize, I am asked "Can you just squeeze this in?". The implied wish is for me to magically complete the new task without impacting the current highest priority.

What often happens to me is that stakeholders request LargeImportantProject and SmallLessImportantTask. The dilemma is, should SmallLessImportantTask wait for LargeImportantProject to finish (or have a roadblock)? There are tradeoffs, and my experience has been that the product owner has to decide. (If the product owner does not decide, the development team has to guess.)

One goal of scrum (and project management in general) is to avoid roadblocks for the highest priority tasks. As you become more efficient, there is less room to squeeze in extra "nice to have" work.

The efficiency can be scary, but I have seen the benefits outweighing the cost in two important ways.

As you become more efficient, you increase your team's capacity to complete valuable work, which includes "nice to have" requests.

If a request truly is a "nice to have", it is probably perfectly legitimate for the request to wait until more important business priorities are addressed.


问:
Data structure for accessing units of measure
TL;DR - I'm trying to design an optimal data structure to define units within a unit of measure.

A Unit of measure is essentially a value (or quantity) associated with a unit.  SI Units have seven bases or dimensions.  Namely: length, mass, time, electric current, temperature, amount of substance (moles), and luminous intensity.

This would be straightforward enough, but there are a number of derived units as well as rates that we frequently use.  An example combined unit would be the Newton: kg * m / s^2 and an example rate would be tons / hr.

We have an application that relies heavily upon implied units.  We'll embed the units within the variable or column name.  But this creates problems when we need to specify a unit of measure with different units.  Yes, we can convert the values at input and display but this generates a lot of overhead code that we'd like to encapsulate within its own class.

There are a number of solutions out on codeplex and other collaborative environments.  The licensing for the projects is agreeable but the project itself usually ends up being too lightweight or too heavy.  We're chasing our own unicorn of "just right."

Ideally, I could define a new unit of measure using something like this:

UOM myUom1 = new UOM(10, volts);

  UOM myUom2 = new UOM(43.2, Newtons);

Of course, we use a mix of Imperial and SI units based upon our clients' needs.

We also need to keep this structure of units synchronized with a future database table so we can provide the same degree of consistency within our data too.

What's the best way of defining the units, derived units, and rates that we need to use to create our unit of measure class?  I could see using one or more enums, but that could be frustrating for other developers.  A single enum would be huge with 200+ entries whereas multiple enums could be confusing based upon SI vs Imperial units and additional breakdown based upon categorization of the unit itself.

Enum examples showing some of my concerns:

myUnits.Volt

  myUnits.Newton

  myUnits.meter

SIUnit.meter

  ImpUnit.foot
  DrvdUnit.Newton

  DrvdUnitSI.Newton

  DrvdUnitImp.FtLbs

Our set of units in use is pretty well defined and it's a finite space.  We do need the ability to expand and add new derived units or rates when we have client demand for them.  The project is in C# although I think the broader design aspects are applicable to multiple languages.

One of the libraries I looked at allows for free-form input of units via string.  Their UOM class then parsed the string and slotted things accordingly.  The challenge with this approach is that it forces the developer to think and remember what the correct string formats are.  And I run the risk of a runtime error / exception if we don't add additional checks within the code to validate the strings being passed in the constructor.

Another library essentially created too many classes that the developer would have to work with.  Along with an equivalent UOM it provided a DerivedUnit and RateUnit and so on.  Essentially, the code was overly complex for the problems we're solving.  That library would essentially allow any:any combinations (which is legitimate in the units world) but we're happy to scope our issue (simplify our code) by not allowing every possible combination.

Other libraries were ridiculously simple and hadn't even considered operator overloading for example.

In addition, I'm not as worried about attempts at incorrect conversions (for example: volts to meters).  Devs are the only ones who will access at this level at this point and we don't necessarily need to protect against those types of mistakes.

答:
My prototype code: http://ideone.com/x7hz7i

My design points:

Choice of UoM (Unit of Measure) as Property get/set ```
Length len = new Length();
len.Meters = 2.0;
Console.WriteLine(len.Feet);
```

Named constructor for choice of UoM ```
Length len = Length.FromMeters(2.0);
```

ToString support for UoM ```
Console.WriteLine(len.ToString("ft"));
Console.WriteLine(len.ToString("F15"));
Console.WriteLine(len.ToString("ftF15"));
```

Roundtrip conversion (negligible rounding loss permitted by double precision) ```
Length lenRT = Length.FromMeters(Length.FromFeet(Length.FromMeters(len.Meters).Feet).Meters);
```

Operator overloading (but lacking dimensional type check) ```
// Quite messy, buggy, unsafe, and might not be possible without using F# or C++ MPL.
// It goes on to say that <strong>Dimensional Analysis</strong> is <strong>not an optional feature</strong> for UoM -
// whether you use it directly or not. It is <strong>required</strong>.
```

Body:
My prototype code: http://ideone.com/x7hz7i

My design points:

Choice of UoM (Unit of Measure) as Property get/set

Length len = new Length();
len.Meters = 2.0;
Console.WriteLine(len.Feet);

Named constructor for choice of UoM

Length len = Length.FromMeters(2.0);

ToString support for UoM

Console.WriteLine(len.ToString("ft"));
Console.WriteLine(len.ToString("F15"));
Console.WriteLine(len.ToString("ftF15"));

Roundtrip conversion (negligible rounding loss permitted by double precision)

Length lenRT = Length.FromMeters(Length.FromFeet(Length.FromMeters(len.Meters).Feet).Meters);

Operator overloading (but lacking dimensional type check)

// Quite messy, buggy, unsafe, and might not be possible without using F# or C++ MPL.
// It goes on to say that Dimensional Analysis is not an optional feature for UoM -
// whether you use it directly or not. It is required.


问:
Implementing non-fixed length array support in a compiler
I'm thinking of building a language for PIC microcontrollers. I want to be able to use non-fixed size arrays, like this:

Declare the variable as int[]

Wait for input from serial connection

Make the variable input long

I think such a feature would be useful, but don't know how I should go about implementing this in a compiler which compiles to assembly. I want the arrays to be stored on consecutive register addresses, of course. Since I'm working with PICs, this has to be very memory efficient.

When someone writes int[], I don't think it's a good idea to reserve memory space for the variable yet, is it? Then the array would have a fixed maximum size. For example, when the array gets reserved memory address 5-100, and other variables get 4 and 101, the array has fixed borders and can't grow any bigger than 96 registers. Also, when I'd reserve memory from the start, let's say x bytes, and I in the end only need y bytes, I'm wasting x - y bytes. I don't want that.

That means the only option I see is initializing the array and reserving space in the microcontroller, on the fly. This will take up some memory and execution time, of course. I thought of a system like this:

Initialize an array int[x] = {int, int} which holds pointers to the start and end of arrays that are not initialized from the beginning - x would be the maximum amount of arrays (this is a concession, but is better than a maximum length for all arrays)

Store a variable c = 0 to indicate the number of arrays used

Store the borders of the initialized (reserved) memory in a variable somewhere

When an array gets a length:

Put pointers to the start (the current border) and end (the current border + the length) in the array from the first point at index c

Increment c

I think this would work (wouldn't it?), but there a few cons, mainly concerning memory: I need to store the array, c and the current memory borders as overhead.

Would there be a better way to implement non-fixed size arrays in a language for PIC microcontrollers? My requirements are:

Low memory overhead

The array length does not have to be changed on the fly

With the system I thought up, one cannot store values in arrays that haven't been initialized yet. If there would be a system that can store values in an array of undefined length, that would be an advantage

Faster systems (at runtime, compile time doesn't matter) are preferable

答:
You will probably make it a lot easier on yourself and avoid a lot of user errors if you allow the declaration of variable-length arrays to be delayed until a point where the size is known. Then you don't have to invent a syntax for assigning the array size and you don't have to deal with differences in the order in which arrays are declared and in which they get their size.

If your compiler uses a stack-based allocation scheme for local variables (they are given addresses relative to the stack-frame of their function, like it is done for the 'big' platforms), the variable-length arrays can just be carved out of the stack frame as needed. Only if there are multiple variable-length arrays in a function do you need an additional pointer to indicate where the data of each starts.

If your compiler uses a fixed allocation scheme (all variables, including locals, are given a fixed (absolute) address by the compiler/linker), I would use an 'array stack' for carving the variable-length arrays from. As overhead, you would need a global pointer to indicate where the free space currently starts,  a pointer for each variable-length array to indicate where its data is located and, in each function using variable-length arrays, a way to restore the global pointer to the value it had when entering the function.

The user would also need some way to indicate to the compiler/linker how much of a 'array stack' he thinks will be needed. A reasonable default could be to use whatever is left after allocating all fixed-size variables and variable-length array overhead.

I think this scheme uses the least amount of overhead, both space and time-wise. The scheme you came up with is essentially a malloc implementation.


问:
What is the difference between a combinator and function chaining?
So from what I've read about combinators I can't quite tell how they're different from simply chaining function calls.  I know I must be missing something but I'm not figuring out what I'm missing.  I mean would g(f(x)) (where f and g are functions) effectively be a combinator if I gave it a different name?

答:
Function chaining is known as function composition in the functional world.

A combinator however is a completely different thing, the composition operator is simply one combinator.

I wrote a blog attempting to explain combinators here it may help.

In it I try to explain a combinator is simply a function that follows a few rules in the general case, but in the common case "combinator" is used to refer to a function which abstracts a bit of common functionality that is very basic in form.

in SKI combinator calculus there are 3 combinators:
Kxy = x
Sxyz = xz(yz)
Ix = x

These are combinators because they act only upon their terms, and they are generalized to not care what their terms are.

This seems useless but realize I can be defined in terms of S and K:
SKKx = Kx(Kx) = x which is equivalent to Ix which = x, the benefit to combinators is they are fully generalized to whatever terms you pass them and therefore live at a higher level abstraction than other things.

The composition combinator is the B combinator from the B,C,K,W combinator system

Now to make combinators useful however, you have to understand when fully generalized functions like this are worth creating. Parsers are the perfect and most common example, because a parser as a concept has to deal with an extremely large number of possible inputs, therefore the ability to generalize the handling of their inputs comes in very handy.