问:
What is the value of workflow tools?
I'm new to Workflow developement, and I don't think I'm really getting the "big picture". Or perhaps to put it differently, these tools don't currently "click" in my head.

So it seems that companies like to create business drawings to describe processes, and at some point someone decided that they could use a state machine like program to actually control processes from a line and boxes like diagram. Ten years later, these tools are huge, extremely complicated (my company is currently playing around with WebSphere, and I've attended some of the training, its a monster, even the so called "minimalist" versions of these workflow tools like Activiti are huge and complicated although not nearly as complicated as the beast that is WebSphere afaict).

What is the great benefit in doing it this way? I can kind of understand the simple lines and boxes diagrams being useful, but these things, as far as I can tell, are visual programming languages at this point, complete with conditionals and loops. Programmers here appear to be doing a significant amount of work in the lines and boxes layer, which to me just looks like a really crappy, really basic visual programming language.

If you're going to go that far, why not just use some sort of scripting language? Have people thrown the baby out with the bathwater on this? Has the lines and boxes thing been taken to an absurd level, or am I just not understanding the value in all this?

I'd really like to see arguments in defense of this by people that have worked with this technology and understand why its useful. I don't see the value in it, but I recognize that I'm new to this as well and may not quite get it yet.

答:
There is only one real benefit, yet its huge:
Separation of Concerns.

So, instead of process orchestration logic being embedded in our system, it becomes and external configuration. A map, basicly. You can change it (much more) independently, you can have multiple processes, multiple versions of processes, multiple versions of multiple processes running at the same time, and that's all out of the box in any decent solution.

Historically, the concept of SoC have won many times - starting from the Unix principle "do one thing, but do it good", and being applied again and again - like having dedicated server components like ESB, different persistance systems, caching, load balancing, monitoring, like splitting CSS from HTML, etc.

Your business process and its flow rules are often orthogonal to your data, UI "screens" or users "hierarchy". So, it makes perfect sense to develop and change it separately from the other aspects of the system. That was the premise on which BPM has appeared in early 1990s.

Since then, many of tools and languages were created to support this concept, with most well known being BPMN - a graphical language for creating "flowcharts" that directly map to processes. While people complain that its large and unwieldy (having over 100 symbols in vocabulary), and advocating modern approaches like S-BPM (has only 5 base symbols), current industry practice is to stick to BPMN or its derivatives, subsets or siblings.

You do not look pleased with BPMN:

Programmers here appear to be doing a significant amount of work in the lines and boxes layer, which to me just looks like a really crappy, really basic visual programming language.

But its not that bad ) There is theory behind it. And version 2.0 took some good insight from 1.0 shortcomings.

If you're going to go that far, why not just use some sort of scripting language?

Imperative paradigm and scripting languages are not always the best answer. As you've probably seen in declarative languages (like HTML, CSS, SQL, Drools or internal ones of Nginx, Graddle and Maven, Puppet etc) the resulting code can be much smaller and cleaner, than a version written in "decent language, like Java or C++".

As for your other point:

as far as I can tell, are visual programming languages at this point, complete with conditionals and loops.

have you looked into the Events and Triggers ? BPMN is a language and you have to learn it before using, or at least get familiar with it.

Under the hood, BPMN is XML, so you can edit it by hand, or generate.  And you can version control them, because XML is text-based. However, just having an XML that can be translated into flowcharts, does not sound like its goona help you, and that's correct - writing your own parser or editor for it is a tough and expensive task with questionable benefits.

Luckily, there are already tools at the market which do exaclty that.

Activiti is free, and quite popular among both developers and busines owners, because of its initial price (zero), availability of information and humbleness. The last point is really unique, as Activi only focuses on managing your business processes, without trying to tie you in with whole-package solutions. Also, its open - so you only need to know Java and REST to get it up and running. The drawback is that client side, integration and application/businesss logic and the whole architecture are left to developer, so if your develpment team is weak - prepare for the fail. Total cost of ownership can be surprisingly high for a free tool ;)

On the other side of spectrum is Pega (Pega PRPC), the reigning king of BPM systems (according to Gartner and Forester), which looks surprisingly good for its age. This and-a-kitchen-sink behemoth is even capable of CRM, OCR and (if i'm not mistaken) speech recognition capabilites, predictive analytics, business rules management and WYSIWYG UI editor. It comes with a serious a price tag, though. Not only does it cost a fortune, but all the development is being done within a web-app, which means you have to use browser, which is IE8 (plus some plugins, plus ugly hacks, like using Excel for editing data tables). Also, Java, Javascript or HTML/CSS editing is also being done withing web browser - so say goodbye to unit tests, IDE code highlighting, refactoring and all your programming toys you've leaned to love.

Good side of it ? you can implement complex system WITHIN WEEKS [PDF, see page 22]. And yes, result is not guaranteed.

IBM have somewhat recently (accoring to the enterprise time pace) have bought Lombardi, and is now offering very competitive solution (but then you will have to buy everything ibm, you'know). Appian is young vendor who has interesting insights and positive feedback, but the way they are written (two extra DSL languages in addition to visual one) just doesn't appeal to me.

There are other players, and their solutions. Most of them are plain horrible. Like - your eyes, brain and heart would literally bleed, when you simply look at them. So, trust your guts and don't make your developers and user hate you.

Closing note:

BPM system is the same for for processes, what Photoshop is for images. Don't be afraid that its  visual. Don't make it do the job not suited for it (remember web sites created entirely in Photoshop, which were next to impossible to edit?). Keep it simple and don't make bugs ;)

Title: What is the value of workflow tools?


问:
Metric by which to hold developers accountable
I asked a question on lines of code per hour and got torn a new one.  So my matured follow-up question is this:

If not lines of code, then what is a good metric by which to measure (by the hour/day/unit-of-time) the effectiveness of remote programmers?

答:
It certainly is possible to devise all kinds of intricate metrics to evaluate performance, but at the end of the day a significant part of your judgment has to rely on subjectivity and input from people who are close to the codebase.

For example, it is quite possible for some team to crank out internally hideous unmaintainable slop at a very fast rate, and this might even meet the required deadline and specification. But is the technical debt accrued from that kind of working style worse than if the team had churned out something robust and maintainable but slipped the deadline a few weeks? It depends.

If the purpose of the question is to resolve some type of productivity problem, I would say that what the manager actually does to facilitate the work of the team is as or more important than any measuring technique used to evaluate the team. It is a two-way street. In other words, I am saying that metrics are fine but if you want more out of any team the ultimate question is whether or not the manager doing everything possible to ensure the team can be productive.

This goes far beyond writing a spec, finding a team, throwing the spec "over the wall" and clicking a stopwatch.

ContentLicense: CC BY-SA 2.5


问:
What's good about statement coverage?
When it comes to code coverage, you often hear about the flaws of statement coverage criteria. We're told that statement does not take into account control structures and branches thereof, and is therefore sub-par compared to other criteria such as branch coverage, condition coverage, and even function coverage.

But what are the actual benefits of statement coverage, and to what degree is it used (perhaps in co-agency with other criteria) in industry? Does statement coverage criteria fill needs that other criteria do not?

答:
The only advantage of statement coverage is that it is much easier to calculate than other coverage variants. You just need to know if some line of code was executed and calculating coverage ratio is as simple as counting statements that were executed and total count of statements.

With other coverage variants, you have to take account code before and after a statement. You need to figure out which branch of if was taken. If for loop was actually executed. You would also need to complexly calculate all possible paths the code would take and count those to be able to get relative coverage.

Statement coverage is also much easier to visualize. You can just show statements that are covered in green and those that are not in red. But for other coverage variants, it becomes hard to visualize in which branch statement was hit (or not) and by which test.

And the primary reason why it is so much used in industry now is that, being simplest, it was first one used. And users then don't see reason to move to more complex variants (or don't even realize they exist) which creates low demand for actual complex version being implemented and entering people's minds.

ContentLicense: CC BY-SA 3.0

Title: What's good about statement coverage?


问:
Ensuring non conflicting components in a modular system
So lets say we are creating a simple "modular system" framework.

The bare bones might be the user management. But we want things like the Page Manager, the Blog, the Image Gallery to all be "optional" components.

So a developer could install the Page Manager to allow their client to add a static home page and about page with content they can easily edit with a wysiwyg editor.

The developer could then also install the Blog component to allow the client to add blog entries.

The developer could then also install the Gallery component to allow the client to show off a bunch of images.

The thing is, all these components are designed to be independent, so how do we go about ensuring they don't clash? E.g. ensuring the client doesn't create a /gallery page with the Page Manager and then wonder why the gallery stopped working, or the same issue with the Blog component, assuming we allow the users to customize the URL structure of the blog (because remember, the Page Manager doesn't necessarily have to be there, so we might not wan't our blog posts to be Date/Title formatted), likewise our clients aren't always going to be happy to have their pages under pages/title formatting.

My core question here is, when building a modular system how to we ensure that the modules don't conflict without restricting functionality?

Do we just leave it up to the clients/developer using the modules to ensure they get setup in a way that does not conflict?

答:
You can't have both without restricting your user:

allowing users to create the URLs freely AND having collision-free URLs for each module.

If you want to allow users to create URLs, you may check if the entered URL is already in use through doing self-queries but that would still result in having the user restructure the URL catalogue in case there has been a misconfiguration (like in your /gallery example).

You could create URLs based on module name automatically i.e. /gallery-one for an entry one for module gallery and then offer user functionality to add non-colliding URL rewrites but this may lead to seriously confusing .htaccess files.


问:
How early does a hashable type need to become immutable?
Suppose I have a class that looks vaguely like this:

```<code>class Foo(object):
    _value = None  # XXX: see below

    def __init__(self, value):
        self._value = value

    @property
    def value(self):
        """The value of this Foo."""
        return self._value

    def __hash__(self):
        return hash(self.value)

    def __eq__(self, other):
        return type(self) is type(other) and self.value == other.value
</code>```

(I know the _value = None line is wrong; in my actual code, _value is a data descriptor with a default value.  It never raises AttributeError even if it hasn't been set yet, and I'm just trying to emulate that behavior in as few lines of code as possible.  I could rip the behavior out, but it would significantly complicate things in other areas of my codebase.)

Have I violated the rules?  Should I be setting _value in Foo.__new__() instead?  Clearly it needs to be set somewhere; there will be a point in time at which the hash() of our instance changes.  When should that happen, exactly?

If there is not anything wrong with the above code, then why is the following code wrong?  Logically, this combination is illegal, but I'm not sure which class has broken the rules:

```<code>bar = set()

class Baz(object):
    def __init__(self, *args, **kwargs):
        bar.add(self)
        super(Baz, self).__init__(*args, **kwargs)  # play nice with MI

class Qux(Baz, Foo):
    pass

assert Qux('quuux') in bar  # assertion fires because the hash() changed
</code>```

答:
I would say Baz has broken the rules. By using self as hashable before even invoking other __init__ methods in the MRO, it assumes that those methods do nothing to influence the hash. That is a wrong assumption. Although technically it is safer to construct immutable classes so that __new__ sets up all state and makes it completely, utterly, immutable (like tuple and namedtuple), this is often very clunky and unnecessary.

Setting some private variables that are read-only for the outside world and never mutated in any methods (other than __init__ where they are first set) is a perfectly valid and common way to implement immutable classes.

This ties into a more general point: Doing anything at all with an object while it's still being initialized is very risky and need to be thought out very well. Many invariants may be violated before __init__ has finished, so most code can't meaningfully interact with a partially-initialized object. Luckily, a sane __new__ implementation takes care that the new object is first passed to __init__ (and, of course, any code that __init__ invokes itself — which, again, needs to be carefully chosen).


问:
Unit testing and reality
I m actually studying Unit testing, reading some articles on the internet and trying to understand how it works exactly, but it's not very clear for me actually.

What should I test ?

I have seen that it's a good thing to create interfaces and make our testable classes implement this interface (permitting to implement it in our test classes) but what are exactly the classes that I should test ?

For example, admitting that I have a project structure like this :

Presentation Layer (MVC)

Business Layer (Service / Repositories)

DB Layer (Mongo, Mysql etc..)

Should I only test services and repositories ? Should I create an interface to all my classes, even if I have only one implementation of it (two with the test) ?

What does my test really return ?

I have read an article, in which the author explaning the way to test a DB access using mocks :

There is a class "S" which is a business service implementing an interface "I", that manage the data taken from the db to provide business objects

There is a class "C" which is a controller and has a renderView method, which send to the view an array of the business objects given by the service "S".

1/ In the first test, he tests the value returned by "S" on getBusinessObject only using a collection of String. So he only tests if he has data in the collection (null, the content after adding an item etc...)

2/ In the second test, he mocks the "S" service to use it to test the renderView and only tests if he gets 2 results when the mock object method returns 2 objects.

My question is, how can we say that the test is a representation of the real state of our programm ?

I mean, it's possible that we have to make a difficult job in a service, with condition, iteration etc...

I dont really understand the impact of making unit tests if we mock a class that

```<code>return "a string";
</code>```

comparing to a classe that transform drastically a String in the method.

Maybe I didn't understand the aim of unit testing in fact.

答:
I've had contractors who have left after creating a database layer with a set of unit tests which passed, but didn't actually save anything to the database so everything was lost if you restarted the system.

So test the parts of your system you want to work by writing tests which compare the expected behaviour with the real behaviour.

If those parts are small enough to be tested without much scaffolding and configuration, then those tests are unit tests. If they require multiple parts to work in concert, then they are integration tests.

If you find that you'd rather be coding than waiting for long integration tests to run, then refactor the system into smaller pieces which can be tested using unit tests.

Often this involves creating interfaces and mocks/stubs so the business and UI layers can be tested quickly without a real database. The UI layer probably doesn't care whether the thing being displayed is 'a string' or real data. Similarly, the DB layer may not care whether it is being asked to store real data or 'a string'†. The business layer which does something to the data may well care.

The key here is that the mocks should return something which will  exercise the functionality under test, but are no more complicated than is necessary to do that.

You end up with more unit tests and fewer integration tests and a better factored system, but you still need the integration tests to check it actually does fulfil the overall spec.

† actually, most of my projects store quantities of time series data so the nature of the real data dictates the design of the database, so generally I test using a few months real data, not made-up data.

Title: Unit testing and reality


答:
Unit tests are essentially tests of a single component. With perhaps some mock objects to fake some interfaces used by the module.

You are testing that for a given input you receive the expected output. If you test involves other modules components or services which cannot be mocked then it is no longer a unit test.

As such they are very useful. You can test edge conditions and extreme values which are unlikely but could possibly appear in real data.

However unit testing is only the beginning. The module needs to be tested with the other modules in the system, and, with any other system that it interacts with.

In addition you need to do realistic tests with real users a program can only be considered to be "working" if the end user thinks it works.

Title: Unit testing and reality

Body:
I m actually studying Unit testing, reading some articles on the internet and trying to understand how it works exactly, but it's not very clear for me actually.

What should I test ?

I have seen that it's a good thing to create interfaces and make our testable classes implement this interface (permitting to implement it in our test classes) but what are exactly the classes that I should test ?

For example, admitting that I have a project structure like this :

Presentation Layer (MVC)

Business Layer (Service / Repositories)

DB Layer (Mongo, Mysql etc..)

Should I only test services and repositories ? Should I create an interface to all my classes, even if I have only one implementation of it (two with the test) ?

What does my test really return ?

I have read an article, in which the author explaning the way to test a DB access using mocks :

There is a class "S" which is a business service implementing an interface "I", that manage the data taken from the db to provide business objects

There is a class "C" which is a controller and has a renderView method, which send to the view an array of the business objects given by the service "S".

1/ In the first test, he tests the value returned by "S" on getBusinessObject only using a collection of String. So he only tests if he has data in the collection (null, the content after adding an item etc...)

2/ In the second test, he mocks the "S" service to use it to test the renderView and only tests if he gets 2 results when the mock object method returns 2 objects.

My question is, how can we say that the test is a representation of the real state of our programm ?

I mean, it's possible that we have to make a difficult job in a service, with condition, iteration etc...

I dont really understand the impact of making unit tests if we mock a class that

```<code>return "a string";
</code>```

comparing to a classe that transform drastically a String in the method.

Maybe I didn't understand the aim of unit testing in fact.


问:
Use of underscores and earmuffs
I have seen underscores and earmuffs (like _good_ or *good*) frequently used in conversations among programmers. All I know is that underscores are used to denote private variables or methods in languages like C, Python etc. and  earmuffs are the preferred way of indicating that something is a global variable.

Do these have any other punctuational meaning when they are used outside code, in mailing lists etc? Does the highlighted word have any hidden meaning or expression added to it? Or, are they just like air quotes?

答:
This is not something that is specific to programmers, but occurs in electronical communications in general.

The underscores are used to mean underlining, and the asterisks are ment to indicate an action.

Example:

```<code>This is a _really_awesome_ feature. *clapping*
</code>```

Title: Use of underscores and earmuffs

Body:
This is not something that is specific to programmers, but occurs in electronical communications in general.

The underscores are used to mean underlining, and the asterisks are ment to indicate an action.

Example:

This is a _really_awesome_ feature. *clapping*


问:
Does cleaning data before saving fall under the term "data sanitization"?
I understand that "data sanitization" mainly deals with cleaning user-entered data to prevent harmful effects (e.g. SQL injection), but what about other forms of cleaning data? E.g., trimming whitespace from text or removing leading zeros from numbers. Do these also fall under "data sanitization"? If not, what is this type of data cleaning called?</p>

答:
Canonicalization

In computer science, canonicalization (sometimes standardization or normalization) is a process for converting data that has more than one possible representation into a "standard", "normal", or canonical form. This can be done to compare different representations for equivalence, to count the number of distinct data structures, to improve the efficiency of various algorithms by eliminating repeated calculations, or to make it possible to impose a meaningful sorting order.


问:
Sending Out Functions To Return Or To Die
Which is preferable for both solid technique and secure coding?

Example #1:

```<code>function_one()
    blah;
    function_two()
        blah;
        print blah;
        exit;
</code>```

...

Example #2:

```<code>function_one()
    blah;
    function_two()
        blah;
        return blah;
    return blah;
print blah;
</code>```

Obviously this is a simplified version, and I am looking for an answer which is universal for larger function trees, if possible.

-1: I liked Example #1 because it seemed faster to terminate as soon as it is done doing what it does, instead of returning to the original call to lives out its natural life. Also, Example #1 seems to compartmentalize code, helping to prevent unintended code from executing further on down (which should be detected during testing, but I prepare for the worst). However, this model has proven difficult to maintain, and doesn't seem to scale well. For instance, I had a web page which I wrote to echo out what it was supposed to and then terminate, but this caused problems down the road with making sure that the http headers were inserted before the output; this would require sending the headers as a function parameter.

-2: Example #2 seems to maintain a standard order of doing things because not all functions will terminate as soon as possible. Also code can be combined easier without passing additional parameters as in Example #1. However, Example #2 requires more return statements, and will continue out the rest of the program. What do you think?

答:
In general, you should avoid exiting your entire program from random functions. Why?

It makes your code hard to read. The function in question becomes difficult because you then need to care about what context it's being called from. The rest of the program becomes difficult because the "normal" exit point isn't correct all the time.

It makes that code less reusable. Just because you consider that a fatal error (with a certain message) doesn't mean that is the end of other programs.

It makes that code hard to unit test. Depending on the language, it may be impossible to unit test.

There are certainly cases where it can make sense - global exception handlers which clean up before dying, for example. But in general, just return or throw.


问:
Should a front-end developer ever specify JSON format for back-end developers?
I am taking the front-end role in a project. Should I be specifying for my back-end teammates the exact format of JSON that their PHP returns to my JavaScript?

For example, should I be telling them they should use a format similar to one described here:

Proper way to structure JSON for front-end consumption

Or should I keep my role as sterile as possible, and simply describe in words the inputs and outputs I need from their back-end interface? (Of course, if this happens it might be more difficult on my part to handle their different data structure formats)

答:
I would just ask Why Not?
When we are talking about a project we also talk about the team working on it and it is expected and should be welcomed to hear opinion about the features and and structure used. As a developer I personally believe and value the contributions of teammates.

You know there is a saying "if you want to go fast go alone. if you want to go far go together".


问:
What programming mindset to have towards JavaScript?
I used to program in Java. As a result of that I got well-versed with Object Oriented Paradigm. In my opinion it is a great approach to programming because it's easy to unit test, use design patterns, and so on.

I am now programming in JavaScript. Should I approach JavaScript with an Object Oriented mindset, too? It seems to me that the answer should be "yes", but I want to hear other people's opinion.

答: Javascript is more different from Java than a surface comparison might suggest. An object-oriented approach can get you a long way, but it is not in itself sufficient to become a good javascript programmer, because a few of the critical patterns of javascript are not really object-oriented in nature. You will find that judicious use of closures is as important as any OO design pattern, and that a firm grasp of the "continuation function" pattern is almost as critical.</p>


答:
JavaScript is a multiparadigm language that you can approach from a pure OOP point of view, but most people don't, for various reasons I'll describe in a bit.  In my experience, idiomatic JavaScript tends to be around 50% functional, 30% OOP, and 20% procedural.  However, that mix varies a lot depending on what you're doing and who you work with.

Some reasons OOP isn't used as heavily in JavaScript as you might expect:

The this pointer has weird scoping rules that complicate things.

Programmers from the Java/C#/C++ schools aren't accustomed to prototype-style objects.

Programs tend to be very asynchronous and callback-heavy, which lends itself very well to a functional style.

Functional capabilities of the language obviate the need for many OOP design patterns.

The smaller, more self-contained, single purpose scripts lend themselves well to a procedural style.

Title: What programming mindset to have towards JavaScript?

Body:
I used to program in Java. As a result of that I got well-versed with Object Oriented Paradigm. In my opinion it is a great approach to programming because it's easy to unit test, use design patterns, and so on.

I am now programming in JavaScript. Should I approach JavaScript with an Object Oriented mindset, too? It seems to me that the answer should be "yes", but I want to hear other people's opinion.


问:
Locking on data vs locking on functions
In concurrent code, what is the difference between, and the pros and cons of each:

Locking on data

Locking on functions

My understanding is that locking on data is better, and I think I'm doing it, but apparently I am not. Could anyone provide some clarification?

答:
There's a subtle distinction between the two.  Let's take a look the following sample code.

public class Foo {
    private static Object lockFunction = new Object();
    private static Bar myBar = new Bar();

    public void fooFunc(float someValue) {
        synchronized (lockFunction){                //locking against the function
            fooPrivates(someValue);
        }
    }

    public void fooFunc2(float someValue1) {
        myBar.setValue(someValue2);             //Danger! unlocked access to myBar
    }

    private void fooPrivates(float someValue){
        myBar.setValue(someValue);
    }
}

public static class Bar {
    private static Object lockBarData = new Object();
    private float myValue;

    public float getValue() {
        synchronized (lockBarData) {            //technically not necessary since it's a read operation
            return myValue;
        }
    }

    public void setValue(value) {
        synchronized (lockBarData) {            //locking against the data
            myValue = value;
        }
    }
}
```

The danger with lockObject is that it is controlling access to calling the function fooPrivate.  It exercises no control over other functions accessing the Bar object myBar.  And if you happen to have static copies of myBar then you're going to have a single copy of myBar used by multiple copies of Foo.

lockBar on the other hand is controlling access to the private variable myValue within the Bar class.

In a trivial example like the one above, they'll both work because you can make sure that functions like FooFunc2 simply aren't allowed to work that way. But in a complex, multi-threaded application it gets easier for subtle situations to arise where myBar is manipulated without being controlled by a lock.

In my experience, locking against data is the better way to go.

When you lock against the function, you have to make sure that everything accessing that function operates the same way.  It introduces greater governance concerns in order to make sure that non-conforming code doesn't sneak into the code base.  Another concern with locking against the function is that you're generally holding the lock for a longer period of time than you would if you're locking against the data.

Locking against the data makes it easier to hide the implementation of the data object from users of the data.  Even in the trivial example above, Foo doesn't necessarily know (or care!) that Bar needs to have a lock in place before anything can be done with myValue.

Title: Locking on data vs locking on functions

Body:
There's a subtle distinction between the two.  Let's take a look the following sample code.

public class Foo {
    private static Object lockFunction = new Object();
    private static Bar myBar = new Bar();

    public void fooFunc(float someValue) {
        synchronized (lockFunction){                //locking against the function
            fooPrivates(someValue);
        }
    }

    public void fooFunc2(float someValue1) {
        myBar.setValue(someValue2);             //Danger! unlocked access to myBar
    }

    private void fooPrivates(float someValue){
        myBar.setValue(someValue);
    }
}

public static class Bar {
    private static Object lockBarData = new Object();
    private float myValue;

    public float getValue() {
        synchronized (lockBarData) {            //technically not necessary since it's a read operation
            return myValue;
        }
    }

    public void setValue(value) {
        synchronized (lockBarData) {            //locking against the data
            myValue = value;
        }
    }
}

The danger with lockObject is that it is controlling access to calling the function fooPrivate.  It exercises no control over other functions accessing the Bar object myBar.  And if you happen to have static copies of myBar then you're going to have a single copy of myBar used by multiple copies of Foo.

lockBar on the other hand is controlling access to the private variable myValue within the Bar class.

In a trivial example like the one above, they'll both work because you can make sure that functions like FooFunc2 simply aren't allowed to work that way. But in a complex, multi-threaded application it gets easier for subtle situations to arise where myBar is manipulated without being controlled by a lock.

In my experience, locking against data is the better way to go.

When you lock against the function, you have to make sure that everything accessing that function operates the same way.  It introduces greater governance concerns in order to make sure that non-conforming code doesn't sneak into the code base.  Another concern with locking against the function is that you're generally holding the lock for a longer period of time than you would if you're locking against the data.

Locking against the data makes it easier to hide the implementation of the data object from users of the data.  Even in the trivial example above, Foo doesn't necessarily know (or care!) that Bar needs to have a lock in place before anything can be done with myValue.


问:
Maintenance cost of SIMD programming code base

Question:

The software industry's consensus is that clean and simple code is fundamental to the long-term viability of the code base and the organization that owns it. These properties lead to lower maintenance costs and increased likelihood of the code base being continued.

However, SIMD code is different than general application code, and I would like to know if there is a similar consensus regarding clean and simple code applying specifically to SIMD code.

Background to my question.

I write plenty of SIMD (single-instruction, multiple data) code for various image processing and analysis tasks. Recently I also had to port a small number of these functions from one architecture (SSE2) to another (ARM NEON).

The code is written for shrink-wrapped software, therefore it cannot depend on proprietary languages without unrestricted redistribution rights such as MATLAB.

An example of typical code structure:

Using OpenCV's matrix type (Mat) for all memory, buffer and lifetime management.

After checking the size (dimensions) of input arguments, the pointers to the start address of each row of pixels is taken.

The pixel count, and the start addresses of each row of pixels from each input matrix is passed into some low-level C++ functions.

These low-level C++ functions use SIMD intrinsics (for Intel Architecture, and ARM NEON), loading from and saving to raw pointer addresses.

Characteristics of these low-level C++ functions:

Exclusively one-dimensional (consecutive in memory)

Does not deal with memory allocations.
(Every allocation, including temporaries, are handled by the outer code using OpenCV facilities.)

The range of name lengths of symbols (intrinsics, variable names, etc) are roughly 10 - 20 characters, which is quite excessive.
(Reads like techno-babble.)

Reuse of SIMD variables is discouraged because compilers are quite buggy in correctly parsing code that is not written in the "single-assignment" coding style.
(I've filed several compiler bug reports.)

What aspects of SIMD programming would cause the discussion to differ from the general case?  Or, why is SIMD different?

In terms of initial development cost

It is well-known that the initial development cost of C++ SIMD code with good performance is about 10x - 100x (with a wide margin) compared to casually-written C++ code.

As noted in the answers to Choosing between performance vs readable/cleaner code? , most code (including casually-written code and SIMD code) is initially neither clean nor fast.

Evolutionary improvements in code performance (in both scalar and SIMD code) is discouraged (because it is seen as a kind of software rework), and the cost and benefit is not tracked.

In terms of propensity

(e.g. the Pareto principle, a.k.a. the 80-20 rule)

Even if image processing only comprises 20% of a software system (in both code size and functionality), image processing is comparatively slow (when viewed as a percentage of CPU time spent), taking more than 80% of time.

This is due to the data size effect: A typical image size is measured in megabytes, whereas the typical size of non-image data is measured in kilobytes.

Within the image processing code, a SIMD programmer is trained to automatically recognize the 20% code comprising the hotspots by identifying the loop structure in the C++ code. Thus, from a SIMD programmer's perspective, 100% of the "code that matters" is performance bottleneck.

Often in an image processing system, multiple hotspots exist and take up comparable proportions of time. For example, there may be 5 hotspots each taking up (20%, 18%, 16%, 14%, 12%) of total time. To achieve a high performance gain, all of the hotspots need to be rewritten in SIMD.

This is summarized as the balloon-popping rule: a balloon cannot be popped twice.

Suppose there are some balloons, say 5 of them. The only way to decimate them is to pop them one by one.

Once the first balloon is popped, the remaining 4 balloons now comprises a higher percentage of total execution time.

To make further gains, one must then pop another balloon.
(This is in defiance to the 80-20 rule of optimization: a good economical outcome can be achieved after the 20% of lowest-hanging fruits has been picked.)

In terms of readability and maintenance

SIMD code is patently hard to read.

This is true even if one follows every software engineering best-practice e.g. naming, encapsulation, const-correctness (and making side-effects obvious), function decomposition, etc.

This is true even for experienced SIMD programmers.

Optimal SIMD code is very contorted, (see remark) compared to its equivalent C++ prototype code.

There are many ways to contort SIMD code, but only 1 out of 10 such attempts will attain acceptably fast results.

(That is, in the tunes of 4x-10x performance gains in order to justify to high development cost. Even higher gains have been observed in practice.)

(Remark)

This is the main thesis of the MIT Halide project - quoting the paper's title verbatim:

"decoupling algorithms from schedules for easy optimization of image processing pipelines"

In terms of forward applicability

SIMD code is strictly tied to a single architecture. Each new architecture (or each widening of SIMD registers) requires a rewrite.

Unlike the majority of software development, each piece of SIMD code is typically written for a single purpose that never changes.
(With the exception of porting to other architectures.)

Some architectures maintain perfect backward compatibility (Intel); some fall short by a trivial amount (ARM AArch64, replacing vtbl with vtblq) but which is sufficient to cause some code to fail to compile.

In terms of skills and training

It is not clear what knowledge prerequisites are required to properly train a new programmer to write and maintain SIMD code.

College graduates who have learned SIMD programming in school seems to despise and dismiss it as an impractical career track.

Disassembly-reading and low-level performance profiling are cited as two fundamental skills for writing high-performance SIMD code. However, it is unclear how to systematically train programmers in these two skills.

Modern CPU architecture (which diverges significantly from what is taught in textbooks) makes training even more difficult.

In terms of correctness and defect-related costs

A single SIMD processing function is actually cohesive enough that one can establish correctness by:

Applying formal methods (with pen-and-paper), and

Verifying output integer ranges (with prototype code and performed outside run-time).

The verification process is, however, very costly (spends 100% time on code review and 100% time on prototype model checking), which triples the already-expensive development cost of SIMD code.

If a bug somehow manages to slip through this verification process, it is nearly impossible to "repair" (fix) except to replace (rewrite) the suspected defective function.

SIMD code suffers from the blunt of defects in the C++ compiler (optimizing code generator).

SIMD code generated using C++ expression templates also suffers greatly from defects of the compiler.

In terms of disruptive innovations

Many solutions have been proposed from academia, but few are seeing widespread commercial usage.

MIT Halide

Stanford Darkroom

NT2 (Numerical Template Toolbox) and the related Boost.SIMD

Libraries with widespread commercial usage do not seem to be heavily SIMD-enabled.

Open-source libraries seem lukewarm to SIMD.

Recently I have this first-hand observation of this after profiling a large number of OpenCV API functions, as of version 2.4.9.

Many other image processing libraries I have profiled also do not make heavy use of SIMD, or they miss the true hotspots.

Commercial libraries seem to avoid SIMD altogether.

In a few cases, I have even seen image processing libraries reverting SIMD-optimized code in an earlier version to non-SIMD code in a later version, resulting in severe performance regressions.
(The vendor's response is that it was necessary to avoid compiler bugs.)

This Programmer's question:  Does low latency code sometimes have to be "ugly"?  is related, and I previously wrote an answer to that question to explain my view points a few years ago.

However, that answer is pretty much "appeasement" to the "premature optimization" viewpoint, i.e. to the viewpoint that:

All optimizations are premature by definition (or, short-term by nature), and

The only optimization that has long-term benefit is toward simplicity.

But such viewpoints are contested in this ACM article.

All of that leads me to ask:

SIMD code is different than general application code, and I would like to know if there is a similar industry consensus regarding the value of clean and simple code for SIMD code.

答:
My organization has dealt with this exact problem. Our products are in the video space, but much of the code we write is image processing that would work for still images, too.

We "solved" (or maybe "dealt with") the problem by writing our own compiler. This isn't quite as crazy as it sounds at first. It's got a restricted set of inputs. We know that all the code is working on images, mostly RGBA images. We set up some constraints, like that the input and output buffers can never overlap, so there's no pointer aliasing. Things like that.

We then write our code in the OpenGL Shading Language (glsl). It gets compiled to scalar code, SSE, SSE2, SSE3, AVX, Neon, and of course actual glsl. When we need to support a new platform, we update the compiler to output code for that platform.

We also do tiling of the images to improve cache coherency, and stuff like that. But by keeping the image processing to a small kernel, and using glsl (which doesn't even support pointers) we greatly reduce the complexity of compiling the code.

This approach isn't for everyone, and it has its own problems (you need to ensure compiler correctness, for example). But it has worked out fairly well for us.


答:
Absolutely! When I write javascript, I mainly write objects. Javascript has a wonderful, subtle, IMHO elegant object model that equates function instances, maps, and objects. So you can take advantage of each of those semantics. In practice, I end up wrapping all my code in objects, not because of my Java experience, but just because it allows for nice modularization.

Also, because of the interesting equivalences between functions, maps, and objects, it may be a lot of folks are using object design without realizing it.

For example, especially with the modern javascript libraries available, I see lots of code that looks like this:

```<code>{ 'thing1': 'blah',
  'thing2': function() { ... }
}
</code>```

This is an object-oriented approach, because it defines a map (or associative array) which in javascript is the same as an object. (or visa-versa, I forget). The important point it, it bundles data with functionality, which is a core concern of OO.

Failing that, I see a lot of code rolled up into objects for the purposes of modularization. jquery is heavily object-oriented, and you see lots of extentions like:

```<code>$.zoom = new function() {
  var privateValue = 0;
  this.visibleFunction = function() { ... };
}();
</code>```

Which tacks an object on top of what jquery provides.

So in my experience, increasingly in the last few years as the javascript frameworks take on an OO aspect, is that OO practice in javascript is really almost the predominant methodology. And, from my experience, no serious javascript developers have given me grief about rolling my code up in objects -- if anything just the opposite.

(By comparison, that's not as true with in the Python world, where in Python the more natural attitude is dropping in functions at the module level.)

But the joy I find with doing OO in javascript is that you can leverage the nice scoping / visibility mechanisms it provides to really create clean object semantics in a way that can be really cumbersome in java. And then you get the nice benefits of a dynamic language, like late binding of functions to objects. Also, in Java, reflection is a great set of utilities, but I've always found it really cumbersome. But in javascript, reflection is just a natural aspect of the language.

So I love taking a break and doing OO in javascript occasionally, because it feels liberating after slogging through Java for a while.

I think the primary difference with OO in java and javacript is that javascript isn't as obnoxious about it.

Body:
Absolutely! When I write javascript, I mainly write objects. Javascript has a wonderful, subtle, IMHO elegant object model that equates function instances, maps, and objects. So you can take advantage of each of those semantics. In practice, I end up wrapping all my code in objects, not because of my Java experience, but just because it allows for nice modularization.

Also, because of the interesting equivalences between functions, maps, and objects, it may be a lot of folks are using object design without realizing it.

For example, especially with the modern javascript libraries available, I see lots of code that looks like this:

{ 'thing1': 'blah',
  'thing2': function() { ... }
}

This is an object-oriented approach, because it defines a map (or associative array) which in javascript is the same as an object. (or visa-versa, I forget). The important point it, it bundles data with functionality, which is a core concern of OO.

Failing that, I see a lot of code rolled up into objects for the purposes of modularization. jquery is heavily object-oriented, and you see lots of extentions like:

$.zoom = new function() {
  var privateValue = 0;
  this.visibleFunction = function() { ... };
}();

Which tacks an object on top of what jquery provides.

So in my experience, increasingly in the last few years as the javascript frameworks take on an OO aspect, is that OO practice in javascript is really almost the predominant methodology. And, from my experience, no serious javascript developers have given me grief about rolling my code up in objects -- if anything just the opposite.

(By comparison, that's not as true with in the Python world, where in Python the more natural attitude is dropping in functions at the module level.)

But the joy I find with doing OO in javascript is that you can leverage the nice scoping / visibility mechanisms it provides to really create clean object semantics in a way that can be really cumbersome in java. And then you get the nice benefits of a dynamic language, like late binding of functions to objects. Also, in Java, reflection is a great set of utilities, but I've always found it really cumbersome. But in javascript, reflection is just a natural aspect of the language.

So I love taking a break and doing OO in javascript occasionally, because it feels liberating after slogging through Java for a while.

I think the primary difference with OO in java and javacript is that javascript isn't as obnoxious about it.


问:
How to store/deal with data available to other classes
I'm writing a CAD program. Let's say I have in input class, this class reads various data from a text file and creates lots of lists/dictionaries and .... These data need to be accessed by other methods in other classes to be modified. Now here is how I have done it so far:

I have one static class: Building.cs When I create/or load a project this class holds all the data like list of columns, beams, points, etc. All of these are stored as private fields. I can access these using the class's public methods like GetColumns or GetPoints ...

Now I also have non-static classes. They contain 2-3 public methods. and do some stuff on various parts of the building.

```<code>public static class Building
{
    private static List<Column> columns;
    private static List<Beams> beams;
    private static List<Points> points;

    public static List<Column> GetColumns()
    {
        return Columns;
    }
}

public class ColumnsService()
{
    private List<Columns> columns;
    public GroupColumns(List<Columns> columns)
    {
        this.columns = columns;
    }

    public void Group()
    {
        // group columns
    }
}

var columns = Building.GetColumns();
var columnsService = new ColumnsService(columns);
columnsService.Group();
</code>```

I was wondering is this the way to go? How else can I store the data. The data needs to be accessible throughout the lifetime of the program to most of the classes.

答:
Because in your first question you professed a lack of knowledge about dependency injection, I will explain it in simple terms:

```<code>public class ThingThatNeedsABuilding
{
  private Building building;
  public ThingThatNeedsABuilding(Building building)
  {
    this.building = building;
  }
}

public class OtherThingThatNeedsABuilding
{
  private Building building;
  public OtherThingThatNeedsABuilding(Building building)
  {
    this.building = building;
  }
}

public class Application
{
  public static void Main(string[] args)
  {
    var building = new Building();
    var thing1 = new ThingThatNeedsABuilding(building);
    var thing2 = new OtherThingThatNeedsABuilding(building);
  }
}
</code>```

It's really that simple. The same instance of building is now shared between those two instances and nothing else. This is important. You do not want more stuff than needed to be available. This helps you avoid it.

If an object needs something, by all means give that something to it. Making it globally accessible to all objects makes it hard to tell if your program will even run, because an issue in the configuration of the dependency may cause a crash, but you can't tell at initialization that an object even has that dependency.

There are other important reasons to inject dependencies, but that one is very simple to understand and very hard to reject.

Body:
Because in your first question you professed a lack of knowledge about dependency injection, I will explain it in simple terms:

public class ThingThatNeedsABuilding
{
  private Building building;
  public ThingThatNeedsABuilding(Building building)
  {
    this.building = building;
  }
}

public class OtherThingThatNeedsABuilding
{
  private Building building;
  public OtherThingThatNeedsABuilding(Building building)
  {
    this.building = building;
  }
}

public class Application
{
  public static void Main(string[] args)
  {
    var building = new Building();
    var thing1 = new ThingThatNeedsABuilding(building);
    var thing2 = new OtherThingThatNeedsABuilding(building);
  }
}

It's really that simple. The same instance of building is now shared between those two instances and nothing else. This is important. You do not want more stuff than needed to be available. This helps you avoid it.

If an object needs something, by all means give that something to it. Making it globally accessible to all objects makes it hard to tell if your program will even run, because an issue in the configuration of the dependency may cause a crash, but you can't tell at initialization that an object even has that dependency.

There are other important reasons to inject dependencies, but that one is very simple to understand and very hard to reject.


问:
Algorithm to find the fewest number of values (from a set of values) where the sum equals a value (+ one more condition)
The simplest way to explain this is with an example.

You're given then number 19 and you have a set of numbers to choose from: 1, 2, 3, 4, 6

When choosing values from the list (which can be duplicates), the smallest number of values that adds up to 19 is 4:

6, 6, 6, 1

6, 6, 4, 3

Now the list of numbers I want to select is 6, 6, 4, 3 because when plotted on a graph, it draws a smoother gradient going from larger numbers to smaller numbers. 6, 6, 6, 1 is a sudden drop as it goes from the final 6 to the 1.

I may be wrong (and please correct me if so) but I think a better way of expressing that desire is to choose the list where the product of each of the numbers is the largest value.

6 * 6 * 6 * 1 = 216

6 * 6 * 4 * 3 = 432 (choose this list)

Another example is choosing 4, 3 instead of 6, 1 for a value of 7.

How could this be calculated using an algorithm (ideally non-brute-force!) for use with any total value and an arbitrary set of numbers to choose from?

答: This is the subset sum problem.  The subset sum problem is NP-complete.  In general the solution requires an exhaustive search, so the run time complexity is O(2^N) where N is the number of values in the set.  Alternatively, if the desired sum S is small, dynamic programming can be used with a run time complexity of O(NS).</p>


问:
Shared & exclusive locking
I need a "lock" that can be either shared or held exclusively, that will provide the following behavior for the following sequence of events:

Process A: Requests and is granted a shared lock

Process B: Requests an exclusive lock, and is blocked by A's ownership of the lock

Process C: Requests a shared lock, and is blocked because B is in line for an exclusive lock

I had thought that the Unix flock would provide these semantics, but then discovered that in step 3, Process C's request for a shared lock would succeed, and Process B's request for an exclusive lock would only be granted when there were no shared locks being held. This has the unfortunate (for me) effect that, the more concurrent processes there are requesting shared locks, the longer a process requesting an exclusive lock will have to wait for a window in which there are no outstanding shared locks being held.

The behavior I seek, in other words, is that, from the time that a process requests an exclusive lock, through the time that it is granted the lock and until the time that it releases the lock, all subsequent shared lock requests will be queued and blocked.

What type of lock has the properties that I need?

答: All you really need to build arbitrary locking structures is a mutex and a state variable.  In the context of the filesystem, a mutex consists of open("/tmp/foo.lock", O_WRONLY|O_TRUNC|O_CREAT|O_EXCL) to acquire (followed by a call to close()) and remove("/tmp/foo.lock") to release.  To block on the mutex, use inotify or (if you know it will not remain locked or suffer heavy lock contention) busy-wait on it.  The state variable can be a secondary file in /tmp (or elsewhere, if you can live with cleaning up stale files by hand after your program crashes).  The rest should be fairly straightforward to design.

Title: Shared & exclusive locking


问:
How to do "image shifting" faster?
I have a grayscale matrix (implemented as a simple array) to be shown on a display:

```<code>line 1: | a | b | c |
line 2: | d | e | f |
</code>```

a-f are grayscale values. This pattern has to be shifted by a variable percentage to the left or to the right.

What I currently do is an easy approach:

For example, shift 20% to left, so:

```<code>a_new = a * (1 - 0.2) + b * 0.2
b_new = b * (1 - 0.2) + c * 0.2
c_new = c * (1 - 0.2) + 0
</code>```

and so on. That is, I have a loop over all elements.

With only six elements I don't care, but in fact the matrix is much bigger.

Of course, the factor (1 - 0.2) can be reused, but I wonder if there is a smarter way of calculating the resulting (shifted) matrix?

答:
Do you know where the majority of time is being spent? There could be all sorts of ways to speed this up. There's not enough code to know for sure, unless you're specifically asking only about the math. </p>

<p>Computers are extremely good at math, so the problem is likely somewhere else than the actual computation of the data. For example, in one toolkit I know if, changing each pixel in an image one at a time is orders of magnitude slower than computing a whole row or the whole image in memory before saving the data to the image object. </p>

Title: How to do "image shifting" faster?

Body:
Do you know where the majority of time is being spent? There could be all sorts of ways to speed this up. There's not enough code to know for sure, unless you're specifically asking only about the math.

Computers are extremely good at math, so the problem is likely somewhere else than the actual computation of the data. For example, in one toolkit I know if, changing each pixel in an image one at a time is orders of magnitude slower than computing a whole row or the whole image in memory before saving the data to the image object.


答:
You need a read-write lock that will prioritize its write locks. A you can set a posix' rwlock to prefer the writer during its initialization.

Or as said in the linked wiki page:

Write-preferring RW locks avoid the problem of writer starvation by
  preventing any new readers from acquiring the lock if there is a
  writer queued and waiting for the lock. The writer will then acquire
  the lock as soon as all readers which were already holding the lock
  have completed. The downside is that write-preferring locks allows
  for less concurrency in the presence of writer threads, compared to
  read-preferring RW locks. Also the lock is less performant because
  each operation, taking or releasing the lock for either read or write,
  is more complex, internally requiring taking and releasing two mutexes
  instead of one. This variation is sometimes also known as
  "write-biased" readers-writer lock.

Title: Shared & exclusive locking

Body:
I need a "lock" that can be either shared or held exclusively, that will provide the following behavior for the following sequence of events:

Process A: Requests and is granted a shared lock

Process B: Requests an exclusive lock, and is blocked by A's ownership of the lock

Process C: Requests a shared lock, and is blocked because B is in line for an exclusive lock

I had thought that the Unix flock would provide these semantics, but then discovered that in step 3, Process C's request for a shared lock would succeed, and Process B's request for an exclusive lock would only be granted when there were no shared locks being held. This has the unfortunate (for me) effect that, the more concurrent processes there are requesting shared locks, the longer a process requesting an exclusive lock will have to wait for a window in which there are no outstanding shared locks being held.

The behavior I seek, in other words, is that, from the time that a process requests an exclusive lock, through the time that it is granted the lock and until the time that it releases the lock, all subsequent shared lock requests will be queued and blocked.

What type of lock has the properties that I need?


问:
How to define "or" logically
Recently, I came across a problem that required me to define the logical "OR" operator programmatically, but without using the operator itself.

What I came up with is this:

OR(arg1, arg2)
  if arg1 = True and arg2 = True
     return True

  else if arg1 = True and arg2 = False
     return True

  else if arg1 = False and arg2 = True
     return True

  else:
     return False
```

Is this logic correct, or did I miss something?

答:
This one remembers me the charasteristic functions:

```<code>or(a, b)
    return a + b - a*b
</code>```

This only applies to languages which can treat booleans as (1, 0). Does not apply to Smalltalk or Python since boolean is a class. In smalltalk they go even further (this will be written in sort of a pseudocode):

```<code>False::or(a)
    return a

True::or(a)
    return self
</code>```

And the dual methods exist for and:

```<code>False::and(a)
    return self

True::and(a)
    return a
</code>```

So the "logic" is perfectly valid in the OP statement, althought it is verbose. Beware, it is not bad. It is perfect if you need a function which acts like a mathematical operator based on, say, a kind of matrix. Others would implement an actual cube (like a Quine-McCluskey statement):

```<code>or = array[2][2] {
    {0, 1},
    {1, 1}
}
</code>```

And you will evaluate or[a][b]

So yes, every logic here is valid (except that one posted as using the in-language OR operator xDDDDDDDD).

But my favorite one is the DeMorgan's law: !(!a && !b)

Title: How to define "or" logically

Body:
This one remembers me the charasteristic functions:

or(a, b)
    return a + b - a*b

This only applies to languages which can treat booleans as (1, 0). Does not apply to Smalltalk or Python since boolean is a class. In smalltalk they go even further (this will be written in sort of a pseudocode):

False::or(a)
    return a

True::or(a)
    return self

And the dual methods exist for and:

False::and(a)
    return self

True::and(a)
    return a

So the "logic" is perfectly valid in the OP statement, althought it is verbose. Beware, it is not bad. It is perfect if you need a function which acts like a mathematical operator based on, say, a kind of matrix. Others would implement an actual cube (like a Quine-McCluskey statement):

or = array[2][2] {
    {0, 1},
    {1, 1}
}

And you will evaluate or[a][b]

So yes, every logic here is valid (except that one posted as using the in-language OR operator xDDDDDDDD).

But my favorite one is the DeMorgan's law: !(!a && !b)


问:
Why is Win32 API's WM_MOVE being called while dragging the window?
I have two callbacks for WM_MOVE and WM_MOVING. Oddly enough, WM_MOVE is being called while dragging the window. I expected WM_MOVE to be called only when I'm done dragging the window.

MSDN says that WM_MOVE is sent after a window is moved, and WM_MOVING is when the window is moving. The docs don't show any changes for Windows 7/VS2010, which is what I'm using.

WM_MOVE:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms632631%28v=vs.85%29.aspx

Has anyone come across this before? On drag, WM_MOVE is called, which it shouldn't be as far as I understand it.

```<code>    case WM_MOVE:
        {
            int x = LOWORD(lParam);
            int y = HIWORD(lParam);

            if (win.m_onMoveCallback)
                win.m_onMoveCallback(x, y);

            return 0;
        }
        break;

    case WM_MOVING:
        {
            int x = LOWORD(lParam);
            int y = HIWORD(lParam);

            if (win.m_onMovingCallback)
                win.m_onMovingCallback(x, y);

            return 0;
        }
        break;
</code>```

答:
When you define a move completion as occurring at a microscopic level, then it makes sense that the WM_MOVE is sent following a WM_MOVING message. Windows determines a move based on a change in the location of the mouse over a very small time period (hundreds or thousands of times a second). Since it can't know if you are done moving, you get both messages.

Way back when the time period was bigger (about 18 times a second), it was easier to tell if you were still moving.


答:
Back in the time of Windows 3.1, before Windows 95, you dragged the window, but it did not move. You were only moving an outline of the window, in form of semi-transparent frame. Only when you ended the drag, the window moved (jumping to final position; back then there were no graphics accelerators, so redraw of a window was expensive operation; resize worked similarly).

In Windows 95, one of the big things was, the window actually moved / resized while you were dragging it / its resizing border. Much nicer.

Well, those two WM_XXX messages are tied to dragging the movement frame (now invisible, but still present as a principle) vs. moving the actual window itself. I hope now it is clearer.

Title: Why is Win32 API's WM_MOVE being called while dragging the window?

Body:
I have two callbacks for WM_MOVE and WM_MOVING. Oddly enough, WM_MOVE is being called while dragging the window. I expected WM_MOVE to be called only when I'm done dragging the window.

MSDN says that WM_MOVE is sent after a window is moved, and WM_MOVING is when the window is moving. The docs don't show any changes for Windows 7/VS2010, which is what I'm using.

WM_MOVE:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms632631%28v=vs.85%29.aspx

Has anyone come across this before? On drag, WM_MOVE is called, which it shouldn't be as far as I understand it.

```<code>    case WM_MOVE:
        {
            int x = LOWORD(lParam);
            int y = HIWORD(lParam);

            if (win.m_onMoveCallback)
                win.m_onMoveCallback(x, y);

            return 0;
        }
        break;

    case WM_MOVING:
        {
            int x = LOWORD(lParam);
            int y = HIWORD(lParam);

            if (win.m_onMovingCallback)
                win.m_onMovingCallback(x, y);

            return 0;
        }
        break;
</code>```


答:
I found that WM_EXITSIZEMOVE is an alternative to WM_MOVE. It detects when I'm done dragging the window, but it passes no values in WPARAM and LPARAM, so you have to do some detection. Here's a rough example:

First, set these as static or global:

```<code>static int x = 0, y = 0;
static bool move = false, resize = false;
</code>```

Second, do your detection in WM_MOVING:

```<code>case WM_MOVING:
        {
            move = true;
            RECT *r = (LPRECT)lParam;

            x = r->left;
            y = r->top;

            if (win.m_onMovingCallback)
                win.m_onMovingCallback(x, y);

            return 0;
        }
        break;
</code>```

Third, check the move state in WM_EXITSIZEMOVE:

```<code>    case WM_EXITSIZEMOVE:
        {
            if (move)
            {
                if (win.m_onMoveCallback)
                    win.m_onMoveCallback(x, y);

                move = false;
            }

            return 0;
        }
        break;
</code>```

Body:
I found that WM_EXITSIZEMOVE is an alternative to WM_MOVE. It detects when I'm done dragging the window, but it passes no values in WPARAM and LPARAM, so you have to do some detection. Here's a rough example:

First, set these as static or global:

static int x = 0, y = 0;
static bool move = false, resize = false;

Second, do your detection in WM_MOVING:

case WM_MOVING:
        {
            move = true;
            RECT *r = (LPRECT)lParam;

            x = r->left;
            y = r->top;

            if (win.m_onMovingCallback)
                win.m_onMovingCallback(x, y);

            return 0;
        }
        break;

Third, check the move state in WM_EXITSIZEMOVE:

    case WM_EXITSIZEMOVE:
        {
            if (move)
            {
                if (win.m_onMoveCallback)
                    win.m_onMoveCallback(x, y);

                move = false;
            }

            return 0;
        }
        break;


问:
What are the valid uses of static classes?
I noticed that nearly every time I see programmers using static classes in object oriented languages such as C#, they are doing it wrong. The major problems are obviously the global state and the difficulty to swap implementations at runtime or with mocks/stubs during tests.

By curiosity, I've looked at a few of my projects selecting ones which are well tested and when I made an effort to actually think about architecture and design. The two usages of static classes I've found are:

The utility class—something I would rather avoid if I was writing this code today,

The application cache: using static class for that is plainly wrong. What if I want to replace it by MemoryCache or Redis?

Looking at .NET Framework, I don't see any example of a valid usage of static classes either. For example:

File static class makes it really painful to switch to alternatives. For example, what if one needs to switch to Isolated Storage or store files directly in memory or needs a provider which can support NTFS transactions or at least be able to deal with paths longer than 259 characters? Having a common interface and multiple implementations appears as easy as using File directly, with the benefit of not having to rewrite most of the code base when requirements change.

Console static class makes testing overly complicated. How should I ensure within a unit test that a method outputs a correct data to console? Should I modify the method to send the output to a writeable Stream which is injected at run time? Seems that a non-static console class would be as simple as a static one, just without all the drawbacks of a static class, once again.

Math static class is not a good candidate either; what if I need to switch to arbitrary precision arithmetic? Or to some newer, faster implementation? Or to a stub which will tell, during a unit test, that a given method of a Math class was indeed called during a test?

On Programmer.SE, I've read:

Don't Use “Static” in C#? Some answers are quite against static classes. Others assert that “Static methods are fine to use and have a rightful place in programming.”, but don't bake it with arguments.

When to use a Singleton and when to use a static class. Utility classes are mentioned, given that I mentioned above that utility classes are problematic.

Why and when should I make a class 'static'? What is the purpose of 'static' keyword on classes? The only valid usage which is mentioned is a container for extension methods. Great.

Aside extension methods, what are the valid uses of static classes, i.e. cases where Dependency Injection or singletons are either impossible or will result in a lower quality design and harder extensibility and maintenance?

答:
what if; what if; what if?

YAGNI

Seriously. If someone wants to use different implementations of File or Math or Console then they can go through the pain of abstracting that away. Have you seen/used Java?!? The Java standard libraries are a fine example of what happens when you abstract things for the sake of abstraction. Do I really need to go through 3 steps to build my Calendar object?

All that said, I'm not going to sit here and defend static classes strongly. Static state is solidly evil (even if I still use it occasionally to pool/cache things). Static functions can be evil due to concurrency and testability issues.

But pure static functions are not that terrible. There are elementary things that don't make sense to abstract out since there's no sane alternative for the operation. There are implementations of a strategy that can be static because they're already abstracted via the delegate/functor. And sometimes these functions don't have an instance class to be associated with, so static classes are fine to help organize them.

Also, extension methods are a practical use, even if they're not philosophically static classes.


答:
I'd recommended a great book JavaScript: The Good Parts by Douglas Crockford as a good guide. JavaScript AFAIK does not have (in practice) a single approach to object-orientation. Also as mentioned in previous answers it is probably more useful to learn functional paradigm mindset, unless you are going to design large frameworks like OpenLayers, jQuery or Node.js.

One more factor to decide is whether you will work alone or in a larger team. In the latter case it is probably wise to have common programming style.


答:
In short...

*Earmuffs* usually denote actions, e.g. "Wow, that's pretty funny! *Laugh*"

_Underscores_ usually denote underlining.  Used for emphasis.

I've also seen /slashes/ used to either suggest italicization or to refer to a 4chan board.

Title: Use of underscores and earmuffs

Body:
I have seen underscores and earmuffs (like _good_ or *good*) frequently used in conversations among programmers. All I know is that underscores are used to denote private variables or methods in languages like C, Python etc. and  earmuffs are the preferred way of indicating that something is a global variable.

Do these have any other punctuational meaning when they are used outside code, in mailing lists etc? Does the highlighted word have any hidden meaning or expression added to it? Or, are they just like air quotes?


问:
Initialize in Field, Constructor, or Argument
If I know what a field will be initialized to, should I initialize it in the field, constructor, or receive it as a parameter?  I am asking about best practices.  All three options effectively provide the same result.  I am not considering what is happening behind the scenes, because I think it would be insignificant.

This is a general question for curiosity's sake.  I do not have a specific problem with this at the moment, although I have in the past.  I will use an ArrayList in the following example, because it is a case where you know what it will be initialized to, and if you end up actually wanting to initialize it to another existing ArrayList, there is no harm by preinitializing it.  Or is there?

For example, where should I initialize an ArrayList?

Initialized in Field:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList = new ArrayList<String>();

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

Initialized in Constructor:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList;

    public Initializer() {
        arrayList = new ArrayList<String>();
    }

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

Initialized as Parameter:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList;

    public Initializer(ArrayList arrayList) {
        this.arrayList = arrayList;
    }

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

答:
Initializing as a parameter breaks encapsulation in that the caller can then do with the passed in list what it wants (clear at odd times).

```<code>ArrayList<String> list = new ArrayList<String>();
Initializer init = new Initializer(list);
//do various thing
list.clear();
//now the list in init is also empty while init may still expect it to be filled
</code>```

For the other 2 options it depends on what you want to initialize with:

For example a class that needs a thread-safe queue could use one of several implementations (linked list, circular buffer, etc.) The constructor can decide which one to use based on parameters or which constructor was called and even pass parameters to the constructor of the object.

```<code>public Initializer(boolean useLinked) {
    if(useLinked){
        this.arrayList = new LinkedList<String>();
    else
        this.arrayList = new ArrayList<String>();
}
</code>```

If the implementation will always be the same then just initialize in the field or initializer block and mark it final.

```<code>public class Initializer {
    private final ArrayList<String> arrayList = new ArrayList<String>();
}
</code>```

This prevents other methods from overwriting it accidentally.

Body:
Initializing as a parameter breaks encapsulation in that the caller can then do with the passed in list what it wants (clear at odd times).

ArrayList<String> list = new ArrayList<String>();
Initializer init = new Initializer(list);
//do various thing
list.clear();
//now the list in init is also empty while init may still expect it to be filled

For the other 2 options it depends on what you want to initialize with:

For example a class that needs a thread-safe queue could use one of several implementations (linked list, circular buffer, etc.) The constructor can decide which one to use based on parameters or which constructor was called and even pass parameters to the constructor of the object.

public Initializer(boolean useLinked) {
    if(useLinked){
        this.arrayList = new LinkedList<String>();
    else
        this.arrayList = new ArrayList<String>();
}

If the implementation will always be the same then just initialize in the field or initializer block and mark it final.

public class Initializer {
    private final ArrayList<String> arrayList = new ArrayList<String>();
}

This prevents other methods from overwriting it accidentally.


答:
If I know what a field will be initialized to

If you know that, then chances are very high that either your class won't be very reusable or you make the mistake of assuming that you will know how your class will be used now and in future - while actually missing some possible usecases.
I can explain this in more detail if you provide more information about what the class is to do semantically.

Usually it is best to go with Initialization by parameter (assuming that the argument will change and not always be an empty list, as stated above). This design is also called depedency injection. It has quite some advantages. E.g. one of them is a much better testability.

You should also use Interfaces instead of accepting only concrete types. So you should go for:

```<code>//(...)
private List<String> someList ;

    public Initializer(List someList ) {
        this.someList = someList ;
    }
</code>```

And someone can then create a new Initializer by new Initializer(new ArrayList()) OR he can do something like new Initializer(new CopyOnWriteArrayList()) as this will be much more performant in some cases. The user of the class can decide best, what kind of list he wants to use. And you, as the class creater, don't care if it is an ArrayList or CopyOnWriteArrayList right? You just want to have a List-thing where you can add items in the same way.

Title: Initialize in Field, Constructor, or Argument

Body:
If I know what a field will be initialized to, should I initialize it in the field, constructor, or receive it as a parameter?  I am asking about best practices.  All three options effectively provide the same result.  I am not considering what is happening behind the scenes, because I think it would be insignificant.

This is a general question for curiosity's sake.  I do not have a specific problem with this at the moment, although I have in the past.  I will use an ArrayList in the following example, because it is a case where you know what it will be initialized to, and if you end up actually wanting to initialize it to another existing ArrayList, there is no harm by preinitializing it.  Or is there?

For example, where should I initialize an ArrayList?

Initialized in Field:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList = new ArrayList<String>();

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

Initialized in Constructor:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList;

    public Initializer() {
        arrayList = new ArrayList<String>();
    }

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

Initialized as Parameter:

```<code>import java.util.ArrayList;

public class Initializer {
    private ArrayList<String> arrayList;

    public Initializer(ArrayList arrayList) {
        this.arrayList = arrayList;
    }

    public void addString(String string) {
        arrayList.add(string);
    }
}
</code>```

Body:
If I know what a field will be initialized to

If you know that, then chances are very high that either your class won't be very reusable or you make the mistake of assuming that you will know how your class will be used now and in future - while actually missing some possible usecases.
I can explain this in more detail if you provide more information about what the class is to do semantically.

Usually it is best to go with Initialization by parameter (assuming that the argument will change and not always be an empty list, as stated above). This design is also called depedency injection. It has quite some advantages. E.g. one of them is a much better testability.

You should also use Interfaces instead of accepting only concrete types. So you should go for:

//(...)
private List<String> someList ;

    public Initializer(List someList ) {
        this.someList = someList ;
    }

And someone can then create a new Initializer by new Initializer(new ArrayList()) OR he can do something like new Initializer(new CopyOnWriteArrayList()) as this will be much more performant in some cases. The user of the class can decide best, what kind of list he wants to use. And you, as the class creater, don't care if it is an ArrayList or CopyOnWriteArrayList right? You just want to have a List-thing where you can add items in the same way.


答:
In a word Simplicity.

When you decouple too much, you get the hello world from hell.

```<code>void main(String[] args) {
    TextOutputFactory outputFactory = new TextOutputFactory();
    OutputStream stream = outputFactory.CreateStdOutputStream();

    Encoding encoding = new EncodingFactory.CreateUtf8Encoding();
    stream.Encoding = encoding;

    SystemConstant constant = new SystemConstant();
    stream.LineEnding = constant.PlatformLineEnding;

    stream.FlushOnEachLine = true;

    String greeting = new FixedSizeInMemoryString(); // We may have to switch to a file based string later!"
    greeting.SetContent("Hello world");
    greeting.Initialize();

    stream.SendContent(greeting);
    stream.EndCurrentLine();

    ProcessCompletion completion = new ProcessCompletion();
    completion.Status = constant.SuccessStatus;
    completion.ExitProgram();
}
</code>```

But hey, at least there is no XML configuration, which is a nice touch.

static class allow to optimize for the quick and common case. Most the point you mentionned can be very easily solved by introducing your own abstraction over them, or using stuff like Console.Out.

Title: What are the valid uses of static classes?

Body:
I noticed that nearly every time I see programmers using static classes in object oriented languages such as C#, they are doing it wrong. The major problems are obviously the global state and the difficulty to swap implementations at runtime or with mocks/stubs during tests.

By curiosity, I've looked at a few of my projects selecting ones which are well tested and when I made an effort to actually think about architecture and design. The two usages of static classes I've found are:

The utility class—something I would rather avoid if I was writing this code today,

The application cache: using static class for that is plainly wrong. What if I want to replace it by MemoryCache or Redis?

Looking at .NET Framework, I don't see any example of a valid usage of static classes either. For example:

File static class makes it really painful to switch to alternatives. For example, what if one needs to switch to Isolated Storage or store files directly in memory or needs a provider which can support NTFS transactions or at least be able to deal with paths longer than 259 characters? Having a common interface and multiple implementations appears as easy as using File directly, with the benefit of not having to rewrite most of the code base when requirements change.

Console static class makes testing overly complicated. How should I ensure within a unit test that a method outputs a correct data to console? Should I modify the method to send the output to a writeable Stream which is injected at run time? Seems that a non-static console class would be as simple as a static one, just without all the drawbacks of a static class, once again.

Math static class is not a good candidate either; what if I need to switch to arbitrary precision arithmetic? Or to some newer, faster implementation? Or to a stub which will tell, during a unit test, that a given method of a Math class was indeed called during a test?

On Programmer.SE, I've read:

Don't Use “Static” in C#? Some answers are quite against static classes. Others assert that “Static methods are fine to use and have a rightful place in programming.”, but don't bake it with arguments.

When to use a Singleton and when to use a static class. Utility classes are mentioned, given that I mentioned above that utility classes are problematic.

Why and when should I make a class 'static'? What is the purpose of 'static' keyword on classes? The only valid usage which is mentioned is a container for extension methods. Great.

Aside extension methods, what are the valid uses of static classes, i.e. cases where Dependency Injection or singletons are either impossible or will result in a lower quality design and harder extensibility and maintenance?

Body:
In a word Simplicity.

When you decouple too much, you get the hello world from hell.

void main(String[] args) {
    TextOutputFactory outputFactory = new TextOutputFactory();
    OutputStream stream = outputFactory.CreateStdOutputStream();

    Encoding encoding = new EncodingFactory.CreateUtf8Encoding();
    stream.Encoding = encoding;

    SystemConstant constant = new SystemConstant();
    stream.LineEnding = constant.PlatformLineEnding;

    stream.FlushOnEachLine = true;

    String greeting = new FixedSizeInMemoryString(); // We may have to switch to a file based string later!"
    greeting.SetContent("Hello world");
    greeting.Initialize();

    stream.SendContent(greeting);
    stream.EndCurrentLine();

    ProcessCompletion completion = new ProcessCompletion();
    completion.Status = constant.SuccessStatus;
    completion.ExitProgram();
}

But hey, at least there is no XML configuration, which is a nice touch.

static class allow to optimize for the quick and common case. Most the point you mentionned can be very easily solved by introducing your own abstraction over them, or using stuff like Console.Out.


问:
How can this allocation of bi-dimensional arrays work?
I was seeing this post on StackOverflow and saw a new way (at least for me) to define a two dimensional array of 5x5, it works well, but I feel I don't understand what is going on in the background.

The code is.

```<code>double (*matrix)[5] = malloc(5 * sizeof *matrix);
</code>```

How can this define a two dimensional array of 5x5?, before I thought that n mallocs were necessary to produce a n-dimensional array, but apparently I was wrong.

答:
matrix is a pointer to a 5-element array of double; this means that the type of the expression *matrix is "5-element array of double"; sizeof *matrix will return the number of bytes required by such an object.

So we're telling malloc to set aside enough memory for 5 5-element arrays of double, and assign the resulting pointer to matrix.

Because of how pointer arithmetic works, matrix points to the first 5-element array, matrix + 1 points to the second 5-element array, matrix + 2 points to the third 5-element array, etc.

Since a[i] is equivalent to *(a + i), the expression matrix[i] gives us the i'th 5-element array, so matrix[i][j] gives us the j'th element of the i'th 5-element array.


问:
Handle server-side/client-side when for showing missing information as placeholders
I have a database table which stores Incident details. Each Incident can have an image saved for it. There are some Incidents which will not have an image for it.

I have an API which pulls the image for a given Incident. I have an issue with handling the API in case an Incident doesn't have the image.

What is the best approach to handle this (and why :) )?:

Telling the client in advance which Incident has the image, and which don't. So the client calling the API should only call it when there is an image for the Incident.

Sending a predefined error message as the result.

Sending a generic placeholder like image to the client when an image is not available for the Incident.

答:
It depends on the technology you use. Normally in this case you would return a Type that describes the possibility of an image. In some languages these types are built in (Option in Scala, Maybe in Haskell). In other ones you can write them yourselves or use a library that includes these types. For javascript (which may be what you can use, I guess), you can find a good explaination and implementation here.

In addition you should - as you mentioned - have a method to ask, if an image exists.


问:
Best (worst) logically impossible task assigned to you?
What was a project or spec that got put on your desk that could not possibly be done? How did you explain the dilemma to the "requester"? More importantly, did they understand after you explained the fundamental issue? </p>

答:
My friend asked me to make him a website.

With C++.


问:
PHP package management with/without Composer
I noticed that many (large) PHP applications use Composer for managing the dependencies. I'm currently trying to figure out if I should switch to Composer (at the moment I'm simply keeping my dependencies in a folder called vendor/ - without being able to automatically update them).

I prefer this way because:

I don't rely on 3rd party websites/repositories to get the source code (I find this as being a security concern as well); and, also, what happens if one of the repositories goes down for any reason? I will no longer be able to update/install my application?

I don't need Composer installed (on the local machine/production machine); this is just a minor issue;

It's faster; obviously, the updates can be pushed faster to production if they come from a single source - it's enough to just update my bundle (that contains all vendor packages) and then simply push/pull the changes to/from production;

I'm using many components that are not aware of Composer (standalone libraries, GitHub repositories that are lacking the composer.json file, many JS/CSS assets, even isolated single PHP classes that I prefer to think as 3rd party libraries etc); if I were to use Composer I would have to use a tool like Satis to manage these (and things get more complicated without a real benefit);

I have my application deployed to several domains (around 25) and it's easier to maintain it this way (a single bundle where all changes get [automatically] pushed and later on the bundle itself can be pushed to production);

There are many changes and updates (a few commits each day to the main application package) and I find it easier to simply push these commits to production rather than setting up tags for each commit so that Composer can see the difference and update to the new version;

On the other hand, there are some problems with my approach:

First of all, I'm including all the vendor packages in my codebase/build; I know this is not a good practice and I understand the arguments behind this (I'd prefer to avoid this);

It may be a good thing that I'm able to trace all source code changes to my application in a single repository (I use git). Obviously, all vendor changes will be reflected in the bundle repository. But on the other hand there's some redundancy - I'm also working myself on some of the dependent vendor packages so it doesn't make much sense to COMMIT the changes there and add the relevant message to the COMMIT and then the same changes to be commited on the bundle package, as well;

More bundles: I have a few applications that are extensions of the main application - they provide their own business logic, additional controllers etc; For these I need to create separate bundles that will again contain all the vendor packages - so, whenever a vendor package gets updates I need to propagate these changes to the additional bundles, too;

It's not that straightforward to work in a team: how should the process go? Each member should update the core application package then update the bundle itself? Then the other members will download both the bundle and the core application? Again, I find it somehow redundant;

My approach worked so far, but I think that I might be missing something in this whole developing/deployment process. I'm trying to stay focused on developing rather than managing dependencies/deploying so I'd prefer that the deployment process is as smooth and transparent as possible. Also, with this approach the update on each server takes about 30 seconds, with composer it would take a lot more (as the update in Composer needs to connect to all 3rd party repositories).

So, as a summary:

I think there's a redundancy in my current way of working;

I'd like to know if there are better ways to handle this (simple deployment process to all my 25 domains - about 20 of them are sharing the same codebase; no source code redundancy; team work friendly);

There are many updates to the source code each day so the update/deployment process should not be a hassle;

I really like the modular approach for developing: having an application that *requires* other modules; if I need the same application just slightly modified I add an additional module that changes some of the existing logic and that's it. I'd like to keep it this way without having to build bundles made up of modules and then install the bundle.

Any ideas for improvement?

Edit:
I found two articles related to this topic that are quite interesting: http://www.redotheweb.com/2013/09/12/should-you-commit-dependencies.html and http://addyosmani.com/blog/checking-in-front-end-dependencies/. I believe that what they are mainly saying is that "it depends". So, I assume that, at least for me it would be better to checkin my dependencies.

答:
It seems like your main issue with composer is speed. Since so much code is shared between projects, you could have symlinks for shared vendor subfolders so you only need to update in one of the projects and it will update all of them. You can then deploy to all the servers using rsync.

I like to have a file with the build version in each project so I can use it in my deployment scripts to update the css/js minified file names with it for cache invalidation. You could even set up a temporary file with a global cache of all the projects' build versions so that you can compare when deploying and only push those that were updated.