问:
Golang Testing Process
I am new to golang and RunC and now doing some research on it as a part of my intership. What kind of contents do the ' _test.go ' functions check during testing a program or a container with Golang (Or RunC in case of Docker containers)? Can someone please explain me the testing process?</p>

答:
Personally, for each package I have one _test.go file with several test functions in (they must take the form func TestX(t *testing.T)). Each of these test functions runs through all of my regular functions in the package, passing in acceptable data, extreme data (data on the boundaries of acceptable), and invalid data (passing nil in, etc.).

I then test the return values and use t.Log() to print useful output. If your code doesn't behave how it is supposed to, you can use t.Fail() to cause the test to fail, or t.Errorf()to log an error message and then fail.

One useful technique to help keep the test files smaller is to have a struct type called testData or something like that, with a field for:

An instance of the data you will pass in.

Each expected return value from various functions.

and a function that sets up and returns a slice of initialised test data.

You then only need to call td := setupTestData(), and loop through each member, passing in and testing the same fields.

Go has useful tools for testing including the very useful cover tool. Open terminal or command prompt in the package directory and do go tool cover -html="cover.out" to see a visual representation of how much of your code is covered by your test.

Title: Golang Testing Process


问:
How do I refactor a loop that does aggregation while doing it's regular job ?
What am I suppose to do

I am working on an API, where I get a list of objects from the output of an SQL query using an ORM. Now this API call needs to send following information down to the client.

Output consists of a list and a dictionary.
List contains dictionaries holding statistical data per day. One dictionary for each day.

Dictionary contains aggregate of all the statistical data in the same time period along with a status indicating how much did the value go down or up depending upon aggregate of previous N days prior to current N days requested.

Current Implementation

So, I got a requested of data for last N days. I fetched last 2N days of data from the database.
Looped over the result and while I prepared data according to the API if the data was for the last N days.
While the loop was in, I kept creating aggregated data for previous N days and current N days for calculating the change.

Once the loop got over, I created the dictionary having current aggregate and the change per stat.

What's the issue?

The function is just too big.
I can separate out the the process into two functions but then I will have to loop over the data again. How do I ensure that the code looks sane and refactored without affecting the performance.

答: When you have a big loop that also mutates state, the way to refactor it is to create a method object. It's an object from a new class whose fields correspond to all the messy local state that you're now maintaining while iterating through the loop.

ContentLicense: CC BY-SA 3.0


问:
Are there any arguments that can make a contractor reconsider working on fixed price?
I've been working for a contractor who brings in some good projects, but they are all fixed-price and often fixed-time.

As a result he always has me making a quote over loose requirements, which never fails to bring a lot of tension due to feature creep.

He claims he'd never get a contract if he couldn't agree on a price with his clients first, but as far as I'm concerned I don't wanna go through another project under these terms.

Is there any argument I could make to have him pay me by the hour, or should I just suck less at estimating ?

答: This is one of the major reasons for the emergence of the agile movement. Using the Scrum + XP combination is one way to present a fair alternative to this relatively common - fixed time or fixed price with variable requirements situation.</p>

ContentLicense: CC BY-SA 2.5


问:
Is recursive-descent parsing a panacea for DoS threats posed by 'Evil' regexes? Or does evilness stem from the grammar?
ReDos attacks exploit characteristics of some (otherwise useful) regular expressions ... essentially causing an explosion of possible paths through the graph defined by the NFA.

So does using a recursive-descent parser (such as ANTLR) necessarily avoid the problem?

答:
Each regular expression defines a regular language. A regular expression can be translated to a non-deterministic finite automaton (NFA) that recognizes its language. Each NFA can be translated to an equivalent deterministic finite automaton (DFA) that recognizes the same language in linear time. Each DFA can be translated to a regular grammar (see e.g. here) that generates the same regular language. A regular grammar is a particular type of context-free grammar, which implies that all regular languages are also context free.

So, if the language you want to recognize is regular, you could use a recursive-descent parser to parse it, but you would not gain anything in terms of time-complexity.

As far as I understand, the attacks you mention in your question are related to

Naive implementations of the parsing algorithm that are not O(n).

Extensions to regular expressions such as back-references, which make the resulting language non-regular and force one to use some ad-hoc algorithm that has an exponential worse-case complexity.

Case 1 is easily solved by using a non-naive implementation that runs in linear time: no need for recursive-descent parsers.

Case 2 depends on the extensions you want to support. If you want to use recursive descent parser your language must be context-free, so you must first check if languages defined by regular expressions with extensions like back references are context free.

These languages are not context free (see the answer to the parallel question on the Computer Science site for very good background on the topic). My intuition as to why they are not context free is that a context free grammar would need to be able to generate strings with arbitrarily long disjoint matching substrings. You can probably use the pumping lemma for context free languages to show that a context free grammar cannot do this.

Bottom line: recursive descent parsers are no use for this kind of extensions.

A pragmatic solution is to first run the fast DFA algorithm and only revert to the potentially exponential algorithm if a back-reference is encountered in the regular expression. This is mentioned here in Section Implementations and running times. Unfortunately there is very little information about this approach in the wikipedia page. If I find more details somewhere else I will add them to this answer.


问:
Azure Webjob Creation and workflow management
I am attempting to create a long running job to be used in Azure.  I would like to be able to run multiple instances of this job if I needed it.

I was thinking of seperating the primary functionality into two different webjobs. With one job called "the brain". while the other job(s) would be called the worker.

Brain would be in charge of actually knowing what jobs needs to get run and put them into the Azure storage ques that would be listened to by worker jobs.  I envision a few different queues and a few different workers handling different tasks listening to queues.

The job list is stored in a regular database with values about when it should be run next and the type of job it is.  Brain will periodically look to the database and get the next job that needs to get run.  Pass it off to a queue for a worker webjob to handle and either write back to the database if the job is regularly run or just update the status in the database.

Question: Given this setup how would you actually handle getting the next job to be run (what is next is based on a date and time) and setting it's status to 'processing' (so it is not picked twice) in such a way that there is not chance for it to be picked up twice (two instances of Brain running)? I am using EF6 in this case.  Is there a way to get the record and update it before anyone else could do that on the database?

A shorter version of the question is How can I pull and update a record and be sure it is not modified while the process is happening using EF6?

Should I be using a different type of job management?  Is there a good tutorial available on a better way to do this?

答:
A shorter version of the question is How can I pull and update a record and be sure it is not modified while the process is happening using EF6?

I usually use the following. Suppose you give a name, or ID, to each "Brain". You start a transaction, read if a job is already taken ("owner" column = NULL), write your own id in the "owner" column. You commit the transaction, then you read it back.
Now the column can be

NULL

your ID

another ID

If it is NULL, you try again (maybe another Brain took it, failed, and you reset the owner to NULL in its finally block?), if it is your ID you "won" and  you can proceed, if it is another, you let it process the entry and you look for another one or go to sleep.
This is quite flexible; it usually works with plain SQL (and with ORMs on top of it, like EF6) but also with other systems, like redis.

Should I be using a different type of job management? Is there a good tutorial available on a better way to do this?

A different "way" which is usually used with massive distributed system is to let two workers process the same item; if it happens, and the process is idempotent (processing it once or twice does not change the final result), you just let it happen, on the basis that synchronization is more expensive than the actual processing. This model works well with Azure queues.
See for example this article

In general, a very good guide for Azure is on MSDN. I also found very interesting this article. It is old, but it is very clear and helped me in the past.


问:
Azure ServiceBus Queue: Consumer Application
I am running a .net MVC4 web application. I want to be able to hit an action on a controller, create an email and send it to my service bus queue to be processed (sent).

What are my options as far as creating a consumer for my queue?

I've considered creating another project that runs when the web app is started that continually checks the queue state and processes messages as they come in.

I was wondering if there were any other solutions to processing messages.

答:
A second project that checks the queue and process messages is what I would do. Background processing belongs to background workers.

Just remember that you don't have to continuously monitor the queue, if you are using a Service Bus queue: plain Azure queues are non-blocking, but Service Bus queues offer blocking receive operations which block the calling thread until a message arrives on a queue or a specified timeout period has elapsed.

So you just have to call your receive function in a loop, and it will stay there and wait for the next call.
You can read the initial paragraphs of this article: it is about performances, which is not your case, but the initial overview is good for grasping the fundamentals.


问:
Best practices for testing deep objects?
I'm writing in javascript with 'should' library, but this doesn't matter.

I should write test for method, which returns a quite big object with a lot of fields. How I should write my expected result?

I just wrote the same big object in expected, but it about 100 lines of code and looks not good.

Any suggestion? Or should I refactor my method in some way? At this moment method calls to 4 another methods, and collect results in some way to one object.

答:
Refactor the 4 other methods into a separate class (could be one class per method, or one class with 4 methods). Chances are the class that is creating the one big object has too many responsibilities and you should split those up.

Test the individual methods to confirm that they work. The result of those calls will be a lot smaller than the big object.

Test the big method by asserting that the big method calls the 4 extracted methods and correctly assigns the values.


问:
What happens to lock's state and methods when a thread acquires a lock?
Assume I have following class

```<code>class Student{
   void method1(){
     // Do Something
   }
   void method2(){
     // Do Something
   }
</code>```

}

And the object of this class is used as monitor in synchronization in some piece of the code as

```<code>class ProcessingThread implements Runnable {
   void someMethod(){
     synchronized(student){    // student object is used as lock. Thread T1 acquires a lock on student.
       // critical section of code
     }
   }
}
</code>```

When a thread T1 acquires a lock on the student object, and enters into the critical section of the code, at the same time can some another thread T2 executes a method on the student object i.e.

```<code>class College{
   void assignTopic(){
      student.method1();   // Thread T2 working on student object when
                           // Thread T1 has acquired a lock on student
   }
}
</code>```

I want to know that what happens to the methods / variables of the locked object when it is acquired by a thread?

答:
What happens when a thread acquires a lock?

No other thread can acquire that lock until it's released.

Note that this is all that happens. Only methods called via other threads are affected. Only other threads that compete for the same lock are affected at all.

This means that the holding thread can do whatever it likes, just as before the lock. Other threads cannot acquire that lock, but they can do everything else. That includes mutating the state of student, unless all of student's mutating methods and fields are protected by the same lock. If there is a mutating method that isn't synchronized, or synchronizes via a different lock, the "locked" object can still change unpredictably.

To elaborate: as ratchet freak's answer says, obtaining a lock on an object doesn't protect it from changing unless all possible users pledge to obtain the same lock before changing it. This is hard to enforce because most languages don't have built-in support for questions of mutability, only for locking. It can also lead to problems because obtaining multiple locks can easily cause deadlock; you need a clear policy in which order to obtain locks, and everyone has to follow them all the time. Again, this is easy to get wrong, and few languages give you any support enforcing it.

This is why reasoning about thread-safety is hard: a single chink in the armor completely negates all consistency assertions you wanted to make, and thread-safety is not preserved under composition (two components that were proven to be thread-safe might not be thread-safe when used together, for instance because they use different locks and not the same one).


答:
To ensure safety both threads must go and acquire the lock before messing with the data of student.

If one thread doesn't do that then it's like neither thread acquired the lock.

Part of acquiring the lock is waiting until no other thread has the lock.

Title: What happens to lock's state and methods when a thread acquires a lock?

Body:
Assume I have following class

```<code>class Student{
   void method1(){
     // Do Something
   }
   void method2(){
     // Do Something
   }
</code>```

}

And the object of this class is used as monitor in synchronization in some piece of the code as

```<code>class ProcessingThread implements Runnable {
   void someMethod(){
     synchronized(student){    // student object is used as lock. Thread T1 acquires a lock on student.
       // critical section of code
     }
   }
}
</code>```

When a thread T1 acquires a lock on the student object, and enters into the critical section of the code, at the same time can some another thread T2 executes a method on the student object i.e.

```<code>class College{
   void assignTopic(){
      student.method1();   // Thread T2 working on student object when
                           // Thread T1 has acquired a lock on student
   }
}
</code>```

I want to know that what happens to the methods / variables of the locked object when it is acquired by a thread?


问:
ASP.NET synchronous commands handlers
A question about DDD, for validating a Proof Of Concept.

Let's say we have a webpage that triggers a Domain Event. For instance, updating the status of a client after an interaction occurred on that page. From a user perspective, we want the event to be handled immediately, and the page to refreshed in the server response, because the status has a lot of weight on the information that is displayed on the page.

Domain Events are meant to lead to "Eventual Consistency". How do you handle synchronous events in an ASP.NET application ?

Thanks

答:
Got a response. I was looking for some kind of "Request/Respond" pattern implementation, that I found.

[Edit for Svick] For instance, the web application should emit a "DoSomething" command and then wait for a "SomethingDone" notification (or a combo of notifications), introducing a timeout. For sagas and more complex workflows, the point is to know what significant (and minimal) notification(s) should be handled/aggregated by the web application to let the user know his request has been handled and is on a good way to completion.

Title: ASP.NET synchronous commands handlers


问:
Manual Dependency Injection Examples Still Tightly Coupled
I've been using AutoFac for some time now to take care of the dependency injection we're using in a web project. All is well, but I now want to demonstrate DI outside of an AutoFac context for reasons irrelevant to this question.

In doing so, I'm having to read up on 'manual' methods of achieving this pattern. All of the examples in tutorials work (obviously), but the issue of tightly coupled objects still exist.

Take this for example: http://www.devx.com/dotnet/Article/34066/0/page/2

The constructor of the dependent class takes an interface:

```<code>public class BusinessFacade {
    private IBusinessLogic businessLogic;
    public BusinessFacade(IBusinessLogic businessLogic) {
        this.businessLogic = businessLogic;
    }
}
</code>```

But then the instantiation of BusinessFacade still requires a concrete implementation to be passed, which means an implementation needs to be instantiated:

```<code>IBusinessLogic productBL = new ProductBL();
BusinessFacade businessFacade = new BusinessFacade(productBL);
</code>```

Question

Now I know, somewhere at some point, something (whether it is a container or manually written code) will have to decide on the implementation to choose, but how can I come to terms with the benefit of moving such decision outside of the dependent class? Of course the BusinessFacade class is no longer tightly coupled, but we're still very explicit about which implementation we choose - so what's been gained here? Is it purely for the fact we can mock out the dependencies?

答:
The whole purpose of DI is to decouple concrete classes from each other. Your example is a good example of DI: BusinessFacade only knows about the IBusinessLogic interface and is passed an implementation of that interface. Other code then decides the instance to pass in. That "other code" might be a container if you are using IoC (such as with AutoFac), it might be hand-rolled code if you are using "poor man's DI" (as in your example), or it might be a mock when unit testing.

Declaring an instance of an interface and then passing it in to another class does not create tight coupling; it remains loosely coupled as the two are only bound together at runtime. For the live application, you want to perform all such instantiation and injection in one place (eg, the container). Unit tests will instantiate and inject in many places as they are deliberately only testing small parts of the code.

There are clear benefits to this approach. Your code is not only easier to test (as mocks can be injected in place of complex business logic) but it makes the code more maintainable too, for it's easier to understand the implications of a change when that change isn't tightly coupled to numerous disparate parts of a system.


答:
Are the four methods - and the data returned - dependent upon, and related to, each other?

If there is no real dependency, then expose those methods as public ones and allow each to return a smaller set of data, which can be independently tested.

If there is a strong dependency, consider making your large object hierarchical. So it may have four fields, each of which is an object with its own collection of fields. Again you can then either split the testing out into four tests and test each sub-object in turn, or you could have four separate should invocations (assuming should.js allows multiple asserts in one test; it's not a testing framework I'm familiar with).

Title: Best practices for testing deep objects?

Body:
I'm writing in javascript with 'should' library, but this doesn't matter.

I should write test for method, which returns a quite big object with a lot of fields. How I should write my expected result?

I just wrote the same big object in expected, but it about 100 lines of code and looks not good.

Any suggestion? Or should I refactor my method in some way? At this moment method calls to 4 another methods, and collect results in some way to one object.


问:
Class design, responsibility granulation, efficiency and performance
I am in a design dilemma. I have a set of data that can be interpreted in numerous ways, but I cannot really decide how finely grained should it be. To illustrate it with some simple code:

class Base {
    Datamembers data;
    virtual void doStuff() = 0;
};

class T1 : public Base {
    void doStuff() {
        // do T1 stuff
    }
};

class T2 : public Base {
    void doStuff() {
        // do T2 stuff
    }
};

class ...

// ---------------------------

class Master {
    enum Role {
        T1, T2 ...
    };

    Role r;
    Datamembers data;

    void doStuff() {
        switch (r) {
        case T1:
            // do T1 stuff
            return;
        case T2:
            // do T2 stuff
            return;
        case ...
        }
    }
};
```

In reality the number of responsibilities is much higher than 2, currently still in the two digit range, but may grow further. Also, it is not one, but several doStuff()s at play.

In the first case, the data is encapsulated in a base class, and doStuff() is a pure virtual, which each derived class implements. Each responsibility is encapsulated in its own, dedicated type.

In the second case doStuff() is not a virtual function, and the responsibility is stored as an extra member in a Role, and the method switches the role and executes the appropriate code.

The two designs achieve the same goal in a different way, but what are the consequences?

Code maintenance:

The first approach seems cleaner, but even though usually clean implies maintainable, I feel it is kind of fragmented, which may end up reducing the ease of maintenance.

In the second approach all the code for the different responsibilities is crammed in the same place, which may sound dirty, but in many cases having it all in the same place may actually be easier, since all you have to do is scroll rather than jump around different files. But then again... it is still a big wall of code, not obvious in the example code above, but in practice it will be 4-5 digit lines of code. Although there is an easy remedy for that, just implement a function for each case that takes this as a parameter to access the data and force inlining to eliminate the overhead of extra function calls (even if not virtual).

Performance:

In my case, I deal with massive trees of objects, where doStuff() will be executed in tight loops, traversing that tree.

In approach one, the method is virtual, this means I get penalty for the indirection. Also, the code is spread across multiple types, which may end up fragmenting it in memory, reducing cache efficiency.

In the second approach the method is not virtual, and all code is essentially in a single function, which means it will be tighter in memory, and potentially more cache friendly. There is the switching penalty vs the indirection and fragmentation of the first approach.

Memory usage:

Although it is missing from the example code above, the hierarchy does have a constructor with plenty of arguments. Meaning that in the first approach each derived class will have its own constructor with the same parameters, redirecting those to the base class constructor. This, aside from tedious to implement for each type, also adds some code bloat. And there is the overhead of the v-tables. Both the extra constructors and v-tables are avoided in the second approach.

Obviously, I've made some considerations, and had this been a smaller undertaking, I'd do both and compare. But it is actually a lot of work and it is not really possible, so I'd appreciate some experienced opinions - are my considerations valid, am I missing something, is there another, better way to do it?

As it seems the scale of the problem isn't entirely clear, I will just summarize it. I am talking about an object tree, potentially tens of millions of nodes big, a responsibility hierarchy of currently 30 roles, expected to grow to over 100. Right now I have approximately about 200 lines of code per role, although the number varies. Parallelization is not an option, since node result depends on parent or children nodes results, so I am limited to a single thread traversing the tree, currently processing 200-300k nodes a second.

答:
Just to supplement what other people have said, worrying about virtual function dispatch time is worrying about trivia.

A large hierarchy has another, hidden, source of performance issues.
You need to know how to clean these out, on an ongoing basis, while developing the code.

Here's an example.

The hidden source of performance issues is this:
Most lines of code are calls to methods of other classes, and a method you can call is like a credit card.
It makes spending easy, so with a credit card, on average you spend more than you would if spending were not so easy.

In software, this effect compounds over many layers.
At one layer you can spend more than you need to by 50%,
because you generally assume things are efficient,
and if they are not perfect, so what?
At a lower layer you (or your teammates) also do it.
Those two layers do not have to be adjacent.
1.5 * 1.5 = 2.25, so you've slowed down the software by more than twice!
This can easily escalate to an order of magnitude or more.

There's a funny psychological effect going on here.
When somebody writes a super-useful function or method foo,
they tend to assume its users will respect it and use it sparingly.
OTOH, each of those users think foo is such a great,
well-written, efficient function
that you can really get a lot done by using it a lot!
Sometimes this happens when the writer and the user are the same person.

All of this is not a reason to avoid layers, just as credit cards are not a bad idea.
It is a reason to know how to find and remove the inefficiencies,
just as credit cards require discipline.


问:
Shell commands in bash or python? How much encapsulation is too much?
I'm thinking about how to decide whether it's better to encapsulate my work behind well-named function names, or to expose it - which will help developers understand what's going on more quickly? Is there a name for the study of this sort of problem?

Specifically, if I'm running a bunch of bash commands ultimately, but I have significantly complex logic around those commands, at what point does it make sense to write this in a high-level language like Python, even though this obfuscates the actual bash commands being run?

Detailed problem

Currently I'm trying to write a Jenkins build script for my project with roughly the following steps:

Pull my code from github

Compile sass files into CSS

Pull down a sub-folder from a different github project

Zip up the project

Upload it to an object store with a unique ID

I'm thinking about how to write this to be as easy for future developers as possible (this code is never going to be seen by end users). These developers are likely, but not definitely, going to be fairly good at Python. They will definitely have a passing familiarity with the command-line, but are likely to be unfamiliar with more complex bash scripting.

The first iteration of this build script was just a list of sequential commands, something like:

```<code>git clone git@github.com:username/project.git
git clone git@github.com:username/sub-project.git project/sub-project
sass --update project/css
tar -czf project.tgz project
swift upload my-container project.tgz --object-name=project-`sha1sum project.tgz`.tgz
</code>```

However, this set of commands quickly became more complex as I started to do things like only clone the git project if it wasn't already there, otherwise update it - to speed up the build. Before I knew it I had 50 lines and a fair few conditionals.

So the first thing I did was encapsulate these into bash functions, e.g. update_git_dir, so my build script looks more like this:

```<code>#!/usr/bin/env bash

source helper_functions.sh

update_git_dir project git@github.com:username/project.git
build_sass project/css
create_archive project project.tgz
upload_to_swift project.tgz
</code>```

This is one level of encapsulation. Now the developer, who would have understood the git clone etc. commands directly, can't actually see what's going on. They have to look in helper_functions.sh.

However, as time went on I realised that many of my helper functions now consisted of more conditional statements, variable assignments and function calls than actual commands. These conditional statements can be quite opaque to someone not familiar with bash scripting:

```<code>function create_archive {
    project_name=${1}
    archive_filename=${2}

    # Get revision ids
    dependencies_requirements_revision=$(cat ${project_name}/sub-project/requirements-revision.txt)

    requirements_context=${project_name}/${requirements_file}
    requirements_dir=$(dirname ${requirements_context})
    if [ "${requirements_dir}" != "${project_name}" ]; then
        requirements_context=${requirements_dir}
    fi
    latest_revision=$(git-revision-hash ${project_name})

    ...
</code>```

So I started migrating my code into Python. So now my build script looks like this:

```<code>#!/usr/bin/env python

from builders import GitProjectBuilder

builder = GitProjectBuilder(
    project_name='my-project',
    swift_container='my-container',
    git_repository='git@github.com:username/project.git',
    sub_project='git@github.com:username/sub-project.git'
)

# Compress and upload
builder.build_sass(directory='css')
builder.get_sub_project(repo='git@github.com:username/sub-project.git')
builder.build_archive(name='archive.tgz')
upload_location = builder.upload_archive_to_swift(archive='archive.tgz')
print upload_location
</code>```

Now, when you look in builders.py, it's much easier to understand the logic - if statements and function calls are much more readable - but now we're even further away from the real shell commands. In my python code the closest I get to directly running shell commands looks like this:

```<code>def build_archive(self, archive):
    print subprocess.check_output(
        (
            'tar --exclude-vcs --create --file '
            '{archive_filename}.tar {project_dir}'
        ).format(
            archive_filename=archive_filename,
            project_dir=self.project_name
        ).split()
    )
</code>```

If the developer needs to work out exactly which commands are being run, it's now much more difficult.

Wrap up

So how do I decide which is the best architecture to maximise transparency while encapsulating complexity?

This problem seems similar to when I'm working with dependency injection where the more dependencies I inject rather than encapsulate, the more complex my initialisation code gets - and I have a similar problem drawing the line.

Is there a name for this field of study?

答:
I will not provide with an immediate yes/no answer, but some thoughts on the situation.

Build scripts since many people depend on them should be the most easy to understand area of your code. I would argue that a long "boring" bash should not be an issue as long as it is easily understood. I'd add a hint towards the "configure && build && build install" of the C in various unices.

Your bash script seems to be doing variable initialization and defaults assignment, none of those would get into a deep if-then-else structure.

An estimation of bash scripts size (although it is surely debatable) is that if it is more than 100 lines of code than it might need to be written into a "proper" program. Although the previous sentence is a matter of opinion.

If you decide not to go through the bash route, then you need to get into build tools who are made just for this purpose. Ant / Maven / Gradle in the Java world and many others for different platforms. I can kind of see your example as a series of build targets for some tools I've used in the past, as a sequence of Rake tasks or bazel ones.

I assume that if you go that route then it should be one in the most used language of the project if possible (easier to maintain).

I do not know if there is a name for the field, but someone doing this on day to day basis is called "integration engineer" in some companies.


答:
I'd give xonsh a go, it's a clever mix of shell and python.

xonsh is a Python-ish, BASHwards-compatible shell language and command prompt. The language is a superset of Python 3.4 with additional shell primitives that you are used to from BASH and IPython. xonsh is meant for the daily use of experts and novices alike.

Take advantage of Python(3)'s abstraction and package system, coupled with nice conditionals, but write what needs to be in shell as just shell.

e.g.,

#!/usr/bin/env xonsh

def exists(filename):
    return filename in $(ls)

if exists(".git"):
    git checkout master
    git pull
else:
    git clone $GITURL
```

Note that only a little bit of ugliness $() is required to in-line shell inside of python, and it Just Works (TM) if you are splitting things clearly by line (eg the if statement lines)

Lots more detail (including embedding python in shell lines with @() ) in the tutorial http://xonsh.org/tutorial.html

You can use it as your system shell. But just because you can doesn't mean you should :-)

Title: Shell commands in bash or python? How much encapsulation is too much?

Body:
I'm thinking about how to decide whether it's better to encapsulate my work behind well-named function names, or to expose it - which will help developers understand what's going on more quickly? Is there a name for the study of this sort of problem?

Specifically, if I'm running a bunch of bash commands ultimately, but I have significantly complex logic around those commands, at what point does it make sense to write this in a high-level language like Python, even though this obfuscates the actual bash commands being run?

Detailed problem

Currently I'm trying to write a Jenkins build script for my project with roughly the following steps:

Pull my code from github

Compile sass files into CSS

Pull down a sub-folder from a different github project

Zip up the project

Upload it to an object store with a unique ID

I'm thinking about how to write this to be as easy for future developers as possible (this code is never going to be seen by end users). These developers are likely, but not definitely, going to be fairly good at Python. They will definitely have a passing familiarity with the command-line, but are likely to be unfamiliar with more complex bash scripting.

The first iteration of this build script was just a list of sequential commands, something like:

```<code>git clone git@github.com:username/project.git
git clone git@github.com:username/sub-project.git project/sub-project
sass --update project/css
tar -czf project.tgz project
swift upload my-container project.tgz --object-name=project-`sha1sum project.tgz`.tgz
</code>```

However, this set of commands quickly became more complex as I started to do things like only clone the git project if it wasn't already there, otherwise update it - to speed up the build. Before I knew it I had 50 lines and a fair few conditionals.

So the first thing I did was encapsulate these into bash functions, e.g. update_git_dir, so my build script looks more like this:

```<code>#!/usr/bin/env bash

source helper_functions.sh

update_git_dir project git@github.com:username/project.git
build_sass project/css
create_archive project project.tgz
upload_to_swift project.tgz
</code>```

This is one level of encapsulation. Now the developer, who would have understood the git clone etc. commands directly, can't actually see what's going on. They have to look in helper_functions.sh.

However, as time went on I realised that many of my helper functions now consisted of more conditional statements, variable assignments and function calls than actual commands. These conditional statements can be quite opaque to someone not familiar with bash scripting:

```<code>function create_archive {
    project_name=${1}
    archive_filename=${2}

    # Get revision ids
    dependencies_requirements_revision=$(cat ${project_name}/sub-project/requirements-revision.txt)

    requirements_context=${project_name}/${requirements_file}
    requirements_dir=$(dirname ${requirements_context})
    if [ "${requirements_dir}" != "${project_name}" ]; then
        requirements_context=${requirements_dir}
    fi
    latest_revision=$(git-revision-hash ${project_name})

    ...
</code>```

So I started migrating my code into Python. So now my build script looks like this:

```<code>#!/usr/bin/env python

from builders import GitProjectBuilder

builder = GitProjectBuilder(
    project_name='my-project',
    swift_container='my-container',
    git_repository='git@github.com:username/project.git',
    sub_project='git@github.com:username/sub-project.git'
)

# Compress and upload
builder.build_sass(directory='css')
builder.get_sub_project(repo='git@github.com:username/sub-project.git')
builder.build_archive(name='archive.tgz')
upload_location = builder.upload_archive_to_swift(archive='archive.tgz')
print upload_location
</code>```

Now, when you look in builders.py, it's much easier to understand the logic - if statements and function calls are much more readable - but now we're even further away from the real shell commands. In my python code the closest I get to directly running shell commands looks like this:

```<code>def build_archive(self, archive):
    print subprocess.check_output(
        (
            'tar --exclude-vcs --create --file '
            '{archive_filename}.tar {project_dir}'
        ).format(
            archive_filename=archive_filename,
            project_dir=self.project_name
        ).split()
    )
</code>```

If the developer needs to work out exactly which commands are being run, it's now much more difficult.

Wrap up

So how do I decide which is the best architecture to maximise transparency while encapsulating complexity?

This problem seems similar to when I'm working with dependency injection where the more dependencies I inject rather than encapsulate, the more complex my initialisation code gets - and I have a similar problem drawing the line.

Is there a name for this field of study?

Body:
I'd give xonsh a go, it's a clever mix of shell and python.

xonsh is a Python-ish, BASHwards-compatible shell language and command prompt. The language is a superset of Python 3.4 with additional shell primitives that you are used to from BASH and IPython. xonsh is meant for the daily use of experts and novices alike.

Take advantage of Python(3)'s abstraction and package system, coupled with nice conditionals, but write what needs to be in shell as just shell.

e.g.,

#!/usr/bin/env xonsh

def exists(filename):
    return filename in $(ls)

if exists(".git"):
    git checkout master
    git pull
else:
    git clone $GITURL

Note that only a little bit of ugliness $() is required to in-line shell inside of python, and it Just Works (TM) if you are splitting things clearly by line (eg the if statement lines)

Lots more detail (including embedding python in shell lines with @() ) in the tutorial http://xonsh.org/tutorial.html

You can use it as your system shell. But just because you can doesn't mean you should :-)


问:
Multiple Canvas in the Browser -- How to Proceed?
Here's something I wrote ages ago as Java applet.  I want to convert it into JavaScript and run in html5 canvas in the browser. From what I can see, canvas makes the most sense here.  (SVG / DIV techniques aren't so friendly for what I want to do.)  The canvas does allow for mouse drag and visual update, something you need to be able to understand the concepts displayed.

Actually that is four separate applets.  This doesn't translate perfectly well to canvas.  I'm not sure the best way to make this update to JavaScript/Canvas.  I think I have three choices:

One large canvas, divided into fourths.  (The problem here is when you grab and drag the control elements there is nothing stopping that drag from hitting a neighboring technique. I could program internal boundaries, but where is the fun in that? This is my least favorite option.)

Four individual canvases (spelling?), each with its own #id.

four iframes with each canvas as a stand alone in its own html file.  These would all be hosted at the same server.

Anybody got an opinion on best technique to proceed for displaying four functional graphic boxes (and why?)  As I understand it, because I'm asking for an opinion, this question doesn't really fit over at stackoverflow...

答:
By all means, the best solution would be to use a canvas for each pane.

You noted in one of your comments:

Using iframes does reduce the coding workload a significant amount.
  You don't have to manage four different contexts. In fact you could
  code the boxes all the same way, with a flag element (flag = ellipse
  or bezier or bspline or nurb) (That was how the original applet was
  coded...)

But I don't see how using iframes would especially reduce the coding. More than likely you'd eventually code more if you used an iframe, you potentially could come to encounter painful issues due to the iframes.

Here's how you may approach resolving your issue, the code speaks for itself:

```<code>/**
 * Pane object is a map from ID to options specific to a pane.
 * The id will be used for each canvas element, and the options will determine
 * what will be drawn on a canvas.
 * @type object
 */
var pane = {
    "ellipse": {
        label: "Simple Straight Line & Ellipse Technique",
        degree: null,
        buttons: [{
                label: "Information",
                execute: function () {
                    // implementation of what happens when the 'Information'
                    // button on the ellipse canvas is clicked.
                }
            },
            {
                label: "Reset Frame",
                execute: function () {
                    // implementation of what happens when the 'Reset Frame'
                    // button on the ellipse canvas is clicked.
                }
            }]
    },
    "beizer": {
        label: "Beizer Technique",
        degree: null,
        buttons: [/* You have a pattern to follow above */]
    },
    "b-spine": {
        label: "B-Spine Technique with ...",
        degree: 3,
        buttons: [/* You have a pattern to follow above */]
    },
    "nurbs": {
        label: "NURBS with uniform ...",
        degree: 3,
        buttons: [/* You have a pattern to follow above */]
    }
};

// Extract the IDs for each canvas, and call `draw` for each one of them.
Object.keys(pane).forEach(draw);

/**
 * Draws on the canvas with the given id.
 * @param {string} id ID of a canvas element
 * @returns {undefined}
 */
function draw(id) {
    var option, canvas, context;

    option = pane[id];
    canvas = document.getElementById(id);

    // Now you are ready to implement your drawing,
    context = canvas.getContext("2d");
    // if you wanted the label you can have it from `option.label`.
    // The rest is up to your imagination...
}
</code>```

Based on the option associated with each pane, you may branch to a special drawing logic.

Body:
By all means, the best solution would be to use a canvas for each pane.

You noted in one of your comments:

Using iframes does reduce the coding workload a significant amount.
  You don't have to manage four different contexts. In fact you could
  code the boxes all the same way, with a flag element (flag = ellipse
  or bezier or bspline or nurb) (That was how the original applet was
  coded...)

But I don't see how using iframes would especially reduce the coding. More than likely you'd eventually code more if you used an iframe, you potentially could come to encounter painful issues due to the iframes.

Here's how you may approach resolving your issue, the code speaks for itself:

/**
 * Pane object is a map from ID to options specific to a pane.
 * The id will be used for each canvas element, and the options will determine
 * what will be drawn on a canvas.
 * @type object
 */
var pane = {
    "ellipse": {
        label: "Simple Straight Line & Ellipse Technique",
        degree: null,
        buttons: [{
                label: "Information",
                execute: function () {
                    // implementation of what happens when the 'Information'
                    // button on the ellipse canvas is clicked.
                }
            },
            {
                label: "Reset Frame",
                execute: function () {
                    // implementation of what happens when the 'Reset Frame'
                    // button on the ellipse canvas is clicked.
                }
            }]
    },
    "beizer": {
        label: "Beizer Technique",
        degree: null,
        buttons: [/* You have a pattern to follow above */]
    },
    "b-spine": {
        label: "B-Spine Technique with ...",
        degree: 3,
        buttons: [/* You have a pattern to follow above */]
    },
    "nurbs": {
        label: "NURBS with uniform ...",
        degree: 3,
        buttons: [/* You have a pattern to follow above */]
    }
};

// Extract the IDs for each canvas, and call `draw` for each one of them.
Object.keys(pane).forEach(draw);

/**
 * Draws on the canvas with the given id.
 * @param {string} id ID of a canvas element
 * @returns {undefined}
 */
function draw(id) {
    var option, canvas, context;

    option = pane[id];
    canvas = document.getElementById(id);

    // Now you are ready to implement your drawing,
    context = canvas.getContext("2d");
    // if you wanted the label you can have it from `option.label`.
    // The rest is up to your imagination...
}

Based on the option associated with each pane, you may branch to a special drawing logic.


问:
Why would I use code contracts
I recently stumbled upon Microsoft's framework for code contracts.

I read a bit of documentation and found myself constantly asking: "Why would I ever want to do this, as it does not and often cannot perform a static analysis."

Now, I have a kind of defensive programming style already, with guarding exceptions like this:

```<code>if(var == null) { throw new NullArgumentException(); }
</code>```

I'm also using NullObject Pattern alot and have rarely any problems. Add Unit Tests to it and you're all set up.

I have never used asserts and never missed them. Quite the contrary. I really hate code that has lots of meaningless asserts in it, which is just noise to me and distracts me from what I really want to see. Code contracts, at least the Microsoft way are much the same - and even worse. They add lots of noise and complexity to the code. In 99% an exception will be thrown anyway - so I don't care whether it's from the assert/contract or the actual problem. Just very, very few cases remain in which the program states really becomes corrupted.

So frankly, what is the benefit of using code contracts? Is there any at all? If you already use unit tests and code defensively I feel that introducing contracts is just not worth the cost and puts noise in your code that a maintainer will curse when he's updating that method, much like I do when I cannot see what the code is doing due to useless asserts. I have yet to see a good reason to pay that price.

答:
I know this is a little late to the question, but there is one other benefit to Code Contracts which bears mentioning

Contracts are checked outside the method

When we have guard conditions inside our method, that is the place where checking happens and where errors get reported. We might then have to investigate the stack trace to find out where the actual source of the error is.

Code contracts are located within the method, but preconditions are checked outside the method when anything attempts to call that method. The contracts become part of the method and determines whether it can be called or not.  That means that we get an error reported much closer to the actual source of the problem.

If you have a contract protected method which is called for many places, this can be a real benefit.

Title: Why would I use code contracts


答: If you can show that what you did falls under the "loose requirements" under any (preferably your) interpretation then any clarification becomes a change request, subject to new estimates and new effort and new payments. Then it falls over your client's shoulder the responsibility to come up with crispier requirements.</p>

Title: Are there any arguments that can make a contractor reconsider working on fixed price?

Body:
I've been working for a contractor who brings in some good projects, but they are all fixed-price and often fixed-time.

As a result he always has me making a quote over loose requirements, which never fails to bring a lot of tension due to feature creep.

He claims he'd never get a contract if he couldn't agree on a price with his clients first, but as far as I'm concerned I don't wanna go through another project under these terms.

Is there any argument I could make to have him pay me by the hour, or should I just suck less at estimating ?


问:
What are the valid uses of static classes?
I noticed that nearly every time I see programmers using static classes in object oriented languages such as C#, they are doing it wrong. The major problems are obviously the global state and the difficulty to swap implementations at runtime or with mocks/stubs during tests.

By curiosity, I've looked at a few of my projects selecting ones which are well tested and when I made an effort to actually think about architecture and design. The two usages of static classes I've found are:

The utility class—something I would rather avoid if I was writing this code today,

The application cache: using static class for that is plainly wrong. What if I want to replace it by MemoryCache or Redis?

Looking at .NET Framework, I don't see any example of a valid usage of static classes either. For example:

File static class makes it really painful to switch to alternatives. For example, what if one needs to switch to Isolated Storage or store files directly in memory or needs a provider which can support NTFS transactions or at least be able to deal with paths longer than 259 characters? Having a common interface and multiple implementations appears as easy as using File directly, with the benefit of not having to rewrite most of the code base when requirements change.

Console static class makes testing overly complicated. How should I ensure within a unit test that a method outputs a correct data to console? Should I modify the method to send the output to a writeable Stream which is injected at run time? Seems that a non-static console class would be as simple as a static one, just without all the drawbacks of a static class, once again.

Math static class is not a good candidate either; what if I need to switch to arbitrary precision arithmetic? Or to some newer, faster implementation? Or to a stub which will tell, during a unit test, that a given method of a Math class was indeed called during a test?

On Programmer.SE, I've read:

Don't Use “Static” in C#? Some answers are quite against static classes. Others assert that “Static methods are fine to use and have a rightful place in programming.”, but don't bake it with arguments.

When to use a Singleton and when to use a static class. Utility classes are mentioned, given that I mentioned above that utility classes are problematic.

Why and when should I make a class 'static'? What is the purpose of 'static' keyword on classes? The only valid usage which is mentioned is a container for extension methods. Great.

Aside extension methods, what are the valid uses of static classes, i.e. cases where Dependency Injection or singletons are either impossible or will result in a lower quality design and harder extensibility and maintenance?

答: In general the only places where I would use static classes are those where the class hosts pure functions which don't cross system boundaries. I realize the latter is redundant as anything that crosses a boundary is likely by intent not pure.  I come to this conclusion both from a distrust of global state and from an ease of testing perspective. Even where there is a reasonable expectation that there will be a single implementation for a class which crosses a system boundary (network, file system, I/O device), I choose to use instances rather than static classes because the ability to supply a mock/fake implementation in unit tests is critical in developing fast unit tests with no coupling to actual devices.  In cases where the method is a pure function with no coupling to an external system/device I think a static class may be appropriate but only if it doesn't hinder testability. I find the uses cases to be relatively rare outside existing framework classes. In addition, there may be cases where you have no choice - for example, extension methods in C#.</p>


问:
Is the time complexity of a while loop with three pointers different than 3 nested for loops?
This program (written in ruby) finds the largest 3 numbers in an array (without sorting the array). It has a while loop with three pointers. My fist instinct, since there is only one loop is to say this solution is O(n). But, the pointer's j and k get reset when they reach n which seems the same as  using 3 for loops, so then is it O(n³)?

```<code>def greatest_of_3_with_ptrs(a)
  greatest = 0
  greatest_tuple = nil
  i,j,k=0,1,2
  n = a.length-1
  while( true) do
    sum = a[i]+a[j]+a[k]
    if sum > greatest then
      greatest = sum
      greatest_tuple =[a[i],a[j],a[k]]
    end
    sum = 0
    if k == n then
      j=j+1
      k=j+1
    else
      k+=1
    end
    if j == n then
      i=i+1
      j=i+1
      k=j+1
    end
    break if k > n || j > n || i==n
  end
  { greatest_tuple: greatest_tuple, greatest: greatest }
end

a =[10,-1,4,2,5,3,0]
expected = 19
print greatest_of_3_ptrs(a)[:greatest] #> 19
print greatest_of_3_ptrs(a)[:greatest_tuple] #> [10,4,5]
</code>```

Please Note: I am aware of the O(n) solution, that is not my question

```<code>def greatest(a)
  max_1,max_2,max_3 = a[0],a[1],a[2]
  for i in 3..a.length-1
    if a[i] > max_1 then
      max_1 = a[i]
    elsif a[i] > max_2 then
      max_2 = a[i]
    elsif a[i] > max_3 then
      max_3 = a[i]
    end
  end
  [max_1,max_2,max_3]
end
</code>```

答:
You are absolutely correct. The function tries out all possible combinations of elements from the array. In this case, it's O(n³) and would be of an even higher power, if you wrote a similiar function for the greatest 4, 5, .. elements. The best solution for any of these problems should be of the order O(n).

I just want to mention that in your example for an O(n) solution you omitted the "down-shift" when replacing a value:

```<code>def greatest(a)
  max_1,max_2,max_3 = a[0..2].sort.reverse #changed
  for i in 3..a.length-1
    if a[i] > max_1 then
      max_3 = max_2 #new
      max_2 = max_1 #new
      max_1 = a[i]
    elsif a[i] > max_2 then
      max_3 = max_2 #new
      max_2 = a[i]
    elsif a[i] > max_3 then
      max_3 = a[i]
    end
  end
  [max_1,max_2,max_3]
end
</code>```

Body:
You are absolutely correct. The function tries out all possible combinations of elements from the array. In this case, it's O(n³) and would be of an even higher power, if you wrote a similiar function for the greatest 4, 5, .. elements. The best solution for any of these problems should be of the order O(n).

I just want to mention that in your example for an O(n) solution you omitted the "down-shift" when replacing a value:

def greatest(a)
  max_1,max_2,max_3 = a[0..2].sort.reverse #changed
  for i in 3..a.length-1
    if a[i] > max_1 then
      max_3 = max_2 #new
      max_2 = max_1 #new
      max_1 = a[i]
    elsif a[i] > max_2 then
      max_3 = max_2 #new
      max_2 = a[i]
    elsif a[i] > max_3 then
      max_3 = a[i]
    end
  end
  [max_1,max_2,max_3]
end


问:
What is the "correct" way to store functions in a database?
Note: Yes, I know that storing functions in databases should be punishable by law.

We are developing a financial web application using PostgreSQL, Sinatra and AngularJS. As you may have guessed, a financial application may need some calculations.

There is a certain database model (called 'Hypothesis') that has a certain function that needs to be executed overnight. The problem is that this function is different for each instance of the model. That being said, whenever a new hypothesis is added a corresponding function also needs to be added.

Because of the nature of financial applications, the function can either do simple arithmetic operations or integrate over an area. This leaves us with the following options:

Implement a DSL (domain-specific language):
This would be ideal, but adding branching and looping logic to a DSL seems more like creating a new programming language.

STDIO pipe: Simply allow the execution of a program in a sandbox on a server. This is most flexible, but the gods of software security would not be merciful. Docker I guess?

Pure sandboxed ruby code: Create a sandbox that allows only functionally pure Ruby code to be executed in it. Then simply evaluate the function inside the sandbox whenever required. This would be a security flaw as well, but not half as much as options 2. For now this option seems to be implementable using trusted-sandbox and pure.

Use the octave-ruby gem: Beautiful idea. But it seems that nobody cares much about the project and we will probably need to fork it and work on it. This seem like the best option, since writing complex math expression is trivial in Octave.

???

Profit

Another concern would be implementing the ability to test the functions with mock data in the admin panel, but this is not mandatory.

Can somebody suggest a better/more flexible/more secure option? Or at least one that doesn't store a function string in the database? I hope that I have explained the problem well enough.

答:
I don't know how different the functions you want to store are, but if you can isolate few different "types of functions" you could do something like this :

Create several generic methods in your model which can make every special calculus needed with the help of some arguments :

```<code>def integrate_over_an_area(args={})
  #Do your calculus with your args.
end

def simple_arithmetics(args={})
  #Do your calculus with your args.
end
</code>```

Create - for each model instance - a hash describing which generic function to use and the parameters :

```<code>{
 function: "integrate_over_an_area",
 args: {
   arg1: "value",
   arg2: "value",
   ...
  }
}
</code>```

Store the hashes in a column named "special_function_hash" or something like this, in your model's table

Create a "special_function" method in your model which calls the function described in the hash, with the arguments described in the hash

Body:
I don't know how different the functions you want to store are, but if you can isolate few different "types of functions" you could do something like this :

Create several generic methods in your model which can make every special calculus needed with the help of some arguments :

def integrate_over_an_area(args={})
  #Do your calculus with your args.
end

def simple_arithmetics(args={})
  #Do your calculus with your args.
end

Create - for each model instance - a hash describing which generic function to use and the parameters :

{
 function: "integrate_over_an_area",
 args: {
   arg1: "value",
   arg2: "value",
   ...
  }
}

Store the hashes in a column named "special_function_hash" or something like this, in your model's table

Create a "special_function" method in your model which calls the function described in the hash, with the arguments described in the hash


问:
When applying the interface segregation, should you separate interfaces for regular setters and mathematical operations?
When applying the interface segregation, should you have separate interfaces for a straightforward setters and interfaces that perform an operation before setting?  For example, say you have a class:

```<code>class FooClass:
     public GettableFoo,
     public SettableFoo
{
    virtual int getRelevantData();
    // member of GettableFoo (which is used by roughly 20 clients)

    virtual void setToOtherFooTemplate(GettableFoo *other);
    // member of SettableFoo (which is used by roughly 20 clients, some of which also need GettableFoo interface)

    void setToSum(GettableFoo *a,GettableFoo *b);
    // requires SettableFoo functions to implement, used by 8 clients, some of which also need SettableFoo interface

    void combineSomeOtherWay(GettableFoo *a,GettableFoo *b);
    // requires SettableFoo functions to implement, used by 4 clients (possibly more in the future), some of which also need SettableFoo interface and/or which require the setToSum function

private:
  // complex data
}
</code>```

The interface segregation principle states that no client should be forced to depend on methods it does not use.  Does that mean that setToSum() and combineSomeOtherWay() should each have their own interfaces?  The answer seems like it should be yes, but I don't think I've ever come across code with interfaces segregated that much.

If the answer to the above is yes, wouldn't that mean if combineSomeOtherWay() were just setToProduct() (multiply) and I used operator overloading for both to implement this in C++, that technically the operator overloading would be breaking the interface segregation principle given the client needs stated above? And if so, wouldn't most instances where operator overloading is used be breaking ISP, and if not, how is it any different from not segregating any of the interfaces in the above class?

答:
I find it helpful to think of an interface as a role an object can play.
So if a client depends on a single role to do it's job, it will only depend on one interface.

If a bunch of methods belongs to the same role, then it doesn't violate ISP to put them into one interface.
For example, a set of mutually-exclusive overloads for doing the same thing in a slightly different way probably belongs to a single role.

Segregating interfaces is a good thing because it gives the clients the ability and encourages them to be designed in more modular way and couple to a narrower contract. And it doesn't really cause significant over-engineering or other overhead, unlike some other techniques/patterns from OOP.

You can always easily merge multiple interfaces back into one if it makes sense.
Much harder is to split an interface after you already have a bunch of clients who now have too much responsibility and coupling because they were allowed to depend on a huge interface. This is why it's not always a good idea to rely on clients to tell you how to split an interface.

Summary:

model roles that make sense for your application

when in doubt, segregate more


答:
You could embed an interpreter (like Lua or GNU guile) into the program using the database.

(I am not sure that coding your entire software in Ruby is best; coding it in C++ might be worthwhile)

You would either store in the database expressions (that is string containing Guile or Lua source) in Lua or Guile, or you might store some bytecode or some serialized closures.

Title: What is the "correct" way to store functions in a database?

Body:
Note: Yes, I know that storing functions in databases should be punishable by law.

We are developing a financial web application using PostgreSQL, Sinatra and AngularJS. As you may have guessed, a financial application may need some calculations.

There is a certain database model (called 'Hypothesis') that has a certain function that needs to be executed overnight. The problem is that this function is different for each instance of the model. That being said, whenever a new hypothesis is added a corresponding function also needs to be added.

Because of the nature of financial applications, the function can either do simple arithmetic operations or integrate over an area. This leaves us with the following options:

Implement a DSL (domain-specific language):
This would be ideal, but adding branching and looping logic to a DSL seems more like creating a new programming language.

STDIO pipe: Simply allow the execution of a program in a sandbox on a server. This is most flexible, but the gods of software security would not be merciful. Docker I guess?

Pure sandboxed ruby code: Create a sandbox that allows only functionally pure Ruby code to be executed in it. Then simply evaluate the function inside the sandbox whenever required. This would be a security flaw as well, but not half as much as options 2. For now this option seems to be implementable using trusted-sandbox and pure.

Use the octave-ruby gem: Beautiful idea. But it seems that nobody cares much about the project and we will probably need to fork it and work on it. This seem like the best option, since writing complex math expression is trivial in Octave.

???

Profit

Another concern would be implementing the ability to test the functions with mock data in the admin panel, but this is not mandatory.

Can somebody suggest a better/more flexible/more secure option? Or at least one that doesn't store a function string in the database? I hope that I have explained the problem well enough.


答:
Let's note that how the code is stored is inconsequential; how it is executed is essential.

Most probably your functions have a limited repertoire. Implement them properly, without storing in the DB. (I leave alone the question if a numeric integration implemented in Ruby a good idea. Let's assume you use something like Numpy of the Ruby world.)

Now you can have proper code revisions, tests, etc.

Then you can store a very limited form of a program in your database: which functions to call with which field values. The choice is limited by your blessed set of functions.

What if you need to store simple but unpredictable formulas?

I'd limit this feature to formulas that do simple arithmetics with no side effects, no loops, and no function definitions. It's pretty easy to write a parser for the basic arithmetic expressions with parentheses; most probably ready-made gems exist.

Then I'd store these expressions in the DB as a text. I'd only let in the expressions that can be parsed (no code constructs beyond the tiny allowed subset), and only execute the expressions that again can be parsed (so that putting  exploit code directly into the DB won't work). This is apparently what you call the DSL approach.

If the above is not enough in your case, I'd pick a well-known embeddable language that can be stripped of unneeded parts and only allow functions you say it should allow. Lua is an obvious choice, Python is a slightly less obvious choice. Be prepared: it would take quite some work.

Docker could be a simpler approach, though: you can hide anything not strictly needed, like network and disk access, from the process.

But learn first if you are going to need these complexities. Chances are high that you don't.


问:
How to simplify structure of a monitoring software that uses the Observer pattern?
I am working on a project where I have to monitor the status of an elevator. So far, my design looks like this:

We are using a third party library to read and monitor real-time values from the elevator's logic controller. A tag is an integer value in a memory register of the logic controller. For example, DM140 contains the elevator direction.

I use an adapters layer in order to keep my application domain isolated from the third party library.

In the domain, the ElevatorDirectionMonitor takes the ElevatorTag integer value, extrapolates the elevator's current direction, and then notifies its clients that the elevator's direction has changed. So far, so good.

Here's my problem: There will be a lot (a hundred or more) of types of information to monitor from the elevator, such as the current floor, whether the  doors are opened/closed, etc. If I continue this way, I will have to create one monitor class for each type of information I want to monitor (e.g. an ElevatorFrontDoorStatusMonitor, ElevatorRearDoorStatusMonitor) and each monitor associated with its own Observer interface. This means a lot of work. I've been trying to find another way to structure my application, but haven't found anything more "elegant".

The basic logic in each monitor class would be pretty much the same across the board: listen for value change on an ElevatorTag instance, extrapolate the information from the new tag value, and then notify the observers. The problem seems to come from the fact that each observer class must have a different interface signature (e.g. ElevatorDirectionObserver's has NotifyDirectionChanged(ElevatorDirection newDirection) when a DoorStatusMonitor would rather have something like NotifyDoorStatusChanged(DoorPosition frontOrRear, DoorStatus openOrClosed)).

Do you think that the solution I am working on is acceptable. If not,could someone suggest which patterns I should use or investigate, or anything that could help me to simplify my design?

答:
You're looking for Boost.Signals2. In this case, every "monitor" is just one instance of boost::signals2::signal<void(ElevatorDirection)>, say. And every "observer" is just some random function of the appropriate signature- like a lambda. You can just call the signal whenever you need and add/remove methods provided. No bullshit required.

The Observer pattern is not a pattern at all. It's just a class. A class you don't even have to implement.


答: I would use Jupyter as my backend execution engine & you then can write your functions in several languages e.g. Julia, python or octave. https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages</p>


答:
Rather than fret about how much abstraction is too much for a build/integration tool, instead opt for a tool commonly used by Python developers to do this sort of thing. You'll get far more use out of a well known, but highly abstracted build tool than you will from the clearest written, simplest build tool that does the same thing.

What is more important to understand: Knowing the gritty details about how a  gzip file is created in Bash, or that these files are gzipped together and all manners of chaos and panic will occur if it is omitted?

I would recommend searching for Python build tools. Ask the python community what they recommend and choose a open source project. It probably has more features than you've built, and it probably has been tested more completely.


问:
Maintaining multiple versions of a library that all need maintenance and development
I'm writing an interface to a lexicon in Java. I just started this project, and the lexicon is on version 3.8. When a new version comes out, say version 4.0, database columns may be renamed, removed, added, re-organized, etc. The database could be totally different, and my 3.8 interface would not work with it, so I would need to update the library to interface with 4.0. However, I also want to maintain backwards compatibility with 3.8, and continue to do development on all versions until I drop support for them.

I believe that source code control software like Git (which I'm using for the project) can make maintenance and feature addition to different versions really easy through branches and release tags, but I'm not sure how to organize the project.

How does one manage a single library that interfaces with multiple versions of a data source, that may each need continued development?

My thoughts:

Most recent version of the library's development goes in master

development for old versions are in OLD_VERSION_XXXX branches where XXXX might be:

stable

development

any random feature/bugfix branch that gets merged into dev or stable for that version

Use Releases and Tags in Github to ensure consistency between these versions, and to release new compiled versions when necessary

Note: This is a personal project that (probably) only I will use, but I want to apply some real-world practices the right way.

答:
Two things come to mind.

Encapsulate the thing that changes into its own class and/or interface, and provide a factory to get the desired version. This works well if the "data access" is the only thing that changes.

Create an adapter layer that acts as the compatibility layer between version X and version Y. You might have a bunch of classes in this layer but it could blunt the impact of non compatible API changes at the expense of some performance.


答:
If I were in your position, I would use http://paver.github.io/paver/
It's basically Rake for python.</p>


问:
Dynamic gesture recognition with fingertip points
I have built a fairly robust program in c++ which tracks several points on a hand. It accurately quantifies the size of the palm, the center of the palm, and the fingertip locations among other hand traits. The points are all stabilized with a Kalman filter.

I want to be able to recognize certain dynamic gestures such as: a wave, a swipe right/left, a circle drawn with the palm open. I am thinking of storing 2D points in a vector over time and trying to pattern match this or apply constraints to recognize the gestures.

As I am already starting to eat up the CPU of an i7 and want this to be performed in real time, efficiency is necessary. What approach should I take to recognize these dynamic gestures. Should I train classifiers to recognize certain patterns, or should I apply certain geometric constraints?

答:
You could use Hidden Markov Models as they are used in tasks similar to your program like speech recognition and handwriting recognition. This research paper might help you out.

http://www.cs.ucsb.edu/~mturk/Papers/ICPR2000.pdf


问:
What's the use of .Any() in a C# List<>?
I've been discussing this with colleagues, and we couldn't figure out what the use is of .Any for any given List<>, in C#.

You can check the validity of an element in the array like the following statement:

```<code>if (MyList.Any()){ ...}  //Returns true or false
</code>```

Which is exactly the same as

```<code>if (MyList.Count() != 0) { ... }
</code>```

and is much more common, readable and clear about the intent of the if statement.

In the end, we were stuck with this thought:

.Any() can be used, will work just as well, but is less clear about
  the intent of the programmer, and it that case it should not be used.

But we feel like this can't be right; we must be missing something.

Are we?

答:
Keep in mind that Any doesn't operate on a List; it operates on an IEnumerable, which represents a concrete type that may or may not have a Count property.  It's true that it's not necessarily the best thing to use on a List, but it definitely comes in handy at the end of a LINQ query.  And even more useful than the standalone version is the override that takes a predicate just like Where.  There's nothing built in on List that's anywhere near as convenient or expressive as the predicate-Any extension method.

Also, if you're using Count() (the LINQ extension method for IEnumerable), rather than Count (the property on List), it can have to enumerate the entire sequence if it can't optimize this away by detecting that your underlying data type has a Count Property.  If you have a long sequence, this can be a noticeable performance hit when you don't really care about what the count is, and just want to know if there are any items in the collection.


问:
Are there any arguments that can make a contractor reconsider working on fixed price?
I've been working for a contractor who brings in some good projects, but they are all fixed-price and often fixed-time.

As a result he always has me making a quote over loose requirements, which never fails to bring a lot of tension due to feature creep.

He claims he'd never get a contract if he couldn't agree on a price with his clients first, but as far as I'm concerned I don't wanna go through another project under these terms.

Is there any argument I could make to have him pay me by the hour, or should I just suck less at estimating ?

答:
Could you just ask if you could be paid by the hour?  All my consulting contracts were hourly wage, no matter what the terms with the client were.

Alternately, firm up the requirements yourself and present that to the client as part of the agreement.  That way, you get to bill additional for feature creep.  Any contract that says you'll get paid $X for an amount of work the client gets to increase more or less arbitrarily is a disaster waiting to happen.


答: There is a run time difference Count() can be O(n) where Any() is O(1).

Title: What's the use of .Any() in a C# List<>?

Body:
I've been discussing this with colleagues, and we couldn't figure out what the use is of .Any for any given List<>, in C#.

You can check the validity of an element in the array like the following statement:

```<code>if (MyList.Any()){ ...}  //Returns true or false
</code>```

Which is exactly the same as

```<code>if (MyList.Count() != 0) { ... }
</code>```

and is much more common, readable and clear about the intent of the if statement.

In the end, we were stuck with this thought:

.Any() can be used, will work just as well, but is less clear about
  the intent of the programmer, and it that case it should not be used.

But we feel like this can't be right; we must be missing something.

Are we?


问:
Who is the Owner of Information? Memento vs. Originator
Imagine for a second that I'm implementing the Memento Pattern, using the following classes.

Classes

An Originator class that has public and private, properties and fields respectively

A Memento class that has many properties/fields representing state of Originator

A CareTaker class that acts on an instance of Originator, and saves a list of Mementos representing the Originator's state.

Using the example classes above, is there an objective reason to use one of the following solutions over the other? The premise here is that I need to expose a few properties that partially represent the state of the Originator.

Solution #1

Properties and fields representing state are duplicated across Originator & Memento

Originator has a CreateMemento method that initializes a new Memento with all of the shared properties/fields between the two

Originator has a SetMemento method that sets Originator's properties/fields to those that correspond with Memento's

Solution #2

Originator has an instance of Memento, representing Originator's current state

Originator exposes as properties, the necessary properties/fields of it's Memento, using the getters/setters to update the memento

Originator doesn't have it's own copy of properties/fields representing its state, instead it uses it's Memento as a backing store.

I understand that ideally there should be yet another class OriginatorState, that Originator & Memento both have instances of, but I'm not asking about which classes to create. I'm asking which class holds the information of the current state of the Originator. Both the Originator and the Memento? Just the Memento? or the Originator vicariously through the Memento as in Solution #2?

答:
From the description of the Memento Design Pattern:

The Memento design pattern defines three distinct roles:

Originator - the object that knows how to save itself.

Caretaker - the object that knows why and when the Originator needs to save and restore itself.

Memento - the lock box that is written and read by the Originator, and shepherded by the Caretaker.

The Originator is responsible for creating the Memento and restoring itself to a previous state. A Memento is just a bag of data devoid of logic. The CareTaker knows when to get a Memento and when the Originator should be restored from a Memento.


答:
Actually, keep in mind that there is the List.Count property, and then there is the Enumerable.Count method.

In your example, you are using the Enumerable.Count() method, which has to iterate through every single item of the enumeration to return a result.  That is clearly slower than calling Any() which only needs to iterate over the first item, if it exists.

EDIT:

In the comments it was pointed out, rightly so, that Enumerable.Count() extension method doesn't need to iterate through all the items if it detects that the enumerable also happens to be an ICollection<T>.  So in the case of a List<T>, using the Count property or method doesn't actually make a difference.

IEnumerable.Count source:

```<code>public static int Count<TSource>(this IEnumerable<TSource> source) {
    if (source == null) throw Error.ArgumentNull("source");
    ICollection<TSource> collectionoft = source as ICollection<TSource>;
    if (collectionoft != null) return collectionoft.Count;
    ICollection collection = source as ICollection;
    if (collection != null) return collection.Count;
    int count = 0;
    using (IEnumerator<TSource> e = source.GetEnumerator()) {
        checked {
            while (e.MoveNext()) count++;
        }
    }
    return count;
}
</code>```

Body:
Actually, keep in mind that there is the List.Count property, and then there is the Enumerable.Count method.

In your example, you are using the Enumerable.Count() method, which has to iterate through every single item of the enumeration to return a result.  That is clearly slower than calling Any() which only needs to iterate over the first item, if it exists.

EDIT:

In the comments it was pointed out, rightly so, that Enumerable.Count() extension method doesn't need to iterate through all the items if it detects that the enumerable also happens to be an ICollection<T>.  So in the case of a List<T>, using the Count property or method doesn't actually make a difference.

IEnumerable.Count source:

public static int Count<TSource>(this IEnumerable<TSource> source) {
    if (source == null) throw Error.ArgumentNull("source");
    ICollection<TSource> collectionoft = source as ICollection<TSource>;
    if (collectionoft != null) return collectionoft.Count;
    ICollection collection = source as ICollection;
    if (collection != null) return collection.Count;
    int count = 0;
    using (IEnumerator<TSource> e = source.GetEnumerator()) {
        checked {
            while (e.MoveNext()) count++;
        }
    }
    return count;
}


问:
Is there a software attribute (like reliability) that describes a bug free experience?
TL;DR

Do non-crashing bugs always affect Functionality, despite possibly affecting other quality attributes as well?  Or is there a more specific quality attribute that non-crashing bugs always affect, like Functional Suitability?

This might seem like an ambiguous or unclear question, but keep in mind that core software attributes are very well defined by standards like ISO/IEC 9126, which describes:

Functionality

Reliability

Usability

Efficiency

Maintainability

Portability

There is a really large list on Wikipedia as well, but many of those approach the fuzzy, flexible land of the blogosphere.  I want to keep this question unambiguous and bound to accepted definitions.

I'm looking for a software attribute that any non-crashing application bug affects.  For instance, I consider a crashing bug to be a Reliability issue, but I'm not sure how to generally classify a non-crashing bug.

Correct me if you think I'm wrong, but here's why crashing bugs are most generally Reliability issues.  If an application crashes (closes unexpectedly, freezes and doesn't recover, etc), it could be a security issue if protected assets are exposed or destroyed somehow.  It could also be a safety issue if the application is, for instance, an X-Ray machine control panel.  But a crash is always a Reliability issue, regardless of where else it falls.

But a non-crashing bug?  Examples:

some valid input is treated as invalid

EX: February 29, 2012 is incorrectly disallowed, but is a leap day.

some valid input is represented incorrectly

EX: 11:59 AM + 1 minute is shown as 11:60 AM rather than 12:00 PM.

configuration or assets that are supposed to be persistent do not persist

EX: User needs to log in every visit, despite selecting "Remember Me"

input is ignored in some cases

EX: File extension is ignored if dragged-and-dropped

These all affect Usability, but that might not always be the case, so Usability is not a completely general classification.  I'm leaning toward Functionality, because Functionality pertains to user needs and requirements fulfillment, and I think a non-crashing bug will always deviate from expected behaviour and/or correct behaviour.  But something more specific than Functionality is desirable.

ISO/IEC 25010, the 2011 revision gives these definitions for Functional Suitability:

4.2.1

  functional suitability

  degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions

Note 1 to entry: Functional suitability is only concerned with whether the functions meet stated and implied needs, not the functional specification (see C.6).

4.2.1.1

  functional completeness

  degree to which the set of functions covers all the specified tasks and user objectives

4.2.1.2

  functional correctness

  degree to which a product or system provides the correct results with the needed degree of precision

4.2.1.3

  functional appropriateness

  degree to which the functions facilitate the accomplishment of specified tasks and objectives

EXAMPLE:A user is only presented with the necessary steps to complete a task, excluding any unnecessary steps.

...which does seem to fit.  I can't fathom a non-crashing bug that doesn't affect either Functional Completeness or Functional Correctness or both.

And finally, by bug, I'm referring to all three steps in the dependability chain:  the fault in source code, the erroneous program states caused by the fault, and the interface failure caused by the error.

Whew, that was a lot of work for terminology's sake!

答:
I would call non-crashing bugs a "Reliability" issue as well, because you don't just have to be able to rely on a program to stay up; you have to rely on it to do what it's supposed to do.

Imagine a program that never crashed but sometimes thought that 2+2=5 when certain edge cases were hit.  I wouldn't consider such a program to be reliable.  Would you?


问:
Practical size limits of a Hashtable and Dictionary in C#
What are the practical limits for the number of items a C# 4 Dictionary or Hashtable can contain and the total number of bytes these structures can reasonable contain?

I'll be working with large numbers of objects and want to know when these structures start to experience issues.

For context, I'll be using a 64-bit system with tons of memory. Also, I'll need to find objects using some form or 'key'. Given the performance demands, these objects will need to reside in memory, and many will be long-lived.

Feel free to suggest other approaches/patterns, although I need to avoid using third-party or open-source libraries. For specification reasons, I need to be able to build this using native C# (or C++\CLI).

答:
I recently updated the GitHub project hash-table-shootout. The standard GCC unordered map has about 1.8 gigabyte of overhead to store 40 million objects. This seems pretty atrocious to me, but even the best performer memory wise, the Google sparse_hash_map, takes 600 megabytes, and you pay a performance penalty for using it.

If you want speed, of the algorithms included, the Glib GHashTable is the fastest and has good memory performance (around 1.3 gigabyte overhead). The benchmark results are posted in Hash Table Shootout on GitHub.

ContentLicense: CC BY-SA 4.0


问:
Manual Dependency Injection Examples Still Tightly Coupled
I've been using AutoFac for some time now to take care of the dependency injection we're using in a web project. All is well, but I now want to demonstrate DI outside of an AutoFac context for reasons irrelevant to this question.

In doing so, I'm having to read up on 'manual' methods of achieving this pattern. All of the examples in tutorials work (obviously), but the issue of tightly coupled objects still exist.

Take this for example: http://www.devx.com/dotnet/Article/34066/0/page/2

The constructor of the dependent class takes an interface:

```<code>public class BusinessFacade {
    private IBusinessLogic businessLogic;
    public BusinessFacade(IBusinessLogic businessLogic) {
        this.businessLogic = businessLogic;
    }
}
</code>```

But then the instantiation of BusinessFacade still requires a concrete implementation to be passed, which means an implementation needs to be instantiated:

```<code>IBusinessLogic productBL = new ProductBL();
BusinessFacade businessFacade = new BusinessFacade(productBL);
</code>```

Question

Now I know, somewhere at some point, something (whether it is a container or manually written code) will have to decide on the implementation to choose, but how can I come to terms with the benefit of moving such decision outside of the dependent class? Of course the BusinessFacade class is no longer tightly coupled, but we're still very explicit about which implementation we choose - so what's been gained here? Is it purely for the fact we can mock out the dependencies?

答:
Sometimes it helps to be really explicit:

BusinessFacade is tightly coupled to IBusinessLogic because it uses it in it's constructor signature

ProductBL is tightly coupled to IBusinessLogic because it implements the interface

The 2-line composer in your example is tightly coupled to IBusinessLogic, BusinessFacade and ProductBL because it creates objects of those types

So there still is a lot of tight coupling going on. That isn't surprising, DI does not intend to make everything loosely coupled. The remaining tight coupling makes sure that there are no runtime errors because of type mismatches.

Decoupling

The main goal of DI is to decouple components from the concrete dependencies that are used when the application is running.

Why would you want that? One possible reason is that you do not know the final applications in which it will run. Maybe because you want your components to support future applications that are not written yet, maybe applications are written by a third party.

Another aspect is that by having an interface as dependency it is much clearer what the component actually depends on: exactly the members of the interface. And you know that everything not explicitly specified in the interface can be changed without touching the code of your component, just by supplying another concrete dependency.

Compositon

I would argue that injecting dependencies even is worth it when you are injecting concrete dependencies - it allows the use of a single composition root for the whole application which documents the application structure.

But with concrete dependencies there isn't a lot you can change. If you are using components that depend on interfaces there is a lot you can easily tweak and adjust at the system composition layer. Even a simple facade with logging-functionality or a caching-component at the right spot can provide huge value with minimal to no changes to existing code.


问:
Swift: Creating an empty array
The following:

```<code>var emptyArray = [String]();
</code>```

What do I need the parentheses () for? [String] says, that the array accepts strings only. But why do I have to put the () there?

答:
Two things are true in Swift:

To instantiate an object of type T in Swift you write code T().

The type "name" of an array of Strings is [String]

Combine the two and you get [String]()

Title: Swift: Creating an empty array


问:
Big-O for Immutable Trees
How would you calculate the time and space complexity for a tree algorithm that creates a copy of a tree, but reuses as much of the original tree as possible?

For example,

```<code>   A
  /|\
 B C G
  /|\
 D E F
</code>```

To change E, we can take a more efficient approach, rather than copying the entire tree:

Copy E and change the value, call this E'

Copy C and point it to E' instead of E, call this C'

Copy A and point it to C', call this A'

The new tree looks like this.

```<code>   A'
  /|\
 B C'G
  /|\
 D E'F
</code>```

The time taken for updates completely depends which value you want to change and the number of parent nodes between it and the root, rather than the total number of items.

In a worst case scenario, to update a node, you must change h items, where h is the height (maximum depth) of the tree. Is it sane to express the complexity as O(h) (rather than in terms of n)?

If so, what would be a sane way to plot this on a graph that was using the y axis for n and the x axis for Big O? Include a secondary y axis as h? How would you show that h is likely to be far smaller than n?

答:
Is it sane to express the complexity as O(h) (rather than in terms of n)?

Absolutely. It's common and useful to state the big-O of an algorithm in terms of multiple variables.

How would you show that h is likely to be far smaller than n?

This depends on properties of the tree that you have not specified. ratchet freak has correctly stated that for balanced trees h is a logarithm of n, which is also the most useful answer I can give, so I'll attempt to provide the justification for that claim.

Let n be the number of nodes in the tree and a be the arity of the tree (i.e. if every node has at most 2 children, that means a is 2). A tree of height 1 can only have 1 node, and 1 < a. A tree of height 2 can have up to 1+a1 nodes, since the 1 node at height 1 can have up to a children, and 1+a < a2. A tree of height 3 can have up to 1+a1+a2 nodes, since each of the a level 2 nodes can have up to a children each, and 1+a1+a2 < a3. It should be clear that for an a-ary tree of height h, the maximum number of nodes n is between ah-1 and ah.

Justifying this last step rigorously can be quite tedious, but suffice it to say that the previous statement is essentially saying that n=Θ(ah), which is equivalent to saying that h=Θ(logan).

Of course, that holds for perfectly balanced trees. If the tree is not balanced at all then the worst case is h=O(n).

Title: Big-O for Immutable Trees