问:
Configuration file for a generic log file parser
Our company uses multiple log file formats.

We would like to develop a series of tools to parse them, often the same core functionality for multiple log file formats.

A classical example is generating Message Sequence Charts from the log files (other candidates are memory usage, stack size and time measurement between events).

To stick with Message Sequence Charts, we basically need to identify:

the title of the columns (i.e processes involved)

the title of the messages

perhaps also message parameters

These, we intend to determine by parsing the log files.

However, for different formats, we might say that the information is

delimited by certain strings (<some text> <process name> SEND <message> TO <process name>)

in fixed column positions

in the Nth word of lines beginning with a certain string

and so on

This could turn into a spaghetti of command line arguments, so we are thinking of an external configuration file.

What sort of file (plain text, .INI, XMl, or what) and how should we structure it?

I would hope that about 90% of the core of our scripts is common, and we just need 10% or so to massage the input to direct our parsing.

Any advice, or references?

答:
Do not reinvent the wheel, use something like AWK. So your "configuration file" is simply an awk script, which is powerful enough to deal with all the requirements you mentioned.

If that turns out to be not powerful enough, use your favorite scripting language like Perl or Python. And if you can live with a closed source Windows-only solution (freeware, but not FOSS), specificially designed for processing log files, you can try Microsoft's log parser.


问:
Converting static utility class into singleton
In company where I work we have lots of "utility" classes, each has lots of code inside (thousands of lines), and they are all static. And one static methods call anothers. The problem here is that when you want to test other classes you have lots of pain when tested function uses these static utility classes inside because you can not override them(as they are static). We are using mockito for testing. Powermock is not an option because it breaks bytecode sometimes and it's much slower than mockito.

What we decided - is to move all them to singletons, step by step, so when testing we can mock singleton instance and test method we want (or even test some utility method because they also need to be covered by autotests). It's simple to do that for small classes, but in our case there are 6006 lines class, 1261 lines class, 2231 lines class and so on. Converting them all into singletons manually is a lot of pain, so is there any way to do that automatically? Or maybe some helpers at least.

Example of untestable code:

```<code>static Object methodA()
{
      *very heavy code here*
}

static void methodB()
{
      *some code*
      xx = methodA();
      *here is code I've changed and want to test*
      *some code*
}
</code>```

Let's imagine I changed something in methodB and want to test my change. So I don't want this methodA to be called as it's not related to the test. But I can't change what methodA does with mockito in such case. However it is possible if inside methodA there is a call to the same singleton method that can be mocked because it is not static.

So if you convert it to something like this

```<code>public class A {
    private A(){}

    public Object methodAInner()
    {
        *heavy code here*
    }
    public void methodBInner()
    {
        *some code*
        xx = methodA();
        *here is code I've changed and want to test*
            *some code*
    }

    private static A getInstance()
    {
        return SpringUtils.getBean(A.class); // here can be anything that supports testing
    }

    static Object methodA()
    {
        return getInstance().methodAInner();
    }

    static void methodB()
    {
        getInstance().methodBInner();
    }
}
</code>```

You can make these SpringUtils return mocked object with real methodB and make almost no changes to existing code(useful for git blame), just remove static modifier and chnge methods names. An what is the most important for legacy - it allows not to change code that uses this utility class.

答:
I'm all for fighting fire with fire, but turning bad code (lots of static methods) into other bad code (lots of singletons) doesn't sound like such a good idea to spend time on really.

From what you have written, this is all in a legacy code context, but you do have change access to the source where those static methods reside (maybe all, but the rest is too large to touch upon). So let's take a look at where you want to go ultimately:

Why do you want to do that work at all? because the current code is untestable and makes new code hard to test as well

Why is it untestable? because of all the static methods

Why do we have those static methods? because they are used everywhere, have recursive calls (rely on each other), and hence are hard to get rid off

Why do they all rely on each other? because utility classes have a tendency to do a lot of things instead of a single responsibility

My next question and suggestion to you would then be: Why don't you extract one responsibility after the other?

You may have a tough time identifying a responsibility, but you're probably having a concrete use case already that made you aware of this problem in the first place, so that should be a good start.

Once you have a separate, single responsibility and testable class available, you can still use it within the old class to replace parts of that class. Maybe you still have some duplication, because you needed that other static method's code in your new class, but it's still called from some other? Then it's a pointer to your next work area - identify what the common thing is that the two places share and extract it out from both.

Due to the sheer size you portray, this is going to be a slow and long running process, so every small responsibility that's out of that class has to be considered a win. Let the old static method rely on the new class, deprecate it, eradicate it whenever you see a call site. These things take time until you get to a point, where you can finally make that last investment and hole your team up for a day or two to get rid of all the remaining old and problematic code. As long as you take care, that no one uses these old methods for any new or modified code, you're getting closer to that goal. I wish you good luck for the journey.

Addendum - based on the additional information that these utility classes differ between different branches and different teams work with each of these: In short, you are trying to shoot a running target. Given the size of your codebase, the task itself is hard enough for a stable target, but I cannot recommend you continue down that path, until you first stopped that movement. Talk with the other teams and get them on board. If they object in any way shape or form, an already hard task becomes pretty much impossible. Hopefully, they all share your pain (or at least not caring about that part of the codebase) and you can get back to a unified state of these classes by merging them all.

Yes, I do suggest to merge all these branches w.r.t. the utility classes. I know it is its own hell. You have but three choices: make it a common problem (and share the solution work!), completely sever yourself, or just let it be as is.

The severing means that your team on your branch flat out defines itself as a different product line. You stop merging with the other guys and roll your own instead. It may not work, and even if it does, I have seen this come back to hit everyone in the company a few years down the road.

Anyways, every problem that cannot be solved locally within your own team('s codebase) is unfortunately immediately a matter of company politics. It's simple to pick a fight with your teammates to get rid of a nasty class in your team's code, but it's worth a second thought on whether you really want to go to war involving the other teams/stakeholders/managers/etc.

ContentLicense: CC BY-SA 3.0


问:
How often do you run & test your code while programming?
Especially when writing new code from scratch in C, I find myself writing code for hours, even days without running the compiler for anything but an occasional syntax check.

I tend to write bigger chunks of code carefully and test thoroughly only when I'm convinced that the code does what it's supposed to do by analysing the flow in my head. Don't get me wrong - I wouldn't write 1000 lines without testing at all (that would be gambling), but I would write a whole subroutine and test it (and fix it if necessary) after I think I'm finished.

On the other side, I've seen mostly newbies that run & test their code after every line they enter in the editor and think that debuggers can be a substitute for carefulness and sanity. I consider this to be a lot of distraction once you've learned the language syntax.

What do you think is the right balance between the two approaches? Of course the first one requires more experience, but does it affect productivity positively or negatively? Does the second one help you spot errors at a finer level?

答:
For me it's not about how much I write. I can write thousands of lines of simple code without having to test it. But when I'm writing more difficult code I tend to test each function individually after having written a cohesive set of them.

Sometimes though, seeing your code running is a huge motivational boost, when you haven't run anything in a while it's good to see it working.

ContentLicense: CC BY-SA 2.5


问:
Lisp/Clojure: Removing unnecessary parentheses through conventions
I am fascinated to Lisp as it is simple yet powerful. I am just a beginner and I know there have been lots of discussions on removing parentheses from Lisp and its dialects. Yet I request Lisp ninja's to take few minutes answering this.

Will there be any side effects in Lisp/Clojure if they had followed 2 conventions below:

```<code>;; function declaration
defn function-name param-1 .... param-n
  ...
  function-body ;; not (function-body)
  ...

;; function call
function-name param-1 ... param-n ;; not (function-name param-1 ... param-n)
</code>```

While still using () or [] for inline and nested expressions.

```<code>(println "hello, ") (println "world !!!") ;; inline
= a (- 2 3) ;; nested
</code>```

答:
As mentioned by Chris, sweet expressions are what you are looking for.

with respect to Clojure, you can write indented code with a plugin called indent-clj.  You'd end up with something like this:

```<code>defn factorial [n]
  if (<= n 1)
    1
    (* n (factorial (- n 1)))
</code>```

Sweet expressions also mention infix, discussed here for Clojure Clojure has yet to use the #[...] reader form, so it is possible to use #[n <= 1] for infix expressions.  This would never get support, but here it is anyway:

```<code>defn factorial [n]
  if #[n <= 1]
    1
    #[n * (factorial #[n - 1])]
</code>```

A fully supported way to implement infix today is to use a macro of course, or even Clojure's tagged literal facility to create a custom infix tag, say #nfx (...).  It's a bit verbose, so I don't see any benefit over the normal way:

```<code>defn factorial [n]
  if #nfx (n <= 1)
    1
    #nfx (n * (factorial #nfx (n - 1)))
</code>```

Sweet expressions also introduce "neoteric" expressions, that is the traditional function call syntax f(...).  This seems impossible in Clojure, since spaces around collection delimiters are optional (e.g. (a(b)) is a valid list).

Body:
As mentioned by Chris, sweet expressions are what you are looking for.

with respect to Clojure, you can write indented code with a plugin called indent-clj.  You'd end up with something like this:

defn factorial [n]
  if (<= n 1)
    1
    (* n (factorial (- n 1)))

Sweet expressions also mention infix, discussed here for Clojure Clojure has yet to use the #[...] reader form, so it is possible to use #[n <= 1] for infix expressions.  This would never get support, but here it is anyway:

defn factorial [n]
  if #[n <= 1]
    1
    #[n * (factorial #[n - 1])]

A fully supported way to implement infix today is to use a macro of course, or even Clojure's tagged literal facility to create a custom infix tag, say #nfx (...).  It's a bit verbose, so I don't see any benefit over the normal way:

defn factorial [n]
  if #nfx (n <= 1)
    1
    #nfx (n * (factorial #nfx (n - 1)))

Sweet expressions also introduce "neoteric" expressions, that is the traditional function call syntax f(...).  This seems impossible in Clojure, since spaces around collection delimiters are optional (e.g. (a(b)) is a valid list).


问:
How to understand if a property is a member of a class or I have to create a different class that holds it?
Sometimes when you create a class you can add there several properties (new data members) that you are not certain if you want to do or not. For example, I have a casino slots game. I have tiles and tiles are spinning on different reels. So once 3 tiles come on the same line then player wins 3$, 4 tiles - 4$ and 5 tiles - 5$ for tile A and for tile B player wins 5$, 10$, 20$ accordingly. Should, for example, each tile store the data of its reward or there should be a reward manager for checking how many tiles are consecutive next to each other to give the reward to the player?

Please note that I don't want to have such a situation. But I find me many times thinking "Should I add this data, and consequently, corresponding logic the my class or not?". I worry about single responsibility principle when I want to have different managers for such things, but on the other hand I came to a situation to create several singletons or singleton-like classes.

答:
Should, for example, each tile store the data of its reward or there should be a reward manager for checking how many tiles are consecutive next to each other to give the reward to the player?

How I would do it

You have a Reel object that can be spun randomly

You have a number of Reward objects that have a threshold required to win, and a prize amount to give out (you can make the threshold and prize objects in of themselves)

The Reward objects register with the Reel objects to listen to results from spins (standard pub/sub)

When the Player spins each Reel object messages the result of its spin to the Reward object, which starts checking thresholds.

A Reward tracks the result as reels report in and if a threshold is matched (eg 3 cherry tiles, but no more) then the reward messages the Player object to increase the value of the players balance by the prize amount.

No need for a reward manager, each reward is an object itself. Managers are generally a sign your objects are not talking to each other properly. In this case the Reel should talk directly to the Reward.


答:
I find it extremely rare that one would need to mock the methods of a utility class. A very important OOP principle is that state is represented by objects, and that dictates that static methods - which don't operate on an object - should be stateless.

You don't need to mock functionality. If you are testing code that depends on your factorial implementation, you don't need to mock the factorial function. Not because it has simple implementation, but because you pay very much and gain very little from mocking it. The only thing you gain is the ability to test that code even when factorial is broken, which hardly worth all the mess you need to introduce to the code and all the programming principles you need to break to make this mocking possible. If you need to make sure that factorial(6) returns 720, instead of(syntax may not be accurate):

```<code>Factorial factorial = Mockito.mock(Factorial.class);
test.when(factorial.factorial(6)).thenReturn(720);
</code>```

Won't it be better to add to Factorial's unittests:

```<code>Assert.assertEquals(Factorial.factorial(6), 720);
</code>```

Now you can be sure that factorial(6) will return 720 and you don't need to mock it.

So, you don't mock functionality. What you do want to mock is resources. You mock a web service so the test won't do actual calls to the web service. You mock a database so the tests won't modify the actual data in your database. Rather than making your tests more complex(like mocking the factorial does), mocking these things make the tests simpler, because it relives you of the need to deploy a web service or a database from inside the unit test. They worth the trouble.

So, you only need to mock resources - and resources translate naturally to objects! You don't call a function in a web service using a static method - you have an object that represents the web service(and holds data like it's URL). By the very nature of resources you should be passing them around even if you don't intend to write unit tests, which means you don't need to mess up your code to mock them.

So, if these utility methods don't represent actions on resources - you don't need to mock them. And if they do - they shouldn't have been utility methods in the first place!

Title: Converting static utility class into singleton

Body:
In company where I work we have lots of "utility" classes, each has lots of code inside (thousands of lines), and they are all static. And one static methods call anothers. The problem here is that when you want to test other classes you have lots of pain when tested function uses these static utility classes inside because you can not override them(as they are static). We are using mockito for testing. Powermock is not an option because it breaks bytecode sometimes and it's much slower than mockito.

What we decided - is to move all them to singletons, step by step, so when testing we can mock singleton instance and test method we want (or even test some utility method because they also need to be covered by autotests). It's simple to do that for small classes, but in our case there are 6006 lines class, 1261 lines class, 2231 lines class and so on. Converting them all into singletons manually is a lot of pain, so is there any way to do that automatically? Or maybe some helpers at least.

Example of untestable code:

```<code>static Object methodA()
{
      *very heavy code here*
}

static void methodB()
{
      *some code*
      xx = methodA();
      *here is code I've changed and want to test*
      *some code*
}
</code>```

Let's imagine I changed something in methodB and want to test my change. So I don't want this methodA to be called as it's not related to the test. But I can't change what methodA does with mockito in such case. However it is possible if inside methodA there is a call to the same singleton method that can be mocked because it is not static.

So if you convert it to something like this

```<code>public class A {
    private A(){}

    public Object methodAInner()
    {
        *heavy code here*
    }
    public void methodBInner()
    {
        *some code*
        xx = methodA();
        *here is code I've changed and want to test*
            *some code*
    }

    private static A getInstance()
    {
        return SpringUtils.getBean(A.class); // here can be anything that supports testing
    }

    static Object methodA()
    {
        return getInstance().methodAInner();
    }

    static void methodB()
    {
        getInstance().methodBInner();
    }
}
</code>```

You can make these SpringUtils return mocked object with real methodB and make almost no changes to existing code(useful for git blame), just remove static modifier and chnge methods names. An what is the most important for legacy - it allows not to change code that uses this utility class.

Body:
I find it extremely rare that one would need to mock the methods of a utility class. A very important OOP principle is that state is represented by objects, and that dictates that static methods - which don't operate on an object - should be stateless.

You don't need to mock functionality. If you are testing code that depends on your factorial implementation, you don't need to mock the factorial function. Not because it has simple implementation, but because you pay very much and gain very little from mocking it. The only thing you gain is the ability to test that code even when factorial is broken, which hardly worth all the mess you need to introduce to the code and all the programming principles you need to break to make this mocking possible. If you need to make sure that factorial(6) returns 720, instead of(syntax may not be accurate):

Factorial factorial = Mockito.mock(Factorial.class);
test.when(factorial.factorial(6)).thenReturn(720);

Won't it be better to add to Factorial's unittests:

Assert.assertEquals(Factorial.factorial(6), 720);

Now you can be sure that factorial(6) will return 720 and you don't need to mock it.

So, you don't mock functionality. What you do want to mock is resources. You mock a web service so the test won't do actual calls to the web service. You mock a database so the tests won't modify the actual data in your database. Rather than making your tests more complex(like mocking the factorial does), mocking these things make the tests simpler, because it relives you of the need to deploy a web service or a database from inside the unit test. They worth the trouble.

So, you only need to mock resources - and resources translate naturally to objects! You don't call a function in a web service using a static method - you have an object that represents the web service(and holds data like it's URL). By the very nature of resources you should be passing them around even if you don't intend to write unit tests, which means you don't need to mess up your code to mock them.

So, if these utility methods don't represent actions on resources - you don't need to mock them. And if they do - they shouldn't have been utility methods in the first place!


问:
Refactoring wordy conditional tests
My first attempt at this question was too theoretical, so I've rewritten it with actual code. See the edit history if you care.

Supposing this logic, "suffering" from the arrow anti-pattern:

/**
 * Function:effective
 * ... the DateTime the org should be considered expired (or null if never).
 *
 * If a non-group org has a parent, inherits the parent's properties, and
 * the parent isn't a root org, return the parent's expiration date.
 * Otherwise, return the org's expiration date.
 *
 * Compare to <get>, which always returns the org's expiration date.
 */
public function effective() {
    if (! $this->org->isGroup()) {
        if (($parent = $this->org->getParentOrg()) instanceof OrgModel) {
            if ($this->org->isParentPropertyInherited()) {
                if (! $parent->isRoot()) {
                    return $parent->expiration()->effective();
                }
            }
        }
    }
    return $this->get();
}
```

<p>I have rewritten this a few other ways (including the "fail early" approach from this question), but the one that seems the most clear to me is:</p>

<pre class="lang-php prettyprint-override"><code>public function effective()
{
    $nonGroup  =               (! $this->org->isGroup());
    $hasParent = $nonGroup  && ($parent = $this->org->getParentOrg()) instanceof OrgModel;
    $inherits  = $hasParent && ($this->org->isParentPropertyInherited());
    $nonRoot   = $inherits  && (! $parent->isRoot());

    if ($nonRoot) {
        return $parent->expiration()->effective();
    } else {
        return $this->get();
    }
}
</code>```

In my opinion, the logic tests are compact and simple, but verbose because of the method names and the language overhead of calling methods.

I am reluctant to go with my second, re-written approach because it may be obscure and not hold up under maintenance.  On the other hand, I don't like the nested if.  I didn't like the fail-early approach, because it repeated code.

Are there other options I'm not considering here?

答:
This seems to disregard the nice and simple:

```<code>return complicated_test_1() && complicated_test_2() && complicated_test_3();
</code>```

This is certainly simpler than either of your options, and I assume you rejected it because:

the complicated_test are not single function calls, but themselves Boolean expressions

But this is a self-inflicted problem! You extracted out those boolean expressions into methods in your question for readability, so why not do the same thing in your real code? Just apply an extract method refactoring and pick some more expressive names than complicated_test_n, and you're good to go. And we didn't even need monads!

Title: Refactoring wordy conditional tests

Body:
This seems to disregard the nice and simple:

return complicated_test_1() && complicated_test_2() && complicated_test_3();

This is certainly simpler than either of your options, and I assume you rejected it because:

the complicated_test are not single function calls, but themselves Boolean expressions

But this is a self-inflicted problem! You extracted out those boolean expressions into methods in your question for readability, so why not do the same thing in your real code? Just apply an extract method refactoring and pick some more expressive names than complicated_test_n, and you're good to go. And we didn't even need monads!


问:
How is a Java reference different from a C pointer?
C has pointers and Java has what is called references. They have some things in common in the sense that they all point to something. I know that pointers in C store the addresses they point to. Do reference also store the address? How they are different except that pointer is more flexible and error-prone?</p>

答:
They are slightly different. In Java a copy of the reference is copied to the stack of a called function, pointing to the same object as the calling function and allowing you to manipulate that object. However you cannot change the object the calling function refers to.

Consider the following java code

```<code>public static void changeRValue(StringBuffer sb){
    sb = new StringBuffer("helllllo"); /*attempt to assign the reference
                                        to a new object*/
}
public static void main(String[] args) {
    StringBuffer sb = new StringBuffer("hi");     //Create a new string buffer
    changeRValue(sb);                             //Call changeRValue
    System.out.println(sb.toString());            //Prints "hi" not "hello"
}
</code>```

Now consider a pointer in c++:

```<code>void func(Dog* dog){
    *dog = Dog("hello world"); //Change the value of dog to a new object
}

int main(int argc, const char * argv[]) {
    Dog dog1("hi");                            //Create a dog object
    func(&dog1);                               //pass the address of dog
    cout << dog1.name;                         //Prints "hello world" not hi.
    return 0;
}
</code>```

Body:
They are slightly different. In Java a copy of the reference is copied to the stack of a called function, pointing to the same object as the calling function and allowing you to manipulate that object. However you cannot change the object the calling function refers to.

Consider the following java code

public static void changeRValue(StringBuffer sb){
    sb = new StringBuffer("helllllo"); /*attempt to assign the reference
                                        to a new object*/
}
public static void main(String[] args) {
    StringBuffer sb = new StringBuffer("hi");     //Create a new string buffer
    changeRValue(sb);                             //Call changeRValue
    System.out.println(sb.toString());            //Prints "hi" not "hello"
}

Now consider a pointer in c++:

void func(Dog* dog){
    *dog = Dog("hello world"); //Change the value of dog to a new object
}

int main(int argc, const char * argv[]) {
    Dog dog1("hi");                            //Create a dog object
    func(&dog1);                               //pass the address of dog
    cout << dog1.name;                         //Prints "hello world" not hi.
    return 0;
}


问:
Clean Abstract Syntax Tree
I'm writing a toy compiler for fun.

Basically, my problem is that I don't want to clutter the AST with stuff like debug information (symbol tokens, locations of tokens, etc) as well as data that the semantic analyzer computes.

For example, the semantic analyzer does some type inference and the result type is stored back into the Node type. It looks something like this:

```<code>/// A variable declaration in my language looks like this:
/// var x = 10
struct VarDec: Statement {
    var varKeyword: Keyword
    var varName: Id
    var assignment: Symbol
    var initializer: Expr
    var type: Optional<Type>
}
</code>```

The main problem here is that now the node type has a state, type inference has either been done or not. This makes it hard to reason about. The other thing is that the whole AST gets cluttered with those tokens that are only really needed for error messages.

It has been suggested that I could create another representation of the AST just for semantic analysis that links back to the AST but that seems like a lot of work and redundant code...

Does anybody have an idea how I can clean up this code without having to create multiple trees?

答:
It has been suggested that I could create another representation of the AST just for semantic analysis that links back to the AST but that seems like a lot of work and redundant code...

I don't think it means a lot of work and a lot of redundant code.

Your typed AST could mostly contain the type information, and a reference (e.g. a pointer) to the syntactic AST.

You won't have a lot of extra work: you'll need to compute the type information anyway.

You probably won't have a lot of redundant code: each AST node (so each kind of such node) needs to be processed.

FWIW, the Ocaml compiler and the GCC compiler are having several internal representations (often tree like) of the compiled source code.

An alternative might be to have several quite large associative tables mapping syntactic AST to computed attributes (including types) and grow them progressively. I am not sure it would be better (and I don't know of compilers using that, except perhaps some Prolog implementations).

Title: Clean Abstract Syntax Tree


问:
Where should I put my key?
I have a Private Key that is regularly used to decode incoming files during off-business hours as part of an automated process.

Currently, it is stored in the application code itself, and only ever used for this process - but this is probably not a good idea, even though our application is internal-facing only.

Where should we store our private key so that it cannot be abused, but can still be used regularly to decode incoming documents?

答:
A best practice about private keys is that they should be time-limited and regularly updated. So there should be a way to synchronize the new public key to the senders. This suggests that the private key should be in the configuration or in a file argument.

Security is all about trust, so the exact answer depends on which colleagues you trust, how the senders will trust the new public keys, who can see the password that protects the private key. Who administers the machine where this application runs?

Title: Where should I put my key?


问:
Cross-language Test-Driven Development
The short question: How do you follow Test-Driven Development on a project that spans multiple languages?

Specifically, I'm writing a web application that uses JavaScript and PHP, and I want to follow the TDD principles, but I'm not sure how to integrate them. Do I run separate test suites for the JS and PHP sections, and use mocks in the JS suite to emulate server responses? Is there a technique for unit testing both components in one run?

This is my first experience using Test-Driven Development, so any advice you can share on how to make it less daunting would be great. The reason I chose it is that as soon as I finished a prototype, the requirements changed, forcing me to change my design. I figured if I'm starting over, I'd like to write more extensible code with built-in regression testing from the start.

I'm writing my PHP tests in SimpleTest and my JavaScript tests in JsTestDriver. I'm used to object-oriented paradigms, so I've got a few classes in PHP, and I'm doing the something similar in JavaScript using prototypal inheritance. I've also started reading this book about TDD in Python and this one about TDD in JavaScript, but from all that I've seen, these don't describe testing an application in full (outside of using something like Selenium or another web driver to perform front-end acceptance testing. Is TDD just not cut out for full-stack developers?

答:
Is there a technique for unit testing both components in one run?

That would actually be the opposite of unit testing - unit testing, especially in TDD style, means to test your components in isolation. Thus the answer is yes, "run separate test suites for the JS and PHP sections", otherwise it is not unit testing and not TDD.

Of course, automated integration tests can test "both components in one run", and you can utilize exactly the tools you already mentioned (like Selenium). But those are typically more complex tests, developed outside the TDD cycles.

Title: Cross-language Test-Driven Development


问:
Would a statically typed alternative to JavaScript on webpages be practical?
Preference for dynamic and static typing is largely a matter of taste, and different people find them more or less suitable in different situations.

My question is, would it be technically possible to have a statically-typed alternative to JavaScript for client-side web page augmentation etc.?

答:
Given you have no hope in getting all browsers that are used in the real world to support a new language; the language will have to compile down to jscript.

As all the examples of the web are in jscript, the language should mostly look like jscript.

I think there is scope with have a “sub set” of jscript that is check by a static checker but is also valid jscript.  E.g:

All variables must have a comment that say there types before the first usage.

All usages of variables must be valid with the above.

Functions/class may not be used if they have not be declared in a comment#

A comment at the top of the js file must list all other js files that it depends on.


答:
The important thing is to distinguish between TDD and ATDD. The AT there stands for "acceptance tests", and this refers to development where you first start with an acceptance test, which is likely to test the entire stack. This is also sometimes called "outside-in test driven development". When people talk about TDD, the "T" there probably refers specifically to unit tests.

A key part of unit tests is isolating the unit under test from its dependencies. This gives you two very important benefits:

You can make your tests extremely fast, so you can run them very often as part of a very short feedback cycle. Rather than running them, say, every hour, you should be able to run all your unit tests after every little change, so you instantly get their feedback on whether that change broke anything.

Your tests can be targeted at once very specific behavior. If one of them fails, you should be able to determine almost instantly the exact nature of the bug that the test is indicating.

Because of the importance of this isolation, you will automatically want tests to be restricted to much smaller units than your language boundaries force you into. Though it's not a hard and fast rule, you would often expect a single class to be a testable unit. So in terms of TDD, the language issue is more or less irrelevant.

If, on the other hand, you want to do ATDD, make sure you look into resources for this specifically (it's often done as part of BDD, so look at tools targeted at that, too). Here's where something like Selenium would naturally fit in. Usually, when doing ATDD you still write unit tests, and in fact in order to pass each acceptance test you might test drive its implementation with unit tests too. So even if you do want to do ATDD, understanding how to write unit tests is still important.

Title: Cross-language Test-Driven Development

Body:
The short question: How do you follow Test-Driven Development on a project that spans multiple languages?

Specifically, I'm writing a web application that uses JavaScript and PHP, and I want to follow the TDD principles, but I'm not sure how to integrate them. Do I run separate test suites for the JS and PHP sections, and use mocks in the JS suite to emulate server responses? Is there a technique for unit testing both components in one run?

This is my first experience using Test-Driven Development, so any advice you can share on how to make it less daunting would be great. The reason I chose it is that as soon as I finished a prototype, the requirements changed, forcing me to change my design. I figured if I'm starting over, I'd like to write more extensible code with built-in regression testing from the start.

I'm writing my PHP tests in SimpleTest and my JavaScript tests in JsTestDriver. I'm used to object-oriented paradigms, so I've got a few classes in PHP, and I'm doing the something similar in JavaScript using prototypal inheritance. I've also started reading this book about TDD in Python and this one about TDD in JavaScript, but from all that I've seen, these don't describe testing an application in full (outside of using something like Selenium or another web driver to perform front-end acceptance testing. Is TDD just not cut out for full-stack developers?


答:
At first glance, your 'clear' approach does look compact and clean, but if you were to unravel the statements...

```<code>// PHP?
public function effective()
{
    $nonGroup  = !$this->org->isGroup();
    $hasParent = false;
    if ($nonGroup) {
        $hasParent = ($parent = $this->org->getParentOrg()) instanceof OrgModel;
    }
    $inherits  = false;
    if ($hasParent) {
        $inherits  = $this->org->isParentPropertyInherited();
    }
    $nonRoot   = false;
    if ($inherits) {
        $nonRoot   = !$parent->isRoot();
    }
    if ($nonRoot) {
        return $parent->expiration()->effective();
    } else {
        return $this->get();
    }
}
</code>```

The use of temporary variables here just to simplify the arrow anti-pattern manifests into another problem, which is what I characterize as weak standalone if statements that doesn't communicate as clearly the original logic. If I am literally reading this:

nonGroup checks if this org is a non-group.

if non-group, hasParent checks if this parent is an instance of OrgModel.

if has-parent, inherits checks if parent property is inherited.

if inherits, nonRoot checks if parent is non-root.

if non-root, return the parent's effective expiration.

otherwise, the org's expiration date.

This means I have to remember four temporary variables, when are they 'skipped' (and given the default false value), and am also unable to see the linkage from one condition to the other, described clearly in your original code documentation:

If a non-group org has a parent,

inherits the parent's properties, and

the parent isn't a root org,

return the parent's expiration date.

Otherwise, return the org's expiration date.

You didn't reveal the fail-early approach you picked, but this is how I might attempt it (pardon my extremely rusty PHP skills):

```<code>public function effective() {
    $org = $this->org;
    if ($org->isGroup()) {
        return $this->get();
    }
    $parent = $org->getParentOrg();
    return $parent instanceof OrgModel
            && $org->isParentPropertyInherited()
            && !$parent->isRoot() ? $parent->expiration()->effective() : $this->get();
}
</code>```

The only unnecessary temporary variable I created here is $org, but that is just so that I don't have to repeat $this->org.

A further refinement of your codebase, is for isParentPropertyInherited() do the instanceof check internally as well, if appropriate. This can greatly simplify the body to just:

```<code>public function effective() {
    return !($org = $this->org)->isGroup()
            && $org->isParentPropertyInherited()
            && !($parent = $org->getParentOrg())->isRoot()
            ? $parent->expiration()->effective() : $this->get();
}
</code>```

This is not far-fetched since if the org inherits a parent's properties, it implies the parent exists.

Title: Refactoring wordy conditional tests

Body:
My first attempt at this question was too theoretical, so I've rewritten it with actual code. See the edit history if you care.

Supposing this logic, "suffering" from the arrow anti-pattern:

/**
 * Function:effective
 * ... the DateTime the org should be considered expired (or null if never).
 *
 * If a non-group org has a parent, inherits the parent's properties, and
 * the parent isn't a root org, return the parent's expiration date.
 * Otherwise, return the org's expiration date.
 *
 * Compare to <get>, which always returns the org's expiration date.
 */
public function effective() {
    if (! $this->org->isGroup()) {
        if (($parent = $this->org->getParentOrg()) instanceof OrgModel) {
            if ($this->org->isParentPropertyInherited()) {
                if (! $parent->isRoot()) {
                    return $parent->expiration()->effective();
                }
            }
        }
    }
    return $this->get();
}
```

<p>I have rewritten this a few other ways (including the "fail early" approach from this question), but the one that seems the most clear to me is:</p>

<pre class="lang-php prettyprint-override"><code>public function effective()
{
    $nonGroup  =               (! $this->org->isGroup());
    $hasParent = $nonGroup  && ($parent = $this->org->getParentOrg()) instanceof OrgModel;
    $inherits  = $hasParent && ($this->org->isParentPropertyInherited());
    $nonRoot   = $inherits  && (! $parent->isRoot());

    if ($nonRoot) {
        return $parent->expiration()->effective();
    } else {
        return $this->get();
    }
}
</code>```

In my opinion, the logic tests are compact and simple, but verbose because of the method names and the language overhead of calling methods.

I am reluctant to go with my second, re-written approach because it may be obscure and not hold up under maintenance.  On the other hand, I don't like the nested if.  I didn't like the fail-early approach, because it repeated code.

Are there other options I'm not considering here?

Body:
At first glance, your 'clear' approach does look compact and clean, but if you were to unravel the statements...

// PHP?
public function effective()
{
    $nonGroup  = !$this->org->isGroup();
    $hasParent = false;
    if ($nonGroup) {
        $hasParent = ($parent = $this->org->getParentOrg()) instanceof OrgModel;
    }
    $inherits  = false;
    if ($hasParent) {
        $inherits  = $this->org->isParentPropertyInherited();
    }
    $nonRoot   = false;
    if ($inherits) {
        $nonRoot   = !$parent->isRoot();
    }
    if ($nonRoot) {
        return $parent->expiration()->effective();
    } else {
        return $this->get();
    }
}

The use of temporary variables here just to simplify the arrow anti-pattern manifests into another problem, which is what I characterize as weak standalone if statements that doesn't communicate as clearly the original logic. If I am literally reading this:

nonGroup checks if this org is a non-group.

if non-group, hasParent checks if this parent is an instance of OrgModel.

if has-parent, inherits checks if parent property is inherited.

if inherits, nonRoot checks if parent is non-root.

if non-root, return the parent's effective expiration.

otherwise, the org's expiration date.

This means I have to remember four temporary variables, when are they 'skipped' (and given the default false value), and am also unable to see the linkage from one condition to the other, described clearly in your original code documentation:

If a non-group org has a parent,

inherits the parent's properties, and

the parent isn't a root org,

return the parent's expiration date.

Otherwise, return the org's expiration date.

You didn't reveal the fail-early approach you picked, but this is how I might attempt it (pardon my extremely rusty PHP skills):

public function effective() {
    $org = $this->org;
    if ($org->isGroup()) {
        return $this->get();
    }
    $parent = $org->getParentOrg();
    return $parent instanceof OrgModel
            && $org->isParentPropertyInherited()
            && !$parent->isRoot() ? $parent->expiration()->effective() : $this->get();
}

The only unnecessary temporary variable I created here is $org, but that is just so that I don't have to repeat $this->org.

A further refinement of your codebase, is for isParentPropertyInherited() do the instanceof check internally as well, if appropriate. This can greatly simplify the body to just:

public function effective() {
    return !($org = $this->org)->isGroup()
            && $org->isParentPropertyInherited()
            && !($parent = $org->getParentOrg())->isRoot()
            ? $parent->expiration()->effective() : $this->get();
}

This is not far-fetched since if the org inherits a parent's properties, it implies the parent exists.


问:
What's the difference between the terms "protocol" and "standard"?
I find the term "protocol" confusing (in the terms of computer science that is). If the protocol is just a set of rules, wouldn't it be easier if we used the term "standard" instead (like in "HTTP standard")?</p>

答:
Communication protocols are, by definition, sets of rules that govern the formats and interactions between communicating parties. These rules may be implicit and informal, as in typical everyday social interactions between people. But electronics, and digital computers especially, require things to be far more explicit and formalized in order for reliable communication to occur.

A Standard in this context is an attempt to resolve an area of potential misunderstanding or disagreement. Communication protocols are one of many areas that standards may apply to. A notable set of standards that are not communication protocols, for example, are the SI weight and measurement standards. These provide a fixed reference to which you can compare any given quantity in order to measure it in a way that will be unambiguous to everyone with access to the standard, or a reasonable approximation of it.

A Communication Protocol Standard is therefore a formalization of the rules of a communication protocol such that those with access to the standard can (ideally) unambiguously determine whether any particular attempt at communication complies with those rules. Just as comparing a particular mass of metal against the SI standard kilogram will determine how close the mass is to a kilogram, comparing a particular protocol implementation against a protocol standard will determine whether it is truly following that protocol according to the standard. When all parties are properly following the protocol as formalized in the standard, it is (again, ideally) guaranteed to result in the level of communication that the protocol was designed to accommodate.

A Communication Protocol Specification is a means by which the rules and formats of a communication protocol can be described at a formal level, and it is often part of (indeed, the largest part of) a communication protocol standard. While the purpose of a standard is to create a means to determine whether something is truly an instance of the standardized thing, the purpose of a specification is to define exactly what the thing is in the case where the thing is defined by rules and formats, as a communication protocol is. An unambiguous definition of what something is can be used both to create a new instance of that thing and to determine whether an existing thing is an instance of the thing being defined.

So a specification can be used as a standard, but being a standard also carries the weight of some social agreement that there is value in complying with the specification well enough for multiple implementations to have some assurance of working together. The parties interested in this agreement work together to form a Standards Body in order to define the standards that they will agree on for the purpose of their desire to be able to rely on compliance to a protocol. Without the presence of this social agreement and the standards body by which the standards are produced, protocol specifications aren't generally considered to constitute standards, though well-defined specifications definitely have the potential to be used as standards.


问:
OOP Principles in Customer and Account classes of a Bank
I am designing a Bank Application which deals with Customers having FD Accounts.

There are two approaches to this :

Approach 1

In this the customer has a list of accounts. The bank application will manage the accounts through the customers.

My mentor feels this is the correct approach. :-P

```<code>class Customer
{
     String name;
     Date   dob;
     .
     .

     Account[] accounts = new Account[5];
}

class Account
{
     int   accNo;
     float amount;
     .
     .
}
</code>```

Approach 2

In this approach each account has its associated customer reference stored. The Bank application will now manage the accounts and customers independently.

I feel this is the correct approach. :-)

```<code>class Customer
{
     String name;
     Date   dob;
     .
     .
}

class Account
{
     Customer customer;
     int   accNo;
     float amount;
     .
     .
}
</code>```

Now my issues are :

Ideally the bank application should worry more about the account than the customer. For example, the bank should have methods like
addAccount() : which will add account and create a customer implicitly if it is the first account of the customer and
deleteAccount() : which will delete the account and delete the customer implicitly if it is the last account.

There can be a use case in the future where an account can have multiple customers. So that can be accommodated easily in the second approach than the first.

Considering all this, please can you say which is a better Object Oriented Approach here?

Also how is this actually implemented in a real Bank?

答:
Ideally the bank application should worry more about the account than the customer. For example, the bank should have methods like addAccount() : which will add account and create a customer implicitly if it is the first account of the customer and deleteAccount() : which will delete the account and delete the customer implicitly if it is the last account.

What is your definition of 'worry'? :)

As mentioned in @James Anderson's answer, the activities of customers and accounts are monitored, so it will not be customary if accounts' records are physically wiped out from your persistence storage (e.g. databases, or possibly simply flat files if this is actually just a small academic assignment borrowing a bank's context) the moment deleteAccount() is performed.

What you can probably do here is to introduce temporal fields that can mark the start and end timestamps of the record, whether the record is logically considered inactive, or both (i.e. four fields). The final option may make sense if you want to track customers/accounts that are temporarily suspended, with the option that they can be activated later. E.g. a simplified example on a timescale:

```<code>01/01 ------------ 04/30 -------------- 08/31 -------------- 12/31
* Account created  * Account            * Account            * Account
  and active         suspended            active               inactive
</code>```

There can be a use case in the future where an account can have multiple customers. So that can be accommodated easily in the second approach than the first.

True... to some extent. I only hope the following line is only meant to signify the intent to use a multi-element data structure, not a hard limit of 5 accounts:

```<code> Account[] accounts = new Account[5];
</code>```

An alternative approach is to simply store references in both domain classes, e.g. in Java:

```<code>public class Customer {
    // ...
    Set<Account> accounts = new HashSet<>();
}

public class Account {
    // ...
    Set<Customer> customers = new HashSet<>();
}
</code>```

In this way, you can easily identify all the accounts of a single Customer, or all the associated Customers of an Account. To refine this even further, maybe these relationships can be stored as Map<Account, AccountType> and Map<Customer, CustomerType> associations too.

Body:
Ideally the bank application should worry more about the account than the customer. For example, the bank should have methods like addAccount() : which will add account and create a customer implicitly if it is the first account of the customer and deleteAccount() : which will delete the account and delete the customer implicitly if it is the last account.

What is your definition of 'worry'? :)

As mentioned in @James Anderson's answer, the activities of customers and accounts are monitored, so it will not be customary if accounts' records are physically wiped out from your persistence storage (e.g. databases, or possibly simply flat files if this is actually just a small academic assignment borrowing a bank's context) the moment deleteAccount() is performed.

What you can probably do here is to introduce temporal fields that can mark the start and end timestamps of the record, whether the record is logically considered inactive, or both (i.e. four fields). The final option may make sense if you want to track customers/accounts that are temporarily suspended, with the option that they can be activated later. E.g. a simplified example on a timescale:

01/01 ------------ 04/30 -------------- 08/31 -------------- 12/31
* Account created  * Account            * Account            * Account
  and active         suspended            active               inactive

There can be a use case in the future where an account can have multiple customers. So that can be accommodated easily in the second approach than the first.

True... to some extent. I only hope the following line is only meant to signify the intent to use a multi-element data structure, not a hard limit of 5 accounts:

 Account[] accounts = new Account[5];

An alternative approach is to simply store references in both domain classes, e.g. in Java:

public class Customer {
    // ...
    Set<Account> accounts = new HashSet<>();
}

public class Account {
    // ...
    Set<Customer> customers = new HashSet<>();
}

In this way, you can easily identify all the accounts of a single Customer, or all the associated Customers of an Account. To refine this even further, maybe these relationships can be stored as Map<Account, AccountType> and Map<Customer, CustomerType> associations too.


答:
The main problem here is that now the node type has a state, type inference has either been done or not. This makes it hard to reason about.

State isn't that hard to reason about - mutable state is hard to reason about.

And you don't need mutable state to represent a syntax tree pre-inference and post-inference, you just need a placeholder type that represents and instance of "some as of yet unknown type". When you do inference, the compiler then returns new nodes with the inferences replaced with real types.

This gets you the benefit of having multiple trees without the code having multiple trees, and without the general problems that arise from mutable state.

Title: Clean Abstract Syntax Tree

Body:
I'm writing a toy compiler for fun.

Basically, my problem is that I don't want to clutter the AST with stuff like debug information (symbol tokens, locations of tokens, etc) as well as data that the semantic analyzer computes.

For example, the semantic analyzer does some type inference and the result type is stored back into the Node type. It looks something like this:

```<code>/// A variable declaration in my language looks like this:
/// var x = 10
struct VarDec: Statement {
    var varKeyword: Keyword
    var varName: Id
    var assignment: Symbol
    var initializer: Expr
    var type: Optional<Type>
}
</code>```

The main problem here is that now the node type has a state, type inference has either been done or not. This makes it hard to reason about. The other thing is that the whole AST gets cluttered with those tokens that are only really needed for error messages.

It has been suggested that I could create another representation of the AST just for semantic analysis that links back to the AST but that seems like a lot of work and redundant code...

Does anybody have an idea how I can clean up this code without having to create multiple trees?


答:
Yes a protocol is a set of rules.

That does not mean any set of rules is a protocol.

A law is set of rules that I would not call protocol.

HTML is set of rules for format that I would not call a protocol.

A standard is something that is widely adopted.

A standard is not limited to protocol or rules.

A 16" rim for car is something I would call a standard.

A communications protocol is a set of rules specifically for communication.  It can be standard or proprietary. If there is a standards body then it is definitely a standard.

You also have Open.  Open is published and may or may not be a standard.  I may have an Open proprietary protocol meaning here it is and here is out I works so that it can be used but I am not proposing it to be an industry standard.

It can even be a standard and proprietary.  In the early days PDF format was proprietary but it was still presented as standard.

In summary protocol and standard are just plain not the same thing.

Title: What's the difference between the terms "protocol" and "standard"?

Body: I find the term "protocol" confusing (in the terms of computer science that is). If the protocol is just a set of rules, wouldn't it be easier if we used the term "standard" instead (like in "HTTP standard")?</p>


答: You can use languages such as haXe to write your code in a statically typed manner and export it to javascript. JavaScript is becoming very fast, so it is sufficient as an output language. Trying to enforce a statically typed language as a web standard is near to impossible. Attempts to introduce static typing into JavaScript have failed for reasons to broad to discuss.</p>

Title: Would a statically typed alternative to JavaScript on webpages be practical?

Body:
Preference for dynamic and static typing is largely a matter of taste, and different people find them more or less suitable in different situations.

My question is, would it be technically possible to have a statically-typed alternative to JavaScript for client-side web page augmentation etc.?


问:
Why Session has implemented this way in Asp.net
I'm not a Asp.net/C# developer myself but recently had to slightly refactor a Asp.net/C# project and I had to use Session at some point. When working with the session, regardless of the data type that I was specifying for the session variable, I had to cast the session variable any time I needed to access it.

```<code>Session["index"] = 0;
Session["index"] = Session["index"] + 1; // throws exception
Session["index"] = ((int)Session["index"]) + 1; // it works here
</code>```

Asking my C# developer colleagues, it seems that the Session is somehow an object that covers up the actual data type of the variable, apparently without losing it, but for some reason you still need to cast it anyway any time you want to access it.

I'm not sure if that's the case or not or even if I have understood it properly, but what I don't understand is why it has to be implemented in a way that the developer has to cast it before using it. As my colleagues say, that's because you can basically have a dynamically typed session object, at run-time.

What I'm thinking is, first, why you need to have such a feature only for the session, in a statically typed language like Asp.net/C#? For example, what is the use case of defining a "session" variable as an integer, but then put string on it at the run-time? Is there any practical use case and requirement for such a feature, not only in Asp.net but any other language?

Second, if we assume that the session variables also has to be statically typed, then is it "technically" possible to implement Asp.net's session in a way that when you define a session variable like below, the compiler allocates the memory in the heap, keep the data type next to it and simply eliminates the need for casting it each time?

```<code>Session["index"] = 0; // integer type, available at compile-time
</code>```

What would happen if you try to assign a string to this integer variable? Well, like everything else in a statically typed language, you will get an exception.

Am I missing anything here? I simply can't understand why such a design decision has been made by the Asp.net development team and how the developer could benefit from that. Also wondering if there is anything related to how other languages like Java or C++ have implemented sessions, either directly in the language itself or via a library, so it led the Asp.net team as well to implement session like that?

Update #1

Just to clear the air, my main question is why the Session has implemented in this way? What are the benefits of having a session like this -- whatever it is, rather how it has implemented -- even though I'm looking to know that as well.

Update #2

Regarding @JacquesB comment: "You have to be able to save all kinds of different objects in a Session, so the type has to be Object, there is no other option"; Why the following code:

```<code>Session["index"] = 0;
Session["name"] = "abc";
</code>```

Can't be translated, by compiler, at the compile-time, to something like this:

```<code>int SessionIndex = 0;
string SessionName = "abc";
</code>```

I don't get why "there is no other option". It's a computer program after all, not anything magical, or am I still missing something?

答:
Session is effectively a map/dictionary with a key that is a string, and a value that is an object.  It predates generic types in C#/ASP.Net, and would more likely be an IDictionary<string,dynamic> today, which would support what you are trying to do.

```<code>//you cannot do this either...
//default from a session is an object/null, not even an integer
object foo = Session["index"];
foo = foo + 1;
</code>```

If you're using newer (as of several years ago) C#, with support for the as keyword, you should probably do something like...

```<code>Session["index"] = (Session["index"] as int?).GetValueOrDefault() + 1;
</code>```

Which will cast the result from Session["index"] to a nullable integer (or null if there's a mismatch), taking the default value or instance value if matching, then adding 1.

As an aside, in general don't rely on state in a Session if you can avoid it, it's usually better to rely on client-side localStorage or sessionStorage and/or include said data in any requests that actually need it.  Session use itself serialize all concurrent requests for a given user/session.  Not to mention the need for a session server/store if you are scaling beyond a single server.

Body:
Session is effectively a map/dictionary with a key that is a string, and a value that is an object.  It predates generic types in C#/ASP.Net, and would more likely be an IDictionary<string,dynamic> today, which would support what you are trying to do.

//you cannot do this either...
//default from a session is an object/null, not even an integer
object foo = Session["index"];
foo = foo + 1;

If you're using newer (as of several years ago) C#, with support for the as keyword, you should probably do something like...

Session["index"] = (Session["index"] as int?).GetValueOrDefault() + 1;

Which will cast the result from Session["index"] to a nullable integer (or null if there's a mismatch), taking the default value or instance value if matching, then adding 1.

As an aside, in general don't rely on state in a Session if you can avoid it, it's usually better to rely on client-side localStorage or sessionStorage and/or include said data in any requests that actually need it.  Session use itself serialize all concurrent requests for a given user/session.  Not to mention the need for a session server/store if you are scaling beyond a single server.


问:
Readiness vs. Completion Async IO Memory usage?
I was watching this talk about implementing Async IO in Rust and Carl mentions two potential models. Readiness and Completion.

Readiness Model:

you tell the kernel you want to read from a socket

do other things for awhile…

the kernel tells you when the socket is ready

you read (fill a buffer)

do whatever you need

free the buffer (happens automatically with Rust)

Completion Model:

you allocate a buffer for the kernel to fill

do other things for awhile…

the kernel tells you when the buffer has been filled

do whatever you need to with the data

free the buffer

In Carl's example of using the readiness model you could iterate over ready sockets filling and freeing a global buffer which makes it seem like it would use much less memory.

Now my assumptions:

Under the hood (in kernel space) when a socket is said to be "ready", the data already exists. It has come into the socket over the network (or from wherever) and the OS is holding onto the data.

It's not as if that memory allocation magically doesn't happen in the readiness model. It's just that the OS is abstracting it from you.
In the Completion model, the OS is asking you to allocate memory before data actually flows in and it's obvious what's happening.

Here's my amended version of the Readiness Model:

you tell the kernel you want to read from a socket

do other things for awhile…

AMENDMENT: data comes in to the OS (some place in kernel memory)

the kernel tells you the socket is ready

you read (fill another buffer separate from the abover kernel buffer (or you get a pointer to it?))

do whatever you need

free the buffer (happens automatically with Rust)

/My assumptions

I happen to like keeping the user-space program small but I just wanted some clarification on what is, in reality, happening here. I don't see that one model would inherently use less memory or support a higher level of concurrent IO. I'd love to hear thoughts and deeper explanation of this.

答:
In the Readiness Model memory consumption is proportional to the amount of data unconsumed by the application.

In the Completion Model the memory consumption is proportional to the amount of outstanding socket calls.

If there are many sockets that are mostly idle then the Readiness Model consumes less memory.

There is an easy fix for the Completion Model: Initiate a read of 1 byte. This consumes only a tiny buffer. When the read completes issue another (maybe synchronous) read that get the rest of the data.

In some languages the Completion Model is extremely simple to implement. I consider it to be a good default choice.


问:
JIT compiler for C, C++, and the likes
Is there any just-in-time compiler out there for compiled languages, such as C and C++? (The first names that come to mind are Clang and LLVM! But I don't think they currently support it.)

Explanation:

I think the software could benefit from runtime profiling feedback and aggressively optimized recompilation of hotspots at runtime, even for compiled-to-machine languages like C and C++.

Profile-guided optimization does a similar job, but with the difference a JIT would be more flexible in different environments. In PGO you run your binary prior to releasing it. After you released it, it would use no environment/input feedbacks collected at runtime. So if the input pattern is changed, it is probe to performance penalty. But JIT works well even in that conditions.

However I think it is controversial wether the JIT compiling performance benefit outweights its own overhead.

答:
Firstly, I assume you'd want a tracing jit rather than a method jit.

The best approach to take would be compiling the code to llvm IR, then adding in tracing code, before producing a native executable. Once a code block becomes sufficiently well used and once enough information about the values (not the types like in dynamic languages) of variables has been collected then the code can be recompiled (from the IR) with guards based on values of the variables.

I seem to remember there was some progress on making a c/c++ jit in clang under the name libclang.

Title: JIT compiler for C, C++, and the likes


问:
How to deal with relative paths in out of source build?
An out of source build is a build which can be in any directory other than the source directory

I have a project which contains relative paths to some resource files, needed at runtime, which are located close to the source directory.

But when I build the project in an arbitrary folder, the resource files are (obviously) not found.

What is the proper way of providing paths to resource files?

答:
I assume that your program is coded in a compiled language (that is a language implemented in a compiler) like C or C++ or Ocaml, and that your operating system is Linux (or perhaps some other POSIX compliant OS).

First, you could embed resources inside your executable. For that, you would compile a large constant byte array (à la XBM), e.g. in C const char resbytes[] = { ... some very long sequence of bytes, e.g. 23, 97, etc... };  . That source file could obviously be generated by some shell script (perhaps with the help of hexdump(1), etc...)

Then, you could decides that resources are installed at some fixed (or compile-time configurable) path, outside of the build tree. This is the case for most GNU software (like gcc, bash, emacs, make etc...), see the directory variables of the GNU coding standard. Your installation step (e.g. make install) would copy these files at some appropriate places, e.g. under /usr/share/ or /usr/lib/  etc..... And the places would be built inside the installed executable (you might have them under /usr/share/resources/myprog/ and pass -DRESOURCEDIR="/usr/share/resources/myprog/" to your g++ compilation command, using relevant make tricks). You might find autoconf helpful.

Notice that resources are needed (to be accessible) for the installed executable, not for the build tree.

Alternatively, at least on Linux, you could decide that resources are installed at some relative place w.r.t. the program executable (.e.g. in some grand-parent directory); IIRC this is the case for xemacs. Then, the issue is to find out the path of the executable and its directory (to later append a relative subpath, like "../lib/fonts/" etc..., to it). On Linux you just have to readlink(2) the "/proc/self/exe" symlink (and use dirname(3) on the result). On many other POSIX systems you could probably use the argv[0] of main and probably getenv("PATH") to find and compute it (perhaps with the later help of realpath(3)...).

Some framework libraries (like Qt or POCO or perhaps GTK or its Glib) might have conventions about resources and functions supporting them.

An example of a complex free software needing resources is the GCC compiler and it has to be built outside the source tree. You might look inside it to get inspired by its "resource management" (notably compiler predefined headers à la <limits.h> or <stdarg.h>...)

For Linux, look also into the Filesystem Hierarchy Standard and hier(7) which defines useful conventions.


问:
Is it considered good practice to use mutables this way?
Lately I came across a way of using mutables which I had not thought of before.

Let's say you have a list comprehension which calls a method, and you put the result in a list like the following (normal stuff):

```<code>my_list = [my_object.do_stuff() for my_object in all_objects]
</code>```

The do_stuff method does some loop computations to build diverse things, it's quite a bit complicated. For now everything's okay, but it turns out you need to do part of that computation again, and put the result in another list, just afterwards:

```<code>my_other_list = [my_object.do_just_a_part_of_stuff() for my_object in all_objects]
</code>```

You find out that to avoid doing the same bit of computation twice, you can use a mutable like a set the following way:

```<code>my_set = set()
my_list = [my_object.do_stuff(my_set) for my_object in all_objects]
my_other_variable = do_something_with(my_set)
</code>```

As you can see, my_set is empty at first, passed as an argument to do_stuff which modifies it in place just enough while it performs the computations mentioned above, and then once modified, my_set gets used for my_other_list.

Is this kind of pattern commonly considered good practice? And does it even have a name? I must say I find this quite simple and elegant, but I'm also wondering if that's not a bit hazardous and perhaps not so easy to read when you first encounter the codebase.

答:
I'd say that mutation inside a list comprehension looks unexpected and thus can be more error prone. On a code review I'd ask to rewrite it using an explicit loop which is typical for mutation.

Also, you don't need [element for element in my_set] to transform a set to a list, list(my_set) suffices. You can also just iterate over a set as you'd do over a list or any other iterable.


问:
How to design resilient web service clients?
I am writing an app that is supposed to talk to a web service. Let's say it's a chat app. It would need to receive a stream of new messages (so that they are pushed to the interface as quickly as possible), as well as send its own messages. Furthermore, I'd like the interface to be as responsive as possible, so immediately after the user sends a message, it should be visible in the list of his messages. However, the network might be faulty, or the chat server may be unavailable. Therefore, after the user presses the "send" button, his message is not guaranteed to be delivered to the server at all. The app should try to reach the network, and in case of failure notify the user. The user then have the option to retry or give up entirely.

I am trying to come up with a good design for this use case. My current idea is this: I am implementing a proxy object for the chat server — essentially, I am re-implementing a part of the server itself. I am running two separate processes, one of which maintains message histories that the user has with all of his peers. Let's say the user has n peers. Then this process runs n+2 threads, of which:

n are responsible for polling for updates from each peer and pushing them to a queue. Let's call them Event Feeds.

1 thread consumes that queue and is responsible for maintaining a local copy of message histories. It also exposes an interface to send new messages and to subscribe for event updates. Let's call it State Maintainer.

1 thread executes requests to send new messages, which it receives from the State Maintainer. Let's call it Sender.

The interface exposed by State Maintainer is powered by ZeroMQ. It is composed of two sockets: one REP and one PUB. On each iteration of the internal loop, this thread retrieves updates from the queue from Event Feeds, pushes them to the PUB socket and checks for new requests to send messages from the REP socket. For each new request, it pushes the update to the message history it maintains (setting a flag maybe sent on it) and transfers it to another queue. This queue is consumed by the Sender. For each new item in the queue, it attempts to perform a request to the chat server. In case of success, it pushes an event confirming success to a queue which is consumed by the State Maintainer, so maybe sent will become sent. In case of failure, it pushes an event to the same queue, so that State Maintainer will change maybe sent to not sent. In either case, the result is sent through the REP socket.

The UI process is connected to the proxy process via two ZeroMQ sockets. It runs two threads. The primary thread receives updates on the SUB socket and handles the UI. It talks to the secondary thread via two queues. When the user sends a new message, it pushes that message to the queue. The secondary thread picks it up and sends it to the REQ socket, storing a unique ID for that request. When the answer arrives (which can be distinguished by the UID), it pushes it to the second queue, which allows the UI thread to update the interface.

So I'd like to ask:

Is this approach good as a general approach to building resilient web service clients? Having a proxy, maintaining a piece of the remote state, two channels for information exchange between the proxy and the app logic?

What are some general practices of solving this problem? For example, I heard that Telegram is very fault-tolerant. Is it worthwhile crunching through its networking code?

答:
It looks like you want to spool the result of HTTP requests. Indeed, you might use some caching HTTP proxy like squid (you might use it, instead of coding your own).

You could use both HTTP client libraries (like libcurl) and HTTP server libraries (like libonion) in the same process (but probably in different threads, or thru some event loop like libev or libevent).

You still have to define some staleness criteria (and that would be application and domain specific; e.g. weather forecasts are surely rotten in a few days!), and you might want (perhaps that is not a very good idea) to pool the servers periodically.

You could also use (and provide) the Expires and Age HTTP headers (in replies).


答:
What you're suggesting is not possible, because the type of the assigned variables cannot be known until runtime. Remember that session is effectively shared between all modules used in creating the page. Some of these may not be known at compile time. Therefore you either need to specify the type, or move the check to runtime by using object or dynamic.

For instance, what would the compiler do with the following?

```<code>public void AssignString()
{
      Session["test"] = "STRING";
}

public void AssignInt()
{
      Session["test"] = 10;
}
</code>```

The two methods could be called in any order. These might even be a library functions in different plugins that are dynamically loaded at run time.

Title: Why Session has implemented this way in Asp.net

Body:
I'm not a Asp.net/C# developer myself but recently had to slightly refactor a Asp.net/C# project and I had to use Session at some point. When working with the session, regardless of the data type that I was specifying for the session variable, I had to cast the session variable any time I needed to access it.

```<code>Session["index"] = 0;
Session["index"] = Session["index"] + 1; // throws exception
Session["index"] = ((int)Session["index"]) + 1; // it works here
</code>```

Asking my C# developer colleagues, it seems that the Session is somehow an object that covers up the actual data type of the variable, apparently without losing it, but for some reason you still need to cast it anyway any time you want to access it.

I'm not sure if that's the case or not or even if I have understood it properly, but what I don't understand is why it has to be implemented in a way that the developer has to cast it before using it. As my colleagues say, that's because you can basically have a dynamically typed session object, at run-time.

What I'm thinking is, first, why you need to have such a feature only for the session, in a statically typed language like Asp.net/C#? For example, what is the use case of defining a "session" variable as an integer, but then put string on it at the run-time? Is there any practical use case and requirement for such a feature, not only in Asp.net but any other language?

Second, if we assume that the session variables also has to be statically typed, then is it "technically" possible to implement Asp.net's session in a way that when you define a session variable like below, the compiler allocates the memory in the heap, keep the data type next to it and simply eliminates the need for casting it each time?

```<code>Session["index"] = 0; // integer type, available at compile-time
</code>```

What would happen if you try to assign a string to this integer variable? Well, like everything else in a statically typed language, you will get an exception.

Am I missing anything here? I simply can't understand why such a design decision has been made by the Asp.net development team and how the developer could benefit from that. Also wondering if there is anything related to how other languages like Java or C++ have implemented sessions, either directly in the language itself or via a library, so it led the Asp.net team as well to implement session like that?

Update #1

Just to clear the air, my main question is why the Session has implemented in this way? What are the benefits of having a session like this -- whatever it is, rather how it has implemented -- even though I'm looking to know that as well.

Update #2

Regarding @JacquesB comment: "You have to be able to save all kinds of different objects in a Session, so the type has to be Object, there is no other option"; Why the following code:

```<code>Session["index"] = 0;
Session["name"] = "abc";
</code>```

Can't be translated, by compiler, at the compile-time, to something like this:

```<code>int SessionIndex = 0;
string SessionName = "abc";
</code>```

I don't get why "there is no other option". It's a computer program after all, not anything magical, or am I still missing something?

Body:
What you're suggesting is not possible, because the type of the assigned variables cannot be known until runtime. Remember that session is effectively shared between all modules used in creating the page. Some of these may not be known at compile time. Therefore you either need to specify the type, or move the check to runtime by using object or dynamic.

For instance, what would the compiler do with the following?

public void AssignString()
{
      Session["test"] = "STRING";
}

public void AssignInt()
{
      Session["test"] = 10;
}

The two methods could be called in any order. These might even be a library functions in different plugins that are dynamically loaded at run time.


答:
It could be worse, but it's overly coupled, and a sign your do_stuff is too big.  I would only do it if I needed the extra performance, and couldn't think of any other way.

You didn't give us much to go on, but usually problems like that can be much more cleanly decomposed in the following way:

```<code>common = [my_object.get_common() for my_object in all_objects]
my_list = list_from_common(common)
my_other_list = other_list_from_common(common)
</code>```

Title: Is it considered good practice to use mutables this way?

Body:
Lately I came across a way of using mutables which I had not thought of before.

Let's say you have a list comprehension which calls a method, and you put the result in a list like the following (normal stuff):

```<code>my_list = [my_object.do_stuff() for my_object in all_objects]
</code>```

The do_stuff method does some loop computations to build diverse things, it's quite a bit complicated. For now everything's okay, but it turns out you need to do part of that computation again, and put the result in another list, just afterwards:

```<code>my_other_list = [my_object.do_just_a_part_of_stuff() for my_object in all_objects]
</code>```

You find out that to avoid doing the same bit of computation twice, you can use a mutable like a set the following way:

```<code>my_set = set()
my_list = [my_object.do_stuff(my_set) for my_object in all_objects]
my_other_variable = do_something_with(my_set)
</code>```

As you can see, my_set is empty at first, passed as an argument to do_stuff which modifies it in place just enough while it performs the computations mentioned above, and then once modified, my_set gets used for my_other_list.

Is this kind of pattern commonly considered good practice? And does it even have a name? I must say I find this quite simple and elegant, but I'm also wondering if that's not a bit hazardous and perhaps not so easy to read when you first encounter the codebase.

Body:
It could be worse, but it's overly coupled, and a sign your do_stuff is too big.  I would only do it if I needed the extra performance, and couldn't think of any other way.

You didn't give us much to go on, but usually problems like that can be much more cleanly decomposed in the following way:

common = [my_object.get_common() for my_object in all_objects]
my_list = list_from_common(common)
my_other_list = other_list_from_common(common)


问:
IBM Jazz: what are Streams?
I'm trying to make sense of IBM Jazz.  I'm trying to figure out what it is, and what it isn't.  "Streams" have been giving me trouble.  From IBM's online library:

The stream is similar in concept to a branch in most other version
  control systems. However, there are some significant differences that
  enable specific operations when using Rational Team Concert.

They loudly lambast freeware solutions for their handling of the development patterns that use branching:

It is true that many agile teams use freeware or low-cost development
  tools that offer limited support for segregated development. Such
  tools often put developers off of a segregated approach due to
  functional limitations and the high overhead of integration within
  such tools. Rational Team Concert is a mechanism for many agile teams
  to raise their game and take their development capability to the next
  level in terms of efficiency, flexibility, and the creativity that can
  be engineered into their development processes.

However, in all the documentation I have read so far, I am having trouble finding anything that Jazz Streams do which have not been de facto standard behaviors for branches for the last 20 or so years.

Is there anything to Jazz Streams that is unique enough to keep in mind as I continue making sense of Jazz, or should I just write them off as IBM adding branching and electing to call it something different?

答:
In my current shop, Streams are tied to Builds, which thereby controls the targets available to deploy to. The "Development" stream is only associated with builds targeting the Dev & Test servers, while only the Integration and Production Streams are associated with Builds targeting Staging or Production.

Yeah, you can still do that using Branches, but as @SBI said, the benefit is in the integration with the change-management suite. I prefer to keep branches as "topic branches" -- a strictly development task.

Title: IBM Jazz: what are Streams?


答:
Java References and C pointers differ in exactly two points:

There's no pointer-arithmetic for the former.

And you cannot create a Java reference to whatever you want, you can only copy those saved somewhere accessible (static fields, fields of objects, local variables) or returned by function-invocations (like constructor-calls), which thus all refer to Java objects (never to basic types like references, char, int and so on).

Someone wrote that References are strongly typed, because you cannot force the compiler to treat an int* as a char*.

Completely aside from the fact that that particular conversion is actually safe, there is no polymorphism in C, so that comparison is a non-starter.

Certainly, Java is more strongly-typed than C, not that that's a feature of C pointers vs Java references, you need to use the JNI to break type-safety (aside from disregarding generic restrictions), but even in C you have to force the compiler.

Someone wrote that Java references might be implemented as C pointers, to which I say sure, as they are strictly less powerful, on 32Bit machines they typically are, if the JVM is implemented in C. Though on 64Bit machines, they are normally compressed ordinary-object-pointers ("compressed OOPs") to save space and bandwidth.

Anyway, those C pointers need not be equivalent to hardware addresses either, even if they typically (>99% of implementations) are for performance reasons.

Finally, that's an implementation detail which is not exposed to the programmer.

Title: How is a Java reference different from a C pointer?

Body: C has pointers and Java has what is called references. They have some things in common in the sense that they all point to something. I know that pointers in C store the addresses they point to. Do reference also store the address? How they are different except that pointer is more flexible and error-prone?</p>


问:
Is it okay to return the "wrong" HTTP status code in order to show a more user-friendly error page?
Located here, RFC 7231 is all about the HTTP protocol (status codes)

I'm developing an in-house web application for my university's department and I'm at a (probably unimportant) impasse.

I'm reducing users' access to resources based on role (student, faculty, staff) and have it working just fine. However, when a user attempts to access a resource and is denied access, the HTTP specification says that I should use a 403: Forbidden, marking that the user does not have access to the resource, or "hide" the resource by returning a 404: Not Found.

I could do that, but I want to make the site a bit more user-friendly and instead redirect them to their previous page with a flash (NOTE that I did not say 'Adobe Flash'; see comments) message instead of having a 403/4 page with links back to different parts of the site. I think it's a pretty UX-friendly feature.

However, that would involve redirecting via a 303: See Other, which more or less violates the standard.

Obviously it really doesn't matter as this is an in-house project for a lab at my university and nobody here is going to be complaining, but imagine this site were under the Google umbrella.

So, is it bad practice to deviate from the [HTTP] standard like this?

Edit: There seems to be a lot of confusion, so let me give an example:

User with id = 1 attempts to access mydomain.com/projects/2 through their profile (by clicking a link). User 1 is the owner of project 2, so the server accepts the requests and renders the page with the project.

User 1 then types mydomain.com/projects/3 into the URL. User 1 is not the owner of project 3. Therefore, when the request gets to the server, the server denies it.

Now, there are two options:

1.) The server redirects the user to /projects/2 (their previously visited page, which would of course always be a valid one) with a helpful notification at the top such as "Sorry, but you do not have permission to access XXXX" or

2.) The server renders a 403: Forbidden page, or a 404: Not Found page, which would probably have that same text (if it was a 403), and maybe even a JavaScript-enabled "back" link

Personally, the redirect seems way more user friendly to me, and eliminates seeing an unfriendly page like 403: Forbidden and having to click a "back" button. However, doing the redirect goes against the HTTP protocol standard.

答:
403, 404 and 401 which you didn't mention have specific meanings.

403 (forbidden) doesn't mean that this particular user isn't allowed to access the resource, but that nobody is. It has nothing to do with that particular user, and there is nothing the user can do to be allowed access.

401 (unauthorized) means that the user wasn't authorized to access the resource, but that is a problem that could be fixed by the user getting authorization. Typically this is used when a user isn't logged in, or a password needs to be provided, or if different users have different rights, and this user doesn't have enough rights.

404 (not found) means the resource doesn't exist. Which is a perfectly normal thing to happen. Could happen if the user mistyped a URL, for example. But you should never give an error 404 if this particular user doesn't have the right to access the resource. 404 is meant for exactly the case that a resource doesn't exist.

410 (gone) is sometimes used; it means a resource was there in the past, but now it's gone, and it's very unlikely to come back. The difference to 404 is that this isn't due to something being spelled wrong. Last week the request might have been fine, but now the resource is gone.


问:
Is it ok if a method returns a new instance of the class it's in?
I have a class called MyClass and a method called Get that looks something like this:

public class MyClass{
    string ClassVariable1 {get; set;}
    string ClassVariable2 {get; set;}
    string ClassVariable3 {get; set;}

    public MyClass Get()
    {
           ..set variable1, variable2, and variable3..

                return new MyClass()
                {
                    ClassVariable1 = variable1,
                    ClassVariable2 = variable2,
                    ClassVariable3 = variable3
                };
            }
        }
        return null;
    }
}
```

<p>Is there anything wrong with doing this from a design perspective? Or any other perspective?</p>

<p>I guess to make this more clear, I'll post my actual code. I'm writing a facebook app and my class is called "FacebookAccount". The idea is that this will hold stuff related to a facebook account, like screen name, user email, recent status updates made by the user, etc. </p>

<pre class="lang-cs prettyprint-override"><code>class FacebookAccount
{
    public AccessToken AccessToken { get; set; }
    public string ScreeName { get; set; }
    public long FacebookUserId { get; set; }
    public string Email { get; set; }
    public List<StatusUpdate> ScheduledStatusNotPosted { get; set; }
    public List<StatusUpdate> StatusPosted { get; set; }
    private Facebook Facebook { get; set; }

    public List<FacebookAccount> Get(string userId)
    {
        var facebookAccountList = new List<FacebookAccount>();

        using (DataContext dc = new DataContext())
        {
            var facebookAccount = dc.FacebookAccounts.Where(x => x.UserId == userId);

            foreach (var account in facebookAccount)
            {
                var faceBook = new Facebook(account.UserId, account.UserId);
                facebookAccountList.Add(new FacebookAccount()
                {
                    AccessToken = new AccessToken() { Token = account.Token, TokenSecret = string.Empty },
                    Email = account.Email,
                    ScreeName = account.Email,
                    ScheduledStatusNotPosted = faceBook.GetScheduledStatusUpdates(),
                    FacebookUserId = account.UserId
                });
            }
        }

        return facebookAccountList;
    }

    public FacebookAccount Get(string userId, long facebookUserId)
    {
        using (DataContext dc = new DataContext())
        {
            var facebookAccount = dc.FacebookAccounts.FirstOrDefault(x => x.UserId == userId && x.UserId == facebookUserId);
            if (facebookAccount != null)
            {
                Facebook = new Facebook(facebookAccount.UserId, facebookAccount.UserId);

                return new FacebookAccount()
                {
                    AccessToken = new AccessToken() { Token = facebookAccount.Token, TokenSecret = string.Empty},
                    Email = facebookAccount.Email,
                    ScreeName = facebookAccount.Email,
                    ScheduledStatusNotPosted = Facebook.GetScheduledStatusUpdates(),
                    FacebookUserId = facebookAccount.UserId
                };
            }
        }
        return null;
    }
}
</code>```

答:
Having methods return a new instance of their class is done for two reasons:

Copying or cloning the current object

Making the object immutable

The C# language makes Strings immutable by having each method with a string return type return a new instance of a string. This allows you to modify data on an object and store the result of that operation without modifying the original object --- programming without side effects, if you will.

More specifically to your question, the FacebookAccount class violates the Single Responsibility Principle: it is doing data access as well as stuff related to a Facebook account. It really seems like moving the data access methods to FacebookAccountRepository would be more beneficial here.