问:
How are Rust Traits different from Go Interfaces?
I am relatively familiar with Go, having written a number of small programs in it. Rust, of course, I am less familiar with but keeping an eye on.

Having recently read http://yager.io/programming/go.html, I thought I'd personally examine the two ways Generics are handled because the article seemed to unfairly criticize Go when, in practice, there wasn't much that Interfaces couldn't accomplish elegantly. I kept hearing the hype about how powerful Rust's Traits were and nothing but criticism from people about Go. Having some experience in Go, I wondered how true it was and what the differences ultimately were. What I found was that Traits and Interfaces are pretty similar! Ultimately, I'm not sure if I'm missing something, so here is a quick educational rundown of their similarities so you can tell me what I missed!

Now, let's take a look at Go Interfaces from their documentation:

Interfaces in Go provide a way to specify the behavior of an object: if something can do this, then it can be used here.

By far the most common interface is Stringer which returns a string representing the object.

```<code>type Stringer interface {
    String() string
}
</code>```

So, any object that has String() defined on it is a Stringer object. This can be used in type signatures such that func (s Stringer) print() takes almost all objects and prints them.

We also have interface{} which takes any object. We must then determine the type at runtime through reflection.

Now, let's take a look at Rust Traits from their documentation:

At its simplest, a trait is a set of zero or more method signatures. For example, we could declare the trait Printable for things that can be printed to the console, with a single method signature:

```<code>trait Printable {
    fn print(&self);
}
</code>```

This immediately looks quite similar to our Go Interfaces. The only difference I see is that we define 'Implementations' of Traits rather than just defining the methods. So, we do

```<code>impl Printable for int {
    fn print(&self) { println!("{}", *self) }
}
</code>```

instead of

```<code>fn print(a: int) { ... }
</code>```

Bonus Question: What happens in Rust if you define a function that implements a trait but you don't use impl? It just doesn't work?

Unlike Go's Interfaces, Rust's type system has type parameters which let you do proper generics and things like interface{} while the compiler and the runtime actually know the type. For example,

```<code>trait Seq<T> {
    fn length(&self) -> uint;
}
</code>```

works on any type and the compiler knows that the type of the Sequence elements at compile time rather than using reflection.

Now, the actual question: am I missing any differences here? Are they really that similar? Is there not some more fundamental difference that I'm missing here? (In usage. Implementation details are interesting, but ultimately not important if they function the same.)

Besides syntactic differences, the actual differences I see are:

Go has automatic method dispatch vs. Rust requires(?) impls to implement a Trait

Elegant vs. Explicit

Rust has type parameters which allow for proper generics without reflection.

Go really has no response here. This is the only thing that is significantly more powerful and it's ultimately just a replacement for copying and pasting methods with different type signatures.

Are these the only non-trivial differences? If so, it would appear Go's Interface/Type system is, in practice, not as weak as perceived.

答:
What happens in Rust if you define a function that implements a trait but you don't use impl? It just doesn't work?

You need to explicitly implement the trait; happening to have a method with matching name/signature is meaningless for Rust.

Generic call dispatching

Are these the only non-trivial differences? If so, it would appear Go's Interface/Type system is, in practice, not as weak as perceived.

Not providing static dispatch can be a significant performance hit for certain cases (e.g. the Iterator one I mention below). I think this is what you mean by

Go really has no response here. This is the only thing that is significantly more powerful and it's ultimately just a replacement for copying and pasting methods with different type signatures.

but I'll cover it in more detail, because it's worth understanding the difference deeply.

In Rust

Rust's approach allows for the user to choose between static dispatch and dynamic dispatch. As an example,  if you have

trait Foo { fn bar(&self); }

impl Foo for int { fn bar(&self) {} }
impl Foo for String { fn bar(&self) {} }

fn call_bar<T: Foo>(value: T) { value.bar() }

fn main() {
    call_bar(1i);
    call_bar("foo".to_string());
}
```

<p>then the two <code>call_bar</code> calls above will compile to calls to, respectively,</p>

<pre class="lang-rust prettyprint-override"><code>fn call_bar_int(value: int) { value.bar() }
fn call_bar_string(value: String) { value.bar() }
</code>```

where those .bar() method calls are static function calls, i.e. to a fixed function address in memory. This allows for optimisations like inlining, because the compiler knows exactly which function is being called. (This is what C++ does too, sometimes called "monomorphisation".)

In Go

Go only allows dynamic dispatch for "generic" functions, that is, the method address is loaded from the value and then called from there, so the exact function is only known at runtime. Using the example above

type Foo interface { bar() }

func call_bar(value Foo) { value.bar() }

type X int;
type Y string;
func (X) bar() {}
func (Y) bar() {}

func main() {
    call_bar(X(1))
    call_bar(Y("foo"))
}
```

<p>Now, those two <code>call_bar</code>s will always be calling the above <code>call_bar</code>, with the address of <code>bar</code> loaded from the interface's vtable.</p>

<h2>Low-level</h2>

<p>To rephrase the above, in C notation. Rust's version creates</p>

<pre class="lang-rust prettyprint-override"><code>/* "implementing" the trait */
void bar_int(...) { ... }
void bar_string(...) { ... }

/* the monomorphised `call_bar` function */
void call_bar_int(int value) {
    bar_int(value);
}
void call_bar_string(string value) {
    bar_string(value);
}

int main() {
    call_bar_int(1);
    call_bar_string("foo");
    // pretend that is the (hypothetical) `string` type, not a `char*`
    return 1;
}
</code>```

For Go, it's something more like:

/* implementing the interface */
void bar_int(...) { ... }
void bar_string(...) { ... }

// the Foo interface type
struct Foo {
    void* data;
    struct FooVTable* vtable;
}
struct FooVTable {
    void (*bar)(void*);
}

void call_bar(struct Foo value) {
    value.vtable.bar(value.data);
}

static struct FooVTable int_vtable = { bar_int };
static struct FooVTable string_vtable = { bar_string };

int main() {
    int* i = malloc(sizeof *i);
    *i = 1;
    struct Foo int_data = { i, &int_vtable };
    call_bar(int_data);

    string* s = malloc(sizeof *s);
    *s = "foo"; // again, pretend the types work
    struct Foo string_data = { s, &string_vtable };
    call_bar(string_data);
}
```

(This isn't exactly right---there has to be more information in the vtable---but the method call being a dynamic function pointer is the relevant thing here.)

Rust offers the choice

Going back to

Rust's approach allows for the user to choose between static dispatch and dynamic dispatch.

So far I've only demonstrated Rust having statically dispatched generics, but Rust can opt-in to the dynamic ones like Go (with essentially the same implementation), via trait objects. Notated like &Foo, which is a borrowed reference to an unknown type that implements the Foo trait. These values have the same/very similar vtable representation to the Go interface object. (A trait object is an example of an "existential type".)

There are instances where dynamic dispatch is really helpful (and sometimes more performant, by, e.g. reducing code bloat/duplication), but static dispatch allows compilers to inline the callsites and apply all their optimisations, meaning it is normally faster. This is especially important for things like Rust's iteration protocol, where static dispatching trait method calls allows for those iterators to be as fast as the C equivalents, while still seeming high-level and expressive.

Tl;dr: Rust's approach offers both static and dynamic dispatch in generics, at the programmers discretion; Go only allows for dynamic dispatch.

Parametric polymorphism

Furthermore, emphasising traits and deemphasising reflection gives Rust much stronger parametric polymorphism: the programmer knows exactly what a function can do with its arguments, because it has to declare the traits the generic types implement in the function signature.

Go's approach is very flexible, but has fewer guarantees for the callers (making it somewhat harder for the programmer to reason about), because the internals of a function can (and do) query for additional type information (there was a bug in the Go standard library where, iirc, a function taking a writer would use reflection to call Flush on some inputs, but not others).

Building abstractions

This is somewhat of a sore point, so I'll only talk briefly, but having "proper" generics like Rust has allows for low level data types like Go's map and [] to actually be implemented directly in the standard library in a strongly typesafe way, and written in Rust (HashMap and Vec respectively).

And its not just those types, you can build type-safe generic structures on top of them, e.g. LruCache is a generic caching layer on top of a hashmap. This means people can just use the data structures directly from the standard library, without having to store data as interface{} and use type assertions when inserting/extracting. That is, if you have an LruCache<int, String>, you're guaranteed that the keys are always ints and the values are always Strings: there's no way to accidentally insert the wrong value (or try to extract a non-String).

OwnerDisplayName: Logan

Body:
What happens in Rust if you define a function that implements a trait but you don't use impl? It just doesn't work?

You need to explicitly implement the trait; happening to have a method with matching name/signature is meaningless for Rust.

Generic call dispatching

Are these the only non-trivial differences? If so, it would appear Go's Interface/Type system is, in practice, not as weak as perceived.

Not providing static dispatch can be a significant performance hit for certain cases (e.g. the Iterator one I mention below). I think this is what you mean by

Go really has no response here. This is the only thing that is significantly more powerful and it's ultimately just a replacement for copying and pasting methods with different type signatures.

but I'll cover it in more detail, because it's worth understanding the difference deeply.

In Rust

Rust's approach allows for the user to choose between static dispatch and dynamic dispatch. As an example,  if you have

trait Foo { fn bar(&self); }

impl Foo for int { fn bar(&self) {} }
impl Foo for String { fn bar(&self) {} }

fn call_bar<T: Foo>(value: T) { value.bar() }

fn main() {
    call_bar(1i);
    call_bar("foo".to_string());
}

then the two call_bar calls above will compile to calls to, respectively,

fn call_bar_int(value: int) { value.bar() }
fn call_bar_string(value: String) { value.bar() }

where those .bar() method calls are static function calls, i.e. to a fixed function address in memory. This allows for optimisations like inlining, because the compiler knows exactly which function is being called. (This is what C++ does too, sometimes called "monomorphisation".)

In Go

Go only allows dynamic dispatch for "generic" functions, that is, the method address is loaded from the value and then called from there, so the exact function is only known at runtime. Using the example above

type Foo interface { bar() }

func call_bar(value Foo) { value.bar() }

type X int;
type Y string;
func (X) bar() {}
func (Y) bar() {}

func main() {
    call_bar(X(1))
    call_bar(Y("foo"))
}

Now, those two call_bars will always be calling the above call_bar, with the address of bar loaded from the interface's vtable.

Low-level

To rephrase the above, in C notation. Rust's version creates

/* "implementing" the trait */
void bar_int(...) { ... }
void bar_string(...) { ... }

/* the monomorphised `call_bar` function */
void call_bar_int(int value) {
    bar_int(value);
}
void call_bar_string(string value) {
    bar_string(value);
}

int main() {
    call_bar_int(1);
    call_bar_string("foo");
    // pretend that is the (hypothetical) `string` type, not a `char*`
    return 1;
}

For Go, it's something more like:

/* implementing the interface */
void bar_int(...) { ... }
void bar_string(...) { ... }

// the Foo interface type
struct Foo {
    void* data;
    struct FooVTable* vtable;
}
struct FooVTable {
    void (*bar)(void*);
}

void call_bar(struct Foo value) {
    value.vtable.bar(value.data);
}

static struct FooVTable int_vtable = { bar_int };
static struct FooVTable string_vtable = { bar_string };

int main() {
    int* i = malloc(sizeof *i);
    *i = 1;
    struct Foo int_data = { i, &int_vtable };
    call_bar(int_data);

    string* s = malloc(sizeof *s);
    *s = "foo"; // again, pretend the types work
    struct Foo string_data = { s, &string_vtable };
    call_bar(string_data);
}

(This isn't exactly right---there has to be more information in the vtable---but the method call being a dynamic function pointer is the relevant thing here.)

Rust offers the choice

Going back to

Rust's approach allows for the user to choose between static dispatch and dynamic dispatch.

So far I've only demonstrated Rust having statically dispatched generics, but Rust can opt-in to the dynamic ones like Go (with essentially the same implementation), via trait objects. Notated like &Foo, which is a borrowed reference to an unknown type that implements the Foo trait. These values have the same/very similar vtable representation to the Go interface object. (A trait object is an example of an "existential type".)

There are instances where dynamic dispatch is really helpful (and sometimes more performant, by, e.g. reducing code bloat/duplication), but static dispatch allows compilers to inline the callsites and apply all their optimisations, meaning it is normally faster. This is especially important for things like Rust's iteration protocol, where static dispatching trait method calls allows for those iterators to be as fast as the C equivalents, while still seeming high-level and expressive.

Tl;dr: Rust's approach offers both static and dynamic dispatch in generics, at the programmers discretion; Go only allows for dynamic dispatch.

Parametric polymorphism

Furthermore, emphasising traits and deemphasising reflection gives Rust much stronger parametric polymorphism: the programmer knows exactly what a function can do with its arguments, because it has to declare the traits the generic types implement in the function signature.

Go's approach is very flexible, but has fewer guarantees for the callers (making it somewhat harder for the programmer to reason about), because the internals of a function can (and do) query for additional type information (there was a bug in the Go standard library where, iirc, a function taking a writer would use reflection to call Flush on some inputs, but not others).

Building abstractions

This is somewhat of a sore point, so I'll only talk briefly, but having "proper" generics like Rust has allows for low level data types like Go's map and [] to actually be implemented directly in the standard library in a strongly typesafe way, and written in Rust (HashMap and Vec respectively).

And its not just those types, you can build type-safe generic structures on top of them, e.g. LruCache is a generic caching layer on top of a hashmap. This means people can just use the data structures directly from the standard library, without having to store data as interface{} and use type assertions when inserting/extracting. That is, if you have an LruCache<int, String>, you're guaranteed that the keys are always ints and the values are always Strings: there's no way to accidentally insert the wrong value (or try to extract a non-String).


问:
Referencing official documentation in source-code documentation
What are your preferences for referencing official documentation (like a white paper) for an algorithm from comments in
the code that implements it? I'm trying to decide between two methods, as illustrated by the examples below, but am
open to other suggestions. RFC 1321, for instance, describes the MD5 hash algorithm. If I were implementing MD5
in Python using the paper as a guide, and documented my code as I went, which of the following implementations of an
example function pad_string(str) would you recommend, if not something entirely different?

option 1

```<code>def pad_string(str):
    """
    Pads the string to be hashed to an appropriate length. See RFC 1321 3.1.
    """
</code>```

where the relevant section of the RFC is:

```<code>3.1 Step 1. Append Padding Bits

    The message is "padded" (extended) so that its length (in bits) is
    congruent to 448, modulo 512. That is, the message is extended so
    that it is just 64 bits shy of being a multiple of 512 bits long.
    Padding is always performed, even if the length of the message is
    already congruent to 448, modulo 512.
</code>```

or

option 2

```<code>def pad_string(str):
    """
    Pads the argument string such that its length is congruent to 448, modulo 512.
    """
</code>```

Here, option 1 simply references relevant parts of the "official" MD5 documentation by section number, where
option 2 doesn't make any mention of the paper and basically reiterates its contents where necessary. The
advantage of the first is that the documentation adheres somewhat to DRY (Don't Repeat Yourself, in this case with
respect to the RFC), but its disadvantage is that it relies on a third-party resource: if it's an obscure technical
specification hosted in one place and the link provided inside the code documentation dies, then you're out of luck.
The advantage of the second approach is that your source is self-contained: you don't need any other information when
reading through it, and our example MD5 script would become as much a description of the algorithm as its
implementation; this, of course, requires more writing on the code author's part, can bloat documentation, and, more
importantly, may end up with mis-copied information and other errors. Is there any consensus on which is better?

答:
It depends on type of documentation. If it is a publicly facing function you probably should provide all information necessary to use API. For example if I want to use SMTP library to send an email I don't want to read RFC 5321, RFC 2045, RFC 2046, RFC 2047, RFC 4288, RFC 4289, RFC 2049 and others as most parts are only vaguely relevant to what I want to achieve - in most cases I just want to send a simple message filling the To/From/Subject fileds and message - maybe add attachment if I feel adventurous. However you might want to reference it if RFC is simple and protocol is not well-known to provide additional information (or webpage describing protocol/format if it exists).

For internal documentation the more important is why rather then how (this part is covered by code) or what. Therefore referencing the specification is useful. It will also allow the person modified the code to see what (s)he can change or at least what part of spec (s)he needs to consider.

For example in your example it is probably clear from the code that pad_string pads string so result length is congruent to 448, modulo 512. However the MD5 spec provides reference why it is done.

Other method is to nearly copy and paste the paper/RFC into code if flow is non-trivial and you cannot avoid it by making the code nicer in one way or another. For example lets say the pseudocode is:

```<code>Ri <- head from stack
Merge Ri and Rj
if Ri' is not well balanced then
   balance(Ri')
push Ri' on stack
</code>```

You can do something like:

```<code>// Ri <- head from stack
Node *Ri = stack_pop(stack);
// Merge Ri and Rj
Node *Rj = NULL;
while(!empty(Ri) && !empty(Rj)) {
   Node *next;
   if (foo(peek(Ri), peek(Rj))) {
       next = get(Ri);
   } else {
       next = get(Rj);
   }
   add(Rj, next);
}
// if Ri' is not well balanced then
//   balance(Ri')
...
</code>```

Also, of course, in such case you need to check the copyright law first (this is not a legal advice that it is permitted, especially that copyright law vary from jurisdiction to jurisdiction).

ContentLicense: CC BY-SA 3.0

Body:
It depends on type of documentation. If it is a publicly facing function you probably should provide all information necessary to use API. For example if I want to use SMTP library to send an email I don't want to read RFC 5321, RFC 2045, RFC 2046, RFC 2047, RFC 4288, RFC 4289, RFC 2049 and others as most parts are only vaguely relevant to what I want to achieve - in most cases I just want to send a simple message filling the To/From/Subject fileds and message - maybe add attachment if I feel adventurous. However you might want to reference it if RFC is simple and protocol is not well-known to provide additional information (or webpage describing protocol/format if it exists).

For internal documentation the more important is why rather then how (this part is covered by code) or what. Therefore referencing the specification is useful. It will also allow the person modified the code to see what (s)he can change or at least what part of spec (s)he needs to consider.

For example in your example it is probably clear from the code that pad_string pads string so result length is congruent to 448, modulo 512. However the MD5 spec provides reference why it is done.

Other method is to nearly copy and paste the paper/RFC into code if flow is non-trivial and you cannot avoid it by making the code nicer in one way or another. For example lets say the pseudocode is:

Ri <- head from stack
Merge Ri and Rj
if Ri' is not well balanced then
   balance(Ri')
push Ri' on stack

You can do something like:

// Ri <- head from stack
Node *Ri = stack_pop(stack);
// Merge Ri and Rj
Node *Rj = NULL;
while(!empty(Ri) && !empty(Rj)) {
   Node *next;
   if (foo(peek(Ri), peek(Rj))) {
       next = get(Ri);
   } else {
       next = get(Rj);
   }
   add(Rj, next);
}
// if Ri' is not well balanced then
//   balance(Ri')
...

Also, of course, in such case you need to check the copyright law first (this is not a legal advice that it is permitted, especially that copyright law vary from jurisdiction to jurisdiction).


问:
What should happen at the start of a software project startup?

A quick introduction

My college semesters include a 8 week project working for an actual company with a software need in order to get some much needed practical experience. I have just started such a project with 5 other students. We're required to spend roughly 40 hours a week per student on this project. We're working with SCRUM as the software development method, this was assigned by our teachers.

The question

Day one of the project just ended which has created some questions for me as to how to start a project in the 'real world'. Our first day included working on a project planning document (not sure what the English term is), creating a appointment with the company for an introduction and the opportunity to start specifying the requirements and setting up some standards for the behavior within the group.

However these items didn't take that long to finish. We've made some concrete plans for tomorrow and the day after we'll meet the company. This still leaves several hours of 'work-time' unspent. Is it usual not being able to fill every hour of a day for work at the start of a project or are we simply too inexperienced to see what work needs to be done at this stage of a project, or are we, perhaps, going through the above list too fast?

How does this work in the 'real world'? Do you spend your time wondering 'what should I do now', or do you have a clear view of what you're supposed to do at that moment?

答:
What should happen at the start of a software project? How does this work in the 'real world'? Do you spend your time wondering 'what should I do now', or do you have a clear view of what you're supposed to do at that moment?

What I look to do at the start of a software project is spend time getting to know the people I am working with.

I like to develop and modify team practices and procedures over time, however the start of a project or a team can be a good time to agree on some initial ground rules or 'charter'.

Look to find a way to work together that works for the team for whatever situation and mix of members you have, and find the best tools for email, im, ticket tracking, wiki, etc. for the situation at hand.  Become comfortable with using and adapting these tools to meet your specific needs so that when the time comes for quick changes you are proficient in making them.


问:
Can one always design for scale out?
When designing an application where scalability is going to be important, it would be useful if the application could be designed to "scale out" on cheap commodity servers. However, there are many scale up applications (see my previous question).

Is it always possible to design an application to scale out, or do some requirements make scale out impossible, even if all software can be designed from scratch and no legacy software is involved?

I am looking for cases where scale out is an inferior solution to scale up, and where good application design could not mitigate this. In other words, the application cannot be designed to scale out effectively, no matter how hard one tries.

I am not looking for cases where legacy software or data structures -- or even entire architectures (as in the answer to the previous question) -- were not designed to scale out and could not be changed.

Alternatively, if there is reason to believe that one can always scale out, if the application is well designed, I would love to know that as well.

Edit: I am looking for an explicit example of such a system, if there is one.

答:
No, the history of computation is doing things that are [were] "impossible."

Cheek aside, at a more practical level - it sounds like you're essentially asking your audience to prove a negative. So again, the answer is pretty easily "no."

OwnerDisplayName: Demetri

Title: Can one always design for scale out?


问:
Early attempt to remove Python GIL resulted in bad performance: Why?
This post from Python creator, Guido Van Rossum, mentions an early attempt to remove the GIL from Python:

This has been tried before, with disappointing results, which is why
  I'm reluctant to put much effort into it myself. In 1999 Greg Stein
  (with Mark Hammond?) produced a fork of Python (1.5 I believe) that
  removed the GIL, replacing it with fine-grained locks on all mutable
  data structures. He also submitted patches that removed many of the
  reliances on global mutable data structures, which I accepted.
  However, after benchmarking, it was shown that even on the platform
  with the fastest locking primitive (Windows at the time) it slowed
  down single-threaded execution nearly two-fold, meaning that on two
  CPUs, you could get just a little more work done without the GIL than
  on a single CPU with the GIL. This wasn't enough, and Greg's patch
  disappeared into oblivion. (See Greg's writeup on the performance.)

I can hardly argue with the actual results, but I really wonder why this happened.  Presumably, the main reason that removing the GIL from CPython is so difficult is because of the reference counting memory management system.  A typical Python program will call Py_INCREF and Py_DECREF thousands or millions of times, making it a key contention point if we were to wrap locks around it.

But, I don't understand why adding atomic primitives would slow down a single threaded program.  Suppose we just modified CPython so that the refcount variable in each Python object was an atomic primitive.  And then we just do an atomic increment (fetch-and-add instruction) when we need to increment the reference count.  This would make Python reference counting thread-safe, and shouldn't have any performance penalty on a single-threaded application, because there would be no lock contention.

But alas, many people who are smarter than me have tried and failed, so obviously I'm missing something here.  What is wrong with the way I'm looking at this problem?

答:
I am unfamiliar with the Greg Stein Python fork, so discount this comparison as speculative historical analogy if you wish. But this was exactly the historical experience of many infrastructure codebases moving from single- to multi-threaded implementations.

Essentially every Unix implementation I studied in the 1990s--AIX, DEC OSF/1, DG/UX, DYNIX, HP-UX, IRIX, Solaris, SVR4, and SVR4 MP--all went through exactly this kind of "we put in finer-grained locking--now it's slower!!" problem. The DBMSs I followed--DB2, Ingres, Informix, Oracle, and Sybase--they all went through it too.

I have heard "these changes won't slow us down when we're running single-threaded" a million times. It never works out that way. The simple act of conditionally checking "are we running multithreaded, or not?" adds real overhead, especially on highly-pipelined CPUs. Atomic operations and occasional spin-locks added to ensure the integrity of shared data structures have to be called quite often, and they're very slow. First-generation lock/synchronization primitives also were slow. Most implementation teams eventually add several classes of primitives, in various "strengths," depending on how much interlock protection was needed at various places. Then they realize where they initially slapped down locking primitives was not really the right place, so they had to profile, design around the bottlenecks found, and systematically roto-till. Some of these sticking points eventually got OS or hardware acceleration, but that whole evolution took 3-5 years, bare minimum. Meanwhile, the MP or MT versions were limping, performance-wise.

Otherwise-sophisticated development teams have argued that such slowdowns are basically a persistent, intractable fact of life. IBM e.g. refused to SMP-enable AIX for at least 5 years after the competition, adamant that single-threaded was just purely better. Sybase used some of the same arguments. The only reason some of the teams eventually came around was that single-thread performance could no longer be reasonably improved at a CPU level. They were forced to either go MP/MT or accept having an increasingly uncompetitive product.

Active concurrency is HARD. And it's deceptive. Everyone rushes into it thinking "this won't be so bad." Then they hit the quicksand, and have to plod through. I've seen this happen with at least a dozen name-brand, well-funded, smart teams. Generally it seemed to take at least five years after choosing to multi-thread to "get back to where they should be, performance-wise" with MP/MT products; most were still meaningfully improving MP/MT efficiency/scalability even ten years after making the shift.

So my speculation is that, absent GvR's endorsement and support, no one has taken on the long trudge for Python and its GIL. Even if they were to do so today, it'd be Python 4.x timeframe before you'd say "Wow! We're really over the MT hump!"

Perhaps there is some magic that separates Python and its runtime from all the other stateful infrastructure software--all the language runtimes, operating systems, transaction monitors, and database managers that have gone before. But if so, it's unique or nearly so. Everyone else removing a GIL-equivalent has taken five-plus years of hard, committed effort and investment to get from MT-not to MT-hot.


答: No, any design is only going to survive scaling out as far as could be conceived at the design phase. Eventually the system will hit a wall or inefficiency where it cannot be scaled out, or scaling out further decreases efficiency without a new architecture.</p>

Title: Can one always design for scale out?

Body:
When designing an application where scalability is going to be important, it would be useful if the application could be designed to "scale out" on cheap commodity servers. However, there are many scale up applications (see my previous question).

Is it always possible to design an application to scale out, or do some requirements make scale out impossible, even if all software can be designed from scratch and no legacy software is involved?

I am looking for cases where scale out is an inferior solution to scale up, and where good application design could not mitigate this. In other words, the application cannot be designed to scale out effectively, no matter how hard one tries.

I am not looking for cases where legacy software or data structures -- or even entire architectures (as in the answer to the previous question) -- were not designed to scale out and could not be changed.

Alternatively, if there is reason to believe that one can always scale out, if the application is well designed, I would love to know that as well.

Edit: I am looking for an explicit example of such a system, if there is one.


问:
Is SQL Server Sevice Broker appropriate for transfering large volumns of records from one database to another
I have a database that collects and process large volumes of records

However due to the legacy nature of the system im working with (and the powers that be) i need to transfer (processed) records to a different database instance (with some small field modifications along the way)

The options i have identified are as follows

I could set up a schedule task with a stored proc to transfer the records with any logic needed

The company i work for is insisting i dump records to a file and ftp them and the use some existing logic to poll and insert

SQL Service broker which seems to have built in fault tolerance, and has facilities to do this actual task

Im just wondering what some potential pitfalls of sql service broker might be
and is it the right technology for the job

Anyone have any suggestions

答: I've written a system to handle offline processing with Service Broker as its core. It's not the most intuitive process, and it definitely sounds like overkill for your needs. A small sproc called by a scheduled task will definitely be easier to program and maintain. If you try Service Broker, I can promise you a steep learning curve and some diagnostic bumps along the way.</p>


问:
Object pointer with privileged access?
Let's say you have

class Car with Car->tune() and Car->drive()

class CarDriver

class CarMechanic

Each CarDriver and CarMechanic object has a reference/pointer to a Car object.
The CarDriver object will call $this->car->drive(), the CarMechanic object will call $this->car->tune().

The question is, how can I prevent a CarDriver from calling $car->tune() ?

There are probably two approaches:

Language level: Using access modifiers like public/private/protected, and then either abuse inheritance or use more exotic language features like a C++ "friend class".

Composition: Split the car into a CarFrontend and CarBackend, or create a DriverCarWrapper, which exposes only the drive() method.

This seems like a good idea, but it will likely introduce clutter and code duplication, which might outweigh the benefits of access control. Especially if there are many methods that should be accessible to the car driver.

Are there any other solutions that I missed?

Background: My real-world use case is in PHP, but I am asking this as a cross-language question.

答:
What you want to do sounds like Double Dispatch.

Basically, you want to dispatch the call to the Car class based on whether the object interacting with the Car is a CarDriver or CarMechanic.

One popular way of implementing this in object oriented languages is with the Visitor pattern.

In your case, you would create DriverVisitor and MechanicVisitor classes that the Driver and Mechanic classes call respectively. These visitors would in turn invoke the appropriate methods on the Car class that are allowable for the Driver and Mechanic roles.

Title: Object pointer with privileged access?


问:
Coupling business logic with class definitions
Perhaps it is my scripting background, but my perspective is that the purpose of an object should be to contain data. Classes are necessary in that they provide templates and standard ways to classify and construct objects with particular data attributes. Standard protocol in object oriented design seems to be to include business logic in class definitions.

In a system that depends on state, I understand the motivation behind binding methods with the sole purpose of altering or retrieving the state of an object to the class definition. So let's exclude simple getters, setters, and constructors from the scope of the conversation.

The problems, as I see them, start when processes are bound to an object.

Consider a simple "string.equals()" method that accepts a string as a parameter, and returns a boolean indicating whether strings are equivalent. I think this simple method can cause enormous problems.

Let's say I have a LocationString class that inherited from String, where equals() is overridden as follows:

```<code>Boolean equals(LocationString otherLocation) {
    return extractCountry().equals(otherLocation.extractCountry());
}

String extractCountry(){
    return this.getCountry();
}
</code>```

Now, assume I have a thousand such different objects with different equals() definitions, some of which are the same, and some of which are vastly different. Suddenly, the definition of equality changes for some of my classes. It becomes extremely painful to work out which equals definitions need to be changed. It becomes painful to work out whether other internal processes which called the equals method need to be changed. And eventually as the system expands, more hours are wasted searching through the code than analyzing it.

As such, to me it makes far more sense that a generic equals rule is defined that modifies its behavior polymorphically depending on the inputs it receives. Consider the following definition:

```<code>public static Boolean equals(Object a, Object b) {
    List locationClasses = Arrays.asList("LocationString")
    if (locationClasses.contains(a.getClass().getName()) && locationClasses.contains(b.getClass().getName()){
        return equals(a.extractCountry(), b.extractCountry());
    }
    else {
        return StringUtils.equals(a,b);
    }
}
</code>```

If all processes were defined externally, a change in process would not be all that difficult to correct and provided these more generic rules could be stored and categorized logically, they would be easy to find and would only need to change once as the business logic changes.

This approach seems relatively simple and straight forward. Why then, has it been generally rejected in favor of the typical object oriented coupling of data with methods?

答:
To me, any given object has a sort of behavioral profile which is the domain and/or scope of varying functions it can participate in. The breadth and character of this behavioral profile will vary widely depending on the object. Object oriented design certainly has some relation with cognitive science, since it is concerned with how the programmer models the given problem space. You might consider this behavioral profile as a collection of affordances: http://en.wikipedia.org/wiki/Affordance, but over a more abstract rather than concrete domain.

With that notion on board, I would say that the decision to couple data and behavior depends on an evaluation of the domain of potential affordances for the object. Sometimes the properties of a given object are so action implying over a comparitively restricted domain that it makes sense to tightly couple the data and functionality. In other cases, the data is more abstract, and the scope of possible functions is large.

Pragmatically speaking, I definitely prefer to keep data and behavior separate if possible. I am partial to using plain data-only classes along with a healthy use of generic fluent interfaces, and especially higher order functions / monadic constructs. Thats my short answer..


问:
Adding field to the class at runtime - design pattern
Imagine your customer want's to have a possibility to add new property (e.g. color) to product in their eshop in their CMS.

Instead of having properties as fields:

```<code>class Car extends Product {
   protected String type;
   protected int seats;
}
</code>```

You would probably end up doing something like:

```<code>class Product {
   protected String productName;
   protected Map<String, Property> properties;
}

class Property {
   protected String name;
   protected String value;
}
</code>```

That is, creating own type system on top of existing one. It feels to me like this could be seen as creating domain specific langauge, or couldn't?

Is this approach a known design pattern? Would you solve the problem differently? I know there are languages where I can add a field in runtime, but what about database? Would you rather add/alter columns or used someting as shown above?

Thank you for your time :).

答:
Is this approach a known design pattern?

In XML and HTML, these would be the attributes to a node/element.  I've also heard them called extended properties, name/value pairs, and parameters.

Would you solve the problem differently?

That is how I would solve the problem, yes.

I know there are languages where I can add a field in runtime, but what about database?

A database would be like Java, in some senses.  In pesudo-sql:

```<code>TABLE products
(
    product_name VARCHAR(50),
    product_id INTEGER AUTOINCREMENT
)

TABLE attributes
(
    product_id INTEGER,
    name VARCHAR(50),
    value VARCHAR(2000)
)
</code>```

This would correspond to the Java

```<code>class Product {
   protected String productName;
   protected Map<String, String> properties;
}
</code>```

Note that there is no need for a Property class, as the Map stores the name as the key.

Would you rather add/alter columns or used someting as shown above?

I've tried the add/alter column thing, and it was a nightmare.  It can be done, but things kept getting out of sync and I never got it to work well.  The table structure I outlined above has been much more successful.  If you need to do searches and order by's, consider using an attributes table for each data type (date_attributes, currency_attributes, etc.) or adding some of the properties as good old database columns in the products table.  Reports are often much easier to write on database columns then sub tables.

Body:
Is this approach a known design pattern?

In XML and HTML, these would be the attributes to a node/element.  I've also heard them called extended properties, name/value pairs, and parameters.

Would you solve the problem differently?

That is how I would solve the problem, yes.

I know there are languages where I can add a field in runtime, but what about database?

A database would be like Java, in some senses.  In pesudo-sql:

TABLE products
(
    product_name VARCHAR(50),
    product_id INTEGER AUTOINCREMENT
)

TABLE attributes
(
    product_id INTEGER,
    name VARCHAR(50),
    value VARCHAR(2000)
)

This would correspond to the Java

class Product {
   protected String productName;
   protected Map<String, String> properties;
}

Note that there is no need for a Property class, as the Map stores the name as the key.

Would you rather add/alter columns or used someting as shown above?

I've tried the add/alter column thing, and it was a nightmare.  It can be done, but things kept getting out of sync and I never got it to work well.  The table structure I outlined above has been much more successful.  If you need to do searches and order by's, consider using an attributes table for each data type (date_attributes, currency_attributes, etc.) or adding some of the properties as good old database columns in the products table.  Reports are often much easier to write on database columns then sub tables.


问:
Returning results of method on batch list?
The title is a bit vague so I'll try to elaborate.

I have a function

```<code>makeFoo(int bar) -> returns Foo or throws Exception.
</code>```

I also have a batch version of this

```<code>makeFoos(int[] bars) -> returns Foo[]
</code>```

which basically loops through bars and runs makeFoo() on them.

Issue is, if while running makeFoos(), makeFoo() throws an Exception, what do I do? I don't want to break out of makeFoos() because I want to continue processing the rest of the bars. But also, I want to retain the Exception that was thrown.

My initial solution is, instead of returning Foo[], I return Result<Foo>[]. Where Result is a wrapper class:

```<code>class Result<T>:
    T data;
    Exception e;
</code>```

Is there a better way that I can approach this?

Edit: Apologies if this is considered a duplicate post, but I posted a more general version of this question here: https://softwareengineering.stackexchange.com/questions/247149/result-object-vs-exceptions

答:
In all honesty, I like your answer best.  The return array should be the same size as the input array, so you can easily determine which inputs resulted in which outputs.  You can also loop through the results array and check to see if the exception is null or not, and if not, then process the result.

Another way to do this would be to pass in an array that has the input parameters, the output result, and an exception all in one area.  In that case, you could return the number of exceptions (or non-exceptions, or the first exception), and you could get a quick check on if everything worked.  For example:

```<code>class BatchedFoo
{
    int i;
    Foo data;
    Exception e;
}

makeFoos(BatchedFoo []) -> returns int
</code>```

Body:
In all honesty, I like your answer best.  The return array should be the same size as the input array, so you can easily determine which inputs resulted in which outputs.  You can also loop through the results array and check to see if the exception is null or not, and if not, then process the result.

Another way to do this would be to pass in an array that has the input parameters, the output result, and an exception all in one area.  In that case, you could return the number of exceptions (or non-exceptions, or the first exception), and you could get a quick check on if everything worked.  For example:

class BatchedFoo
{
    int i;
    Foo data;
    Exception e;
}

makeFoos(BatchedFoo []) -> returns int


答:
No, it is not always possible to "scale out". As Amdahl's law shows us, your abilities to parallelize tasks are limited by sections of your algorithm that must be done sequentially.

I think there are few CRUD applications where you cannot get significant performance benefits from scaling out. However, there are situations where you have significant sequential computing requirements, such as many scientific applications, where you are fundamentally required to compute one thing before another. This CS stack exchange question shows a few examples.

Title: Can one always design for scale out?


问:
Do you have any techniques to help you get started in the morning?
I'm not a morning person.  I got into the habit of working from 10 to 7/8, sometimes 9.

Nevertheless, many of the places I've worked for have asked for something more like 8 to 6 or 9 to 6.

Honestly, I have a real tough time getting my brain going in the morning and getting into the code.  Once I do, I can concentrate for a long time and be quite productive.

It's getting into the code and starting to get going with it that I have trouble with.  Sometimes I will waste time in the morning just because I can't focus on the problem (yes, I am writing this in the morning).

I've put all my busy work in the morning, but as a developer, I don't have a lot of busy work.

It kind of becomes a self reinforcing thing; because you don't get things done in the morning, you have to stay later.  Because you have to stay later, then its harder to get started the next morning, etc.

Do other people have trouble with this?  If so, any techniques you've learned?

Thanks!

答:
Leave work at the end of a day with a
specific problem.

Create some sort of prompt so you see
it first thing when you get to the
office the next morning.

Go to bed early (I know easier said
than done.).

Wake up with plenty of time to get ready for work.

Eat breakfast

Shower it helps wake you up, so don't do it before bed (unless so stinky you can't stand it.)

Don't start caffiene intake until you are at your desk.

See the problem from #1? Get to work.

Much easier if you get a routine going.

ContentLicense: CC BY-SA 2.5


问:
When is Singleton appropriate?
There is a widely accepted opinion that Singleton is an anti-pattern. As usual, there are always exceptions to the rule. Can you explain why Singleton is a bad choice in general and give an example of some valid use cases for it?</p>

答:
Why do people use it?

I have seen quite a number of singletons in the PHP world. I don't remember any use case where I found the pattern to be justified. But I think I got an idea about the motivation why people did use it.

Single instance.

"Use a single instance of class C throughout the application."

This is a reasonable requirement e.g. for the "default database connection". It does not mean you won't ever create a second db connection, it just means you usually work with the default one.

Single instantiation.

"Do not allow class C to be instantiated more than once (per process, per request, etc)."

This is only relevant if instantiating the class would have side effects that conflict with other instances.

Often these conflicts can be avoided by redesigning the component - e.g. by eliminating side effects from the class constructor. Or they can be solved in other ways. But there might still be some legit use cases.

You should also think about whether the "only one" requirement really means "one per process". E.g. for resource concurrency, the requirement is rather "one per entire system, across processes" and not "one per process". And for other things it is rather per "application context", and you just happen to have one application context per process.

Otherwise, there is no need to enforce this assumption. Enforcing this also means you cannot create a separate instance for unit tests.

Global access.

This is legit only if you have no proper infrastructure to pass objects around to the place where they are used. This could mean that your framework or environment sucks, but it might not be within your powers to fix that.

The price is tight coupling, hidden dependencies, and everything that's bad about global state. But you are probably already suffering these.

Lazy instantiation.

This is not a necessary part of a singleton, but it seems the most popular way to implement them. But, while lazy instantiation is a nice thing to have, you don't really need a singleton to achieve that.

Typical implementation

The typical implementation is a class with a private constructor, and a static instance variable, and a static getInstance() method with lazy instantiation.

In addition to the problems mentioned above, this bites with the single responsibility principle, because the class does control its own instantiation and life cycle, in addition to the other responsibilities that the class already has.

Conclusion

In many cases, you can achieve the same result without a singleton, and without global state. Instead, you should use dependency injection, and you might want to consider a dependency injection container.

However, there are use cases where you have the following valid requirements remaining:

Single instance (but not single instantiation)

Global access (because you are working with a bronze age framework)

Lazy instantiation (because it is just nice to have)

So, here is what you can do in this case:

Create a class C that you want to instantiate, with a public constructor.

Create a separate class S with a static instance variable and a static S::getInstance() method with lazy instantiation, that will use class C for the instance.

Eliminate all side effects from the constructor of C. Instead, put these side effects in the S::getInstance() method.

If you have more than one class with the above requirements, you may consider to manage the class instances with a small local service container, and use the static instance only for the container. So, S::getContainer() will give you a lazy-instantiated service container, and you get the other objects from the container.

Avoid calling the static getInstance() where you can. Use dependency injection instead, whenever possible. Especially, if you use the container approach with multiple objects that depend on each other, then none of these should have to call S::getContainer().

Optionally create an interface that class C implements, and use this to document the return value of S::getInstance().

(Do we still call this a singleton? I leave this to the comment section..)

Benefits:

You can create a separate instance of C for unit testing, without touching any global state.

Instance management is separated from the class itself -> separation of concerns, single responsibility principle.

It would be quite easy to let S::getInstance() use a different class for the instance, or even dynamically determine which class to use.

ContentLicense: CC BY-SA 4.0

Title: When is Singleton appropriate?


问:
What's a good simple way to combat the n+1 problem?
I'm trying to better understand performance in PHP.  One issue I'm thinking about is the n+1 problem. By n+1 I mean something like this:

```<code>$posts = Posts::getPosts();

foreach($posts as $post) {
  $comments = Comments::getComments(array('post_id' => $post->id));
  // do something with comments..
}
</code>```

It's quite inefficient as we have to do many queries for every post to get the comments.

Is something like this better? It's more PHP code but only two queries will be executed:

```<code>$posts = Posts::getPosts();

// get the ids into an array
$post_ids = array();
foreach($posts as $post) {
  array_push($post_ids, $post->id);
}

// get comments for ALL posts in one query by passing array of post ids (post_id IN (...))
$comments = Comments::getComments(array('post_id' => $post_ids));

// map comments to posts
foreach($posts as $key => $post) {
  foreach($comments as $comment) {
    if($post->id == $comment->post_id) {
      $post->pushComment($comment);
    }
  }
}

foreach($posts as $post) {
  $comment = $post->comments;
  // do something with comments..
}
</code>```

This is much more PHP, and kinda messy too, but this time I'm only using two queries (one for posts, the other for fetching ALL comments of those posts in one query). Is this a good proposal to tackle the n+1 problem in PHP?

Also, how do frameworks generally deal with this under the hood?

答:
The inefficiency/messiness is coming from "hydrating" your data too early and too often. By "hydrating," I mean instantiating data objects from (what I assume are) records in your database.

You don't always need to deal with data objects. For example, with this code...

```<code>$posts = Posts::getPosts();

// get the ids into an array
$post_ids = array();
foreach($posts as $post) {
  array_push($post_ids, $post->id);
}
</code>```

...I assume Posts::getPosts() fetched some rows from the database, and created a new Post data object for each row, only to extract the IDs and throw out the data objects. If you added a function like Posts::getPostIds() that returned an array of numbers, you could call it in place of that block of code.

```<code>$post_ids = Posts::getPostIds();
</code>```

The remainder of your code brings up a tougher question. Ideally you still want to be able to do something like your original example, except you want the "comment" data objects populated from a recordset you got beforehand rather than running a separate query for each post. Maybe in place of Comments::getComments, you could add a getComments instance method to Post to handle this, and pass it a recordset.

This is all assuming you actually need to get comments for multiple posts at once; I can't think of a situation where you'd need to do that offhand but I assume you know what you're doing.

As for understanding how other frameworks handle ORM, I'd recommend taking a look at redbean's documentation to see how it's used, and if you're interested, have a look at the source.

Body:
The inefficiency/messiness is coming from "hydrating" your data too early and too often. By "hydrating," I mean instantiating data objects from (what I assume are) records in your database.

You don't always need to deal with data objects. For example, with this code...

$posts = Posts::getPosts();

// get the ids into an array
$post_ids = array();
foreach($posts as $post) {
  array_push($post_ids, $post->id);
}

...I assume Posts::getPosts() fetched some rows from the database, and created a new Post data object for each row, only to extract the IDs and throw out the data objects. If you added a function like Posts::getPostIds() that returned an array of numbers, you could call it in place of that block of code.

$post_ids = Posts::getPostIds();

The remainder of your code brings up a tougher question. Ideally you still want to be able to do something like your original example, except you want the "comment" data objects populated from a recordset you got beforehand rather than running a separate query for each post. Maybe in place of Comments::getComments, you could add a getComments instance method to Post to handle this, and pass it a recordset.

This is all assuming you actually need to get comments for multiple posts at once; I can't think of a situation where you'd need to do that offhand but I assume you know what you're doing.

As for understanding how other frameworks handle ORM, I'd recommend taking a look at redbean's documentation to see how it's used, and if you're interested, have a look at the source.


问:
"Cross-cutting" domain in DDD
I've recently started studying Domain Driven Design and so far it looks like it will help tremendously with my team's current project. I'm running into a bit of a problem though actually determining what the domains are in our project.

After my inital pass, I ended up with something like this:

I have three domains: A, B, and C for different areas of the business. However, for each of those I need to track current data as well as create plans for the future. This led me to add a Planning domain, but I don't see a clear way to separate Planning from the other domains. Either A, B, and C need to have detailed knowledge of Planning or vice versa.

Example: Let's say Domain A is "inventory". The implementation has messages for things like ItemsAdded and ItemsRemoved and it tracks quantities of various products. Then if I want to allow for planning of future quantities of items I need to add classes to the model which will allow me to associate future product info with various time windows. This kind of feels like a new domain except the same planning and time window logic also applies to Domain B and C.

Essentialy, I have a cross-cutting domain. I feel like I must be looking at this the wrong way.

Maybe I actually have 6 domains and a library used by 3 of them?

答: What is stopping you from creating a "Planning Domain" and have other 3 depend on this domain? This seems quite normal dependency in library/module.</p>

Title: "Cross-cutting" domain in DDD


答:
First, what you are describing sounds quite a lot like functional approach to things. Separating data and functions operating on those data is modus operandi of all functional language. Try looking at Haskell for inspiration.

Second thing that I see is that you seem to lack concept of abstraction. If you have identified multiple classes, which have same or similar behavior, it should be your top priority to refactor this behavior into a reusable form in either inheritance or composition. This is actually the hardest part of software and OO design.

And why is separation of data and behavior frowned upon in OO design? First, encapsulation. Making sure there is only limited scope, which can access and change a state, simplifies reasoning about code, because you don't have to worry about variables changing without your knowledge. By having outside functions access this data, it means encapsulation is broken. Second are virtual methods. Virtual methods are ways to achieve late binding in OO languages. You cannot have virtual methods if they are not parts of the class.

And last thing, every time I see someone talking about OO design, I cannot help myself and link to this excersise.

And some final words: Just because some approach might seem simple and straightforward on simple and straightforward example doesn't mean it will work on complex and huge example. Try applying your approach to project which has thousands of classes with millions of functions and extremely complex business logic. You will soon realize it is far from simple.

Title: Coupling business logic with class definitions

Body:
Perhaps it is my scripting background, but my perspective is that the purpose of an object should be to contain data. Classes are necessary in that they provide templates and standard ways to classify and construct objects with particular data attributes. Standard protocol in object oriented design seems to be to include business logic in class definitions.

In a system that depends on state, I understand the motivation behind binding methods with the sole purpose of altering or retrieving the state of an object to the class definition. So let's exclude simple getters, setters, and constructors from the scope of the conversation.

The problems, as I see them, start when processes are bound to an object.

Consider a simple "string.equals()" method that accepts a string as a parameter, and returns a boolean indicating whether strings are equivalent. I think this simple method can cause enormous problems.

Let's say I have a LocationString class that inherited from String, where equals() is overridden as follows:

```<code>Boolean equals(LocationString otherLocation) {
    return extractCountry().equals(otherLocation.extractCountry());
}

String extractCountry(){
    return this.getCountry();
}
</code>```

Now, assume I have a thousand such different objects with different equals() definitions, some of which are the same, and some of which are vastly different. Suddenly, the definition of equality changes for some of my classes. It becomes extremely painful to work out which equals definitions need to be changed. It becomes painful to work out whether other internal processes which called the equals method need to be changed. And eventually as the system expands, more hours are wasted searching through the code than analyzing it.

As such, to me it makes far more sense that a generic equals rule is defined that modifies its behavior polymorphically depending on the inputs it receives. Consider the following definition:

```<code>public static Boolean equals(Object a, Object b) {
    List locationClasses = Arrays.asList("LocationString")
    if (locationClasses.contains(a.getClass().getName()) && locationClasses.contains(b.getClass().getName()){
        return equals(a.extractCountry(), b.extractCountry());
    }
    else {
        return StringUtils.equals(a,b);
    }
}
</code>```

If all processes were defined externally, a change in process would not be all that difficult to correct and provided these more generic rules could be stored and categorized logically, they would be easy to find and would only need to change once as the business logic changes.

This approach seems relatively simple and straight forward. Why then, has it been generally rejected in favor of the typical object oriented coupling of data with methods?


问:
Track performance numbers
I have a (large) software project in a git repository. There is a benchmark suite that spits out lots of numbers (time per benchmark, allocations done per benchmark) I’d like to track.

What is the best way to get a grip on intended and unintended performance changes in such a setup?

(Ideally, I think, there is a piece of free software that takes a git repository, a script to obtain the numbers, schedules jobs for new commit, also checks old commits when idle and presents a slick web interface with lots of graphs and analysis tools.)

答:
Some inspiration of what is possible can be found on these existing Performance Dashboards. Unfortunately, not all of them are free software, and neither of them seems to be ready-to-use by others:

https://datazilla.mozilla.org/

http://goperfd.appspot.com/perf

http://speed.pypy.org/

https://chromeperf.appspot.com/

Title: Track performance numbers


问:
To depend on source code or on binary?
We have two in-house projects, A and B, developed by different teams with B dependent on A. Since source code for both projects is stored in git, I've included project A as a submodule into project B and configured the build system to build both in right order. An alternative solution would be to consume A via a binary repository manager such as Artifactory or Nexus.

I wonder about pros and cons of depending on source code vs depending on binary artifacts. When is one better than the other? So far I managed to come up with following factors, but I am really keen on hearing other opinions.

Depending on source code is better

if you don't have a binary repository manager

if you need to depend on pre-release version of another project

if you need to patch another project

because it is easier to browse dependency source code in IDE

Depending on binaries is better

to minimize build time

to avoid the hassle of setting up another project's build environment

答:
I would always recommend binary dependencies over source dependencies. Most of the pros you list for source code can be easily incorporated into binary dependencies as well.

First, it is very quick to set up a Nexus repo on some local server. The benefits of having a binary repo far outweigh the setup effort/cost. This is almost a pre-requisite for the rest of my answer :)

For pre-release versions, your projects should be deploying -SNAPSHOT versions into the artifactory. There can be a nice and clear distinction between versions, such as:

projectA-3.2.0-SNAPSHOT : Active development, may change at any time

projectA-3.2.0-RC1 : Release candidate

projectA-3.2.0 : Production release

All these versions can be stored together in your artifactory. Your projects would know exactly what they are compiling against.

For patches, git is your friend. Fork the repo and put in a patch version number, such as projectA-3.2.1-FOR_PROJ_B. Notice the .1 showing a patch release and also the descriptor. This will also make it easy for the project to pull in those changes back to the master later.

For source code, you can configure your build tool to generate a "source jar" and deploy it alongside the binary jar in the artifactory. Most IDEs can look-up the source jar name and automatically download the source for you.

Another big reason to stay away from source code is that you are tied to the build system, tool-chain and compile-time dependencies of Project A. If you see a build failure in Project B, you have to first investigate if Project A or Project B caused an error. If it was Project A, you have go track down people from that project to fix their build. This adds a lot of overhead to your build cycle.

Title: To depend on source code or on binary?


答:
Your original approach does lazy loading while your modified code does eager loading.

You are absolutely right that eager loading is more efficient in your situation. In most cases, minimising the number of queries is the best thing you can do to speed up your app. If you were only going to look at one of the posts then lazy loading would be faster.

Most ORMs (at least, all the good ones!) support lazy and eager loading. For example, SQLAlchemy has extensive support. You were asking about PHP; I expect the main PHP ORMs have similar features. Usually the ORM defaults to lazy loading, but you can tell it to eager load a particular table when executing a query.

In fact, it is possible load all your data in one query. SQLAlchemy has two eager loading modes: joined and subquery. joined does it all in one query, which is sometimes the most efficient way, but it does end up querying duplicate data. subquery eager load is just like your approach. I'm not sure whether PHP ORMs support this distinction.

Title: What's a good simple way to combat the n+1 problem?

Body:
I'm trying to better understand performance in PHP.  One issue I'm thinking about is the n+1 problem. By n+1 I mean something like this:

```<code>$posts = Posts::getPosts();

foreach($posts as $post) {
  $comments = Comments::getComments(array('post_id' => $post->id));
  // do something with comments..
}
</code>```

It's quite inefficient as we have to do many queries for every post to get the comments.

Is something like this better? It's more PHP code but only two queries will be executed:

```<code>$posts = Posts::getPosts();

// get the ids into an array
$post_ids = array();
foreach($posts as $post) {
  array_push($post_ids, $post->id);
}

// get comments for ALL posts in one query by passing array of post ids (post_id IN (...))
$comments = Comments::getComments(array('post_id' => $post_ids));

// map comments to posts
foreach($posts as $key => $post) {
  foreach($comments as $comment) {
    if($post->id == $comment->post_id) {
      $post->pushComment($comment);
    }
  }
}

foreach($posts as $post) {
  $comment = $post->comments;
  // do something with comments..
}
</code>```

This is much more PHP, and kinda messy too, but this time I'm only using two queries (one for posts, the other for fetching ALL comments of those posts in one query). Is this a good proposal to tackle the n+1 problem in PHP?

Also, how do frameworks generally deal with this under the hood?


答:
TLDR: scaling out is not possible for sequential algorithms or atomic data, and can be a bad way to spend a development budget.

As others have pointed out, sequential algorithms cannot be scaled out. An example could be financial transaction processing, where transactions must be applied in the right order. However, in typical CRUD scenario's you are unlikely to run into sequential long-lived algorithms, so my answer will be directed towards cases where you are not dealing with such a sequential algorithm.

The key to scaling out is avoiding synchronization work between servers, because it's the synchronization work that introduces bottlenecks as you scale out. For example, application servers can avoid storing data locally, fetching everything fresh from the database layer for each request. This way no data needs to be synchronized between application servers (e.g. session data). Another example is sharding of databases, where different database servers hold different parts of the data, which means no synchronization work is needed between those servers.

So, the answer is that even in cases where there are no algorithmic bottlenecks to scaling out there are situations where scaling out is not possible, and these have to do with atomicity of data. If you are dealing with a set of data which cannot be sharded and must be shared across nodes to implement your logic (e.g. a non-divisible social graph), then your only option is scaling up: one big server that holds the whole dataset instead of a bunch of smaller ones that each hold a subsegment. These situations are however pretty rare, so in most cases software can be designed to be scaled out, provided the design takes this into account from the beginning.

Another thing to take into account is design cost. Designing for scaling out increases the development cost. Database sharding for example requires additional application-level logic to deal with the shards. For that reason, it makes sense to consciously choose to scale up, especially at the DB layer. For an example, see the site you are looking at right now.

Title: Can one always design for scale out?


问:
Should we always create web application scalable?
Fact that now a days everyone is moving to cloud due to ease in scalability, configuration and maintenance.

But my question is should we develop all web application scalable event if it is not required?
If an application is low budget with no such special requirements then why we should add efforts to make it support sclability?

There are many factors to consider while making application scalable. If client has no specific requirement, small application contains 3-4 modules, client budget is low and is using on premises servers.

To make application scalable, we have to make sure of session, IO files, caching etc...

答:
It is difficult to imagine a situation in which being scalable is not important enough to take into consideration when building a web application, however, I imagine it could happen.  If you are making a web application in order to try out new technologies, it does not need to be "production ready."  Likewise if you know that this is going to be a web application used by few that is not likely to grow, then scalability would be low on the list of priorities.

However, if the client did not explicitly specify, it could be because A) it is not important for the client or B) the client doesn't even know what "scalability" entails and thus didn't specify it.

In the case of A, no problems there.  However, in the case of B, it should be your responsibility to verify if scalability is, in fact, an issue.  Even though budget is low and you think it might be obvious enough, you should make it clear.  Make it cost more, whatever you want, however at least both parties are at an agreement.  If the client expects a web application that works just as well with 1 user as with a million users, it's not going to end well for him and ultimately for you as well.

There are some programming practices that improve scalability and have no serious downfalls, save for taking slightly longer to implement (caching common requests, reducing session data usage, database connection pooling).  For these, I would encourage you to always take the longer route whether or not the client values scalability.  This makes you a better programmer, and it helps you think in the mentality of always striving to write a better program.  That's just my 10 cents though.


答:
As always in such cases YAGNI (You Ain't Gonna Need It) and KISS (Keep It Simple Stupid) applies.

In other words, keep your application as simple as possible, and only add features which are currently needed (don't add features which may be needed in the future).

Simple systems are easier to develop, easier to maintain and are often easy to scale.

Title: Should we always create web application scalable?

Body:
Fact that now a days everyone is moving to cloud due to ease in scalability, configuration and maintenance.

But my question is should we develop all web application scalable event if it is not required?
If an application is low budget with no such special requirements then why we should add efforts to make it support sclability?

There are many factors to consider while making application scalable. If client has no specific requirement, small application contains 3-4 modules, client budget is low and is using on premises servers.

To make application scalable, we have to make sure of session, IO files, caching etc...


答:
I would go for binary dependency, because almost none of your considerations favouring me is without critic:

if you don't have a binary repository manager: that is ok, but I do not think it is that difficult to stablish one. In the most basic level, a shared folder where only the people in charge to deciding which version use can write.

if you need to depend on pre-release version of another project: metacubed already covered it. Either with source code or binary dependencies, you should stick to a well-stablished version, even if it is pre-release. Of course, when developing against a pre-release, odds are that you will need switch to an updated version often due to bug solving.

if you need to patch another project: do you mean that your team is going to patch the project made from another team in the same house? I would say the most sane strategy is tell the other team the issues you have with their project.

because it is easier to browse dependency source code in IDE: You should not need to snoop into the inner workings of your dependencies, because it will mean:

project A is poorly documented

your coders will end using "undocumented features" of project A that may disappear without previous warning at any update.

Title: To depend on source code or on binary?

Body:
We have two in-house projects, A and B, developed by different teams with B dependent on A. Since source code for both projects is stored in git, I've included project A as a submodule into project B and configured the build system to build both in right order. An alternative solution would be to consume A via a binary repository manager such as Artifactory or Nexus.

I wonder about pros and cons of depending on source code vs depending on binary artifacts. When is one better than the other? So far I managed to come up with following factors, but I am really keen on hearing other opinions.

Depending on source code is better

if you don't have a binary repository manager

if you need to depend on pre-release version of another project

if you need to patch another project

because it is easier to browse dependency source code in IDE

Depending on binaries is better

to minimize build time

to avoid the hassle of setting up another project's build environment


问:
Designing database related methods, which is better to return: true/false or row affected?
I have some methods that perform some data changing in a database (insert, update, and delete). The ORM I'm using return row-affected int values for those type of method. What should I return for "my method", in order to indicate the success/failure state of the operation?

Consider the code that is returning an int:

A.1

```<code>public int myLowerLevelMethod(int id) {
    ...
    int affectedRows = myOrm.deleteById(id)
    ...

    return affectedRows;
}
</code>```

Then usage:

A.2

```<code>public void myOtherMethod() {
    ...
    int affectedRows = myLowerLevelMethod(id)

    if(affectedRows > 0) {
        // Success
    } else {
        // Fail
    }
}
</code>```

Compare to using boolean:

B.1

```<code>public boolean myLowerLevelMethod(int id) {
    ...
    int affectedRows = myOrm.deleteById(id)
    ...

    return affectedRows > 0;
}
</code>```

Then usage:

B.2

```<code>public void myOtherMethod() {
    ...
    boolean isSuccess = myLowerLevelMethod(id)

    if(isSuccess) {
        // Success
    } else {
        // Fail
    }
}
</code>```

Which one (A or B) is better? Or pros/cons of each?

答:
Returning the number of affected rows is better because it gives additional information about how the operation proceeded.

No programmer will blame you because he/she has to write this to check if they got some changes during the operation:

```<code>if(affectedRows > 0) {
    // success
} else {
    // fail
}
</code>```

but they will blame you when they'll have to know the number of affected rows and they realize there are no method for getting that number.

BTW: if by "failure" you mean a syntactical query error (in which case the number of affected rows is obviously 0) then throwing the exception would be more appropriate.

Body:
Returning the number of affected rows is better because it gives additional information about how the operation proceeded.

No programmer will blame you because he/she has to write this to check if they got some changes during the operation:

if(affectedRows > 0) {
    // success
} else {
    // fail
}

but they will blame you when they'll have to know the number of affected rows and they realize there are no method for getting that number.

BTW: if by "failure" you mean a syntactical query error (in which case the number of affected rows is obviously 0) then throwing the exception would be more appropriate.


问:
Am I wrong in thinking that needing something like AutoMapper is an indication of poor design?
Automapper is an "object-object mapper" for .Net, which means copying objects from a class into another class that represents the same thing.

Why is this ever useful? Is the duplication of classes ever useful/good design?

答:
Is the duplication of classes ever useful/good design?

Its good practice to have a separate view model class for use in the UI layer rather than using the same class that's used in the data layer. Your UI/web page might need to display other bits of information that aren't strictly related to the data entity.  By creating this extra class, you are giving yourself the freedom to tweak your UI easily as requirements change over time.

As far as using AutoMapper goes, I personally avoid it for 3 reasons:

Silent failures more common

Because AutoMapper maps between properties automatically, changing a property name on one class and not the other will cause the Property mapping to be skipped.  The compiler won't know.  Automapper won't care.

Lack of static analysis

So you've been given a large code base to deal with.  There's a million classes with a million properties. Alot look like they aren't used, or are duplicates. That "Find all references" tool in Visual Studio will help you see where properties are used and help build a map in your head of how the whole application hangs together.  But wait, there's no explicit references to half the properties because Automapper is being used.  My job is now a hell of a lot harder.

Late requirements that increase complexity

Automapper is all fine and dandy when all you want to do is copy the values from ONE class to another (as is often the case at the START of development), but remember those requirements that change over time?  What if you now need to get values from other parts of your application, perhaps specific to the user that's logged in or some other contextual state?

AutoMapper's pattern of creating the one-to-one class mappings at application start doesn't lend itself well to these sorts of context-specific changes.  Yes, there's probably ways of making it work, but I usually find it cleaner, simpler and more expressive to write the logic myself.

In summary, before reaching for Automapper to save yourself 30 seconds of manually mapping one class to another, think about this.

Programming is the art of telling another human what one wants the computer to do. - Donald Knuth

With this in mind, ask yourself "is AutoMapper helpful today and will it be tomorrow?"


问:
Why do operating systems do low level stuff in C and C++? Why not just C++?
On the Wikipedia page for Windows, it states the Windows is written in Assembly for the bootloader and task switcher, and C and C++ for kernel routines.

IIRC, you can call C++ functions from an extern "C"'d block. I can get using C for the kernel functions so pure C apps can use them (like printf and such), but if they can just be wrapped in an extern "C " block, then why code in C?

答:
There are various level of embedded platforms using C as programming language
( of course it is your freedom to use assembly language at any time )

For 'Level' I am talking about the Internal SRAM and ROM resource level for a system.

These platforms sometimes are resource constrained ( e.g. some 8051 platforms only have 128 bytes of User SRAM ).

It is meaningless to support dynamic memory allocation with such a small amount for RAM. (new/delete) or even malloc in C.

One of the main improvement from C to C++ is Object-oriented paradigm. C++ is suitable in software with larger memory footprint

but not in embedded firmware that have its size limitation up to 32KB. (e.g. in a 16-bit MCU plaform  )

There is no need to have a C++ compiler which is generally more complicated than C compiler.
( at least SDK providers won't bother to do this ).

In fact I can hardly find a C++ compiler over an  32-bit ARM7 platform.

It just doesn't worth the complexity

In Some
8051(8 bit) : 1MB ROM, 128B RAM

TI MSP430(16 Bit) : 32KB ROM , 4KB RAM

ST Microelectronics ARM 32-bit Cortex™-M3 CPU Core ( STM32F103T4 ) :
16 or 32 Kbytes of Flash memory
6 or 10 Kbytes of SRAM


答:
"Is this better than that?" is not a useful question when the two alternatives don't do the same thing.

If you need to know the affected row count, then you must use version A. If you don't have to, then you can use version B - but any advantage you might gain in terms of less code-writing effort is already gone since you took the trouble to post both versions to an online forum!

My point is: which solution is better depends entirely on what your requirements specifically for this application are, and you know those circumstances much better than we do. There is no industry-wide, safe-to-use, best-practice, won't-get-fired-over-it choice that is better in general; you have to think about it yourself. And for a decision as easily revised as this one, you needn't spend all that much time thinking either.

Title: Designing database related methods, which is better to return: true/false or row affected?

Body:
I have some methods that perform some data changing in a database (insert, update, and delete). The ORM I'm using return row-affected int values for those type of method. What should I return for "my method", in order to indicate the success/failure state of the operation?

Consider the code that is returning an int:

A.1

```<code>public int myLowerLevelMethod(int id) {
    ...
    int affectedRows = myOrm.deleteById(id)
    ...

    return affectedRows;
}
</code>```

Then usage:

A.2

```<code>public void myOtherMethod() {
    ...
    int affectedRows = myLowerLevelMethod(id)

    if(affectedRows > 0) {
        // Success
    } else {
        // Fail
    }
}
</code>```

Compare to using boolean:

B.1

```<code>public boolean myLowerLevelMethod(int id) {
    ...
    int affectedRows = myOrm.deleteById(id)
    ...

    return affectedRows > 0;
}
</code>```

Then usage:

B.2

```<code>public void myOtherMethod() {
    ...
    boolean isSuccess = myLowerLevelMethod(id)

    if(isSuccess) {
        // Success
    } else {
        // Fail
    }
}
</code>```

Which one (A or B) is better? Or pros/cons of each?


答:
I would not recommend any of them. Instead, return nothing (void) on success and throw an exception on failure.

This is for exactly the same reason I choose to declare certain memebers of class private. It also makes function easier to use. More operational context does not always mean better, but it certainly means more complex. The less you promise, the more you abstract away, the easier it is for the client to understand and the more freedom you have in choosing how to implement it.

The question is how to indicate success/error. In this case it's enough to signal failure by throwing an exception and return nothing on success. Why do I have to provide more than the user needs?

Failures/exceptional situations can happen and then you'll have to deal with them. Whether you use try/catch to do it or examine the return codes, is a matter of style/personal preference. The ideas behind try/catch are: separate normal flow from exceptional flow and let exceptions bubble up to the layer where they can be handled most appropiately. So, as many have pointed out already, it depends on whether failure is really exceptional or not.


问:
Why is software OS specific?
I'm trying to determine the technical details of why software produced using programming languages for certain operating systems only work with them.

It is my understanding that binaries are specific to certain processors due to the processor specific machine language they understand and the differing instruction sets between different processors. But where does the operating system specificity come from?
I used to assume it was APIs provided by the OS but then I saw this diagram in a book:

Operating Systems - Internals and Design Principles 7th ed - W. Stallings (Pearson, 2012)

As you can see, APIs are not indicated as a part of the operating system.

If for example I build a simple program in C using the following code:

```<code>#include<stdio.h>

main()
{
    printf("Hello World");

}
</code>```

Is the compiler doing anything OS specific when compiling this?

答:
Actually, the real answer is that if every OS did understand the same executable binary file layout, and you only limited yourself to standardized functions (like in the C standard library) that the OS provided (which OSes do provide), then your software would, in fact, run on any OS.

Of course, the reality is that's not the case. An EXE file doesn't have the same format as an ELF file, even though both contain binary code for the same CPU.* So each operating system would need to be able to interpret all file formats, and they simply didn't do this in the beginning, and there was no reason for them to start doing so later (almost certainly for commercial reasons rather than technical ones).

Furthermore, your program probably needs to do things that the C library doesn't define how to do (even for simple things like listing the contents of a directory), and in those cases every OS provides its own function(s) for achieving your task, naturally meaning there won't be a lowest common denominator for you to use (unless you make that denominator yourself).

So in principal, it's perfectly possible. In fact, WINE runs Windows executables directly on Linux.

But it's a ton of work and (usually) commercially unjustified.

*Note: There's a lot more to an executable file than just binary code. There's a ton of information that tells the operating system what libraries the file depends on, how much stack memory it needs, what functions it exports to other libraries that may depend on it, where the operating system might find relevant debug information, how to "re-locate" the file in memory if necessary, how to make exception handling work correctly, etc. etc.... again, there could be a single format for this that everyone agrees on, but there simply isn't.

Title: Why is software OS specific?


问:
Python API: Extension Module or Pure Module
We have a C++ application, with C++ interfaces changing regularly during development. We also have a C API which is designed to be stable in spite of such changes; the C API is tighly focused on operations which client code might want to do.

Now, we want to add a Python API to make it easy for outside clients (or our own team) to create scripts controlling the application. Given the volatility of the C++ code and the by-design stable nature of the C API, it makes sense to build the Python API over the C one. I know the way to do this is to provide an extension module (implemented in C++) which will be usable in Python and wrap the C API functions.

What I am considering is whether I should make this extension module the Python API (and thus make it as Pythonic as possible), or whether I should keep this an "implementaion detail" of the Python API and build a pure Python module on top of it, which would serve as the actual external API. The extension module would then be a very thing wrapper around the C functions, resigning on being nice Python code.

I see the following pros and cons:

Python API is a C++ extension

This would be more efficient, as there are less intermediary layers. At the same time, it requires boilerplate code to provide Pythonic behaviour in C++, and the code is generally harder to write and maintain (reference counting, pointer tables etc.).

Python API is a module wrapping the C++ extension

This allows us to write Pythonic code in Python, which is easier and less error-prone. It would probably lead to a richer, more complete API as well, since adding functionality is easier and faster in Python than in C++. As a downside, there's runtime and memory overhead of the extra layer, as well as the need to maintain two separate bits of code for one thing.

My primary question is: are there any obvious upsides or downsides to some of the approaches that I failed to consider, especially ones that make one choice clearly superior to the other? Also, is there a widely accepted "best practice" for this type of thing? Or a "checklist" to help choose the correct approach for a particular situation?

答:
You pretty much answered the question yourself, and in a way that confirms my own experience: A python wrapper is better in all accounts except for speed and memory efficiency. You are not telling us what sort of application is that so you'll need to carefully weigh how important those factors are for you. In my experience, most of the time they matter very little.

As for having to maintain two bits of code for one thing, you get that with both approaches anyway, and in the case of a C++ written API, it's much harder to maintain.


问:
Which is better design: Determining if a function should execute from outside of it, or inside of it?
I'm often presented with these two options.  From a design point of view, which of these is subjectively better, and why?

Option A

```<code>class foo {
  private boolean bar() {
    //Stuff
  }

  private void A() {
     if(bar())
       B();
  }

  private void B() {
      // more stuff
  }
}
</code>```

Option B

```<code>class foo {
  private boolean bar() {
    //Stuff
  }

  private void A() {
      B();
  }

  private void B() {
     if(!bar())
        return;

     // more stuff
  }
}
</code>```

Let's assume that code duplication isn't an issue here, and the division is only to separate, logically, the code for better readability/understanding (A, B, and bar have meaningful names in context).    Is it more readable to have the reader know that B might not do anything when looking at just A?  Or is the encapsulation of state better put into B?  Obviously, this can be refactored to separate B and bar() into a separate class entirely, which separates concerns even more, but assuming mostly shared state, which is better?

答:
It depends on whether foo() and B() lie on the same level of abstraction.

If they do, then Option A is preferable.

Otherwise (foo() is an implementation detail of B() and therefore lower than B()) Option B is better.

So, the guiding principle here is: don't mix levels of abstraction.


答:
Another option is to return a result object instead of basic types. For example:

```<code>OperationResult deleteResult = myOrm.deleteById(id);

if (deleteResult.isSuccess()) {
    // ....
}
</code>```

With this, if for some reason you need to return the numbers of rows affected, you simply can add a method in OperationResult:

```<code>if (deleteResult.isSuccess()) {
    System.out.println("rows deleted: " + deleteResult.rowsAffected() );
}
</code>```

This design allows your system to grow and include new functionality (knowing about affected rows) without modifying existing code.

Body:
Another option is to return a result object instead of basic types. For example:

OperationResult deleteResult = myOrm.deleteById(id);

if (deleteResult.isSuccess()) {
    // ....
}

With this, if for some reason you need to return the numbers of rows affected, you simply can add a method in OperationResult:

if (deleteResult.isSuccess()) {
    System.out.println("rows deleted: " + deleteResult.rowsAffected() );
}

This design allows your system to grow and include new functionality (knowing about affected rows) without modifying existing code.


问:
What is a good strategy to develop apps that run in console and as API?
I develop quite often scripts that are primary used as a console application but later they are used in other scripts, webservices and other things where it is very convenient, to just import the script and call methods directly rather than using a Process and parsing stdout.

The question is now, what is a good design strategy to develop such scripts? I have an example here in python:

```<code>def main():
   # do stuff
   print("Found several results here:")
   for foo in bar:
       print("result: %s" % foo)

if __name__ == "__main__"():
    main()
</code>```

this one could be easily rewritten to something like this:

```<code>def main():
   result = some_function()
   print("Found several results here:")
   for foo in result:
       print("result: %s" % foo)

def some_function():
   # do stuff
   return bar

if __name__ == "__main__"():
    main()
</code>```

But in most of my cases this is not so easy. But how to deal with this problem when the program does quite a lot of output, like different analyses or several steps or the return value of an API function would be something like a list of dict of list of tuples (or even more complicated structures)? Would it be good to encapsulate every step into a single method and run them one after another? What design patterns can be used to write good progamms that can be easily reused as APIs?

答:
If you are working on a script where you know (or suspect) that it will later be used as a library in a larger project, then it is easier to start out with writing the functionality in a library and then tagging on a driver script to use the library from the command line.

There are no specific design patterns to help in creating good API's. It requires mostly a shift in thinking from "a script that produces human readable output" to "a library whose output can be easily used by other software components". The biggest change there is that a library usually does not interact directly with the user.