问:
DAO design for writing big XML file on database
I am currently working on JavaEE application (Spring, Hibernate). I have to put a big XML file (more than 1 gigabyte) on a relational database (Postgres).

The application does not use batch processing. I've done some searching but I did not find any solution for the design of the DAO layer: if I use only one transaction, the server will not response to any request until it finishes the insertion of rows to fill a complex database schema (a huge number of rows: the order of added rows is thousands (for every table)). So, using 1 transaction is not a good idea.
I can split XML file basing on its tags data: every tag content will be inserted on a row.
The idea is to use multithreading to manage transactions (every transaction inserts a defined number of rows). Is it a good idea?
I have found a difficulties to would know how to define the necessary number of transactions to maintain a good time response of the application. I also search    how to manage failure of certain transactions. For example, If only 3 transactions write over 1000000 fail, I should try again all the transactions?

When searching, I find that batch processing like Spring batch manages database records and transactions failure. But in my application, we did not use batch processing.

Unfortunately, I can not change the database to Nsql database or add Spring Batch framework to the project.

N.B: I can not bypass Spring and Hibernate in this project, but I am open to any suggestion even for curiosity.

答:
Your problem and technology stacks are very similar to a project that I am working on as an application architect at this time so I am going to give you my best advice on how to proceed given the information and constraints you have provided.

Your instincts are correct that the best choice for this project would be to use Spring Batch or something similar to it.  What you are effectively doing is exactly what batch processing is and your attempts to introduce multi threading and working to avoid running out of memory in processing are easily handled in Spring Batch.  It sounds like from my perspective that your client had a poorly designed application for the intended features, and you were asked to clean up the mess but not at the expense of a rewrite.

So I am not saying that you must use Spring Batch but I want to give you some context as to why Spring Batch is the best choice.  This will help you design your approach properly.

Readers, Processors and Writers

The idea behind Readers is to read in a subset of the data to be processed.  This can typically be done however you are reading the XML file now.  Your reader keeps track of where it is on the file position.  It is creating objects for the processor.

The processor will perform any business or integration logic that you might have.

The writer can use a tool like Hibernate to write out individual records to the relational database.

Chunking and Transactions

A chunk of data is just a subset of data objects that you read in, process and write out in a single contiguous transaction.  If the transaction completes through all the way then it is clearly okay to commit to the database.  In the event of an exception you will want to define exception behaviour to where you rollback the transaction at the database level and properly log which chunk of records failed to be completed successfully.  Perhaps as part of this rollback behaviour you want to include some notification event behaviour to email a support group to look at the problem.  Utilizing a transaction framework through Spring + JTA is the best approach.

Realistically though you can't have a discussion about what to do when there is an exception without looking at your business requirements (or as I suspect, perhaps the lack of business requirements from your client here).  Defining what happens when some records don't process isn't something we can tell you, it is something that has to be addressed in your business requirements or else it is a gap.

Regardless how you approach what to do in your rollback behaviour, 1GB of data for a single file is too much for a single transaction and it would be wasteful to throw away all of the processing that went into that file because of what might amount to an unexpected character in some arbitrary record.

You want to chunk your input data to a reasonable size such that there is adequate memory for all currently processing files at the same time.

You want your chunk to be individually transacted such that once it completes you will not have to revisit these records again

You want to process these files one chunk at a time at first, and only after you are not reaching desired performance metrics should you consider a multi threaded or distributed approach.

You want to log what chunk you are currently processing in the database in some kind of meta data table, and if a chunk fails then in your exception rollback behaviour you want to update within the database which chunk failed on the job.

If a chunk fails on a file, you should stop processing altogether until the problem is identified and fixed.  This might be human involvement so you probably need to consider a support functionality to restart a failed job where it left off.

Performance and Scaling

This is hard for me to help with since I don't know where the file comes from, how the file processing job is invoked, and what non-functional requirements you have around performance.  My advice here of course is that the safe bet is to process as individual transactions in a single threaded way to start off.  Multi threading or even introducing parallel processing and distributed computing here could be potentially very complicated if you are trying to roll your own.  Frameworks like Spring Batch help you manage this if you need it but there is a good chance that you won't if the client did not offer any strict performance requirements.  Your concerns about deadlocking the database and staying within the memory constraints on your server are alleviated by handling this in a single threaded way.


问:
How to correctly implement key=value storage REST API
I'm new to REST API, so I decided to get familiar with it by designing a small web service API. I have its design written down and would like you to review it. I feel like I have made some mistakes in designing it and understanding REST concepts, which I try to address with questions on my design at the very end. I'm mostly unsure about my use of URLs for the API.

The web service I try to design provides a way for a user to store key=value pairs, i.e. retrieve value knowing its key, update the value of an existing key and delete an existing key. The whole CRUD set of actions!

Here is what I have came up with.

Versioning

Since my API might evolve, I want a notion of API versions, so I have

```<code>GET api.example.com/supported-versions
</code>```

Which returns a JSON list of integers denoting the API versions supported.

The API will be available at api.example.com/{VERSION}/ endpoint, e.g. api.example.com/1/ for the first version.

Key=value storage

```<code>GET api.exmaple.com/1/keys/{KEY}
</code>```

Allows a user to get a value associated with a key KEY. The server responds with a status code (200, 404, etc.) and a text value on success, both encoded in JSON.

```<code>POST and PUT api.exmaple.com/1/keys/{KEY}?value={VALUE}
</code>```

Allows a user to create/update the value associated with the key KEY. The server returns a status code (200, 404, etc.). VALUE is a text string.

```<code>DELETE api.exmaple.com/1/keys/{KEY}
</code>```

Allows a user to delete a key=value pair with key KEY. The server Which will delete the key=value pair and return a status code (200, 404, etc.).

Let's imagine that there is an OAuth2 authentication set up, which is used for the up above POST/PUT/DELETE methods (but not GET anyone can GET, you don't need authentication for that), so there is a way to uniquely identify a user and keep track which key=value pairs belong to which user.

Now, I want an authenticated user to be able to get a list of all keys they have, so that they don't have to store this information on client-side in order to send DELETE or PUT request later.

```<code>GET api.exmaple.com/1/keys?page={PAGE}
</code>```

This allows a user to get a list of all existing keys they have created. PAGE is an optional parameter. User can increase page count until the user gets an error status code. It returns a paged list of all existing keys a user have created and also a status code (200, 404, etc.).

Is there anything wrong with this API?

Should the first GET/POST/PUT/DELETE from Key=value storage section be on /keys/ or /values/?

Is it fine that the last "GET", the one which returns all keys a user has, is also /keys/?

答:
Your solution is OK, save for a few things:

Versioning REST APIs at the resource level is generally not a good practice, although this not necessarily a consensus view. See here for some comments and links to further discussion.

To your second question, you could probably argue either way as to whether /keys/{id}/ or /values/{id}/ is correct, but my instinct would be to either use keys since it's the "parent" of the tuple, or I'd use an entirely different name such as /data/ or /elements/, or whatever it is these things actually represent.

I would probably have different GET methods for the whole key-value dictionary, e.g. /keys/{id}/, and a separate resource for those for a particular user, e.g. /users/{id}/keys/{keyId}.

Finally, it's not great (although possible) to use query strings with POST/PUT/DELETE methods. It is a query string, after all. Instead, you could use POST /data/, and put the key-value pair as JSON data in the request body.
See this question as well.

ContentLicense: CC BY-SA 3.0


问:
Does making a method static save memory on a class you'll have many instances of?
In response to Aaronaught's response to the question at:

Can't I just use all static methods?

Isn't less memory used for a static method? I am under the impression that each object instance carries around its own executable version of a non-static member function.

Regardless of how much overhead is involved in calling a static method, regardless of poor OO design, and possible headaches down the road, doesn't it use less memory at runtime?

Here is an example:

I make a vector of zero-initialized objects. Each object contains one piece of data (a triangle consisting of nine double's). Each object is populated in sequence from data read from a .stl file. Only one static method is needed. Proper OO design dictates that a method dealing with the the data directly should be distributed to each object. Here is the standard OO solution:

```<code>foreach(obj in vec) {
  obj.readFromFile(fileName);
}
</code>```

Each obj carries readFromFile's compiled code alongside the data!

Memory is more of a concern than performance in this case, and there is a LOT of data on a constrained system.

Solutions:

Namespace method (great for C++ but not possible in Java)

One static method in obj's class. Executable code is kept in one place at runtime. There is a small overhead to call the method.

A parent class from which obj is derived, which contains the private method readFromFile. Call with super.callPrivateMethod() which calls readFromFile. Messy, and still some memory overhead in each object.

Implement readFromFile outside obj's scope, so in vec's class or in the calling class. This, in my opinion, breaks data encapsulation.

I realize for large amounts of data one explicit object for each triangle is not the best approach. This is only an example.

答:
Methods are not stored on a per-instance basis, even with virtual methods. They're stored in a single memory location, and they only "know" which object they belong to because the this pointer is passed when you call them.

The only extra memory required in C++ is if you're using virtual methods, which require a single extra pointer per instance to point at the virtual method table (in Java of course you always have a base class with virtual methods, Object, so it's unavoidable).

If it worked the way you described, OO languages would not be very popular! Feel free to add as many methods as you need to your objects. It won't affect their memory usage.


问:
Having a TestContext to test methods instead of Dependency Injection
I am working on a legacy code base and I need to find a way to write unit tests for this project.

The project has a three layer architecture (UI-Biz-DAL as we call them) and DAL is totally implemented using ADO.Net and Typed-Datasets and it is full of SQL Scripts.

Our Biz classes have methods that are responsible for doing business logic stuffs and they are dependent on other helper classes and DAL classes.

I know I can use DI to inject these classes to my Biz classes but I think that I should change a lot of code. Here's a solution that I can think of :

There's this TestContext class that acts as a container and can contain mock objects for tests but it does not have anything when it comes to run the actual code so that real objects can be used instead , here is an example :

```<code>var dal=TestContext.Current.Resolve<IMyDAL>(@default:new MyDal());
</code>```

as you can see Resolve method accepts an argument of type IMyDAL that will be used in case of not running our tests.

First I would like to know what do you think about this solution

Second I am still thinking about a way to test SQL scripts that are hardcoded in our code base . How can I test them ?

答:
What you're suggesting is homebrewed DI tightly coupled to each business object and auto-wiring the current behavior.

Just. no. Use a real DI solution, and auto-wire THAT to the legacy DAL.

Change the code. You're writing tests here, man, that's the Lord's work.


答:
What Spring Batch will do for you is decouple the reading, writing and business logic tasks of your problem while preserving causality. It doesn't introduce any form of database transaction batching, you'd still be on the hook for that (though the Reader abstraction would make that a tad simpler). So if all you want to do is perform your inserts in efficient batches of SQL, Spring Batch provides nothing, really, but an opinionated framework that you don't know.

When you get right down to it, if you're traversing an XML document and building Hibernate objects from it, what really matters to the database is:

causal consistency, which uncontrolled multi-threading of small transactions might violate

minimum application time spent inside a database transaction (which means not opening a transaction until all data is available and not performing business logic while the transaction is active)

the ability to control the size of a commit (so as not to commit 1 record at a time OR commit everything all at once)

For maximum efficiency WITH consistency and WITHOUT a framework, what you need is a minimum of two threads:

One that's reading and creating objects, as fast as it can

One that's writing objects to a database, as many as are available and in the order they were read

A simple way to provide this behavior with Hibernate involved:

Have one Reader-Creator thread that reads the XML file and creates Hibernate objects, but does not persist them. Instead, it stores them on a bounded BlockingQueue (BQ).

Have a second Persister-Writer thread that shares the BQ with the first thread. In a loop, it polls for a new object to be added to the BQ. When one is discovered, it is persisted using a new EntityManager (EM), along with N-1 additional elements that are currently available (using drainTo, which doesn't block). Then, flush is called on the EM, and we continue the next loop (throwing out the old EM in the process).

In this pattern, the queue that connects the two threads is generally called a "Mailbox" and what we're doing is a cheap approximation of Actor Model programming. But in the end:

you are protected from out of memory issues by properly sizing the BlockingQueue, as the first thread will block if it becomes full and the second thread throws away all of the state it cares about on each loop

causal consistency is guaranteed because all objects will be persisted and flushed to the database in the order they are read

database performance is improved because N objects are flushed at a time rather than 1 or ∞ (you may experiment for an optimal value of N or just use 1000)

No new frameworks are involved -- indeed, this is likely less than 50 LoC

You will need some way to ensure that the second thread stops after the first thread runs out of the work and the last piece of work in the BQ has been committed, otherwise you'll exit the program before all content has been saved. A common technique for this is to create a "poison pill" message -- something that doesn't look anything like the types of message usually received by the Persister-Writer that causes it to flush any remaining objects and break out of its loop.

Title: DAO design for writing big XML file on database

Body:
I am currently working on JavaEE application (Spring, Hibernate). I have to put a big XML file (more than 1 gigabyte) on a relational database (Postgres).

The application does not use batch processing. I've done some searching but I did not find any solution for the design of the DAO layer: if I use only one transaction, the server will not response to any request until it finishes the insertion of rows to fill a complex database schema (a huge number of rows: the order of added rows is thousands (for every table)). So, using 1 transaction is not a good idea.
I can split XML file basing on its tags data: every tag content will be inserted on a row.
The idea is to use multithreading to manage transactions (every transaction inserts a defined number of rows). Is it a good idea?
I have found a difficulties to would know how to define the necessary number of transactions to maintain a good time response of the application. I also search    how to manage failure of certain transactions. For example, If only 3 transactions write over 1000000 fail, I should try again all the transactions?

When searching, I find that batch processing like Spring batch manages database records and transactions failure. But in my application, we did not use batch processing.

Unfortunately, I can not change the database to Nsql database or add Spring Batch framework to the project.

N.B: I can not bypass Spring and Hibernate in this project, but I am open to any suggestion even for curiosity.


问:
What is a proper way to implement object cloning with deep copy, using generally accepted OOP principles?
Short version of the question: What is a proper way to implement object cloning with deep copy, using generally accepted OOP principles?

I ran into this while looking into the Prototype Design Pattern in the GoF Design Patterns book, but I think it applies to general object cloning.

Wouldn't it be, each class has to properly implement its own instance method of deep_copy, because each class has its own way to "going through" all elements, such as left and right for a binary tree, and sometimes, an object A having 2 other references to 2 other objects: B and C, may mean A own B and C, and therefore B and C should also be cloned, while in some cases, such as a node object in a graph, A having a reference to B and C just means it is pointing to B and C and DO NOT own B, C (other nodes in the graph may also point to B and C).

There is a way to clone, which is serialize it and unserialize it (which should be same as data marshalling?) but it doesn't handle the case when the object doesn't own another object, or in the case of a node in a graph, can you serialize and unserialize, and get back a cloned node that points to the proper nodes in the graph as the original node object does?

Another complication may arise, if object A has an instance variable foo, and it has a data structure that reference object B twice, so we really should not clone B twice.  Or, if foo reference it once in its data structure, but another instance variable bar also reference B, then also we should not clone B twice but once.  And if A doesn't own B, then we should not clone B at all.

But let's say we ignore the complication above:

Then roughly speak, all classes in your application should implement its own method of deep_copy, and it roughly is this:

```<code># Pseudo code:

class SomeClass
  def deep_copy
    new_object = self.clone()  # to have all the instance variables and
                               # methods cloned, but just a shallow copy, and
                               # also, all the inheritance, access to
                               # class variables, methods, and inheritance
                               # hierarchy should be properly set up

    for all objects that is referenced by my instance variables
      if I own the object (by the design of my class), then
        # rely on polymorphism to make a proper deep_copy of this object
        new_object.this_instance_variable = self.this_instance_variable.deep_copy()
      end
    end

    return new_object
  end
end
</code>```

and depending on whether the primitive types are object or not, it may just say: if I own the object, but it is primitive, then don't clone it.  Or in case the primitive types (like Fixnum, 1, 2, 3) are objects too, as in Ruby, then just let it clone it (because you don't want to do type checking to see whether it is clone-able), but in the self.clone line, it will raise an exception to say that this type is not for cloning, and in that case, just catch the exception and return the same object without cloning it (which is the base case of the recursion).

But the key point is, using generally accepted OOP principles, every class in your app has to have the deep_copy implemented, and its contract (the interface contract) is that it will indeed return a clone of "myself" together with deep copies of objects that I own (recursively).  And it may be difficult because a lot of times, we define a class, and we don't really implement a deep_copy.  If our app has 12 classes, and we need a clone with deep copy, then we actually have to implement such clone with deep copy for all 12 classes (or for all classes that may need to participate in the deep copy).  Is the above correct, or are there some corrections according to OOP principles?

答:
You have got the gist of it.

It should be noted that, depending on the language you use, there might be existing conventions for which function performs a deep copy of an object.

In many languages, your deep_copy function would be called clone. In C++, this is also the territory of the copy-constructor.


问:
Managing widgets in a simple GUI framework
So, I'm still working on my small GUI "library". (C++ with boost)
The goal is to provide a simple solution for small SPI displays, using a Raspberry or some embedded board.

Thus, I ended up having a Widget class from which concrete widgets are derived.
Examples would be a textbox or an image frame.
Widgets can contain other widgets to allow groups.
This is achieved via aggregation using a STL container.
In the end, the complete GUI dataset can be seen as some kind of tree, with some nodes having child nodes.

Each widget shall have a draw-method which looks like

```<code>    void drawMe(brush *myBrush);
</code>```

A brush provides primitive drawing methods like "setPixel" or "drawRectangle"
This allows the widget to draw itself, using the methods provided by the brush.
A widget first draws itself, then it invokes the drawMe-method of each child widget, if any.

Now lets talk about my problem.
Each widget has its coordinates, relative to the parent.
This is not enough, the brush needs the absolute coordinate of the widget, because a brush does not know anything about children or parents. A brush just provides methods like setPixel(int x, int y) and draws into a pixel buffer.

But I somehow have a bad feeling letting a widgets know its absolute coordinates. This feels wrong.
On the other hand, at the moment a child does not know it is owned by a parent, thus, there's no way a widget can compute its absolute coordinates.

I can implement workaround in the brush, so that the brush supports multiple layers instead of just one canvas (the pixel buffer). A parent then has to set up the layers for its child.
Sounds like a dirty hack.

Also, is it a good idea to et the parent know its child coordinates?

You see, I'm somewhere lost with this issues.
There are GUI frameworks out there, like QT and stuff.
So, my problem should be known and solved already.
Do you have any suggestions?
How would you implement the drawing process?

The whole project is about learning new stuff, mainly OOP paradigms.
I like to hear your opinion on that.

答:
I think that there is a key scenariio that you need to keep in mind.

The widgets should be able to redraw themselves

If you don't have this ability, then when a widget changes in some way, you only can call the root widget to do a full redraw (that, of course, can be optimized in some way, let's say a dirty flag).

So

On the other hand, at the moment a child does not know it is owned by a parent, thus, there's no way a widget can compute its absolute coordinates.

will be probably wrong . a child needs to know that it's owned by a parent, and also know the parent, at least to be able to trigger the parent update.

That would be something in the lines of the child asking the parent for a redraw, and then the parent doing the redraw, in a recursive way.

Also

because a brush does not know anything about children or parents. A brush just provides methods like setPixel(int x, int y) and draws into a pixel buffer.

is probably wrong. Of course the brush should know that. But probably you will need a higher order equivalent, that will know something about the widgets.

When you delete a widget, for instance, probably there was another widget that was hidden but the now deleted widget. And this widget needs to be drawn. Again, if there isn't a widget manager that knows something about this, your only option will be a massive call for all the widgets to be drawn.

This widget manager doesn't need to be your brush, but then it's another component that needs to know about the layout


问:
How and when did it happen that, a type is an interface?
I came from a highly functional and procedural background in programming, and never knew that a type is the same as an interface.

As in the Design Patterns book by GoF, it says:

A type is a name used to denote a particular interface. We speak of an object as having the type "Window" if it accepts all requests for the operations defined in the interface named "Window." An object may have many types, and widely different objects can share a type. (p. 13)

The surprising thing is, I thought of type as char (like a character, or 1 byte), or int (a word, or 4 bytes or 8 bytes), or a pointer to character (a string in C language) before.  Maybe even a struct with x and y as coordinate of a point, or an array, as a type, but I never thought of a type being "an interface".

So it looks like a Car object can be of the type Moveable and Soundable, and a Dog object can be of the type Moveable and Soundable, while a Circle object may be of the type Moveable only, until we decide that a Shape object also need to give out sound, when a user clicks on it, and we let the Shape class implement the Soundable interface and now a Circle object is also of the type Soundable?

I wonder when and how it happened?  Is it actually said to be so by the GoF book for the first time in 1994 when the book got published?  Or is it actually existing idea that came from long time ago?

It actually sound exactly the same as Duck Typing, but Duck Typing seems like a new concept that began about in 2003 in the Python and Ruby community, not like an idea that was in 1994 or earlier.

答:
The idea of having a type defined by it's interface (in a general meaning, including syntax and semantics) is well known under the name Abstract Data Type. The oldest reference to this term I could find is here, ("Programming with Abstract Data Types", B. Liskov and S. Zilles), it refers back to 1974. According to this paper, which contains a full historic overview, that was exactly the time when the idea of type abstraction was invented.

The ADT concept has not much to do with "duck typing" or "dynamic typing". You can implement ADTs easily in a statically typed language like, for example, C++ by utilizing classes.


答:
Depending on your version of studio you can use Microsoft Fakes, in particular shims to be able to mock most things - https://msdn.microsoft.com/en-us/library/hh549176.aspx .

Once your library is covered with tests you can have these as a safety net while you introduce proper DI and fully isolated tests.

The danger is that shims are evil and can let developers get away with bad practice.

Title: Having a TestContext to test methods instead of Dependency Injection

Body:
I am working on a legacy code base and I need to find a way to write unit tests for this project.

The project has a three layer architecture (UI-Biz-DAL as we call them) and DAL is totally implemented using ADO.Net and Typed-Datasets and it is full of SQL Scripts.

Our Biz classes have methods that are responsible for doing business logic stuffs and they are dependent on other helper classes and DAL classes.

I know I can use DI to inject these classes to my Biz classes but I think that I should change a lot of code. Here's a solution that I can think of :

There's this TestContext class that acts as a container and can contain mock objects for tests but it does not have anything when it comes to run the actual code so that real objects can be used instead , here is an example :

```<code>var dal=TestContext.Current.Resolve<IMyDAL>(@default:new MyDal());
</code>```

as you can see Resolve method accepts an argument of type IMyDAL that will be used in case of not running our tests.

First I would like to know what do you think about this solution

Second I am still thinking about a way to test SQL scripts that are hardcoded in our code base . How can I test them ?


问:
How to simulate method overloading in langs without such feature?
Langs like Java knows about true method overloading:

```<code>class Overload {
  void demo (int a) {
   System.out.println ("a: " + a);
  }
  void demo (int a, int b) {
   System.out.println ("a and b: " + a + "," + b);
  }
}
</code>```

Whether langs like PHP — doesn't. So when you stand in front of a problem of processing something depending of context, which ways will you think of and which usually choose?

Typical problem could be some abstract payment system, where outside user should not think about concrete payment provider details (which could be different) and just call Provider.pay(PayDetails). In interface of PaymentProvider we must fix that it works with PayDetails. But in concrete implementations concrete provider could work for example only with it's own ConcretePayDetails.

Is it worth to polyfill such things as ad-hoc polymorphism in langs that dont' support it?

How to better solve context-related methods dispatching in such langs?

Example of possible implementation available at https://github.com/garex/php-ad-hoc-polymorphism-polyfill/

答:
I think you have conception issues here. You are saying you want to have a generic "pay" method. The problem is that this method will always be specific to given payment provider. For example PayPal provider cannot take CreditCardDetails as an argument to "pay". And vice versa.

I would suggest a "factory" approach. May not be the best one in this case but it solves the overloading "issue". This is only a raw draft that needs more work but it show one of possible approaches.

```<code>interface PaymentProvider
{
    public function pay();
}

class PayPalProvider implements PaymentProvider
{
    private $details;

    public function __construct(PayPalPayDetails $details)
    {
        $this->details = $details;
    }

    public function pay()
    {
        //Do something with $this->details
    }
}

class ProviderFactoriesCollection
{

}

interface ProviderFactory
{
    /**
     * @returns PaymentProvider
     */
    public function create(array $data);
}

class PayPalProviderFactory implements ProviderFactory
{
    public function create(array $data)
    {
        // Do the magic here
    }
}

class PaymentProviderFactory
{
    private $providerFactoriesCollection;

    public function __construct(ProviderFactoriesCollection $providerFactoriesCollection)
    {
        $this->providerFactoriesCollection= $providerFactoriesCollection;
    }

    /**
     * @returns PaymentProvider
     */
    public createFromInput($type, array $details)
    {
        return $this->getFactoryForType($type)->create($details);
    }

    private function getFactoryForType($type)
    {
        // Find in collection or throw exception if not supported.
    }
}
</code>```

Please also keep in mind that there are languages that are completely unaware of method overloading. C, Objective-C, Rust, Ruby, Go and so on. And it is not necessarily a bad thing.

Body:
I think you have conception issues here. You are saying you want to have a generic "pay" method. The problem is that this method will always be specific to given payment provider. For example PayPal provider cannot take CreditCardDetails as an argument to "pay". And vice versa.

I would suggest a "factory" approach. May not be the best one in this case but it solves the overloading "issue". This is only a raw draft that needs more work but it show one of possible approaches.

interface PaymentProvider
{
    public function pay();
}

class PayPalProvider implements PaymentProvider
{
    private $details;

    public function __construct(PayPalPayDetails $details)
    {
        $this->details = $details;
    }

    public function pay()
    {
        //Do something with $this->details
    }
}

class ProviderFactoriesCollection
{

}

interface ProviderFactory
{
    /**
     * @returns PaymentProvider
     */
    public function create(array $data);
}

class PayPalProviderFactory implements ProviderFactory
{
    public function create(array $data)
    {
        // Do the magic here
    }
}

class PaymentProviderFactory
{
    private $providerFactoriesCollection;

    public function __construct(ProviderFactoriesCollection $providerFactoriesCollection)
    {
        $this->providerFactoriesCollection= $providerFactoriesCollection;
    }

    /**
     * @returns PaymentProvider
     */
    public createFromInput($type, array $details)
    {
        return $this->getFactoryForType($type)->create($details);
    }

    private function getFactoryForType($type)
    {
        // Find in collection or throw exception if not supported.
    }
}

Please also keep in mind that there are languages that are completely unaware of method overloading. C, Objective-C, Rust, Ruby, Go and so on. And it is not necessarily a bad thing.


问:
If Class is to define attributes and methods, and Interface is to define (a set of) methods, then how to think of interface needing new attributes?
I thought a class is supposed to define, or give a blueprint, of attributes and methods for an object.  And then, an interface is to provide a set of methods, as a contract for its clients. (and so a class also gives an interface as well, because a class also define a set of methods).

But say, if we have a class called Shape.  Now we add an interface to it, called Moveable.  Now we may have to add attributes to the class, such as velocity.  But interface is only about methods, then how would we think about adding attributes to the class due to interface?

答:
Attributes like velocity are implementation details and therefore don't belong to interfaces. Interface doesn't dictate you to add attributes, it's the chosen implementation that does that. One implementation may use the velocity attribute and other would use the acceleration.

So you don't add attributes due to interface, you add attributes due to selected implementation. And it is usually the programmer who decides how to implement an interface, not the interface itself.


答:
Regardless of how much overhead is involved in calling a static
  method, regardless of poor OO design, and possible headaches down the
  road, doesn't it use less memory at runtime?

This is rather complicated, but I would say "mostly no".

Title: Does making a method static save memory on a class you'll have many instances of?

Body:
In response to Aaronaught's response to the question at:

Can't I just use all static methods?

Isn't less memory used for a static method? I am under the impression that each object instance carries around its own executable version of a non-static member function.

Regardless of how much overhead is involved in calling a static method, regardless of poor OO design, and possible headaches down the road, doesn't it use less memory at runtime?

Here is an example:

I make a vector of zero-initialized objects. Each object contains one piece of data (a triangle consisting of nine double's). Each object is populated in sequence from data read from a .stl file. Only one static method is needed. Proper OO design dictates that a method dealing with the the data directly should be distributed to each object. Here is the standard OO solution:

```<code>foreach(obj in vec) {
  obj.readFromFile(fileName);
}
</code>```

Each obj carries readFromFile's compiled code alongside the data!

Memory is more of a concern than performance in this case, and there is a LOT of data on a constrained system.

Solutions:

Namespace method (great for C++ but not possible in Java)

One static method in obj's class. Executable code is kept in one place at runtime. There is a small overhead to call the method.

A parent class from which obj is derived, which contains the private method readFromFile. Call with super.callPrivateMethod() which calls readFromFile. Messy, and still some memory overhead in each object.

Implement readFromFile outside obj's scope, so in vec's class or in the calling class. This, in my opinion, breaks data encapsulation.

I realize for large amounts of data one explicit object for each triangle is not the best approach. This is only an example.


问:
Should I use a single repo when multiple parts of the same project are running on the same server?
I'm planning a project that consists of the following parts:

REST API in Lumen

Web client in Laravel

Product website in Jekyll

These separate products are going to be running on the same server.

I'd like to have a development environment that mimics the production environment as closely as possible, that's why I'm going with Vagrant in combination with a single Ansible playbook that provisions both environments.

What's the best way to organise such a project? Should I keep everything in one repository, or should I split up the parts into multiple repositories?

What about the development environment and provisioning scripts? Should I put these in a separate repository? How would I go about referencing the separate parts of the project in the development environment?

I'd like to be able to clone a single repository and spin up a development environment with as little work as possible.

Any recommendations?

Edit: My question differs from this question because I'm especially struggling with things related to production and development environments.

答:
I'd suggest using same repo even if parts are running on different servers. And I wouldn't split setup scripts (e.g. ansible playbook) into different repository too, because it is the same project any way.

It is just a matter of convenience: you only need one checkout to get started as a developer, tester or support engineer. And I wouldn't bother about provisioning unneeded parts of a system to a server, because ansible could take care of that.


问:
In C++, is it a reflection of poor software design if objects are deleted manually?
With the advent of smart pointers, is it a sign of poor design if I see objects are deleted? I'm seeing some software components in our product that people are still doing this. This practice strikes me as un-idiomatic, but I need to be sure this is the industry consensus. I'm not starting a crusade but it'd be nice to be prepared theory wise.

Edit: legit uses of delete, Klaim mentioned the object pool use case. I agree.

Bad examples of using delete, I am seeing many new's in constructor or start() and corresponding delete's in the destructor or stop(), why not use scoped_ptr? It makes the code cleaner.

答:
RAII and Exceptions

With the advent of smart pointers, is it a sign of poor design if I
  see objects are deleted?

It depends on the context. Let's have a look at an example like this:

```<code>Foo* foo = new Foo;
Bar* bar = new Bar;
func(foo, bar);
delete foo;
delete bar;
</code>```

* Note: not utilizing the stack in this example is silly. Let's imagine in the real world that Foo and Bar are pimpls, or are ultimately abstract types to be stored/owned in a polymorphic data structure, are created through a factory, are destroyed only when the user requests to unload the resource, etc.

Is there anything wrong with this code? Some people might think, "no". Except there's all kinds of problems with this code, and not relating at all to practices or style -- I mean this code is buggy. The problem is this:

```<code>Foo* foo = new Foo; // this can throw
Bar* bar = new Bar; // this can throw
func(foo, bar);     // this might throw
delete foo;
delete bar;
</code>```

It's the interaction of manual release of resources with exception-handling that makes destructors that automate this often an absolute necessity (see RAII). To make this above code no longer buggy for exceptional paths without reaching for destructors to automate it, we might need to do something like this:

```<code>Foo* foo = new Foo;     // this can throw
try
{
    Bar* bar = new Bar; // this can throw
    try
    {
        func(foo, bar); // this might throw
    }
    catch (...)
    {
        delete bar;
        throw;
    }
    delete bar;
}
catch (...)
{
    delete foo;
    throw;
}
delete foo;
</code>```

... phew! That should, provided I didn't make any mistakes (which is still very possible), fix the bugs. Another way is like this:

```<code>Foo* foo = new Foo;     // this can throw
Bar* bar = 0;
try
{
    bar = new Bar;      // this can throw
    func(foo, bar);     // this might throw
}
catch (...)
{
    delete foo;
    delete bar;
    throw;
}
delete foo;
delete bar;
</code>```

... which might be a little bit easier, but still pretty complicated (and imagine how easy it is to mess this all up if we came back and made changes to it later, or if one of our teammates did).

Now let's consider how we can fix the bugs using smart pointers:

```<code>unique_ptr<Foo> foo(new Foo);
unique_ptr<Bar> bar(new Bar);
func(foo.get(), bar.get());
</code>```

Tada! I don't know about you, but that seems a whole lot easier to read, write, manage. Through destructors which automatically release resources, we end up making our lives so, so, so much easier.

Exception-Safety

Some people might think the above example is kind of moot, since it's fixing bugs in exceptional paths of execution. For toy programs at least, worrying about correct behavior in the face of an exception might be overkill.

Yet for production code, I work in an area which is not mission-critical that much, but people still tie their professional livelihood (not lives) to the software. It pays, in these cases, to give some attention to exception-safety, to allow the software to gracefully recover in the face of exception (and also write tests that simulate exception paths by throwing while checking against leak detectors and such). Destructors which automatically release resources and/or roll back side effects are a lifesaver here when interacting with exceptions.

Smart Pointers

That said, do we have to use smart pointers everywhere, per se? Here I'll offer a biased answer that might be unusual as one working on very low-level, performance-critical areas.

There's no cost to something like std::unique_ptr provided we don't need a custom deleter. But often in these areas I work the most in, we use custom allocators, requesting chunks from a memory pool and returning them when we're done. Often my code doesn't look like this:

```<code>Foo* foo = new Foo;
...
delete foo;
</code>```

Instead it looks like this using placement new:

```<code>Foo* foo = new(allocator.allocate()) Foo;
...
foo->~Foo();
allocator.deallocate(foo);
</code>```

With these kinds of cases, it can get a bit awkward to use smart pointers. The standard ones tend to have an overhead when custom deleters are involved which defeats some of the purpose of using fixed allocators and such to improve locality of reference (fewer smart pointers fitting into a cache line). Plus the code just gets a bit more awkward.

It's impossible in these cases to even roll my own smart pointers without paying a runtime cost for it, since they'd need to be injected with the custom allocator I'm using across individual objects, and that would at least equate to doubling the size of the smart pointer (I work in areas like graph data structures where shaving 8 bytes per instance can make the difference between 1.2 gigabytes of memory and a gigabyte, e.g.).

That said, this is no excuse to avoid RAII. So what I do is very carefully still free memory in my destructors (for whole data structures, not for each element inside), to wrap this kind of code into objects that manage memory through their constructor and destructor, and still benefit from RAII that way. The resource management is still automated to those using my classes, I just have to be careful and implement my destructors with manual memory management involved and write tests to make sure I don't forget to free memory in a destructor.

This is an exceptional case: for most people, I'd say it's pretty wise to just use smart pointers as much as possible for pointers owning memory if the alternative strategy is using default, throwing operator new and operator delete. It's only when custom allocators are involved that things get a bit murky.

ContentLicense: CC BY-SA 2.5

Body:
RAII and Exceptions

With the advent of smart pointers, is it a sign of poor design if I
  see objects are deleted?

It depends on the context. Let's have a look at an example like this:

Foo* foo = new Foo;
Bar* bar = new Bar;
func(foo, bar);
delete foo;
delete bar;

* Note: not utilizing the stack in this example is silly. Let's imagine in the real world that Foo and Bar are pimpls, or are ultimately abstract types to be stored/owned in a polymorphic data structure, are created through a factory, are destroyed only when the user requests to unload the resource, etc.

Is there anything wrong with this code? Some people might think, "no". Except there's all kinds of problems with this code, and not relating at all to practices or style -- I mean this code is buggy. The problem is this:

Foo* foo = new Foo; // this can throw
Bar* bar = new Bar; // this can throw
func(foo, bar);     // this might throw
delete foo;
delete bar;

It's the interaction of manual release of resources with exception-handling that makes destructors that automate this often an absolute necessity (see RAII). To make this above code no longer buggy for exceptional paths without reaching for destructors to automate it, we might need to do something like this:

Foo* foo = new Foo;     // this can throw
try
{
    Bar* bar = new Bar; // this can throw
    try
    {
        func(foo, bar); // this might throw
    }
    catch (...)
    {
        delete bar;
        throw;
    }
    delete bar;
}
catch (...)
{
    delete foo;
    throw;
}
delete foo;

... phew! That should, provided I didn't make any mistakes (which is still very possible), fix the bugs. Another way is like this:

Foo* foo = new Foo;     // this can throw
Bar* bar = 0;
try
{
    bar = new Bar;      // this can throw
    func(foo, bar);     // this might throw
}
catch (...)
{
    delete foo;
    delete bar;
    throw;
}
delete foo;
delete bar;

... which might be a little bit easier, but still pretty complicated (and imagine how easy it is to mess this all up if we came back and made changes to it later, or if one of our teammates did).

Now let's consider how we can fix the bugs using smart pointers:

unique_ptr<Foo> foo(new Foo);
unique_ptr<Bar> bar(new Bar);
func(foo.get(), bar.get());

Tada! I don't know about you, but that seems a whole lot easier to read, write, manage. Through destructors which automatically release resources, we end up making our lives so, so, so much easier.

Exception-Safety

Some people might think the above example is kind of moot, since it's fixing bugs in exceptional paths of execution. For toy programs at least, worrying about correct behavior in the face of an exception might be overkill.

Yet for production code, I work in an area which is not mission-critical that much, but people still tie their professional livelihood (not lives) to the software. It pays, in these cases, to give some attention to exception-safety, to allow the software to gracefully recover in the face of exception (and also write tests that simulate exception paths by throwing while checking against leak detectors and such). Destructors which automatically release resources and/or roll back side effects are a lifesaver here when interacting with exceptions.

Smart Pointers

That said, do we have to use smart pointers everywhere, per se? Here I'll offer a biased answer that might be unusual as one working on very low-level, performance-critical areas.

There's no cost to something like std::unique_ptr provided we don't need a custom deleter. But often in these areas I work the most in, we use custom allocators, requesting chunks from a memory pool and returning them when we're done. Often my code doesn't look like this:

Foo* foo = new Foo;
...
delete foo;

Instead it looks like this using placement new:

Foo* foo = new(allocator.allocate()) Foo;
...
foo->~Foo();
allocator.deallocate(foo);

With these kinds of cases, it can get a bit awkward to use smart pointers. The standard ones tend to have an overhead when custom deleters are involved which defeats some of the purpose of using fixed allocators and such to improve locality of reference (fewer smart pointers fitting into a cache line). Plus the code just gets a bit more awkward.

It's impossible in these cases to even roll my own smart pointers without paying a runtime cost for it, since they'd need to be injected with the custom allocator I'm using across individual objects, and that would at least equate to doubling the size of the smart pointer (I work in areas like graph data structures where shaving 8 bytes per instance can make the difference between 1.2 gigabytes of memory and a gigabyte, e.g.).

That said, this is no excuse to avoid RAII. So what I do is very carefully still free memory in my destructors (for whole data structures, not for each element inside), to wrap this kind of code into objects that manage memory through their constructor and destructor, and still benefit from RAII that way. The resource management is still automated to those using my classes, I just have to be careful and implement my destructors with manual memory management involved and write tests to make sure I don't forget to free memory in a destructor.

This is an exceptional case: for most people, I'd say it's pretty wise to just use smart pointers as much as possible for pointers owning memory if the alternative strategy is using default, throwing operator new and operator delete. It's only when custom allocators are involved that things get a bit murky.


答:
Exactly as Bart explained. By convention in C++ or Java this is done with either a clone() instance method or a copy constructor. Indeed, it could be good practice, for code that you plan to release, to provide both.

```<code>public class Foo
{
    private Bar m_bar ;
    private Baz m_baz ;

    // copy constructor
    public Foo( Foo that )
    { // other classes might use either pattern
        this.m_bar = that.m_bar.clone() ;
        this.m_baz = new Baz( that.m_baz ) ;
    }

    // simply reuse copy constructor implementation
    public Foo clone()
    {
        return new Foo( this ) ;
    }
}
</code>```

Body:
Short version of the question: What is a proper way to implement object cloning with deep copy, using generally accepted OOP principles?

I ran into this while looking into the Prototype Design Pattern in the GoF Design Patterns book, but I think it applies to general object cloning.

Wouldn't it be, each class has to properly implement its own instance method of deep_copy, because each class has its own way to "going through" all elements, such as left and right for a binary tree, and sometimes, an object A having 2 other references to 2 other objects: B and C, may mean A own B and C, and therefore B and C should also be cloned, while in some cases, such as a node object in a graph, A having a reference to B and C just means it is pointing to B and C and DO NOT own B, C (other nodes in the graph may also point to B and C).

There is a way to clone, which is serialize it and unserialize it (which should be same as data marshalling?) but it doesn't handle the case when the object doesn't own another object, or in the case of a node in a graph, can you serialize and unserialize, and get back a cloned node that points to the proper nodes in the graph as the original node object does?

Another complication may arise, if object A has an instance variable foo, and it has a data structure that reference object B twice, so we really should not clone B twice.  Or, if foo reference it once in its data structure, but another instance variable bar also reference B, then also we should not clone B twice but once.  And if A doesn't own B, then we should not clone B at all.

But let's say we ignore the complication above:

Then roughly speak, all classes in your application should implement its own method of deep_copy, and it roughly is this:

```<code># Pseudo code:

class SomeClass
  def deep_copy
    new_object = self.clone()  # to have all the instance variables and
                               # methods cloned, but just a shallow copy, and
                               # also, all the inheritance, access to
                               # class variables, methods, and inheritance
                               # hierarchy should be properly set up

    for all objects that is referenced by my instance variables
      if I own the object (by the design of my class), then
        # rely on polymorphism to make a proper deep_copy of this object
        new_object.this_instance_variable = self.this_instance_variable.deep_copy()
      end
    end

    return new_object
  end
end
</code>```

and depending on whether the primitive types are object or not, it may just say: if I own the object, but it is primitive, then don't clone it.  Or in case the primitive types (like Fixnum, 1, 2, 3) are objects too, as in Ruby, then just let it clone it (because you don't want to do type checking to see whether it is clone-able), but in the self.clone line, it will raise an exception to say that this type is not for cloning, and in that case, just catch the exception and return the same object without cloning it (which is the base case of the recursion).

But the key point is, using generally accepted OOP principles, every class in your app has to have the deep_copy implemented, and its contract (the interface contract) is that it will indeed return a clone of "myself" together with deep copies of objects that I own (recursively).  And it may be difficult because a lot of times, we define a class, and we don't really implement a deep_copy.  If our app has 12 classes, and we need a clone with deep copy, then we actually have to implement such clone with deep copy for all 12 classes (or for all classes that may need to participate in the deep copy).  Is the above correct, or are there some corrections according to OOP principles?

Body:
Exactly as Bart explained. By convention in C++ or Java this is done with either a clone() instance method or a copy constructor. Indeed, it could be good practice, for code that you plan to release, to provide both.

public class Foo
{
    private Bar m_bar ;
    private Baz m_baz ;

    // copy constructor
    public Foo( Foo that )
    { // other classes might use either pattern
        this.m_bar = that.m_bar.clone() ;
        this.m_baz = new Baz( that.m_baz ) ;
    }

    // simply reuse copy constructor implementation
    public Foo clone()
    {
        return new Foo( this ) ;
    }
}


答:
After some fiddling around, I came up with a solution that seems to work pretty well.

I keep the subprojects in separate repositories:

projectname-infrastructure

projectname-api

projectname-client

projectname-website

The project-infrastructure-repository contains a Vagrantfile, some provisioning files and an Ansible playbook that is run locally in order to set up the development environment:

---
- hosts: 127.0.0.1
  connection: local
  tasks:
  - git: repo=https://name@example.com/projectname-api.git dest=www/api/
  - git: repo=https://name@example.com/projectname-client.git dest=www/client/
  - git: git: repo=https://name@example.com/projectname-website.git dest=www/website/

When the above playbook is run, the separate repositories are cloned into the www-folder that nginx on Vagrant uses to serve files.

Of course, the www-folder is inside of the .gitignore of projectname-infrastructure.

The only thing I need to do is git clone https://name@example.com/projectname-infrastucture.git, followed by ansible-playbook setup.yml -i 127.0.0.1,. Then I can just execute vagrant up and everything is up and running.

Any downsides to this approach?

Title: Should I use a single repo when multiple parts of the same project are running on the same server?

Body:
I'm planning a project that consists of the following parts:

REST API in Lumen

Web client in Laravel

Product website in Jekyll

These separate products are going to be running on the same server.

I'd like to have a development environment that mimics the production environment as closely as possible, that's why I'm going with Vagrant in combination with a single Ansible playbook that provisions both environments.

What's the best way to organise such a project? Should I keep everything in one repository, or should I split up the parts into multiple repositories?

What about the development environment and provisioning scripts? Should I put these in a separate repository? How would I go about referencing the separate parts of the project in the development environment?

I'd like to be able to clone a single repository and spin up a development environment with as little work as possible.

Any recommendations?

Edit: My question differs from this question because I'm especially struggling with things related to production and development environments.


答:
There's already a nice answer but I noticed this part:

I might think of Interface as just multiple inheritance, but just that
  we don't want to have the issues of multiple inheritance [...]

I'm guessing you're working in a language where the notion of "interface" is strictly stateless. I often find those somewhat heavy-handed and think multiple inheritance isn't quite as messy as some make it out to be.

Nevertheless, at a conceptual level, the stateless interface is often the most general, the most ideal from the contractual and maintenance point of view.

Robots With Wheels

For example, imagine I wrote a boatload of code using a robot interface which commanded robots to go to various places (rooms, buildings, countries).

If said interface included attributes like the idea that a robot has wheels, that would couple my requests to the notion that a robot has wheels. My requests would then only work for robots that have wheels (they might still be able to go to new locations using other means of transportation, but they would at least require possessing wheels to do it).

That degeneralizes the concept of a robot to one who at least possesses wheels, and likewise degeneralizes all the requests I've made for robots to go to new places which might span in the tens of thousands of lines of code, e.g..

Now let's say a new robot is introduced which has legs, or a jetpack, or a molecular transporter and really had no need for wheels at all. This spells trouble for all the existing robot classes and all the existing code using them through this interface. Now we're in a pretty awkward position of needing to rewrite a lot of code which we wouldn't be in if the interface/contract for robots was stateless.

Flexible Interfaces Allow Variety

In general, the most ideal interfaces from the standpoint of leaving code that is resilient to change will leave a lot of room for a rich variety of concrete implementations without violating the interface contract. Once state is introduced to an interface, it often very quickly degrades the richness and can significantly narrow the variety of possible solutions.

Should this be absolute forced in a language design? I don't know -- I'm biased as one who prefers languages that don't force this. Nevertheless, I think it's generally far better if we erred towards stateless interfaces. An attribute/state will far more quickly degrade the generality of your abstractions than functions.

But interface is only about methods, then how would we think about
  adding attributes to the class due to interface?

These are implementation concerns. If the goal of a class that implements a Movable interface is to move to new locations, you have all kinds of ways to implement it and also represent the data required to do so. Try to appreciate that flexibility/freedom.

Probably a concern here is that, after the dozenth class or so, you feel like you're duplicating a lot of the same kind of boilerplate to implement an interface. In that case, you can reach for an abstract base class potentially, or use composition (store a member which does a lot of the gruntwork), possibly static methods (maybe most gross, however, if it starts to look like nondescript helper functionality), etc.

But try to keep in mind how flexible the interface is as a result of it being stateless, of not mentioning such concrete details as part of its contract.

Title: If Class is to define attributes and methods, and Interface is to define (a set of) methods, then how to think of interface needing new attributes?

Body:
I thought a class is supposed to define, or give a blueprint, of attributes and methods for an object.  And then, an interface is to provide a set of methods, as a contract for its clients. (and so a class also gives an interface as well, because a class also define a set of methods).

But say, if we have a class called Shape.  Now we add an interface to it, called Moveable.  Now we may have to add attributes to the class, such as velocity.  But interface is only about methods, then how would we think about adding attributes to the class due to interface?


答:
Type theory actually predates the Gang of Four by about a hundred years at the least.  Russell and Frege developed what is called "Type Theory" around the turn of the 20th century.  What was very important in type theory was that every operation between instance of two (or more) types resulted in an easily calculable result type.

For computer programmers, this was very valuable, because the more you know about something, the more it can be optimized.  If the "type" of a variable describes a great deal about how it behaves, the compiler can emit efficient code without needing to know the values of those variables until run time.

The connection between interfaces and types was natural for the era of the GoF.  If the design patterns could only be implemented with high level features, they wouldn't go far.  The current standard process was to boil as much valuable information into types as possible, so the compiler can help you.  So it became very natural to treat interfaces as types.

Title: How and when did it happen that, a type is an interface?

Body:
I came from a highly functional and procedural background in programming, and never knew that a type is the same as an interface.

As in the Design Patterns book by GoF, it says:

A type is a name used to denote a particular interface. We speak of an object as having the type "Window" if it accepts all requests for the operations defined in the interface named "Window." An object may have many types, and widely different objects can share a type. (p. 13)

The surprising thing is, I thought of type as char (like a character, or 1 byte), or int (a word, or 4 bytes or 8 bytes), or a pointer to character (a string in C language) before.  Maybe even a struct with x and y as coordinate of a point, or an array, as a type, but I never thought of a type being "an interface".

So it looks like a Car object can be of the type Moveable and Soundable, and a Dog object can be of the type Moveable and Soundable, while a Circle object may be of the type Moveable only, until we decide that a Shape object also need to give out sound, when a user clicks on it, and we let the Shape class implement the Soundable interface and now a Circle object is also of the type Soundable?

I wonder when and how it happened?  Is it actually said to be so by the GoF book for the first time in 1994 when the book got published?  Or is it actually existing idea that came from long time ago?

It actually sound exactly the same as Duck Typing, but Duck Typing seems like a new concept that began about in 2003 in the Python and Ruby community, not like an idea that was in 1994 or earlier.


答:
You should think of an interface as a requirement made by somebody else.

You should think of an implementation that meets the requirement as being a decision you make for your own class.

In most OO languages, class state is an implementation concern.

As such, a well implemented getVelocity is something that you need to provide behavior that meets somebody else's requirement. You may implement that requirement in any technique that works for you. If you want to store velocity as an integer, or compute it on the fly from a queue of recently sampled position and time events, that is your prerogative.

Sure, not having to constantly re-implement simple methods is convenient. There is a solution to this in many OOP languages -- the Mixin. Depending on the language, Mixins can have vastly different constraints, especially w/r/t state.

Java default impls, for example, have no state of their own but they can access the other methods of the interface. So you might be required to implement getVelocity, but all othe calculations involved with velocity could offer a default implementation.

Meanwhile, traits in Scala are so full featured it's common to see programs whose implementations are written almost exclusively in traits rather than classes (Concrete realizations are then provided by Objects and Case Classes)

In languages without mixins, delegation is a common solution, where the class that adopts an interface will also have a private member class that provides the behavior. The original class will still directly implement the interface's methods, but in the body of those methods will simply delegate the call to the private member.


问:
When is optimization not premature and therefore not evil?
"Premature optimization is root of all evil" is something almost all of us have heard/read. What I am curious what kind of optimization not premature, i.e. at every stage of software development (high level design, detailed design, high level implementation, detailed implementation etc) what is extent of optimization we can consider without it crossing over to dark side.</p>

答:
When is optimization not premature and therefore not evil?

It is difficult to say what is good and evil. Who has that right? If we look at nature, it seems we are programmed for survival with a broad definition of "survival" which includes passing on our genes to offspring.

So I would say, at least according to our basic functions and programming, that optimization is not evil when it is aligning with the goal of reproduction. For the guys, there are the blondes, brunettes, red-heads, many lovely. For girls, there are guys, and some of them appear to be okay.

Perhaps we should be optimizing towards that purpose, and there it helps to use a profiler. The profiler will let you prioritize your optimizations and time more effectively on top of giving you detailed information about hotspots and why they occur. This will give you more free time spent towards reproduction and its pursuit.


问:
What is benefit that a compiler is implemented in the same language it compiles?
I've seen it's very common for a compiler to be made in the language it's compiling.  What is the benefit of this?  Seems like it makes the process for outsiders (and the developers for a while) more difficult.

Take for instance:

coffee-script

gcc

typescript

jdk

elixir

etc.

It seems like it just makes things more tricky for the developers (at least in early stages) and for users of the compiler...

答:
It's true in one sense that bootstrapping a language with itself can lead to complications. On the other hand, the people writing a compiler for say C++ are likely to be the world's foremost experts at C++. To a degree, then, it's only natural that they'd choose that language in which to write their code.

It also gives the authors complete autonomy — they don't have to worry about Language X becoming unsupported, because when they're writing compilers for Language Y Version 2 in Language Y Version 1, they know that the compiler for Language Y Version 1 isn't going to disappear or cease to be supported: they are literally the people responsible for keeping it around and supporting it.

Besides, there aren't much better tests of whether a compiler is working properly than to pass its own source code through it. :)


答: It is an example of "Eating your own dog food". By writing the compiler in the target language, you verify if the language works and is sufficiently powerful for a complex task.</p>

Title: What is benefit that a compiler is implemented in the same language it compiles?

Body:
I've seen it's very common for a compiler to be made in the language it's compiling.  What is the benefit of this?  Seems like it makes the process for outsiders (and the developers for a while) more difficult.

Take for instance:

coffee-script

gcc

typescript

jdk

elixir

etc.

It seems like it just makes things more tricky for the developers (at least in early stages) and for users of the compiler...


问:
Tracking column (offset) in presence of tab characters
Imagine that you first write a compiler for your language where you necessarily report errors to the user. Compiler also collects location information for backend tools. They must know where the program elements are located. Later, when you are done with your compiler, you decide to provide IDE support as well. The editor is actually one more back-end tool. Having correct locations for program components helps a lot to syntax highlighting and error reporting. At this moment, you suddenly realize that locations reported by compiler are questionable.

It seems like EOL definitions is more or less specified in the language so that you can report lines correctly -- there is always a agreement between compiler and editor. But what about the column? If compiler reports that there is a blunder for an identifier located at line:col, editor may wonder, highlighting something different, depending the Tab settings. It seems impossible to have exact line:col location, no matter how useful it is, if tab width in the editor-specific. Nevertheless, I see that JavaCC provides getLine altogether with getBeginColumn method. I wonder how is it implemented, how is it possible in principle to track the offset? How does lexer match your Editor's width?

答:
JavaCC answers this way

The generated parser has a JavaCharStream named jj_input_stream that
  has setTabSize(int) and getTabSize() methods. Use the tab width value
  from your IDE and pass it to the setTabSize method, this will make the
  token locations more accurate.

Mine proposal is better. The compiler should stuck at constant tab width, say 1 to count tab as any other character. The editor, which is aware of both the current tab width as well as the number of tabs on the current line (editor can easily compute this information since it has the file loaded in memory and already displays the line of code), adjusts the column information from the compiler accordingly. The only problem would be programmer's surprise when he juxtaposes the compiler-reported locations with the objects selected in the editor.


答: I have personally found that writing a Python compiler and concrete set of classes that implement Python's grammar to be an extraordinary learning experience in the inner workings of Python. My goals are different than something like PyPy (also implements a self-hosting compiler) but it sure lets you turn the magic dial up to 11! Hahaha</p>


问:
Best way to handle variables used in a for loop?
From previous experience, I had always thought that, if you are going to use variables inside of a for loop, it was much better to declare them outside of the loop vs. inside the loop itself. I recently had a code review done and had several younger developers claim that this was not true and that I should be putting my variables declarations inside the {} of the for loop.

Maybe compilers have just become more efficient, but it seems this would cause quite a number of memory releases/garbage collects since each iteration would be declaring a new instance of each variable, especially given that the majority of them are strings which are immutable.

答:
Generally you want to restrict the scope of your objects as much as possible. If your variable is only used inside the loop, and you do not require state to carry between iterations, then yes you usually want to declare it inside the loop so that each iteration contains a fresh variable.

Sometimes this carries performance penalties, as you suggest. I'd wager it's pretty rare, especially with modern optimising compilers, but it's still conceivable.

As always, the best advice I can give is to measure it if you think there's a problem. If you really need to, then fine move the declaration outside and be sure to "reset" the variable at the start of each iteration, if that's what turns out to be cheapest. Just be aware that your code will make slightly less sense to readers, so you need to write a comment to explain your choice (if for no other reason than to prevent future maintainers from undoing it) and be extra careful that you're not causing bugs.

So, were the younger developers right? Yes, in the main. But — as always — context rules.


问:
Taking credit for the public domain work of others
I’ve released a piece of code into the public domain (henceforth “PD”), via The Unlicense. Recently, I’ve found someone forked that code and made some modifications without any acknowledgment/attribution. That is absolutely fine (it’s to allow that kind of freedom that I’ve released it into the PD in the first place). Fine, that is, until I see that no code was actually changed.

The only changes he did were replacing my name and website for his, and changing the icon.

So what this person did was the equivalent of taking Alice’s Adventures in Wonderland, stripping every reference to Lewis Carroll and replacing them with the equivalent references to himself, changing the cover, and republishing.

No license was attached to it by the person (and the PD one was removed), but this still feels like a dickish thing to do. I’m not exactly bothered by it and am not thinking about pursuing it (it was my decision to put it into the PD, after all) but I’m wondering about not only the ethics but the legality of this. If I understand correctly, PD still does not allow you to take credit for something you haven’t done, and that is still plagiarism. Is that correct, or is it jurisdiction-specific?

答:
If I understand correctly, PD still does not allow you to take credit for something you haven’t done, and that is still plagiarism. Is that correct, or is it jurisdiction-specific?

I don't believe that to be correct, at least not in the general sense.  Plagiarism can have academic, professional and social implications, but none of those have the force of law unless -- and this is the important part -- doing so infringes on the owner's copyright.

The public domain exists so works can be used by anyone for any purpose once the term of exclusivity (i.e., any copyright or patent) has expired or the author explicitly releases it.  Among those uses is the creation of slightly-derivative versions of public-domain works that don't credit the original author.  It may not be moral or ethical, but according to the U.S. Supreme Court, it is legal.  (Scalia makes some interesting comments in the decision about the implications creating what is essentially a mutant zombie version of copyright law.)  I don't imagine that the courts in any country that's ratified the Berne Convention would rule differently.

Preventing that kind of plagiarism is why the plethora of licenses with relatively few restrictions exist and are a better alternative to relinquishing all of your rights by releasing into the public domain.

There are a couple of states (Florida is one of them) that proscribe selling work that the seller knows will be turned in for academic credit.  I don't know how well a law like that would stand up to a legal challenge if the seller has the rights to distribute the work (either through license or because it's in the public domain).


问:
Should I be using advanced techniques most of the time at my new job just because I can?
Simply put I'm new to the company, should I rather write advanced techniques with things like templates, std techniques..etc to make a first good impression and have my colleagues trust/be impressed at my work or should I be more concerned in writing more solid/based standard code for each current problem to be solved?

答:
Absolutely not.

If you joined my team and spent your whole time using more advanced techniques than were necessary for the task at hand, I would certainly be less than impressed. You'd be making it more difficult to read, understand and maintain the code … not only for your teammates, but also for yourself. And for what reason? Showing off? Not cool.

Though at least you'd be making it easy for me to find content for your first performance review!

The way to impress me is to use the right tool for the job. If a problem comes up which is best solved with an "advanced technique", and you are capable of delivering such a solution in such a way that code stability and maintainability is not compromised, then I'd be impressed.

To be clear, though, the use of templates and standard library facilities is hardly "advanced". I require anyone joining my C++ team to be familiar with these as a matter of course.


答:
In the context of C#, it is hard to imagine how declaring the variable in one scope or another affects the garbage collection.  In C# and Java, scope does not dictate storage reclamation (while on the other hand, it has implications in C++).

Local variables, in particular in C#, are mostly free.  So, declaring one in an inner scope, even if looping is basically free.

It is possible that some initializer expression for something that could be reused (e.g. is unchanging) in the loop is more efficient when outside the loop, however, your question doesn't directly mention that, and seems to be concerned with the cost of the variables instead.

In general, we are better off to declare variables in the smallest scope possible.  In general it is premature to consider performance before other factors, but in this case there is virtually no tension between performance and maintainable coding style.

Title: Best way to handle variables used in a for loop?

Body:
From previous experience, I had always thought that, if you are going to use variables inside of a for loop, it was much better to declare them outside of the loop vs. inside the loop itself. I recently had a code review done and had several younger developers claim that this was not true and that I should be putting my variables declarations inside the {} of the for loop.

Maybe compilers have just become more efficient, but it seems this would cause quite a number of memory releases/garbage collects since each iteration would be declaring a new instance of each variable, especially given that the majority of them are strings which are immutable.


答:
It's worse than that, Jim ...

Since you tagged the question c#, you're not only wrong in assuming that there is any performance benefit (this is a trivial optimization, one any halfway-decent compiler will perform), but also in assuming that the declarations are re-executed on each loop pass.  In fact, the variable will have space allocated for it at some point between entry into the loop and its first use, and will be freed sometime after its last use.  In the case of a non-primitive variable (i.e., an object), only executing a new operator will actually cause the object to be created, regardless of where the variable is declared, and every new will create a new object, every time.


问:
How important is multithreading in the current software industry?
I have close to 3 years experience writing web applications in Java using MVC frameworks (like struts). I have never written multithreaded code till now though I have written code for major retail chains.

I get a few questions on multithreading during interviews and I answer them usually (mostly simple questions). This left me wondering how important is Multithreading in the current industry scenario ?

答:
This left me wondering how important is Multithreading in the current
  industry scenario?

In performance-critical fields where the performance isn't coming from third party code doing the heavy-lifting, but our own, then I'd tend to consider things in this order of importance from the CPU perspective (GPU is a wildcard I won't go into):

Memory Efficiency (ex: locality of reference).

Algorithmic

Multithreading

SIMD

Other Optimizations (static branch prediction hints, e.g.)

Note that this is list is not solely based on importance but a lot of other dynamics like the impact they have on maintenance, how straightforward they are (if not, worth considering more in advance), their interactions with others on the list, etc.

Memory Efficiency

Most might be surprised at my choice of memory efficiency over algorithmic. It's because memory efficiency interacts with all 4 other items on this list, and it's because consideration of it is often very much in the "design" category rather than "implementation" category. There is admittedly a bit of a chicken or the egg problem here since understanding memory efficiency often requires considering all 4 items on the list, while all 4 other items also require considering memory efficiency. Yet it's at the heart of everything.

For example, if we have a need for a data structure that offers linear-time sequential access and constant-time insertions to the back and nothing else for small elements, the naive choice here to reach for would be a linked list. That's disregarding memory efficiency. When we consider memory efficiency in the mix, then we end up choosing more contiguous structures in this scenario, like growable array-based structures or more contiguous nodes (ex: one storing 128 elements in a node) linked together, or at the very least a linked list backed by a pool allocator. These have a dramatic edge in spite of having the same algorithmic complexity. Likewise, we often choose quicksort of an array over merge sort in spite of an inferior algorithmic complexity simply because of memory efficiency.

Likewise, we can't have efficient multithreading if our memory access patterns are so granular and scattered in nature that we end up maximizing the amount of false sharing while locking at the most granular levels in code. So memory efficiency multiplies the efficiency multithreading. It's a prerequisite to getting the most of out threads.

Every single item above on the list has a complex interaction with data, and focusing on how data is represented is ultimately in the vein of memory efficiency. Every single one of these above can be bottlenecked with an inappropriate way of representing or accessing data.

Another reason memory efficiency is so important is that it can apply throughout an entire codebase. Generally when people imagine that inefficiencies accumulate from little bitty sections of work here and there, it's a sign that they need to grab a profiler. Yet low-latency fields or ones dealing with very limited hardware will actually find, even after profiling, sessions that indicate no clear hotspots (just times dispersed all over the place) in a codebase that's blatantly inefficient with the way it's allocating, copying, and accessing memory. Typically this is about the only time an entire codebase can be susceptible to a performance concern that might lead to a whole new set of standards applied throughout the codebase, and memory efficiency is often at the heart of it.

Algorithmic

This one's pretty much a given, as the choice in a sorting algorithm can make the difference between a massive input taking months to sort versus seconds to sort. It makes the biggest impact of all if the choice is between, say, really sub-par quadratic or cubic algorithms and a linearithmic one, or between linear and logarithmic or constant, at least until we have like 1,000,000 core machines (in which case memory efficiency would become even more important).

It's not at the top of my personal list, however, since anyone competent in their field would know to use an acceleration structure for frustum culling, e.g. We're saturated by algorithmic knowledge, and knowing things like using a variant of a trie such as a radix tree for prefix-based searches is baby stuff. Lacking this kind of basic knowledge of the field we're working in, then algorithmic efficiency would certainly rise to the top, but often algorithmic efficiency is trivial.

Also inventing new algorithms can be a necessity in some fields (ex: in mesh processing I've had to invent hundreds since they either did not exist before, or the implementations of similar features in other products were proprietary secrets, not published in a paper). However, once we're past the problem-solving part and find a way to get the correct results, and once efficiency becomes the goal, the only way to really gain it is to consider how we're interacting with data (memory). Without understanding memory efficiency, the new algorithm can become needlessly complex with futile efforts to make it faster, when the only thing it needed was a little more consideration of memory efficiency to yield a simpler, more elegant algorithm.

Lastly, algorithms tend to be more in the "implementation" category than memory efficiency. They're often easier to improve in hindsight even with a sub-optimal algorithm used initially. For example, an inferior image processing algorithm is often just implemented in one local place in the codebase. It can be swapped out with a better one later. However, if all image processing algorithms are tied to a Pixel interface which has a sub-optimal memory representation, but the only way to correct it is to change the way multiple pixels are represented (and not a single one), then we're often SOL and will have to completely rewrite the codebase towards an Image interface. Same kind of thing goes for replacing a sorting algorithm -- it's usually an implementation detail, while a complete change to the underlying representation of data being sorted or the way it's passed through messages might require interfaces to be redesigned.

Multithreading

Multithreading is a tough one in the context of performance since it's a micro-level optimization playing to hardware characteristics, but our hardware is really scaling in that direction. Already I have peers who have 32 cores (I only have 4).

Yet mulithreading is among the most dangerous micro-optimizations probably known to a professional if the purpose is used to speed up software. The race condition is pretty much the most deadly bug possible, since it's so indeterministic in nature (maybe only showing up once every few months on a developer's machine at a most inconvenient time outside of a debugging context, if at all). So it has arguably the most negative degradation on maintainability and potential correctness of code among all of these, especially since bugs related to multithreading can easily fly under the radar of even the most careful testing.

Nevertheless, it's becoming so important. While it may still not always trump something like memory efficiency (which can sometimes make things a hundred times faster) given the number of cores we have now, we're seeing more and more cores. Of course, even with 100-core machines, I'd still put memory efficiency at the top of the list, since thread efficiency is generally impossible without it. A program can use a hundred threads on such a machine and still be slow lacking efficient memory representation and access patterns (which will tie in to locking patterns).

SIMD

SIMD is also a bit awkward since the registers are actually getting wider, with plans to get even wider. Originally we saw 64-bit MMX registers followed by 128-bit XMM registers capable of 4 SPFP operations in parallel. Now we're seeing 256-bit YMM registers capable of 8 in parallel. And there's already plans in place for 512-bit registers which would allow 16 in parallel.

These would interact and multiply with the efficiency of multithreading. Yet SIMD can degrade maintainability just as much as multithreading. Even though bugs related to them aren't necessarily as difficult to reproduce and fix as a deadlock or race condition, portability is awkward, and ensuring that the code can run on everyone's machine (and using the appropriate instructions based on their hardware capabilities) is awkward.

Another thing is that while compilers today usually don't beat expertly-written SIMD code, they do beat naive attempts easily. They might improve to the point where we no longer have to do it manually, or at least without getting so manual as to writing intrinsics or straight-up assembly code (perhaps just a little human guidance).

Again though, without a memory layout that's efficient for vectorized processing, SIMD is useless. We'll end up just loading one scalar field into a wide register only to do one operation on it. At the heart of all of these items is a dependency on memory layouts to be truly efficient.

Other Optimizations

These are often what I would suggest we start calling "micro" nowadays if the word suggests not only going beyond algorithmic focus but towards changes that have a minuscule impact on performance.

Often trying to optimize for branch prediction requires a change in algorithm or memory efficiency, e.g. If this is attempted merely through hints and rearranging code for static prediction, that only tends to improve the first-time execution of such code, making the effects questionable if not often outright negligible.

Back to Multithreading for Performance

So anyway, how important is multithreading from a performance context? On my 4-core machine, it can ideally make things about 5 times faster (what I can get with hyperthreading). It would be considerably more important to my colleague who has 32 cores. And it will become increasingly important in the years to come.

So it's pretty important. But it's useless to just throw a bunch of threads at the problem if the memory efficiency isn't there to allow locks to be used sparingly, to reduce false sharing, etc.

Multithreading Outside of Performance

Multithreading isn't always about sheer performance in a straightforward throughput kind of sense. Sometimes it's used to balance a load even at the possible cost of throughput to improve responsiveness to the user, or to allow the user to do more multitasking without waiting for things to finish (ex: continue browsing while downloading a file).

In those cases, I'd suggest that multithreading rises even higher towards the top (possibly even above memory efficiency), since it's then about user-end design rather than about getting the most out of the hardware. It's going to often dominate interface designs and the way we structure our entire codebase in such scenarios.

When we're not simply parallelizing a tight loop accessing a massive data structure, multithreading goes to the really hardcore "design" category, and design always trumps implementation.

So in those cases, I'd say considering multithreading upfront is absolutely critical, even more than memory representation and access.


答: Sometimes the environment is conservative or just not open to do things any other way then what people are used to and thus comfortable with. Then you will meet opposition whenever you go off the path your co-workers know and they will throw bogus reasoning at you for not using a particular technique rather than trying to catch up and learn something new. There is no standard way to deal with that. Sometimes it is better to just do it and let the results speak for themselves, other times it is better to not stir the pot too much and let everyone get used to the new guy first, to gain some credit, acceptance and willingness. It is more of a social challenge than a technical one. </p>

Title: Should I be using advanced techniques most of the time at my new job just because I can?

Body: Simply put I'm new to the company, should I rather write advanced techniques with things like templates, std techniques..etc to make a first good impression and have my colleagues trust/be impressed at my work or should I be more concerned in writing more solid/based standard code for each current problem to be solved?


答:
As a disclaimer, I'm interpreting "advanced" here as not like a basic class template for a data structure or a function template for a generic algorithm which inputs iterators. This hardly seems like showboating territory unless it's among a team of people who don't know how to use C++. I'm assuming things like, say, policy class templates that allow infinite combinations in policies when only one combination is actually useful: exotic solutions far beyond the norm... maybe even more advanced than policy classes: things without a name yet that people have never seen before, tricks using the language that have never yet been applied in production code.

Noooooooooooo!

The hallmark of an experienced developer is not one who applies recursive metatemplates to create something resembling a basic DSEL to solve a routine task. That's programming masturbation -- it's best saved for scrap projects on the side to better understand how the language works or when writing a book on how to explore the deepest boundaries of a programming language.

It doesn't even come from an elaborate mechanism to replace 3 lines of straightforward boilerplate in 50 source files with 1 line of code. If anything, experience will have you initially excited with such solutions only to, over the years, come back to the straightforward solution when realizing that maintainability was actually degraded instead of improved by getting so fancy.

One of the key hidden metrics of maintainability that isn't discussed so often is "familiarity". Exotic code is unfamiliar code, and likely to be unfamiliar even to yourself after some time has passed upon revisiting it. This is why idiomatic code is so valuable -- it improves maintainability by improving familiarity. We're saturated by idiomatic code, and so it tends to remain ever familiar. "Exotic" is the polar opposite, and it's usually coupled by a desire to reshape the programming language being used rather than to embrace its strengths and accept its weaknesses.

Templates can be wonderful and straightforward ways of achieving compile-time abstractions at no runtime penalty, and sometimes are even properly used in a recursive or DSEL-like context for something the language would otherwise be incredibly inept to handle. I'm assuming that, when you say "advanced", it's to use templates in a very unusual way. But embraced too quickly, they can quickly end up counteracting the goals they were intended to serve: they can turn into the elaborate macro.

If you want to impress people, use the simplest, most idiomatic, straightforward solution you can get away with -- the less exotic the better. Sometimes there will be sections in a codebase that have an unusually complex demand where we have to break away from the norm. Yet the key here is to do this sparingly, reluctantly, to not apply complex solutions for simple tasks. The genius-level solution is to find a simple, symmetrical solution to a seemingly very complex task -- this is often very difficult, but it's not so hard to avoid applying complex solutions for simple tasks.

While this is C instead of C++, check out the Linux kernel. It's written in the most astoundingly straightforward way given the combination of complexity and the critical nature of what it's doing, yet it's organized logically and better than some basic codebases that are doing far simpler things. Take these kinds of sources of simplicity and straightforwardness as inspiration, and as a way to impress your colleagues.

It's great to understand really advanced techniques like the intricacies of SFINAE, but save such advanced knowledge for the most advanced problems where the advanced solutions end up being the most elegant and straightforward solutions.