问:
Any programming languages that support Generics exclusively and have no OOP support?
I am writing a paper on the tension between OOP and Generic programming created by Stepanov. He widely criticizes OOP and says it is "technically flawed" when compared to Generic Programming.

Now I know we have plenty of programming languages that support OOP exclusivley and have no Generic support including Google GO for example, that is a modern language that they chose not to implement Generics because of their complexity. I know we have plenty of languages that support both Generics and OOP, for example C++ and Stepanov's famous STL library.

My question is: Do we have any modern programming languages or any at all that support Generics exclusively without OOP?

EDIT: I would like to add I tried looking around and cant seem to find much so I figured I would ask here.

Just to add that when I refer to Generic vs Object Oriented Programming I am refering to:

Generic Vs Object Oriented Programming

On the Tension Between Object-Oriented and Generic Programming in C++

答:
Generic programming encompasses more than just “Generics”, which in turn is another name for parametric polymorphism. This concept is rather widespread and not constrained to object-oriented languages. It is a distinguishing feature of the ML language family which includes Standard ML, Ocaml (which also supports OOP, but is primarily functional) and Haskell.

Parametric polymorphism isn't the only way to do generic programming. For example, macro systems (like templates in C++) or a dynamic type system are other ways.

However, OOP in its various flavours is another good and widespread way to do generic programming, e.g through the use of interfaces (or an equivalent concept) or duck typing. In this sense Go is generic, as a function in that language can take an argument that is only defined by its interface.


问:
What is the difference between "Syntax" and "Syntactic Sugar"

Background

The Wikipedia page on Syntactic Sugar states:

In computer science, syntactic sugar is syntax within a programming language that is designed to make things easier to read or to express. It makes the language "sweeter" for humans to use: things can be expressed more clearly, more concisely, or in an alternative style that some may prefer.

I don't really understand what the difference is between Syntactic Sugar and Syntax.

I appreciate the point that the sugary version can be clearer, more concise, perhaps boil off some boilerplate. But I feel at some level, all the syntax is essentially doing that to form an abstraction over what the code gets compiled down to.

From the same Wikipedia page:

Language processors, including compilers, static analyzers, and the like, often expand sugared constructs into more fundamental constructs before processing, a process sometimes called "desugaring".

As a thought exercise, if I take "often" in this statement to mean "always": If the difference were just whether the compiler "desugars" the syntax before moving to a next stage, how might a coder who does not know the innards of the compiler know (or care) what is Sugar'd Syntax or not?

A very much related question on this site "Rigorous Definition of Syntactic Sugar?" has an answer which commences:

IMHO I don't think you can have a definition for syntactic sugar, because the phrase is BS and is likely to be used by people that talk about "real programmers" using "real tools" on "real operating systems"

Which might indicate to me that perhaps there isn't a huge difference to the coder using the language?
Perhaps the difference is only perceptible to the compiler writer? Though there may be instances in which it's helpful for the coder using the language to know what's under the hood of the Syntactic Sugar? (But perhaps in reality any discourse on the subject tends to use the term as flame bait?)

The heart of the question

So... the short version of the question:

Is there a real difference between Syntax and Syntactic Sugar?

Who does it matter to?

Extra Food for Thought

Bonus on topic contradiction:

On the Wikipedia page an example is given:

For instance, in the C language the a[i] notation is syntactic sugar for *(a + i)

Whereas another answer on the above linked question talks about the same example:

Now consider a[i] == *(a + i). Think about any C program that uses arrays in any substantive way.

And summarizes that:

The [] notation facilitates this abstraction. It's not syntactic sugar.

The opposite conclusion for the same example!

答:
I think the most obvious example would be the "+=" syntax in C.

```<code>i = i + 1;
</code>```

and

```<code> i +=  1;
</code>```

do exactly the same thing and compile to exactly the same set of machine instructions. The second form saves a couple of characters typing, but, more importantly makes it very clear that you are modifying a value based on its current value.

I was going to cite the "++" post/prefix operator as the canonical example but realized that this was more than syntactic sugar. There is no way to express the difference between ++i and i++ in a single expression using the i = i + 1 syntax.

ContentLicense: CC BY-SA 3.0

Body:
I think the most obvious example would be the "+=" syntax in C.

i = i + 1;

and

 i +=  1;

do exactly the same thing and compile to exactly the same set of machine instructions. The second form saves a couple of characters typing, but, more importantly makes it very clear that you are modifying a value based on its current value.

I was going to cite the "++" post/prefix operator as the canonical example but realized that this was more than syntactic sugar. There is no way to express the difference between ++i and i++ in a single expression using the i = i + 1 syntax.


问:
What must I take into consideration when designing a UI around a 0..1:1 relationship?
I'm designing the database schema for a new product feature. In my current design I have some related optional data. Rather than have nullable fields I have a separate table with a 0..1:1 relationship to the main table. I chose this design because the queries are simpler* if null data doesn't have to be taken into consideration.

The team lead pointed out to me that it will complicate data binding in the UI and suggested that I just use nullable fields. I am wondering what complications the optional table approach will introduce?

The obvious thing that comes to mind is the data aware controls can't bind to a closed data set so I'll need to either create a record on the fly when the user attempts to fill in the optional data or create the record at the same time as the main record, negating the purpose of have a separate table. Is there anything else I should be aware of?

Clarification

*Simpler, by which I refer to the dizzying amount of rules and ambiguities concerning null's behavior under different query scenarios in the SQL standard and the fact that no two vendors agree on which of these rules to implement.

答:
I do not think you design simplfies anything as far as NULL handling is concerned. To use your second table you will need a "LEFT OUTER JOIN" to your main table which will return NULL values if there is no matching row in the second table. Or you will need to do the equivalent logic in your program code and have two separate code paths; one for "found a row" and one for "row not found". So what would be the net gain?

There may be other reasons to separate the data into another table but reducing the complexity of NULL handling is not one of them. What little code you save in checking for NULLs will be far exceeded by the complexity of maintaining a separate table.


问:
Random access (read/write) in datastructures
Certain datastructures, like Python's Dictionary, are unordered/random read/written.
As programming in python is iterative (and programming in general is?), how do these unordered datastructures work?

I understand that Python's Dictionary is in essence a hashtable. But I figured that the datastructure gets stored in the memory by order of inserting, and are thus read, when iterating over the full datastructure, in a similar order. But this is not the case.

Aside from not understanding how this works, I also wonder of the benefit of this. But that may be all too clear when I understand how it works :)

ps. Didn't think this was a StackOverflow Question, so put it here...

答:
A hash table has some parts that are ordered, such as the list of buckets, but you very rarely access them in that order because that order has little to no correlation with any useful part of your data.

Say you want to store a lot of strings. If your goal is to efficiently pull out the 73rd (or other arbitrarily selected) string, then you might use an array, which is ordered. Then getting the 73rd string is a matter of referencing the 73rd element in the array directly.

However, say your goal is to efficiently determine if a particular string is in your set. If you have an unsorted array, you need to search through them all of the strings. If you have a sorted array then you need to sort through log(n) of the strings, which is better but still not great. So the solution is a hash table.

Naively... You make an array with 256 entries (each representing one character), each pointing to a list or array of strings that start with that character. Now if you want to find out if "Bob" is in your set of strings, you immediately jump to the 'B' bucket and only look through those strings.

Of course, this example still means you get the strings out in alphabetical order, which sounds nice, but in the real world you would use a hash function more complex than "the first character", preferably one that distributes strings as evenly as possible between your buckets. The result of which is that simply reading through your hash table in order will give you a pseudo-random ordering of your strings.


问:
Examples of Java APIs that demand an action sequence
As part of a research I'm working on, I'm looking for public APIs that only work correctly when you apply a certain sequence of actions on them.

For example, the java.nio.channels.SocketChannel class, from the Java standard library, only works correctly with sequences such as open() -> connect() -> read() -> read() -> close(). A more complete demonstration if how it may be used may be represented in the following graph:

Additional examples of Java standard library APIs that require certain sequences are java.io.PrintStream (very similar to the one above) and java.util.Iterator (which requires a next() call between every two remove() calls, thus enforcing a certain sequence).

So, does you favorite API for doing X also behave that way? I would very much like to know about additional APIs that require a certain method sequence for correct usage; especially classes that are not part of the Java standard library. The more complex the sequence(s) required, the better.

Some APIs require a sequence that spans across multiple classes, for example:

```<code>X x = new X();
x.setup();
Y y = x.createNewY();
Z z = new Z(y);
z.doSomething();
</code>```

These examples are also interesting, but I'm mostly looking for sequences that all appear in the same class.

EDIT added bounty for greater visibility. I'm sure many of you have encountered many APIs that will match this description - I would really appreciate some good examples.

答:
The C++ STL has a variety of rules about when iterators are invalidated. For example, adding an element to a vector may cause all the iterators into that vector to be invalidated.

Any container library must have special cases for empty containers.

I've dealt with a database api, IBPP, it required a ->Start() call to the transaction object, before creating any statements associated with the transaction. Eventually you would call ->Commit(); (or ->Rollback() if things didn't go well) Other APIs are similar, but probably a bit more complex because of issues such as autocommit.

I've certainly dealt with internal code that acted that way. However, I considered that a bug.

The pygame module in python requires that call .init() and then .display.set_mode() to setup a graphics mode.

Everything I can think of has relatively trivial versions of this. I can't think of any cases that has a really complex version like you really want.

ContentLicense: CC BY-SA 2.5


答:
Dictionaries/Hashes are often implemented as a Array of linked lists.

The key is first hashed to give a subscript for the array which will point to the first entry of a linked list.
Then the linked list is searched to see if an entry with the key exists.

So for any reasonably large dictionary there will be a randomly ordered array, of linked lists. Each linked list will be ordered by entry sequence.

Title: Random access (read/write) in datastructures

Body:
Certain datastructures, like Python's Dictionary, are unordered/random read/written.
As programming in python is iterative (and programming in general is?), how do these unordered datastructures work?

I understand that Python's Dictionary is in essence a hashtable. But I figured that the datastructure gets stored in the memory by order of inserting, and are thus read, when iterating over the full datastructure, in a similar order. But this is not the case.

Aside from not understanding how this works, I also wonder of the benefit of this. But that may be all too clear when I understand how it works :)

ps. Didn't think this was a StackOverflow Question, so put it here...


问:
Alternative to Singleton when D.I. is not an option
I have been researching this extensively, and I am stuck. I am writing a Drupal module that uses a class that I have created. This class is a classic Singleton - it only needs to be instantiated once, since it is assembling a bunch of data from a database. Note that my potential Singleton will not update any data in the database - it will just read from existing data. Also, note that processes outside the context of my program may update the data.

I know the take home answer here is 'use Dependency Injection'. However, I can not use Dependency Injection - Drupal uses hooks, and these hooks accept certain parameters. I.e., I can't (to my knowledge) insert a object that way.

I should point out that testing is a concern of mine as well (although testing my module in Drupal has had its own series of challenges). However, I am willing to accept that a Singleton may be the lesser of two evils here.

Here is an example:

```<code>function mymod_block_view($var) {
    //$var is required by drupal
    $obj = new Object()
    ....
}

function mymod_block_info() {
    ...
    $obj = new Object()
}
</code>```

My options right now are to use a global variable (not ideal) or a Singleton (according to my research, also not ideal, but may be my only option). Do I have any other options, besides declaring a new variable object inside each function?

答:
You seem to be stuck between a rock and a hard place. Neither a global variable nor a singleton are very "good" practice. The main issue I see is that singletons are not really to be used when you only need to instantiate it once, but instead when you MUST only instantiate them once. For example, you may only need one instance of a scoreboard in a game, but nothing breaks when you have two or three instances (if done well). Writing to a log file across multiple threads might break if writes are not threadsafe (they almost assuredly won't be natively), and so having multiple instances of the file handle could be destructive. This is a better reason for having a singleton.

Since I don't believe you have a real use case for a singleton, I suggest a global variable. That is essentially what a singleton would be anyway, here, but you gain the advantages of being variable. This would make switching between databases mid-execution simpler, for example, and you might even find it easier to test than a singleton.


问:
What is the name of the following (anti) pattern? What are its advantages and disadvantages?
Over the last few months, I stumbled a few times over the following technique / pattern. However, I can't seem to find a specific name, nor am I a 100% sure about all its advantages and disadvantages.

The pattern goes as follows:

Within a Java interface, a set of common methods is defined as usual. However, using an inner class, a default instance is leaked through the interface.

public interface Vehicle {
    public void accelerate();
    public void decelerate();

    public static class Default {
         public static Vehicle getInstance() {
             return new Car(); // or use Spring to retrieve an instance
         }
    }
 }
```

<p>For me, it seems that the biggest advantage lies in the fact that a developer only needs to know about the interface and not its implementations, e.g. in case he quickly wants to create an instance. </p>

<pre class="lang-java prettyprint-override"><code> Vehicle someVehicle = Vehicle.Default.getInstance();
 someVehicle.accelerate();
</code>```

Furthermore, I have seen this technique being used together with Spring in order to dynamically provide instances depending on the configuration. In this regard, it also looks like this can help with modularization.

Nevertheless, I can't shake the feeling that this is a misuse of the interface since it couples the interface with one of its implementations. (Dependency inversion principle etc..) Could anybody please explain to me how this technique is called, as well as its advantages & disadvantages?

Update:

After some time for consideration, I rechecked and noticed that the following singleton version of the pattern was used far more often. In this version, a public static instance is exposed through the interface which is initialized only once (due to the field being final). In addition, the instance is almost always retrieved using Spring or a generic factory which decouples the interface from the implementation.

public interface Vehicle {
      public void accelerate();
      public void decelerate();

      public static class Default {
           public static final Vehicle INSTANCE = getInstance();

           private static Vehicle getInstance() {
                return new Car(); // or use Spring/factory here
           }
      }
 }

 // Which allows to retrieve a singleton instance using...
 Vehicle someVehicle = Vehicle.Default.INSTANCE;
```

In a nutshell: it seems that this is a custom singleton/factory pattern, which basically allows to expose an instance or a singleton through its interface. With respect to the disadvantages, a few have been named in the answers & comments below. So far, the advantage seems to lie in its convenience.

答:
Default.getInstance is IMHO just a very specific form of a factory method, mixed up with a naming convention taken from the singleton pattern (but without being an implementation of the latter). In the current form this is a violation of the "single responsibility principle", because the interface (which already serves the purpose of declaring a class API) takes the additional responsibility of providing a default instance, which would be much better placed in a separate VehicleFactory class.

The main problem caused by this construct is that it induces a cyclic dependency between Vehicle and Car. For example, in the current form it won't be possible to place Vehicle in one library, and Car in another. Using a separate factory class and placing the Default.getInstance method there would solve that problem. It may be also a good idea to give that method a different name, to prevent any confusion related to singletons. So the result could be something like

VehicleFactory.createDefaultInstance()


答:
It seems to me that you are trying to Apply the best practices for Java (multi-threaded, long lasting processes etc., general purpose language) to PHP (Single thread, processes initialized on each request, specialized for delivering HTML pages).

There are good reasons to avoid singletons in Java, but I do not see how they apply in the single threaded short life cycle of a php program.

Title: Alternative to Singleton when D.I. is not an option

Body:
I have been researching this extensively, and I am stuck. I am writing a Drupal module that uses a class that I have created. This class is a classic Singleton - it only needs to be instantiated once, since it is assembling a bunch of data from a database. Note that my potential Singleton will not update any data in the database - it will just read from existing data. Also, note that processes outside the context of my program may update the data.

I know the take home answer here is 'use Dependency Injection'. However, I can not use Dependency Injection - Drupal uses hooks, and these hooks accept certain parameters. I.e., I can't (to my knowledge) insert a object that way.

I should point out that testing is a concern of mine as well (although testing my module in Drupal has had its own series of challenges). However, I am willing to accept that a Singleton may be the lesser of two evils here.

Here is an example:

```<code>function mymod_block_view($var) {
    //$var is required by drupal
    $obj = new Object()
    ....
}

function mymod_block_info() {
    ...
    $obj = new Object()
}
</code>```

My options right now are to use a global variable (not ideal) or a Singleton (according to my research, also not ideal, but may be my only option). Do I have any other options, besides declaring a new variable object inside each function?


问:
Best books on the theory and practice of software architecture?
I have a couple of developers at my company who wish to move from programming into architecture. What are the best books out there on the theory and practice of software architecture? Include a cover picture if you can.

Feel free to include general books, and also books that relate to a specific technology.

答: But, I would encourage these developers not to move into architecture and totally forget about programming. If they do, they won't have any respect from programmers and they won't be properly equipped to make architectural decisions.


问:
Best way to test a reimplemented web service
In my team, we are about to start the reimplement of a service. One of the important steps to do to accomplish this is how do we ensure that we are doing it in the right way and  we are not introducing new bugs.

So, what we have in mind is create a bunch of test to verify the following:

The behavior. Both services should behave the same way (store data in the same places, send same, notificacions, etc...)

Results. Object returned by the service's calls should be exactly the same.

So, the things that we have thought is to do the following:

Create a set of test that verifies the behavior and results of old service.

Create the new service adding unit test, and whitebox integration test for the new service

Create some kind of "Mirror test" that checks if the new service is working as the old one. Is there a way to do this?

Thanks for any help.

Note: None of the code of the previous version is reusable. There is no test for the older version of the service

[Edited]replaced original word "migrate" with "reimplement"

答:
What you have described is basically exactly what Unit Testing can provide for you when doing large refactoring or in your case, an entire rewrite.

You'll want to pay particular attention to making sure your interface does not change at all as you will have existing clients, and having a full on integration test might be worth looking at as well.


答:
This looks like Null Object pattern to me.

But that heavily depends on how Car class is implemented. If it is implemented in "neutral" way, then it is definitely a Null Object. That means, if you can remove all null checks on the interface and put instance of this class instead of each occurrence of null, then code still has to work correctly.

Also if it is this pattern, then there is no problem of creating tight coupling between the interface itself and implementation of null object. Because null object can to be everywhere where said interface is used.

Title: What is the name of the following (anti) pattern? What are its advantages and disadvantages?

Body:
Over the last few months, I stumbled a few times over the following technique / pattern. However, I can't seem to find a specific name, nor am I a 100% sure about all its advantages and disadvantages.

The pattern goes as follows:

Within a Java interface, a set of common methods is defined as usual. However, using an inner class, a default instance is leaked through the interface.

public interface Vehicle {
    public void accelerate();
    public void decelerate();

    public static class Default {
         public static Vehicle getInstance() {
             return new Car(); // or use Spring to retrieve an instance
         }
    }
 }
```

<p>For me, it seems that the biggest advantage lies in the fact that a developer only needs to know about the interface and not its implementations, e.g. in case he quickly wants to create an instance. </p>

<pre class="lang-java prettyprint-override"><code> Vehicle someVehicle = Vehicle.Default.getInstance();
 someVehicle.accelerate();
</code>```

Furthermore, I have seen this technique being used together with Spring in order to dynamically provide instances depending on the configuration. In this regard, it also looks like this can help with modularization.

Nevertheless, I can't shake the feeling that this is a misuse of the interface since it couples the interface with one of its implementations. (Dependency inversion principle etc..) Could anybody please explain to me how this technique is called, as well as its advantages & disadvantages?

Update:

After some time for consideration, I rechecked and noticed that the following singleton version of the pattern was used far more often. In this version, a public static instance is exposed through the interface which is initialized only once (due to the field being final). In addition, the instance is almost always retrieved using Spring or a generic factory which decouples the interface from the implementation.

public interface Vehicle {
      public void accelerate();
      public void decelerate();

      public static class Default {
           public static final Vehicle INSTANCE = getInstance();

           private static Vehicle getInstance() {
                return new Car(); // or use Spring/factory here
           }
      }
 }

 // Which allows to retrieve a singleton instance using...
 Vehicle someVehicle = Vehicle.Default.INSTANCE;
```

In a nutshell: it seems that this is a custom singleton/factory pattern, which basically allows to expose an instance or a singleton through its interface. With respect to the disadvantages, a few have been named in the answers & comments below. So far, the advantage seems to lie in its convenience.


问:
Package diagram for an MVC patterned project?
We are required to make a package diagram for our senior project. Since our project uses MVC patter design, we created an MVC class diagram, now, our problem is in creating the package diagrams from our class diagram.

Is it possible to have packages with MVC at the same time? so it would be something like this:

Package: Account

Account Model

Profile Controller

Registration Controller

Profile View

Registration View

These are the controller that cannot exists without the Account model, so I included them.

Thanks in advance!

答:
First thing you have to do is to decide about the semantics of your UML packages. UML does not define any clear semantics, you have to define one that suits your needs.

For example, a package could be a layer in your software (i.e. model layer, view layer, controller layer), so if that is the semantics you pick, this will obviously fit to MVC. I guess that is not the semantics you have in mind, otherwise you would not see any problem here.

A different possible semantics is a namespace, a component, a library, the deployment packaging, or whatever things you want to group together. This is a different point of view, and may or may not conform to the MVC view. If this it doesn't, you may consider making two diagrams, one showing the MVC view and one showing your specific package view.


问:
Is functional programming actually used to create applications?

Possible Duplicate:

  What are some well known applications written in F#?

I see a lot of people talking about how cool functional programming is, how awesome Lisp and Haskell and F# are, etc, but what I don't see is them actually being used.  Just about every program on my computer is written in either something in the C family, Delphi, or Python.  I see tutorials talking about how easy it is to use functional languages to do complicated math problems, but no one talking about using them to do things most people actually care about using computers for, like business applications and games.

Does anyone have any examples of actual programs that people have heard of and are using, written in a functional language?  The only one I can think of off the top of my head is Abuse, from almost 15 years ago.  (Things like Emacs, where the core program is written in C with a functional language scripting layer on top, don't count.)

答: Xmonad, a windowing manager, is written in Haskell.</p>


问:
Which design pattern is more suitable for logging?
I should log some events in a program but as far as I know it would be better to keep the logging code outside the program because it is not about the real functionality of the program. So may you tell me if I should keep it completely out of code and only use Observers and Listeners to log the events? Or I can add a line of code like the following wherever I need to log something:

```<code>MyGloriousLogger.getXXXLogger().Log(LogPlace, new LogObject(z1, z2, z3, z4, ..., z99));
</code>```

Do I make a mistake to use Observer design pattern? I need another design pattern? Or I should stop thinking about design patterns?

PS1. If I want to log using only listeners and observers I will certainly need to add and improve the observers and listeners of the program.

PS2. I certainly know that there are different libraries for logging in Java and I'm using java.utils.logging but I need to have a wrapper for it to log my special objects.

答:
Use Aspect Oriented Programming which uses After, Before and Around advices over methods.
There, as per your need, you can add logs before start of api, after or on some conditions and also separate your main code from logging code.</p>

OwnerDisplayName: yogx


答:
Well, Observer sounds unfit to me. Also, tossing logger calls "wherever you need" will shatter your code and violate the SRP.

You might - for example - be interested in AOP for this, so you can attach logger calls via method annotations.

Title: Which design pattern is more suitable for logging?

Body:
I should log some events in a program but as far as I know it would be better to keep the logging code outside the program because it is not about the real functionality of the program. So may you tell me if I should keep it completely out of code and only use Observers and Listeners to log the events? Or I can add a line of code like the following wherever I need to log something:

```<code>MyGloriousLogger.getXXXLogger().Log(LogPlace, new LogObject(z1, z2, z3, z4, ..., z99));
</code>```

Do I make a mistake to use Observer design pattern? I need another design pattern? Or I should stop thinking about design patterns?

PS1. If I want to log using only listeners and observers I will certainly need to add and improve the observers and listeners of the program.

PS2. I certainly know that there are different libraries for logging in Java and I'm using java.utils.logging but I need to have a wrapper for it to log my special objects.

OwnerDisplayName: Powerslave


答: Logging is usually implemented with the Chain of responsibility pattern. Of course you can (and I would) combine that with a Facade. I really wouldn't use Listener(s) or Observer(s) myself.

OwnerDisplayName: Elliott Frisch


问:
Pointers for transitioning from SOAP to REST based services
We currently have a SOAP based web service which is being used by various teams as the primary data source. We eventually want to transition to REST based services. We are reluctant to do this now as the clients would have to do changes. </p>

<p>What can we do to make this transition smooth? </p>

答:
Don't replace, add.

When you achieve feature parity, stop making updates to the SOAP API and declare it deprecated.


答: Use a database for data collection purposes. A kind of production pipeline of data flow. Application level data, user level data, session level data (same page in two tabs). A NoSQL database would do nicely, but a classical database might be more known and faster.


答:
(link to the book)

This is a pretty good book, although it deals not with software architecture in general, but with architecture of business applications.

Title: Best books on the theory and practice of software architecture?

Body:
I have a couple of developers at my company who wish to move from programming into architecture. What are the best books out there on the theory and practice of software architecture? Include a cover picture if you can.

Feel free to include general books, and also books that relate to a specific technology.


问:
MongoDB: Is this a good schema design?
I'm working on a Meteor app which lets users create events and assign them crew members. I've played around with MongoDB before and made some experiences along the way, where for instance my first attempt went overboard with embedded objects and quickly became sluggish and hard to maintain. Several people therefore suggested to try a more traditional design approach and that's what I've done this time.

I've split my collections up with manual refs in mind,

Events

```<code>_id : ObjectID,
propA: "Some prop",
propB: "Some other prop",
employees: [
    {employee_id: ObjectID, crew_id: ObjectID},
    {employee_id: ObjectID, crew_id: ObjectID},
]
</code>```

Employees

```<code>_id : ObjectID,
propA: "Some prop",
crew_id: ObjectID
</code>```

Crews

```<code>_id : ObjectID,
propA: "Some prop"
</code>```

The reasoning is that the events collection will become quite substantial during the applications life cycle and thus I consider duplication to be a bad thing.
Another noteworthy feature is that an employee must belong to a certain crew, but on an actual event can work in any crew.

Intially I was quite happy with this design, but I have however found some problems which has lead me here.

1) When listing an event I have to manually join refs in order to show the actual properties of the employees and crews (such as name), it's a bit tedious but what worries me most is performance: Rather then running one query I have to run three (first get the event, then loop trough the employee array and then join employee with the Employee collection and finally join the crew with the Crew collection).

2) My biggest problem so far though is what I believe is known as deep querying. Before adding an employee to an event I have to know if he/she is already assigned, thus requiring to query the Events collection for an Event with a given ID and also find check if the given Employee ID exists in the employee array. So far I've been unable to do this in a single query.

So my question boils down to this. Is my design fine as it is or should I change it given my use cases? Or alternatively, if I keep my current design how should I tackle problem 1 and 2 (that is, manual joins and refs are fine. But is performance a real possible showstopper down the line?).

答:
Why are you using a non-relational database in this case?

You said:

When listing an event I have to manually join refs in order to show the actual properties of the employees and crews (such as name), it's a bit tedious but what worries me most is performance:

and

My biggest problem so far though is what I believe is known as deep querying

[emphasis added]

You already have some strong indicators that a non-relational database is not the right fit for your needs.  Use a relational database instead.  Changing the schema in this case, won't compensate for issues within the toolset.

Try your design out with a relational database and see if the issues go away.  That will provide the proof you need that you haven't picked the right tool to use in this case.

In a comment, you added that your stuck with this particular environment and are seeking alternative schema designs.

From what I have seen, the best way to compensate in this case is to denormalize all of your tables.  Instead of 3 tables, you'll have 1 table where you lay out the crew and the employees per event.

That does raise some challenges with making sure you don't double assign employees, but having a status flag for the event should help control for that.  Essentially you'll query to see if employee ABC is already assigned to an event (or crew) that is still active.

Title: MongoDB: Is this a good schema design?


问:
Program unable to write data (read-only file) and losing it -- a bug or a feature?
I found, that some very famous app (I'm not disclosing name intentionally) has a "problem", when writing meta-data of a file and the file turns out to be read-only. Program doesn't handle that too gracefully.

It does not check, whether a file is read only upon opening meta data dialog box. Thus, it allows the user to fill out (quite long) form, enter quite large amount of data without even checking if it will be able to save it. It only checks, if the file is read-only, when saving meta-data (attempting to do so). And if it finds that the file is actually read-only, it only displays a warning to a user and... closes the dialog box, thus causing user to lose data.

For me, this is an obvious bug, that it is beyond any discussion.

Since creator of that program, is my e-friend, I've immediately contacted him, writing (as usual) "Hey, just spotted another bug in your program". His reply simply shocked me, as he claimed and started to convince me, that this is not a bug, it can be only treated as an additional feature, that user may want (talking about my proposition to add some warning, that file is read-only and thus writing meta-data is impossible).

He even went as far (I think) as writing:

read-only file is a user problem, not a bug in a program

It was Saturday, a working and tiring day for me. I thought to myself, that maybe there isn't enough hot coffee in my blood system, so I contacted some of the developers, that I've been working with on a various projects. All the replies were the same: "A certain bug, nothing to discuss further".

I replied to that app's author and my friend, giving him a dumbest example I could come with:

Imagine yourself, that you are writing something in Word, save the document, Word tells you, that it can't write file, because it is read-only and... closes the document, losing everything you wrote since last save. Would you called that an extra feature and no data loss? (...)

And you call it a feature and my problem? I know many people ready to crucify you for such feature! :)

In the reply to that, I heard, that the comparison is not very good, because I don't write in meta-data so many text like in a Word document, so the "loss" is irrelevant. I'm completely lost, with both hands down!

As I said, for me this is a certain bug beyond any discussion. But maybe I slept over some radical revolution in the way, how we define a bug, so maybe someone would kindly enlighten me about that. Who of us two is right, and if it is program's author -- then why?

答:
It depends on your point of view.  Here are a couple articles that discuss the topic.  Developers see bugs as a mistake they made when translating requirements into code.  Users also see deficiencies in the requirements as bugs.  Developers get defensive when you say they made a coding mistake when really the mistake was a missing requirement, even if they're the ones who should have noticed the missing requirement.

The real answer is it doesn't matter what you call it (aside from contractual obligations).  The user's experience with the product is ultimately what matters.


问:
Is using os.environ to store the value of command line switches pythonic?
I write CLI-executable Python 3 scripts regularly to do data compilation and some maintenance tasks, and try to adhere to PEP 8 and the Google Python Style Guide when doing so. Google's guide says to avoid using global variables, while PEP 8 offers naming conventions but doesn't really encourage or discourage use of them, so I don't use them.

However, I've lately found myself using a "verbose" option/switch a lot along with a "printv" function to use while debugging. It usually looks like this:

```<code>def printv(opts, msg, **kwargs):
    if opts.verbose: print(msg, **kwargs)
</code>```

As incredibly lazy as I am, I'd like to make it so I don't need to pass the opts argument (which is an OptionParser object) to printv. The simplest method would be to use a global variable, sure. Another thought I had would be to set a "VERBOSE" variable in os.environ, since the switch is, conceptually, making a change to the execution environment. Obviously it's counter-intuitive, but is it pythonic?

答:
IMHO the level of visibility where you grab your options from should typically match the level of visibility of your printv function. So when there is just one global printv function visible everywhere in your program, it is ok to have the "verbose option" also visible globally. If your printv is instead a class member (for example, of a "Logger" class, see comment from DXM), the "verbose option" should be a member of the same class. If it is a function within a module - same rule.  This feels somewhat natural to me, however I hardly believe that this is a python specific idiom.

Using an environment variable is appropriate for controlling the "verbose option" from outside the program, this makes no sense if the "verbose state" is only switched on and off inside your program. If you need this (for example, because you have two or more programs which shall be controlled by the same VERBOSE environment variable), read the environment once after the program start and store the state in your internal globalOpts.verbose variable (for later usage, for example, in your printv function).


问:
How to store collections of linked Text objects in database
I'm writing a Java program that works as a presentation of a written document, which needs to be stored in a database (I'm using MySQL, but am open to suggestions of other DB types).

The Essay class is a collection of objects of the Paragraph class (more specifically Essay extends LinkedList<Paragraph>, which consists primarily of an int ID and a String content. The Essay object is stored as its own table in the DB, which is fairly straightforward (with columns int paragraphId and Blob text), and I do a simple query that creates a new Paragraph object for each row in the table when the program gets initialized.

Within each paragraph, certain hyperlinked words link to other Essay objects, which also need to be stored in the database. My question is how best to save them - the options I'm considering are:

Creating a single additional DB table where each essay is stored on a row, with paragraphs being stored in a single text object separated by a delimiter, and the program separates them into separate Paragraph object as it pulls them from the DB. (this is fairly manageable because these linked essays are much shorter than the main essay).

Create a single additional table, with a large number of content columns, storing each paragraph in a separate column. Then, when the data is pulled from the database, it creates a new essay for each row and a new paragraph for each column in that row, until it comes upon one that is null.

Dynamically create a separate DB table for each linked essay, setting it up identically to the primary one. This doesn't seem ideal because it would end up being a very large number of tables, and many of them would only have two or three paragraphs, making a new table seem wasteful.

Do a different arrangement altogether: Have one Paragraphs table, where all paragraphs from all essays are stored, and another table called Essays, which uses a SET datatype to store the IDs (foreign keys) of the paragraphs included in each essay.

I've modified my actual use case here to make it more simple to explain. I'm new to database programming, and am doing this largely as an exercise, so I'm looking for general tips about the pros/cons of these approaches, as well as any additional suggestions.

答:
There should be a table for each type of entity: Essay and Paragraph. I'm assuming the essays have some external identifier that the hyperlinks are referencing which I'm just going to call essay_name for this example. It sounds like you also need an attribute to indicate which is the "primary" essay. In pseudo-DDL:

CREATE TABLE Essay (
    essay_id INT NOT NULL,
    essay_name TEXT -- ...,
    is_primary BOOLEAN NOT NULL,
    PRIMARY KEY (essay_id)
)
```

<p>A Paragraph has a <em>many-to-one</em> relationship with an Essay (via the <code>essay_id</code> column). It also needs to have some way to represent the ordering of paragraphs within an essay. The following example uses <code>position</code> to capture the ordering of paragraphs.</p>

<pre class="lang-sql prettyprint-override"><code>CREATE TABLE Paragraph (
    paragraph_id INT NOT NULL,
    essay_id INT NOT NULL,
    position INT NOT NULL,
    content TEXT,
    PRIMARY KEY (paragraph_id),
    FOREIGN KEY (essay_id) REFERENCES Essay (essay_id)
)
</code>```

Body:
There should be a table for each type of entity: Essay and Paragraph. I'm assuming the essays have some external identifier that the hyperlinks are referencing which I'm just going to call essay_name for this example. It sounds like you also need an attribute to indicate which is the "primary" essay. In pseudo-DDL:

CREATE TABLE Essay (
    essay_id INT NOT NULL,
    essay_name TEXT -- ...,
    is_primary BOOLEAN NOT NULL,
    PRIMARY KEY (essay_id)
)

A Paragraph has a many-to-one relationship with an Essay (via the essay_id column). It also needs to have some way to represent the ordering of paragraphs within an essay. The following example uses position to capture the ordering of paragraphs.

CREATE TABLE Paragraph (
    paragraph_id INT NOT NULL,
    essay_id INT NOT NULL,
    position INT NOT NULL,
    content TEXT,
    PRIMARY KEY (paragraph_id),
    FOREIGN KEY (essay_id) REFERENCES Essay (essay_id)
)


问:
Build times for small incremental changes to C# Web Applications
I have recently been moving away from ASP.NET Websites in favor of Web Applications. More specifically I have recently been picking up MVC as an alternative to developing ASP.NET Forms websites.

Something that I find highly frustrating is the constant build->change->re-build process when testing small changes to compiled code. I find myself 'bundling' several changes between builds just to avoid waiting 20-25 seconds for Visual Studio to crunch it's way through the compilation process. I created a clean MVC 4 project from the VS template and timed the build after making a single line code change, it took 19 seconds.

When making a change in the code-behind in an ASP.NET website I've always found that the page is seemingly re-compiled on the fly when you next hit the page. This would seem to me to be a better environment for web development rather than VS forcing an entire rebuild after every small code change.

In the past I have enjoyed the luxury of making a change, hitting refresh in the browser, and waiting 1-2 seconds for my result. Is this achievable with web applications built using Visual Studio?

答:
It seems I had overlooked the usefulness of the "Edit and Continue" option (Tools -> Options -> Debugging -> Edit and Continue) after struggling to get past the infernal dialog message:

Changes are not allowed in the following cases:

When the debugger has been attached to an already running process.

The code being debugged was optimized at build or run time.

The assembly being debugged is loaded as domain-neutral.

The assembly being debugged was loaded through reflection.

Despite having the option checked in VS options you have to enable it for each separate web application. In the Properties Page it's on the Web tab when you scroll right to the bottom "Enable Edit and Continue".

Making small changes using Edit and Continue isn't exactly the same as making code-behind changes to a Website project, however, it's far more palatable than restarting the app for every code change.


问:
Choosing an ubiquitous language across different bounded contexts
If my domain has several Bounded Contexts, but only ONE team will work on all contexts, should I develop an Ubiquitous language for each context? or should I have only one and force to into all contexts?

The bounded context definition from Evan's book states:

A BOUNDED
  CONTEXT delimits the applicability of a particular model so that team
  members have a clear and shared understanding of  what has to be
  consistent and how it relates to other CONTEXTS. Within that CONTEXT,
  work to keep the model logically unified,  but do not worry about
  applicability outside those bounds. In other CONTEXTS, other models
  apply, with differences in terminology,  in concepts and rules, and in
  dialects of the UBIQUITOUS LANGUAGE.

I don't understand what is meant by "dialects of the UBIQUITOUS LANGUAGE". Should I develop a universal ubiquitous languge then modify it for each bounded context?

My main problem is if a single team is going  to work on all context they might get confused by the constant change in the terminology.

UPDATE:

Let's use an example to illustrate the problem. If I have 2 bounded contexts Operations and CustomerService and an entity Order.

A customer may request a refund. In the Operations context this is called a refund while in the CustomerService context it is called a cancellation. In my models I gonna have something like order.refund() or order.cancel().

The question is should I have 2 models for the order entity one with a method called refund() while in the other context a method called cancel()? or should I force a single terminology?

The implementation of the refund process might the same or different.

答: The purpose of the ubiquitous language is to use terminology that both your customer and the programmer can understand.  The terms that describe business entities must be terms that  apply to that particular business.  Consequently, it follows that the ubiquitous language is going to change somewhat with the specific problem domain being worked on.</p>


答: If I understood correctly, you should be cautious with the ambiguities. You need to create a Context Map, see the "Example 1: Same term, different meaning".</p>

Title: Choosing an ubiquitous language across different bounded contexts

Body:
If my domain has several Bounded Contexts, but only ONE team will work on all contexts, should I develop an Ubiquitous language for each context? or should I have only one and force to into all contexts?

The bounded context definition from Evan's book states:

A BOUNDED
  CONTEXT delimits the applicability of a particular model so that team
  members have a clear and shared understanding of  what has to be
  consistent and how it relates to other CONTEXTS. Within that CONTEXT,
  work to keep the model logically unified,  but do not worry about
  applicability outside those bounds. In other CONTEXTS, other models
  apply, with differences in terminology,  in concepts and rules, and in
  dialects of the UBIQUITOUS LANGUAGE.

I don't understand what is meant by "dialects of the UBIQUITOUS LANGUAGE". Should I develop a universal ubiquitous languge then modify it for each bounded context?

My main problem is if a single team is going  to work on all context they might get confused by the constant change in the terminology.

UPDATE:

Let's use an example to illustrate the problem. If I have 2 bounded contexts Operations and CustomerService and an entity Order.

A customer may request a refund. In the Operations context this is called a refund while in the CustomerService context it is called a cancellation. In my models I gonna have something like order.refund() or order.cancel().

The question is should I have 2 models for the order entity one with a method called refund() while in the other context a method called cancel()? or should I force a single terminology?

The implementation of the refund process might the same or different.


问:
Pros and cons of namespaces vs include/require in PHP?
I recently started using namespaces in PHP. When I first saw them, I thought that their syntax was ugly and I'd never use them. However, I created an autoloader (spl_autoload_register) that makes it so that I never have to write an include/require statement again.

I like namespaces, but is there any objective benefit over include/require statements, or are they the exact same method to accomplish the same goal?

答:
Namespaces are not just for autoloading classes. They also prevent naming conflicts. In fact, that's their primary purpose.

Say you have a project that needs a class named User, to store info about users of your application, but a plugin also uses a (different) class named User to store information. Namespaces let you create your class within one namespace (say, MyApp) and let the plugin use another namespace (say, CoolPlugin). Code within the MyApp space can just refer to User (e.g., new User();), and so can code in the CoolPlugin space; each will get the expected result. When you need to use code from another namespace, you just prefix it. For example, code in the CoolPlugin space can access the User class in MyApp via new \MyApp\User();

The alternative is that every class needs a complex name everywhere, such as class MyApp_User and class CoolPlugin_User. Namespaces let you simplify things most of the time and avoid naming conflicts all of the time.

Edit: To answer the question, "Is there any performance difference between the two?"

Not a meaningful one, no. I haven't benchmarked it, but there is probably a difference on the nanosecond level. That said, sacrificing code quality for super-small performance tweaks is not a good strategy, so you should use namespaces regardless. For benchmarks of similar kinds of problems, see PHPbench.com and this StackOverflow answer.

Your code needs to be incredibly tight and incredibly time-sensitive (think high-frequency trading or managing nuclear reactions) before you need to worry about micro-optimizing it in this kind of context. If it really is that time-sensitive, you should probably be coding in c or even assembly, not interpreted languages like PHP.


问:
Entity Framework Entities - Some Data From Web Service - Best Architecture?
We are currently using Entity Framework as an ORM across a few web applications, and until now, it has suited us well as all our data is stored in a single database. We are using the repository pattern, and have services (the domain layer) which use these, and return the EF entities directly to the ASP.NET MVC controllers.

However, a requirement has come up to utilise a 3rd party API (through a web service) which will give us extra information that relates to the user in our database. In our local User database, we will store an external ID which we can provide the to API to get additional information. There is quite a bit of information available, but for the sake of simplicity, one of them relates to the user's company (name, manager, room, job title, location etc). This information will be used in various places throughout our web apps - as opposed to being used in a single place.

So my question is, where is the best place to populate and access this information? As it is used in various places, it's not really sensible to fetch it on an ad-hoc basis wherever we use in the web application - so it makes sense to return this additional data from the domain layer.

My initial thought was just to create a wrapper model class which would contain the EF entity (EFUser), and a new 'ApiUser' class containing the new information - and when we get a user, we get the EFUser, and then get the additional info from the API, and populate the ApiUser object. However, whilst this would be fine for getting single users, it falls over when getting multiple users. We can't hit the API when getting a list of users.

My second thought was just to add a singleton method to the EFUser entity which returns the ApiUser, and just populate it when needed. This solves the above problem as we only access it when we need it.

Or the final thought was to keep a local copy of the data in our database, and synchronise it with the API when the user logs in. This is minimal work as it's just a synchronisation process - and we don't have the overhead of hitting the DB and API every time we want to get user information. However, these means storing the data in two places, and also means the data is out of date for any user that hasn't logged in for a while.

Does anyone have any advice or suggestions on how best to handle this kind of scenario?

答:
With proper separation of concerns, nothing above the Entity Framework / API level should even realize where the data is coming from.  Unless the API call is expensive (in terms of time or processing), accessing data that uses it should be as transparent as accessing data from the database.

The best way to implement this, then, would be to add extra properties to the EFUser object which lazy-load the API data as needed.  Something like this:

```<code>partial class EFUser
{
    private APIUser _apiUser;
    private APIUser ApiUser
    {
       get {
          if (_apiUser == null) _apiUser = API.LoadUser(this.ExternalID);
          return _apiUser;
       }
    }
    public string CompanyName { get { return ApiUser.UserCompanyName; } }
    public string JobTitle{ get { return ApiUser.UserJobTitle; } }
}
</code>```

Externally, the first time either CompanyName or JobTitle is used there will be a single API call made (and thus a small delay), but all subsequent calls until the object is destroyed will be just as fast and easy as database access.

Body:
With proper separation of concerns, nothing above the Entity Framework / API level should even realize where the data is coming from.  Unless the API call is expensive (in terms of time or processing), accessing data that uses it should be as transparent as accessing data from the database.

The best way to implement this, then, would be to add extra properties to the EFUser object which lazy-load the API data as needed.  Something like this:

partial class EFUser
{
    private APIUser _apiUser;
    private APIUser ApiUser
    {
       get {
          if (_apiUser == null) _apiUser = API.LoadUser(this.ExternalID);
          return _apiUser;
       }
    }
    public string CompanyName { get { return ApiUser.UserCompanyName; } }
    public string JobTitle{ get { return ApiUser.UserJobTitle; } }
}

Externally, the first time either CompanyName or JobTitle is used there will be a single API call made (and thus a small delay), but all subsequent calls until the object is destroyed will be just as fast and easy as database access.


答:
In Semantic Processing, automatically generated annotations are created that relate to a particular block of text.  Annotations would flavour the context of individual words or phrases within the block of text.  The phrases were tagged with ideas like 'city' or 'government department', or 'medical field'.  While this is not exactly what you're doing, the idea is the same - how do you relate additional information associated with a block of text.

The blocks of text were stored in a single table with an autogenerated id for each row.  The annotations were stored in a separate table that referenced the id.  The fields it had were: major_type, minor_type, start, end, and value.  Start and end were numbers that indicated character numbers within the block of text.  "The desktop was large" would annotate desktop as a 'physical object', 'furniture', 4, 10, 'desktop'.  It used the character values to reference the annotated details.  While it might not be ideal, it's one way a real system did something similar.

If I were you, I'd strongly consider a markup language to modify the text to include the links.  The character reference seems easy to break.  It really depends on your data - how often it is updated, or how much control you have over it.

Hope this helped!

Title: How to store collections of linked Text objects in database

Body:
I'm writing a Java program that works as a presentation of a written document, which needs to be stored in a database (I'm using MySQL, but am open to suggestions of other DB types).

The Essay class is a collection of objects of the Paragraph class (more specifically Essay extends LinkedList<Paragraph>, which consists primarily of an int ID and a String content. The Essay object is stored as its own table in the DB, which is fairly straightforward (with columns int paragraphId and Blob text), and I do a simple query that creates a new Paragraph object for each row in the table when the program gets initialized.

Within each paragraph, certain hyperlinked words link to other Essay objects, which also need to be stored in the database. My question is how best to save them - the options I'm considering are:

Creating a single additional DB table where each essay is stored on a row, with paragraphs being stored in a single text object separated by a delimiter, and the program separates them into separate Paragraph object as it pulls them from the DB. (this is fairly manageable because these linked essays are much shorter than the main essay).

Create a single additional table, with a large number of content columns, storing each paragraph in a separate column. Then, when the data is pulled from the database, it creates a new essay for each row and a new paragraph for each column in that row, until it comes upon one that is null.

Dynamically create a separate DB table for each linked essay, setting it up identically to the primary one. This doesn't seem ideal because it would end up being a very large number of tables, and many of them would only have two or three paragraphs, making a new table seem wasteful.

Do a different arrangement altogether: Have one Paragraphs table, where all paragraphs from all essays are stored, and another table called Essays, which uses a SET datatype to store the IDs (foreign keys) of the paragraphs included in each essay.

I've modified my actual use case here to make it more simple to explain. I'm new to database programming, and am doing this largely as an exercise, so I'm looking for general tips about the pros/cons of these approaches, as well as any additional suggestions.


答:
I agree with GlenH7 answer - you probably want a relational DB.

That said:

1) When listing an event I have to manually join refs in order to show
  the actual properties of the employees and crews (such as name) [...]
  what worries me most is performance: [...]

Consider caching employees and crews - you won't have that many active, and that should solve the performance issue.

2) Before adding an employee to an event I have to know if
  he/she is already assigned, thus requiring to query the Events
  collection for an Event with a given ID and also find check if the
  given Employee ID exists in the employee array.

You could actually use some 'normalization' here:

Events:

```<code>_id : ObjectID,
propA: "Some prop",
propB: "Some other prop",
</code>```

Events-Employees:

```<code>_id : ObjectID,
eventId: "_id of event",
employeeId: "_id of employee",
event_start: "denormalized event data for querying employee availability",
</code>```

That's two queries to retrieve full event information (assuming employee/crew cache), but a single query to see who's available at a given time. You can even make that a covered query as an index with two ids and dates should remain small(ish).

Title: MongoDB: Is this a good schema design?

Body:
I'm working on a Meteor app which lets users create events and assign them crew members. I've played around with MongoDB before and made some experiences along the way, where for instance my first attempt went overboard with embedded objects and quickly became sluggish and hard to maintain. Several people therefore suggested to try a more traditional design approach and that's what I've done this time.

I've split my collections up with manual refs in mind,

Events

```<code>_id : ObjectID,
propA: "Some prop",
propB: "Some other prop",
employees: [
    {employee_id: ObjectID, crew_id: ObjectID},
    {employee_id: ObjectID, crew_id: ObjectID},
]
</code>```

Employees

```<code>_id : ObjectID,
propA: "Some prop",
crew_id: ObjectID
</code>```

Crews

```<code>_id : ObjectID,
propA: "Some prop"
</code>```

The reasoning is that the events collection will become quite substantial during the applications life cycle and thus I consider duplication to be a bad thing.
Another noteworthy feature is that an employee must belong to a certain crew, but on an actual event can work in any crew.

Intially I was quite happy with this design, but I have however found some problems which has lead me here.

1) When listing an event I have to manually join refs in order to show the actual properties of the employees and crews (such as name), it's a bit tedious but what worries me most is performance: Rather then running one query I have to run three (first get the event, then loop trough the employee array and then join employee with the Employee collection and finally join the crew with the Crew collection).

2) My biggest problem so far though is what I believe is known as deep querying. Before adding an employee to an event I have to know if he/she is already assigned, thus requiring to query the Events collection for an Event with a given ID and also find check if the given Employee ID exists in the employee array. So far I've been unable to do this in a single query.

So my question boils down to this. Is my design fine as it is or should I change it given my use cases? Or alternatively, if I keep my current design how should I tackle problem 1 and 2 (that is, manual joins and refs are fine. But is performance a real possible showstopper down the line?).

Body:
I agree with GlenH7 answer - you probably want a relational DB.

That said:

1) When listing an event I have to manually join refs in order to show
  the actual properties of the employees and crews (such as name) [...]
  what worries me most is performance: [...]

Consider caching employees and crews - you won't have that many active, and that should solve the performance issue.

2) Before adding an employee to an event I have to know if
  he/she is already assigned, thus requiring to query the Events
  collection for an Event with a given ID and also find check if the
  given Employee ID exists in the employee array.

You could actually use some 'normalization' here:

Events:

_id : ObjectID,
propA: "Some prop",
propB: "Some other prop",

Events-Employees:

_id : ObjectID,
eventId: "_id of event",
employeeId: "_id of employee",
event_start: "denormalized event data for querying employee availability",

That's two queries to retrieve full event information (assuming employee/crew cache), but a single query to see who's available at a given time. You can even make that a covered query as an index with two ids and dates should remain small(ish).


问:
Plain old struct vs class in this case?
Stroustrup says that you should use structs unless you can specify an invariant for the data structure. I have to represent a physical object which holds things such as position, velocity, mass and inertia (and so on). I also want to render the object, for which I will need setup in the constructor and unloading stuff in the destructor. So I guess that's an invariant, but it feels wrong to put all this physics stuff into it that doesn't really have invariants.

The drawing parts are pretty trivial, so it doesn't feel worth it to separate them from the rigid body in this case.

Any advice?

edit: I suppose I just realized the answer to my own question. Rigid bodies do have invariants, namely that it has to follow the physical laws.

答:
You could of course separate your concerns.

Hold position, velocity, mass, etc in a struct and create another class such as NewtonianObject which contains an instance of your struct and applies Newtonian physics to the values. This would give you the freedom to later create a NonNewtonianObject that applies a different set of physical laws to the same type of data.

Title: Plain old struct vs class in this case?


问:
Should Repositories return IQueryable?
I have been seeing a lot of projects that have repositories that return instances of IQueryable. This allows additional filters and sorting can be performed on the IQueryable by other code, which translates to different SQL being generated. I am curious where this pattern came from and whether it is a good idea.

My biggest concern is that an IQueryable is a promise to hit the database some time later, when it is enumerated. This means that an error would be thrown outside of the repository. This could mean an Entity Framework exception is thrown in a different layer of the application.

I have also run into issues with Multiple Active Result Sets (MARS) in the past (especially when using transactions) and this approach sounds like it would lead to this happening more often.

I have always called AsEnumerable or ToArray at the end of each of my LINQ expressions to make sure the database is hit before leaving the repository code.

I am wondering if returning IQueryable could be useful as a building block for a data layer. I have seen some pretty extravagant code with one repository calling another repository to build an even bigger IQueryable.

答:
Exposing IQueryable to public interfaces is not a good practice.
The reason is fundamental: you cannot provide IQueryable realisation as it is said to be.

Public interface is a contract between provider and clients. And in most cases there is an expectation of full implementation. IQueryable is a cheat:

IQueryable is nearly impossible to implement. And currently there is no proper solution.
Even Entity Framework has lots of UnsupportedExceptions.

Today Queryable providers have large dependance on infrastructure. Sharepoint has its own partial implementation and My SQL provider has distinct partial implementation.

Repository pattern gives us clear representation by layering, and, most important, independence of realisation. So we can change in future data source.

The question is "Should there be possibility of data source substitution?".

If tomorrow part of the data can be moved to SAP services, NoSQL data base, or simply to text files, can we guaranty proper implementation of IQueryable interface?

(more good points about why this is bad practice)

Title: Should Repositories return IQueryable?


问:
How to make the leap from classification to clustering
I have a clustering problem which I can't seem to solve, although if I treat it as a labeled classification problem, I can solve it with satisfactory precision. Is there an elegant way to make the leap from being able to solve the classification problem, to being able to solve the clustering one?

The details:

I have a labeled dataset of movements, from which I extract a features vector for each movement. I can then perform classification of the movements with several tools (SVM, decision trees, etc.) and I get satisfactory results (0.85 precision, which in my case is ok).

The problem is, that the real world data I am facing, will of course not be labeled, and is not modeled after the labeled data I have (which means I can't train a classifier on the labeled data, and expect it to perform well on other data). The real world data will for sure not be comprised just of these two classes, it will be comprised of K unknown classes, which I would like to be able to cluster with satisfactory precision.

So is it logical to assume that if I can solve the classification problem with high precision, I will be able to solve the clustering problem with high precision? Does it mean that my feature extraction, which works well for the classification, will work well for clustering?

Because just throwing clustering algorithms at the problem (k-means, meanshift, dbscan, etc.) doesn't give good results. Usually I just get too many clusters, or just 1 cluster (although I know I have 2 classes), with horrible precision.

Btw I am using Python's sklearn.

答:
If you have only one or two feature variables, I would hesitate to recommend clustering methods, because you may even not know there will be clusters or how many clusters. I would rather try kernel density estimation on the data to observe the peaks of probability density. And it is not difficult to write a python code with SciPy package (the example code is shown on that page too).

If you have much more features, which makes your problem more complicated, you can try clustering methods. And you can estimate the number of clusters from the data by "one-added-at-a-time" and "split-the-cluster" strategy (called xmeans), or brute-force on k to determine the optimal number of cluster based on sum-of-squares. Refer to this post with similar topic.

OwnerDisplayName: Oren Solomianik


问:
High performance computing and interconnects
High performance computing means different things to different people. From a pure programming perspective what I gather is that these are loads of individual machines connected by extremely fast interconnect not necessarily based on usual TCP/IP.

My question is on the interconnect side of things: How do you make interconnects faster without changing your socket based code much?

答:
You can use TCP over just about any high-performance network. OpenFabrics (a software stack for RDMA networks like InfiniBand and iWARP) has the "sockets direct protocol". Further, 10 GigE in all its forms was built specifically to provide compatibility with legacy socket code.

So feel free to use TCP on your high-performance network. It won't have the absolute best latency advantages compared to, say, using the underlying verbs. But it should be fast enough for most customers' needs.


答:
You need to use a customized clustering approach that integrates your requirements and domain knowledge.

DBSCAN is quite flexible, at least if you have a good implementation. But for your problem at hand you really need to define what "similar" means. Euclidean distance won't do the trick, and when Euclidean distance doesn't work, no clustering using euclidean distance will work. So first make sure, you have the appropriate similarity measure (and this may mean implementing one yourself!)

Title: How to make the leap from classification to clustering

Body:
I have a clustering problem which I can't seem to solve, although if I treat it as a labeled classification problem, I can solve it with satisfactory precision. Is there an elegant way to make the leap from being able to solve the classification problem, to being able to solve the clustering one?

The details:

I have a labeled dataset of movements, from which I extract a features vector for each movement. I can then perform classification of the movements with several tools (SVM, decision trees, etc.) and I get satisfactory results (0.85 precision, which in my case is ok).

The problem is, that the real world data I am facing, will of course not be labeled, and is not modeled after the labeled data I have (which means I can't train a classifier on the labeled data, and expect it to perform well on other data). The real world data will for sure not be comprised just of these two classes, it will be comprised of K unknown classes, which I would like to be able to cluster with satisfactory precision.

So is it logical to assume that if I can solve the classification problem with high precision, I will be able to solve the clustering problem with high precision? Does it mean that my feature extraction, which works well for the classification, will work well for clustering?

Because just throwing clustering algorithms at the problem (k-means, meanshift, dbscan, etc.) doesn't give good results. Usually I just get too many clusters, or just 1 cluster (although I know I have 2 classes), with horrible precision.

Btw I am using Python's sklearn.

OwnerDisplayName: Anony-Mousse


问:
Is there a standard for encoding recurring patterns of dates?
I'm trying to re-write a backend interface doohickey for a calendar, a todo list and another sort of calendar and found that they all were programmed to have subtly different formats for patterns of dates that could be used to configure certain properties in them.

I have three screens that have different ways of essentially doing the same thing:

and they essentially have the same information.  I've got objects for all these, but what I'd like to do is serialize the objects, not to bytecode, but to a semi-human readable format that could be transferred between not only these three screens, but stored as one field in a database and XML.

答:
You should be able to use Cron notation for this. It originated for scheduling jobs within Unix, but there are now many libraries for different languages to convert to and from it.

Essentially, Cron stores date/time intervals in 5 (text) columns representing Years, Months, Days, Hours and Minutes.

Using a combination of * (meaning every day, hour minute etc), numbers representing repetitions and special characters to represent more specialized scenarios you should be able to store any kind of date information.

More: http://en.wikipedia.org/wiki/Cron


问:
GUI programming - who should be an owner
I know nothing about gui programming. I try to learn. I have simple question...

For example, I have small program. I have class that extends JFrame with main Window (MyWindowClass) and my "logic" class (MyClass) that do something (for example calculate something and I want to display it in window).

Should I implement this like:

```<code>class MyClass {
    // ...
    // ...
    MyWindowClass window;
};
</code>```

Or like this:

```<code>class MyWindowClass extends JFrame {
   // ...
   MyClass c;
};
</code>```

I don't know which class should be an owner and which class should be have main method...

Please help me...

I try to write in Java (Swing) but this is general question I guess.

答:
Between your two options having the view know about your "logic" class would be the better method.  This would allow you to create or modify your view without having to change your logic classes.  If you put your view inside of your "logic" classes it dramatically limits your flexibility.

As people mentioned in the comments read up on the MVC pattern as that will demonstrate some ideal solutions.

OwnerDisplayName: jzd


问:
Algorithm for fast tag search
The problem is the following.

There's a set of simple entities E, each one having a set of tags T attached.
Each entity might have an arbitrary number of tags.
Total number of entities is near 100 million, and the total number of tags is about 5000.

So the initial data is something like this:

```<code>E1 - T1, T2, T3, ... Tn
E2 - T1, T5, T100, ... Tk
..
Ez - T10, T12, ... Tl
</code>```

This initial data is quite rarely updated.

Somehow my app generates a logical expression on tags like this:

T1&T2&T3 | (T5&!T6)

What I need to is to calculate a number of entities matching given expression (note - not the entities, but just the number). This one might be not totally accurate, of course.

What I've got now is a simple in-memory table lookup, giving me a 5-10 seconds execution time on a single thread.

I'm curious, is there any efficient way to handle this stuff? What approach would you recommend? Is there some common algorithms or data structures for this?

Update

A bit of clarification as requested.

T objects are actually relatively short constant strings. But it doesn't actually matter - we can always assign some IDs and operate on integers.

We definitely can sort them.

答:
After having written this answer, I should probably flag the question as “too broad”  – we can talk for ages about various strategies, in the end a benchmark will have to be run with your data.

Each tag can be efficiently represented by an integer. Each entity has a set of tags. Choosing the correct set implementation is important – both B-trees and sorted arrays are possible. With this set, we will only do membership tests. As both structures do this in O(log t) (with t tags per entity), I would prefer arrays due to their denser representation.

We can now filter through all entities in an O(n · log t · p) operation, where p is the average path length in the predicate decision tree. This decision tree can be ordered so that a decision can be reached quickly. Without statistical data, it is only possible to factor out common subexpression.

The order in which the entities are searched is not really important. On the other hand it can be advantageous to sort it such that the entities at indices 0 to i all have a certain tag, while the rest doesn't. This reduces the n when searching for this specific tag (in the decision tree, this should be the first test). This can be expanded to multiple levels, but this complicates things and takes O(2k) memory with k levels. With multiple levels, the tags with the highest gain should be decided first, where the gain is the number of entities that don't have to be looked up times the probability of discarding them. The gain becomes maximal for 50:50 chances or when 50% of the entities have this specific tag. This would allow you to optimize even if the access patterns are not known.

You could also create sets that index the entities by each tag used – one set with all entities for T1, the next for T2. An obvious (space and time) optimization is to stop when a set contains more than half of all elements, and to save those elements that don't have this tag  – this way, building indices for all tags will take less than ½ · n · t space (with t tags in total). Note that saving complementary sets can make other optimizations more difficult. Again, I would (sorted) arrays for the sets.

If you also represent your entities through an integer range, you can compress the space used for the index sets by only storing the starting and ending member of a continuous range. Implementation-wise this would probably be done with a high bit to indicate whether an entry is a range bound or regular entry.

If we now have index sets (and thus statistics on the tags) we can optimize our predicates so that unlikely properties are tested first (fail-fast strategy). This means that if T1 is common and T2 is rare, then the predicate T1 & T2 should be evaluated by iterating through all T2 index set entries and testing each element for T1.

If we use sorted arrays to implement the index sets, then many evaluation steps can be implemented as merging operations. T1 & T2 means that we take the T1 and T2 lists, allocate a target array the size of the larger inputs and perform the following algorithm until both inputs are empty: If T1[0] < T2[0], then T1++ (discard the head). If T1[0] > T2[0] then T2++. If both heads are equal, then copy that number over to the target array, and increment all three pointers (T1, T2, target). If the predicate is T1 | T2, then no elements are discarded but the smaller one copied over. A predicate of the form T1 & ¬T2 can also be implemented using a merging strategy, but ¬T1 or T1 | ¬T2 can't.

This should be considered when ordering the predicate decision tree: Complements should either occur on the RHS of an &, or at the end, when the final count is being determined and the actual elements don't have to be looked at.

Without using index sets, each thread can filter through its part of the entities and return the count of elements that match the predicate, which can be summed up. When using index sets, then each thread would be assigned one node in the decision tree. It takes two input streams which correspond to the ordered sets, and return a merged stream. Notice that each node in the decision tree has a corresponding set that represents all entities which fulfill that sub-expression, and that due to the ordering of the sets, it isn't necessary to know the whole set at once in order to merge them.

Different strategies like merging indexed sets or filtering through a list of entities can be combined to a certain degree. Filtering has very predictable performance. If a query is very specific so that the usage of index sets reduces the search space drastically, then merge operations could be better. It is important to note that merging many large input sets can result in much worse performance than brute-force searching. A very optimized algorithm will pick a suitable strategy based on the input size, query structure and statistical indicators.

As an aside, caching partial results can be beneficial if it is expected that similar queries will be run in the future, even though they don't speed up the inital run.

Title: Algorithm for fast tag search


答:
Ideally, wouldn't have either your logic class or your window class contain the other, but would instead use a "application" object to contain them both.

However, if for some reason that isn't practical, the choice of which object contains the other when both objects are from entirely different areas of concern should be determined by which object is dependent upon the other.  If MyWindowClass will never run without a MyClass object, it may contain such as a member; if MyClass will never run without MyWindowClass, it may contain such instead.

(Although this should only be a concern when the overall application design doesn't allow for proper separation, which is an instance that occurs less and less frequently as languages and frameworks have evolved from the bad old days.)

As an aside, if you are forced into cross-concern composition, make sure that you keep the references as generic as practicable.  You want to be able to substitute a MyModifiedClass for MyClass in MyWindowClass with a minimuim of fuss down the road.

Title: GUI programming - who should be an owner

Body:
I know nothing about gui programming. I try to learn. I have simple question...

For example, I have small program. I have class that extends JFrame with main Window (MyWindowClass) and my "logic" class (MyClass) that do something (for example calculate something and I want to display it in window).

Should I implement this like:

```<code>class MyClass {
    // ...
    // ...
    MyWindowClass window;
};
</code>```

Or like this:

```<code>class MyWindowClass extends JFrame {
   // ...
   MyClass c;
};
</code>```

I don't know which class should be an owner and which class should be have main method...

Please help me...

I try to write in Java (Swing) but this is general question I guess.


答:
Classical Model-View-Controller question.

MyClass is a model. It has the domain data and implements domain logic.

SWING components (JTextField, JTable) are a view. They know how to show data to the user and how to accept user input, but know nothing about your domain model.

MyWindowClass is a controller. It needs to mediate your model and view - e.g. set text field value to object property value or update object in response to button press.

Hence, MyWindowClass needs to be able to access both the model and the view. There is little reason for MyClass to know about the UI - e.g. you may use the same domain class in different UI (e.g. list of objects and edit form for a single object).


答:
When Stroustrup is talking about invariants, he is not in the first place talking about the rules that apply to the real world objects that our data structures represent, but about the rules that are required to safeguard your design.

For instance, if you have a flat array of objects, that should be kept sorted because your class assumes the list is sorted when doing binary search lookups, the fact that the list is sorted is an invariant that is logically guarded by the class.

In order to make sure your class will always work correctly, you can make sure nobody outside the class has direct access to the list, and that inserts/removals keep the list sorted.

If you make a struct in C++, by default all fields are exposed as public. If you have constraints on how the fields are modified, such a struct can be sabotaged: For instance, a struct that has a pointer to a refcounted object, should not expose it's pointer field to the outside: Users of such structs could set the pointer to null without decreasing it's refcount, or vice versa, causing memory leaks or corrupted memory.

A lot of time, I choose invariants for my own convenience: For instance, I might choose that a View always has non-null pointer to a valid Model. I can ensure this by preventing that the Model property on the view can not be removed from the outside, and I add a check to the constructor that the Model passed in is non-null, and I throw an exception otherwise. Then I can be sure that I always have a valid Model in my View, so I never need to handle the case that the Model pointer in the view is null.

Back to your example: Invariants can be thought of as assumptions that the code depends on elsewhere. So if your calculations assume a non-negative mass, you can guard this requirement by not letting users of your object class set the mass directly, and check all values passed in.

It could very well be that your code will happily work with negative mass objects, except that they will behave weirdly. Then guarding this aspect at the object level may not add any real value, so you could consider leaving it out.

Title: Plain old struct vs class in this case?

Body:
Stroustrup says that you should use structs unless you can specify an invariant for the data structure. I have to represent a physical object which holds things such as position, velocity, mass and inertia (and so on). I also want to render the object, for which I will need setup in the constructor and unloading stuff in the destructor. So I guess that's an invariant, but it feels wrong to put all this physics stuff into it that doesn't really have invariants.

The drawing parts are pretty trivial, so it doesn't feel worth it to separate them from the rigid body in this case.

Any advice?

edit: I suppose I just realized the answer to my own question. Rigid bodies do have invariants, namely that it has to follow the physical laws.


答:
i would do this in sql having a link table EntityCategory between eid referencing entity and  cid referencing category using self-joins:

```<code>    select count(ec1.eid)
    from EntityCategory ec1
    left join EntityCategory ec2 on ec1.eid=ec2.eid
    left join EntityCategory ec3 on ec1.eid=ec3.eid
    ...
    where
      ec1.cid={categoryId1} and
      ec2.cid={categoryId2} and
      ec3.cid={categoryId3} ...
</code>```

Title: Algorithm for fast tag search

Body:
The problem is the following.

There's a set of simple entities E, each one having a set of tags T attached.
Each entity might have an arbitrary number of tags.
Total number of entities is near 100 million, and the total number of tags is about 5000.

So the initial data is something like this:

```<code>E1 - T1, T2, T3, ... Tn
E2 - T1, T5, T100, ... Tk
..
Ez - T10, T12, ... Tl
</code>```

This initial data is quite rarely updated.

Somehow my app generates a logical expression on tags like this:

T1&T2&T3 | (T5&!T6)

What I need to is to calculate a number of entities matching given expression (note - not the entities, but just the number). This one might be not totally accurate, of course.

What I've got now is a simple in-memory table lookup, giving me a 5-10 seconds execution time on a single thread.

I'm curious, is there any efficient way to handle this stuff? What approach would you recommend? Is there some common algorithms or data structures for this?

Update

A bit of clarification as requested.

T objects are actually relatively short constant strings. But it doesn't actually matter - we can always assign some IDs and operate on integers.

We definitely can sort them.

Body:
i would do this in sql having a link table EntityCategory between eid referencing entity and  cid referencing category using self-joins:

    select count(ec1.eid)
    from EntityCategory ec1
    left join EntityCategory ec2 on ec1.eid=ec2.eid
    left join EntityCategory ec3 on ec1.eid=ec3.eid
    ...
    where
      ec1.cid={categoryId1} and
      ec2.cid={categoryId2} and
      ec3.cid={categoryId3} ...


问:
Licensing libraries (which do code generation, etc) and GPL boundaries
I want to really opensource a project that did start as a personal hobby, but I'm an ignorant of licensing details.

This project, provides some libraries for a given $language, and some wrapper command line utilities for those libraries.

Those libraries, across many things, do generate more code in the same language, code which can be reused by the caller.

On my drafts, I always used a MIT style license for the sake of brevity, but it's unreleased copies I'm reviewing right now to merge on a finally released version.

I would like to use GPL style policies for usage/contribution of my code (lets call it a framework).

But.

Could choosing GPLv2 (3?) for my project, limit the things that a given user can do with the code generated or managed by my libraries/utilities? (i.e. commercial profit of such generated code without release her improvements/changes to my code)

Is there something to consider (in a plain language) when licensing code that generates code ?

Update:

To try to answer some comments:

What do you wish to accomplish with your license?

The best goals for the project health (from the view point of a community-based opensource project).

Who do you want to use your code?

Anybody

What do you want to happen to changes?

To be back-ported to the project as much as possible.

What do you want to happen to generated code?

To don't be affected by my licensing choice. It belongs to the user who generated it.

What about money - if the user makes money do you want that to affect the licensing?

If the user makes money from the generated code, great. If the user makes money modifying my code, I would like to force to publish the changes.

I did a group-therapy with those answers. Now, choosing GPL (2? 3?) I could be ok or not?

Update2

So they have to bundle/compile your library in with theirs when they deploy it, in order for it to run?

yes, the library needs to be installed previously, or bundled with the result (there is an option for this).

答:
Disclaimer - IANAL and the following is not a legal advice. The exact detail may vary depending on your jurisdiction as well.

The interpretation of FSF, which tend to be the strictes, is that the generated code is not covered by license. So you can use gcc to compile a commercial project and distribute the binary. Similarly using a GPL implementation of System API is allowed. However the linking with library is prohibited. I know that not every layer agree with latter interpretation but I don't know the details (the main point is what's a 'derived project') - if you want to know the details you might want to consult a layer. In such case bundling library would move toward 'derived project' (again IANAL).

If you don't want to limit in such way the general practice is use LGPL, GPL with linking exception or any other copyleft license. The latter is more liberal as the former require the third party to be able to relink with different version of library (again - if the clauses are enforceable is debated). In general I'd advice to stick to well know licenses such as (L)GPL, MIT/BSD3, Apache 2.0 etc.

However license is just giving permission. If you publish code on GPL you can republish it under different license as long as you are the copyright holder or have a license to do so (so, for example, if you take the contribution from third party you need to ask for permission). You can even sell a license to some company as was a monetization model of some FLOSS projects.

Final remainder - I am not a layer and this was not a legal advice.