问:
Improve the speed of a Tomcat Server with MySQL
I'm developing an application with 3 sides, desktop (JavaFX), server (Tomcat) and Android.

And a MySQL Database Server.

The app receive mails and convert them into tasks. The tasks are sent to the android devices from the desktop app. The user of the Android device do stuff with the task and send the task with new information to the server.

The desktop app takes the data from the database.

Now it's working, but goes too slowly from the desktop side, due to the server possibly having a big payload.

The server has 3 tasks:

JavaMail email reader, and write in the database.

Send PUSH messages to the Android devices and receive some information from them, and write in the database.

Receive the GPS position from the Android devices, and write it in the database.

My doubts are:

A)Separate the server side in 3 projects.

Mail reader

Send PUSH to the Android devices.

GPS position receptor.

B)Separate the database in 2 databases:

For the GPS position.

For the info of the tasks.

This steps will make the app faster?

Edit to answer  Clinton Bosch:

I´m using Hibernate to connect to the database, here is my hibernate config file:

```<code><?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
    <session-factory>

        <property name="hibernate.c3p0.min_size">2</property>
        <property name="hibernate.c3p0.max_size">150</property>
        <property name="hibernate.c3p0.timeout">100</property>
        <property name="hibernate.c3p0.max_statements">0</property>

        <property name='connection.username'>root</property>

        <!-- Database connection settings -->
        <property name='connection.driver_class'>com.mysql.jdbc.Driver</property>

        <!-- JDBC connection pool (use the built-in) -->
        <property name='connection.pool_size'>10</property>

        <!-- SQL dialect -->
        <property name='dialect'>org.hibernate.dialect.MySQLDialect</property>

        <!-- Echo all executed SQL to stdout -->
        <property name='show_sql'>false</property>

        <!-- Mapping files -->
    </session-factory>
</hibernate-configuration>
</code>```

答:
Keep the db writes async, so when user sends gps or anything else, make a thread safe POJO with all the details, set it to a class that implements Runnable and put it in to a queue.

Natural question is what happens if server fails? Question from me is : how important is that? If a few 100 inserts fails will the users lose money? Stop using the app? if no then let it be!

You can then push jobs in to a queue of java managed threads using a thread pool executor.


问:
continuous integration with Xcode
I'm looking to setup CI.  I only own one Mac.  I do have a copy of OS X 4.0 Server (free with my Apple Developer Program license).  I currently use Git for source control.  I would prefer something with tight Xcode and Git integration.  I will buy a second Mac with Yosemite if I have to.

Do I have an option that will allow me to avoid buying more hardware?  E.G. Virtualization, 3rd party etc...

Update

I tried Travis-CI and sadly this will not work as my project is based on IOS8 Swift.  Travis-CI currently does not support swift.

答:
If you're using GitHub as your central Git repository, you could take a look at Travis CI.  It's a cloud-based CI system, so you don't have to manage the server yourself.  I'm thinking of trying this out for some of my open-source projects.

If you want to stick with Apple's tools all the way, try out Bots.  It ships as part of Xcode Server.  I've been considering using this but haven't got around to setting it up yet.  It's obviously very tightly integrated with Xcode and Git, which is fantastic if you're doing Apple-ony work, but as soon as you decide to branch out onto other platforms you'll find that you have to maintain two CI systems.

I've used TeamCity in the past.  Compared to Jenkins it's very easy to set up and manage.

Lastly, you could take a look at Jenkins.  From what I remember Jenkins is far more complex to install than it should be, but I admittedly didn't try terribly hard to get it working.

Up until fairly recently I ran my own CI system.  I had a VirtualBox virtual machine running OSX Mountain Lion with an install of TeamCity.  I encountered issues where the combination of Java (TeamCity and Jenkins both require it) and Xcode exhausted the virtualized RAM and caused the VM to completely lock up.  If you have an abundance of RAM on the host machine that shouldn't be an issue, but it was a little annoying for me.

ContentLicense: CC BY-SA 3.0

Title: continuous integration with Xcode


问:
Should the async update-server be on the same physical server as the website?
I want to setup a comet/web-socket server for a forum sitting in front of a message queue i.e. rabbitmq, that will maintain client connections and update them about relevant events (i.e. new posts, topics, etc.)

The popular way to go about this is to use some async server that does this in one thread and hands off the blocking tasks to other worker threads. My question is, should this async update-server be on the same physical server as the website? In other words, the task of maintaining client connections and updating them as needed should be separated from the task of creating and returning dynamic web-pages right? Should they be on the same server or separated completely?

The criteria for evaluating the decision is performance and ease of scalability.

Also, this question isn't about what specific platforms to use. I'm asking about the general practice of seperating concerns: Serving dynamic webpages vs updates; when is it best to place them together and when is it best to seperate them? (physically on the server as well as in code base)

答:
You could use NodeJS and the node package module Socket.io for this. Separating the tasks of client connections and creating/returning dynamic web pages into node modules could work the way you described. Node describes their modules in their API docs.

The book NodeJS in Action has some projects that perform similar functionality (even the basic ChatApp in Chapter two performs this to some degree.


问:
Creating/updating entity save strategy
Please note: although I'm talking about Java in this question, what I'm asking about here should really be language-agnostic.

I am dabbling in OR/M for the first time, and have cooked up the following "update strategy" for all of my entities:

All entity POJOs extend an abstract base class BaseEntity that has a Long id property, for that entity's unique id/primary key

Applications are free to create instances of an entity, but all constructors/factory methods omit the setting of this id property

Hence if Person extends BaseEntity, then I can write Person p = new Person(...) but p.getId() is NULL.

Only when you call the entity's DAO method does the entity get saved to persistence and is given an id.

My DAOs will only have a save* creational method (such as savePerson(Person)), and they will do a check for the id being NULL or not to determine whether we should attempt to INSERT a new entity record, or UPDATE an existing entity record.

My DAO save* methods will always return an instance of the entity that was just created/updated, but will make sure that the id is now properly set (if it wasn't before).

To me this is logical/sensible, but seems a bit complicated/convoluted. Am I going about this the wrong way, are there better "update patterns" for OR/M that take care of this for me?  I plan on doing everything in raw JDBC at first, and then eventually migrating to Hibernate once I'm more comfortable.

答:
It sounds/looks good, I would just offer these suggestions:

Might want to include a timestamp in the BaseEntity to resolve conflicts.  For example, what if two people are editing the record at the same time.  You can then develop conflict resolution strategy based on this value (last one in wins, etc.)

Ideally every table should have a natural key to check for uniqueness.  But the surrogate key works well.  Just bear in mind if it's a surrogate key it should always come from your application not the client or caller.  Don't expect the information from the client or caller to always be accurate.  This is where natural keys help out.

Title: Creating/updating entity save strategy


问:
Escape a GameLogic god class.
When programming a simple game as an example. I always tend to get left with a large GameLogic class somewhere which is reponsible for gluing together all of the other portions of the software.

A Common example of the GameLogic class I talk about contains functions such as these.

```<code>startNewGame();
showAd();
prepareLevel();
startLevel();
gameComplete();
restartLevel()
gameOver();
</code>```

This class can become very large.

Is there any advice for how better to structure my future games? Should I be splitting these functions out into dedicated classes somehow as to reduce the complexity of the god class?

答:
Personally, I like the idea of stages, scenes and actors a lot, even for small projects. That way, you can always add new classes and just plop them in, and they act as individual objects that can be thought of as such.

By creating classes for all of your rendered items like so:

Stage (MainMenu)

Scene (MenuNavigator)

Actor (MenuItem)

Actor (MenuItem)

Actor (MenuItem)

Actor (MenuItem)

Stage (InGame)

Scene (Gameplay)

Actor (StarUnderlay)

Actor (Ship)

Actor (Enemy)

Scene (ScoreOverlay)

Actor (Text)

Actor (Text)

Actor (Text)

Scene (GameOver)

Actor (Text)

Actor (MenuItem)

You get a world of benefits.

By classing out your entities, you'll have reusability (Such as MenuItem, Text, and creating enemies that use a common class or anonymous classes). This means that you'll be rewriting less code and sharing logic across all objects.

By classing out scenes, you can also reuse this code, such as having an InGame Stage that uses a ScoreOverlay, as well as an InReplay Stage that also uses that score display, where the data is filled in by the stage.

By classing out your stages, you can put your core logic there as it is where your user is currently at in the game. A MainMenu stage can swap out for an InGame stage, and just by swapping out the object being rendered, you won't have to worry about deallocating the right objects as you'll either have garbage collection or the ability to request child objects be deallocated.

The idea is to reduce the responsibility both now and in the future of all of your classes, and thus you end up with individual units working together to produce a final product. If one thing breaks, you're more likely to know where and why, and you won't be digging through a 800 line file, but maybe a 100 line file.

When you go to pull it together, you can think of it as if you're putting on a play (hence why stages, scenes and actors are common used wordings). You need a monster to appear? Then you add a new actor to portray that monster. When that monster is no longer needed, it's pulled out of the scene.

Yes, you will likely end up with a good amount of classes, but a lot of people will argue that 100 classes is better than 1,000 lines, because again, you'll always know what's what when concepts are separated.

Title: Escape a GameLogic god class.


问:
moving around virtual keyboard

I'm  creating  virtual keyboard using javascript/html/css. And I  can't  find a good algorithm for  moving from one button  to another in  different directions (top-bottom, left-right). Lets say  I have focus set to any of the buttons - how should I  find which  button  should  be focused next.  any  thoughs and  ideas would  be  appreciated.

Here is  code structure  sample

row with functional  keys

```<code><div class="keyboard-row">
        ENG
        <div class="charRow">
             1

             2

             3

             4

             5

             6

             7

             8

             9

             0

             _
        </div>
        Search
    </div>
</code>```

row only  with  chars

```<code><div class="keyboard-row">
        <div class="charRow">
             a

             s

             d

             f

             g

             h

             j

             k

             l
        </div>
    </div>
</code>```

Also layout is formed dynamically based on language file config. So rows that have  letter characters can have different  number of  keys  depending  on language  selected.

Functianal keys are  static  for all languages.  Language setting  is saved in a javascript object

```<code>var langObj = {
......
  row3: [
        {
            value:"q"
        },{

            value:"w"
        },{

            value:"e"
        },....
],
row 4 :[...].....
}
</code>```

where each row  property responds to keyboard row layout

I've added "data-index" attribute  to each key in a row as suggested by   Kilian Foth so now with jquery I  can move horizontally. code is next(might need some  improvements)

```<code>   function moveLeft(e){
            e.preventDefault();
            var $el = $(e.target),
                $parentRow = $el.parents(".keyboard-row");

            if(!$el.is($("a:first", $parentRow ))){
                var nextIndex = parseInt($el.attr("data-index")) - 1
                $parentRow.find("[data-index='" + nextIndex + "']").focus();
            }
            else $parentRow.find("a:last").focus();
        }
</code>```

Now the main problem is moving verticaly. I  would suggest finding coordinates of currently focused  button and  then figuring out which  button vertically intersects by Y-coordinates with the current one and if it even exists. But dont  know how to do it yet

答:
While it would result is much more code, there's nothing inherently wrong with adding "nextLeft, nextDown, etc." to your keys. You'll be able to 100% ensure which key is next when navigating, so all of the guess work is gone.

If you insist on a solution that uses solving, give each key an x offset and width, then when down is pressed, move to the key whose center is closest. You may not even have to define that information if it's already being laid out visually in the DOM, there's already properties for that.

Title: moving around virtual keyboard

Body:
I'm  creating  virtual keyboard using javascript/html/css. And I  can't  find a good algorithm for  moving from one button  to another in  different directions (top-bottom, left-right). Lets say  I have focus set to any of the buttons - how should I  find which  button  should  be focused next.  any  thoughs and  ideas would  be  appreciated.

Here is  code structure  sample

row with functional  keys

```<code><div class="keyboard-row">
        <a href="#" id="langSwitch" class="largeKey" lang-code="en" data-index="0">ENG</a>
        <div class="charRow">
             <a href="#" class="key" id="key2" data-index="1">1</a>

             <a href="#" class="key" id="key3" data-index="2">2</a>

             <a href="#" class="key" id="key4" data-index="3">3</a>

             <a href="#" class="key" id="key5" data-index="4">4</a>

             <a href="#" class="key" id="key6" data-index="5">5</a>

             <a href="#" class="key" id="key7" data-index="6">6</a>

             <a href="#" class="key" id="key8" data-index="7">7</a>

             <a href="#" class="key" id="key9" data-index="8">8</a>

             <a href="#" class="key" id="key10" data-index="9">9</a>

             <a href="#" class="key" id="key11" data-index="10">0</a>

             <a href="#" class="key" id="key12" data-index="11">_</a>
        </div>
        <a href="#" id="enter" class="largeKey" data-index="12">Search</a>
    </div>
</code>```

row only  with  chars

```<code><div class="keyboard-row">
        <div class="charRow">
             <a href="#" class="key" id="key1" data-index="0">a</a>

             <a href="#" class="key" id="key2" data-index="1">s</a>

             <a href="#" class="key" id="key3" data-index="2">d</a>

             <a href="#" class="key" id="key4" data-index="3">f</a>

             <a href="#" class="key" id="key5" data-index="4">g</a>

             <a href="#" class="key" id="key6" data-index="5">h</a>

             <a href="#" class="key" id="key7" data-index="6">j</a>

             <a href="#" class="key" id="key8" data-index="7">k</a>

             <a href="#" class="key" id="key9" data-index="8">l</a>
        </div>
    </div>
</code>```

Also layout is formed dynamically based on language file config. So rows that have  letter characters can have different  number of  keys  depending  on language  selected.

Functianal keys are  static  for all languages.  Language setting  is saved in a javascript object

```<code>var langObj = {
......
  row3: [
        {
            value:"q"
        },{

            value:"w"
        },{

            value:"e"
        },....
],
row 4 :[...].....
}
</code>```

where each row  property responds to keyboard row layout

I've added "data-index" attribute  to each key in a row as suggested by   Kilian Foth so now with jquery I  can move horizontally. code is next(might need some  improvements)

```<code>   function moveLeft(e){
            e.preventDefault();
            var $el = $(e.target),
                $parentRow = $el.parents(".keyboard-row");

            if(!$el.is($("a:first", $parentRow ))){
                var nextIndex = parseInt($el.attr("data-index")) - 1
                $parentRow.find("[data-index='" + nextIndex + "']").focus();
            }
            else $parentRow.find("a:last").focus();
        }
</code>```

Now the main problem is moving verticaly. I  would suggest finding coordinates of currently focused  button and  then figuring out which  button vertically intersects by Y-coordinates with the current one and if it even exists. But dont  know how to do it yet


答:
I use https://github.com/centrifugal/centrifuge, it is a standalone application that uses websockets and falls back to long polling if that fails. Unlike centrifuge, I had lots of issues with comet with high volumes, it did not scale well for various reasons.

Centrifuge even has a dockerfile so you could literally spin up the server on a server in a few seconds. Your web front end requires a tiny bit of javascript and server side is also really simple to push messages to

Title: Should the async update-server be on the same physical server as the website?

Body:
I want to setup a comet/web-socket server for a forum sitting in front of a message queue i.e. rabbitmq, that will maintain client connections and update them about relevant events (i.e. new posts, topics, etc.)

The popular way to go about this is to use some async server that does this in one thread and hands off the blocking tasks to other worker threads. My question is, should this async update-server be on the same physical server as the website? In other words, the task of maintaining client connections and updating them as needed should be separated from the task of creating and returning dynamic web-pages right? Should they be on the same server or separated completely?

The criteria for evaluating the decision is performance and ease of scalability.

Also, this question isn't about what specific platforms to use. I'm asking about the general practice of seperating concerns: Serving dynamic webpages vs updates; when is it best to place them together and when is it best to seperate them? (physically on the server as well as in code base)


问:
A separate endpoint specifically for sync in a REST api
I am working on a multi client/single server app. Communication between the two happens via RESTful API. There are several resources that need to stay in sync across all clients, including images captured from camera, likes and comments on these images, user profiles and user followers/followings. My current approach involves GETing all resources changed on server since last sync, like GET /images?since=<last-sync-timestamp>, GET /users/:id/followers?since=<last-sync-timestamp> etc. and POSTing all client changes since last sync to the same endpoints, like POST /images, POST /user/:id/followers

The main drawback to this approach is way too many network calls - For each resource I need to sync, I need to perform at least 2 network calls.

So I am thinking about introducing a new endpoint like /sync which provides all syncable resources that have changed since last sync timestamp. Both client and server will need to parse the JSON and put the respective data in appropriate tables, but now, no matter how many resources I have to sync, I will only need to perform two network calls, but change the structure of the JSON as reosurces get added/removed.

Any thoughts on why the second approach is superior/inferior to the first one?

答:
How much difference in application performance or server overhead is having one network call rather than two per update going to make? Well, it depends on how many simultaneous clients you expect, and how often they'll be polling for updates. If you have a "small number of clients" like 10 or 50, and they request updates every N seconds (say N between 10 and 120), you can do the math and find it's not a substantial network or server load, so you'd probably be wasting your time optimizing it.

On the other hand, if you had 100s, 1Ks, or 1Ms of simultaneous clients, and/or they're requesting updates every second or few seconds, and/or you have not just two kinds of objects to communicate but M>2, by all means do everything possible to optimize the interactions.

This still leaves open a key question: Is adding new mega-REST endpoint (/sync) really the best way to do that?

Maybe not. Cormac Mulhall outlined some of the downsides. A more subtle problem is that sync is fundamentally hard. It seems not so difficult, but a lot of shops have repeatedly broken their picks on it.

Why is sync hard? Well, let's just scratch that surface. You mention URLs specifying ?since=<last-sync-timestamp>. Is that the server's timestamp, or the client's? They will be different. Distributed timekeeping is itself hard. Clocks vary. You can help this along by adding another rest call /time that helps the client estimate how far its clock is off the server's clock. But now you've got the complexity of time drift estimation, just like inside Network Time Protocol. REST isn't so simple any more. And just what does a timestamp mean, anyway? ?since=1418070480 means...? Is it the start of that second, or the end? Keep in mind that CPUs can execute ~10**11 instructions per second. Seconds are a crude metric. So you say: "You know what? There's going to be error, so I'll just put in a margin for error." You coordinate clocks as best you can, then add another 20 or 30 or 60 seconds back, just to catch stragglers. Now you've think you've got the "accidentally missing updates" problem licked--you haven't really, in a system of any complexity, but let's just pretend. But you've created another problem: The possibility of duplication, because you're asking for another N seconds of data before where you really think you need it. You'll have to build client-side code to make sure objects aren't double-posted. REST was supposed to be simple!

Even with all this, you haven't addressed the elephant in the room: You still have a polling-style update, where the clients have to constantly ask, ask, and re-ask: "Anything new? No? Okay. How about now? No? Okay. How about now? No? Okay...." Your server is still getting hit with needless requests. 90+% of which will always answer "Nothing new to report."

The truth is, you do not yet have a communication style that is even half-way suited to synchronization requirements. Polling is always going to be a chatty, server-loading technique. What you really need aren't AJAX or AJAJ update polls, but at a minimum Comet (long-polling reverse AJAX calls)--or even better, a full back-and-forth communication mechanism such as WebSockets. These allow your server-side code to establish push techniques or push-pull interactions that, while not automagically fixing all of the semantic difficulties of synchronization in a distributed world, substantially reduce the network call, server overhead, and code complexity required to have truly synchronized endpoints.

In summary, one mega endpoint rather than two individual ones is a minor issue. If you want efficient temporal synchronization, the real issue is you need a better interaction mechanism.


问:
Allow iteration of an internal vector without leaking the implementation
I have a class that represents a list of people.

```<code>class AddressBook
{
public:
  AddressBook();

private:
  std::vector<People> people;
}
</code>```

I want to allow clients to iterate over the vector of people. The first thought I had was simply:

```<code>std::vector<People> & getPeople { return people; }
</code>```

However, I do not want to leak the implementation details to the client. I may want to maintain certain invariants when the vector is modified, and I lose control over these invariants when I leak the implementation.

What's the best way to allow iteration without leaking the internals?

答:
If iteration is all you need, then perhaps a wrapper around std::for_each would suffice:

```<code>class AddressBook
{
public:
  AddressBook();

  template <class F>
  void for_each(F f) const
  {
    std::for_each(begin(people), end(people), f);
  }

private:
  std::vector<People> people;
};
</code>```

Body:
If iteration is all you need, then perhaps a wrapper around std::for_each would suffice:

class AddressBook
{
public:
  AddressBook();

  template <class F>
  void for_each(F f) const
  {
    std::for_each(begin(people), end(people), f);
  }

private:
  std::vector<People> people;
};


答:
One could provide member functions:

```<code>size_t Count() const
People& Get(size_t i)
</code>```

Which allow access without exposing implementation details (like contiguity) and use these within an iterator class:

```<code>class Iterator
{
    AddressBook* addressBook_;
    size_t index_;

public:
    Iterator(AddressBook& addressBook, size_t index=0)
    : addressBook_(&addressBook), index_(index) {}

    People& operator*()
    {
        return addressBook_->Get(index_);
    }

    Iterator& operator ++ ()
    {
       ++index_;
       return *this;
    }

    bool operator != (const Iterator& i) const
    {
        assert(addressBook_ == i.addressBook_);
        return index_ != i.index_;
    }
};
</code>```

Iterators can then be returned by the address book as follows:

```<code>AddressBook::Iterator AddressBook::begin()
{
    return Iterator(this);
}

AddressBook::Iterator AddressBook::end()
{
    return Iterator(this, Count());
}
</code>```

You'd probably need to flesh the iterator class out with traits etc but I think that this will do what you've asked.

Title: Allow iteration of an internal vector without leaking the implementation

Body:
I have a class that represents a list of people.

```<code>class AddressBook
{
public:
  AddressBook();

private:
  std::vector<People> people;
}
</code>```

I want to allow clients to iterate over the vector of people. The first thought I had was simply:

```<code>std::vector<People> & getPeople { return people; }
</code>```

However, I do not want to leak the implementation details to the client. I may want to maintain certain invariants when the vector is modified, and I lose control over these invariants when I leak the implementation.

What's the best way to allow iteration without leaking the internals?

Body:
One could provide member functions:

size_t Count() const
People& Get(size_t i)

Which allow access without exposing implementation details (like contiguity) and use these within an iterator class:

class Iterator
{
    AddressBook* addressBook_;
    size_t index_;

public:
    Iterator(AddressBook& addressBook, size_t index=0)
    : addressBook_(&addressBook), index_(index) {}

    People& operator*()
    {
        return addressBook_->Get(index_);
    }

    Iterator& operator ++ ()
    {
       ++index_;
       return *this;
    }

    bool operator != (const Iterator& i) const
    {
        assert(addressBook_ == i.addressBook_);
        return index_ != i.index_;
    }
};

Iterators can then be returned by the address book as follows:

AddressBook::Iterator AddressBook::begin()
{
    return Iterator(this);
}

AddressBook::Iterator AddressBook::end()
{
    return Iterator(this, Count());
}

You'd probably need to flesh the iterator class out with traits etc but I think that this will do what you've asked.


答:
Serving smaller and more numerous requests should not be so much of a risk to the server, unless you leave the synchronization as a user-triggered event.

See, when you leave the synchronization to the user, it is always going to be heavy, regardless of whether the requests are small but many, or few but big, you're always at risk of having too much network traffic to handle - users tend to hit the refresh button a lot more than necessary, especially if the app seems slow or unresponsive to the refresh event (which means, as soon as you have a heavy load, it will get even heavier).

But there are other ways. And I don't mean to tell you to put a timer on your app, and poll for changes every x seconds. I mean using something truly efficient, like push notifications.

Push notifications can be used to tell the client app that there are changes to be synched, and optionally, which exact resources need to be synched. This technique is called send-to-sync.

With send-to-sync, when something is changed in the server (by receiving a POST/PUT operation), you can make the server notify all outdated clients that they have to synchronize with that change. You can even get rid of the timestamp in your requests, and leave things as simple as possible in terms of REST resources, and also completely avoid having to account for time differences in the many devices.

Title: A separate endpoint specifically for sync in a REST api

Body:
I am working on a multi client/single server app. Communication between the two happens via RESTful API. There are several resources that need to stay in sync across all clients, including images captured from camera, likes and comments on these images, user profiles and user followers/followings. My current approach involves GETing all resources changed on server since last sync, like GET /images?since=<last-sync-timestamp>, GET /users/:id/followers?since=<last-sync-timestamp> etc. and POSTing all client changes since last sync to the same endpoints, like POST /images, POST /user/:id/followers

The main drawback to this approach is way too many network calls - For each resource I need to sync, I need to perform at least 2 network calls.

So I am thinking about introducing a new endpoint like /sync which provides all syncable resources that have changed since last sync timestamp. Both client and server will need to parse the JSON and put the respective data in appropriate tables, but now, no matter how many resources I have to sync, I will only need to perform two network calls, but change the structure of the JSON as reosurces get added/removed.

Any thoughts on why the second approach is superior/inferior to the first one?


问:
Why using string[] args in all main methods?
I'm a beginner in learning programming. I ask about using the string array in main method as a parameter. Why not writing the Main()  method without the string array? What is the point of having this array in Main method?

For example:

```<code>public static void Main(string[] args)
{
Console.WriteLine(args.Length);
Console.WriteLine(args[1]);
}
</code>```

答:
Console applications predate GUI applications, and these take command-line parameters for very long time (at least from CP/M time, which preceded MS-DOS, which preceded Windows, which preceded Windows NT — all preserving more or less the same logic).

You can pass command-line parameters to Windows GUI apps, too.

These are the args.


问:
Should you use Fields or just Properties in C#?
I've seen some people who make properties for every single member, private or not... does this make any sense?

```<code>private string mWhatever;

private string Whatever
{
    get
    {
        return this.mWhatever;
    }
    set
    {
        this.mWhatever = value;
    }
}
</code>```

I could see it making sense in 1% of the cases at times when you want to control access to the member inside the class containing it.

Because if you don't use properties for every member it would lead to inconsistencies and checking to see if the member has an access or not (since you have access to both in the scope of the class).

答:
I know this is an old question and everything @Yusubov said is correct, but with respect to his second bullet point about specific logic in the setter, and since I didn't see anyone specifically mention this, a perfect example would be when your class implements the INotifyPropertyChanged interface.

This is used a ton in WCF and Silverlight which allows you to know when a specific property has changed via the logic in its setter like in the class below:

```<code>public class Settings : INotifyPropertyChanged
{
    public Settings()
    {
        this.CountryField = String.Empty;
    }

    private string CountryField;
    public string Country
    {
        get { return this.CountryField; }
        set
        {
            if (Object.ReferenceEquals(this.CountryField, value)) { return; }

            this.CountryField = value;
            this.RaisePropertyChanged("Country");
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;
    private void RaisePropertyChanged(string propertyName)
    {
        PropertyChangedEventHandler propertyChanged = this.PropertyChanged;

        if (propertyChanged != null)
            propertyChanged(this, new PropertyChangedEventArgs(propertyName));
    }
}
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I know this is an old question and everything @Yusubov said is correct, but with respect to his second bullet point about specific logic in the setter, and since I didn't see anyone specifically mention this, a perfect example would be when your class implements the INotifyPropertyChanged interface.

This is used a ton in WCF and Silverlight which allows you to know when a specific property has changed via the logic in its setter like in the class below:

public class Settings : INotifyPropertyChanged
{
    public Settings()
    {
        this.CountryField = String.Empty;
    }

    private string CountryField;
    public string Country
    {
        get { return this.CountryField; }
        set
        {
            if (Object.ReferenceEquals(this.CountryField, value)) { return; }

            this.CountryField = value;
            this.RaisePropertyChanged("Country");
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;
    private void RaisePropertyChanged(string propertyName)
    {
        PropertyChangedEventHandler propertyChanged = this.PropertyChanged;

        if (propertyChanged != null)
            propertyChanged(this, new PropertyChangedEventArgs(propertyName));
    }
}


问:
Where should I store test data?
I have smaller unit tests that use small snippets from real data sets. I would also like to test my program against full data sets for a multitude of reasons. The only problem is that a single real dataset is about ~5GB. I haven't found any hard numbers for what Git repositories can store but that seems like too much.

(According to this Programmers post I should keep all of my data needed to test the project in the repository.)

The solution that my team has adopted is that the project has a file that contains a path to a network attached file system that holds our test data. The file is Git ignored.

I feel like this is an imperfect solution for two reasons. When the NAS isn't working, is slow, or is down than we can't run a full test. The second reason is that when someone first clones a repository the unit tests fail so they have to figure out how to mount things with a certain name and the syntax used to build the testing path file.

So my question is two fold. How much data is too much data to store in revision control?

What is a better way to handle large amounts of test data?

答:
How to handle large files in a build chain

I like to use a build tool that does dependency management - such as maven or gradle. The files are  stored in a web repository, and the tool takes care of downloading and caching automagically when it encounters the dependency. It also eliminates extra setup (NAS configuration) for people who want to run the test. And it makes refreshing the data fairly painless (it's versioned).

What's too big to put in revision control

There is a large gray area. And if you decide something doesn't belong in a RCS, what are your alternatives? It's an easier decision if you limit your choices between the RCS and a binary repo (maven style).

Ideally, you'd only want in the RCS stuff that is humanely editable, diffable, or where you'd want to track the history. Anything which is the product of a build or some other sort of automation definitely doesn't belong there. Size is a constraint, but not the main one - a giant source file (bad practice) definitely belongs in the source control. A tiny compiled binary doesn't.

Be ready to compromise for developer convenience.

Title: Where should I store test data?


问:
online database with partial access for clients
I'm trying to come up with an online database that my clients would have partial access to. This is for an artwork storage company. We have about 40 clients and would like each to have access to their collection. Is that possible? </p>

答:
Sure, you can create a database create an account on it and put it "out there" for the clients to have access to.  It is possible.

You will encounter things like "how many connections do you have?" and "can the admin log in from remote?" and "just how bad is it if one of the clients gets more access than they should have?"

Once its "out there" anyone can try to connect to it.  Having a database "out there" will be the subject to cracking attempts.  Data over the wire for databases isn't typically encrypted. Having it out there as a database also means that your clients are getting connections - possibly persistent ones and can accidentally denial of service themselves (or you).

All of this takes very good management from the DBA to make sure everything is set up properly.  That the table visibility or views or column visibility or connection information is all set up correctly. That long running queries (they ran a query that consumes how much CPU time?) are properly dealt with. That the log files are monitored for inappropriate access attempts. It is a fair bit of work.

People kind of tried to do this in the past with Web SQL but that kind of fizzled (and probably isn't too useful unless you are using SQLite as your backend).

What you would see more of today is writing a RESTful interface to the database and present that.  Don't give them full SQL access to the database. They probably don't need it.  But write an interface to the system that allows them to fetch what they need and not kill your own database performance at that time.

Opening access to a raw database access across the WAN is typically a 'bad thing' and should be avoided if possible.  Find out what your clients really need (not what they want) and build that.


答:
if you want exact implementation of functions from std::vector, use private inheritance as below and control what is exposed.

```<code>template <typename T>
class myvec : private std::vector<T>
{
public:
    using std::vector<T>::begin;
    using std::vector<T>::end;
    using std::vector<T>::push_back;
};
</code>```

Edit:
 This is not recomended if you also want to hide internal data structure i.e. std::vector

Body:
if you want exact implementation of functions from std::vector, use private inheritance as below and control what is exposed.

template <typename T>
class myvec : private std::vector<T>
{
public:
    using std::vector<T>::begin;
    using std::vector<T>::end;
    using std::vector<T>::push_back;
};

Edit:
 This is not recomended if you also want to hide internal data structure i.e. std::vector


问:
How is CoffeeScript influenced by Haskell?
I've been using CoffeeScript for a while now. On Wikipedia, it is said that CoffeeScript is influenced by Haskell. But after I check out the syntax of Haskell, I have found little resemblance from CoffeeScript.

Which aspect of CoffeeScript is influenced by Haskell?

答:
For me, CoffeeScript sometimes looks really similar to Erlang.

For example:

CoffeeScript:

```<code>f_qsort([]) -> []
f_qsort([Pivot, Rest...]) ->
  [
    f_qsort((X for X in Rest when X < Pivot))...,
    Pivot,
    f_qsort((Y for Y in Rest when Y >= Pivot))...
  ]
</code>```

Erlang:

```<code>qsort([]) -> [];
qsort([Pivot|Rest]) ->
  qsort([ X || X <- Rest, X < Pivot])
  ++ [Pivot] ++
  qsort([ Y || Y <- Rest, Y >= Pivot]).
</code>```

There is a library which provides a way to write Erlang-like constructions right in CoffeeScript without precompilation, just wrap up it in special function. You can find it here: https://github.com/nogizhopaboroda/f_context.

OwnerDisplayName: lightblade

Body:
For me, CoffeeScript sometimes looks really similar to Erlang.

For example:

CoffeeScript:

f_qsort([]) -> []
f_qsort([Pivot, Rest...]) ->
  [
    f_qsort((X for X in Rest when X < Pivot))...,
    Pivot,
    f_qsort((Y for Y in Rest when Y >= Pivot))...
  ]

Erlang:

qsort([]) -> [];
qsort([Pivot|Rest]) ->
  qsort([ X || X <- Rest, X < Pivot])
  ++ [Pivot] ++
  qsort([ Y || Y <- Rest, Y >= Pivot]).

There is a library which provides a way to write Erlang-like constructions right in CoffeeScript without precompilation, just wrap up it in special function. You can find it here: https://github.com/nogizhopaboroda/f_context.


问:
Historical precedent for why Prolog is less popular than SQL in Imperative Programming?
It seems that writing Declarative SQL is very popular in Imperative Programming. However, it also seems that writing Declarative Prolog could save a lot of complexity but this is not very common.

Is there a historical precedent for this apparent preference of SQL over Prolog?

If the reason is lack of native support by Imperative languages, then is it possible to answer why the language creators didn't find it useful to natively support Prolog in the first place?

To provide some specific examples:

Example 1

Evaluating a loan application might be just a few lines of code in Prolog, like the SELECT/JOIN query that is just a few lines of code in SQL, but it seems the advantage is not as obvious as SQL.

Example 2

Here is another example problem and the solution in Prolog. The following constraint logic program represents a simplified dataset of john's history as a teacher:

```<code>teaches(john, hardware, T) :- 1990 ≤ T, T < 1999.
teaches(john, software, T) :- 1999 ≤ T, T < 2005.
teaches(john, logic, T) :- 2005 ≤ T, T ≤ 2012.
rank(john, instructor, T) :- 1990 ≤ T, T < 2010.
rank(john, professor, T) :- 2010 ≤ T, T < 2014.
</code>```

The following goal clause queries the dataset to find out when john both taught logic and was a professor:

```<code>:- teaches(john, logic, T), rank(john, professor, T).
</code>```

Result:

```<code>2010 ≤ T, T ≤ 2012.
</code>```

In the above example it will be easy with SQL to get the same result. But suppose that you have this data in an Array.  Then it is not as easy to get the same results using SQL.  And in the case of data stored in an array, I believe that the Prolog code will be easier to write and maintain.

答:
One aspect not mentioned so far is the push for "open" systems in the 1980s and 1990s. In many places, software vendors would have to provide industry standard access to the data in their databases. At the time, SQL was an established standard which was well know and understood; Prolog was pretty esoteric and academic. Once you started getting interfaces like ODBC to easily connect systems, no-one was interested in looking at other technologies.

I worked at a place in the late 80s which had a quite successful ISAM database that was forced by market pressures/procurement regulations to add a SQL interface to.


问:
Enumerated types and their interpretation by compilers
It seems to me that a lot, if not most, compilers treat enumerated types as int underneath. In C/gcc, enums are compiled to int. In C#/Visual C#, you can change the underlying data type with something like:

```<code>enum Days : byte {Sat=1, Sun, Mon, Tue, Wed, Thu, Fri};
</code>```

but the default is int (second paragraph) if you don't change the type.

For obvious reasons, ints are efficient. They fly through the ALU with ease and avoid useless string comparisons.

Upon first glance, it doesn't seem that interpreted languages have enumerated types built-in (Ruby, Perl, JavaScript, others I'm sure). They can be emulated though; i.e. in JavaScript.

My questions:

Why do (seemingly) most compilers use int as the underlying type? Is the reason historical/performance-related?

Are there languages that compile enumerated types to something other than int (or byte, long, numerical types...)? If so, why did they choose to do it differently?

答:
The first part of your question will be difficult to answer because that decision was made long ago and one would need to ask the creators of languages such as C to know for sure.

Your second point is quite answerable.

Java created the enum type in response to the widespread use of int constants which do not behave well (Effective Java, Second Edition, Bloch, Item 30/Page 147). For example, you can multiply two integer constants in a nonsensical manner but you cannot do the same with two enum values. It is possible to multiple RED with BLUE when using integer constants, but not enums. What does that mean, anyway?

In addition, this decision was made because the enumeration type can be "smarter." It can contain values as well as encapsulate behavior inside each enum value. Rather than having a switch over the enum values which is brittle and a maintenance problem, invert the logic and put the body of each case inside each individual enum. Now call a method on your enum variable, and use dynamic dispatch to abstract your switch. Now the enum has its values and behavior localized to the same class.

C++ 11 now includes the enum class construct which was done because languages such as Java implemented "intelligent" enums with good success. (The C++ Programming Language, 4th Edition, Stroustrup, Section 8.4/Page 218).


答:
This is not the sole province of "old" console (command line) applications. Every running program has a "command line" that points at the executable image and includes any command line parameters. Plenty of GUI apps take arguments that alter either their initialization behavior or runtime behavior or both.

Most languages do let you specify an entry-point function that takes no arguments. But virtually all of them also allow you to pass arguments. This args array is simply an ordered collection of whatever is passed on the command line after the name of the executable file when the program is executed. EDIT: And of course in the case of the C language (as one example), the first argument (index 0) is actually the name of the program.

For example, imagine a pointless little program that displays a message in a message box, and lets you specify the message and the title for the window on the command line, like this:

myprogram.exe -title Foo -message Bar

Your args array in this case will look like this, presuming a language like C# which does not include the program name in the args array:

```<code>args[ 0 ] = "-title"
args[ 1 ] = "Foo"
args[ 2 ] = "-message"
args[ 3 ] = "Bar"
</code>```

Making sense of the order of the arguments, and which ones are valid or not, is totally up to your own application code.

Title: Why using string[] args in all main methods?

Body:
I'm a beginner in learning programming. I ask about using the string array in main method as a parameter. Why not writing the Main()  method without the string array? What is the point of having this array in Main method?

For example:

```<code>public static void Main(string[] args)
{
Console.WriteLine(args.Length);
Console.WriteLine(args[1]);
}
</code>```

Body:
This is not the sole province of "old" console (command line) applications. Every running program has a "command line" that points at the executable image and includes any command line parameters. Plenty of GUI apps take arguments that alter either their initialization behavior or runtime behavior or both.

Most languages do let you specify an entry-point function that takes no arguments. But virtually all of them also allow you to pass arguments. This args array is simply an ordered collection of whatever is passed on the command line after the name of the executable file when the program is executed. EDIT: And of course in the case of the C language (as one example), the first argument (index 0) is actually the name of the program.

For example, imagine a pointless little program that displays a message in a message box, and lets you specify the message and the title for the window on the command line, like this:

myprogram.exe -title Foo -message Bar

Your args array in this case will look like this, presuming a language like C# which does not include the program name in the args array:

args[ 0 ] = "-title"
args[ 1 ] = "Foo"
args[ 2 ] = "-message"
args[ 3 ] = "Bar"

Making sense of the order of the arguments, and which ones are valid or not, is totally up to your own application code.


问:
Broadcasting - Listening to replies
We use JS by the way, but I think it's language agnostic. I'm open to ideas.

We have this "pub-sub" framework that we use at work to fix the problem of tightly-coupled code. Works fine. Modules listen to events, things run, broadcasts events, more things run. It's amazing. However...

For instance I wanted to click on a link to open a modal, modify the passed data, and send it back to the called. Sending it to the modal would be as simple as broadcasting an event with the data. Modal, subscribed to the event, responds and does some alteration to the data. The problem is getting the data back to the caller.

Scenarios:

Module broadcasts an event, and at the same time listens to an event in the hopes of a response (event from another module). How would I know that the response was in effect of the event that was broadcast just before it?

Another is that I plan to send out a callback function with the data sent through the event. That way, the receivers can just call the callback with the data from their side. However, the module would be receiving an unknown amount of callback calls, depending on how many subscribers.

The scenario is more of directed messaging, which is somewhat contradictory to the fire-and-forget nature of a broadcast. Are there any patterns which still maintains loose coupling, modularity and the "unknown to the other" type of architecture?

答:
You're basically using a mediator pattern. To start, be aware that you might be overusing it if you're using it everywhere. Event driven programs can be harder to debug or to even reliably predict behavior, so you'll want to use this pattern only when there's an actual pain caused by the normal coupling. For a modal I very much doubt I'd use it.

With that said if you still want to go ahead with this, there's nothing wrong with having a publisher listen on a new unique event and then send information about what event it's listening on when it publishes. That way the receiver knows what event to trigger, and since it's unique the publisher knows who it's from when it receives the event. What you are doing in effect is creating a private event used only once by the publisher and the subscriber.

If instead you want to go with the callback type solution, consider investigating javascript promises. Rather than saying "here's a function you should execute when you're done", promises say "tell me when you're done and I'll decide what to do".

Title: Broadcasting - Listening to replies


问:
Hardware accelerated text processing
Graphics processing units (GPUs) are very common and allow for efficient, parallel processing of floating point numbers.
PPUs (Physics Processing Units) used to be a buzzword several years ago but never really caught on and this kind of calculation is now handled by GPUs as well.
Both of these types of hardware are specialized in the processing of a specific kind of data.

Looking at the kinds of applications being developed worldwide, it seems that a great many of them have to do with text processing. Particularly when web development is considered.

It seems to me that it would make sense to implement basic string operations (concatenation, reversal, maybe search, character-level operations such as replacement) at hardware level, possibly making those operations orders of magnitude faster than they are when performed on a typical CPU.

Does this sort of hardware exist? If it doesn't, what makes it infeasible? Varying length? Arbitrary encoding? I believe these are a bit similar to what we've got in case floating point numbers (varying precision, different encodings possible).

I know some vendors sell devices specifically designed to handle text processing/search. The Google Search Appliance is an example thereof but as far as I know, it's just a normal computer with dedicated software and the hardware does not feature anything as exotic as a processor specifically design to crunch text.

答:
(Disclaimer: I don't have information on the percentages of occurrence of various elementary string operations found in common software. The following answer is just my two cents of contribution to address some of your points.)

To give a quick example, Intel provides SIMD instructions for accelerating string operations in its SSE 4.2 instruction set (link to wikipedia article). Example of using these instructions to build useful language-level string functions can be found on this website.

What do these instructions do?

Given a 16-byte string fragment (either 16 counts of 8-bit characters or 8 counts of 16-bit characters),

In "Equal Each" mode, it performs an exact match with another string fragment of same length.

In "Equal Any" mode, it highlights the occurrence of characters which match a small set of characters given by another string. An example is "aeiouy", which detects the vowels in English words.

In "Range comparison" mode, it compares each character to one or more character ranges. An example of character range is azAZ, in which the first pair of characters specifies the range of lower-case English alphabets, and the second pair specifies the upper-case alphabets.

In "Equal ordered" mode, it performs a substring search.

(All examples above are taken from the above linked website, with some paraphrasing.)

Before beginning a discussion on this topic, it is necessary to gather the prerequisite knowledge needed for such discussion.

It is assumed that you already have college-level introductory knowledge of:

CPU architecture

Digital design and synthesis, which teaches introductory level of Hardware Description Language, such as Verilog or VHDL.

And finally, a practicum project where you use the above knowledge to build something (say, a simple 16-bit ALU, or a multiplier, or some hardware input string pattern detection logic based on state machine), and perform a cost counting (in logic gates and silicon area) and benchmarking of the things being built.

First of all, we must revisit the various schemes for the in-memory representation of strings.

This is because the practicum in hardware design should have informed you that a lot of things in hardware had to be "hard-wired". Hardware can implement complex logic but the wiring between those logic are hard-wired.

My knowledge in this aspect is very little. But just to give a few examples, rope, "cord", "twine", StringBuffer (StringBuilder) etc. are all legit contenders for the in-memory representation of strings.

Even in C++ alone, you still have two choices: implicit-length strings (also known as null-terminated strings), and explicit-length strings (in which the length is stored in a field of the string class, and is updated whenever the string is modified).

Finally, in some languages the designer has made the decision of making string objects immutable. That is, if one wish to modify a string, the only way to do so is to:

Copy the string and apply the modification on-the-fly, or

Refer to substrings (slices) of the original immutable string and declare the modifications that one wish to have applied. The modification isn't actually evaluated until the new result is consumed by some other code.

There is also a side question of how strings are allocated in memory.

Now you can see that, in software, a lot of wonderful (or crazy) design choices exist. There has been lots of research into how to implement these various choices in hardware. (In fact, this has been a favorite way of formulating a master's thesis for a degree in digital design.)

All this is fine, except that due to economic reasons, a hardware vendor cannot justify the cost of supporting a language/library designer's crazy ideas about what a string should be.

A hardware vendor typically has full access to every master's thesis written by every student (in digital design) in the world. Thus, a hardware vendor's decision of not including such features must be well-informed.

Now let's go back to the very basic, common-sense question: "What string operations are among the most-frequently performed in the typical software, and how can they benefit from hardware acceleration"?

I don't have hard figures, but my guess is that string copying verbatim is probably the #1 operation being performed.

Is string copying already accelerated by hardware? It depends on the expected lengths of strings. If the library code knows that it is copying a string of several thousand characters or more, without modification, it could have easily converted the operation into a memcpy, which internally uses CPU SIMD (vectorized instructions) to perform the memory movement.

Furthermore, on these new CPU architectures there is the choice of keeping the moved string in CPU cache (for subsequent operations) versus removing it from the CPU cache (to avoid cache pollution).

But how often does one need to copy such long strings?

It turns out that the standard C++ library had to optimize for the other case:

https://stackoverflow.com/questions/21694302/what-are-the-mechanics-of-short-string-optimization-in-libc

That is, strings with lengths in the low-ten's occur with such high frequency that special cases have to be made to minimize the overhead of memory management for these short strings. Go figure.

Title: Hardware accelerated text processing


问:
If immutable objects are good, why do people keep creating mutable objects?
If immutable objects¹ are good, simple and offer benefits in concurrent programming why do programmers keep creating mutable objects²?

I have four years of experience in Java programming and as I see it, the first thing people do after creating a class is generate getters and setters in the IDE (thus making it mutable). Is there a lack of awareness or can we get away with using mutable objects in most scenarios?

¹ Immutable object is an object whose state cannot be modified after it is created.

² Mutable object is an object which can be modified after it is created.

答:
Additional to many other reasons given here, a problem is that mainstream languages do not support immutability well. At least, you are penalized for immutability in that you have to add additional keywords like const or final, and have to use illegible constructors with many, many arguments or code lengthy builder patterns.

That's much easier in languages made with immutability in mind. Consider this Scala snippet for defining a class Person, optionally with named arguments, and creating a copy with a changed attribute:

```<code>case class Person(id: String, firstName: String, lastName: String)

val joe = Person("123", "Joe", "Doe")
val spouse = Person(id = "124", firstName = "Mary", lastName = "Moe")
val joeMarried = joe.copy(lastName = "Doe-Moe")
</code>```

So, if you want developers to adopt immutability, this is one of the reasons why you might consider switching to a more modern programming language.

Body:
Additional to many other reasons given here, a problem is that mainstream languages do not support immutability well. At least, you are penalized for immutability in that you have to add additional keywords like const or final, and have to use illegible constructors with many, many arguments or code lengthy builder patterns.

That's much easier in languages made with immutability in mind. Consider this Scala snippet for defining a class Person, optionally with named arguments, and creating a copy with a changed attribute:

case class Person(id: String, firstName: String, lastName: String)

val joe = Person("123", "Joe", "Doe")
val spouse = Person(id = "124", firstName = "Mary", lastName = "Moe")
val joeMarried = joe.copy(lastName = "Doe-Moe")

So, if you want developers to adopt immutability, this is one of the reasons why you might consider switching to a more modern programming language.


问:
Designing a scalable schema for college database
I am trying to build back end for college ERP system. The system will be based on LAMP.

These is the scenario:

Colleges have 5 branches

There are 4-6 classes in each branch.

There are 80 students in each class.

Each class has almost 6-7 subjects.

There are around 250 Faculties.

Each of them teaches one or more subjects and prepares a Teaching Plan for each of them.

There are two semesters in a year and two test are conducted in each semester.

Continuous assessments (CA) are conducted for per subject & per student basis. It comprises of 12 Experiments and 3 Assignment marks.

CA leads to the term work marks calculation at the end of semester.

Hence the problem is to record the data regarding attendance, Continuous assessment, Test Marks  of each student etc

I have build a sample schema but the problem is that it is not scalable. Because of the following:

If a subject is of 40 lectures then it requires almost 80*40=3200 records for recording attendance of a single subject per class; so what about 6 more subjects in each semester.

Take a look here

So my questions are:

Which tables should be kept as they are [i.e should be created only once] and which tables should be created dynamically and at what interval? [i.e year-wise/ semester-wise/ any other suggestions]

How to make data retrieval optimal?.

How to redesign schema so that it scales well.

How can I estimate size of each table in my database. Whether the fields which remains null / empty occupy space [whether they are taken into consideration in size calculations]

Thank you in advance for any suggestion or help.

答:
Initial response

Some preliminary matters:

As a practical matter I find it's helpful to throw in date fields for created/modified times; it makes your life much easier when debugging data problems. Pretty much any table that's not a lookup table would benefit from it. If a table is write-once, consider foregoing the modified time column.

Be sure that you're not storing passwords in plaintext. It's beyond the scope of this question but I thought I'd take the time to mention it.

That said, I think you're prematurely worrying about things like scalability until you've actually got some data to back up your problems. That said, I do have one easy suggestion.

If a subject is of 40 lectures then it requires almost 80*40=3200 records for recording attendance of a single subject per class; so what about 6 more subjects in each semester.

It's true that you're adding a lot of records but those records so far look to be... 10 bytes each? I'm not going to look up the sizes of the data types but you're not breaking the bank with 10 byte rows. I suspect that while the users always want to have attendance available, it's likely that agreeing on retention policy can help you to cap the growth and allow you to prune attendance records from years past.

Still, you've got room to optimize. Rather than recording attendance for each Student in a one-to-one relationship, I suggest you record attendance by picking the smaller of "attended" or "missed" and record only that. For example, if you assume most students go to lecture you would pick "missed," which means the Attendance table will only have rows for Students who missed lecture. If a Student has a row in the Attendance table he missed lecture, otherwise he was present. Of course in such a case, Attendance should be renamed to something like Absence or something.

Spend more time considering what you might do when your stakeholders want to add more data to your schema. Examples include adding categories or groupings, aliases or secondary names or secondary emails.

Regarding dynamic table generation

Which tables should be kept as they are [i.e should be created only once] and which tables should be created dynamically and at what interval? [i.e year-wise/ semester-wise/ any other suggestions]

This is a premature concern as you don't know what your performance problems are yet. I'd be inclined to look for badly performing tables after release and building up an archiving process depending on your needs.

I don't recommend dynamically creating tables as you'd have to make edits to the database schema on the live host that impact your application, which have the potential to bring down the application if you commit a typo. To answer your question, I recommend archiving data at the row level.

If you need old data to be highly available but infrequently accessed, you could surface it using a separate module of the application, with its own separate database. This database could be populated by an external process that moves rows (during a maintenance window) from the live/active module to the archive module.

If you don't need it to be as highly available you could export rows from the live database (during a maintenance window) to a compressed archive that is properly backed up and can be reviewed on-demand using developer tools.

The benefit with this approach is that your row operations won't take down the site; worst case is that some data is missing for awhile or performance is degraded.

As the lead developer it's up to you to determine what the needs are and act accordingly. Again, I recommend you put this kind of work off until you have a concrete problem.


答:
You can use the pimpl idiom, and provide methods to iterate over the container.

In the header :

```<code>typedef People* PeopleIt;

class AddressBook
{
public:
  AddressBook();

  PeopleIt begin();
  PeopleIt begin() const;
  PeopleIt end();
  PeopleIt end() const;

private:
  struct Imp;
  std::unique_ptr<Imp> pimpl;
};
</code>```

In the source :

```<code>struct AddressBook::Imp
{
  std::vector<People> people;
};

PeopleIt AddressBook::begin()
{
  return &pimpl->people[0];
}
</code>```

This way, if your client uses the typedef from the header, they will not notice what kind of container you are using. And the implementation details are completely hidden.

Body:
You can use the pimpl idiom, and provide methods to iterate over the container.

In the header :

typedef People* PeopleIt;

class AddressBook
{
public:
  AddressBook();

  PeopleIt begin();
  PeopleIt begin() const;
  PeopleIt end();
  PeopleIt end() const;

private:
  struct Imp;
  std::unique_ptr<Imp> pimpl;
};

In the source :

struct AddressBook::Imp
{
  std::vector<People> people;
};

PeopleIt AddressBook::begin()
{
  return &pimpl->people[0];
}

This way, if your client uses the typedef from the header, they will not notice what kind of container you are using. And the implementation details are completely hidden.


问:
Why is the hashCode method usage of HashSet not specified in the API?
I was trying to debug my code which uses a HashSet and searching through the SO, I found out that I needed to override the hashCode method as well. The strange part is, checking the related API, I did not see any part in it mentioning about the hashCode method. Quoting the definition of the add method of HashSet as seen in the API:

public boolean add(E e)

Adds the specified element to this set if it is not already present.
  More formally, adds the specified element e to this set if this set
  contains no element e2 such that (e==null ? e2==null : e.equals(e2)).
  If this set already contains the element, the call leaves the set
  unchanged and returns false.

Now in the quotation above, I don't see anywhere that mentions about the hashCode method. Shouldn't the correct statement have been like:

... if this set contains no element e2 such that (e==null ? e2==null : e.equals(e2)) AND if this set contains no element e2 such that (e==null ? e2==null : e.hashCode() == e2.hashCode()).

Now if you say that: "If o1.equals(o2) returns true, o1.hashCode() == o2.hashCode() MUST evaluate to true as well.", then I would ask three questions:

Where is that fact specified? (in general, or in the API)

Even if that fact is specified somewhere, where in the API it is specified that HashSet makes use of the hashCode method?

If that fact is indeed correct, why isn't the compiler enforces overriding the hashCode method, whenever the equals method is overridden?

答:
Where is that fact specified? (in general, or in the API)

It is specified clearly in the documentation of the .equals() method.

Note that it is generally necessary to override the hashCode method
  whenever this method is overridden, so as to maintain the general
  contract for the hashCode method, which states that equal objects must
  have equal hash codes.

You overrode the .equals() method without looking at the documentation of the method you were overriding, and you broke its requirements.

Even if that fact is specified somewhere, where in the API it is
  specified that HashSet makes use of the hashCode method?

It doesn't need to be specified anywhere. .hashCode() is a method on Object and thus all objects have it. Any class is allowed to make use of it.

That HashSet uses .hashCode() is an implementation detail, not part of its API. The API of HashSet is basically the same as the API of the Set interface which it implements. HashSet does not add any additional requirements on the type, any more than Set does. The Set contract make sure that there are no two .equals() elements in the set, and the lookup methods lookup using .equals(). HashSet does the same. HashSet uses .hashCode() as part of doing those operations, but it should be able to safely do so because .hashCode() is supposed to be consistent with .equals() as part of the contract of .equals().

If that fact is indeed correct, why isn't the compiler enforces overriding the hashCode method, whenever the equals method is
  overridden?

There is no mechanism in the language to enforce this.


问:
Is Singleton enough for a EMailUtil or there is any better implementation?
I am developing an application, where I have to send lots of emails for user account creations, validation, invites etc.

I have implemented a static class EMailUtil which has different methods like

```<code>public static void sendInvite(from, to, Subject){...}
public static void sendVerificationCode(from, to, subject, code){...}
</code>```

Afterwards, I converted that class to a singleton. I was wondering if there is a way I can improve my approach here. What other pattern I could use here and what is the best approach to implement this utility class?

答:
Ok, here's one suggestion.

First of all, you should get rid of repeating parameters (from, to, subject etc) and methods (sendInvite, sendVerificationCode etc).

To achieve that you could start with defining an EMail-class holding the parameters as attributes. This is quite a standard approach when facing repeating parameters. Now all invitations and verifications and such would be just instances of EMail and all EMails could be send using only one send-method. Thus you would also get rid of the repeating methods.

EMail could be subclassed or it could provide factory methods for creating all kinds of messages. So either

```<code>EMail verificationMessage = new VerificationEMail(recipient, verificationCode);
</code>```

or

```<code>EMail verificationMessage = EMail.createVerificationMessage(recipient, verificationCode);
</code>```

I'd choose the latter approach if it does not lead to an overly bloated EMail class.

What goes into from and subject fields of a message should probably be decided in the method that's creating a message of a certain type. All verification messages should probably have a similar title etc.

After defining the structure of the messages you only need some kind of dispatcher or such to send them:

```<code>EMailDispatcher dispatcher = new EMailDispatcher();
dispatcher.send(someMessage);
</code>```

Dispatcher could have only one method to send messages:

```<code>public void send(EMail mail);
</code>```

I'd try to avoid having a static send method like your EMailUtil has. Static methods are hard to replace for unit tests and otherwise. You don't want to get thousands of actual emails sent every time automated tests run.

Now, if you're using JavaMail-API, then it already has a Message-class. However it could prove useful to have your own message class to define the contents and types of messages your application needs and have the dispatcher deal with the more technical Java-API.

Body:
Ok, here's one suggestion.

First of all, you should get rid of repeating parameters (from, to, subject etc) and methods (sendInvite, sendVerificationCode etc).

To achieve that you could start with defining an EMail-class holding the parameters as attributes. This is quite a standard approach when facing repeating parameters. Now all invitations and verifications and such would be just instances of EMail and all EMails could be send using only one send-method. Thus you would also get rid of the repeating methods.

EMail could be subclassed or it could provide factory methods for creating all kinds of messages. So either

EMail verificationMessage = new VerificationEMail(recipient, verificationCode);

or

EMail verificationMessage = EMail.createVerificationMessage(recipient, verificationCode);

I'd choose the latter approach if it does not lead to an overly bloated EMail class.

What goes into from and subject fields of a message should probably be decided in the method that's creating a message of a certain type. All verification messages should probably have a similar title etc.

After defining the structure of the messages you only need some kind of dispatcher or such to send them:

EMailDispatcher dispatcher = new EMailDispatcher();
dispatcher.send(someMessage);

Dispatcher could have only one method to send messages:

public void send(EMail mail);

I'd try to avoid having a static send method like your EMailUtil has. Static methods are hard to replace for unit tests and otherwise. You don't want to get thousands of actual emails sent every time automated tests run.

Now, if you're using JavaMail-API, then it already has a Message-class. However it could prove useful to have your own message class to define the contents and types of messages your application needs and have the dispatcher deal with the more technical Java-API.


问:
Stop re-iterating over duplicates in nested for loop
I'm working with a JavaScript canvas, and I'd like to recreate quite a common effect I've seen where moving particles create lines between them when then get close to each other.

My issue is that I'm essentially performing two loops inside each other. Please see pseudo code:

```<code>var particles = [ /* array of particles with x and y values */ ]
for(var i = 0; i < particles.length; i ++){
    for(var j = 0; j < particles.length; j ++){
        // check if j is close to i
        // if so, draw a line between them
    }
}
</code>```

First of all I found that this would double check one particle against itself. So I can add:

```<code>         if(i == j) continue
</code>```

But then actually, I need a further case. For instance, if I have already checked when i = 1 and j = 8, I don't need to check when j = 1 and i = 8

Is there a common method of simplifying this checking procedure? I'm losing a fair bit of performance with the multiple checks.

答:
The usual way of saving effort here is

for(var j = i+1; j < particles.length; j++) {

This saves half of the comparisons in the naive double loop; and if you really do have to check every possible pair, you can't save any more than that.


问:
Which are the cases when 'uint' and 'short' datatypes are a better fit than the standard int(32)?
I understand the differences in capacity and values they can represent but it seems as though people always use Int32 regardless of whether it is appropriate. No one ever seems to use the unsigned version (uint) even though a lot of the time it fits better as it describes a value that cannot be negative (perhaps to represent an ID of a database record). Also, no one ever seems to use short/Int16 regardless of the required capacity of the value.

Objectively, are there cases where it's better to use uint or short/Int16 and if so, which are they?

答:
Often forgotten and possibly tangential to your question, when dealing specifically with .NET types, is CLS Compliance. Not all types are available to all languages built on the .NET Framework.

If you're writing code to be consumed by languages other than C# and wish that code to be guaranteed to interoperate with as many .NET languages as possible then you must restrict your type usage to those that are CLS Compliant.

For example, early versions of VB.NET (7.0 and 7.1) did not support unsigned integers (UInteger):

http://msdn.microsoft.com/en-us/library/aa903459(v=vs.71).aspx

Unsigned integers are not CLS compliant and so should be used with care if you are unsure who your class library consumer will be.


答:
When the NAS isn't working, is slow, or is down than we can't run a full test.

Obviously, this can only be solved by copying the 5GB from the NAS to your local drive. But there is no need to do this manually.

The second reason is that when someone first clones a repository the unit tests fail so they have to figure out how to mount things with a certain name and the syntax used to build the testing path file.

You could provide a simple shell script which does exactly this - mount the NAS with a certain name, and copy the data to your local drive when it is not already there, or when the dataset at the NAS is newer than the local dataset. Make sure the script will run automatically during the initialization stage of your unit tests.

Of course, when there is not only one of those data sets, but a whole bunch of dependencies to external files outside of your source code repository, then a tool like the ones mentioned by @ptyx might be the better solution.

Title: Where should I store test data?

Body:
I have smaller unit tests that use small snippets from real data sets. I would also like to test my program against full data sets for a multitude of reasons. The only problem is that a single real dataset is about ~5GB. I haven't found any hard numbers for what Git repositories can store but that seems like too much.

(According to this Programmers post I should keep all of my data needed to test the project in the repository.)

The solution that my team has adopted is that the project has a file that contains a path to a network attached file system that holds our test data. The file is Git ignored.

I feel like this is an imperfect solution for two reasons. When the NAS isn't working, is slow, or is down than we can't run a full test. The second reason is that when someone first clones a repository the unit tests fail so they have to figure out how to mount things with a certain name and the syntax used to build the testing path file.

So my question is two fold. How much data is too much data to store in revision control?

What is a better way to handle large amounts of test data?


问:
Can a single GitHub repository include source code released under multiple licenses?
I want to gather several sub-projects in a single GitHub repo for the convenience of the users. But I wonder if it's actually possible to have folders in a single repo each of which is covered by a different license (all of them are BSD-license variants, with small differences), or do I need to split the project into multiple repos?</p>

答:
If a licence file exists, it's rather natural to assume it applies to the folder (and its subfolders), so you can have a licence in each subproject folder and all is well.

Other than that, you could have the license in each file and thus even change between licences on a per-file basis.

Beyond that, I would point out that you could also have a repo for each project and then just add them as submodules to the "convenience" project (personally I find that more convenient, because I also have the possibility to clone just the part that interests me).

Lastly, if this is about convenience, then don't overdo it. Especially try to avoid variants. When I see a project, where I see clearly "this part is GPL, that part is MIT", then that's acceptable. But if I have to deal with a jungle of licences and custom variants of them, I'll probably just go look some place else or just write the part I need myself. Programmers love to reinvent the wheel. Don't give them excuses to do that ;)


问:
Why isn't software abstract on a grander scale?
Consider the following example:

The user wants a program to calculate a few fibonacci numbers.
Sounds easy enough. pseudocode:

```<code>stdout.write("How many fibonacci numbers do you want to calculate? ")
int count = int(stdin.readline())
while count>0:
    stdout.writeline(calculate_next_fibonacci_number())
    count--
</code>```

Even a very simple program like this is already flawed:

The program writes to stdout. That is not actually what the programmer intended, is it? The intention is to display a bunch of number to the user, not to write some text to stdout - there's no guarantee the user will ever see the text that's written to stdout.

Similarly, the user input is read from stdin, which is a text (or file, if you prefer) interface, when in reality, the program requires a number, not text. This seems fundamentally wrong.

Let's think about what we're trying to do more abstractly. We want to:

Calculate a bunch of numbers. How many is up to the user.

display the results to the user.

Why, then, don't we write code exactly like that?
(Most) programming languages provide this thing called a "function", which accepts parameters and uses them to do something. Does that not sound exactly like what we're trying to do?

```<code>void display_fibonacci_numbers(
        int number "how many fibonacci numbers to calculate"):

        Sequence<int> numbers
        while number>0:
            numbers.append(calculate_next_fibonacci_number())
            number--

        notify(numbers)

display_fibonacci_numbers()
</code>```

This code is, of course, incomplete - the notify function isn't implemented anywhere, and the user needs some way of inputting a number. I would imagine the user's operating system or desktop manager or whatever to take care of that - it could display a terminal, it could generate a GUI, it could tell the user do write the number into the air with his nose; either way it does not (should not) concern me, the programmer.

It is my belief that software should be more abstract.

A few more examples.

inter-process communication. How do you do it? Sockets? Signals? DBus? Files? The goal isn't to use sockets, signals or dbus, it's to communicate with another process. Why not something like Process.from_name("Music player").play_random_song()?

Downloading files. local_file.write(remote_file.read())? Why not download(url), which could, depending on the system configuration, start the download right away, but with a low priority so as not to slow down other downloads, or add it to the download queue to be downloaded later, or whatever?

File paths. Why on earth are file paths still strings (in most languages, at least)? Why do we have to deal with whether the path separator is / or \, whether it's ., .., or whatever? Why is there no FilePath class that takes care of this?

Going a step further, why not apply the duck-typing concept to modules? For example, in python, HTML is often parsed using the beautifulsoup module:

```<code>from bs4 import BeautifulSoup

page= BeautifulSoup(urlopen('http://test.at'))
</code>```

Again, the programmer's goal wasn't to use BeautifulSoup, but to parse HTML. Why, then, would he explicitly tell his code to use the BeautifulSoup module? What he really wants is to

```<code>import HTMLParser
page= HTMLParser.parse(urlopen('http://test.at'))
</code>```

Duck-typing for modules: Who cares what module it is as long as it does what I want?

Why is programming not like this? Am I overlooking something, something that makes all this impossible (or impractical)?

答:
You've discovered good factoring. Congratulations.

Why do more people not program like this? Because it's hard to see the advantage of doing things in a way that seems more effort and will only pay off later, when requirements change. People are short-sighted; even the ones who can think abstractly enough to become programmers are short-sighted. Some of them are a little less short-sighted, and they tend to be the ones inventing the rare trick or pattern that eventually manages to get established and raise overall software quality a tiny bit.

Then someone invents a shiny new distribution channel with a weird ad-hoc kludgy programming language to go with it, and all progress goes promptly out of the window. Don't feel bad, that's just the way the world is :)


答:
Why aren't programs more abstract?

Simply put: because when you get down to it, abstractions don't do anything.  Implementations do.  And the more abstract your program is--the more divorced it is from what's actually happening--the harder it is to understand what the code is doing.

At first, to a naive developer, it certainly seems like focusing on the high-level goal of what the code is supposed to be doing and "abstracting away the implementation details" makes the code easier to understand, but I respectfully submit that anyone who believes this has very little experience at debugging.  When something is going wrong in your code, especially if it's code that's been released already and you have clients who are unhappy about it and want a fix now, the faster you're able to find where the problem is, the better.  And having to dig through layers upon layers upon layers of abstractions makes this more difficult, not easier.

Since the modern software development lifecycle spends far more time in maintenance than in original development, anything that favors quick original development and high abstraction at the expense of maintenance, readability and debuggability should be considered a meta-example of premature optimization.

Don't get me wrong.  I'm not against abstraction where it makes sense.  But all too often you see people adding more and more abstraction just for abstraction's sake, and it turns the code into a mess of complexity rather than reducing complexity.  Remember the advice famously attributed to Einstein: Everything should be as simple as possible, but not simpler.

Title: Why isn't software abstract on a grander scale?

Body:
Consider the following example:

The user wants a program to calculate a few fibonacci numbers.
Sounds easy enough. pseudocode:

```<code>stdout.write("How many fibonacci numbers do you want to calculate? ")
int count = int(stdin.readline())
while count>0:
    stdout.writeline(calculate_next_fibonacci_number())
    count--
</code>```

Even a very simple program like this is already flawed:

The program writes to stdout. That is not actually what the programmer intended, is it? The intention is to display a bunch of number to the user, not to write some text to stdout - there's no guarantee the user will ever see the text that's written to stdout.

Similarly, the user input is read from stdin, which is a text (or file, if you prefer) interface, when in reality, the program requires a number, not text. This seems fundamentally wrong.

Let's think about what we're trying to do more abstractly. We want to:

Calculate a bunch of numbers. How many is up to the user.

display the results to the user.

Why, then, don't we write code exactly like that?
(Most) programming languages provide this thing called a "function", which accepts parameters and uses them to do something. Does that not sound exactly like what we're trying to do?

```<code>void display_fibonacci_numbers(
        int number "how many fibonacci numbers to calculate"):

        Sequence<int> numbers
        while number>0:
            numbers.append(calculate_next_fibonacci_number())
            number--

        notify(numbers)

display_fibonacci_numbers()
</code>```

This code is, of course, incomplete - the notify function isn't implemented anywhere, and the user needs some way of inputting a number. I would imagine the user's operating system or desktop manager or whatever to take care of that - it could display a terminal, it could generate a GUI, it could tell the user do write the number into the air with his nose; either way it does not (should not) concern me, the programmer.

It is my belief that software should be more abstract.

A few more examples.

inter-process communication. How do you do it? Sockets? Signals? DBus? Files? The goal isn't to use sockets, signals or dbus, it's to communicate with another process. Why not something like Process.from_name("Music player").play_random_song()?

Downloading files. local_file.write(remote_file.read())? Why not download(url), which could, depending on the system configuration, start the download right away, but with a low priority so as not to slow down other downloads, or add it to the download queue to be downloaded later, or whatever?

File paths. Why on earth are file paths still strings (in most languages, at least)? Why do we have to deal with whether the path separator is / or \, whether it's ., .., or whatever? Why is there no FilePath class that takes care of this?

Going a step further, why not apply the duck-typing concept to modules? For example, in python, HTML is often parsed using the beautifulsoup module:

```<code>from bs4 import BeautifulSoup

page= BeautifulSoup(urlopen('http://test.at'))
</code>```

Again, the programmer's goal wasn't to use BeautifulSoup, but to parse HTML. Why, then, would he explicitly tell his code to use the BeautifulSoup module? What he really wants is to

```<code>import HTMLParser
page= HTMLParser.parse(urlopen('http://test.at'))
</code>```

Duck-typing for modules: Who cares what module it is as long as it does what I want?

Why is programming not like this? Am I overlooking something, something that makes all this impossible (or impractical)?


答:
Why is programming not like this?

Why do you think that? Actually, programming is like that. Of course, you can't do that in assembler. Therefore we invented languages that can be more similiar to the human thoughts.

Good programmers program alike examples you gave. But it is not as easy. First you have to understand, what the requirement is. This is hard enough already and most people can't elaborate correctly what they need - therefore a good software designer needs to actually understand what people want, not what they tell.

After that is done you need to see the concepts behind the real world things. That is quite philosophical and needs much experience. And then, after this is done, you have to translate these results into a language of your choice, working around the incapabilities that every language has and choose the correct abstraction level (you don't want to overengeneer anything right?).

Not all people are good at these steps or even aware of them. Even in small examples one can see that it is not trivial. Let's take your display_fibonacci_numbers function and see what we can improve there (even though its not codereview.stackexchange here).

void display_fibonacci_numbers(

The naming is actually not very precise because you are very specified in the kind of numbers you will display right? That should be described by the function name.

```<code>    int number "how many fibonacci numbers to calculate"):
</code>```

Why is it an int? That is waaay to concrete and even wrong, because ints can of course have negative values right? So what you might really want is a type that can be used for calculating the fibonacci numbers. Maybe a natural number?

```<code>    Sequence<int> numbers
    while number>0:
        numbers.append(calculate_next_fibonacci_number())
        number--
</code>```

Does that really describe what you intend to do? In natural language (which is often a good approximation) we would say "create a list of the specified number of fibonacci numbers". I can't read anything about decrementing a number here, I can't read anything about a static 0 value here. Also nothing about a "calculate_next_fibonacci_number()" function.

Therefore, in Scala I would e.g. write the code as List.fill(number)(randomFibonacci())

It does not read exactly as in natural language, but it at least contains the same information.

I wanted to give you these examples to show you, that it is hard to write software that way. You need to have experiences, spent good amounts of time to think about things, be able to find the correct concepts and abstractions and THEN have a language that does not hinder you to express them in a concise way.
This is, why not many people program like that. And even worse: humans get used to things. So if one always programs in a language that hinders him in doing the last step, he will maybe continue to program in a new language as in the old one.

Body:
Why is programming not like this?

Why do you think that? Actually, programming is like that. Of course, you can't do that in assembler. Therefore we invented languages that can be more similiar to the human thoughts.

Good programmers program alike examples you gave. But it is not as easy. First you have to understand, what the requirement is. This is hard enough already and most people can't elaborate correctly what they need - therefore a good software designer needs to actually understand what people want, not what they tell.

After that is done you need to see the concepts behind the real world things. That is quite philosophical and needs much experience. And then, after this is done, you have to translate these results into a language of your choice, working around the incapabilities that every language has and choose the correct abstraction level (you don't want to overengeneer anything right?).

Not all people are good at these steps or even aware of them. Even in small examples one can see that it is not trivial. Let's take your display_fibonacci_numbers function and see what we can improve there (even though its not codereview.stackexchange here).

void display_fibonacci_numbers(

The naming is actually not very precise because you are very specified in the kind of numbers you will display right? That should be described by the function name.

    int number "how many fibonacci numbers to calculate"):

Why is it an int? That is waaay to concrete and even wrong, because ints can of course have negative values right? So what you might really want is a type that can be used for calculating the fibonacci numbers. Maybe a natural number?

    Sequence<int> numbers
    while number>0:
        numbers.append(calculate_next_fibonacci_number())
        number--

Does that really describe what you intend to do? In natural language (which is often a good approximation) we would say "create a list of the specified number of fibonacci numbers". I can't read anything about decrementing a number here, I can't read anything about a static 0 value here. Also nothing about a "calculate_next_fibonacci_number()" function.

Therefore, in Scala I would e.g. write the code as List.fill(number)(randomFibonacci())

It does not read exactly as in natural language, but it at least contains the same information.

I wanted to give you these examples to show you, that it is hard to write software that way. You need to have experiences, spent good amounts of time to think about things, be able to find the correct concepts and abstractions and THEN have a language that does not hinder you to express them in a concise way.
This is, why not many people program like that. And even worse: humans get used to things. So if one always programs in a language that hinders him in doing the last step, he will maybe continue to program in a new language as in the old one.