Calculating RAM requirements for an in-memory database In general, how much RAM does an in-memory database require? I would assume it requires enough RAM for:


Operating system and general system services, plus
DBMS, plus
the entire database (same amount as stored on disk), plus
temporary database tables
some slack


Considering that the largest component of this would normally be the size of the entire database, then it would require the normal size of RAM required to run a regular on-disk database (e.g. 16 GB) plus the on-disk size of the database (let's say 50 GB for a fairly large DB); in this example it would be at least 76 GB plus room for gradual growth of the database.

I know that the answer would be different for different DBMSs (e.g. SAP Hana or Oracle TimesTen), but I am asking about the general components of the calculation.

Is this analysis accurate, or am I missing some important components of the calculation?
Check out these links - the first is for Michael Stonebraker's VoltDB. He is one of the main drivers behind what he terms NewSQL - he believes that OldSQL (i.e. traditional RDBMS systems, your Oracles, MySQLs and SQL Servers) are doing far to much work buffering and latching &amp;c. In his VoltDB (Java based) system, transactions run to completion and don't compete with other transactions for system resources.

The key thing to bear in mind is that it is a shared-nothing in-memory architecture. He believes that this is good for OLTP, but not for OLAP for which he advocates columnar based stores such as the one he founded - Vertica, since bought by HP.

Stonebraker also believes that NoSQL has thrown the baby out with the bathwater in sacrificing SQL and ACID for performance.

Microsoft's Hekaton is based on very similar ideas (1, 2) which is hardly surprising since there was a Microsoft guy as coauthor of some of Stronebraker's work (can't find reference at the minute). The main difference AFAICS is that Hekaton's stored procedures are compiled to native code.

The references I have given are kind of vague and general, but the first Microsoft link says that your memory should be twice your data. AIUI, this rule is not cast in stone, but will depend on your app's functionality, but it's a good place to start :-). This is a fascinating area and worth reading up on (did an essay on it last year) and there's a lot of stuff out there.

Check out these also (1, 2, 3).