<story><title>Waltz: A Distributed Write-Ahead Log</title><url>https://wecode.wepay.com/posts/waltz-a-distributed-write-ahead-log</url></story><parent_chain><item><author>georgewfraser</author><text>This design seems to be an example of a deterministic database system. There&amp;#x27;s an excellent review of deterministic databases here: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.cs.umd.edu&amp;#x2F;~abadi&amp;#x2F;papers&amp;#x2F;abadi-cacm2018.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.cs.umd.edu&amp;#x2F;~abadi&amp;#x2F;papers&amp;#x2F;abadi-cacm2018.pdf&lt;/a&gt;&lt;p&gt;The core concept of all deterministic databases is simple: if your database is deterministic, multiple geographically-distributed replicas can execute the same transaction log independently, and they will all reach the same state. The problem of implementing a distributed database is reduced to implementing a distributed log.&lt;p&gt;However, there is one oddity of the Waltz design: the central locking mechanism. Other deterministic databases don&amp;#x27;t have anything like this, because it&amp;#x27;s not necessary. You can keep track of locks locally on each replica, and you can rely on every replica to reach the same conclusions about which transactions succeeded and failed, because they are deterministic.&lt;p&gt;Can anyone clarify why they are managing centralized locks?</text></item></parent_chain><comment><author>majidazimi</author><text>Essentially if each instance can keep track of validating transactions locally, the whole thing can be easily implemented with Bookkeeper and distributed log API [0]. Bookeeper guarantees single writer (the master node which is appending to log) for each ledger. All it needs to be done is to put concurrency control metadata (like version, timestamp, ...) inside the message so each node can validate stuff locally.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;bookkeeper.apache.org&amp;#x2F;docs&amp;#x2F;4.9.2&amp;#x2F;api&amp;#x2F;distributedlog-api&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;bookkeeper.apache.org&amp;#x2F;docs&amp;#x2F;4.9.2&amp;#x2F;api&amp;#x2F;distributedlog-...&lt;/a&gt;</text></comment>
<story><title>Waltz: A Distributed Write-Ahead Log</title><url>https://wecode.wepay.com/posts/waltz-a-distributed-write-ahead-log</url></story><parent_chain><item><author>georgewfraser</author><text>This design seems to be an example of a deterministic database system. There&amp;#x27;s an excellent review of deterministic databases here: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;www.cs.umd.edu&amp;#x2F;~abadi&amp;#x2F;papers&amp;#x2F;abadi-cacm2018.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;www.cs.umd.edu&amp;#x2F;~abadi&amp;#x2F;papers&amp;#x2F;abadi-cacm2018.pdf&lt;/a&gt;&lt;p&gt;The core concept of all deterministic databases is simple: if your database is deterministic, multiple geographically-distributed replicas can execute the same transaction log independently, and they will all reach the same state. The problem of implementing a distributed database is reduced to implementing a distributed log.&lt;p&gt;However, there is one oddity of the Waltz design: the central locking mechanism. Other deterministic databases don&amp;#x27;t have anything like this, because it&amp;#x27;s not necessary. You can keep track of locks locally on each replica, and you can rely on every replica to reach the same conclusions about which transactions succeeded and failed, because they are deterministic.&lt;p&gt;Can anyone clarify why they are managing centralized locks?</text></item></parent_chain><comment><author>dmix</author><text>Daniel Abadi also has an excellent paper on column databases: &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;db.csail.mit.edu&amp;#x2F;pubs&amp;#x2F;abadi-column-stores.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;db.csail.mit.edu&amp;#x2F;pubs&amp;#x2F;abadi-column-stores.pdf&lt;/a&gt; He&amp;#x27;s a great person at giving high level overviews of database systems before digging into the implementations.</text></comment>