I wanted to keep people updated on what we've been doing to help with the stability issues of the Stellar network.
In the past few days we've completed moving to one validator, that came with a few bonus features.
Even though the one validator was churning along, public nodes, and in particular the heavily loaded ones (wrt amount of data in the historical node store), were going in and out of sync with the network very often.
Think of this situation with public nodes as runners trying to keep up with leaders (that don't care about the non-leaders) that run faster than them while running inside a maze! A single leader is actually faster as it doesn't spend time talking to other leaders on where to go next.
I thought this would continue to happen and decided to revive some of the changes I had on the consensus front to bias towards safety and help with debugging.
With the changes I made I could isolate where delays were being injected. It gives more leg to the theories I had on potential causes for the quorum to be lost over time (delays range from sub-second to double digit ones).
Quorum loss is a really bad problem to have with multiple validators especially when quorum is re-established between peers that may be different than the ones that had quorum before. In the case of public nodes this translates to availability problems that slow down clients a lot: the node doesn't accept new transactions or accepts transactions that are never validated.
Those changes worked well with our problematic nodes (those slow nodes stay in sync for much longer period of time: from minutes at a time to hours/days at a time) and I imagine that they should help other public nodes.
You can find updated config files to run your own public nodes in the tree.
thank you for your update (and of course for your hard work)!
Are there any plans to get rid of some of stellard's dependencies? I actually thought about running a public node, but compiling stellard just seems to be a bit of a hurdle. Are there any plans for stellard to be integrated into Linux or BSD packages and/or ports repositories? Would that even make sense given the changes that will be implemented into the consensus algorithm?
Concerning ease of building from source, yes we're working on that.
I don't think we talked about maintaining distro specific collaterals, I imagine that it will be next after making build push button.
b) Keeping in sync with the ledger.
In my opinion, those can be separated.
In the first case, we need the server to transmit transactions and receive confirmations from a number of validators and he believes that the transaction has been processed if 81% have returned true.
Keeping in sync with the ledger alone doesn't require that it is current. There can be a delay if need be. This process is simply to keep track of the history of transactions.
The reason that a) doesn't require b) is that if our topology is bad and the percentage of bad actors is bigger than what our protocol can handle, then those factors will continue to exist in all the subsequent transactions. I can't see why b) would help a).
Also, are there any news on the new consensus?
I actually thought about running a public node, but compiling stellard just seems to be a bit of a hurdle.
It dont think it work until they enable more validators, but I was able to start a stellard server in no time running that dockerfile. Make sure you have enough memory available, I believe it requires 4GB Memory.
when in sync, if a transaction "makes sense" against the current ledger, it can be forwarded to the network, otherwise it can just be outright rejected (and no fee is collected).
Going back on topic, did anybody try 0.28 to see if you can stay in sync?
As a point of reference: for the larger nodes (with 100TB+ rocksdb), we typically stay in sync days at a time, for smaller nodes (with only the last 2-3 weeks worth of ledgers) they have been staying in sync for much longer period of time (order of week+). Original sync can take a few hours, so be patient.
This was not the case before those changes where we would see nodes staying in sync only minutes at a time (for the larger nodes, or even get stuck in "we're not running on the consensus ledger" for days).
To experiment with this, I would start with a very conservative setup (like only download the last 10 ledgers), and progressively increase it (from what I've seen, downloading older ledgers can take its toll on a node that is barely in sync).
Page created in 0.095 seconds with 72 queries.