Is Drupal scalable across multiple geographical locations? I'm investigating how a Drupal site can be scaled across multiple geographical locations. The primary requirement for this is high availability. The site should be able to run despite some sort of outage on a single location.
The site is a &quot;heavy read, low write&quot; site, meaning that users will never create accounts and edit content. All content is edited by moderators only.
The following two options pop into my mind. I'd be interested in hearing if anyone else have experiences with similar setups - or just general feedback on how viable these options are.
1 WAN-wide cluster
A WAN-wide cluster could be achieved using a database cluster and a storage cluster. However, the immediate implications - as far as I see it - is latency. If we choose to go with a MariaDB/Galera cluster (which is synchronous replication), performance will be very dependent on network latency. For storage, something like GlusterFS (separate storage tier) could solve our needs. However, this again is latency dependent and I'm not so sure replication across an entire WAN is a good idea. The idea of a synchronous cluster (Galera/GlusterFS) is compelling as it removes a few timing issues (replication lag) which you need to take into account otherwise.
The other option for a WAN-wide cluster would be a &quot;regular&quot; master-slave database cluster, where we would have a single master database where all content editing is made. The master database will be located in one location, and the other geographical location(s) will have slave databases only. However, you can never guarantee that some Drupal node not will attempt to write to a database - so all Drupal nodes regardless of location will need to have the possibility to get in touch with the master database to execute write queries. This could be achieved using the AutoSlave module. However, this design makes the other geographical locations dependent on the single master node. If some sort of outage happens here, the master node will no longer be available and all other sites will possibly fail. This defeats the entire idea of high availability. With regards to storage, something simple like lsyncd/csync2 or BitTorrent sync could solve file replication. This would be asynchronous file replication, but I don't think that would be a problem as the site is &quot;heavy read, low write&quot;. Only a couple of images a day gets uploaded. Cached JS/CSS and thumbnails would possible be excluded from synchronization, which would put a little more load on all frontends - but that would be a question of testing how much load that would be.
2 Content sharing between locations
I'm not sure how this one could be done, and I'm afraid complexity grows. However, separate - but identical - Drupal installations could be set up at multiple locations. All content editing could be made at one of the sites (master), which would publish content to the other sites. There would be no database or storage dependencies between the sites. However, I'm not sure if this is all that easy to to - especially when content gets complex with multiple fields, media types, images etc.
I'll address your two points one by one and then discuss suggestions. This is a massive question in scope so I'm doing the best I can with what we've been provided.

WAN-wide cluster

Galera would be your best option if you wish to do an Active/Active setup. It's a tricky setup though and only the best hosts have figured it out (platform.sh comes to mind). Your other option is, by far, the most common. i.e. Master-Slave replication. You may find support for that in the Drupal community is more prevalent and that is worth its weight in database queries. :)

Content sharing between locations

You have some options when it comes to this. If your editors are to be collocated into a single geographical space, then sharing content isn't a problem. If you have a need to allow editors access from many geographical locations, you can do this in a two primary ways:


Active/Active using a Galera cluster as you indicated.
Active/Passive sites using a content deployment model using the Deploy module, Migrate module, Feeds module, or some combination of those.


Suggestions


Start thinking in terms of a single site, not sites. Ultimately, you'll employ a CDN of some kind to deliver content across regions. You had mentioned a concern with freshness of content but there are many ways to achieve the balance. i.e. Look to the Drupal modules Purge, Cache Actions, and Rules. 
Consider Edge Side Includes which will allow the static serving of some content on a page but assets served from the edge for others.
I can't stress the importance of dedicated Memcache servers enough.
CDNs, Varnish and a module such as Boost are not mutually exclusive concepts. They can all be employed to keep things fast on the anonymous side. Bear in mind, however, the more layers you add; the greater the possible complexity.


Hope this is helpful.