Hazelcast 3.8.3 - <AUTHOR> https://jepsen.io/analyses/hazelcast-3-8-3 <comment><USER2> "In addition, the names of Hazelcast's datatypes, and the functions provided on those types, imply a certain fitness-for-purpose, e.g. that users can use these types and functions in a meaningful way. What is the point of an ID Generator which emits duplicate IDs? A lock that doesn't lock? Who wants an AtomicReference which is not atomic? Of what possible use is a queue which doesn't, well, queue?"

Wow. I wonder if the documentation and naming schemes are bad because they're aspirational, or because the developers don't know what they're doing?

<comment><USER3> Not with respect to Hazelcast specifically, but it's often the latter, but not maliciously so. It's fairly difficult to implement a lot of those things correctly, and fairly easy to convince yourself you were thorough.

<comment><USER4> That's why we have QA, no?

A developer can only test up to the point that their code works as they thought it should work. QA takes over from that point.

<comment><USER3> Probably not. You as a developer could write a Jepsen harness for your system and find the same results seen here. You could also write property-based tests to improve your testing model and input space coverage. You could also use mutation testing to see how good your tests are, etc. etc. etc.

At my last job we held off running a chaos test suite against our platform in the certification environment until _after_ the QA department came back with their report and gave the platform all green checkmarks. This was an exercise in both validating the value of the chaos engineering/testing work as well as partially discrediting the idea that QA is capable of exercising enough of the system in enough ways to expose non-obvious/nuanced edge cases.

I mean I guess there's an argument to be made that QA could be engineering and building their own rich tooling like a chaos testing framework, but that seems extremely unlikely to happen in the current typical formulation of QA.

</comment></comment></comment></comment><comment><USER5> Is there some award Jepsen and Aphyr can be nominated for? Kyle's work has improved so many open source projects it needs acknowledgement.

Also, his Twitter is a great thing for confusing people who want to read about lock schemes and relationship management etc...

<comment><USER6> ... and bears dressed up as Wonder Woman :-)

</comment></comment><comment><USER7> Hazelcast has officially responded to the Jepsen Analysis here. [https://blog.hazelcast.com/jepsen-analysis- hazelcast-3-8-3/](https://blog.hazelcast.com/jepsen-analysis-hazelcast-3-8-3/)

<comment><USER8> You write that Hazelcast belongs to a category of software that characteristically chooses speed/availability over consistency in case of network partitions[1]. A web search leads me to your your competitor who sounds like they are tackling the issue by putting consistency first: [https://docs.gridgain.com/v8.1/docs/network- segmentation](https://docs.gridgain.com/v8.1/docs/network-segmentation) [https://www.gridgain.com/products/software/enterprise- editio...](https://www.gridgain.com/products/software/enterprise- edition/network-segmentation-protection)

"When there is a network partition we favour availability over consistency. Which is what the Jepsen test shows. [...] It is also the PACELC contract of the entire category of IMDG systems. IMDGs are used for maximum speed and low latency. They represent a set of design trade-offs to achieve this primary purpose."

</comment></comment><comment><USER9> For anyone wondering if there is a set of Hazelcast like data structures and primitives that actually have safe, strong consistency (based on Raft consensus), check out Atomix:

[http://atomix.io/atomix/](http://atomix.io/atomix/)

<comment> dyu- > that actually have safe, strong consistency

Based on what data? Even hazelcast had that same claim before they got exposed?

I've only seen 2 systems that did rigorous in-house fault-tolerant testing (foundationdb and cockroachdb) and later when "jepsen-verified", they actually backed their claim or had few modifications to uphold their claim.

<comment><USER9> Atomix has a Jepsen test suite[1], which while not perfect, has helped fix the bugs that existed in the Raft implementation.

1: [https://github.com/atomix/atomix-jepsen](https://github.com/atomix/atomix- jepsen)

</comment></comment></comment><comment><USER10> Ouch. (Someone please give this guy an industry medal.)

Talk of Hazelcast reminded me of Coherence (now Oracle, before Tangosol): [https://www.javalobby.org//java/forums/t78008.html](https://www.javalobby.org//java/forums/t78008.html)

Oracle docs sensibly call it a distributed cache: [http://www.oracle.com/technetwork/middleware/coherence/distr...](http://www.oracle.com/technetwork/middleware/coherence/distributed- caching-100021.html)

</comment><comment><USER11> We use Hazelcast at Verizon Labs for a couple services, and one of the issues we ran into is how much _work_ it takes to replicate split brain to test various and custom merge policies.

When reading that the distributed map implements the ConcurrentHashMap interface we were immediately skeptical, but it took more work to prove our skepticism was founded.

Hazelcast has a _fantastic_ foundation for building clustered applications, but some of the design choices they made both internally and API wise are not choices we would have made.

We are moving to a custom merge policy because of some of these choices.

</comment><comment><USER12> If a system is so "broken by design", shouldn't it include a flag for running under worst-case mode, so that users can more easily detect faults in their assumptions about the system?

</comment><comment><USER13> Wait, is this Kyle Kingsbury? If so, where is his name? Who are the authors? I notice this text doesn't mention the authors:

"We wish to thank Jordan Halterman for his discussion of Hazelcast use cases. Luigi Dell'Aquila & Luca Garulli from OrientDB, and Denis Sukhoroslov from BagriDB, were instrumental in understanding those systems' use of Hazelcast. Thanks also to Julia Evans, Sarah Huffman, Camille Fournier, Moishe Lettvin, Tim Kordas, André Arko, Allison Kaptur, Coda Hale, and Peter Alvaro for reading and offering comments on initial drafts. This research was performed independently by Jepsen, without compensation, and conducted in accordance with the Jepsen ethics policy."

I'd like to know who stands behind this research

<comment><AUTHOR> Hahaha I suppose that's an entirely legitimate question, isn't it? Just to reassure you, Lisa, yes, this is me, and I'm the only person who works at Jepsen. I'm still exploring voices for the Jepsen brand--since at some point I might hire more researchers, more recent stuff is written with an organizational "we". Suppose I should start adding bylines. :)

<comment><USER14> hey, Kile. Thanks for the awesome work you do.

completely off-topic: have you ever thought about giving your training ([https://jepsen.io/training](https://jepsen.io/training)) in an online way (mooc, udemy, whatever)? I would love to learn about distributed systems from you, but today I think it would be almost impossible since you only seems to give in organization trainings.

<comment><AUTHOR> Thank you. Unfortunately, it's just not a good fit for a MOOC, in terms of class format and cost.

<comment><USER10> Have you explored high level consulting in inception phase of distributed system design? (and p.s. thank you for your exceptional work and knowledge sharing.)

<comment><AUTHOR> You're welcome, and yes, I do design consulting as well: [https://jepsen.io/services](https://jepsen.io/services).

</comment></comment></comment></comment></comment></comment><comment><USER15> Apache Ignite seems to be doing things right, they don't have jepsen results yet though...

<comment><USER16> Another solution in the same space is Apache Geode. Its commercial version has been around since a long time and has been well proofen on very large loads in systems that require low response times.

Disclaimer: I work on Geode.

<comment><USER15> What is the current state of Gemfire/Geode vs GridGain/Ignite? Any big differences?

<comment><USER17> Also vs snappydata which is based on gemfire?

<comment><USER18> Worked on GemFire prior to Pivotal acquisition (before Geode) and currently work for SnappyData.

You can imagine GemFire/Gridgain as an apples-to-apples comparison. Both are "enterprise" in-memory data grids originally intended for managing data in low-latency OLTP applications which later added analytics/OLAP features. Geode/Ignite are the open source options for these two IMDGs and also a good apples-to-apples comparison. (Hazelcast also has enterprise/OSS verisons I would compare accordingly)

I can't speak to the current comparison between these systems, but I can compare them to SnappyData. SnappyData deeply integrates GemFire with Spark to bring high concurrency, high availability and mutability to Spark applications. In the world of combining Spark with a datastore over a connector (cassandra, hive, mysql, mongo etc) to enable "database-like" features in Spark, SnappyData has taken the next step of integration. In Snappy, the database (GemFire) and the Spark executors share the same block manager and VM so the systems no longer communicate over a "connector." This, along with our database optimizations, provides the best performance for Spark applciations in what I like to call the "Spark Database Ecosystem."

As such, comparing SnappyData to GemFire/Hazelcast/Gridgain does not make much sense unless you are trying to use Spark in conjunction with these systems. In that case, the main difference I would point out is that SnappyData will necessarily perform better as any of them would need to use a connector to interact with Spark. The better comparison would be between SnappyData and Ignite, as Ignite contains a direct Spark abstraction called "IgniteRDD." That said, the majority of the comparisons/benchmarks we've run have been against MemSQL+Spark and Cassandra+Spark, so I don't have much to say about Ignite vs SnappyData.

User manigandham mentions SnappyData's Approximate Query Processing features (called Synopses Data Engine) which is unique within this space, but a discussion of which would take this too far afield.

</comment></comment></comment></comment></comment><comment><USER19> It's going to take me a while to go through all of this, so I'll miss the boat on the HN comments

However, I'd really like to credit the author on the design of the site, a pleasure to read.

</comment><comment><USER20> Question: why do we still use sequential ID's for identifying objects in data stores? There are many known problems with using numeric ID's (anyone remember Twitter's overflow issue), so when they are sent to a remote system, they should probably be strings anyways. Sorting or doing math on ID's is basically useless. Memory savings are the only thing I can think of, and while yes that's a nice feature, when a cluster of N nodes is generating ID's that are supposed to be globally unique, aren't UUID-type things better?

<comment><USER21> Because (B-tree) indexes on sequentiel IDs are more efficient.

</comment></comment><comment><USER22> Is this use of Hazelcast in Apero CAS safe? It is built on IMap. [https://apereo.github.io/cas/5.1.x/installation/Hazelcast- Ti...](https://apereo.github.io/cas/5.1.x/installation/Hazelcast-Ticket- Registry.html)

<comment><AUTHOR> I assume that if you lose your ticket you can always reauthenticate to get a new one, so... that doesn't seem like it'd be too bad, right?

</comment></comment><comment><USER23> We have been using Hazelcast for a while now. Currently evaluating K tables if we could get the same performance. Any learnings or advices would be welcome if anyone has done the switch.

<comment><USER24> What are K tables?

<comment><USER25> Presumably Kafka Streams KTables

</comment></comment></comment><comment><USER26> Fairly brutal for a Jepsen report.

What fascinates me about the now-known-to-be-untrue claims in the Hazelcast documentation is that the claims were made despite the developers having know way of knowing if they were true because they'd not conducted these kinds of tests themselves.

Documentation that reflects what the developers wish were true rather than what is actually true is not a new phenomenon, but is potentially fatal for this kind of software.

<comment><USER27> For someone listed as a partner on their homepage to be actively attempting to remove the system entirely is also pretty bad news. Presumably a number of users of Hazelcast have been bitten by problems, and enough of them are able to attribute them to Hazelcast...</comment></comment>


Show HN: Magnetico – self-hosted BitTorrent DHT search engine suite - <AUTHOR> https://github.com/boramalper/magnetico <comment><USER2> Skimming over the code it seems like it's doing fairly egregious bittorrent spec violations. In other words its very selfish and other clients may blacklist it if they detect such behavior.

<comment><AUTHOR> > Skimming over the code it seems like it's doing fairly egregious bittorrent > spec violations.

It's true and even understated: _magneticod_ is not only selfish but also incredibly aggressive. It will literally contact a thousand (and if your network allows, even more) nodes every single second and deceive them by forging thousands of ids to create the impression that it is "very close" (in terms of XOR-distance used in DHT Kademlia) to the every single node.

Please have a look at this website (although it is a bit old): [https://dsn.tm.kit.edu/english/2936.php](https://dsn.tm.kit.edu/english/2936.php)

There are on average 7,000,000 DHT peers in the network and less than a quarter of the nodes stay longer than 12 hours in the network (i.e. the node you contacted this morning might be long gone at the evening). There are quite a few public papers[0] describing various Sybil- _attacks_ on BitTorrent DHT, so it's not something unknown. BEP 42[1] tries to mitigate the issue by making node IDs dependent on the IP address of the node, but as far as I know not many clients (if any) implements it.

I spoke too much but one last point: These _attacks_ -given the scale of BitTorrent DHT- I doubt to be disruptive, but they are more dangerous for the privacy of the peers. One can gather thousands of IP addresses each day as easily using the same technique. (See [http://opentracker.blog.h3q.com/2007/02/12/perfect- deniabili...](http://opentracker.blog.h3q.com/2007/02/12/perfect- deniability/))

[0]: [https://www.usenix.org/legacy/event/woot10/tech/full_papers/...](https://www.usenix.org/legacy/event/woot10/tech/full_papers/Wolchok.pdf)

[1]: [http://www.bittorrent.org/beps/bep_0042.html](http://www.bittorrent.org/beps/bep_0042.html)

<comment><USER2> I'm not sure what you're trying to argue here. That it's ok to misbehave because there are papers describing how to do it? That it's ok to be selfish because there are enough compliant nodes to compensate for it?

<comment><AUTHOR> > That it's ok to be selfish because there are enough compliant nodes to > compensate for it?

Yes. Of course, I do not have data nor the foresight to claim that its short or long term effects are negligible, but I really doubt if it's not the case.

> That it's ok to misbehave because there are papers describing how to do it?

Not because of that, but I was pointing out that the technique is nothing new, and probably numerous real attacks of much greater scale (with the intention of actually disrupting the network) using the same techniques must have been launched many times before magnetico. If BitTorrent DHT survived those, surely it is more resilient than you tend to think.

\----

I would love to hear the opinion of a BitTorrent developer though. I'll try to e-mail them tonight and ask for their opinion. =)

<comment><USER2> > Yes. Of course, I do not have data nor the foresight to claim that its short > or long term effects are negligible, but I really doubt if it's not the > case.

And what if people see your solution and try to build an end-user thing out of it so that millions of users could run it at home?

When building something in a P2P network you always should consider whether it would still work if the majority of the network behaved like that.

> If BitTorrent DHT survived those, surely it is more resilient than you tend > to think.

Yes, the rain forest is also quite resilient. Nothing wrong with logging a few more trees, right? I'm saying your kind of reasoning is responsible for tragedy of the commons outcomes. You're not even trying to make your nodes a good citizen in the network because "others do it too".

\--

Anyway, such behavior can be detected and if it becomes more common other clients will probably implement blacklisting for it.

<comment><AUTHOR> > DHT indexing already is possible and done in practice by passively observing > get_peers queries. But that approach is inefficient, favoring indexers with > lots of unique IP addresses at their disposal. It also incentivizes bad > behavior such as spoofing node IDs and attempting to pollute other nodes' > routing tables.

Source: DHT Infohash Indexing < [http://www.bittorrent.org/beps/bep_0051.html](http://www.bittorrent.org/beps/bep_0051.html) >

Figurative speech aside, bittorrent.org acknowledges the need & the issue, and calls it _bad_ , not " _egregious_ ". The paper[0] mentioned in the BEP 51 proposal describes more or less how magnetico works ("horizontal attack") and states in conclusion that

> Through an extensive measurement study since December 2010, we have > identified that both of these attacks are happening in the real network. We > have analyzed their exact behavior through honeypots and have shown the > scale of the on-going activities. We must stress that we have no concrete > proof of actual malicious activities; our work only shows that the scale of > attacks is large enough for this to be a concern.

Repeating for the last time (as I feel this slowly creeps into a flame war):

1\. As I have said, these technique is already in use. This means that the network is more resilient than you claim and the chances that a few hundred (or thousand, in future) magnetico instances will harm the network is near zero. I could answer in a more direct way if you did not use rain forest metaphor and addressed the issues you see more directly.

2\. magnetico uses a _similar_ but also _a lot different_ technique. The _horizontal (Sybil) attack_ requires the offending node to maintain its identity: "First, the attacker only has to answer two types of messages, `PING` and `FIND_NODE`; he can ignore every other message."[0] magneticod, in contrast, answers only `GET_PEERS` and `ANNOUNCE_PEER` queries, which are _essential_ for the remote node to register itself as a peer downloading the torrent. magneticod will send hundreds of `FIND_NODE` queries per second[1] using forged node IDs, _and that 's all_! According to the BitTorrent DHT protocol[2], "after 15 minutes of inactivity, a node becomes questionable." Hence, presumably, magneticod should be forgotten & removed from the routing table after it does not answer several `PING` queries. Consider that `FIND_NODE` queries that magnetico makes are targetting (at every single time) totally random nodes, so same set of nodes will not be targeted ever (statistically speaking).

The problem is, BEP 51 is still a _draft_ and AFAIK no clients in the wild implements it, so it's simply impractical to write a DHT search-engine depending on this particular behaviour. Once it becomes more widespread (hopefully together with BEP 42), magnetico will (and has to) drop its current technique and support the new and proper one.

[0]: [https://www.cl.cam.ac.uk/~lw525/publications/security.pdf](https://www.cl.cam.ac.uk/~lw525/publications/security.pdf)

[1]: [https://github.com/boramalper/magnetico/blob/257912a12f862d9...](https://github.com/boramalper/magnetico/blob/257912a12f862d9f4ca3c5970df57f3a62d29c66/magneticod/magneticod/dht.py#L225)

[2]: [http://www.bittorrent.org/beps/bep_0005.html](http://www.bittorrent.org/beps/bep_0005.html)

<comment><USER2> > bittorrent.org acknowledges the need & the issue

It does not acknowledge or imply that what you are doing (forged replies) is necessary. It only acknowledges that indexing happens in the wild and that when it is happening nodes are currently _incentivized_ to misbehave, that does not mean they _need_ to misbehave to achieve their goals.

> and calls it bad, not "egregious".

That's nitpicking words, a spec is obviously written in a more conservative tone. Your implementation is far outside specs, intentionally so to manipulate the behavior of other clients.

> This means that the network is more resilient than you claim and the chances > that a few hundred (or thousand, in future) magnetico instances will harm > the network is near zero.

I have not denied that the network has some resilience or claimed that a single instance of this code running will destroy the network. I am saying that your code is misbehaving and greedy, which makes it a bad citizen in a p2p environment.

> BEP 51 is still a draft

Hardly relevant, many widely deployed BEPs are in draft status.

> AFAIK no clients in the wild implements it

That's not entirely true, but even assuming that it is it is not relevant either because BEP51 is not the only approach to be a good citizen while indexing the DHT.

</comment></comment></comment></comment></comment></comment></comment><comment><USER3> Nice project! Last year I made a similar PoC on Elixir based on the paper "Crawling DHTs for fun and profit"[1].

In that paper the researcher uses a pool of IPs to introduce multiple nodes (to cover the hash space). On my PoC I used less IPs but multiple ports per IP. How does this project manages to cover that ?

[1]: [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.172....](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.172.8202)

<comment><AUTHOR> I too read that paper, but magnetico crawls Mainline DHT (which is the _de facto_ standard) whilst the paper is on Vuze DHT, though the idea is more or less the same.

This project uses only _one_ instance of crawler running at the same time, so one IP and one port for crawling the DHT. Whenever it receives a `GET_PEERS` query from another peer in the network, it forges itself a new ID that has the first 15 bytes in common with the info hash in the query, so that the querying node would think that it found the closest (or a close enough) node to announce. After responding back, most of the times the same querying node sends a `ANNOUNCE_PEER` query to register itself as a peer downloading that particular torrent. Upon receiving `ANNOUNCE_PEER` query, magneticod then connects the peer using BitTorrent (TCP) protocol, receives the complete metadata, and closes the connection.

Keep repeating these and you'll end up with thousands of torrents at the end of each day.

Possibly, utilizing multiple IPs might increase the throughput and prevent IP based bans, but I didn't bother to be honest, as the current solution seems to work well enough.

</comment></comment><comment><USER4> A couple of thoughts on this.

1\. I assume it is "dangerous" to run this on an IP address that is easily traceable back to you, since it claims, to various peers, to be downloading all the things, even though it doesn't. Is this correct?

2\. A site, skytorrents.in, was posted to Show HN a while ago, that appears to be similar to a centrally hosted version of this, in that it builds its index of magnet links by trawling the DHT rather than relying on user submissions.

<comment><USER5> 1\. I'd expect any decent logging system to actually attempt transferring a file before marking an IP

2\. BTDigg is/was the most known one, dozens of other followed.

<comment><USER6> I expect the content mafia just to sue whoever they can

</comment></comment></comment><comment><USER7> Its not really decentralized. Readme says:

" _magnetico liberates BitTorrent from the yoke of centralised trackers & web- sites and makes it truly decentralised. Finally!_ "

Source says: ( [https://github.com/boramalper/magnetico/blob/42b2d64d200e2af...](https://github.com/boramalper/magnetico/blob/42b2d64d200e2aff106eb0ce4241d7bcc411ceb5/magneticod/magneticod/dht.py#L31-L34) )

        BOOTSTRAPPING_NODES = [
            ("router.bittorrent.com", 6881),
            ("dht.transmissionbt.com", 6881)
        ]

Also, i think its going to be very difficult to find anything in the DHT (spam) chaos without some of the pre-filtering that exists by using torrent sites. Fun project though..

<comment><USER8> >Its not really decentralized

Every DHT has to be bootstrapped from something. If that makes it not distributed then i'm not sure anything is. You can use whatever bootstrap nodes you want by changing the values.

<comment><USER7> Readme says "truly decentralised" which is the point, not "not distributed". It certainly is distributed, just not truly. It requires bootstrapping with prefixed domain names. Bootstrapping is not bad (when done securely), but the false claim is.

<comment><USER2> DHT bootstrapping is an implementation detail, not part of the protocol. You don't need DNS or any dedicated bootstrapping mechanism since _any_ node in the network can be used to join it. You could call your friend and ask for the IP and port of his bittorrent client.

So the protocol is fully decentralized. Implementations tend to use bootstrapping mechanisms that rely on an expandable, customizable list of known bootstrap nodes, but this is not essential.

</comment></comment></comment></comment><comment><USER9> This looks rather nice though two things seem to be missing:

* A docker-compose file for easy testing * Screenshots of the WebUI

<comment><AUTHOR> I have no experience with docker so I couldn't (nor can) create docker-compose file but the installation is pretty straightforward:

        pip3 install magneticod magneticow --user

Also, I added some screenshots, thanks for the suggestion! =)

<comment><USER10> "Dockerizing" Python apps is pretty straightforward, too. This might help if you ever decide to try it: [https://runnable.com/docker/python/dockerize-your- python-app...](https://runnable.com/docker/python/dockerize-your-python- application)</comment></comment></comment>


Build and ship native GUI apps on Python with Beeware [audio] - <AUTHOR> https://talkpython.fm/episodes/show/79/beeware-python-tools <comment><USER2> This is worth a listen simply for the brilliant ways he figured out how to convert python to native function calls for both ios and android.

Listening to the author, Russell Keith-Magee, gives me the feeling that he might just be able to pull this off. Partly for his experience on the Django core dev team and partly because he just seems like a really smart guy while at the same time having the ability to rally community engagement.

</comment><comment><USER3> [https://github.com/pybee/toga](https://github.com/pybee/toga)

</comment><comment><USER4> Sounds like a great proposition, but a cursory search on Google found no specifics, and I am not going to listen to hour and a half (!) of talk just to get the feeling of the state of things.

So, does anyone know if this is feasible, or should I stick to Android + iOS native for now?

<comment><USER5> I've listened to the podcast before, it's mostly about the projects in this github organization: [https://github.com/pybee/](https://github.com/pybee/)

One of these projects is the "toga" toolkit, which maps to native toolkits on the platforms it runs on. On iOS, this is done through the FFI (because objective-c uses the same calling convention as C, apparently), on Android it's done by compiling the Python bytecode to JVM bytecode, and calling native Java-level methods on the existing GUI toolkit. (That's what the "voc" project does).

I haven't tried it. To me, it all sounds promising, but some of it might be in early-ish stage.

</comment></comment><comment><USER6> really interested in checking this out. I keep trying to pick up Kivy, but I have found the documentation(kivy) is really hard to understand.</comment>


Do you want to be bought by Yahoogle? - <AUTHOR>

Money is a nice thing.  No need to elaborate on that.  But since this is an incubator site I think it's a valiant question to ask actually, because as engineers or entrepreneurs, I think you or me or we would like to differentiate ourselves from those in med school and law school.
Talk to anyone in those schools and they will readily tell you that the law and medicine are wholly secondary to the goal of making piles of money.
Not enough money.  Piles of money.
If software and the products we create with it are for the exact same purpose.  We as a community will end up with the exact same result as those people.
So ask yourself if thats your gig, and maybe "are we already there".  And when you say you want to change the world, maybe stop bullshitting yourself a little bit. <comment><USER2> "And when you say you want to change the world, maybe stop bullshitting yourself a little bit."

I've already mentioned this in another post. That line above is simply a keyphrase to attract VCs. Nobody is changing the world with the next cool-dog- food-pic-sharing-6-second-iOS-app.

There's 3 things humans desire: fame, power, wealth

It's tough determining which one is the most sought after, but I would say that it all begins with wealth first (this is in the general sense, in terms of regular folk).

</comment><comment><USER3> I'd have no problem getting bought by Google, Microsoft, Apple, etc. if it meant my technology/product could get much broader reach and success than it would on its own; skipping 5-10 years of growth and risk would be worth putting up with some corporate BS.

</comment><comment><USER4> I'll take the piles of money, thanks. No, wait, it's not exactly what you're thinking... It's not that I want to be a "sell out", but if I sold my company for piles of cash, I could do a lot of the things I want to do anyway (depending on how big the piles are) AND then I'd have the financial security and the seed capital to launch my next thing. And knowing that I would have the luxury of already having my personal financial situation taken care of, I could be even bolder next time.</comment>


Netflix Conductor: Open-source workflow orchestration engine - <AUTHOR> https://netflix.github.io/conductor/ <comment><USER3> I set up Conductor where I work while evaluating workflow engines, and overall wasn't too happy with it. The default datastore is this Netflix-specific thing (Dynomite) that's built on top of redis. It's not particularly easy to integrate operationally into non-Netflix infrastructure, and Conductor itself hard dependencies on several services.

The programming model for workflows/tasks felt a little cumbersome, and after digging into the Java SDK/Client, I wasn't impressed with the code quality.

We did have some contacts at Netflix to help us with it, but some aspects (like dyomite itself, and its sidecar, dynomite-manager) felt abandoned with unresponsive maintainers.

We've started using Temporal[0] (née Cadence) recently, and while it's not quite production-ready, it's been great to work with, and, just as critically, very easy to deal with operationally in our infrastructure. The Temporal folks are mostly former Uber developers who worked on Cadence, and since they're building a business around Temporal, they've been much more focused and responsive.

[0] [https://temporal.io/](https://temporal.io/)

<comment><USER4> The Temporal founders worked on AWS SWF too before building Cadence at Uber. They have a lot of experience in this area and are making the product better with each iteration no doubt. I enjoyed using Cadence at Uber and definitely sad about not having it at my current company.

One of the founders, mfateev, is around elsewhere on this thread answering questions.

</comment></comment><comment><USER5> Workflows and orchestration are my jam -- that's what we're trying to simplify over at [https://refinery.io](https://refinery.io)

Conductor is a cool piece of tech, and it's a well-established player in a rapidly growing space for workflow engines.

I used to work at Uber and that company had microservice-hell for a while. They built the project Cadence[0] to alleviate that. It is similar to Conductor in many ways.

One project to watch out for is Argo[1] which is a CNCF-backed project.

There are also some attempts[2] to standardize the workflow spec.

Serverless adds a whole new can of worms to what orchestration engines have to manage, and I'm very curious to see how things evolve in the future. Kubernetes adds a whole dimension of complexity to the problem space, as well.

If anybody is interested in chatting about microservice hell or complex state machines for business logic, I'd be excited to chat. I'm always looking for more real world problems to help solve (as an early stage startup founder) and more exposure to what others are struggling with is helpful!

0: [https://github.com/uber/cadence](https://github.com/uber/cadence)

1: [https://argoproj.github.io/argo/](https://argoproj.github.io/argo/)

2: [https://serverlessworkflow.github.io/](https://serverlessworkflow.github.io/)

<comment><USER6> Hey, wait a second. Are these just a modern incarnation of the enterprise service bus? Is there a significant difference?

<comment><USER7> Yes. The only difference from what I can tell is that each service/node on the bus is a VM/Container rather than a bespoke built machine.

<comment><AUTHOR> dragonwriter below you says the opposite - based on a plane-level analysis. would be interested if that changes your mind.

<comment><USER7> The real answer is it depends on what you think what responsibilities exactly ESB or Workflow Orchestration should have. dragonwriter is of the reasonable opinion that ESBs focus should be a message plane. If your of my opinion, ESBs generally include some of the features found in workflow. devonkim is also right in that ESBs in general let the business data/process bleed into the rest of the stack, where as with workflow, its completely agnostic to the business data/process.

The most accurate answer would be from comparing various ESB products to Workflow Orchestration products, as every vendor/product will have slightly different opinions on where their responsibilities are.

<comment><AUTHOR> fair enough. appreciate the thoughts!

</comment></comment></comment></comment></comment></comment><comment><USER8> Quick notes from skimming the docs:

* Conductor implements a workflow orchestration system which seems at the highest level to be similar to Airflow, with a couple of significant details.

* There are no "workers", instead tasks are executed by existing microservices.

* The Orchestrator doesn't push work to workers (e.g. Airflow triggering Operators to execute a DAG), instead the clients poll the orchestrator for tasks and execute when they find them.

My hot take:

If you already have a very large mesh of collaborating microservices and want to extract an orchestration layer on top of their tasks, this system could be a good fit.

Most of what you're doing here can also be implemented in Airflow, using an HTTPOperator or GRPCOperator that triggers your services to initiate their task. You don't get things like pausing though. On the other hand, you do get the ability to run simple/one-off tasks in an Airflow operator, instead of having to build a service to run your simple Python function.

I'm unsure on whether push/pull is better; I think it largely depends on your context. I'm inclined to say that for most cases, having the orchestrator push tasks out over HTTP is a better default, since you can simply load-balance those requests and horizontally scale your worker pool, and it's easier to test a pipeline manually (e.g. for development environments) if the workers respond to simple HTTP requests, instead of having to provide a stub/test implementation of the orchestrator. (In particular I'm thinking about "running the prod env on your local machine in k8s" \-- this isn't practical at Netflix scale though.)

<comment><USER9> Is there a workflow tool that's designed with micro service in mind?

My particular use case: \- several workers process the data on the workers' local threads \- several workers serve as relays to interface with external third party services, hold all the necessary credentials, and conduct cron- like checking. \- the ETL tool doesn't directly provision these workers.

The second point is part of the reason why we don't want to use Airflow's k8s operator.

But it doesn't seem like there is a better option in terms commonly used and robustness. So we are leaning towards write some custom operators and sensors to make Airflow more friendly to micro services.

Thought?

</comment></comment><comment><USER10> We've used Conductor at my workplace for about a year now. The grounding is pretty solid but the documentation is pretty pants once you dig into it. We have to resort to digging into github issues to find fairly fundamental features that aren't really documented. I feel Conductor is something Netflix has open-sourced and then sort of dumped on the OS community.

For example there isn't any examples of how to implement workers using their Java client, we had to dig up a blog post to do that, although it is fairly simple a very basic example of implementing the Worker interface would be nice.

They also do not make it clear the exact relationship between tasks and workflows and it's hard to find any good examples of relatively complex workflows and task definitions available on the internet other than Netflix's barebones documenatation and the kitchen-sink workflow they provide, which is broken by default on the current API.

Also the configuration makes mention of so many fields that are pretty much undocumented, like you can swap out your persistence layer for something else but I would have no idea how that works.

</comment><comment><USER11> Suprised to see Camunda isn't mentioned here more.

Open-Source BPMN compliant workflow processing with a history of success. Goldman Sachs supposedly runs their internal org with it.

Slightly different target use case, but Camunda has really shined in microservices orchestration and I find implementing complex workflow and managing task dependencies much easier with it.

<comment><AUTHOR> do you have some recommended resources to learn more about BPMN? what's your take on BPMN vs other approaches? (JSON or cadence/temporal style "workflow as code")

</comment></comment><comment><USER12> Very interesting. Looks a lot like zeebe [0], which uses BPMN for the workflow definition. This makes it easier to communicate the processes with the rest of the company. I never used it in production, just played around with it for a demo.

[0] [https://zeebe.io/](https://zeebe.io/)

<comment><USER8> I've looked at Zeebe, and Camunda too - likewise, just in a demo capacity.

Interested in folks' experiences deploying these tools, as this sounds like a potentially very useful way of modeling business workflows that span multiple services.

<comment><USER13> I've used Conductor, Zeebe, and Cadence all in close to production capacity. This is just my personal experience.

Conductor's JSON DSL was a bit of a nightmare to work with in my opinion. But otherwise, it did the job OK-ish. Felt more akin to Step Functions.

Arguably, Zeebe was the easiest to get started with once you get past the initial hurdle of BPMN. Their model of job processing is very simple, and because of that, it is very easy to write an SDK for it in any language. The biggest downside is that it is far from production ready, and there are ongoing complaints in their Slack about its lack of stability, and relatively poor performance. Zeebe does not require an external storage since workflows are transient, and replicated using their own RocksDB, and Raft set-up. You need to export, and index the workflows if you want to keep a history of it or even if you want to manage them. It is very eventually consistent.

With both Conductor, and Zeebe however, if you have a complex enough online workflow, it starts getting very difficult to model them in their respective DSLs. Especially if you have a dynamic workflow. And that complexity can translate to bugs at an orchestration level which you do not catch unless running the different scenarios.

Cadence (Temporal) handles this very well. You essentially write the workflow in the programming language itself, with appropriate wrappers / decorators, and helpers. There is no need to learn a new DSL per se. But, as a result, building an SDK for it in a specific programming language is a non-trivial exercise, and currently, the stable implementations are in Java, and Go. Performance, and reliability wise, it is great (relies on Cassandra, but there are SQL adapters, though, not mature yet).

We have somewhat settled on Temporal now having worked with the other two for quite some time. We also explored Lyft's Flyte, but it seemed more appropriate for data engineering, and offline processing.

As it is mentioned elsewhere here, we also use Argo, but I do not think it falls in the same space as these workflow engines I have mentioned (which can handle the orchestration of complex business logic a lot better rather than simple pipelines for like CI / CD or ETL).

Also worth mentioning is that we went with a workflow engine to reduce the boilerplate, and time / effort needed to write orchestration logic / glue code. You do this in lots of projects without knowing. We definitely feel like we have succeeded in that goal. And I feel this is an exciting space.

<comment><USER8> Thanks for the thoughtful reply, this is very useful.

The concept of having business users able to review (or even, holy grail, edit/author) workflows was one of the potentially appealing aspects of the BPMN products; did you get a signal on whether there were any benefits? "the initial hurdle of BPMN" sounds like maybe this isn't as good as it seems on the face of it?

Also, how do you go about testing long-lived workflows? Do any of these orchestrators have tools/environments that help with system-testing (or even just doing isolated simutions on) your flows? I've not found anything off-the- shelf for this yet.

<comment><USER13> You raised a pretty good point about being able to review the BPMN. I did not immediately think of this, but now that you have mentioned it...

1\. It was good for communicating the engine room

I remember demo'ing the workflows within my team, and to non-technical stakeholders. It was very easy to demonstrate what was happening, and to provide a live view into the state of things. From there, it was easy to get conversations going, e.g. about how certain business processes can be extended for more complex use-cases.

2\. It empowered others to communicate their intent

Zeebe comes with a modeller which is simple enough even for non-technical users to stitch together a rough workflow. The problem is, the end-result often requires a lot of changes to be production-ready. But I have found that this still helps communicate ideas, and intent.

You do not really need BPMN for this, but if this becomes the standard practice, now you have a way of talking on the same wavelength. In my case, we were productionising ML pipelines so data scientists who were not incredibly attuned to data engineering practices, and limitations, were slowly able to open up to them. And as a data engineer, it became clearer what the requirements were.

On the point about testing, the test framework in Zeebe is still a bit immature. There is quite a few tooling / libraries in Java, but not really in other languages. The way we approached it was lots of semi-auto / manual QA, and fixing live in production (Zeebe provides several mechanisms for essentially rescuing broken workflows).

The testing in Cadence / Temporal is definitely more mature. But you do not have the same level of simplicity as Zeebe. That said, the way I like to see it / compare them, you could build something like Zeebe or even Conductor on Cadence / Temporal, but not vice versa.

</comment></comment></comment></comment></comment><comment><USER14> Can someone explain how and where to use a Workflow Orchestration Engine ?

<comment><USER15> These are useful for tasks that can last an arbitrarily long amount of time. Think about the process of signing up a user and waiting for her to click on a email verification link. This process can literally never end (the user never clicked) but more commonly it takes a few minutes.

It's easier to implement these things if you can write the code like:

       await sendVerificationEmail();
       await waitForUserToClick();   // This could take forever
       await sendWelcomeEmail();

If you do the above in a "normal" program, said program could stay in memory forever (consuming RAM, server can't restart, etc). The workflow engine will take care of storing intermediate state so you can indeed write the above code.

The other option is to implement a state machine via your database and some state column, but the code doesn't look as pretty as the above three lines.

Note that this particular tool seems to be more declarative than my example above (it uses JSON to do define the steps), so instead of using an `if` statement, you'd need to declare a "Decision".

Hope this helps!

<comment><USER16> So is it like [https://github.com/uber/cadence](https://github.com/uber/cadence), AWS SWF, Amazon Step Functions?

<comment><USER17> Yes, they all solve similar problems. This is a space with a lot of differing requirements, so they all share some common features, but many have different strengths for various types of tasks.

</comment></comment></comment></comment><comment><USER18> I prefer Power Automate / Logic Apps interface, it would be cool if there was a Power Automate open source imagine the number of plugins for that cloud tool that would come up? It's a valuable tool and part of the O365 ecosystem and could be even greater, more strategy and vision into that product would make O365 and Azure a leader in components and integrations this is the most valuable thing in the end of it all.

<comment><USER19> There is, it's called Apache nifi

It's ripe for someone to make it saas

<comment><USER18> Dude awesome tip, just saw it now thank you! Great project! Watching the videos now... this is ground breaking stuff and little marketing for now on it... impressive I'm impressed with Nifi.

</comment></comment></comment><comment><USER20> Does this have the same limitations as Airflow? How does it compare to something like Prefect?[0]

[0] [https://medium.com/the-prefect-blog/why-not- airflow-4cfa4232...](https://medium.com/the-prefect-blog/why-not- airflow-4cfa423299c4)

<comment><USER21> That blog post is a good rundown of the problems with Apache Airflow.

</comment></comment><comment><USER22> We started using it around 2016 in the company I work for. We decided to use it to automate the often manual setup of new clients for each product. It grew to use our own security and rights system, and we also added a different database support (which we are working on the open source). We also changed the Jason API to conform to our company wide standard.

At the time, we wanted something that we could host ourself, maintained, open source and that was working!

Nowadays internal teams also use it to automate their own processes as well.

We'd probably go for a "push based" workflow engine, maybe based on events, mainly for latency and load reasons, but it's something we're ok with so far (there is a way to listen to event for some tasks though, but it's not that easy)

If I'm not mistaken, Netflix uses it to automate video encoding for shows, but that might be outdated.

Overall, we're pleased about it. But here are some cons about it: we wished we could split some services out (such as read only ones, or the workflow definitions from the executions, etc, but the code isn't architecture for such an easy split: for example, pushing the result of a task computation by a worker triggers the current workflow to determine the next task to schedule, but it's doing this internally, and not through the defined interfaces) Security of the api is not so easy, as it's not really modular (unlike the database implementation, which is great). That point is being worked on though, so there is some hope for the future.

</comment><comment><USER23> I have been exploring workflow orchestration for sometime now - <USER2> Temporal. Temporal's authors don't recommend it for very high throughput (per workflow) use cases, although I haven't benchmarked it myself. Also using it in a SaaS environment, I would prefer some serverless deployment strategy which possibly allows scaling down to zero.

I have my eyes on flink stateful functions [http://statefun.io/](http://statefun.io/) The abstractions are quite low level as compared to Temporal but the overall ability to write tasks/activities as serverless functions which have access to state is quite attractive.

Would be happy to talk to someone who has explored this further.

</comment><comment><USER24> For better or worse, we ended up creating our own workflow engine at my company. Unfortunately, everyone who ends up using it hates it. We've also run into the problem where the entire process of producing our end product is encoded in the workflow. Downstream steps depends on earlier steps etc. If any part of the process changes, managing this data becomes tough.

Additionally, we have software engineers writing these workflows. Ideally we would have tooling so that those who know the process can write these things. The difficulty we have had though is making it easy to join/match up earlier parts of the process with later steps. We do this now by keeping a lot of data in the workflow and by occasionally persisting data in other places. Software engineers, not the process people, are the ones who understand the data model and how to munge everything together.

Have others dealt with this issue?

<comment><USER25> I've been really interested in creating my own workflow engine for personal use and I kind of had the same thought while trying to plan it out.

One approach that comes to mind to solve this sort of end user issue would be to explicitly define all inputs/outputs for the dataflow of a node as "requires" and "provides". In this manner each node would run in parallel the moment their requirements (ie. dependencies) are met. Additionally, since what each piece of data a node needs and provides is explicitly defined you could technically automatically wire nodes together without having to even really needing know the underlying data model itself.

So it just means needing to define clear unique labels for each port. In a UI a user could just drop nodes into a space which would automatically wire up to matching ports. You can then display which needs are not met and even have an interface for choosing matching nodes that fit what might be missing.

In the end all you would really need to know is how to compose the pieces of logic to get the desired outcome.

I'm a novice at this stuff though so take that with a grain of salt.

<comment><USER24> Yup in theory it makes sense. We do define inputs/outputs for each task and workflow but it's somewhat crude; for instance, we just check if the variable name will be available but not if a specific key is in a dictionary. We can definitely improve this though with schemas (probably json schema) and validation.

Wiring this together _without any user input_ (which is our goal) though is very, very hard. Let's say we have two parallel branches-- one that orders boxes with holes in them and another that orders shapes that can fit in those holes. When both orders are in the workflow can join and we want to put the shapes into boxes. How do we know which shape can go in each box? Maybe we have a BoxType with a list of possible shapes. This gets very complicated though when you have many attributes and care about different attributes at different stages of the process. Additionally, if the process should update the database at some point to, say, change a flag from `false` to `true` the user would need to know the underlying data model.

<comment><USER25> So if it's a long running task that's definitely a more complex problem that would require some sort of queue/pause/resume system presumably but I imagine the same concept could still apply.

The node that merges context would simply have a requirement/dependency on the results "provided" by each branch. It would wait to execute until those requirements were met.

The user wouldn't need to know about the data model still, the "merge node" would just intrinsicly wait for the results provided form the seperate branches.

Fundamentally when each node completes the system itself just needs to check against the list of nodes to see if its dependencies have just been met and keeps track of which nodes have been already executed so that they don't end up getting triggered again when the next check happens.

There will always be a need for some sort of workflow level state or context management that governs all of this orchestration that you would want to persist to a database somewhere if this is a long running workflow but this is a systems concern and the user doesn't need to know about it.

That was just a long way of me more or less saying that it doesn't matter how many branches there are, all that matters ultimately is that a node waits to execute when its requirements are met.

<comment><USER24> I guess what I'm trying to say is that the logic in the merge node is where the magic happens. So you have parallel branches, A and B. A orders boxes while B orders shapes. Somewhere in A and B we created Box and Shape instances and those are outputs of A and B, respectively. The merge node, C, waits until A and B are completed and takes their outputs as input. The merge node needs to know how to match up the Box and Shape instances to say shape 2 can fix in box 1.

</comment></comment></comment></comment></comment><comment><USER26> Since this has come up several times, Airflow is an orchestration tool for ETL jobs (long running complex processes) and Netflix Conductor is an orchestration tool for micro-services (short running simple processes).

</comment><comment><USER6> Seems neat. I guess this partially solves the problem of having some workflow stuck/dropped.

I wonder how much overhead there is. How much latency does each task cause?

Is it feasible to complete workflows while a user/client is waiting for a RESTful response?

</comment><comment><USER23> There is also [https://github.com/dapr/workflows](https://github.com/dapr/workflows) which uses Azure Logic Apps engine on [https://github.com/dapr/](https://github.com/dapr/)

</comment><comment><USER27> > Almost no way to systematically answer "How much are we done with process > X"?

is this a typo?

</comment><comment><USER28> Any difference with airflow?

</comment><comment><USER29> If curious see also

2016 [https://news.ycombinator.com/item?id=13174743](https://news.ycombinator.com/item?id=13174743)

</comment><comment><USER30> It's great to see corporations getting more involved in open source software; giving back and empowering the developer community.

</comment><comment><USER31> can somebody ELI5, why would someone need such a workflow orchestration engine? What problems are best solved with workflow engines?

<comment><AUTHOR> really good answer in another comment [https://news.ycombinator.com/item?id=24215303](https://news.ycombinator.com/item?id=24215303)

</comment></comment><comment><USER32> How does this compare with Cadence/Temporal?

<comment><USER33> You need to specify a DAG, rather than just write regular code

</comment></comment><comment><USER34> I wonder what the relationship is to StackStorm[1]? StackStorm is older and lists Netflix as a sponsor / user.

[1] [https://stackstorm.com/](https://stackstorm.com/)

</comment><comment><USER35> Comparison with Airflow will be helpful.</comment>


XKCD 1190 is now more individual frames than there are XKCD comics. - <AUTHOR> http://geekwagon.net/projects/xkcd1190/ <comment><USER2> If only they could find an old shed door to cross the river.

The similarities between this ongoing xkcd and _why's most recent publican are really interesting. Thematically I find the stories very similar, they feel as though they come from the same author to me (I know Randall is not _why). Their choice of delivery also come to mind.

I am very happy there are more dedicated people than myself in the world, otherwise I would miss out on these stories unfolding.

<comment><USER3> _> I know Randall is not _why_

Now _that's_ a conspiracy theory I can get behind.

<comment><USER4> That Jack Black looking dude really _is_ Jack Black. Randall hired him to play his alter ego, _why, in public appearances.

</comment></comment></comment><comment><USER5> The (now) 621 page thread on xkcd forums is essential reading:

[http://forums.xkcd.com/viewtopic.php?f=7&t=101043](http://forums.xkcd.com/viewtopic.php?f=7&t=101043)

<comment><USER6> On page 1:

"What an interesting comic. This is probably the most simplistic xkcd yet..."

<comment><USER7> Also:

"I don't really get it. Nothing is happening. Yet I am sitting here, staring at it. Thinking about what those two might be thinking about. Maybe they just talked about something, something important perhaps. Maybe they're content just to sit there, together in silence. Watching time happen.

Somehow, I feel that this is the strongest xkcd yet. It's in my head."

People like what they want to like, I guess.

<comment><USER8> Well, that's the fun of art.

A lot of comes from playing with your expectations. I was expecting a somewhat humorous geeky pun or reference; about 90% of XKCD is figuring out the reference. What does he mean by "Time"? They're just sitting there. Where are they sitting? What are they looking at? What's their relationship like? This reminds me of times when I too have just sat there, with a friend or a girlfriend or whatever, contemplating.

And in the meanwhile Randall has just made you _FEEL EMOTIONS_.

I used to live near this grafitti: <http://instagram.com/p/WcpTGfSzx1/> and I would often go and stand and quietly contemplate it while I had my morning coffee and cigarette. It's hard to articulate but it feels rewarding to think about what whoever the artist is had in mind when they made it.

</comment></comment></comment></comment><comment><USER9> At around frame 170 I wrote a script[1] that grabbed the images automatically as well as logging all of the file names. Was a very fun and perfectly timed project for helping me learn python. I completely forgot about it and am happy to say it's still running and has been collected the images correctly for the past two months. I've been pushing the collection of images to github as well in a separate repo[2] because at the time I couldn't find anywhere to one hit download all the images. The linked viewer however is at 1354 whilst I'm up to 1355, I'm curious as to at what point I grabbed an excess frame.

[1] <https://github.com/Lockyy/newpix> [2] <https://github.com/Lockyy/xkcdTimeImages>

</comment><comment><USER10> Some part of me wonders how he can keep up this pace and thinks that he must have a computer program generating new frames somehow. Then I look at things like the sand castle and doubt that a computer program could have come up with anything like that and assume that we'd have caught on to any formulaic repetition by now.

<comment><USER11> That's assuming he's drawing this live. Who's to say he hasn't been planning this from Day One and has been drawing the frames all along.

I know, it's a stretch but ... what if?

<comment><USER12> Randall has shown the capability to draw _enormous_ amounts before. Don't forget <http://xkcd.com/1110/> . See [http://www.wired.com/wiredscience/2012/09/how-big-is-the- xkc...](http://www.wired.com/wiredscience/2012/09/how-big-is-the-xkcd-click- drag-world/) for an analysis of how big 1110 is.

</comment></comment></comment><comment><USER13> _now more individual frames than there are XKCD comics._

That's kinda a silly claim. Most of the Time frames are part of an ongoing scene, so each new frame is just an addition on top of the old frame. Your Time viewer has a differential view against the next/previous frame so you can see it in action.

Saying it's "more frames than all the other comics" is like saying one episode of My Little Pony: Friendship Is Magic has more frames than X. Yeah, numbers can be bigger than other numbers, but they aren't the sameness of comparison.

<comment><AUTHOR> Perfectly true - it was more a point about how many drawings there are for this comic given the total number of comics he has done. So I guess I'm commenting on the amount of work he had to do. If you think about the massive 'pan and zoom' comic year or so ago, that was a similarly vast undertaking. I thought it was a catchy headline, but you make a very good point.

<comment><USER13> _I thought it was a catchy headline_

It is a very catchy headline! I think that's what made me stop and consider why it was so catchy.

The next amazing Time news would be finding out if it's still being created or if it was made all up front. We have to watch for clues to see if it references any current events (not created by Randy himself) that happened after the first Time frame.

</comment></comment></comment><comment><USER14> What´s this exactly? is this original XKCD?

<comment><USER15> <http://xkcd.com/1190/> changes every 30 minutes if I'm not mistaken. The link shows all the frames since the comic was published.

<comment><USER16> When was this comic published?

<comment><AUTHOR> March 25!

</comment></comment></comment></comment><comment><USER17> Its lack of context or goal, plus the obsession with water, is starting to read like they're on acid

<comment><USER18> The strongest theory I've heard is that it's a metaphor for Randall's wife's cancer (ie "the sea" == "the C"). Lines like "the sea can't keep making more of itself forever!" and "we don't understand how the sea works" lends a bit of credence to that.

<comment><USER19> I suspect that Randall would prefer that at least one of his comics wasn't interpreted as a reference to cancer. It must get tiring after a point.

<comment><USER20> I'm not aware of any other comics being interpreted as a reference to cancer, but there are a handful of comics directly referencing his wife's cancer. If it is a recurring theme of his, I'd say it's perfectly legitimate for people to consider it a candidate for an otherwise unexplained plot line.

<comment><USER19> The discussion on the xkcd forums frequently attributed every single comic to be a cancer reference. The forums have since been removed from the main page though. so it's not quite as visible anymore.

</comment></comment></comment></comment></comment><comment><USER21> Here's an animation of all the frames

<http://www.explainxkcd.com/wiki/index.php?title=1190:_Time>

<comment><USER22> This one is nice too: <http://xkcd.aubronwood.com/>

<comment><USER23> I like this one: <http://geekwagon.net/projects/xkcd1190/></comment></comment></comment>


Japan's Cat Island - <AUTHOR> http://www.theguardian.com/lifeandstyle/gallery/2015/mar/04/japans-cat-island-in-pictures

<comment><USER2> Could be an issue for local wildlife? [http://www.nature.com/ncomms/journal/v4/n1/full/ncomms2380.h...](http://www.nature.com/ncomms/journal/v4/n1/full/ncomms2380.html)

</comment><comment><USER3> Since it's an island, a trap/neuter/release program could be fantastically successful.

<comment><USER4> You're assuming the inhabitants consider the cats to be a problem.

</comment></comment><comment><USER5> More info / video: [http://www.ibtimes.co.uk/japan-aoshima-island-overrun-by- cat...](http://www.ibtimes.co.uk/japan-aoshima-island-overrun-by-cats-1490198)</comment>


Why programmers can't make any money: dimensionality and the Eternal Haskell Tax - <AUTHOR> http://michaelochurch.wordpress.com/2014/06/06/why-programmers-cant-make-any-money-dimensionality-and-the-eternal-haskell-tax <comment><USER2> Sounds like elitism to me. Programming is just a craft, it's the other skills you bring, called "specialization" in the article, that determines your value to the employer. If you don't bring anything other than programming skills then there are fewer jobs to choose from. If you insist on your favorite tools there are even fewer options. And if your favorite tool is that of the smart elite you'll be competing with a whole bunch of those guys so you'll get to sleep in the bed you made for yourself or maybe you'll just get to sleep on the couch.</comment>


The United States Is Now the Largest Global Crude Oil Producer - <AUTHOR> https://www.eia.gov/todayinenergy/detail.php?id=37053&src <comment><USER3> It's been fascinating to watch Saudi influence wane over the last few years as this sinks in as the new reality. We live in a completely different power structure now from 20 years ago. They're certainly still obscenely rich, but the power they had over geopolitics via OPEC is effectively gone at this point due to tar sands, renewables, and natural gas.

<comment><USER4> I wonder what the next 20 years will bring, especially with the shift to electric vehicles.

<comment><USER5> Most people I talk with in my peer group wouldn't even think of getting an all-electric car (except maybe as a secondary vehicle), due to range anxiety. However, what I think will happen over the next 20 years (with battery improvement, etc) is more and more hybrids and plugin hybrids. For example, see Ford's plans starting in 2020.

But the plugin hybrids won't be sold as "fuel efficient", but sold as "more powerful". However once people start doing most of their driving in all- electric mode, that will cause many gas stations to go out of business (since they run on razor thin margins). That will then be the push to make the leap to all electric in the following generation. In other words, I fully expect plug-in hybrids to be the bridge to all electric.

<comment><USER6> How much range do people need before there's no longer range anxiety?

People used to complain about 200 miles not being enough then Tesla released 300 miles then they said that's not enough either.

<comment><USER7> For me, it's not really the range at all, it's what you do at the end of the range. Charging stations are rare and slow, fuel stations are common and quick.

I don't see why gas-electric hybrids, with say, 20-50 miles of electric-only range aren't more popular. That would eliminate the vast majority of fossil fuel usage [1], and avoid this problem entirely. It also seems a lot cheaper than loading a car with hundreds of miles worth of batteries.

1\. [http://www.solarjourneyusa.com/EVdistanceAnalysis.php](http://www.solarjourneyusa.com/EVdistanceAnalysis.php)

<comment><USER8> The public is always reluctant to change established behavior and expectations... mass adoption requires revolutionary change that overcomes perceived functional and financial costs of switching, or evolutionary change with no perceived cost.

So EV will see mass adoption as soon as people can top up at a "charge station" in under 3 minutes and get 200+ mile range - because that's what people expect from "a car". The public doesn't care about the evolution of implementation details like gas vs electric.

The beautiful part is that once home charging is no longer required, people will be happy to charge at home! We are an adorably irrational species.

Alternatively, there is a revolutionary path: autonomous vehicles. If AI-EV offered completely hands-off transportation, no driving or maintenance required, people would happily ignore any perceived transition costs. However, this path seems unlikely in the near future.

<comment><USER9> >The beautiful part is that once home charging is no longer required, people will be happy to charge at home! We are an adorably irrational species.

The problem isn't that people are _unhappy_ to charge at home now, it's that _if_ and _when_ I need to travel more than 200 miles without stopping at home, I need supporting infrastructure (and planning to find it) or to use another vehicle.

<comment><USER10> Plus, I live in an apartment block with the under-building garage not having any plug sockets. Unless I drape an extension lead down 5 storeys and about 150m away, I can't see the upgrade path at the moment.

There probably are sockets closer, but it's not as simple as "park and charge", but either "lobby the building management and get sockets installed" or "find a hacky solution with long cables".

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> There are some question marks on how sustainable the US oil boom is as the extraction methods are quite capital intensive. We are at historic low capital price and have a fairly high oil price and still the profitability picture is not pretty.

To dig deeper see: [https://wolfstreet.com/2018/08/15/shale-profits-remain- elusi...](https://wolfstreet.com/2018/08/15/shale-profits-remain-elusive/)

<comment><USER12> So is most of the new oil coming from shale then? Or is a lot of it coming from hydraulic injection (fracking)?

<comment><USER13> Fracking is the technology used to extract shale oil and gas.

</comment></comment></comment><comment><USER14> This ignores the fact that Saudi Arabia has a large spare capacity that it doesn't deploy - except when it's absolutely necessary.

[https://www.reuters.com/article/us-oil-opec-saudi- trump/can-...](https://www.reuters.com/article/us-oil-opec-saudi-trump/can- saudi-arabia-pump-much-more-oil-idUSKBN1JR1HI)

<comment><USER15> That's right. This article is about the production of crude oil, not necessarily the capacity for production.

</comment></comment><comment><USER16> Good on them. But what a sorry state to be in, when global warming and sea level rise related problems already are a reality in Miami or - of all places - Norfolk's naval base.

<comment><USER17> 10 cm are already a problem? did I miss some news / tides?

<comment><USER18> A) It's a foot compared to historical levels, it's 10cm since the 1990s.

B) It's a constant problem in Miami where high tide is now above the level of many roads so they've had to jack the roads up several feet more.

C) Venice, Italy is routinely flooded because of sea-level rise and it's only getting worse.

<comment><USER17> ok 30 cm is a lot.

one thing that I don't fully understand yet: there were always storm surges above 2m (?) how come that 50cm sea-rise without any storm surges are already causing problems?

<comment><USER19> An extreme event that happened once every few years (or decades) might be tolerable, and become a vastly larger problem if it starts happening once a month instead.

New York recovered from Sandy, but the city would need massive changes if that water level became routine.

</comment></comment></comment></comment></comment><comment><USER20> That's odd. The gwfp.com link says:

"the Permian region in eastern Texas and western New Mexico,"

that's got it backwards actually (western TX and ...).

And the _current_ EIA's Short-Term Energy Outlook (reference by gwpf.com) says nothing about either Russia or Saudi Arabia.

I'd question the gwpf's research.

The latest weekly US production figures are here:

[https://www.eia.gov/petroleum/weekly/crude.php](https://www.eia.gov/petroleum/weekly/crude.php)

(3rd chart down - look at number immediately below).

In the most recent wk, 9/7, production dropped back from 11 to 10.9.

Production for SA, Russia and the US have been nip and tuck for a while and (provided capital and markets hold up) the US will likely surpass the other two - but maybe not just yet.

</comment><comment><USER21> Meanwhile and in other news, MCC:s carbon clock indicates that we just shattered the 1.5 degrees warming limit. [https://www.mcc- berlin.net/fileadmin/data/clock/carbon_clock...](https://www.mcc- berlin.net/fileadmin/data/clock/carbon_clock.htm) (Calculations according to IPCC:s most probable scenario).

</comment><comment><USER22> Not that I doubt that they're right about this data point, but interesting choice of source... appears to be some sort of climate-change-denial organization?

<comment><USER23> The link is to something maybe questionable, but the information source is the EIA, the same graph is on the front page of [https://www.eia.gov/](https://www.eia.gov/)

<comment><USER24> <sarcasm ahead /> Isn't that some sort of climate-change-denial organization now?

I wonder if this mean the US will stop making war on oil rich nations, and instead have to contend with invasions now? Would certainly be a change.

(in all seriousness, I'm not advocating wars of agression on anyone - <USER2> natural resources or other reasons)

<comment><USER23> More like another Russian revolution dragging in old satellites or a regional war in the middle-east.

Less wars of aggression and more overthrowing the powers which were held there by oil profits. Mix in world powers trying to "help" and we've got something coming.

</comment></comment></comment></comment><comment><USER25> As much as I hate to admit it, a lot of the credit for that goes to George W and Dick Chaney for pushing for local production.

</comment><comment><USER26> i still have the feeling that the next big bust will come from the fracking industry by 2020. The reserves are vastly overestimated, profitability muddled by financially wizardry, and capital flow about to get squeezed as interest rates rise.

<comment><USER27> Fracking is actually far more resilient to market swings than traditional oil fields/wells. If prices for crude/natural gas drop, you just shutdown the well. Spinning it back up is far faster and cheaper than normal wells.

<comment><USER28> I work in the O&G industry and I think this is backwards. You're far more likely to see conventional wells be choked back or shut-in during price drops. That is what we saw in the recent oil price drop.

With unconventional wells (horizontal fracked wells) decline rates are so high that you will see a dramatic decrease in production by simply stopping drilling of new wells.

</comment></comment></comment><comment><USER29> There was an interesting article The Next Financial Crisis Lurks Underground [https://www.nytimes.com/2018/09/01/opinion/the-next- financia...](https://www.nytimes.com/2018/09/01/opinion/the-next-financial- crisis-lurks-underground.html)

arguing:

>...fracking could not have taken off so dramatically were it not for record low interest rates after the 2008 financial crisis. In other words, the Federal Reserve is responsible for the fracking boom.

>Frackers haven't proven that they can make money. "The industry has a very bad history of money going into it and never coming out,"

and that it may not be sustainable.

<comment><USER30> If we are talking strictly about the economic measures going into a business decision, using (cheap QE) debt to finance capital expenditures was a good idea. The profit being made off of the capex justifies it well. It is not unsustainable in the sense of a business decision.

<comment><USER29> The article argues much of the industry may not be profitable on using normal accounting and:

>the public markets have been valuing fracking companies not based on a multiple of profits, the standard way of valuing a company, but rather according to a multiple of the acreage a company owns.

A bit like valuing dot coms on eyeballs rather than GAAP profits. It all depends on oil prices and the like I guess though.

<comment><USER31> There's a difference between a company being profitable (revenues>income) and being profitable to investors (stock goes up). Oil companies are profitable, but bad investments, especially long term.

</comment></comment></comment></comment><comment><USER32> There is not much here about whether or not the US can sustain this position long term but in the short term this development puts in an advantageous position geopolitically. NATO is very reliant on Russian oil and if the US is a top producer we can export more to Europe, although Russia will most likely remain the preferred exporter.

Also, this allows us to have a more aggressive stance with Saudi Arabia, especially when it comes to their disastrous human rights record in Yemen. Although the issue with that is that the current administration has shown no interest in SA's human rights record and defense contractors would lobby strongly against any sort of stirring of the relationship.

<comment><USER33> _...their disastrous human rights record in Yemen._

"Their"? That should be "our". Every bomb and bullet says "Made in USA", and has since 2015. Oh and of course we've also had troops there since then as well. Who in either branch of the mainstream party do we expect to pressure KSA about the atrocities in Yemen? How about the media? Oh, wait:

[https://fair.org/wp-content/uploads/2018/07/MSNBC-Yemen- Dani...](https://fair.org/wp-content/uploads/2018/07/MSNBC-Yemen-Daniels.png)

</comment></comment><comment><USER34> I take pleasure in knowing that on another timeline the US is a truly great nation that has managed to completely remove its dependency on fossil fuels, and is now switching its focus to helping poorer nations do the same.

It's just a shame we are all stuck on this timeline...good for them though!

<comment><USER35> Hey now, oil is fantastic. The U.S. will be able to use the proceeds from this bounty of natural resources to fund an excellent public health system, just like Norway!

<comment><USER34> Watch out, jokes get downvoted, this isn't Reddit ;)

<comment><USER36> Depends on the intellectual level of the joke ...

</comment></comment></comment></comment><comment><USER37> Looking at China and India, I forget how big the US actually is compared to an average country. Same goes for our abundant natural resources which we don't really talk about much (in the media at least).

<comment><USER38> I'm curious what you mean relative to China. China is almost the same size as the US (10 million square kilometers) as the US, though of course normal distorted projections make the US look much larger.

<comment><USER39> With over a billion people! That's the point the US is huge, massive urban areas and modern wealth yet it's old fashioned commodity wealth is still massive.

Easy to forget the sheer scale.

</comment></comment></comment><comment><USER40> Daniel Yergin is an energy analyst who has written several good books on oil and energy, The Prize and The Quest. The Prize is a history explaning how to got to 1990. And The Quest explains how we got to 2011. By the end of The Quest, you will understand how we got to here. Why Russia is a major natural gas provider to Europe. Why Venezuela's economy is collapsing. Why Saudi Arabia may follow. And mostly importantly, why the US is now one of the top energy exporters in the world (TL;DR we have tons of resources and the capital and business structures to exploit them).

Right now the US seems to be using oil exports to weaken other countries, rather than banking the oil and exploiting it when supply starts to dry up. I also think that some methods aren't long-term a good idea.

<comment><USER41> _Right now the US seems to be using oil exports to weaken other countries, rather than banking the oil and exploiting it when supply starts to dry up. I also think that some methods aren 't long-term a good idea._

I will second the book recommendations and I will also note that by far the best thing the U.S. can do in the short term is switch to electric and plug-in hybrid vehicles. (And I mean "vehicles" in the broadest sense, including but not limited to cars: [https://www.vox.com/the- goods/2018/9/10/17631318/electric-sc...](https://www.vox.com/the- goods/2018/9/10/17631318/electric-scooters-bird-city-regulations- sustainability)). That will cut the influence of bad political actors considerably. And almost everyone can, as individuals, take important steps in that direction.

<comment><USER12> Purchasing an electric vehicles causes more pollution than simply driving your exiting vehicle if it's relatively fuel efficient.

Drive your current car until the wheels fall off and then, if you have to buy a hybrid or electric, try to buy used.

<comment><USER42> That was my conclusion, too. The personal economics and the carbon footprint correlate surprisingly strongly. The CO2 costs of hybrids look attractive on a 5-6 year timescale, but it gets bad when the environmental costs of a new set of batteries get factored in. People underestimate the various costs of the vehicle itself, and how they look over a longer haul.

I just threw in the towel on my 20 year old vehicle for a new small gasoline engine car. I figure my odds of getting >12 years out of this vehicle are quite high.

Maybe the picture will be very different in 10 years, with improvements to battery technology. But today I stayed away from the hybrids and electric vehicles, for both fiscal and environmental reasons.

</comment></comment></comment></comment><comment><USER43> I am going to side track the discussion a bit, what are the implication of this to Petrol Dollar as the world's reserve currency?

One of the reason US dollar is being used as reserved is its absolute influence / Protection in Oil and OPEC. Now US is producing more Oil than anyone else, China is buying more from and start to settle those deals in RMM, what happen next?

</comment><comment><USER44> So if USA is producing so much oil why is it not able to pay for things like Healthcare? Presumably oil coming out of the ground is free. Where is that money going?

<comment><USER45> > So if USA is producing so much oil why is it not able to pay for things like > Healthcare?

We have 'better' things to spend money on, like 'defense' (/s).

> Presumably oil coming out of the ground is free.

It's not free, it costs money to discover, extract, transport, process into anything usable, transport again, etc.

> Where is that money going?

Probably the companie(s) that discover, extract, transport, process, transport, etc. And their shareholders. And politician friends.

<comment><USER46> The US spends plenty on health care, we just don't get good value. Medicaid and Medicare alone are 1.5x defense spending.

</comment></comment></comment><comment><USER47> As a side note, the GWPF (Global Warming Policy Forum / Foundation) is a climate change denialist organisation set up by Nigel Lawson (UK politician of the Thatcher years) [0]:

> On 23 November 2009 Lawson became chairman of a new think tank, the Global > Warming Policy Foundation, a registered education charity, involved in > promoting climate change scepticism.

[0]: [https://en.wikipedia.org/wiki/Nigel_Lawson#Position_on_globa...](https://en.wikipedia.org/wiki/Nigel_Lawson#Position_on_global_warming)

<comment><USER48> We changed the url from [http://www.thegwpf.com/the-united-states-is-now-the- largest-...](http://www.thegwpf.com/the-united-states-is-now-the-largest- global-crude-oil-producer/) to the government page it points to (where by "pointing to" I mean copying).

</comment></comment><comment><USER49> Thanks, Obama.

</comment><comment><USER50> This is a political disaster. Fracked crude is only profitable because the externalised costs of the catastrophic environmental damage are born by others.

This is another direct wealth transfer from the general population i.e. 99% to the 1%.

If instead of allowing fracking we invested massively in renewables we'd have more jobs, a path to much-cheaper-than oil energy, and we wouldn't be leaving toxic residue all over our water tables to make our children and grandchildren sick with any number of diseases.

As usual capitalism is taking the quick easy cash with large externalities because they aren't affected by them.

<comment><USER51> Fracked crude might not even be first-order profitable. There is an assertion here by a book author being interviewed that the fracking industry is largely not long-run profitable and that only low interest rates allowed them to run speculatively at all.

[https://www.marketplace.org/2018/09/11/economy/economics- fra...](https://www.marketplace.org/2018/09/11/economy/economics-fracking)

This was new to me, and I don't really know if it's a correct assessment. But it's very interesting to mull over. I wonder if anyone has other information in this vein?

<comment><USER52> I'm not seeing where he says they are not long-run profitable, only that the oil industry is hard to predict, which has been obviously true for decades.

American shale plays can be profitable down to $40/bbl

<comment><USER51> The author is a she, and when she says this:

"And what fascinated me about this industry is it doesn't make money. These companies lose billions of dollars. They've never produced free cash flow. They're really dependent on Wall Street's willingness to fund them. "

That's where I made the interpretation that the idea being presented here is that fracking is not long-run profitable. So I now have to wonder, when "profitable" at $40/bbl is quoted, does that might mean some form of operational profit but not payback of capital layout - or maybe not including debt service or equipment wear out if she's talking about free cash flow.

</comment></comment></comment></comment><comment><USER53> Energy independence achieved.

Except it hasn't been at all, because much of the oil is more profitable to sell and refine abroad than use domestically. So if we actually wanted to force independence we'd need to prohibit companies from exporting their oil.

I doubt that's ever happening. Watch the "energy independence" talking point evaporate now that the new millionaires have been minted. It's almost as if that's what it was always about, isn't it?

<comment><USER54> We're energy independent in the 1970's sense, which is when the term originated. We are no longer subject to the sort of oil embargo OPEC initiated in 1973 over Western nations' support of Israel, because the West now has sufficient capacity to plug any gap in supply.

<comment><USER55> I'm not sure about that. The US still consumes a lot more than it produces. It might be the largest oil producer by a little, but it is the largest oil consumer by a lot. Looks like, net of exports, US production needs to rise several million barrels a day more to be really capable of meeting its own supply. Unless you are thinking of other Western countries that have enough surplus to cover the difference for the rest?

Relevant data: [https://www.eia.gov/dnav/pet/pet_sum_snd_d_nus_mbblpd_m_cur....](https://www.eia.gov/dnav/pet/pet_sum_snd_d_nus_mbblpd_m_cur.htm)

</comment></comment></comment><comment><USER56> The word "producer" is misleading. No new oil is being produced. Should be largest "Extractor." Too bad. Wouldn't it be better to buy from other countries and leave your reserves in the ground? In the future, once all the easy to extract oil is gone, the US will have to buy from abroad at potentially higher prices.

<comment><USER57> It's not the US extracting (or buying) oil, it's private companies. Who can make a profit today.</comment></comment>


Rat Robot Beats on Live Rats to Make Them Depressed - <AUTHOR> http://spectrum.ieee.org/automaton/robotics/medical-robots/ratbot-beats-on-live-rats-to-make-them-depressed#.USPi5ZZC5ws.hackernews <comment><USER2> What are the implications for our understanding of how to induce depression in humans?</comment>


Consciousness: Where Are Words? - <AUTHOR> http://www.nybooks.com/daily/2017/12/27/consciousness-where-are-words/ <comment> bra-ket the author's Phd thesis "Intentional robots ", 2007:

[https://www.researchgate.net/profile/Riccardo_Manzotti/publi...](https://www.researchgate.net/profile/Riccardo_Manzotti/publication/242281589_Intentional_robots_The_design_of_a_goal- seeking_environment-driven_agent/links/004635341619b9bc1c000000/Intentional- robots-The-design-of-a-goal-seeking-environment-driven-agent.pdf)

</comment><comment><USER2> Meaning and subjectivity come from (machine) learning, or relativity as Riccardo calls it.

I think, consciousness from how our brains learn, namely by having internalized the feedback. Which is another way of saying by modeling the world, and updating those models by constantly monitoring the effectiveness of those models.

We are self-observers, integrating our sensory information into a subjective first-person perspective.

</comment><comment><USER3> Riccardo Manzotti's website:

[http://www.consciousness.it/](http://www.consciousness.it/)

<comment><USER4> [http://www.thespreadmind.com/The_Spread_Mind_S7.php](http://www.thespreadmind.com/The_Spread_Mind_S7.php)

This is obviously false to me.*

If a robot vacuum recognizes a roadblock and stops, its inner representation of the roadblock is not merely an agreement (with whom anyway?).

That representation is internal and (normally) only accessible to the software of that particular robot (it may even could change over time - of course it would be a bad programming practice except for malwares - so no agreement is needed but only some kind of behavioral self consistency). And it have real causal power: it makes the robot stop and turn.

That representation is stored in memory cells wich are (usually in a normal household) not roadblocks.

*(AFAIK this viewpoint is called functionalism. It just somehow feels natural to me as a developer, but maybe it is just taste, and it is just as twisted for a philosopher as the above identity theory for me...)

</comment></comment><comment><USER5> For a better understanding of consciousness, thinking, knowledge, I would refer you to a short thesis correcting some of the errors of ideas about same: [http://wn.rsarchive.org/Books/GA003/English/GC1981/GA003_c03...](http://wn.rsarchive.org/Books/GA003/English/GC1981/GA003_c03.html)

"ALL PROPOUNDERS of theories of knowledge since Kant have been influenced to a greater or lesser degree by the mistaken way he formulated the problem of knowledge. As a result of his "a priorism" he advanced the view that all objects given to us are our representations..."

</comment><comment><USER6> I'm 3 sentences into the interview, and this seems absurd on its face. Does anyone take this seriously?

"When we say we are thinking, what we are actually doing is rearranging causal relations with past events, objects that we have encountered before, to see what happens when we combine them. We don't need a mental sofa to put next to a mental armchair. We allow the sofas and armchairs encountered in our past to exert an effect in the present, in various combinations."

<comment><USER7> I had the same reaction, but you have to keep reading. He's building a stack. He's not saying that thoughts and mental images don't actually exist, it's that they don't exist before the game of language is played. And the game of language is turning sensations of physical objects into auditory representations of them. There's no reason to start constructing the abstract objects yet, we're not at that layer of the 'stack'.

It's kind of like saying that when you dive down to the network layer, video doesn't exist. It's up to the application running the network to assemble all the packets to turn it into video.

Thoughts are constructed from the stream of physical experiences that get formed into words, not the other way around. It's perfectly possible for the brain to still operate without thoughts the way we experience them, in fact, deaf people think visually rather than auditorially like most of us do.

<comment><USER6> I think it's obvious that brains can operate without thoughts because animals and insects do it.

<comment><USER8> What makes you think other living beings besides humans don't have thoughts? IMHO we're just a slightly souped up version of everything else, exist along a continuity, and are not 'special' except in contrived and technical ways.

<comment><USER9> 'slightly' seems a bit understated to me. There is no other species whose members are anywhere close to sitting around typing messages to each other across hundreds or thousand of miles.

We are steering our evolution as a species outside of biology, in large part, via social and cultural institutions. What other species does that?

<comment><USER10> > What other species does that?

None. Sometimes I wonder if they know something we don't.

</comment></comment></comment></comment></comment></comment><comment><USER11> How about this question:

What is consciousness?

What is existence?

They seem to be duals of each other.

I have found that, if you are careful to state the subject and object of a sentence, then it's really hard to pose any question that doesn't have a trivial and obvious answer.

TRY IT!

So could many of these things just be an artifact of language?

<comment><USER12> Consciousness is: what it is like to experience something. Note this is different from self consciousness which is: what is it like to be something which experiences and know you are that thing.

Humans are certainly self conscious. Crabs are certainly conscious. Crystals certainly exist.

Where the boundaries are is an interesting question, but the categories certainly exist even if I can't draw a clean boundary around members.

<comment><USER11> Let's translate your sentences to illustrate what I mean.

What it is _like_? So you are comparing one entity experiencing something to another. Experiencing is a process that takes place inside the entity and among its component parts.

 _Know you are that thing_ \- every thing is itself, if it has well defined boundaries. Knowing something is just having access to that information and good reasons to believe it to be correct. Here we have a tautology - that a thing is itself - so it doesn't require much to know that. What you are probably more interested in is a mechanism that can take advantge of that knowledge in various ways. Like self preservation.

Humans are self conscious.

Crabs are self conscious too because they act with a purpose of self preservation, which requires knowledge of what is them and what is not them.

Crystals do not have this knowledge or do not have a mechanism to act on it in a way that manipulates semantic symbols.

<comment><USER13> > Crabs are self conscious too because they act with a purpose of self > preservation, which requires knowledge of what is them and what is not them.

It's a bit more complicated. Self consciousness is the ability to _contemplate_ mental states, some philosophers say. Crabs do not contemplate their own mental states, they simply act and feel.

<comment><USER11> Ok well there you go. I was basing it on your earlier definition.

So define the word _contemplate_.

But do you see how defining things and using the subject and object makes it hard to ask any of the major questions?

<comment><USER14> Guuz's comment does not change the earlier definition (regardless of who stated it.) What it does do is point out a flaw in your argument from that definition to your claim that crabs are self-conscious.

<comment><USER11> _what is it like to be something which experiences and know you are that thing._

According to this definition, crabs are self conscious.

<comment><USER14> As this is one of several places where you have asserted the same claim, one response is sufficient: [https://news.ycombinator.com/item?id=16041077](https://news.ycombinator.com/item?id=16041077)</comment></comment></comment></comment></comment></comment></comment></comment>


Ask HN: What's the biggest traffic spike you got? - <AUTHOR> Tell us your story. Please do share even if your site was still small at the time. It's always relative to your usual traffic figures.<p>Where did the crowd come from? How many requests per second did it reach at peak? How did your site handle the load? How long did the surge last? How many unique visitors came to your site during the surge? How did this affect your business? <comment> techpulse-co For me it was always from HN.</comment>


The Great Divide - <AUTHOR> http://andymangold.com/post/the-great-divide

<comment><USER2> One way to solve the problem - make a new "culture tax", let people vote how to distribute the tax between artists, legalize piracy.</comment>


Hacking Magic with Lisp and Unix - "Wizards's Bane" (fantasy novel, free) - <AUTHOR> http://www.baen.com/library/0671878468/0671878468.htm

<comment><AUTHOR> It's pretty old, but some of you might not know this. Amusing story of a computer programmer "summoned" to another world, who tries to use CS to tame the wild magic.

"backslash cd slash grep moira" isn't that much different from "abracadabra alakazam" after all...</comment>