Neo4j nabs $80M Series E as graph database tech flourishes - <AUTHOR> https://techcrunch.com/2018/11/01/neo4j-nabs-80m-series-e-as-graph-database-tech-matures/ <comment><USER2> I've been using Neo in production on GrapheneDB with py2neo and Flask for about a year. Love it.

The reason I use a graph is for consistency from my product level business logic to my implementation.

Basically, to solve my problem, I started with a set of english statements, which yielded a grammar, that I described as "objects and morphisms," (things and relationships) then implemented it in a graph - put a front end on it and built a product.

Graphs provide coherence to my problem. Could an RDBMS do this? Yes, but not without a complex intermediate query layer. I think of using a graph as analogous to specifying your problem in terms of a functional language instead of imperatively. The reason to do that is because your product is the result of maintaining consistency of an abstraction, like a DSL or a game, instead of just retrieving stored values, documents and their variations.

It's disruptive to a lot of orgs as well, since there is a lot of sunk cost in RDBMS experience, so I think the applications are all net new projects. I don't foresee anyone migrating to one, but I do see a point where majority of new products use one.

<comment><USER3> Great comment. Along similar lines, I can see GraphQL replacing traditional REST APIs for an increasingly large percent of new projects, and reaching a majority.

<comment><USER4> It's interesting that you mention GraphQL. For me, using Neo4j and GraphQL were similar experiences, in that once I started using them they offered a more intuitive and productive approach over the "standard" tools (RDBMS and REST, respectively).

Once I learned Cypher and some common graph data modeling constructs, I found I could build more complex applications faster using Neo4j, largely because I found the graph model of my project's domain more intuitive and easier to work with than a relational database model.

Similarly, building and using GraphQL APIs has been a huge productivity win once I figured out how to build GraphQL services.

Of course, when used together, Neo4j and GraphQL have some great synergies, since it's graph all the way down ;-)

It's also worth pointing out that Neo4j has some great GraphQL integrations [1].

[1] [https://grandstack.io/docs/neo4j-graphql.html](https://grandstack.io/docs/neo4j-graphql.html)

<comment><USER5> Oh man did I struggle with Cypher to start. Conceptually I felt like I understood it but 2/3rds of my queries would go out to lunch and never come back.

Lessons learned for me:

\- start with very small datasets (mine was 10k nodes/40k edges/5 types and that still sent it to the moon in many cases)

\- get to know the query planner right away

Cypher seems to reward a more deliberate and iterative approach to learning it rather than the ol' grip it and rip it. Once I settled down a bit it came together quite nicely.

</comment></comment></comment></comment><comment><USER6> I had a toy project which housed all its data in Postgres and I needed a bit of graph traversal, really basic stuff. I almost prepared some ETL to load the data into neo4j/redisgraph, but then I found out about recursive CTEs in Postgres.

Granted, they won't let you do nearly as much as some advanced graph algorithms, but the ease with which you can use it in your operational data store is amazing. And with proper indexing, I could do a traversal in hundreds of milliseconds.

[https://www.postgresql.org/docs/current/static/queries- with....](https://www.postgresql.org/docs/current/static/queries-with.html)

<comment><USER7> Many of us who have worked in the graph database space have been tempted to use the graph abstractions on top of a relational database. It's a reasonable first step. Tables are nodes, join tables are relationships and so forth.

It's also however a bit of a dead end once you go beyond the basics. The costs of joins get worse the deeper you go, and "hundreds of milliseconds" is at least an order of magnitude slower than what Neo4j would do for you.

Once you take that major performance penalty, and then layer it into more complex graph algorithms or analytics, it gets really, really painful quickly. Granted, you might not notice this if you never needed to go further than 2-3 hops in a graph. But once you start working with graphs you're not going to want to stick to such basics.

More technical detail on the difference between a graph abstraction on top of another database, and a native graph database, can be found here:

[https://neo4j.com/blog/note-native-graph- databases/](https://neo4j.com/blog/note-native-graph-databases/)

<comment><USER8> Well, of course it would be expensive to use tables as an arc.

So say A points to B you have 3 tables right? Table 1 for A, Table(2) for B and a join table (3) showing that A points to B right? Why would you do that? What's stopping you from having one table that contains A and what A points to? So you only have 2 tables?

What if you have a node that can point to many items, a column can contain a list in postgres, so we can still have one Table containing your node data and a list of items they point to.

I'll concede that graph databases are easier to write query for, most people already struggle with basic SQL, let alone CTE and recursive CTE.

I'm yet to be convinced that a problem can't be reshaped and mapped on a traditional RDMS and yet remain performant.

<comment><USER7> > What's stopping you from having one table that contains A and what A points > to? So you only have 2 tables?

So you can do that. Suppose you denormalize the graph into a single table. Either your denormalizing, and to connect A to many B's you have to duplicate A's data in the table, or you have the constraint that the A -> B link can only have a cardinality of 1.

This would be not a good choice for nodes with many relationships to other things, say for example "Person friended Person". It might work if the cardinality was somewhat capped, like say for example "Customer ordered Product" (a common denormalization).

For a real application, you're not going to have 1 of these. You're going to end up with 20+.

> I'm yet to be convinced that a problem can't be reshaped and mapped on a > traditional RDMS and yet remain performant.

Any problem can be re-shaped to any database formalism. For that matter we can re-shape everything we're discussing for a straight K/V store like Redis. The expressive power of a database isn't at issue, because any database can store any dataset. Period.

More relevant questions are about performance and about conceptual fit for the problem. First on performance, have a look at the performance growth graph here:

[https://neo4j.com/blog/oracle-rdbms-neo4j-fully-sync- data/](https://neo4j.com/blog/oracle-rdbms-neo4j-fully-sync-data/)

If you reason from the computer science of how these systems work, this result makes sense.

From the conceptual simplicity standpoint, that's kind of a matter of personal taste and application. Can you do it all with an RDBMS? Sure. But all of these different tech niches exist because sometimes you want more than one kind of tool for the wide variety of jobs you need to accomplish.

I'd argue that it's conceptually simpler to think of your graph as nodes and relationships, rather than to remember each time which node/rel set was denormalized into one table, which node label was split out into its own separate table, what that join table was, how the key naming differed between tables, etc. etc. etc. (Because once you have a non-trivial sized graph, you'll have a lot of these, and maybe you made different decisions at different spots).

The prize if you do remember all of that is that you get to write quite complex SQL to join together the datastructures for non-trivial traversals, because substantial graph use cases that you can answer on the basis of a single denormalized table are going to be rare.

In terms of conceptual simplicity, compare a recursive join SQL query to a cypher snippet like "MATCH (user:User {login:'bob'})-[:KNOWS*..5]->(foaf)". The equivalent SQL is...difficult.

It's just a "use the right tool for the job" situation at its core.

</comment></comment></comment></comment><comment><USER9> Neo4J has been around so long -- is anyone here using it in production and really happy with it, where it's like _really_ crushing a use-case?

I feel like 90% of the applications in existence can go so far with a regular RDBMS that they never try out Neo4j... I know that's the case with me. Half the time I think I'd just try throwing Agensgraph[0] at the problem instead of jumping to the community version of Neo4J.

[0]: [https://github.com/bitnine-oss/agensgraph](https://github.com/bitnine- oss/agensgraph)

<comment><USER10> There are two kinds of graph databases, property graph and rdf semantic graph. Neo 4J is the former, which I don't think is that useful in primarily only showing some kind of relation between things. The rdf semantic graph approach, however, captures the semantics of the relationship allowing inference and queries not normally possible with rdbms.

It's an easier cultural change in thinking from rdbms to property graph, but also not a huge improvement in terms of what you can do vs an rdbms. Going with a full semantic graph where the relationship is also represented by a node with unlimited relationships, and moving and thinking in hierarchies and inference, is a complete cultural change with an impressive productivity and capability improvement not possible with rdbms. Allegrograph is a good example of a semantic graph which can handle trillions of triples.

<comment><USER11> Do you have a good example of when an RDF is superior? My looking into RDF's was summarized as "way too complex for little gain" but I admittedly was looking at it through my SQL/Neo4j lens.

<comment><USER12> I'm working on a project with old school RDF & semantic web, and I tend to concur with you. I often miss my usual RDBMS. Maybe it's the way my data is structured, but any complex display of data quickly becomes a weird maze of async calls.

I'd be happy to be proven wrong, I love graphs!

<comment><USER10> You have to think completely differently from rdbms to use graph well. It's a hard cultural change to make.

<comment><USER13> True, but a very worthwhile one.

Before getting into Neo4j (and later joining the company), I'd worked with SQL databases a lot and, at one point, as a DBA. I spent the first few months trying to draw direct analogies between graphs and tables but after a while realised that I needed to "unlearn" a load of what I thought I already knew.

It's now pretty refreshing to be able to look from a higher vantage point at the different types of data store available, and work out which is the right one for each job.

It turns out that the graph is actually the best solution more often than you'd think!

</comment></comment></comment></comment></comment></comment><comment><USER14> To everyone @ Neo4j, on behalf of the Graphistry team, big congrats!

This may help non-graph folks understand the community a bit. Neo4j has a bunch of cool bits, and it's been a pleasure watching them bring two specific "aha!" moments to customers. Our tech helps teams build scalable visual workflows that include visual graph, so we're often brought in near the beginning of a graph project, and have repeatedly seen two situations where a DB at Neo4j's quality shines:

1\. Performance: A teams starts using their existing data stores -- SQL, Splunk, etc. They'll get quite far. Often, however, they will hit some query that just cannot perform. E.g., for two bank accounts, all paths between them. For different DBs and workloads, these can be different things.

2\. Ease: Asking for something like a 360 view around a device, user, patient, account, etc. is hard in sQL - you don't know what column, table, etc. to look at. Or imagine the above shortest-paths query. Cypher makes writing this stuff EASY, so in a world where a lot of people can barely do SQL, that's a superpower.

Neo4j has been broadening by entering the scaleout world, app dev world, and adding multi-modal & ML capabilities, which are all important things and help grow the eco-system. Congrats again!

</comment><comment> frant-hartm This is great news for Neo4j, and for the thousands of organisations that are just starting to realise that they have an unmet need that graph technology can solve.

What we are seeing here is the 'commodification' of graph, a trend that happens in technology in general. Companies that launched ten years ago, on a massive investment with their own proprietary graph technology - I'm talking the likes of Twitter, Facebook and so on - today the same features could be implemented with a fraction of the investment. They'll do this by leveraging Neo4j.

This funding we'll broaden the reach of graph technology, while reducing the overall cost for individual organisations to adopt. Social networks, recommendation engines, fraud detection systems are all now easily within reach. Check out our own free and open-source recommendation engine, which was built on top of Neo4j, for example: [https://github.com/graphaware/neo4j-reco](https://github.com/graphaware/neo4j-reco).

We live in exciting times. While the commodification of what we call 'graph 1.0' is in progress, what Tesla's head of A.I. Andrej Karpathy brands "Software 2.0", that is the intersection of machine learning and software development is rapidly picking up pace. We're only at the beginning of the hype cycle on this. And guess what? IT is an established fact, is that graph is playing a central role in this transformation process.

We are proud to say that our organisation is at the forefront of using graph technology to derive insight and meaning from unstructured data - we call this GraphAware Hume. We're really excited about this!

As you can see we're are pretty passionate about graph technology, and Neo4j in particular, and in our opinion we're at the beginning of what is going to be a very transformative adoption. If you're thinking about exploring how graph might fit into _your_ organisation, of course feel free to reach out.

Disclaimer: GraphAware ([https://graphaware.com/](https://graphaware.com/)) is Neo4j's solution partner

</comment><comment><USER15> The frustrating thing with Neo4j is that they have two modes - a fairly neutered open source version, and a ~$35k/node enterprise version, with absolutely nothing inbetween. They're apparently doing a cloudy hosted enterprise version somewhere inbetween but that's some time out and not a self-hostable option.

<comment><USER16> I hear you. We're working on it. I will note that we have a startup program where you can get Neo4j Enterprise for free if you're below 50 employees and also have several other programs for educational institutions, journalists, etc. And obviously the long term solution here is a DBaaS with grow-as-you- grow pricing. Watch this space! :)

<comment><USER17> There are a ton of use cases in bigger enterprises (1000+ employees) for smaller applications that are not worth the costs you are offering currently, so I'm glad you're working on it. Cypher is amazing (as is most of Neo4j). I've used Neo4j for many years, going back to pre 2.0 versions. The database has evolved massively since then.

</comment></comment></comment><comment><USER18> People think Neo4J is for things like social, but Facebook's social graph doesn't actually work like Neo4J does – Facebook's load is dashboard shaped and read dominated, so query/storage/writer separation is really important. [http://www.dustingetz.com/:datomic-facebook- tao/](http://www.dustingetz.com/:datomic-facebook-tao/)

</comment><comment><USER19> Neo4j seems to have performance issues: [https://www.arangodb.com/2018/02/nosql-performance- benchmark...](https://www.arangodb.com/2018/02/nosql-performance- benchmark-2018-mongodb-postgresql-orientdb-neo4j-arangodb/)

And as far as I can remember from a former project the scalability is pretty limited (but this could have changed).

<comment><USER20> For those unfamiliar with the tech can you give numbers to frame your scaling challenges. Performance issues is too broad for me to rationalize in a meaningful way without more specificity.

<comment><USER21> Neo4J is mainly a one-machine database. For larger clusters, you probably want to look at ArangoDB or Dgraph, for examples. Meaning, as long as your data can be stored in one running machine, you're alright. When wanting to scale horizontally, other solutions were built with that first in mind.

<comment><USER20> That makes sense, thank you!

</comment></comment></comment></comment><comment><USER22> I have only used Neo4j for one customer project, but I experiment with it myself (I also use RDF datastores) and have used Neo4j in examples for two books I wrote. Good technology, and I am glad they are successfully transitioning from the open source community version to also making money with their enterprise version.

It is still a research thing, but I am starting to see occasional papers on inducing relations in graphs using deep learning. If this proves useful, that should help the growth of graph databases and the use of knowledge graphs.

However, as another commenter 'hardwarsofton' said, just using Postgres is very often all I need.

<comment><USER16> Hey Mark. Yours was the first ever book on AI that I read back in the 90s so thank you for that. :)

I think we're in the early parts of an exciting journey connecting (no pun intended) graphs and AI. I'm personally really excited about connected feature extraction. I wrote a little bit about it here: [https://neo4j.com/emil/80-million- series-e/](https://neo4j.com/emil/80-million-series-e/) There's more in depth info here in this graphs & AI overview video from GraphConnect last month: [https://neo4j.com/graphconnect-2018/session- topics/?topic=AI...](https://neo4j.com/graphconnect-2018/session- topics/?topic=AI%20and%20Machine%20Learning)

<comment><USER22> Thanks!

</comment></comment></comment><comment><USER23> Huge fan of Neo4j and there are many data stores that would benefit from a graph, security data being one. And Cypher is a fantastic query language for graphs and should be the standard.

</comment><comment><USER24> Neo4j, and graph databases in general, are an excellent use case for IoT access management.

Our schema involved taking physical assets/personnel and representing them as different labels: machine, factory, production line, user, usergroup, etc. We then drew complex relationships between different user/groups in the organization and the assets they were responsible for.

At first, we used a relational database, but it soon became difficult to go more granular than simply: user belongs to usergroup, usergroup belongs to client, client has factories, factories have lines, lines have machines.

As many have pointed out here, it's not that you can't do this with non-graph databases, it just requires a more complex query layer. Neo4j allowed us to represent complex business relationships as natural language, and that really helped us as the business scaled.

</comment><comment><USER25> Is "nabbing a Series E" really a sign of something flourishing? The way I view investment rounds is as a letter grade. Once you've reached a "Series F" maybe you've just failed? Has this company made any profit, for instance? Just conjecture.

<comment><USER26> You need a graph database to track all the levels of funding.

</comment></comment><comment><USER27> Nice to see a Java based technology taking off.

Java will always be king of enterprise despite all the drama with the future of the JRE.

</comment><comment><USER28> Love Neo4j. The console makes it SO freaking easy to use!

<comment><USER29> Agreed.... The console is one of the better parts of the overall tool.

</comment></comment><comment><USER30> Good to see interest in this space brewing.

I used to use OrientDB but moved away due to stability issues (long ago now so hopefully they have that sorted). I also just noticed they've been acquired by SAP!

For me graph layouts are conceptually superior vs relational when explaining to non techie users.

Practically however I now stick to Postgres - it's 'good enough' (for what I'm doing) and has a heap of benefits in and of itself.

I looked st Agensgraph but I couldn't get enough info on it, plus it is a custom version of Postgres (not a plug-in) and i think they recently switched to AGPL which makes it overall less exciting to investigate.

I know there are places where having a real graph db helps but I've not personally hit those scenarios yet.

</comment><comment><USER31> The problem with neo4j is that Cassandra is a brilliant alternative with a superb hosted story - lots of providers have hosted Cassandra which scales to petabytes levels.

And there is a huge scalable graph stack with Cassandra - Datastax Enterprise Graph, Titan,JanusGraph (where Google is involved), Tinkerpop,etc.

The production readiness of neo4j is something I'm still not quite sure about.

It truly shines as an embedded graph db though. I wonder if there is a Blockchain story around neo4j (as a replacement for leveldb) that makes this more interesting. After all there is a lot of excitement around DAG based blockchain alternatives.

<comment><USER32> Cassandra supports cypher?

<comment><USER31> No. All of the big scale graph databases usually use Tinkerpop Gremlin .

[https://tinkerpop.apache.org/gremlin.html](https://tinkerpop.apache.org/gremlin.html)

</comment></comment></comment><comment><USER33> Does anyone have experience with both Datomic and Neo4j? Are they comparable?

<comment><USER34> I do, but only in smaller side projects and I'd love to hear from someone who has experience with both in larger settings. Although they work very differently, the query language (Cypher vs. Datalog) have a very similar feel to them. When I tried them it seemed that Datomic would be the more flexible of the 2 (eg. for use cases where you'd want a bit of relational and a bit of graph support), and that Neo4j you'd end up with PostgreSQL next to Neo4j in many practical applications. I have to say though that the query/visualization UI that Neo4j provides is nice and helps with making sure your data is indeed stored in the way you want.

</comment></comment><comment><USER35> Has anyone used stuff like ontotext graphdb or stardog?

I'm also interested in peopels opinions of Neptune in that domain.

Mostly for medical reasoning on RDF OWL ontologies such as "Ontology Development and Debugging in Protégé using the OntoDebug Plugin"[1]

[1] [https://www.youtube.com/watch?v=vHmC- rRuMYM](https://www.youtube.com/watch?v=vHmC-rRuMYM)

</comment><comment><USER36> This is a little off topic, but there are so many graph databases out there. It's hard to know which one to start playing with..

One would think that Neo4j probably is the most stable one? But it's unclear to me if the full version is open source or not? [1][2]

Has anyone tried several?

Has anyone tried Dgraph? [3]

      [1] Community is limited according to Wikipedia
      [2] Trying to download enterprise takes me to a "Start a Free 30-Day Trial" page
      [3] https://dgraph.io/ ?

<comment><USER37> We're currently using it and it's got some pros/cons, though the cons are a little scary. Jepsen did an evaluation of Dgraph[0] shortly after we started using it and I can confirm seeing a lot of what was noted. Dgraph is _fast_, but there are problems with the underlying data store (their own DB called Badger). Some of those issues have been remedied, but many still exist.

As a graph database, it has some non-typical tradeoffs. You can't easily discern incoming edges and there's no true node deletion. There's a pretty narrow happy-path where the DB works as advertised/expected, but it's just a fairly young DB from an understaffed startup. Probably worth waiting a year or two for the kinks to be ironed out.

[0] [https://jepsen.io/analyses/dgraph-1-0-2](https://jepsen.io/analyses/dgraph-1-0-2)

<comment><USER38> (author of Dgraph and Badger here)

Congrats to Neo4j on the raise! I hope it changes the perception of US VCs w.r.t. graph DBs, who are falling behind the dev enthusiasm and readiness for adoption in this field.

As Jepsen report mentioned, it had identified 23 issues, 19 of which were resolved before the report released and another one right after. Dgraph has gone a long way since v1.0 release in terms of production stability. I'd recommend trying out the latest v1.0.9 release or the upcoming v1.0.10.

Dgraph itself is close to being launched in production at a few very big and well-known companies (that we can't mention publicly yet), who moved away from Neo4j to Dgraph. Needless to say, Dgraph's performance and scalability far exceed any other graph DB in the market.

Dgraph is tackling a lot harder problem of doing distributed joins and traversals, while providing distributed ACID transactions, synchronous replication and linearizable reads. The equivalent of Spanner, which can also do efficient joins (something relational DBs suck at, so technically more complex). There's no graph product out there like this or even a single paper which Dgraph is based on, rather we had to do original research to perfect this technology -- which is why it took time to build and stabilize Dgraph.

Badger, the underlying kv DB, itself was never found to have an issue. It is serving several petabytes of data in production use at various companies. We built Jepsen style bank tests for Badger, which run successfully nightly, and there's an open bounty of $1337 for finding any data loss bugs in Badger.

Dgraph is decently staffed (7 engineers) for a seed-stage startup, but we're definitely hiring and planning to grow in SF. No need to wait, this is the right time to run Dgraph in production.

<comment><USER39> Regarding performance: have you seen Roi Lipman's RedisGraph presentation ([https://www.slideshare.net/mobile/RoiLipman/graph- algebra](https://www.slideshare.net/mobile/RoiLipman/graph-algebra))? GraphBLAS is elegant and very promising. There some benchmarks comparing RedisGraph to Neo4j and others at the end of the slides. I would love to see Dgraph benchmarked against RedisGraph the same way.

</comment></comment></comment></comment><comment><USER40> It would be great if the Neo4j team helped jepsen.io do an analysis of their enterprise clustering features.

</comment><comment><USER41> I'm curious if any of you have tried it against another commercial graph db like AllegroGraph?</comment>


Handling five billion sessions a day in real time - <AUTHOR> https://blog.twitter.com/2015/handling-five-billion-sessions-a-day-in-real-time?hn

<comment><USER2> What i never see is how much servers/hardware/resources they are spending. It's _always_ like "we process a quadrillion messages per second" but it never mentions that you'll need a quadrillion expensive servers to do so. Same for many many database benchmarks, sadly...

<comment><USER3> It's a couple years old but _What it takes to run Stack Overflow_ [1] lists the number of DB servers, Web servers, etc

[1] [http://nickcraver.com/blog/2013/11/22/what-it-takes-to- run-s...](http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack- overflow/)

<comment><USER4> This update is more recent:

[http://highscalability.com/blog/2014/7/21/stackoverflow- upda...](http://highscalability.com/blog/2014/7/21/stackoverflow- update-560m-pageviews-a-month-25-servers-and-i.html)

</comment></comment></comment><comment><USER5> I had a hunch I'd see the data structures du jour in there: Bloom Filters and HyperLogLog. Since they are pretty accurate by themselves already but run much faster than the brute force approach, why keep the latter? The error margin, if used properly, should be small enough so people shouldn't really care.

<comment><USER6> They are not for accuracy. One usage for instance is to recompute stats if there are bugs in the stats processing code. Or if you want to add more metrics. (I work at Twitter but not on Answers).

</comment></comment><comment><USER7> > What happens if our speed (real-time) processing layer goes down? Our on- > call engineers will get paged and address the problem.

Innovative!

But for real, that's some architecture, and very insightful to see what they've got. Totally nothing I'd have guessed.

</comment><comment><USER8> First thing I noticed was Amazon in there Stack. Amazon really owns the internet...

<comment><USER9> In what way does Amazon own the Internet?

Yes, they own some of the computers that make up the Internet, but by far not all of them.

<comment><USER10> they are the dominant vendor of hardware/hosting to the largest and most complex web applications on the internet. AWS is the pillar that some of the most high volume sites on the internet (like Twitter and Netflix) rely on. Its safe to say that without AWS (or some other similar vendor) the internet as we know it wouldn't exist.

<comment><USER11> That's ridiculous. It's like you're saying: without Honda, the car industry, as we know it, wouldn't exist.

The hosting industry is huge. There's a ton options, many of them better, larger and older than AWS. There are _so_ many sites that don't use AWS. For example, I believe that, pre-Google, youtube used ServerBeach.

<comment><USER12> > There's a ton options, many of them better, larger and older than AWS.

Some examples would be more interesting than a general "you're wrong".

<comment><USER11> For every site that's reliant on AWS, there's probably more than 10 000 that aren't. The giants, for example: Google. Facebook. Wikipedia. Microsoft.

It's hard to get actual numbers, but Equinix might be the largest. I _believe_ their Singapore data center is actually what AWS, Digital Ocean and Azure use. (there's a lot more of these, such as Internap and Level 3 (which provided (and might still) services for Netflix and iTunes.))

AT&T is quite large. World of Warcraft used to be hosted on AT&T's infrastructure (might still be).

The above tend to be for large setups. Going smaller, you have the Rackspaces and Softlayers (now IBM). And, to a lesser extent, providers like OVH and Hetzner.

And, you can't forget the likes of Digital Ocean and Linode.

It's hard to come up with examples because more things are examples than aren't. AWS' market share is growing, but that's to be expected, in a lot of ways, despite EC2 being ~10 years old , it's still a new player.

AFAIC, what's more interesting about AWS is:

1 - How dominating it is over Microsoft and Google

2 - How popular the less invasive offerings are. A lot of people are sticking to their colocated, dedicated or VPSs, while using some part of AWS for DNS, backups or email. I hope they keep this up.

<comment><USER12> Thanks for the reply—exactly what I was hoping for!

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> twitter: @copypastaa and then appending ?hn is at least consistent :)

</comment><comment><USER14> "How we handle spying on your mobile phone activity on a global scale"

<comment><USER15> hahaha but it is a good architecture you have to admint :D</comment></comment>


Why Do People Kill Themselves? - <AUTHOR> https://www.scientificamerican.com/article/why-do-people-kill-themselves/ <comment><USER2> Please no. Suicide prevention is not important. What's important is giving people the tools to end their life safely, painlessly and with dignity if they choose to end their suffering.

I was in a coma for a month exactly a year ago from a blotched suicide attempt. It took the majority of the year to learn how to walk again, and use my hands. I am not sure if I'm thankful for another chance at living, but I do know that it I had a safe and guaranteed option to take my life, things would have been a lot easier. I know I will attempt again, and it is my right to do so.

How can these hypocrites pat their own backs sitting on their high horses? They talk about rights and suffering, yet never acknowledge that life itself starts with a violation of consent. Nobody asks to be born into suffering. And nobody deserves it.

Stop this madness and let people end their life in peace. It is the only true way to gain freedom.

<comment><USER3> I'm sorry you went through that. Though I've never attempted, I can relate to a lot of that. Certainly I've spent way too much time thinking about all of this.

I do think that everyone should be allowed a humane exit, perhaps after having to show deliberation over a period of months (agony notwithstanding).

Also, though, the phenomenon is worth studying. If we could figure out how to reduce the suffering that leads people to suicide, we should do it.</comment></comment>


Hackathon and Challenge to build a new Terminal – $20k investment - <AUTHOR> https://pioneer.app/blog/pioneer-challenge-a-new-terminal/ <comment><USER2> > What would this look like in multiplayer mode?

Uh… no thank you.

> How about chat

What are you on about? (also we already have `wall`)

This list of ideas is terrible and none of them make sense. It almost feels like one of these start-up name generators that mash up a bunch of buzzwords together.

Better auth support and all can be done completely independently of the terminal app, much like ssh-agent manages my keys and I don't need to re-enter my password every time I connect somewhere. But that's all done in the shell, not in the terminal app.

It looks like some money is available here for terminal developers? Instead of making something useless that no one wants or need, it would be a lot better to support people who actually build a useful app: [https://www.iterm2.com/donate.html](https://www.iterm2.com/donate.html)

</comment><comment><USER3> I genuinely don't understand this project.

Use the right tool for the job. I chat using a chat program, and write code using a terminal. Pair programming involves taking turns, not multiple people typing into the same document at the same time.

Beating Terminal for speed might be harder than they realize.

Beating iTerm 2 for functionality might be harder than they realize.

Features neither of them include might be unwanted, but I guess good luck, and I'll wait to see if any actual good ideas comes out of the hackathon?

</comment><comment><USER4> I'd be curious to see how this plays out.

> We expect real-time collaboration to dominate future markets as the pandemic > runs its course and forces people and businesses online en masse

I'd think that the area for the easiest investment in this problem space wouldn't be on the creation of a new terminal emulator (like iTerm2 or the Windows Terminal), but instead as a piece of software running on the remote server, something like tmux. Imagine instead a tmux that could allow multiple users to connect to it simultaneously, and drive different panes at the same time, or view the commands that another user is using in a given pane.

Then, regardless of what terminal emulator you choose, you'd be able to collaborate with your teammates, once you connect to the same server session.

When I think about a "Terminal 2.0", I imagine something more like [this]([https://github.com/microsoft/terminal/issues/3121](https://github.com/microsoft/terminal/issues/3121)). Ideally though, whatever protocol for rich feedback and content embedding wouldn't be tightly coupled between the shell and the Terminal, which I don't believe is something that [UpTerm]([https://github.com/railsware/upterm](https://github.com/railsware/upterm)) did well (though correct me if I'm wrong.)

<comment><USER5> >Imagine instead a tmux that could allow multiple users to connect to it simultaneously, and drive different panes at the same time, or view the commands that another user is using in a given pane.

I think this is what tmate does? I haven't used it, but I saw it mentioned on HN a couple days ago.

[https://tmate.io/](https://tmate.io/)

</comment></comment><comment><USER6> I disagree on the point that the terminal needs innovation. Clis are what need the innovation. They're rather inaccessible to anyone who's never used vim or emacs, and modern workflows just give no incentives to switch to them.

Obviously if you're already proficient in clis you'd never trade them for a gui, but if you're proficient in a gui then why use a cli (for a reason other than bloat)?

</comment><comment><USER7> I'd like a terminal that can finally actually show graphics. Not exactly sure what's missing to do that since I have no idea about terminal internal. Maybe i3 is what I'm looking for. But, it would be great to have lynx that allows showing images or videos or simply streaming a video over SSH to a video player in my terminal.

</comment><comment><USER8> I would absolutely love to have a Free terminal that had a feature set similar to SecureCRT. [https://www.vandyke.com/products/securecrt/](https://www.vandyke.com/products/securecrt/)

</comment><comment><USER9> Modern collaborative terminal...so, tmux running in hyper (for electron goodness)? :)

</comment><comment><USER10> Work for free for us for 48 hours to explore this problem space.

5 best teams will win the chance to work for free for 1 month for us.

Best team we will crown the winner and let you keep 95% of your company.

</comment><comment><USER11> Thought this was a hardware project until I clicked the link. Hardware would provide serious scope for innovation. Software has been done.

<comment><USER12> what do you mean "has been done"? is software a one-shot game?

<comment><USER11> I mean exhaustively modified with every bell and whistle imaginable. It is likely that any novelty here will be so minor as to be insignificant to UX versus established implementations. Interested to be proven wrong!</comment></comment></comment>


Brain Is a Co-Conspirator in a Vicious Stress Loop (2009) - <AUTHOR> http://www.nytimes.com/2009/08/18/science/18angier.html?em <comment><USER2> I was trained as a neuroscientist, took me over a decade to realize two important, basic, biological facts I wasn't taught:

1) Stress is always on - always. It's the survival instinct that can be reduced but not turned off.

2) Sleep is quite simply the recovery from stress. Without proper sleep, we don't recover.

Now as we develop technologies for stress management, I'm still amazed that #1 and #2 are not as commonly understood - and talked about - as the need for good nutrition and hydration. The body was designed to stress and sleep. We ignore those primal drives to our short- and long-term peril.

<comment><USER3> if stress is chemical, is there not a good way to dampen or come close to removing it chemically? Would there be major health downsides to working to reduce stress in your life to this degree?

<comment><USER4> Adrenaline uptake can be prevented with beta blockers[0], such as propanolol[1] which is often prescribed for blood pressure, arrhythmia, and performance anxiety. The down sides are when you stop taking it, you may be hypersensitive to stress since other methods of dealing with the adrenaline have been neglected. Exercise and meditation are better ways to reduce the effects of stress in the long term.

[0][https://en.wikipedia.org/wiki/Beta_blocker](https://en.wikipedia.org/wiki/Beta_blocker) [1][https://en.wikipedia.org/wiki/Propranolol](https://en.wikipedia.org/wiki/Propranolol)

<comment><USER5> I take propranolol for both BP and stress. I also meditate.

I can attest to propranolol's effectiveness for acute anxiety. Meditation is about equally effective for me, but not always practical.

</comment></comment></comment></comment><comment><USER6> tl;dr: "We tortured some rats, and they reacted as though they'd been tortured. When we stopped torturing them, they got better."

Don't mistake me for someone who has a problem with animal research in its own right, but I'm having a hard time seeing the novelty here.

<comment><USER7> They probably got some fMRI, and they framed it in terms of cortisol, adrenaline and cytokines.

Old knowledge is useless to modern science, we need to continually re-express it in the modish jargon of the day or people won't accept it as true. Thus, generations of studies restating the same old crap with new methods.

<comment><USER8> This is good!

It sucks as a learner. But for science and progress of knowledge, it provides an inefficient but lacks-false-positives mechanism.

It sucks for Liebniz, but was a positive for our understanding

<comment><USER7> No, it isn't good. It is brittle and dogmatic, and these dogmas are often thinly-evidenced ("now we know stress is all about cortisol levels!"). It means vast categories of human knowledge are regularly discarded without understanding or examination because of epistemological prejudice. And the process is not by any means lacking in false positives; this presumes that our current modes of knowledge are actually better, less error-prone, etc., than the forms of knowledge they are refusing to countenance, which they often are not.

</comment></comment></comment></comment><comment><USER9> >It's still August. Time to relax, rewind and remodel the brain.

Who the hell is the entity that is doing the remodeling. What a bunch of garbage.

<comment><USER10> It's even worse… some ppl here believe we are designed. Disappointing.</comment></comment>


Configuration (mis)management or why I hate puppet, ansible, salt, etc. - <AUTHOR> http://www.scriptcrafty.com/configuration-mismanagement-or-why-i-hate-puppet-ansible-salt-etc/ <comment><USER2> > So that ladies and gentlemen is why I hate anything and everything that uses > YAML or some other weird custom DSL to do the obvious.

Sure, because I would trust a random clueless programmer to do system administration or to even know what is necessary there. Your rant is totally justified.

> Everything that seems intuitive and obvious to me is either an anti-pattern > or straight-up impossible to do with any of those tools.

The same can be said about functional languages from a perspective of imperative programmer. You just don't understand the paradigm. There's really nothing to brag about here.

> Somehow I'm the only one that ended up with the right set of experiences > that taught me to avoid all the things these tools champion.

Yes, somehow you're the only one smart out there. All the other people are just plain stupid.

> [...] but not before a whole bunch of hair is lost trying to figure out why > some snippet of YAML is not creating the right unix user (this by the way > has happened more than enough times now that I've stopped counting).

As previously, there's nothing to brag about. You just don't understand the paradigm.

</comment><comment><USER3> > The stuff I had built at my first job taught me that my intuitions about > software systems in general were mostly correct

Oh boy. Here we go.

> The guy that had built this part was pretty smart .. He probably should have > trusted his intuitions a bit more and not bowed to peer pressure

Or maybe he knew enough to understand how Puppet could fit into the overall architecture and strategy at this company that is now the second company you've ever worked for.

> Why the fuck are build/ops folks making decisions about how the application > should be deployed and configured?

Because that's what build/ops folks are there to do, and congratulations, you've just discovered dev ops. Not every application is nginx, node.js, and mongodb running on a single VM. Sometimes deployments get complicated.

> How is it that these things are obvious to me but not obvious to others?

Wow.

This person hates on Puppet for "reasons" and decries all configuration management tools as awful pieces of garbage, then says Chef is great while consistently asserting that people are sheep for using configuration management tools, then throws in digs at Ansible and Salt with absolutely no context. It seems like he/she extrapolated one bad example of one Puppet use case into a general flaw with the whole lot. It's like he/she drove a Pinto for a bit, had trouble going around one corner one time, and deduces that all motor vehicles are bad and anyone who drives a car is an idiot.

</comment><comment><USER4> Yup.

I'm trying to create a build-and-configuration-management system for distributed embedded systems development.

The product will have some physical components with embedded software, and some back-end components running on some sort of server somewhere. Some of the embedded devices may be networked. Some may not.

I want to use Continuous Integration to ensure that the front & back end (and all the various testing tools) play nice together, and Continuous Deployment to reduce the risk of manual missteps.

So ... I'm trying to create the build scripts that build the system components (including cross-compiling the embedded SW), run tests and deploy the various components to UAT / HIL / SIL & simulation environments, orchestrate overnight and over-the-weekend simulation runs & generally provide tools to help manage deployment -- I also want to make the whole thing enforce DO-178B-style process compliance, requirements traceability etc.. etc...

Most of this is done in Python, 'cos I want to be able to reproduce the process on my workstation. Moreover, I want it to be portable (Linux, OSX & Windows), location-agnostic (works in any directory), and free from all but the most essential dependencies.

</comment><comment><USER5> You can run Puppet in a decentralised way:

        sudo puppet apply --verbose --debug --modulepath=./system system.pp

You can also use BASH/Python whatever _with_ Puppet and still take advantage of Puppet's declarative semantics if that's what you need:

         exec { "install_node":
            require => Package[["build-essential", "openssl", "libssl-dev", "pkg-config"]],
            command => "${settings::modulepath}/main/install_node.sh v0.12.7",
            creates => "/usr/local/bin/node",
            logoutput => true,
            timeout => 1800
        }

</comment><comment><USER6> On another blog I read a very wise notion, which captured the underpinnings which are "wrong" with tools like ansible/salt/puppet.

You can verify that something is set, but this guarantees nothing about the things you haven't set.

Now, this is fine if you purely use the above tools as provisioning, i.e. run the configuration tools on an immutable image. However, often these configuration tools are run on long running machines. And then we're back to pets vs. cattle. With all the faults and difficulties of Docker - at least it does that right.

<comment><USER7> Slight clarification, this isn't new in Docker, you're describing "Immutable Systems". It's also not mutually exclusive with configuration management tools.

The systems management tools have been doing this a long time with baked cloud images, most prominently Packer invoking configuration-tools-of-choice for baking cloud images straight (which I'd say STILL has a wider adoption than Docker) and this can also be done inside Docker files, just like it can be done for AWS image baking.

You're still going to have some things that need to be done in runtime, especially in managing your stateful servers.</comment></comment>


Find out if your brain solves problems like a software engineering pro - <AUTHOR> https://words.okwolf.com/problem-solving-pros <comment><USER2> Nice article in general and a good breakdown of problem solving.

I don't get the tangential step to dismiss the notion that there are actually 10x developers. Particularly as your article goes on to explain exactly why there could be.

People who as you describe that can:

* Understand the problem and how to know when it's solved

* Determine multiple possible ways to solve the problem

* Pick a solution and test if it solves the problem

Are very likely to be much more efficient that the programmer who can't even understand what the problem is to begin with or worse programmers who don't understand that they don't understand what the problem is but insist that it can be solved using that new and shiny toolkit they just found.

<comment><AUTHOR> My point about 10x developers is that they're not 10x better than average per the stereotype, but avoiding 90% of the wasted effort the average dev expends.</comment></comment>


Four software lessons you can learn from HBO's Silicon Valley - <AUTHOR> https://www.techrepublic.com/article/software-lessons-from-hbos-silicon-valley/ <comment><USER2> Love this! Plus I didn't realize Silicon Valley was already on Amazon Prime, which means it's binge watching time!</comment>


Tesla gets $520 mn funding for first Chinese plant - <AUTHOR> https://www.dawn.com/news/1468374/tesla-gets-520-mn-funding-for-first-chinese-plant <comment><USER2> I think the idea behind the tesla is too creative & innovative</comment>


Ask HN: What was something you predicted in tech before it's time? - <AUTHOR> Mine:
* 1996-7: talking to Kent, a friend who designed MIPS procs- "* why can't we have a rack that we just can universally slot machines on rails with a consolidated backplane so we don't have to plug in any cables? (Blade servers)
* 1997-8, working at intel: "why can't we stack multiple processors on top of each other?
 1998, "apple will switch to intel processors" them: never!
* 2004, on a hike with my intel chip designers, talking about their 56-core experimental chips... "just wait until google starts hosting systems in their own datacenters" - them: never gunna happen...
Not claiming anything other than questioning why we couldn't have made these moves earlier rather than later...
I have a bunch more, as I am sure you did too... what are some of your "holy crap I thought of that ten years before it happened" on which you like to reminisce? <comment><USER3> A long time ago (forget the year, but it was during the era of 32-bit main- memory buses), I predicted that CPU chips would have 512- or 1024-bit-wide main memory buses.

Not technology, but: After seeing Star Wars: A New Hope, I predicted that Vader was Luke's father. My friend: "Vader killed Luke's father! Did you miss that part?" Me: "I know it said, that, but I still think..."

</comment><comment><USER4> 2000: we'll be able to comparison shop online and brick and mortar stores using one interface.

2004: in the future we'll use the same language on the server and in the browser (though I incorrectly predicted PHP would move to the browser)

2005: someday we're not going to worry about minutes and text messages on our cell phones...it will all just be data.

</comment><comment><AUTHOR> EDIT: the Apple to intel year was wrong: that was said by me in 2004 - <USER2> know why I said 1998, apologies</comment>


Justice Department Suspects Sci-Hub Founder Works for Russian Intelligence - <AUTHOR> https://www.washingtonpost.com/national-security/justice-department-investigates-sci-hub-founder-on-suspicion-of-working-for-russian-intelligence/2019/12/19/9dbcb6e6-2277-11ea-a153-dce4b94e4249_story.html <comment><USER2> [https://en.wikipedia.org/wiki/McCarthyism](https://en.wikipedia.org/wiki/McCarthyism)

</comment><comment><USER3> Do we like them? No. Are they Russian? Yes.

Must be FSB/GRU/KGB

</comment><comment><USER4> TDS knows no limits, lol.</comment>


Intelligence Improperly Collected on U.S. Citizens - <AUTHOR> http://www.nytimes.com/2009/12/17/us/17disclose.html?partner=rss&emc=rss

<comment><USER2> I think what's scary is that this doesn't get me nearly as angry as it used to.

EDIT: Actually the incidents in the article are fairly tame. There are much more egregious examples.

</comment><comment><USER3> Doesn't sound good at first glance. In defense of their actions is that by studying different groups (and I mean that in the mathematical sense) you will make whatever you are looking for stand out much clearer. Creating patterns of each group might make forecasting their actions a bit easier and be able to pick up an unknown group within the general population. Just my thoughts.

</comment><comment><USER4> I am shocked. SHOCKED!

</comment><comment><USER5> "In all three cases, after other Homeland Security Department officials raised concerns, copies of the reports were destroyed. The agency also held a workshop on intelligence-gathering "while ensuring the protection of civil rights and civil liberties"

Someone made an error. They corrected themselves. iow, nothing happened.

<comment><USER6> It's great that they corrected themselves and all, but why didn't this come out in public? It seems to have taken an FOIA to get it out there.

Also, the "error" is particularly egregious and Stasi-like: collect info on people doing something legal, nay, constitutionally protected. It's also very typical for US Secret Police to do. The COINTELPRO program infiltrated Quaker organizations in the 60s, some DoD office did it before Gulf War I, and someone seems to be trying it today: <http://www.afsc.org/ht/display/ContentDetails/i/5264/pid/268>

I suspect the Secret Police tries to infiltrate Quaker organizations because it's so easy: they welcome just about anyone, and do all their work out in public, unlike less trustworthy organizations like the DHS and TSA.</comment></comment>


DNS over HTTPS Explained - <AUTHOR> https://blog.apnic.net/2018/10/12/doh-dns-over-https-explained/ <comment><USER2> The weight of setting up and tearing down an HTTP(S) connection compared to the lightweight DNS query is barely touched.

<comment><USER3> HTTP has keep-alive, you can definitely win for a single UDP DNS query, but web browsers don't do a single DNS query they often do dozens per page displayed and HTTPS can make that faster.

The HTTPS connection stays alive but idle for seconds, maybe minutes, and it always has the capabilities that DNS can only get by punting to TCP, which requires a round trip.

At the layer above the TLS connection can be brought back quickly from cached credentials using a PSK

And at the layer above that the TCP connection can do Fast Open so that there isn't a round-trip there either after the very first session.

</comment></comment><comment><USER4> It's presented from a biased point of view. It doesn't discuss the problems with DoH.

<comment><USER5> Most of the "problems" with DoH tend to be implementation based (e.g. applications individually resolving DNS or corporations assuming if they intercept UDP 53 they have secured connectivity) not necessarily problems with DoH.

The main drawback of DoH itself is it moves DNS to a session based service which causes increases in workload for DNS infrastructure but to be fair the article does mention that so I'm not sure "it doesn't discuss the problems with DoH" is totally fair.

<comment><USER6> I use my system resolvers on all my hosts, hand the IP to my unbound DNS server out and recurse over DoHs, that way everything is doing DoHs even if the underlying software doesn't implement it. DNSCrypt can do both it's protocol and DoHs.

[https://wiki.archlinux.org/index.php/Dnscrypt- proxy#Local_DN...](https://wiki.archlinux.org/index.php/Dnscrypt- proxy#Local_DNS_cache_configuration)

</comment></comment></comment><comment><USER7> HTTPS can't be trusted due to nation States both being abusive certificate authorities and the source of demand for dns censorship. Herefore, DNS over HTTPS is a bad idea at best. Please think about using dnscrypt or similar solutions instead.

<comment><USER8> The certificates can be pinned. This is a poor argument against DNS over HTTPS.</comment></comment>


Researchers have accidentally made batteries that could last 400 times longer - <AUTHOR> http://thetodayevents.com/researchers-accidentally-made-batteries-that-last-400-times-longer/ <comment><USER2> GREAT NEWS</comment>


Procrastination is more about managing emotions than time: study - <AUTHOR> https://www.bbc.com/news/health-45295392 <comment><USER3> One of the counterintuitive things that I've learned is that procrastination is often a side-effect of perfectionism -- but in cases where the "perfect" path isn't obvious. It's better to do nothing at all until the perfect path is understood.

Defeating this is a real challenge and requires serious conscious effort. People who have reputation as "doers" often gain it by finding the "good enough" target and driving towards it. Many "do nothings" are really people who have an overdeveloped desire to "do it right" and can't overcome it.

It's absolutely a matter of emotional control.

<comment><USER4> Which is why I am much more productive working on bad code in projects I get thrown into than starting from scratch. People trust me to fix things and add features fast in codebases I have never seen before but are known as being unmaintainable. Pays well by the way. If I have to start from scratch I can only start if I have the perfect picture in my head or it is better to do nothing and go hiking. The perfect picture (almost?) always emerges during hiking/walking though.

<comment><USER5> The trick sometimes is to write the bad version yourself first, then fix it. Hard to do when you know there must be a better way.

<comment><USER6> Exactly this. Just write _anything_ , and then start improving it.

</comment></comment></comment></comment><comment><USER7> Purely anecdotal, but observed over a long period and with many different colleagues:

(1)I see a direct correlation between how much people care about the outcome of something, and procrastination.

People that treat their work as just something they do 9-5 then switch off, are detached from the outcome for the project or the client never procrastinate. People that care do. I think this goes in the direction of the article.

(2) People that are stronger at analysis,able to grasp complex systems and the effects of changes, are much more likely to suffer from procrastination than those that are less apt at this. This is often confused with perfectionism. In my experience procrastinators are not looking for perfection, but are looking for a course of action that will have a reduced negative outcome. This would be in line with the general 'loss-aversion' [1] bias from psychology.

Tangent: Procrastination is often presented as a symptom of depression, but my personal feeling is that the causation is reversed. Procrastination is a symptom looking for but not finding good solutions. As such it is a lot more likely to be encountered in engaged, intelligent people. Depression can be a consequence of the stress built up overtime by that process. So you are not procrastinating because you are depressed, procrastination is a precursor to depression

[1] [https://www.psychologytoday.com/us/blog/science- choice/20180...](https://www.psychologytoday.com/us/blog/science- choice/201803/what-is-loss-aversion)

<comment><USER8> > _People that treat their work as just something they do 9-5 then switch off, > are detached from the outcome for the project or the client never > procrastinate._

Yeah, I noticed that too, and always was surprised about it. I've never seen a detached 9-5er procrastinate. In the field of programming, the only procrastinators I know are those who show some actual interest in the field beyond getting their paycheck.

> _Procrastination is often presented as a symptom of depression, but my > personal feeling is that the causation is reversed. Depression can be a > consequence of the stress built up overtime by that process._

That... resonates. It seems that it's usually prolonged procrastination that destroys me emotionally, not bad emotional state causing me to procrastinate.

</comment></comment><comment><USER9> I'm definitely a procrastinator. I've noticed three things. First, I get bored with tasks, so I make things exciting by putting them off until the pressure mounts. Second, I have a hard time actually finishing something and being content with it. Papers in school, for example, where I could easily turn an essay into a life-long project, constantly tweaking it, or of course in software with feature creep. Third, I dread the frustration that will come from inevitable disappointment in the outcome.

The only thing that has helped me is to break tasks down and use Pomodoros to track time. It creates some accountability and sense of progress that, I guess, provides enough pressure to motivate me. For the disappointment, I just have to remind myself that it almost always takes three tries to get something really right and try to focus on taking satisfaction in learning from errors rather than focusing on the sense of disappointment.

Some amphetamines would probably help, though. ;)

<comment><USER10> > Some amphetamines would probably help, though. ;)

Amphetamines _can_ help. What they actually do, at least for me, is raise the stakes. Imagine hyper-focusing on procrastinating.

</comment></comment><comment><USER11> Doesn't this article itself kind of undercut the study by showing that the brain can be altered through behaviors? What I am asking is, is the amygdala larger in procrastinators causing them to procrastinate, or did it grow BECAUSE they procrastinated? A leads to B, or A & B go together.

As an aside, I'd love to procrastinate significantly. It doesn't make me happy anymore. I've read the books, read the articles, tried the Pomodoro Technique, nothing. I don't think I have ADHD, since I can concentrate fine, just never on what I am meant to be concentrating on.

<comment><USER12> I suggest the following as this has worked well for me in the past.

Make a list of things that interest you, that you want to get done, and break each one down into a set of smaller tasks necessary to get that thing done. It doesn't have to be perfect initially, just make the initial list over the course of a day and rank order the tasks by how easy you think they will be to accomplish.

The next morning pick one task off the top of the list and either work on that task or think about how to better define the task to make it easier, this will often involve breaking it further down into smaller tasks. Some of the tasks will be easier to work on in sets, and some will be dependent on others. You can also pick a different task if you want, it is totally up to you.

As you finish a task, move it to the bottom of the list, mark it as complete, and make notes on what you did and additional paths to pursue. Typically this step will inspire additional things that you would like to get done, so add those as new tasks to the top of the list and reorder those tasks at the end of the day.

To make this easy for yourself, you will need to reserve 20-30 minutes at the end of each day to update the list, marking and describing accomplished tasks, adding new ones, and sorting the list.

Windows Sticky Notes is the perfect thing for this as it auto-saves (still backup though) has minimal editing features and is always there for you :)

<comment><USER13> If you're making lists of tasks and then actually doing those tasks you're not really procrastinating, are you?

<comment><USER12> The trick is that while you are procrastinating you add items to the list, or add more detail to the existing items, so that they are easier to get started on. Maybe you could call that taking a break from procrastinating :)

</comment></comment></comment></comment><comment><USER14> What helps me from procrastinating is writing excruciating detailed notes on tasks breaking them down to the smallest detail. Then all I have to do is sit down and start coding. It helps me a lot because my mind stays focused on the task at hand without the need to start analyzing things which most times means I lose focus. Furthermore, nothing seems daunting when you compartmentalize it, which I guess is the reason most of us procrastinate in the first place.

The one thing that really does the trick though is to get something out there and start having paying customers. Then you know that if you procrastinate you'll lose them and that the best motive to do the work that anyone can find.

<comment><USER15> The detailed notes is one of the principles of Getting Things Done (GTD). I started using it to help me with procrastination and it is really effective.

<comment><USER16> I love the extensive note taking. I began doing it in high school; I just write down every single word in a textbook or video about what I will be learn in class the next few days. It makes class so much more engaging and the homework is way easier, so I don't want to put it off as much. I highly recommend it if you have the time.

</comment></comment></comment><comment><USER8> Another HN procrastination topic, yay! I'm late to the discussion, but maybe someone will still answer:

I've been a procrastinator for as long as I remember, and have the following problem I struggle with: whenever I try to force myself to do a task - <USER2> _any_ task that isn't fun or just following curiosity - I feel this strong emotional block. The more I force myself, the stronger it gets, quite often turning into powerful anxiety, and sometimes even physical pain. Usually the mental pain itself is powerful enough to dumb me down to the level I am not smart enough to complete the task at hand (perils of working with one's head instead of hands...), which means the work won't get done anyway.

Is this normal? Is this just lack of discipline, as some would suggest? Does lack of discipline manifest itself with debilitating levels of anxiety? Do people here, who overcame procrastination, suffered from something similar at the beginning and don't suffer from it now?

I ask because I've tried the usual tricks - from pomodoros, through commitment schemes (hello Beeminder), imagination exercises, GTD, DeepWork planning, edw519's A/B mode of work... and everything eventually lost to the emotional pain. It's a small miracle that I'm able to get enough stuff done to make ends meet, but I look at how much more I could be doing if I weren't procrastinating that hard, and the very thought of years wasted makes me feel bad.

(#HNTherapy once again, I guess.)

<comment><USER17> If I remember [https://sideways-view.com/2017/02/19/the-monkey-and-the- mach...](https://sideways-view.com/2017/02/19/the-monkey-and-the-machine-a- dual-process-theory/) right, it suggests that this sort of problem comes down to a misalignment or mistrust between your conscious agency and the rest of your brain -- that is, the 'deliberator' and the 'monkey'. If whatever you're doing isn't getting the monkey enough bananas over time, it doesn't care how effective you are at getting Zorkmids or whatever such nonsense you consciously value (nonsense from the POV of the monkey), and it will intervene. The most obvious intervention feels like I-don't-wanna -- find something else to do -- anything else.

Tactical interventions ultimately fail because the monkey, not the deliberator, has its hands on the levers. The monkey can learn that letting the deliberator do its thing and earn zorkmids will ultimately also bring in plenty of bananas, but that has to be true for it to work in the long run.

(I may be adding a few of my own thoughts about how this monkey-and-machine business applies to procrastination -- I don't remember if those were on that long page.)

<comment><USER8> That was quite a good read, thanks! Among the couple of insights, the concept of getting two conflicting optimization systems to agree by making them trade was something... new to me, at least expressed in that form.

</comment></comment></comment><comment><USER18> Tim Pychyl is my favorite procrastination researcher and this article is so good. It's the most I've ever highlighted a Medium article.

[https://betterhumans.coach.me/how-to-use-psychology-to- solve...](https://betterhumans.coach.me/how-to-use-psychology-to-solve-the- procrastination-puzzle-6e6a56cdd535)

He calls it short term mood repair. Once you realize that, there are a lot of interesting ways to beat it.

</comment><comment><USER19> Where I struggle most with procrastination is when making talks or writing academic papers. I rarely or never procrastinate on coding. I think the key difference is emotional: if I'm writing something for a computer it just had to work. If I'm writing something for a human it has to be judged interesting or worthwhile according to some nebulous criteria that exists in the minds of the readers.

</comment><comment><USER20> I find procrastination is like trying the walk a cat. The cat rebels, making it as difficult as possible. The cat being your inner child. Ask it what it wants, apologize to it for being a tyrant, offer it some rewards and then don't you dare skimp on the deal. It'll turn more dog-like. "Heal the boy and the man will appear" -Tony Robbins

</comment><comment><USER21> Emotions are involved, but is it the root cause of procrastination ?

My understanding is that fear related emotions are closer to the root cause. Fear of failure, fear of imperfection, fear of wasting time and energy, fear to be rejected, ... I don't think that emotions like Anger, love or sadness are root causes.

Fear can worsen through time with new painful experiences. But fear can also be tamed.

One way is by adopting the development mindset. Failure is normal when we learn and practice. It is through practicing, and thus some failures, that we get better. No one is borned perfect at something. (See Dr. Dweck's book "Mindset").

Another way is by analyzing and identifying the root cause of fears. I could for instance fear to go in kitchens. That would be very disabling. By searching the root cause, I could find out that I got burned by touching an oven when I was a toddler. Or I could deduce that the kitchen is not dangerous in itself. Only the sharp knives and a hot oven are dangerous. One can then enter the kitchen and avoid the real dangers. Etc.

<comment><USER22> I don't think it's fear as much as being spoiled. Sure a student dreads failing an exam, but there's a lot of machinery starting up every time that fear is invoked, to find apologies and calm, soothing prospects on how it's going to work out eventually. That machinery existing, not even running through to a satisfying result, is comforting enough to overrule the fear. The oversized emotional attachment the article mentions must relate to the activities done instead of the thing which apparently too little emotion is attached. What is called Fear of Missing Out.

</comment></comment><comment> EZ-E Procrastination comes from anxiety (for me)

When I'm anxious about a task it's usually because I don't know where to start OR I'm because afraid I'll be dissapointed by how the result will be.

</comment><comment><USER23> Are there any (legal) drugs that helps to focus or "inhibit" the amygdala or something in order to be more productive?

<comment><USER24> I'd love to avoid drugs. My very limited experience hints at meditation (just the simple calm, slow, breathing exercise, thoughtless relaxation nothing more) is very potent. Not enough though but more than enough to keep digging this trail.

<comment><USER25> Try taichi or qigong. They can teach you to meditate while standing up, which makes it a lot more practical in daily life. Just having something invisible you can do ('work on your standing') in place of fidgeting or getting stressed out when get stuck waiting for something is remarkably effective.

Anxiety and trauma tend to manifest in disassociating your mind from your body. Working to bridge that gap can help a lot in dealing with uncomfortable situations.

Also, just stoop to bribing yourself. If I finish doing my taxes today I'll get those headphones or the fancy chocolates or my favorite ice cream. Lots of small wins, even contrived ones, gives you confidence for bigger ones. I have done this small annoying thing ten times. I can do this moderately annoying thing twice.

<comment><USER24> I did taichi in college (part of a wushu class) and it was damn amazing.

i did the self bribing, it works but not a lot. Hard to be judge and party. For bureaucracy I'm often more proactive to use the system against itself, namely giving them willfully stupid forms so they curate the required data, fill half of it and give me clear directions for the remaining part.

</comment></comment></comment></comment><comment><USER26> For me at least, one of the biggest causes is writing and part of that is having multiple experiences of asking for help and being told to "just write the thing" or "stop overthinking it" in response to my questions. I've gotten much better since developing the confidence to say to myself "no really. I need to get the answer to this question."

So if you are procrastinating, consider the possibility that your subconscious still remembers that you need some information but some other part of you is blocking you from getting it.

</comment><comment><USER27> On a related topic, I would add that managing energy is often more important than managing time, especially if your line of work is fairly intense.

</comment><comment><USER28> The following book helped me immensely. I'm sure many of you are familiar with it. I can't say enough about how great this book is.

The War of Art - Steven Pressfield

[http://a.co/d/dEjPUgf](http://a.co/d/dEjPUgf)

If you need some help with procrastination and not moving forward, a must read....

<comment><USER29> Seconding this recommendation. I just read "The War of Art" and "Do the work" (another book by the same author), and I wish I'd read them earlier. As a bonus, these books are a great example of how to write without fluff.

</comment></comment><comment> hal-eisen I can't believe no one has mentioned [https://waitbutwhy.com/2013/10/why- procrastinators-procrasti...](https://waitbutwhy.com/2013/10/why- procrastinators-procrastinate.html) yet

</comment><comment><USER30> [https://en.wikipedia.org/wiki/Test- driven_development](https://en.wikipedia.org/wiki/Test-driven_development) can solve procrastination for software developers

<comment><USER31> (when writing new code)

</comment></comment><comment><USER32> I often don't want to write any code before I can think of a good solution, but at the same time I can't come up with a good solution until I've tried a bunch of bad solutions.

</comment><comment><USER33> People who were procrastinators and have found a trick or technique that has worked in the long run and have who have made lots of progress please share

</comment><comment><USER34> The closing lines: "You do have the time. You just need to make it." are demonstrably false.

One cannot simultaneously conduct Nobel-caliber research, train as a top-tier professional athlete, be deeply involved with running a serious business, and spend a fulfilling/requisite amount of time with family. There is not enough time.

For me, the critical part of dealing with procrastination/perfectionism has been mastering, and executing upon, prioritization.

The rest of the tips in the article are useful, but that last one, I couldn't abide (and hence have been delayed by ~5 minutes doing the things I _want_ to do today).

<comment><USER35> At the same time, I look back on how much time I wasted by constantly moving the goalpost for any given project, or even about what projects I should spend my time on. And a _ton_ of this was due to me worrying about how to make money, and which projects would make the most money, and all kinds of useless questions like that.

But historically, the projects of mine that got the most positive feedback and have the most users, are ones I did purely because I wanted to, because I saw a need and knew how to fill it and knew it would be an interesting and satisfying challenge.

So the other night I decided to go back to that. I've got a _super cool_ new app that will make Mjolnir/Hydra/Phoenix/AppGrid look like stone age tools, that I plan to dedicate serious time to. I've also started re-typesetting a very fascinating public domain book printed in 1885, which I found on archive.org, and got Tesseract to do most of the work for me last night in about an hour flat.

So yeah, I can get a lot of really cool stuff done in a very short amount of time, as long as I _stop worrying so much_ about how things will be provided for me and trust in Divine Providence to take care of that. That's how all those other cool projects were made anyway, so I have even more evidence against my worrying and overthinking it.

<comment><USER36> There's a lot to be said for a "keep moving" approach to project work. I'm a worrier with a perfectionist streak, but I do better with multiple projects because I ruminate less on each one: If I cycle through them over the course of a day, I forget what I was worried about and just focus on accomplishing something I had put in the to-do, which levels things out and lets me do more work at less than 100% when it's called for. When I do work at 100% it means I'm destroying myself trying to calculate every possible option. The 100% work only needs to come out some of the time.

</comment></comment></comment><comment><USER37> Yeah, I figured out years ago that dealing with my feelings first is the more efficient path forward, basically.

</comment><comment><USER38> Already well known in the literature.

<comment><USER39> Then cite it

<comment><USER38> Fair point. So far I haven't found the underlying papers (Google isn't great at turning up papers I know I've read about in books, it turns out..) however the Internet's non-scholarly literature on the subject consistency identifies procrastination as essentially an anxiety disorder, not one of poor time management skills. e.g. :

[https://www.psychologicalscience.org/observer/why-wait- the-s...](https://www.psychologicalscience.org/observer/why-wait-the-science- behind-procrastination)

[https://www.nbcnews.com/better/careers/6-myths-you-ve- been-t...](https://www.nbcnews.com/better/careers/6-myths-you-ve-been-told- about-procrastination-productivity-n763571)

[https://finishyourthesis.com/procrastination/](https://finishyourthesis.com/procrastination/)

One of the leading researchers in this field is Tim Pychyl. I learned most about the subject by reading his books and listening to all his podcast interviews with fellow researchers. It is possible the papers I'm thinking of are listed on his group's web site. The paper titles tend to be hard to parse, unfortunately:

[https://www.procrastination.ca/](https://www.procrastination.ca/)

If and when I track down the source research studies, I'll follow up here.

</comment></comment></comment><comment><USER40> TL;DR Emotions are somewhat studied in terms of how to deal with them, check Cognitive Behavioral Therapy. Check out Feeling Good by Burns[1], there are specific chapters addressing perfectionism. And his podcast on this topic ""I don't feel like doing it!" Quick Cure for Procrastinators""[2] (suggest to find it on iTunes[3]/Spotify to listen with 1.5x speed, episode 053)

I'm going through a stage in my life when I don't enjoy doing things anymore. Obviously, if a task is considered somewhat hard and I know I will barely enjoy it, there is very little internal emotional support (motivation?) for doing it, thus procrastination.

I did an exercise from the book which was to list tasks I did during the day and mark a satisfaction rate as well as an efficiency rate. All the tasks are like 10% satisfaction, 99% efficiency. Seems like a mismatch. Then I asked myself a question "What would a person who does useful and efficient things tell himself to feel that miserable? What would he think about?".

I got the following, which goes really deep into my internal motivations and fears: "All these tasks are worth nothing. I shouldn't be happy with them because I can forget that I don't grow and don't accomplish much in general. I may end up being a silly guy who only chills and has fun but in fact, has nothing meaningful going in his life. I shouldn't forget that things are not that good at the moment and it's too early to relax, to enjoy small accomplishments. A person becomes weak if he always enjoys what's going on around."

So if you experience a huge mismatch between satisfaction and efficiency (paradoxically I naturally have lots of fun doing things I am like 0% efficient in because I don't know yet how perfect result looks like), I suggest you to ask the same question "what thoughts could make a person that miserable?".

After listing advantages of believing the above and deep appreciation of my beliefs, it became clear that some of them are crazy self-sadistic. Even though they have the best intentions in the universe (making my life better) what they in fact achieved was getting me to the verge of having suicidal thoughts. And now I am doing exercises mentioned in the book and the podcast to give up these beliefs.

[1] - [https://www.amazon.co.uk/Feeling-Good-New-Mood- Therapy/dp/03...](https://www.amazon.co.uk/Feeling-Good-New-Mood- Therapy/dp/0380810336) [2] - [https://feelinggood.com/2017/09/18/053-ask- david-i-dont-feel...](https://feelinggood.com/2017/09/18/053-ask-david-i-dont- feel-like-doing-it-quick-cure-for-procrastinators/) [3] - [https://itunes.apple.com/us/podcast/feeling-good-podcast- tea...](https://itunes.apple.com/us/podcast/feeling-good-podcast-team-cbt-the- new-mood-therapy/id1171155453?mt=2)

<comment><USER31> > A person becomes weak if he always enjoys what's going on around.

For any person not retired or similar, this is a _true statement_ , not a mere sample of "internal motivations and fears".</comment></comment>


Programming Language Analysis - <AUTHOR> http://ec2-54-224-80-201.compute-1.amazonaws.com:8888/languages_visualization/71c6f3773a3cbe2aa440949bc906bd4d/index.html <comment><USER2> This is tough to do. A while back, I sold off an effort I'd maintained for several years called [http://langpop.com](http://langpop.com) , which was, IMO, the best effort out there because I looked for a lot of different data sources to utilize including source code, google searches, books, and jobs.

One of my conclusions is that there is "popularity" in terms of overall use, and "acceleration" in terms of something gaining popularity quickly. Cobol is probably something in the first category. Ruby, when Rails started to take off, was in the 'accelerating' category.

<comment><USER3> I'm curious, did you consider a 3rd factor – something like enthusiasm or interest – where a language may be neither in wide use nor necessarily gaining much adoption but nevertheless garners a lot of attention? I'm thinking maybe Rust or Haskell?

 _Edit: I don 't mean to slight those two languages, by the way. Both, I'm sure, are gaining actual adoption at non-zero rates. I just chose them to illustrate that, perhaps, the level of interest around them may be a bit out of proportion with either their adoption or their acceleration._

<comment><USER2> I included 'talked about' in the stats for that reason. I guess you could say that, yes, there are some languages that a lot of people look at and admire but don't really use that much. Lisp comes to mind as being #1 on that list.

</comment></comment></comment><comment><USER4> Where's top 20 business language Cobol? Only 500 questions on Stack Overflow! No-one out there running a portmanteau of handles creating screes of trivial questions and answers to nudge its rank up.

And on Github a message that cries opportunity: "It looks like we don't have any trending developers for COBOL. If you create a COBOL repository, you can really own the place. We'd even let it slide if you started calling yourself the mayor."

<comment><USER5> heh yes I always wonder about these analyses -- is the # of questions on SO an actual correlate to popularity? Or is it the sign of an opaque or buggy language

It's funny (be warned, this is sortof flamebait i know) I left RoR to enter the Java world & since then it seems like RoR's popularity is _still_ increasing on these metrics & it is very strong, even though the heyday of RoR seems to have passed. Many are critical of the new releases / constant breaking updates etc. and many viable MVC alternatives have emerged.

I think an interesting metric that is never really considered in these reviews would be "Number of SO Questions per user of the language" \-- basically SO issues per capita. I think that may actually reveal some interesting facts about which dev platforms are most seamless to work on.

I appreciate the open source mantra & everything, still work on open source projects, but it is slightly worrying when I see these languages gaining so much traction yet having such a massive slew of banal issues ("trivial questions and answers" \+ things that are broken that shouldn't be, leading to workaround requests). I guess I have been converted to the evil Java empire (in fairness, I am diving into Clojure as well). JVM just seems more stable to me after dealing with everything in Rails breaking all the time & trying to balance dependencies ("I need to update this to use this, but then this will break. So I guess I need to update both gems plus Rails itself, but we aren't budgeted to do a bunch of integration testing right now, so... I guess we'll get that feature in 2 years or if it is ever backported" \-- Java libs just seem a lot more robust to me, haven't yet faced an issue like this except for 1 or 2 workable IDE integration issues). OK that last bit really _was_ a tangent.

<comment><USER6> Number of questions on SO correlates to number of people using the language that are likely to ask questions on SO.

SO skews towards web-programmers and MS stack developers. That clearly affects a lot of rankings.

Github similarly skews towards web-programmers (as a lot of the first users of it were RoR related) so you see the upper-right of this graph is mostly web related.

Also, some langauges have their own preferred forums for support, with a lack of the critical mass of SO users. If you want an answer to a Common Lisp question, please visit #lisp on freenode; if you post to SO it will go unanswered for weeks; it's likely to get answered in a few hours on #lisp.

[edit] And SO also skews towards homework questions.

<comment><USER5> yes i see what you're saying. thank you for the intro to freenode! it's real- time (well, the time it takes for someone to return to computer & read your question) IRC q/a for Lisp? very cool.

upload the logs & i bet it would be pretty parallel in function to SO. lol, i feel like their moderation usually only hurts the quality of posts if anything... if i ask a question that begs any thought or is open-ended in anyway it is immediately shutdown as an opinionated "asking for favorite library" question even if the functionality i'm looking for is clearly delineated -- which is what the real strength of a q/a community should be!!

"I want to do x thing that's never been done before. Based on the existing libs that do tangentially related things or offer partial functionality, how should I proceed?" > "My RoR form is giving me an error does anyone know why?" (usually a typo or outdated syntax because older version docs are no longer available in the gem community)

</comment></comment></comment></comment><comment><USER7> I was very sad to discover that most of the hits (I only checked one page) for slash on stackoverflow are actually questions involving the slash character (Generally http request parsing)

<comment><USER4> Developers who use popular language `Visual Basic` for their Github projects mustn't ever have any problems as they don't ever ask questions on Stack Overflow. Oh wait, it's tagged differently there, `vb6` and `vb.net`.

<comment><USER8> Yep the tagging needs to be worked on, julia get tagged as julia-lang on SO. And I am sure many of the languages ranking high on GH but very low on SO will have the same issue.

</comment></comment></comment><comment><USER9> I wish I could click on the languages in the legend and have them color- coded/labeled in the chart with unselected languages washed out.

</comment><comment><USER10> Great job!

You might want to prevent anyone from updating your data and influence the results by applying a document update validation function: [http://docs.couchdb.org/en/latest/ddocs.html#validate- docume...](http://docs.couchdb.org/en/latest/ddocs.html#validate-document- update-functions)

</comment><comment><USER8> Very cool, I have wanted to see something like this for a while.

This chart would be more useful if the dots were language. Expand the chart out to where the legend is, and drop the legend. (Or move the legend below the chart, and have it as a table with Stackoverflow as one column and Github as another).

</comment><comment><USER11> I think an analysis of which programming language/framework is popular in which domain/industry would be really helpful.</comment>


CSAW CTF 2013: An entry-level CTF for students who are new to security. - <AUTHOR> https://ctf.isis.poly.edu/

<comment><USER2> I tried and I honestly don't understand how to crack half of them. They do take a lot of time and I just heard this today. I hope they keep the challenge going (but don't account the score after 22nd). It's fun to play with.</comment>


20 Years Ago Today: Dijkstra's "On the Cruelty of Really Teaching Computer Science" - <AUTHOR> http://www.cs.utexas.edu/users/EWD/ewd10xx/EWD1036.PDF

<comment><USER2> Thanks for posting this Michael (saw it on /. too). A much better read than the terrorism submissions that seem to be showing up here lately.

A title correction however: Should be 'Computing' Science. I might be wrong but I think Dijkstra drew a deliberate distinction between the two.

The precision of his words is refreshing.

<comment><AUTHOR> Thanks for the correction.</comment></comment>


Eugene Chung, new Managing Director for TechStars NYC - <AUTHOR> http://www.techstars.com/eugene-chung-techstars-in-nyc/

<comment><AUTHOR> People can vote on whether or not it's the right choice on <https://poutsch.com/question/1006881></comment>