"It's The Future" - <AUTHOR> https://circleci.com/blog/its-the-future <comment><USER3> The article perfectly summarizes my frustration and sentiment. These days I hear these buzzwords all time. I work as a consultant for an enterprise product and most people whom I meet they somehow catch these buzzwords and blurt it out in front of everyone during meetings and discussions to either showoff that they know technology and things that are in the market these days(also latest iphone, apple news, tesla, space exploration and what not) or I feel they are somewhat trying to hide their insecurities.

Anyway long story short, most of these people do not really understand why they need all this rocket science to manage < 500 internal users. One of the new buzzwords I am hearing these days is mostly related to bigdata and machine learning. One of my managers came to me and asked me why dont we integrate our product with hadoop it will solve the performance problems as it can handle lot of data.

I am frustrated by the industry as a whole. I feel industry is simply following marketing trends. Imagine the no. of man-hours are put into investigating technologies and projects dropped mid-way realizing the technology stack is still immature or not suitable for at all.

<comment><USER4> This is how we manage this problem at the times when Visual Basic was the king and we use _instead_ Visual FoxPro.

People want theirs apps to be made with Visual Studio (BTW, FoxPro was part of the package).

So they ask: "In what is the app made"?

"In Visual, Sir."

Done. End of story (like most of the time, obviously some times people are more dangerous and press it ;) ).

\----

The point is not focus in the exact _word_ but in what the people _know_ the word will give to them.

So, for example "Big Data". The meaning for us matter zero. The meaning to some customer is that it have a largeish excel file that with his current methods and tools take _too long_ to get results.

So. Do you use "Big Data Tools"?

"Yes Sir."

And what about use Hadoop?

"We use the parts of big data tech necessary for solve this, and if we need to use hadoop or other _similar tools_ that fit better with your industry and that use the same principles will depend in our evaluation. Not worry, we know this"

Or something like that ;). Know that worry the people behind the words have help me a lot, even with people with WORSE tech skills (damm, I have build apps for almost iliterate people with big pockets but only witch cellphones as references of tech!)

And the anecdote about the largeish excel file that was too big and take too long? Yep, true. And was for one of the largest companies in my country ;)

<comment><USER5> I think the buzzword abuse exists because of people who don't want to take the time to learn to real skill, and just want shortcuts to sound smart and relevant.

I am very skeptical of people who are "BizDev" or "Project Managers" or "Managers" or "Scrum Master" they generally don't know what they're talking about and rely on buzzwords.

<comment><USER6> I'm wondering why you have managers in scare quotes. Are you suggestion that they aren't really managers or that managers as a position is just some sort of fraud?

Also Project Manager and Scrum Master are just positions that describe roles and responsibilities an organization / on a team. The people filling those roles needn't be clueless.

<comment><USER5> I feel people in those positions are just people who want a job in tech / startup and have gotten there not because they deserve those positions but because, they've given up on learning the skills and resorted to becoming a "manager". At least from my experience this is what I've seen. They generally don't really add any value because they don't have any skills, don't know how to do anything.

The positions I mentioned above are usually the position people who failed at picking up any valuable skill seem to resort to.

</comment></comment></comment></comment></comment><comment><USER7> "-No, look into microservices. It's the future. It's how we do everything now. You take your monolithic app and you split it into like 12 services. One for each job you do.

That seems excessive"

A 100 times yes. We tried to split our monolithic Rails app into micro- services built in Go. 2 years and many fires later, we decided to abandon the project. It was mostly because the monitoring and alerting were now split into many different pieces. Also, the team spent too much time debating standards etc. I think micro-services can be valuable, but we definitely didn't do it right, and I think a lot of companies get it wrong. Any positive experiences with micro-services here?

<comment><USER8> Yeah, we do micro services, the "real" kind. Not the "SOA with a new name kind", but the "some services are literally a few douzan lines of code and we have 100x the amount of services as we do devs" kind.

The thing is, you need a massive investment in infrastructure to make it happen. But once you do, its great. You can create and deploy a new service in a few seconds. You can rewrite any individual service to be latest and greatest in an afternoon. Different teams don't have to agree on coding standards (so you don't argue about it).

But, the infrastructure cost is really high, a big chunk of what you save in development you pay in devops, and its harder to be "eventually consistant" (eg: an upgrade of your stack across the board can take 10x longer, because there's no big push that HAS to happen for a tiny piece to get the benefits).

Monolithic apps have their advantages too, and many forget it: less devops cost, easier to refactor (especially in statically typed languages: a right click -> rename will propagate through the entire app) and while its harder to upgrade the stack, once its done, your entire stack is up to date, not just parts of it being all over. Code reuse is significantly easier, too.

<comment><USER9> "Different teams don't have to agree on coding standards (so you don't argue about it)."

Unsure if sarcastic.

<comment><USER10> Not at all sarcastic. I've seen endless wheel warring over spaces/tabs, level of indents. Mostly I ignore it.

</comment></comment></comment></comment><comment><USER11> Author here.

To answer some questions: yes this is obviously poking fun at Docker, but I also do really believe in Docker. See the follow-up for more on that: [https://circleci.com/blog/it-really-is-the- future/](https://circleci.com/blog/it-really-is-the-future/)

In a self-indulgent moment I made a "making of" podcast about this blog post, which is kinda interesting (more about business than tech): [http://www.heavybit.com/library/podcasts/to-be- continuous/ep...](http://www.heavybit.com/library/podcasts/to-be- continuous/ep-16-its-the-future/)

And if you like this post you'll probably like the rest of the podcast: [http://www.heavybit.com/library/podcasts/to-be- continuous/](http://www.heavybit.com/library/podcasts/to-be-continuous/)

<comment><USER12> It's a good post. It capture both the innate complexity of the problems some of us are working on, and the incredible WTF moments involved. Both sides of your theoretical conversation have their own pitfalls. Everyone in the comments here seems stuck on the "I don't really need microservices train", (and there _is_ such a thing as overdoing it … but I've never seen it) but I can't help but think that Italics nailed it here:

> _-It means they're shit. Like Mongo._

> I thought Mongo was web scale?

> _-No one else did._

It's so incredibly true, and I laugh (and cry, b/c we use Mongo) at this section each time I read it. Also, this gets me every time:

> _And he wrote that Katy Perry song?_

</comment></comment><comment><USER13> There was a time when Heroku seemed just as foreign to me as Docker does in this article.

\- So shared webhosting is dead, apparently Heroku is the future?

\- Why Ruby, why not just PHP?

\- Wait, what's Rails? Is that different from Ruby?

\- What's MVC, why do I need that for my simple website?

\- Ok, so I need to install RubyGems? What's a Gemfile.lock? None of these commands work on Windows.

\- I don't like this new text editor. Why can't I just use Dreamweaver?

\- You keep talking about Git. Do I need that even if I'm working alone?

\- I have to use command line to update my site? Why can't I just use FTP?

\- So Github is separate from Git? And my code is stored on Github, not Heroku?

\- Wait, I need to install both PGSql and SQLite? Why is this better than MySQL?

\- Migrations? Huh?

<comment><USER14> To be fair, a lot of these questions are valid. You arguably /don't/ need MVC for yours simple website, if Dreamweaver floats your boat you might as well use it, it's unclear why a small website needs an elaborate deployment infrastructure, using a VCS for personal code can be overkill, PGSql and SQLite aren't better than MySQL for every use case -- and so on.

Frameworks, orchestrations, even just new technologies -- these are great if they actually make your job easier or if they make your product better. Unfortunately, they often do exactly the opposite.

<comment><USER15> Agree until

> using a VCS for personal code can be overkill

I've been burned before, have you? If you're using something like Google Drive, you should use DropBox instead, since it seems less likely to lose your work.

</comment></comment></comment><comment><USER16> Obligatory link to "The S stands for simple", a SOAP-bashing classic: [http://harmful.cat-v.org/software/xml/soap/simple](http://harmful.cat-v.org/software/xml/soap/simple)

<comment><USER17> Everything must be in XML. Except the SoapAction header. Which has no defined standard. Yeah I remember all that madness.

<comment><USER18> remember? Thomson Reuters on demand APIs are still largely SOAP based.

<comment><USER19> never touch a running system...

<comment><USER18> fair enough, but I posit X users started consuming this API when SOAP was prevalent and Y users started when ReST was prevalent and Y >> X. Furthermore, SOAP is hard to maintain these days because it's so ancient. i.e. the libraries are not new and/or actively maintained.

As such, I maintain SOAP should be gone for the good of the running system.

<comment><USER20> In Python you simply don't have good SOAP libraries. They were all started at the tail end of its popularity and then all died quiet deaths when attention shifted to ReST before they were actually production ready, and if you now want to talk to a SOAP service… well, better don't do it in Python. 2, that is. Forget about 3.

<comment><USER21> Have you seen Zeep?

It's literally billed as "A fast and modern Python SOAP client". Python 2 and 3 compatible. Last commit was two weeks ago.

[http://docs.python-zeep.org/en/master/](http://docs.python- zeep.org/en/master/)

<comment><USER20> Nope. We needed one last September, zeep didn't yet exist back then.

And going by the bugtracker, it's running into quite a few problems with almost-but-not-quite compliant servers/WSDL files, which is a real issue when you're trying to interface ass-old legacy APIs (we're talking "not upgraded since 2006"-old) made by $BigEnterprise. Maybe _this_ time the project won't die before they work out all the little kinks.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER22> Man, I dont think this is the future at all. OK, Docker is good and has its propose, and is very good on what its do: "Run only one process in one brand new kernel", but beyond than that, its just a daemon that uses and abuses of linux containers, you can easily scale, but is a pain in the ass to upgrade apps, also you _need_ to run only _one_ process on that. Does not looks like the future for me to have 30 different linux containers running just only one process in each of them, dude, you have a kernel in your hand, why the hell you will run only one process on it? (what the heck, you can protect yourself and scale without be the bitch of a daemon, you just need to know your best friend kernel), you dont need to make micro services for everything, its good ok, but its not the solution for everything like the people are saying...

I really dont have any idea why the people are are so excited about "docker" all the things.

<comment><USER23> It's all about simplifying deployment. That's it, that's what's so good about using containers.

I don't know if you understand what Docker really is when you say something like this: "Run only one process in one brand new kernel", the kernel is shared between containers, that's the whole idea, you package the things your application need and be done with it.

The current problem with containerization is that there are no really good or understood best practices, people are still experimenting and that's why it's a big moving target and, consequently, a pain in the ass if you need to support a more enterprise-y environment. You will need to be able to change and re-architecture things if the state-of-the-art changes tomorrow.

I agree with your sentiment about going overboard on "docker all the things", that's dumb and some people do it more because of the hype than by understanding their needs and using a good solution for it but I think you are criticising something you don't really grasp, these two statements:

 _> "Run only one process in one brand new kernel"_

 _> you have a kernel in your hand, why the hell you will run only one process on it?_

I'm not trying to be snarky, I really recommend you doing a bit more of research on Docker to understand how it works. Also, Docker doesn't make it a pain in the ass to upgrade apps, quite the contrary if you do it in some proper ways.

<comment><USER24> Doesn't statically compiling programs solve the deployment issue better? I mean, as far as I can tell Docker only exists because it's impossible to link to glibc statically, so it's virtually impossible to make Linux binaries that are even vaguely portable.

Except now Go and Rust make it very easy to compile static Linux binaries that don't depend on glibc, and even cross-compile them easily.

Hell I think it's actually not even _that_ hard to do with C/C++: [https://www.musl-libc.org/how.html](https://www.musl-libc.org/how.html)

If I have a binary built by Go, what problems does Docker solve that just copying that binary to a normal machine doesn't?

<comment><USER25> Have to admit, as a fellow Go dev, with single binary static compiles, I don't really GET why I need docker... all it seems to offer is an increased workload and complicated build proc

<comment><USER26> In my humble case, Docker solves the problems I have to manage the systems on which my application runs (and that's mainly it). A single dockerfile of 20-30 lines describes a whole system (operating system, versions, packages, libraries, etc), and cherry on the cake, I can version it in my git repository.

This is not revolutionary in itself, but having the creation and deployment of a server being 100% replicable (+ fast and easy!) on dev, preproduction, and production environments, plus it's managed with my usual versionning tool, that is something I appreciate very much.

Sure, there are other tools to do the same, but docker does the job just fine.

<comment><USER27> > _having the creation and deployment of a server being 100% replicable_

The problem of ensuring that upstream dependencies can be reproducibly installed and/or built is, of course, left as an exercise for the reader.

</comment></comment></comment></comment></comment></comment><comment><USER28> I am going to ramble. Just move on if you don't care to hear the ramblings of a 62 year old development manager.

I'm pretty docker ignorant. I think I get it in concept. I manage >150 web sites (~15,000 pages total) that are php based with eXist-db and oracle (overkill but forced to use it) for database backends. My team develops on mac os x and pushes code to RHEL. We have never had a compatability problem between os x and RHEL except for some mgmt scripts in bash that were easily coded around.

Big data to me is a 400 MB apache log file.

I go home grateful I don't have to be in the buzz word mix.

I do read a lot about technology and over time that informs some changes like using apache camel for middleware, splunk for log file analysis yada dada...

I have had bosses that brought me buzz word solutions that don't ever match the problems we have. I hate that but right now I am not in that position. My boss leaves technology decisions to us.

Least you think we are not modern at all we do use a CDN, git and more.

Some days I get anxiety from reading HN, feeling stupid. Some days I get a lift from HN from reading articles like this one and the comments.

I am so glad I'm not in the business of chasing technology.

</comment><comment><USER29> Please also read the followup ([https://circleci.com/blog/it-really-is-the- future/](https://circleci.com/blog/it-really-is-the-future/)).

I read both articles a year ago and it really helped me grasp the whole container movement.

</comment><comment><USER30> "Why don't I just use Google's thing?

"-You think that's going to be around in 6 months?"

Isn't reputation a thing of beauty?

<comment><USER31> So true. Why people often use "it's backed by Google" as an argument in favor of anything is beyond me.

<comment><USER32> "it's backed by Google" is the reason I avoided Go for years and is still the only reason I'm nervous using it.

<comment><USER33> If it's free software it won't just disappear. If it's proprietary and hosted by an org that isn't making real money from it, that's a different story....

<comment><USER34> Is the development open though (serious question, I don't know how they do it with go)?

Look at android, it's a closed project that the occasionally release some source code for. If google decided to drop android then it's a critical blow because there isn't much of a community around it.

<comment><USER32> It depends what you mean by "open." Anyone can subscribe to the go-codereview mailing list. Anyone can submit a patch, but you need to a) use gerrit and b) sign a (digital) contribitors license agreement with Google.

It's "open", but it would almost certainly collapse entirely if Google decided to drop support for it. (There's no sign that they'll do that, but it's not a risk you take with a language like C.)

<comment><USER35> Even by the most lax standard you couldn't call the app store open; and _that 's_ kind of what's being shuttered here. The technology wasn't ever particularly interesting (not a bad thing), but the distribution model was. And that's entirely closed - they're not about to give anyone else permission to run even parts of the app store to anyone else, and that's that.

</comment></comment></comment></comment></comment></comment></comment><comment><USER36> Sometimes it seems the webdev world is unaware of the complexity its creating simply to execute instructions....

<comment><USER37> What do you mean? I'm just running bytecode on a virtual machine ontop of a virtualized container on top of virtualized hardware on a CPU where the instruction set is virtualized in microcode...

<comment><USER38> Reasonable.

</comment></comment></comment><comment><USER39> There are 2 major issues with this:

1) Small teams (~1-5 people) trying to seem "big" by working at Google's scale.

2) Heroku's prices. We are currently (successfully so far) migrating a small Django project from bare Amazon EC2 instances to ECS with Docker. Even using 3 EC2 micro instances (1 vCPU, 1 GB RAM) for the Docker cluster we would spend ~8 USD/month/instance. With Heroku the minimum would be 25 USD/month/dyno. That's a 3x increase in expenses.

It's very possible to take advantage of technologies like containers without getting too caught in the hype.

<comment><USER40> wait. you're comparing $25 with $75. it is 3x but it's still accounting noise by any standard imaginable unless you're running a charity server for an open source project.

<comment><USER41> What about the standard of "I'm young and this is a side project I'm doing in a couple of hours at the weekends"? Of course once you have a real company with more than two customers $75 is nothing. But version 0.1 is often a tool that's only useful to you.

<comment><USER42> Heroku has free dynos for side projects, and hobby dynos ($7/dyno/month) for slightly-less-side projects. So that original $75/m quote isn't quite right for that situation.

<comment><USER39> Yes it is, our EC2 instances have 1 GB of ram, the 7$ dyno has 512 MB.

</comment></comment></comment></comment></comment><comment><USER43> Previous discussion from 434 days ago: [https://news.ycombinator.com/item?id=9688383](https://news.ycombinator.com/item?id=9688383)

</comment><comment><USER44> I think I will be fine, thanks. I'll stick to my shell scripts, so far they've outlived any other devops fad.

<comment><USER45> Why not use Python instead? Shellscript is so... chaotic.

<comment><USER46> How do you write "foo | bar" in Python?

<comment><USER24> How often do you _actually_ need to do that. 99% of `foo | bar` command could easily be `foo > a && bar < a` which is pretty trivial to do in Python.

<comment><USER46> Well, how do you write that in Python?

I'm curious because these simple things that are delightfully easy in bash often turn out to be surprisingly tedious in other languages.

Of course, some things are tedious in bash too. But a basic principle of shell scripting is that you call other programs to do the stuff you don't want to do in shell.

<comment><USER24> Like this: [http://stackoverflow.com/a/1996540](http://stackoverflow.com/a/1996540)

I agree it is tedious, but to be honest, reading and writing to stdin/out isn't something that would commonly need to be done in a robust system. If the world were perfect you would use library functions.

I definitely think there is scope for a language that works well as an interactive shell, _and_ as a general purpose language. They have somewhat conflicting constraints but I'm sure we can do better than Bash. Have you seen how [ is implemented?

<comment><USER46> No stdout or stdin for robust systems? I disagree.

Yeah, a nicer shell-like language would be cool. I've been thinking about it for a while.

Bash is quirky but it gets a lot of stuff right and once you understand it it can be extremely ergonomic and productive.

And not depending on language run times other than shell can be really glorious in some situations, too...

</comment></comment></comment></comment></comment></comment></comment><comment><USER47> Or just put your app(s) into containers and run them through docker compose on a single VPS. That bypasses about 99% of the things listed in this article.

You can still easily set things up so it's a git based deploy which is hands free after the initial push.

Now you have a single $5-10/month server that runs your app's stack without a big fuss. Of course it's not "web scale" with massive resiliency but when you're just starting out, 1 server instance is totally fine and exactly what you want.

I've ran many projects for years on 1 server that did "business mission critical" tasks like accepting payments, etc..

</comment><comment><USER48> I wonder if the original HN "Heroku is Dead..." title will cost Heroku money / market share in indirect ways.

When I see titles like that (despite the fact that it was intended as sarcasm), I think to myself, e.g., "I bet at least hundreds of people who scrolled past it thought it was sincere, and now they will have this subconscious 'Heroku is Dead... Docker...' thought at times when deploying projects. Maybe they'll even check out Docker. Maybe these hundreds of people will represent a tipping point of sorts for Heroku->Docker migrations, because one of them will write a really great blog post about it, and it will receive thousands of views..." (alternate endings of the same thought continue to be brute-forced for a few moments).

Along the same vein of thinking, back in 2008 I had this "realization" that Google could control the world by simply showing results based on headline titles (e.g., a search for "Obama" during the election could have resulted in articles / results whose titles have words/phrases whose presences are positively correlated to lower or higher stress levels, assumptions, other emotions, etc., resulting in a net positive or negative sentiment, respectively, about the subject of the search query, all while simply scanning the results to determine which one to click).

<comment><USER49> > I wonder if the original HN "Heroku is Dead..." title will cost Heroku money > / market share in indirect ways.

This would be true for an average BuzzFeed-consuming-crowd, which -to my knowledge- isn't the case here.

</comment></comment><comment><USER50> This pretty much sums it up. New stuff, stacked on existing stuff, with the theory that older stuff was hard, and new stuff is better, but you still needs the old stuff, so you basically end up doing exactly what you did, but now with more stuff in between, adding cost, complexity and additional failure modes while solving nothing.

Any of the proposed problems that containerization was supposed to fix are already fixed by using proper configuration management. In almost all cases so far, people yammering on about docker and containers (and CoreOS), it ended up being their idea of configuration management, because they didn't have any in the first place.

Say you want to fix your 'problems' with setting up servers, how about doing it the right way. You will need deployment services, regardless of containers, VMs or bare metal. You will also need configuration management services, and monitoring. Containers and special distributions solve none of it, knowledge to run systems is still required and not actually fixing your problems and layering stuff on top of it doesn't actually help.

Get something like SaltStack or Chef, and configure the ____out of everything. It doesn 't care what you're running on, and actually solves the problems that need fixing.

</comment><comment><USER51> I've spun up a lot of kubernetes clusters to test it out. A few months ago I also tested out Flynn, Deis, Deis Workflow, Openstack, and a lot of other options. I still haven't found a simple bootstrap script that gets everything set up on AWS and lets me simply deploy my application. And it's true that storage still seems to be an unsolved problem with kubernetes.

Heroku is great, and free for small services. On the other hand, a highly- available kubernetes cluster is going to set you back at least $100 per month, which is just too much for small startups and side projects before they take off.

I think I'm going to forget everything and head towards [http://serverless.com/](http://serverless.com/). No Heroku, no Docker, no micro-services, no servers. Just everything running on AWS Lambda and DynamoDB. And everything static in S3 behind Cloudfront.

Or maybe just Firebase. But I really am tired of managing servers.

<comment><USER52> So, why not ditch AWS and use GCP? Kubernetes cluster setup in one command, or use AppEngine.

Maybe the problem is AWS.

<comment><USER53> > So, why not ditch AWS and use GCP?

It's a Google product offering, which means it could be EOLed tomorrow. Or this afternoon. Or maybe it already has — better go check their blog.

<comment><USER52> Amazon and Google have almost the exact same language in their ToS for deprecation policies. Also, k8s is open source, so you can at least go somewhere else if you really must.

GCP isn't "Google Labs".

</comment></comment></comment></comment><comment><USER54> Thumb rule: Divide number of backend engineers by 5 you get optimal number of micro-services.

<comment><USER55> I got 0.2

Should I round up or down?

<comment><USER56> Congratulations! It's serverless!

</comment></comment></comment><comment><USER57> I got a genuine laugh out of this quote "Yeah, BDSM. It's San Francisco. Everyone's into distributed systems and BDSM.".

<comment><USER58> Thought it was a joke, then quickly googled Aphyr's blog...

</comment></comment><comment><USER59> Making sort of funny articles about Docker doesn't change the fact that circleci's support for docker is horrible. I don't use it on prod (and I probably won't) but since circleci forces me to use ubuntu 12.04 which is not something I use on prod I want a docker container which looks like my production host. Having said that - circle really tries to support docker, but It doesn't.

I have to use ECS for caching (I am not happy about it)

Builds might fail due to the custom docker version/compilation

You can mock docker, but people are using it in one way or another and you should support it properly.

</comment><comment><USER60> This needs a "(2015)" adding to the title.

<comment><USER61> Because 2015 was, like, 100 years ago...

<comment><USER62> Containers are obsolete. AWS Lamba is the new hot thing.

</comment></comment></comment><comment><USER63> Most commenters don't seem to realise that this is satirical, and that the author actually thinks Docker is 'the future':

[https://circleci.com/blog/it-really-is-the- future/](https://circleci.com/blog/it-really-is-the-future/)

But there is, as the author notes, truth in the satire.

</comment><comment><USER64> I assume this entire article is one piece of sarcasm. Because after reading it, how could any sane person not prefer Heroku?

<comment><USER65> From your comment I see you didn't read the post.

Read it, it's a lovely 5 minutes piece of writing.

<comment><USER64> It looks like I was not the only one to be confused: [https://circleci.com/blog/it-really-is-the- future/](https://circleci.com/blog/it-really-is-the-future/)

</comment></comment></comment><comment><USER66> show me an easy way to push a rails application to aws (with docker) that uses RDS ?

is there ANY way i can spin up a server, add the ssh keys to some configuration file somewhere and just "docker-magic push" and have my rails application running ?

or do "docker-magic bundle exec db:migrate" and have that command run on the server.

Or push a Procfile with worker definitions and have the PAAS automatically pick it up, add it to supervisord/systemd and run it ?

<comment><USER47> Once you finish taking my Scaling Docker on AWS course, you'll have access to a magic command that deploys your app and you're free to optionally run migrations.

[http://nickjanetakis.com/courses/scaling-docker-on- aws](http://nickjanetakis.com/courses/scaling-docker-on-aws)

It covers using RDS, ElastiCache and also handles load balancing your app + much more.

<comment><USER67> You should mention it costs $20 for information that can be found for free anywhere on the web. While other users answer the question, you simply refer to your piggy bank, shameless.

<comment><USER47> > You should mention it costs $20 for information that can be found for free > anywhere on the web. While other users answer the question, you simply refer > to your piggy bank, shameless.

Yep, it costs $20. Basically the cost of chinese food for 2, to ensure guaranteed victory in learning the essentials of AWS' platform while having a guided tour on how to deploy a fault tolerant web app with Amazon ECS from start to finish.

You can definitely learn everything for free, but the value in a course is that you're getting a cohesive learning path that was carefully planned and tested. You get a system that you can apply to your own projects and plenty of source code to reference.

You're paying the $20 so you can avoid spending 6 months trying to figure out everything on your own while stringing together a bunch of half-assed blog posts and tutorials.

You pay the small fee for certainty and it's well worth it because your time (and sanity) is not infinite.

<comment><USER67> I love the comparison to chinese food for 2. As if everyone lives in the US with the same economic situation. Your posts are truly despicable.

<comment><USER47> I'm sorry you feel that way. Unfortunately time is something we all share, and it brings me a lot of joy to have students say that my courses have saved them a ton of time and helped them meet their goals.

I see that you recently left Amazon after making 200k/year there (a comment you made 5 days ago). I can see why you don't like people promoting Amazon products, I fully understand.

<comment><USER67> Condescension abounds! :-)

</comment></comment></comment></comment></comment></comment></comment><comment><USER68> Glad to know there's the same kind of hipsterism going on in the backend world as much as in the frontend world.

You could basically substitute all these backend buzzwords with "Webpack", "Grunt", "Gulp", "Requirejs", "React", "Angular", "Ember", "Backbone", etc. and it would have same effect on the readers--they think you're an annoying hipster.

</comment><comment><USER69> [https://www.gitignore.io](https://www.gitignore.io) runs on Heroku and it gets 40k+ visitors a month on a free Dyno. I use Heroku because I don't want an IAAS solution, I want a PAAS solution. If I wasn't using Heroku, I would probably find another PAAS, before switching to Docker (even though I do love Docker).

<comment><USER70> That's 1 request per minute, you could slap that into a raspberry pi on a 3G connection, and do the same for your next 400 apps that has 1 request per minute.

People seem to underestimate just how powerful modern machines really are. And I don't get why people seem to think it's hard to deploy simple web applications. Just write a 4-line shell script that rsync's, runs whatever DB migrations you may have and restarts the thing.

<comment><USER71> 40k visitors per month isn't one per minute. It's anywhere from one per minute to 40k per second. Division of requests by time is the worst possible mistake in calculating load.

<comment><USER72> Plus visitors != requests.

I have 200k visitors per month generating 8m page views and about 50m hits on the servers (with CDNs taking another few hundred million hits).

These all peak during the UK weekdays and wind down at nights and weekends.

Divisions over time aren't going to work, but neither is translating visitors into requests, especially as it's only the page views that have a beyond trivial computation cost.

<comment><USER69> Yeah, I just checked Cloudlfare and here are the requests

      Total Requests
      Last Month
      622,482

</comment></comment></comment></comment></comment><comment><USER73> One thing that I liked about Heroku was the ease of deployment for Java applications (or any language for that matter as long as you had the right build pack)

Since they upped the cost of their small tier, I moved to Digital Ocean and installed Dokku, which gives me that Heroku-like deployment experience so managing my (admittedly very small) website isn't that much of a hassle.

<comment><USER74> We at Boxfuse ([https://boxfuse.com](https://boxfuse.com)) provide the same ease of deployment for running Java apps (as well as Go and Node.js ones) on AWS. All you need to type is literally: boxfuse run myapp-1.0.jar -env=prod

And you automatically get things like auto-scaling, database auto- provisioning. easy debugging and more.

Disclaimer: I'm Boxfuse's founder and CEO

<comment><USER75> Hey, it looks like your "center-iframe" is pointing to an invalid link. It threw me off while I was looking around. Hope this helps

</comment></comment></comment><comment><USER76> In an earlier age, all we had to hate were frameworks.

[http://discuss.joelonsoftware.com/?joel.3.219431.12](http://discuss.joelonsoftware.com/?joel.3.219431.12)

(Factory factory factory factory.)

</comment><comment><USER77> Out of topic but, man, is coreos not ready for prime time. I've been using it following the stable channel and ended up having to turn off automated updates because it would break docker (docker would just hang with the last 4 updates)... Not a very convincing test of coreos :)

</comment><comment><USER78> Kubernetes/Docker will become increasingly accessible to developers and it will loosen the reliance on lock-in PaaS like Heroku - This is the future; I'm betting everything on it.

With tools like Rancher [http://rancher.com](http://rancher.com), you can already see things moving in that direction. Next step is rancher-as-a- service.

When it comes to developers, I think open systems will always prevail in the end (it's just more flexible).

<comment><USER79> I thought it's more important WHAT you run inside containers than containers itself.

I got really upset about this rancher tool because it doesn't design my database schema.

Shit, future was so close.

<comment><USER78> You still have to write code, define your database schema and declare config files to specify how containers should be orchestrated but soon open source developers will start creating frameworks/boilerplates which capture some of these requirements and which can automatically run and scale on Kubernetes/Swarm/Mesos and this will greatly speed up application development and deployment.

Right now, we think of frameworks as being components (part of) larger software systems - But in the future, frameworks will provide the foundations for entire software systems - They will be be responsible for declaring their own network topologies and resource requirements and they will be capable of scaling automatically to any number of machines.

Developers will extend and customize the framework with their own logic but the framework itself will handle all the difficult stuff related to its own operations.

<comment><USER79> [sarcasm]

</comment></comment></comment></comment><comment><USER80> Don't listen to hype. Look at the problems you need to solve. If you need a big, complex, distributed system, than maybe microservices are a good idea. If you're building a simple webapp... not so much. Things that work in the large don't necessarily work in the small.

I write stuff in Scheme. I'm a hobbyist, there's no reason for me not to, and I love the language. The apps I write are sometimes single-threaded (or coroutine-based) monoliths. But I only have one machine available for me, and the things I'm writing are fairly simple. It's good ENOUGH. And Worse really is Better[1].

1:and I truly mean that in the Gabriel sense. As in the New Jersey model. Not any other way.

<comment><USER81> For people who do not know what "Worse is Better" means, here's the link: [https://www.jwz.org/doc/worse-is-better.html](https://www.jwz.org/doc/worse- is-better.html)

<comment><USER15> WTF is at that link? It's NSFW...

</comment></comment></comment><comment><USER82> This definitely reflects some of my experiences when trying out Docker and some of the other stuff associated with it.

Serious question though: I would absolutely love to have an introduction on how to use Docker to deploy one or two web applications that use a typical amount of backend services, say some sort of database and a redis server. All of this would probably run on a single VM (whether Amazon, DigitalOcean, Linode, ...) and you mainly use Docker to isolate the applications from each other in terms of the environment/dependencies that they need.

How do I do this with Docker in a way that gets me an easy deploy process? (Or maybe the question is actually, should I even do this with Docker?)

<comment><USER83> My personal blog is basically an example of deploying a simple app using docker. HTTPS://GitHub.com/pbecotte/devblog ... The makefile wraps up the actual commands.

</comment></comment><comment><USER84> > No, look into microservices. It's the future. It's how we do everything now. > You take your monolithic app and you split it into like 12 services. One for > each job you do.

 _reader implements and gets massive bill for personal blog hosting_

"Am I doing this right?"

</comment><comment><USER85> I guess this is meant to be hyperbolic but honestly it's true.

> So I just need to split my simple CRUD app into 12 microservices, each with > their own APIs which call each others' APIs but handle failure resiliently, > put them into Docker containers, launch a fleet of 8 machines which are > Docker hosts running CoreOS, "orchestrate" them using a small Kubernetes > cluster running etcd, figure out the "open questions" of networking and > storage, and then I continuously deliver multiple redundant copies of each > microservice to my fleet. Is that it?

> -Yes! Isn't it glorious?

> I'm going back to Heroku.

</comment><comment><USER86> What a nice clickbait title. A is better than B, even though A is an apple while B is an orange, and we use them for entirely different purposes. Some functionality provided by Heroku can be replaced with Docker, and some missing features of Heroku are in the Docker infra, that can be added to Heroku using software (like service discovery: [https://blog.heroku.com/managing_your_microservices_on_herok...](https://blog.heroku.com/managing_your_microservices_on_heroku_with_netflix_s_eureka))

<comment><USER87> It's a humorous lament on the fragmented state of devops, not to be taken too seriously.

</comment></comment><comment><USER88> I actually learned a couple of things from this article even though it's satire. I struggled through the de-monolithing of an application into microservices and then into Docker containers. Do people actually do this without having pressing issues that haven't lead them to this path? Is there any reason to spread into microservices unless the monolith is not keeping up with requests?

I would have never considered Docker containers unless artifact preservation/isolation and deployment issues hadn't forced me to look toward a solution.

</comment><comment><USER89> Look, if you're a one man shop doing a small project — do LAMP. Do perl. Do cgi. Do whatever you're comfortable with; if you try to switch to the latest silver bullet tech, you'll just get disappointed.

But if you're a CTO with a startup with 10+ server-side developers and plan to hire at least as much in near future, suddenly all these dockers and microservices actually make sense.

So, unless you'll start conversations with _who_ you are and _what problem_ are you trying to solve, of course the other side will seem stupid.

<comment><USER90> > _if you 're a CTO with a startup with 10+ server-side developers and plan to > hire at least as much in near future, suddenly all these dockers and > microservices actually make sense_

As a consultant, I often get asked those kinds of questions: "Should we use X?"

Whether it's programming languages, databases, operating systems, whether it's Chef vs Puppet vs Ansible vs Docker vs Whatever, it's a question that comes up a lot.

I generally answer it with "What are your team good at? What have they used, what do they know well?"

There are always exceptions to the rule, but in general I encourage people to play to the strengths of their team, rather than recommending Technology X because it's shiny and bang on-trend.

<comment><USER89> Exactly my point. I used two broad categories just to paint the overall picture; the fact that they "make sense" doesn't mean that they're the silver bullet.

</comment></comment></comment><comment><USER91> I'm not a web developer but I sometimes put some programs on the web for me and my friends. I use FreeBSD Jails for that.

Can someone explain to me the advantages of Docker compared to Jails?

<comment><USER92> In your use case: pretty much no advantages. Stick with jails and be happy.

</comment></comment><comment><USER93> Reminds me of [https://www.youtube.com/watch?v=b2F-DItXtZs](https://www.youtube.com/watch?v=b2F-DItXtZs)

</comment><comment><USER94> Also

Hitler uses Docker: [https://www.youtube.com/watch?v=PivpCKEiQOQ](https://www.youtube.com/watch?v=PivpCKEiQOQ)

<comment><USER95> Really spot on - recommended! "You are a bunch of Node.js hipsters that just HAVE to install everything you read on Hacker News!"

By the way, why all the downvotes to the parent?

</comment></comment><comment><USER62> Since we are seeing so many "ads" here in this discussion. Does the average HN reader prefer this type of advertising to regular adsense ads? There is no violation of privacy, no autoplaying video, they are actually relevant to the discussion and it's obvious that they are trying to sell you something. Oh and probably the most important thing si you can downvote them.

<comment><USER43> I dislike it because they just seem to have read the title and not the actual post.

</comment></comment><comment><USER96> He slowly caressed his docker image while tenderly inserting it in the continuous delivery pipeline. The slow throbbing of the Jenkins service increased in intensity as the automated tests started firing off in a crescendo of etcd writes leaving a quivering micro-service that lay panting in its pod. That was a memorable deployment. The first of many that night...

</comment><comment><USER97> See .... microservices are where you take a process using IPC and force it to go over a network and use TCP ...then you pretend it will run better and scale more but avoid the whole 10x infrastructure growth and the fact the devops team has tripled in size to manage the frankenstein ... dont forget to add in the whole SDN layer the network guys absolutely love

</comment><comment><USER71> I'm not sure I get the message.

<comment><USER98> heroku is dead, docker is awesome, gluten free food does not have gluten ;-)

<comment><USER71> I'm sure I don't get that message.

<comment><USER99> It's all sarcastic. It's poking fun at how trendy advances might not necessarily be better than what existed before, as well as how much complexity is involved in them. It makes the most sense if you're already familiar with the concepts in the article (Docker, Heroku, CoreOS, etcd, and so on), and happenings in the technology industry related to them.

As usual, if a joke needs to be explained (to a person) then it's not funny (to them). I found this amusing since it aligned with my experiences:

> -Since no-one understands Paxos, this guy Diego…

> Oh, you know him?

> -No, he works at CoreOS. Anyway, Diego built Raft for his PhD thesis cause > Paxos was too hard. Wicked smart dude. And then he wrote etcd as an > implementation, and Aphyr said it wasn't shit.

> What's Aphyr?

> -Aphyr is that guy who wrote, 'Call Me Maybe.' You know, the distributed > systems and BDSM guy?

> What? Did you say BDSM?

There are several jokes here based on cultural references related to the aforementioned topics.

<comment><USER100> I learned that <x> will laugh at a joke three times:

* First when you tell it,

* then when you explain it,

* then finally when they get it.

(x used to be English people when I was a kid.)

<comment><USER101> A gentle reminder that it's against the Hacker News guidelines to leave unclosed xml tags in your comments.

<comment><USER102> Surely that's BNF rather than XML?

</comment></comment></comment></comment></comment></comment></comment><comment><USER103> Hype is as bad as seeing a promising technology just as hype without trying understand the problems it might solve.

This is quite frustrating for both people who are aware of those issues and trying to fix them as well as the people missing out on the real advantages of such technologies.

This reminds me of similar sentiment around virtualization and cloud computing later in my peer group:

Some sold VMs as security feature and people focused their criticism on that, without understanding other advantages like quick/self-service provisioning of systems. Later one, cloud computing was trivialized as "it now just somebody elses computer" which completely ignored advantages like no ramp up costs and the ability to problematically manage your systems life cycle.

PS: Considering every new thing a fad probably also makes you consider 'hadoop' the latest shit in big data processing and assume today's tech companies hipster are fighting over wordpress plugins. (Like, really?)

</comment><comment><USER104> As someone who recently drove down the path of learning all the new "hotness" in the DevOps world, I can safely say I came out of it with two learnings:

1\. I have a much better understanding of what's happening behind-the-scenes

2\. For most small startups, you should seriously consider the time (and therefore, cost) of investing in your own infrastructure.

For point #1, I think understanding your options and how they benefit your company is essential for you transition from a small -> medium -> large size company. The paradigms you learn by virtue of researching the new technologies might end up being applicable in other parts of your development process.

On point #2, I partially regret not deploying to Heroku, seeing where our system became stressed, and optimizing. Attempting to scale for things you don't know about yet is tough, and can lead you down a path of wasted time and money.

</comment><comment><USER105> Well I think that people and developers in particular needs to start realising that companies like docker has gotten really good at marketing them selfs towards them. Most of the stuff you hear about a technology is just marketing stuff that makes you sound smart when telling your peer about a new technology.

</comment><comment><USER106> > So I just need to split my simple CRUD app into 12 microservices, each with > their own APIs which call each others' APIs but handle failure resiliently, > put them into Docker containers, launch a fleet of 8 machines which are > Docker hosts running CoreOS, "orchestrate" them using a small Kubernetes > cluster running etcd, figure out the "open questions" of networking and > storage, and then I continuously deliver multiple redundant copies of each > microservice to my fleet. Is that it?

 _exactly. I mean look, if you have a lifestyle business that 's only going to support 5-10 people, it's totally a waste of time. if you have some hope of scaling this is the way to go. I get it, just use Heroku. It's easy and convenient. If you're planning on a billion dollar exit, this way is way better._

> I need to decide if i believe my own hype?

 _yeah. sorry._

</comment><comment><USER107> You know what's better than microservices? Nodes. Nodes running open source software that can power a distributed network.

Microservices often hit the same database. You want to be able to split up the database. Not just into shards, but into distributed nodes.

And by doing this, you split up the whole stack.

</comment><comment><USER108> I believe microservice is the wrong term used to describe SOA. The word 'micro' make it looks simple & applicable for tiny apps 10-50K SLOC. I believe you should start chopping off your monolithic app only when it reach > 100K SLOC. Still you can split it up by well defined modules with clear interface, without necessarily using SOA if it is running on same box.

Having monolithic app does not make it bad. What makes it bad is not having proper modules with proper interfaces.

SOA comes handy when you want to distribute your workload, so now we have proper modules but those modules needs more computing power, so split them up into boxes and pay the pain for managing that, because you have no option.

</comment><comment><USER109> As the author of that "Docker is the Heroku Killer" post that was popular a couple of years ago I have to say that I agree with this.

When I wrote that article it was largely focused on the potential for Docker to create a bunch of Heroku competitors as well as a simplified development experience across multiple languages.

The businesses aren't there yet although a ton are trying. The local dev experience has not materialized yet either outside of native Linux due to performance issues with volumes that only a 3rd party rsync plugin have come close to fixing.

I still use and advocate for Heroku pretty heavily for just about any non- enterprise environment.

<comment><USER110> I'm working on one of those Docker powered Heroku "competitors". This post and your post both rung true for me.

It's a constant balancing act. Too flexible, it becomes overwhelming. Too constrained and you sacrifice a bunch of the perks of using Docker.

The conclusion I've come to is the only way to do it is to be unashamedly opinionated about keeping things simple for the average user. Otherwise you end up having that exact conversation

<comment><USER109> Yea. The most realistic way to get things working IMO is Dockerfiles with 80% use case defaults for parts of the stack. From there if people want/need to dive in and tweak them they'd have the ability to do so but ideally the average user doesn't need to know they are touching Docker much beyond knowing that it's there if they need it.

</comment></comment></comment><comment><USER111> When it comes to micro services and containers, we are are missing fairly detailed descriptions of real world architectures from successful projects. And actually that applies to many other technologies as well.

When it comes to micro service, it would be interesting to know simple things like what kind of services were created, how large are, how communiction is handled, how large team(s) behind the service etc.

For some companies these are of course trade secrets, but sometimes opening things up might be good marketing. An example is Backblaze with their very detailed descriptions of their storage pods.

<comment><USER112> Would you find this presentation useful?

[https://www.infoq.com/presentations/microservices- comparison...](https://www.infoq.com/presentations/microservices-comparison- evolution)

</comment></comment><comment><USER113> Is there a computer-voice teddy-bear version of this?

I find them much better than walls of text.

</comment><comment><USER114> Clickbait titles like that don't make me want to do business with circleci , it feels like a business full of immature kids yelling at each others. Not even going to click on the link.

</comment><comment><USER115> Not exact but somewhat similar feelings expressed here: Let Me Just Code: [http://www.drdobbs.com/tools/just-let-me- code/240168735](http://www.drdobbs.com/tools/just-let-me-code/240168735)

Getting Back To Coding [http://www.drdobbs.com/architecture-and- design/getting-back-...](http://www.drdobbs.com/architecture-and- design/getting-back-to-coding/240168771)

</comment><comment><USER116> What containers do for you is two and only two coarse-grained things. 1. Make you more productive rolling out validated infra 2. Better utilize your hardware. Both of these things imply that you were using VM/AMIs before and you were hand-crafting your entire stack using things like Chef and Ansible. If you weren't doing that before (like, if you were using Google App Engine or Elastic Beanstalk or Lambda), Docker will make you _less_ productive than you are today.

</comment><comment><USER117> I think words are very powerful, in particular "microservices" vs "monolith". By accepting those words, we imply the conclusion: microservices sound sexy and lean and elegant - who can argue with separation of concerns? And a monolith is a big unchangeable rock.

I think we need a better word for apps that are single tight self-contained systems than "monolith". You can design elegant interfaces, and avoid creating a sloppy mess, with function calls or objects too.

</comment><comment><USER118> It's amazing that an article from a year ago can still create a buzz on HN like this. Goes to show there are still a lot of people talking about this.

</comment><comment><USER101> Where do Beowulf Clusters fit into all of this?

<comment><USER119> I'm actually looking into using Docker as an easier way to deploy my software to a Beowulf cluster..

</comment></comment><comment><USER120> "Hi, my name is dokku, and I have no idea what you're talking about" :)

This rant sounds just like any rant from old dev mocking a new tech. "This is less efficient, this is too complicated, this can't be taken seriously, this won't last".

Creating a character obsessed with "this is dead" hardly dissimulate the obsession with "this won't work". Do whatever you please, we don't care. But don't mock others about what they please.

Passing through that, let's address the critics.

Microservices and docker are not necessarily tied. I write only monolithic apps, and use them with docker through dokku.

Etcd is a microservice problem, not a docker one.

You don't need coreos or kubernetes to use docker in production. You need them if you want massively scaled applications, just like you would have many servers running the same app with replication without docker. Most of us don't need that (and those who need it probably won't find it more complicated than what is needed to do that without docker).

If you don't want to manage servers, well, don't manage them. That's what cloud services are made for. But please tolerate some people love devops and not spending much direct money into infrastructure.

<comment><USER121> >This rant sounds just like any rant from old dev mocking a new tech.

Probably because we have seen it all before, and there isn't much "new" most of the time.

<comment><USER120> I've been a professional developer for ten years myself, I've seen my part of hype. But I've seen way more people not even trying new things (and trying does not mean sticking with it on your main project), probably because they could not stand the idea that the experience they are so proud of could be invalidated in any way.

</comment></comment></comment><comment><USER122> I must've missed a tech cycle (or two) - I had heard of Heroku but didn't know what it did. I've used lxc and Docker and read bits about CoreOS/rkt/appc/kubernetes/etcd.

I know it's tongue-in-cheek but few if any of these new fangled things are critically dependent on one another.

</comment><comment><USER123> This comment is over simplistic and reads like it was written by someone who doesn't know what they're talking about. All of these technologies have their place, but they should be adopted incrementally and where it makes sense. Posting a frenetic conversation benefits no one.

<comment><USER124> [https://en.wiktionary.org/wiki/joke](https://en.wiktionary.org/wiki/joke)

</comment></comment><comment> a-priori There's a theory in economics about the optimal size of a firm. How big should a company be? The optimal size could be infinite, where all of society acts as one firm. Think Communist-style command economy. Or it could be one, where everyone acts as networks of individual contractors or single-owner businesses. Think anarcho-capitalism. But in reality it's neither extreme and falls somewhere in the middle; why?

It turns out that the optimal size depends on the balance between the overhead costs associated with allocating resources within one firm and the transaction costs associated with two firms doing business with each other. The overhead costs are higher with large firms because there's more internal resources, including people, to allocate. On the other hand, transaction costs are higher with small firms because each firm does less themselves so they need to transact more with others to accomplish their goals.

As the relative costs vary over time, the optimal size varies too, and firms in an industry will grow and shrink. If it increases, then you'll see mergers and acquisitions produce larger firms. If it decreases then you'll see firms start splitting or small startups disrupting their lumbering competition.

I suspect a similar thing happens in software, where there's an optimal service size. It could be infinite, where it makes sense to build large monoliths to reduce the cost of two systems communicating. Or it could be one, where it's optimal to break the system at as fine a granularity as possible (function level?).

The optimal size depends on the balance of costs. All else being equal, by drawing a service boundary between two bits of functionality you shrink the services on either side but you increase the number of services and add communication costs for them to exchange data and commands.

How these costs balance out depends on the technology, and there are competing forces at work. As languages, libraries and frameworks improve, we can manage larger systems at lower costs. That tends to increase the optimal service size. As platforms, protocols and infrastructure tools improve, the costs to run large numbers of services decreases. That tends to decrease the optimal service size.

The microservices movement, and to an extent the serverless movement, assume that in the medium- and long-term the technological improvements are going to tip the scales sharply in favour of small services. I agree that's likely the case. But we're not there yet, except in some specialized cases such as large distributed organizations (Conway's law). But it's going to be at least a few years before it's worthwhile to build most software systems in a microservice architecture.

</comment><comment><USER125> This is structured like one of those Xtranormal do-it-yourself cartoons from a few years back.

</comment><comment><USER126> Well, repeating buzzwords and pushing for early technology adoption for the sake of early technology adoption seems to be dumb, as the article implies.

But new technology is necessary and early adopters are necessary. Iteration is necessary. Don't punish it.

</comment><comment><USER127> I love everything about this.

</comment><comment><USER128> The post got one critical detail wrong. The microservices won't have their own API. APIs are dead. They'll use Kafka for messaging. That's the future.

</comment><comment><USER129> Duplicate story: [https://news.ycombinator.com/item?id=9688383](https://news.ycombinator.com/item?id=9688383)

</comment><comment><USER130> This reminds me of React state. Sagas, selectors, constants, components, containers, routes, reducers, actions, connectors, shoot me.

</comment><comment><USER131> Check back next year for Docker is dead article.

</comment><comment><USER79> Yet another comparison of hot and soft.

<comment><USER79> sounds like "POSTGRESQL IS DEAD, USE KUBERNETES INSTEAD"

</comment></comment><comment><USER132> Is there really something called "Ew" ? Its hard to Google something thats just two chars.

<comment><USER11> The search term is "Jimmy Fallon ew".

</comment></comment><comment><USER18> Question:

Is there an advantage to using docker when it takes 3 hours to rebuild our relatively small database?

<comment><USER133> Did you read it?

<comment><USER18> Sorry, I was lazy. I did not read it.

</comment></comment></comment><comment><USER134> this old gem is back trending now!

</comment><comment><USER135> doesn't anyone think about performance anymore? I mean just add more vms or increase their config, no one talks about efficient resource utilization asfaik. And there might be times where you containerizing everything is an over overkill.

<comment><USER11> Funnily enough, people do. In fact, containerization came from Google caring a lot about performance! A good intro to how containers help, and to the probable future of computation: [http://m.youtube.com/watch?v=7MwxA4Fj2l4](http://m.youtube.com/watch?v=7MwxA4Fj2l4)

</comment></comment><comment><USER136> Really Hilarious and Reality. Lot of noise about docker, kubernates, microservices etc.

</comment><comment><USER38> Wow, what a strawman. Seems like someone was blowing off some steam.

<comment><USER137> Depends on how you choose to read it. I love microservices and all the new tools we have to scale gracefully, but I read this as satire. It's more hyperbole on how some people always seem to be over-engineering and pretending that's the conservative way to write software without considering the time- sink required to be able to dynamically scale with all of those tools.

<comment><USER38> Calling this satire insults the skill and wit of competent satirists the world over.

It's not clever or funny. It's lazy. It's exaggerated to the point of questionable emotional stability on the part of the writer.

Actually it is funny, but only in an ironic way.

"-Well, Amazon has ECS, but you gotta write XML or some shit."

</comment></comment></comment><comment><USER138> "I thought Mongo was web scale?"

This joke will never get old to me

</comment><comment><USER139> Is it a blank page ? I don't see any text.

<comment><USER76> No, it's not.

Even renders on console browsers.

<comment><USER71> Did you read this on an Xbox?

</comment></comment></comment><comment><USER140> Lovin' it!

</comment><comment><USER141> Did you just tell me to go containerize myself?

</comment><comment><USER142> (2015)

</comment><comment><USER143> Self admittedly, I am one of the P2P psychotic pundits. This is a good reminder that we need to tone our language down.

That said, if you can get your system to work with a single Heroku box, you really truly can simplify your life. That is what we're trying to do with [http://gun.js.org/](http://gun.js.org/) , be able to start with a single machine and no configuration/setup/complexity. Then grow out.

We just had a discussion on the WebPlatform Podcast about all of this P2P stuff ([https://www.youtube.com/watch?v=NYiArgkAklE](https://www.youtube.com/watch?v=NYiArgkAklE)) although, like I said, I probably got too jargony.

But props to circleci for calling out the elephant in the room. Great marketing actually.

</comment><comment><USER144> I'm sure I could go back 5 years and see an almost identical complaint about Heroku.

</comment><comment><USER133> Solution: .NET

</comment><comment><USER78> That's why we started [https://baasil.io/](https://baasil.io/).

The idea is you can deploy any app to any infrastructure of your choice (inside Docker containers). This means that you are not locked into Heroku and it gives you much more flexibility.

It's basically a hosted Rancher [http://rancher.com/](http://rancher.com/) service with a focus on a specific stack.

I think in the future, there will be a lot of services like Baasil.io (specializing in various stacks/frameworks) and managed by various open source communities.

Docker and Kubernetes WILL become more accessible to developers - I would bet my life on it.

I'm currently building a CLI tool to allow deploying in a single command - <USER2> you can get the simplicity of Heroku while not losing any flexibility/control over your architecture.

<comment><USER145> But all you've done is shift the problem to, er, you. If someone is uncomfortable taking out a dependency on say Azure or AWS - the two leading Docker hosting platforms, then they sure as hell aren't going to take out a dependency on "baasil.io" are they?

<comment><USER78> Um no, Rancher is open source can run and manage ANY infrastructure (including Amazon EC2) - You can run it on your own datacenter or even on your own local machine.

Also Baasil.io is essentially just a control panel/dashboard (Rancher-as-a- service), you can quit Baasil.io at any time and switch to your own hosted Rancher instance and you don't have to change any of your application code or change infrastructure providers.

The main benefit of Baasil.io is that it was built by an open source community using open source software and so we can offer the best possible support for apps built on top of our own open source stacks.

<comment><USER145> I took one look at "baasil.io" and saw the typical landing page with a "Plans" link at the top and questioned why anyone would want to take a dependency on this. If there is some OSS project behind the plan-based charade then that's fine.

"Also Baasil.io is essentially just a control panel/dashboard (Rancher-as-a- service), you can quit Baasil.io at any time and switch to your own hosted Rancher instance and you don't have to change any of your application code or change infrastructure providers."

But so you can with AWS and Azure - especially with their Docker offerings. So I'm not sure what problem baasil.io actually solves? If anything it just adds to the list of dependencies and points of failure.

<comment><USER78> The main benefit of Baasil.io is that it offers developers a boilerplate/framework which they can extend with their own code (to build scalable realtime apps and services) - Any app/service built on top of this boilerplate can be automatically scaled to 1000 hosts/nodes using a single command.

Users don't have to use Baasil.io, but if they do, they will get the best possible support - For example, a customer can give us access to their Rancher control panel and this would allow us to SSH into their machines to help resolve any problems in a hands-on way.

It's probably more accurate to describe it as "DevOps as a service - With a focus on realtime apps/services". The value proposition is probably closest to Cloud 66 [http://www.cloud66.com/](http://www.cloud66.com/) except more focused on realtime apps.

Another similar service is Zeit.co [https://zeit.co/](https://zeit.co/) \- Except Zeit.co only runs Node.js. Baasil.io can be extended with components written in any language.</comment></comment></comment></comment></comment>


Ask HN: What's your biggest gripe with CSS Media Queries? - <AUTHOR> I'm currently finishing up with a complete re-write of a 2-year old JQuery plugin for Responsive Web Design (restive.js). It's going to lose the JQuery dependency and have a couple more nice features for modern Web design.
I've never used CSS Media Queries to build Responsive websites, primarily because I found them quite frustrating when I started learning Responsive Web design.
Still, I know a lot of folks use CSS Media Queries and I'd like to find out what your biggest issue with it is? <comment><USER2> I don't have a complaint, but I do have a piece of advice. Mobile-first design is real. Developers work largely on desktops, so they make the desktop version first. Then try to use media queries to apply mobile css using max-width. This seems sensible, but just does not seem to work out well. Instead do the unintuitive thing, design it for mobile first, and then scale up to tablet/desktop using min-width to apply styles.

No matter your choice, go one way. try to avoid mixing min/max-width and the cognitive load is way less.

<comment><AUTHOR> Thanks for your feedback. Tradition is a hard thing to break. I've come across a number of websites that fit the viewport, but don't look quite right on mobile devices and I guess it's because of the way min/max-width works, especially in different orientations. Have you had any challenges here?

</comment></comment><comment><USER3> Watch for iPhone retina and other hi-res devices that have resolution adjustments beyond the standard screen size.

Example query line

      @media only screen and (-webkit-min-device-pixel-ratio: 2) { ...

<comment><AUTHOR> Great! Another thing to worry about. So happy I don't use them. I'm hoping we [as in the Web community] finds a better way to make mobile- friendly/responsive happen, and soon, because it's things like this that drive people crazy. Thanks for sharing.

</comment></comment><comment><USER4> Not really a complaint, but knowing what the minimal set of screen sizes to use to be the most useful to the most number of devices.

<comment><AUTHOR> I feel you on this. Being certain/definitive about your media queries is an impossible expectation.</comment></comment>


Wind, solar farms produce 10% of US power in the first four months of 2018 - <AUTHOR> https://renewablesnow.com/news/wind-solar-farms-produce-10-of-us-power-in-4-mo-617830/ <comment><USER2> There are big battery storage moves coming up too.

A 300 MW/1200 MWh project planned in California by Vistra Energy:

[https://www.prnewswire.com/news-releases/vistra-energy-to- de...](https://www.prnewswire.com/news-releases/vistra-energy-to- develop-300-megawatt-battery-storage-project-in-california-300674786.html)

Another California project, 182.5 MW/700 MWh from Tesla/PG&E:

[https://www.sfchronicle.com/business/article/PG-E-Tesla- team...](https://www.sfchronicle.com/business/article/PG-E-Tesla-team-up-on- big-batteries-13038571.php)

Have We Reached Peak Peaker? 'I Can't See Why We Should Build a Gas Peaker After 2025'

[https://www.greentechmedia.com/articles/read/battery- storage...](https://www.greentechmedia.com/articles/read/battery-storage-is- threatening-natural-gas-peaker-plants)

A Reckoning for Power Giants: Siemens May Sell Its Gas Turbine Business

[https://www.greentechmedia.com/articles/read/siemens-may- sel...](https://www.greentechmedia.com/articles/read/siemens-may-sell-gas- turbine-business)

I remember when solar power was _ridiculously_ expensive. I'm amazed that in sunny regions it is transitioning so quickly from "more expensive than the most expensive fossils" to "cheaper than the cheapest fossils." Not long ago I thought that natural gas was going to have significantly more time to savor its victory over coal before being dethroned in turn.

<comment><AUTHOR> The Tesla/PGE project is estimated to be up to 1GW of storage.

<comment><USER3> Do you mean 1 GW hour?

</comment></comment></comment><comment><USER4> The article says "A bit over 10% of all power produced in the US came from solar..." but given the table doesn't include oil, they clearly mean only 10% of all _electrical_ power. It's only 6-7% if you include oil.

(And don't get me started on the stupidity of measuring power as GWh per 4 months)

<comment><USER5> Technically, if you want total power defined by physics that equates to aka power as defined by physics, then the US is 99% solar just like every other country. Look at the solar numbers for all that farm land which is converted to chemical enegery at low efficiency. Then add in passive solar heating etc.

Interestingly 80% of farmland is used to feed animals which give us 20% of our calories. So the total solar energy required for the average diet ends is significantly boosted. More so if you include fish that are high on the food chain.

PS: Might seem pedantic, but it's more often relevant that you might think.

</comment></comment><comment><USER6> Take note: 10% PRODUCED, not 10% CONSUMED.

This is counting all output by wind and solar regardless if it is needed and usable when the power is being produced. This is quite important because wind and solar are not on-demand sources of power.

<comment><USER7> > _This is counting all output by wind and solar regardless if it is needed > and usable when the power is being produced. This is quite important because > wind and solar are not on-demand sources of power._

I think you have that backwards: in the US, we lack the ability to scale down coal and nuclear plants. Solar and Wind are generally the first to get pulled offline when generated capacity exceeds demand and storage.

TIL this is called "curtailment" and it's an argument that utilities have used to justify not spending on renewables that are saving the environment from global warming (which is going to require more electricity for air conditioning).

Solar energy production peaks around noon. Demand for electricity peaks in the evening. We need storage (batteries with supercapacitors out front) in order to store the difference between peak generation and peak use. Because they're unable to store this extra energy, they temporarily shut down solar and wind and leave the polluting plants online.

Consumers aren't exposed to daily price fluctuations: they get a flat rate that makes it easy to check their bill; so there's no price incentive to e.g. charge an EV at midday when energy is cheapest.

The 'Duck curve' shows this relation between peak supply and demand in electricity markets: [https://en.wikipedia.org/wiki/Duck_curve](https://en.wikipedia.org/wiki/Duck_curve)

Developing energy storage capabilities (through infrastructure and open access basic research that can be capitalized by all) is likely the best solution. According to a fairly recent report, we could go 100% renewable with the energy storage tech that exists today.

But there's no money for it. There's money for subsidizing oil production (regardless of harms (!)), but not so much for wind and solar. There's money for responding to natural disasters caused by global warming, but not so much for non-carbon-based energy sources that don't cause global warming. A film called "The Burden: Fossil Fuel, the Military, and National Security" quotes the actual unsubsidized price of a gallon of gasoline.

Wouldn't it be great if there was some kind of computer workload that could be run whenever energy is cheapest ( ' _energy spot instances_ ') so that we can accelerate our migration to renewable energy sources that are saving the environment for future generations? If there were people who had strong incentives to create demand for power-efficient chips and inexpensive clean energy.

Where would be if we had continued with Jimmy Carter's solar panels on the roof of the White House (instead of constant war and meddling with competing oil production regions of the world)?

It's good to see wind and solar growing this fast this year. A chart with cost per kWhr or MWhr would be enlightening.

</comment></comment><comment><USER8> See what we can do? In a relatively short amount of time, a revolution happened. As prices get lower, unless the state taxes them to the wazoo, DIY- ers can get close to free electricity as after a few years the panels will pay for themselves.

<comment><USER9> Consider what happens when people are mostly energy independant, efficient, and have open source software to turn this into quite a lot of different things.

<comment><USER8> Everything, or virtually anything, is closely priced to energy cost. Virtually everything we use, uses energy to be transformed and then transported to us. If we take care of AC (correlates with hot and sunny days), cooking, charging our cars (why nor charge them when the sun is out?) and other household stuff (we can program to do the laundry at 1pm when the sun is at max) a lot of energy is freed for other stuff. Price drops...

<comment><USER9> I'm jumping even further.. beside the price of renewable energy transformer (solar, wind, hydro..), the tools (heating, cutting, radio, etc) and the metatools (computer, network) what else is there to buy ?

How close to a closed loop synergy are we ?

</comment></comment></comment></comment><comment><USER10> What is interesting is to look at the breakdown in increases of non-hydro renewables:

Non-hydro renewables 149,652 +10.8%

\- of which wind 104,801 +10.5%

\- of which utility-scale solar 18,510 +33.6%

This equates to:

Type 2017 2018 difference

Non-hydro renewables 135,065 149,652 +14,587

\- of which wind 94,842 104,801 +9,958

\- of which utility-scale solar 13,854 18,510 +4,654

In other words, utility-scale solar is increasing faster, but it is still falling behind. The increase in wind power is _almost_ the starting point of utility scale solar.

I would love to know how the residential scale solar numbers fit into this, but that isn't reported here.

<comment><USER11> Aside: Figuring out (roughly) when we can start decommissioning hydro is on my todo list.

Motivation: Restore habitat. Try to save the pacific salmon, humpback chub, others.

<comment><USER12> british Columbia has been building half a dozen new hydro electric power plants per year for the last 20 years. It takes 3-6 years to do the environmental studies. Goats, insects, tadpoles, bears, eagles, fish, plants, you name it, everything is studied to see how it will be impacted. On a recent project they discovered a frog that was on the endangered list actually wasn't endangered at all as it was in every creek in the whole area.

Impacts can be mitigated by creating new habitat.

there are environmental monitors during all construction and weekly reporting to the goverment ministries of fisheries and forests during operation.

The run of river plants only cause unnatural changes in river flows if human life is at risk. Some owners would rather destroy a bearing than kill a fish.

My point is that hydro can operate without much impact on the environment. Flow releases and fish ladders might decrease the absolute amount of energy generated but that small decrease is in the big picture preferable to switching to natural gas for the entirety of the hydro plant capacity.

I don't think any struggles of the pacific salmon or humpback chub can be blamed solely on hydro.

Disclaimer: I program hydro plants for the last 15 years.

<comment><USER11> Solely? No. Significantly? Yes.

I'll google your new BC hydro, of which I know nothing. See what's what. But that doesn't change the situation with USA's dams.

Further, any hope of restoring the pacific salmon runs requires removing hydro. Among many other measures.

You didn't answer my question. So I'll try to restate it: At what point will we have enough solar and wind capacity to moot hydro?

I can't speak to the chub situation.

<comment><USER12> I believe salmon runs that have been impacted can be helped without having to remove dams entirely, but I'm an EE not a biologist.

I also don't defend all dams. Many dams have had terrible impacts on people and the environment.

Hydro could be mooted when there is enough battery storage in MWH as there is stored in the potential energy in the water in the reservoirs of the dams.

The economics on batteries aren't there yet. To be able to depend on having x MWH of energy in the battery in 5 years you need a battery that is 10x, or you buy a battery that is only 1.5x but then replace it every year. A project in Hawaii just chose pumped hydro storage over batteries.

I'm more hopeful geothermal will become a strong resource but all of the drilling equipment and expertise is chasing oil.

<comment><USER11> We cool.

After some quick googling, skim reading, it looks like your govt has tried harder to preserve your salmon runs. Honestly, that's pretty encouraging. And the restoration work on Vancouver Island is exciting.

Since the Columbia (Basin) is also used for irrigation, agriculture, it's been hard, slow work balancing the needs of all the stakeholders. Just guessing: Maybe that's less of a factor for your govt.

Aside: I do think we'll need to be build more reservoirs, cisterns. Higher up. As you know, the snowpack, which used to hold the water, has all but disappeared with climate change. We'll need something to replace that functionality, capacity. So even as an EE, you'll still have a lot of work in the future (assuming you stay with hydro).

Peace.

</comment></comment></comment></comment></comment></comment><comment><USER13> Nice. Wind has passed hydro.

There's a ceiling on wind, though. Without storage, you need some other source. And no, averaging over a large area won't work. We have that now, and there's still about 4:1 variance in a day over the CAISO or PJM control areas.

With solar, peak air conditioning load and peak solar output pretty much coincide.

<comment><USER14> > Without storage, you need some other source

You need storage</comment></comment>


When the Suicide Bomber Is a Woman - <AUTHOR> http://www.marieclaire.com/print-this/world-reports/news/international/woman-suicide-bomber-2?page=all

<comment><USER2> It's fascinating that once you let someone talk, how it changes the perspective.

You start reading feeling sick, to feel sorry at the end.

</comment><comment><USER3> A kid at my college was a Sinhalese from Sri Lanka. He was seriously Buddhist; defending it as the state religion(/philosophy?) and being critical of the Tamils.

On one hand I'd like to see a separate state of Tamils so they can live as they choose, but on the other hand I'd like them to be forced to work out some compromise with the Sinhalese. There are always going to be diverse factions, and we need to learn how to live together.</comment>


Kindergarten class in San Francisco adopts new learning method - <AUTHOR> http://abclocal.go.com/kgo/story?section=news/education&id=6350828 <comment><USER2> I am a very proud product of a Montessori classroom, attending from kindergarten through my "fifth year." To my peers with children, I suggest it is as the ONLY option for elementary education (but lean towards AMI- accredited schools). If you want your child to understand how to problem- solve, instead of memorize facts and tables, I urge you all to seek the same.

I wrote about the perfect world that a Montessori-based education system would yield here:

[http://patrick.snajder.net/blog/index.php?/archives/44-Save-...](http://patrick.snajder.net/blog/index.php?/archives/44-Save- Our.-.-.-Something.html)

By the way: I never had to wear a uniform and, how did I turn out? I like to think people enjoy my presence and I like to solve problems.

</comment><comment><AUTHOR> My daughter starts at this school next week - it's hard to believe this is a public school. Apparently, if the experiment continues to go well, San Francisco will convert additional schools to the Montessori method. This could actually help stem the tide of families leaving the City once their children reach school age - quite a development.

</comment><comment><USER3> There was a Montessori school down the road from my primary school. We thought the students there were pretty weird (they were easily identifiable by their lack of uniforms) but of course their weirdness was more likely due to their parents than due to the school.

I have particularly vivid memories of the fat boy who wore skirts.

In later life I never actually met anyone who went to that school, though, so I couldn't tell you how they turned out. Whatever they did, they did it in very different social circles to mine.

</comment><comment><USER4> Montessori FTW. This is awesome.

The Montessori method is not without its problems. Namely, that it has to end at some point.

I've written more thoughts on this subject on my blog:

[http://jamtoday.beehold.us/post/37396861/solving-school- part...](http://jamtoday.beehold.us/post/37396861/solving-school- part-1-personalization-and)</comment>


S&P 500 Return Attribution: Its 1% economy - <AUTHOR> https://www.putnam.com/advisor/content/perspectives/7816 <comment><USER3> What's with the editorialized title? The actual title is "S&P 500 Return Attribution," and the phrase "1% economy" appears nowhere in it.

<comment><USER4> It's not editorialized. "Those 5 stocks are up 37% on average while the S&P 500 is up 2%, and the remaining 495 stocks are down 5%."

5/500 = 1%.

<comment><USER5> The title is different from the title on the article, so it _is_ editorialized.

<comment><AUTHOR> It can be mine mistake, is it not allowed?

<comment><USER3> Generally not. The guidelines say you should submit the actual title, unless the title is clickbait, uninformative, otherwise misleading, or longer than the character limit.

<comment><AUTHOR> Thanks. Should not have added my text.

</comment></comment></comment></comment></comment></comment><comment><USER6> Stock market returns have always been about a small handful of big winners. It's one of the reasons why indexing usually outperforms stock picking.

The median stock actually performs worse than short-term treasury bills. In other words, cash is a better investment than most companies.

<comment><USER7> Yeah, except that when you pick a company to invest in you are not just picking randomly. Just because most companies perform poorly, says nothing about the ability to single out high performing companies. This advise is completely flawed. Of course if you don't want to spend time on picking the right companies to invest in, an index will likely be a better and safer choice. But with an index, your gains will always be excessively capped. It's trading gains for effort spend. Not even risk. If there is a crash, indices will go down as well. By picking your companies wisely, you can run circles around any index.

<comment><USER8> Buying wisely is difficult enough. Another challenge in picking stocks is selling wisely.

It can be difficult to hold onto the winners and not sell them too early even as they suddenly seem expensive, you're already up 3x or whatever, and besides there's this shiny new IPO/turnaround/whatever where you could put the money instead...

Sticking with the index saves you from these bad decisions.

(I had some Apple stock in 2002 that I sold a year later; some Tesla in 2013 that I sold in 2016; some AMD and Nvidia that I sold in 2017; and some Shopify that I sold last year.)

<comment><USER9> Buy and hold is the better strategy. Even when you think a stock may be overpriced, you incur capital gains to sell and switch somewhere else. Usually better to just hold and defer the gains indefinitely.

</comment></comment></comment></comment><comment><USER10> Stay away from following anything that looks like advice in these comments. I'm frankly shocked at some people's opinions on investing here.

I meet so many investors who think they are geniuses because the portfolio they constructed did so well.

One of the best lessons I've heard about risk is that you don't judge a decision by the outcome, but by the risk being taken when the decision is made.

For example, go to a casino and put your life savings on 10. If you win, did you make a good decision? No.

That's what people who pick stocks are doing. Just massive amounts of hindsight bias and survivorship bias.

</comment><comment><USER11> I would prefer to see this comparison (top 1% vs the rest) for the NASDAQ.

To me it seems kind of normal, that tech is eating the world and is more volatile. So FAAMG making the rest of the S&P 500 look stuck in comparison is no surprise.

But does the same hold true if we compare FAAMG to the rest of the tech market? How about the rest of the software market? If so, that would be alarming.

</comment><comment><USER12> It should be remembered that the S&P 500 is market cap weighted, and so will always be concentrated. This is nothing new:

* [https://theirrelevantinvestor.com/2018/11/26/the-nifty-fifty...](https://theirrelevantinvestor.com/2018/11/26/the-nifty-fifty/)

* [https://awealthofcommonsense.com/2020/07/the-nifty-fifty-and...](https://awealthofcommonsense.com/2020/07/the-nifty-fifty-and-the-old-normal/)

* [https://en.wikipedia.org/wiki/Nifty_Fifty](https://en.wikipedia.org/wiki/Nifty_Fifty)

And while the article isn't wrong, it may not show the whole story:

> _While the S &P 500 is up a 3% or so this year, there are 142 stocks (as of > yesterday's close) in the S&P 500 that are up at least 10% or more. On the > other hand, there are 220 stocks down 10% or more this year._

* [https://awealthofcommonsense.com/2020/08/concentrated-in-the...](https://awealthofcommonsense.com/2020/08/concentrated-in-the-stock-market/)

When the above article was posted (8/4), the top ten YTD percentage risers were, largest first: Carrier Global, DexCom, NVIDIA, West Pharma, PayPal, Abiomed, Regeneron Pharma, AMD, Amazon, Cadence Design Systems. Was anyone surprised that Clorox (53.7%) rose more than Apple (49.1%)?

The S&P 500 being concentrated is the same as it always was over the decades:

* [https://etfdb.com/history-of-the-s-and-p-500/](https://etfdb.com/history-of-the-s-and-p-500/)

* [https://www.qad.com/blog/2019/10/sp-500-companies-over-time](https://www.qad.com/blog/2019/10/sp-500-companies-over-time)

* [https://ritholtz.com/2013/02/visual-history-of-the-sp-500/](https://ritholtz.com/2013/02/visual-history-of-the-sp-500/)

AT&T was in the top ten for _seventy years_.

Absolutely no one was excited about the S&P 500 during the "Lost Decade" of 2000-2009, but now people are probably too excited. Everyone is freaking up about the un-reality of the stock market: let's see how things go for the next ten years and decide then.

</comment><comment><USER13> >There are a handful of stocks doing the heavy lifting and driving YTD returns in the S&P 500 index.

This is interesting, and this analysis is done for 2020 returns only. Were the same analysis be done for any other year, like 1995 or 1985 or 1975, we might find a similar dynamic. Which might make the insight here less interesting.

</comment><comment><USER14> I would love to see a return distribution, not by day, but rather by stock, to see to what extent the distribution is right tailed. I suppose the answer is very.

</comment><comment><USER15> i would be interested to see how the return distribution varies (if it does at all) over the medium/long term 5yrs+ - i bet it does varies - and that's the whole point of the index - it's not news that some stocks do better in the short terms than others. the real question is which ones are good in the long run... so i think this article is rather pointless in its essence

<comment><USER12> > _i would be interested to see how the return distribution varies (if it does > at all) over the medium /long term 5yrs+_

As the P/E goes up, the expected return goes down; see Schiller / CAPE:

[https://en.wikipedia.org/wiki/Cyclically_adjusted_price- to-e...](https://en.wikipedia.org/wiki/Cyclically_adjusted_price-to- earnings_ratio)

This has happened to popular companies before:

* [https://en.wikipedia.org/wiki/Nifty_Fifty](https://en.wikipedia.org/wiki/Nifty_Fifty)

</comment></comment><comment><USER16> Is there an S&P 5 index fund or ETF?

<comment><USER17> if you're only dealing with five stocks, why not just rebalance your holdings yourself every month or so? plenty of zero-commission options these days.

<comment><USER18> No need to rebalance if it is market cap weighted

<comment><USER19> You need to rebalance if the stocks included in your top 5 index change, i.e. if some new stock displaces one of your original 5. Otherwise you've got a fund of 5 stocks chosen by some metric on historical date blah -- which may be a reasonable idea but doesn't seem to align with the spirit of "S&P 5".

</comment></comment></comment></comment><comment><USER20> The Russell 3000 is also up YTD, but is also cap-weighted and therefore I suspect most returns are from the mega corps.

Many people (politicians?) like to point to the S&P 500 when talking about stock market growth.

What is a better index or metric that captures the overall health / growth of the economy?

<comment><USER21> none of them?

stocks have never been more detached from the underlying economy than now - <USER2> mean look at the total market cap of US stocks to GDP - it's something like 170%

<comment><USER12> > _stocks have never been more detached from the underlying economy than now_

Economic indicators (CPI, GDP, unemployment) are backwards looking, while stocks are generally forward looking.

The S&P 500 peaked around mid-February and then started tanking. Meanwhile, in early March:

> _On March 2, New York City Mayor Bill de Blasio tweeted that people should > ignore the virus and "go on with your lives + get out on the town despite > coronavirus."[73][74] At a news conference on March 3, New York City > Commissioner of Health Oxiris Barbot said "we are encouraging New Yorkers to > go about their everyday lives."[75]_

* [https://en.wikipedia.org/wiki/COVID-19_pandemic_in_New_York_...](https://en.wikipedia.org/wiki/COVID-19_pandemic_in_New_York_\(state\)#March)

Cuomo then declared a state of emergency on March 7— _at least_ two weeks after stocks started 'acting'.

The stock market (as 'an entity') dislikes _unexpected_ news: as new information ripples out, the Efficient Market Hypothesis purports that will effect prices. So if a company says a quarter or two will be good, and it turns out bad, their stock price will effected by that _new_ information. But if everyone already knows that things will be bad, when the bad news is announced there won't be much of a reaction.

You can have bad news as long it's expected: the pandemic was unexpected and so everyone had to re-organize their strategies. That was February to March.

Now that the pandemic is more of a know quantity, and its effects of society are more well-known, people can focus on how companies will deal with it: it's generally bad for cruise lines and airlines (stocks down), it's generally good for companies that help with work/shelter-in-place stuff like Zoom/telcomm and cloud (stocks up). What was April onwards.

Yes, the economy does suck. But stocks are about forward looking expectations, not about the now or past.

<comment><USER21> i don't think you can look at Tesla surpassing Walmart in market cap today and rationalize it as normal forward looking behavior when there's been zero news over the last 10 days outside of a stock split, which in theory should have zero impact on the market cap

my point is we aren't living in a world where a stock is simply the present value of future cash flows and no one should rationalize the market as such

<comment><USER12> It's going up because it has gone up. See the momentum investing factor:

* [https://www.investopedia.com/terms/m/momentum.asp](https://www.investopedia.com/terms/m/momentum.asp)

Also, it has just recently become eligible to become part of the S&P 500 (through some perhaps 'amusing' accounting), so a bunch of folks may be front running it's possible actual inclusion:

* [https://www.thebeartrapsreport.com/blog/2020/07/19/gaming-th...](https://www.thebeartrapsreport.com/blog/2020/07/19/gaming-the-sp-500-inclusion-process/)

Though long-term stock holders may not actually want inclusion:

* [https://www.nber.org/papers/w27593](https://www.nber.org/papers/w27593)

<comment><USER21> that has nothing to do with forward expectations, which is my point.

<comment><USER22> If that's what you believe then you can always put your money where your mouth is and go short on Tesla.

<comment><USER21> there's no point in trying to act rationally when the market is in extreme moral hazard mode.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER23> How does this compare to pre-covid S&P 495?</comment>