Drupal Remote Code Execution vulnerability exploited widely - <AUTHOR> https://drupal.sh/drupal-remote-execution-vulnerability-exploit-sa-core-2018-004 <comment><USER2> I work at a company specializing in Drupal services. In fact, it was one of my coworkers who discovered the Drupalgeddon2 exploit. My opinion is that people and companies should be careful with using any CMS. All of our customers have a genuine need for using Drupal, but the truth is that for most companies, there is no need for an outward facing CMS. If there is no user interactivity, I strongly recommend to anyone to build their site to static HTML and serve that if at all possible.

Maintaining a CMS is a lot of work. We all stayed up late last night patching all our supported sites. I think it's worth it for companies to completely outsource maintenance to specialized companies. These recent patches seem back up that claim. I'd guess that the vast majority of unpatched sites are self- maintained.

<comment><USER3> > I strongly recommend to anyone to build their site to static HTML and serve > that if at all possible.

Is there any products that support CMS as backend and automatically generate HTML as frontend OOTB?

<comment><USER4> I recommend gatsbyjs (react.js based static generator) and it has plugin for Drupal and Wordpress.

Here is the location of the Drupal plugin: [https://github.com/gatsbyjs/gatsby/tree/master/packages/gats...](https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby- source-drupal)

And the headless Drupal GatbyJS demo: [https://using- drupal.gatsbyjs.org/](https://using-drupal.gatsbyjs.org/)

<comment><USER5> I really like Gatsby & static site generators, especially when hosted on something like Netlify.

Have you used it with a Drupal or WordPress on a larger site? To do a decoupled site, every time someone publishes a new article or a change, you are going to have to do a ton of HTTP requests during build time for Gatsby since incremental builds are not a thing yet. If you use the Drupal Paragraphs module, that can really complicate things as well. The only way around this with Gatsby that I can think of is to have Drupal create a repository of static files that get incrementally updated. Then Gatsby could grab a compressed version of this during build & create a new version of the site.

I would love to hear thoughts from anyone else though as I'm sure better ideas must exist.

<comment><USER6> I noticed this too. I wrote a WordPress plugin to trigger a TravisCI build when I publish a post and this works great, but it takes 5 minutes or so to publish a new version of the site. For me this isn't a huge deal, although waiting 5 minutes to fix a grammar issue sucks, but for some this is going to be a bigger deal when they're used to being able to make changes instantly.

I noticed that the Gatsby WordPress plugin hits every endpoint on your site to build the graphql data store, you could probably modify it to just hit the ones you need. Additionally I feel like there should be a way to do persistent incremental builds. At least there should be a way to cache the graphql stuff. Maybe a incremental webpack build plugin exists.

<comment><USER5> Incremental builds via Webpack being worked on. Gatsby is waiting on these - [https://github.com/gatsbyjs/gatsby/issues/179](https://github.com/gatsbyjs/gatsby/issues/179)

</comment></comment></comment></comment></comment></comment><comment><USER7> I offer free hosting and administration for a non-profit's Drupal site. It has been a wild month. A few weeks after the March RCE issue, Drupal security team announced that the bug was being actively exploited in the wild. For the RCE announced yesterday, they sent a heads-up on Monday about upcoming critical security release. I was on stand-by and applied the patch within minutes of it being released yesterday. And sure enough - it was still rated "Critical" when I applied the patch, but within only a few hours the ongoing exploits had been adapted to take advantage of this new RCE and the bug was re-assessed as "Highly critical".

Kudos to the Drupal security team for their work. I bet it's not the easiest codebase to work with, but at least the security team are doing their job well.

</comment><comment><USER8> My biggest gripe with tools like Drupal (& WP and other FOSS CMS like things) is that updating them often seems to break things. You have a custom layout and upgrade? Shit breaks. Have some custom plugins? Shit breaks. After the first couple of updates, your client no longer wants to pay you to fix things after an upgrade, so you stop upgrading. Inevitably a remotely executable flaw is found, and you're now fucked.

<comment><USER9> While I do agree, Drupal does publish just the patch for these critical updates for both D7 and D8. You can patch an old version of the site pretty quickly without upgrading or working back in the inevitable core hacks. Each of the big 3 Drupal fixes have been maybe 30 lines of code across 3 or so files, IIRC.

Now if they aren't willing to even pay for that minimal level of service I have little sympathy.

<comment><USER8> There are so many people out there who pay someone 500eur/usd for a simple drupal website, and then expect to pay maybe 50/year for hosting, but are entirely unwilling to pay for anything else. After all, they can get dirt cheap hosting or free wordpress sites all over the internet, so paying anything at all makes them seem like a good customer.

You get what you pay for. Unfortunately, many of those unpatched websites end up causing trouble for others...

</comment></comment></comment><comment><USER10> > Nowadays Drupal is mostly marketed as a robust enterprise tool for markets > to hold critical data

Who markets it as this? That's utter madness!

<comment><USER11> This is what millions in enterprise funding gets you. A decade+ of sunk cost fallacy.

<comment><USER12> What are the alternatives in ""enterprise"" CMS, though? Would you rather use Sharepoint?

<comment><USER13> When someone suggests an enterprise CMS, the usual best case outcome is to go back to the requirements analysis and remove enough requirements until an enterprise CMS is not required.

I've done my fair share of work on the things over the years and the outcome has never been the best one for the organisation.

<comment><USER14> Ah yes, training up people to handcraft html every time a new press release needs to go up on the site, or a VP changes on thecabout us page. Not fun.

<comment><USER13> Hire someone to do it. Cheaper than an enterprise CMS is to run and/or commission.

<comment><USER14> Lets see, my Wordpress install running a well known theme, using Wordfence to alert for plugin updates, with me keeping an eye on it cost about £2,000 to set up and has ongoing costs of about £1,000 a year.

No. Its not cheaper to hire someone to hand-code the site and then hand-code every change.

<comment><USER13> If you're doing that, just use wordpress.com. Wordpress is way out of scope for the term "enterprise CMS".

<comment><USER14> So we've gone from "there's no need for a CMS, write HTML flat files" to "Use a hosted CMS" \- OK.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> I really need to take a day or so and stick those 10 odd static pages in a jekyll theme to get rid of this ageing D7 site I am maintaining. This endless stream of updates is getting very tiresome.

The price one pays for being somewhat lazy (and bad at designing websites).

</comment><comment><USER16> Am I reading the announcement correctly, that only registered users on the site can exploit this specific vulnerability?

The seriously scary part about the previous vulnerabilities like Drupalgeddon was that _anyone_ could exploit them without having an account on the site.

<comment><USER17> There used to be many Drupal-based sites which offer anonymous registration. Last time I worked with Drupal was 5 years ago so maybe the situation changed.

</comment></comment><comment><USER18> > To make matters worse for Drupal security records, the vulnerability is > being actively exploited hours after the patch was released by the Drupal > core team.

This is true for Drupal 7, but not for Drupal 8. Nothing in wild has surfaced regarding SA-CORE-2018-004 for D8 yet.

</comment><comment><USER5> A giant warning label should be required for Drupal, Wordpress & any CMS that offers a large market of plugins.

"Plugins are dependencies that if not maintained actively, may cause you to fall behind on updating the core CMS. Not keeping your CMS up to date will crush your website if an emergency security patch comes out & you can't update to it."

I get that this could be said with any framework or code base, but as someone whose job is to update a couple Drupal sites, I've almost been burned on this way to many times. Content editors rely on a plugin. That plugin isn't getting updated as fast as it should & Drupal introduced breaking changes so the plugin won't work with the new Drupal version. We have to hold back an update until the plugin gets patched. Fortunately I had just finished updating our sites before this security announcement.

The best advice I can give to anyone managing multiple Drupal sites would be pick your plugins carefully, make sure you have a testing server, make sure you can set aside some time each month to run updates & fix the many issues created by those updates, and create integration tests.

</comment><comment><USER19> It's ugly but I updated just a few minutes ago, although the patch has been available for 16 hours. The thing is, `drush up` did not show available updates for me yesterday, and I was hesitating between going to bed and manually applying a patch. Finally, I went to bed.

It's a pity Drupal infrastructure was not up to the task of distributing updates to everyone at the moment of announcement.

<comment><USER20> what you were probably missing is drush pm-refresh.

drush updates its information about the latest available versions only every few hours. Which... in times of urgent severe vulns that need to be patched immediately is probably something they should reconsider.

<comment><USER21> Similarly, if you use Composer, it can cache things for up to 10-15 minutes. Also, Drupal.org's subtree split process takes 5-10 minutes to push out a new core update to Packagist, so if you wait for the update using Composer it could be 20-30 minutes.

The most reliable method is to use Git, Drush, or the patch plugin with Composer to apply the patch immediately, and push it out to your servers. Then update to the latest core version and push that out to your servers once it's ready.

The security team usually links to the raw patch file for each new version in the CVE.

</comment></comment></comment><comment><USER22> Looking over the code pre-patch, can someone with more knowledge explain to me how that allows RCE?

<comment><USER23> more info here [https://research.checkpoint.com/uncovering- drupalgeddon-2/](https://research.checkpoint.com/uncovering-drupalgeddon-2/)

</comment></comment><comment><USER24> Being a Drupal user since 2000(yes, that's 18 years) I now am moving all my sites to wordpress. Never liked Drupal 8 is the main reason.

That been said, Drupal 8 might be a good fit for large sites that needs flexibility and customization, for most (smaller) projects though, wordpress seems a better choice these days, its popularity proves that.

<comment><USER5> Have you tried Ghost or Netlify's CMS? I would highly suggest them for small sites over Wordpress. I find Wordpress runs into many of the same issues as Drupal.

<comment><USER24> heard about both.

Ghost looks like more for the hosting service similar to wordpress.com. I want to self-host, so it is not the best option.

a quick check did not reveal how Netlify does its CMS, e.g. post can be public/private/user-access-by-permissions, note CMS means I can classify various levels of access rights, i.e. content-management(instead of just publishing). Most CMS are geared towards public viewing, which is what blog does, but CMS should do more,

Drupal with its access module got CMS right(still not built-in though), Wordpress is similar, default for public viewing, but you have plugin to control each posts' access rights. I have yet to see an OSS CMS without the need of modules/plugins to make it a true CMS.

<comment><USER5> You can host Ghost on your own but it doesn't have much for user access levels.

Netlify offers something but I haven't used it - [https://www.netlify.com/docs/identity/](https://www.netlify.com/docs/identity/)

</comment></comment></comment></comment><comment><USER25> Will formal verification reverse the trend of old root bugs as a law of nature?

We applied the D6 patch immediately, 911b/admin/reports/status says 6.35. The avg active acct age is >10y. Anon users can do nothing. Is there anything else to fix? Moved to Drupal 04/2005.

What is the longest running D* site?

<comment><USER20> >Will formal verification reverse the trend of old root bugs as a law of nature? >We applied the D6 patch immediately,

Formal verification can do nothing to help people who run software that's been outdated and unsupported for several years.

<comment><USER25> IDK why you seem annoyed, Drupal is so far from formal verification it's obviously a different subject... this bug didn't care what D version you were on anyway. There are many advantages to staying behind the upgrade curve, especially on simple software.

You can spend a bunch of time going from 6 to 7, or you can spend almost the same amount of time going from 6 to 8. It's not like there are major features the users need, most of the work goes into not breaking existing customization they expect.

</comment></comment></comment><comment><USER26> Is there a patch for drupal 6, as there was for SA-CORE-2018-002?

<comment> ar-jan Yes: [https://github.com/d6lts/drupal](https://github.com/d6lts/drupal)

</comment></comment><comment><USER27> Is there some pattern or rule you can put into Varnish/CDN/nginx to prevent this??

<comment><USER28> You really need a Layer 7 kind of thing to be able to inspect the actual payload of the request. I don't think any of those can do that. You'd need an actual WAF.

<comment><USER29> Cloudflare was able to protect against Drupalgeddon 2

[https://blog.cloudflare.com/keeping-drupal-sites-safe- with-c...](https://blog.cloudflare.com/keeping-drupal-sites-safe-with- cloudflares-waf/)

<comment><USER28> That's a WAF layer though. I don't know of a way to do this with Varnish or a straight CDN.

</comment></comment></comment></comment><comment><USER30> there is this shit [https://github.com/dreadlocked/Drupalgeddon2](https://github.com/dreadlocked/Drupalgeddon2) that i saw in my logs.

how to report it to github

<comment><USER31> I'm not sure why you think GitHub ought to take down code for a proof of concept.

<comment><USER30> its not exactly proof of a concept, there is also a version that allows you to execute anything "as in wget a backdoor" look at the code, with this code you can execute anything at the target/victim.

hell, look right at the message when the target is vulnerable.

Good News Everyone! Target seems to be exploitable (Code execution)! w00hooOO!

<comment><USER31> Pretty much any non-obfuscated PoC that anyone could come up with for this vulnerability would be trivial to adapt to run malicious code. This doesn't really lower the bar for anyone, and as long as there's no malicious payload, it doesn't seem legally or morally wrong.

(This is not to say that you couldn't end up in a situation where you'll spend a few years and a truckload of money in a legal fight because you've released something like this.)

</comment></comment></comment></comment><comment><USER32> That's why i did't work on drupal after 2012..

<comment><USER33> what you work on then?

</comment></comment><comment><USER34> I legitimately wish that there were procedures and infrastructures in place for the core teams of things like Drupal to exploit such RCE vulnerabilities themselves (finding installations by the standard means that the bad guys also use) with a payload that applies a suitable patch to fix the vulnerability (definitely in a very-cautious way, e.g. only if the entire file to be patched matches an expected checksum), and then email the site owner if possible to declare what has been done.

<comment><USER17> Good intentions but honestly, that would be a legal nightmare.

<comment><USER35> Well, you could make it and opt-in service with all legal disclaimers applied.

<comment><USER34> In that case it's completely useless.</comment></comment></comment></comment>


Dropbox + Google Doc? - <AUTHOR> https://www.producthunt.com/upcoming/schema-closed-beta <comment><AUTHOR> Most all in workspaces have three problems:

1\. They are incredibly difficult to get started. With so many features in one package, users often get lost. 2\. Their version of sheet, trello etc. is often not as good. Additionally, they usually feel slow and clunky. 3\. None of them address the problem of document management. If you rely on pro desktop software (Photoshop, Sketch etc.), you end up still having to jungle between stuff that's online and stuff that's offline.

In short, they offer an incomplete experience. We found a new way to crack this problem by integrating different services rather than building our own version of everything.

And the coolest thing is, you can use pro desktop apps along with your favorite web apps and organise them all in one place!

Put simply, it's like Dropbox + Google doc and much better. Check out the link to find out more. It's now on closed beta.</comment>


Ask HN: What is wrong with this approach? - <AUTHOR> Hi,
In this last paper http://lambdaway.free.fr/workshop/?view=els_2018
1) Following a short introduction understandable by an average people, I show how three rules and a single predicate function working on simple words are sufficient to introduce the key concepts of a functional language, data structures and recursion. I am not aware of any presentation of recursion being more simple, relegating the Y-combinator to the rank of a fun exercise, and in any case secondary. A kind of lambda-calculus for dummies!
2) As a first application, I build the set of natural numbers defined as lists, replacing the rather old Church's approach and leading to a simple and "symetric" definition of the standard operators [+,-,*,/,%]. Just an exercice demonstrating what can be done with so little foundations.
3) Finally I show how to use the modern web browsers' powerful functionalities to build on this single and strong basis, "on demand" and online, more and more sophisticated libraries leading to a true programmable programming language.
Since a long time I try to share - for free - such an open source personal work with people around me, especially here in Hacker News. I thought it was the place to do that. But zero echo, nada, at least concerning the "substance".
So my question is « What is wrong with this approach? »
Your opinion is welcome.
Please, don't be evil! <comment><AUTHOR> The workshop: [http://lambdaway.free.fr/](http://lambdaway.free.fr/)

The paper: [http://lambdaway.free.fr/workshop/?view=els_2018](http://lambdaway.free.fr/workshop/?view=els_2018)

</comment><comment><USER2> I didn't read further than the abstract because I couldn't tell why we need a new language.

<comment><AUTHOR> Maybe to understand the making of a language, it's the subject of this paper.

And « There are hundred of wiki engines and hundred of languages! Why yet another wiki and another language nobody will ever want to use? »

Here some elements to begin an answer:

A wiki[1] is a web application which allows collaborative modification, extension, or deletion of its content and structure. Following the first wiki created in 1995 by Ward Cunningham hundred of wikis have moved us from simple consumers to creators of shared informations. The best known of wikis is Wikipedia, full of rich documented pages written by authors supposed to be neither web designers nor coders. Wikis come generally with rather rudimentary and heterogeneous syntaxes to enter, enrich and structure texts. At the lowest level documents are created using HTML & CSS syntaxes. But writing HTML/CSS code being rather complex and at least tiresome, intermediate syntaxes, for instance Markdown syntax, have been created to make things a little bit easier. Everything works well but the code quickly becomes rather obfuscated, difficult to write, read, edit and maintain. In fact, the Markdown syntax is not intended for writing rich documents.

Works have been done to build enhanced syntaxes, true languages, in order to unify authoring, styling and coding, for instance CURL, LML, Scribble, SXML, LAML, Pollen ... But these tools are definitively devoted to coders, not to web designers and even less to authors, making difficult - if not impossible - a true collaborative work. Hence the {lambda way} project!

More on that in [http://lambdaway.free.fr/](http://lambdaway.free.fr/)

</comment></comment><comment><USER3> Try asking on lobste.rs, more people there who like this sort of thing.

<comment><AUTHOR> Thank you for the link. Signup is by invitation only to combat spam and increase accountability. Could you invite me?</comment></comment>


Drugmakers hiked prices 1k% in price-fixing scheme, states allege - <AUTHOR> https://arstechnica.com/science/2019/05/drug-makers-hiked-prices-1000-in-massive-price-fixing-scheme-states-allege/ <comment><USER3> > "We have hard evidence that shows the generic drug industry perpetrated a > multi-billion dollar fraud on the American people,"...

So, force them to reset prices. Don't slap them on the wrists with easily paid fines. Make them reset prices.

<comment><USER4> And put some people behind bars as well.

</comment></comment><comment><USER5> There is a serious issue with price fixing in pharma. Whether it's generics colluding, fake patent extensions, or name brand producers paying off generic; it needs to stop.

The current practice is to mark drugs up sky high and rebate the prices. Maybe someone can educate me, but it looks very similar to money laundering IMHO.

<comment><USER6> It certainly helps that the U.S. government _can 't negotiate_ Medicare prices, too.

<comment><USER7> If there ever was a clear case for a law that should be revoked, it's that one. The whole point is that you can negotiate prices and make medical care cheaper for everyone. Banning that is the clearest evidence that the system is designed to increase corporate profits, not heal people.

</comment></comment></comment><comment><AUTHOR> Submission statement of sorts: the NYT article for this was submitted twice in the last two days but got no attention; the pharmaceutical industry seems to be a topic of interest here (including some non-mainstream views on characters like Martin Shkreli), so I'm interested to see HN's take on this.

<comment><USER8> It's important to note that this case is for generic drugs, not brand name drugs. The expensive drugs that make the news are generally the branded kind. Generic drugs are usually very cheap due to the level of competition.

Interestingly, the US has some of the cheapest generic drug prices in the world due to the level of competition. I had read that for some generic companies, they make almost 100% of their profit by wining patent cases. Once competition ramps up, their margin is razor thin.

So based on that, I guess it's not that surprising companies would collide in order to improve margins.

<comment><USER9> Can you please share where you read that "the US has some of the cheapest generic drug prices"?

My mom, who lives in a southeast asian country, came to visit me in the US last month and I asked her to buy generic antibiotics; ear drops; artificial tear drops; generic antihistamines; and they all are much cheaper than what I would have paid here (and are equally effective in a way that I've been weathering fine in this allergy season with these drugs). My significant other, who is from the same country as I, is a medical resident in the US and she still orders meds from her parents.

<comment><USER10> This is a complicated one: US generics are vastly more expensive than international generics, but at least part of that is because US manufacturers sell those drugs abroad at the cost of production, making up their remaining costs in the US. People in the industry have described it to me as a tactic of "we help people in need, please don't regulate us".

This isn't the whole story; east Asian manufacturers also produce cheaper products, for a bunch of reasons from lower costs to lower regulation to lower profits. And it's notably not the case for non-essential, non-basic medicines (e.g. most psychiatric meds), expensive new generics not used abroad, and certain other high-demand generics like insulin. Those are either expensive _everywhere_ , or more expensive in the US than other wealthy countries and largely unused in poorer countries. (How all this relates to your story depends a lot on whether we're talking about Bangladesh or Taiwan.)

But as far as generic antibiotics or antihistamines, it's going to be hard to separate "US manufacturers overcharge" from "US manufacturers sell cheap abroad".

<comment><USER4> > US manufacturers sell those drugs abroad at the cost of production

Not everyone agrees with that statement.

First of all, purchasing negotiations are all secret. US manufactures could very well be saying one thing, and doing another.

Donald Trump said something along those lines some time ago, calling EU "Freeloaders". There is a Wall Street Journal article echoing his comments.

Here's an answer from EU's Health Commissioner Vytenis Andriukaitis:

[https://sciencebusiness.net/news/drug-prices-row-eu- health-c...](https://sciencebusiness.net/news/drug-prices-row-eu-health-chief- hits-back-trump-view-europe-free-loading-us-innovation)

tl;dr: according to him, Trump was just lying.

<comment><USER10> I should have been clearer about "abroad"; I didn't mean Europe. Trump is referencing the claim that Americans pay for drug development and Europe gets cheap products from it, which is dubious at best. And I don't think US- developed medicines are being sold at-cost there, even if they are cheaper.

What I'm talking about is medicines that are donated, sold with differential pricing, or assigned 'voluntary licenses' to increase medical access in underdeveloped countries. Voluntary license medicines are required to be produced with substantially different appearances from their developed-nation analogues, specifically so that they won't be exported back to undercut those products.

All of those programs are fundamentally good things which save lives, but they lead to some pretty strange outcomes and in turn strange reporting. It can be vastly cheaper to obtain a medicine in an underdeveloped country with supply shortages than in the first world, and that sometimes also prompts articles saying that such-and-such a medicine "only costs 10 cents in other countries!" As far as I know, this is mostly an issue in the US; most other high- differential-price countries have socialized healthcare or other functional solutions like Singapore's, but the US has a lot of extremely poor people who end up paying much higher prices than equally impoverished people abroad.

[https://apps.who.int/medicinedocs/documents/s17815en/s17815e...](https://apps.who.int/medicinedocs/documents/s17815en/s17815en.pdf)

</comment></comment></comment></comment></comment></comment><comment><USER11> They should levy punitive measures in the hundreds of billions of dollars and go back to old pricing.

</comment><comment><USER12> Fuck em - just buy overseas knock offs. If they're going to abuse the system I have no qualms about abusing it as well. China is happy to make generic knock- offs and sell them at a fair market price.

<comment><USER13> That a person buying medicine for personal use from abroad could be considered 'abuse', just goes to show how sickeningly skewed in favor of corporations the system is.

There was recently a story on HN of a journalist's home raided to identify his source. Perhaps that is the kind of system abuse that is called for - <USER2> SWAT teams to the homes of each of the CEOs, board-members, and major shareholders, and ransacking them for any signs of wrongdoing, while holding the accused in handcuffs.

It happens every day to regular people for non-violent crimes in the US. Why should the rich have a different law-enforcement system? Yet this call for _equal treatment_ sounds almost extremist...

</comment></comment><comment><USER14> The whole industry needs to be nationalized, full stop, and the current people in charge need to all be in jail.

<comment><USER15> Yes, put this in the hands of the same bureaucrats that allow this sort of thing to happen without penalty in the first place. That sounds like a fantastic idea.

<comment><USER16> It's not like it doesn't work in other countries (see France).

</comment></comment></comment><comment><USER17> So they're taking a playbook page out of pharma bro Shkreli.</comment>


YouTube Displaying video in 3D - <AUTHOR> http://blog.siliconverse.com/2012/01/18/youtube-displaying-3d-video/

<comment><USER2> welcome to internet. It's available since mid 2009.

<comment><AUTHOR> available on YouTube videos since 2009? Never seen them before. I am a regular youtube user.</comment></comment>


Net.wars (1997) - <AUTHOR> https://nyupress.org/netwars/contents/contents.html <comment><USER2> The thing that strikes me the most about the writings here is that even at their most snarky they are not cruel or unkind...

I'm not totally free of guilt in this respect but I wish we could find some way to not default to "vitriol" in our internet communications.

<comment><USER3> this is probably the first step

</comment></comment><comment><USER4> Seeing the 1997 in the link made visiting so tempting as I knew there would be a fast easy to navigate design.

Of course it also has the nostalgic appeal.</comment>


Comparison of Abbott ID Now and Abbott m2000 methods for detection of SARS-CoV-2 [pdf] - <AUTHOR> https://jcm.asm.org/content/jcm/early/2020/04/23/JCM.00798-20.full.pdf <comment><USER2> So, if I'm reading this report correctly, if either of them say you are positive, then you are almost certainly positive.

But IDNow seems to lack the ability to detect some weak positive cases, and give you a false negative. So, if you come up as negative on IDNow, then you need to be retested with the more sensitive process.</comment>


Ask HN: What talks or videos do you suggest to watch on a long flight? - <AUTHOR> <comment><USER3> I can't do anything productive or mentally taxing on long flights, I don't know if it's the air pressure or premature jetlag or what, but I have a very hard time exercising my will. What I do enjoy, though (in addition to the Marvel comics movies – I only watch them on planes) is reading documentation. It's a great time to sit down with an in-depth manual and just soak up information you didn't have before. You don't have to do anything, or make anything, you just learn how something works. Another related activity is reading a bunch of code that I know I'll be working on in the near future. There's something very pleasurable about absorbing a codebase while at the same time completely excusing myself from actually _doing_ anything.

<comment><USER4> I've experienced the same thing. I had planned to spend a 14hr flight gaming and after 4hrs, had neither the ability not will to continue. I'm completely uninformed on the subject, but wondered whether it was the effect of a thinner atmosphere on cognitive abilities.

<comment><USER5> Same here. I think it's because most of the time, I have to get up early to catch a flight and I don't get enough sleep.

</comment></comment></comment><comment><USER6> Another one of those threads that causes intense navel-gazing, shame, and mounting disgust in my gut for myself as I realize that everyone around me is working hard to better themselves while I just binge-watch movies.... and then my eyes glaze over as I mollify myself with the movies until I forget that this thread exists.

<comment><USER7> Life is short, enjoy your time watching movies. I have wasted a lot of time being busy (reading non-fiction, TED talks etc) that did not translate into any noticeable self improvement or progress in my life. I have feeling a lot of people makes that mistake too.

<comment><USER8> I find I enjoy watching documentaries and reading technical/history/biography books often more than fiction. Because fiction is, ultimately, fake and manipulative.

<comment><USER9> I'm reading your comment as implying that fiction is more manipulative than nonfiction? If that's true, I'd be curious to hear more about your reasons for that. Don't mean that in a challenging way, this is just a topic I've been thinking about.

<comment><USER8> It's manipulative in the following ways:

1\. The good guys win, the bad guys lose

2\. Going along with (1) there's the morality message

3\. The events in the book serve the purpose of the plot

4\. Lots of books are formulaic, i.e. the author found a formula that sells.

5\. There's got to be a beginning, middle, and end.

6\. Many novels are written as a series of cliffhangers. Crichton would do that, as would the Outlander series, etc.

Real life is much more messy and unpredictable, hence more interesting.

<comment><USER10> The problem I have with biographies and documentaries is that they overlay order and narrative on this 'messy and unpredictable' reality. If anything, that's more manipulative than a good novel that is rooted in deep research/domain knowledge and that doesn't claim to be 'objective'.

<comment><USER8> Historical accounts are always biased, usually according to the political correctness of the time they were written. For example, the Founding Fathers went from hero to goat to hero to goat and are now morphing back to heroic status with the play "Hamilton".

Civil War heroes are goats today, even having their memorials removed.

Edison has gone from hero to goat. He'll be a hero again in a few decades.

It's the job of a professional historian to try and discern the truth from all this. But whatever truth there is in fiction is impossible to determine if you know nothing about history.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> If you enjoy history, check out Dan Carlin's Hardcore History series on WWI, "Blueprint for Armageddon." About 20+ hours of content.

[https://www.dancarlin.com/product/hardcore- history-50-bluepr...](https://www.dancarlin.com/product/hardcore- history-50-blueprint-for-armageddon-i/)

<comment><USER12> The History of Rome is another history podcast among the all time greats of the medium I think, worth checking out.

<comment><USER13> Link: [http://thehistoryofrome.com/episodes](http://thehistoryofrome.com/episodes) .

</comment></comment></comment><comment><USER14> I'm a huge fan of the series of Ultimate talks from C3 conferences.

The Ultimate Game Boy Talk [https://media.ccc.de/v/33c3-8029-the_ultimate_game_boy_talk](https://media.ccc.de/v/33c3-8029-the_ultimate_game_boy_talk)

The Ultimate Apollo Guidance Computer Talk [https://media.ccc.de/v/34c3-9064-the_ultimate_apollo_guidanc...](https://media.ccc.de/v/34c3-9064-the_ultimate_apollo_guidance_computer_talk)

The Ultimate Atari 2600 Talk [https://media.ccc.de/v/28c3-4711-en- the_atari_2600_video_com...](https://media.ccc.de/v/28c3-4711-en- the_atari_2600_video_computer_system_the_ultimate_talk)

More here:(Commodore 64, Amiga 500, etc.) [https://media.ccc.de/search/?q=ultimate](https://media.ccc.de/search/?q=ultimate)

</comment><comment><USER15> It's kind of a classic, but I really enjoyed Richard Feynman lectures[0], especially on subjects I thought I knew well.

[0] - [https://www.youtube.com/watch?v=j3mhkYbznBk&list=PLLzGzdSNup...](https://www.youtube.com/watch?v=j3mhkYbznBk&list=PLLzGzdSNup63lMYeOpU9Hax6MBsTjdDas)

<comment><USER16> Also _reading_ Feynman's Lectures on Physics [0]. He presents loads of topics in a way that I think captures a lot of the beauty and cleverness of physics while focusing more on building intuition and understanding than on specific math techniques that often come up in physics courses.

I think they're a good way to learn about physics without a ton of background. I also find they're nice to read when studying for my physics exams, just because of Feynman's unique approach to teaching.

[0] - [http://www.feynmanlectures.caltech.edu/index.html](http://www.feynmanlectures.caltech.edu/index.html)

</comment></comment><comment><USER17> Never had a mindset to understand what sometimes are they talking about in humanities. Recently found this series of short and clear leactures on many of such topics. Philosophy of the Humanities [https://www.youtube.com/playlist?list=PLPeStI124dee1ByfcDzRv...](https://www.youtube.com/playlist?list=PLPeStI124dee1ByfcDzRvPxKDNb0GQjmo)

Then this huge series of lectures on western philosophy introduces to a lot of ideas about everything. Can listen anytime. Arthur Holmes: A History of Philosophy [https://www.youtube.com/playlist?list=PL9GwT4_YRZdBf9nIUHs0z...](https://www.youtube.com/playlist?list=PL9GwT4_YRZdBf9nIUHs0zjrnUVl- KBNSM)

Then this category theory keeps popping up everywhere, completly incomprehensible, seems like mathematicians are gone mad. I found that this lectures help like no others to catch up to them. Category theory for programmers by Bartosz Milewski [https://www.youtube.com/playlist?list=PLbgaMIhjbmEnaH_LTkxLI...](https://www.youtube.com/playlist?list=PLbgaMIhjbmEnaH_LTkxLI7FMa2HsnawM_)

</comment><comment><USER18> I've started watching Raymond Hettinger's talks lately. He's a core dev on Python. I love them, the guy is charismatic, very smart and his Python knowledge constantly impresses me.

[https://www.youtube.com/playlist?list=PLRVdut2KPAguz3xcd22i_...](https://www.youtube.com/playlist?list=PLRVdut2KPAguz3xcd22i_o_onnmDKj3MA)

My favourite: [https://www.youtube.com/watch?v=9zinZmE3Ogk](https://www.youtube.com/watch?v=9zinZmE3Ogk)

<comment><USER19> This is cool. I really love these type of posts where the people under discussion is not in the main stream but should have been popular in programming groups. John Carmack talks also is of that type. Very charismatic and delivers his speech standing at the same place with minimal movement for over an hour and talking from the fundamentals to the extreme details of the topics.

</comment></comment><comment><USER20> Just as a side note for all these wonderful youtube resources. You can use this recently discussed tool on hacker news [0] youtube-dl [1] for downloading youtube videos. There's also this video transcoder Handbrake [2] that uses ffmpeg [3] under the hood if the previous tool spits out a video in a format that is not specifically mkv or mp4 (whichever of those two you prefer).

[0]: [https://news.ycombinator.com/item?id=17460060](https://news.ycombinator.com/item?id=17460060)

[1]: [https://github.com/rg3/youtube-dl/](https://github.com/rg3/youtube-dl/)

[2]: [https://handbrake.fr/](https://handbrake.fr/)

[3]: [https://ffmpeg.org/](https://ffmpeg.org/)

</comment><comment><USER21> I like to watch the map that shows the where the plane is. As the agony of travel bares on after hour and hour, you watch as you get closer closer to escaping your hellish existence in a tin can at 30k feet.

<comment><USER22> > _As the agony of travel bares on after hour and hour_

What agony?

> _you watch as you get closer closer to escaping your hellish existence in a > tin can at 30k feet_

Might as well bring some paint, to watch it dry...

<comment><USER23> The agony is from body aches from sitting in an uncomfortable chair and not getting physical movement for a few hours.

<comment><USER22> > _The agony is from body aches from sitting in an uncomfortable chair and not > getting physical movement for a few hours._

That sounds no different from my regular day at the office -- except that even cross-atlantic flights are shorter.

</comment></comment></comment></comment><comment><USER24> Kung Fu movies. Seriously. Jackie Chan, Bruce Lee, whatever.

You need something engaging but not too mentally taxing while under the influence of jet lag.

What could be better than the righteous grist of King Fu movies involving good guys and bad guys duking it out?

<comment> Al-Khwarizmi A kindred spirit. I always spend long flights watching martial arts movies and drinking tomato juice. I'm too uncomfortable and tired on flights to do anything productive.

<comment><USER25> You might be more comfortable if you drank a less salty drink.

<comment><USER24> I don't know what it is about tomato juice and airlines. The only place I _ever_ see anyone drinking straight-up tomato juice is on flights!

<comment> Al-Khwarizmi It is quite well documented (I think at least a couple of studies have been made) that tomato juice is one of the few things that actually taste better on airplanes. The low pressure and high noise level affects how we perceive flavors.

[https://www.dw.com/en/lufthansa-investigates-the-science- of-...](https://www.dw.com/en/lufthansa-investigates-the-science-of-airline- food/a-6114748)

[https://www.travelandleisure.com/travel-tips/airlines- airpor...](https://www.travelandleisure.com/travel-tips/airlines-airports/why- tomato-juice-tastes-better-on-airplane)

I would never buy tomato juice at home, but I drink a lot of it on planes. If you have never tried it on a plane maybe you should!

</comment></comment></comment></comment></comment><comment><USER26> I know this is not answering the question from the poster but I found meditation very helpful during my recent 15 hours flight. It is very rare to find such a long time alone in an almost quite and non-stimulating environment. Getting meals served to you on your seat and knowing that you are stuck here for next many hours takes away the planning ahead part of the head. I found it easier to be with my breath and thoughts. I also drifted in and out of sleep during this time and felt very relaxed. It helped me upon landing to minimize effects of Jet lag. I believe it was more helpful than any talk or video and could also help getting some good insights.

<comment><USER27> I'd like to add that my morning 1-2 hour commute is much more enjoyable if I do some exercise before it. If not, I have trouble staying put/sleeping/concentrating on work/meditating, etc. Also, someone recommended to me recently to get a massage before a flight. I'll try that next time.

</comment></comment><comment><USER28> Anybody even remotely involved in designing anyth9ing connected to a _network_ should watch Raph Koster's GDC talk about the ethical issues involved in developing AR and VR games. Except it's not really about "games" as we usually use the term. "Twitter" is a "virtual world"/MMO where they only bothered to built the chat feature.

This is really about the ethics and responsibilities that come with managing social spaces or infrastructure that people use to interact socially (which necessarily includes the _problems_ that humans always have). The MMOs (and the MUDs/etc before 0them) have been working on these problems for _decades_. That doesn't mean they have the solution, but that experience does include a lot of lessons about what _not_ to do; far too many projects are choosing to learn those lessons the hard way.

[http://www.gdcvault.com/play/1024060/Still-Logged-In-What- AR](http://www.gdcvault.com/play/1024060/Still-Logged-In-What-AR)

<comment><USER29> Awesome. I've been intending to look for writings and talks by Raph Koster lately.

For unaware readers: he was the lead designer for Ultima Online.

<comment><USER30> All of my talks are linked from here: [https://www.raphkoster.com/games/presentations/](https://www.raphkoster.com/games/presentations/)

Most of my writings are on that site as well.

I just recently put out a new book including postmortem material on Ultima Online and other games. Should be front page on the blog.

</comment></comment></comment><comment><USER31> For 5-6 hour flights, it's usually a combination of things for me:

\- Videos related to whatever technology will be pertinent at the meeting I'm flying in for.

\- Distraction movies: Bourne Identity, etc.

\- Anime I'm currently binge-watching.

\- Games on my phone.

\- Printed technical articles I've been procrastinating about reading.

Honestly, how virtuous/edifying my in-flight activities are depends on how much sleep I got the night before. This is one reason I dislike early-morning flights - they leav me zonked and unable to make good use of my in-flight solitude.

Side note: On Android, I just discovered "Movies Anywhere". It was a great tool for downloading DRM's movies/shows for my last trip.

</comment><comment><USER32> This is a talk by Professor Fritz Henglein about how to sort general datasets in linear time. Most people in industry do not believe this is possible, but it's actually part of a family of well-researched techniques. [https://youtu.be/sz9ZlZIRDAg](https://youtu.be/sz9ZlZIRDAg)

<comment><USER33> Actually linear, or O( _n_ log _n_ )?

<comment><USER32> Actually linear. O(n) + k.

Check my submission history. I keep trying to show people this work. Sadly, it never seems to take to a wide audience.

As a consequence, selective joins across 2 fields are also linear time.

<comment><USER33> Interesting. Can you point me at the timestamp or a paper which shows the algorithm?

<comment><USER32> It's complicated enough that you should just watch the whole talk. The paper is 90 pages.

<comment><USER33> It's been six years since the talk... Why hasn't someone drilled down through all the irrelevant typing goo, and shown how to sort integers in linear time? Where does the standard proof that optimal algorithms are Ω( _n_ log _n_ ) break?

<comment><USER32> > Why hasn't someone drilled down through all the irrelevant typing goo, and > shown how to sort integers in linear time?

I mean, if that's all you want Radix sort and American Flag sort are wikipedia fodder.

As for the "typing goo" is, uh, fairly important for how it's derived and what it does. A lot of advanced CS work uses it now, because it's quite powerful. A huge amount of pure-CS research is done in that world.

I guess we don't see this work in more contexts because it's just very very difficult to get it right. Edward Kmett, who is globally recognized for his talent in the field, did a haskell implementation which we all use and love, but it's pretty excruciating to implement it outside of a functional and generic language (and it is genuinely hard to do it in a language without general side effects, so you get difficulty from both sides).

I think a lot of folks in industry just don't think it matters. Certainly when I talk to my peers they suggest it doesn't matter to their work at all.

As for "6 years", they author has continued to push the work. It's also been used in a variety of applications. You can cite search the original paper. [https://pdfs.semanticscholar.org/f425/7af9221ca7fe21dc84a049...](https://pdfs.semanticscholar.org/f425/7af9221ca7fe21dc84a049a8545a28a874ae.pdf)

[https://scholar.google.dk/scholar?oi=bibs&hl=en&cites=180821...](https://scholar.google.dk/scholar?oi=bibs&hl=en&cites=18082135325500003099&as_sdt=5)

One particularly interesting work was computing specific types relational algebras producing products in linear time, which is pretty cool: [https://dl.acm.org/citation.cfm?id=1706372](https://dl.acm.org/citation.cfm?id=1706372) (btw, sorry, ACM is down tonight they're showing us their cloudflare).

This is not new stuff in the world of graduate CS, anymore than succinct data structures or wavelet trees. It's seldom applied in industry, but many of these techniques and sorts are taught as upper division courses in recent, algorithm-and-theory intensive CS programs. That stuff just doesn't make it out of academia very fast these days. Academics face a lot of pushback from industry now with disruptive findings. Look at how Emin Gun Sirer has been blackballed by the cryptocurrency community, for example. Look at how steadfastly folks have stood by JWT despite Lentczner's team's work on a better alternative in Macaroons.

<comment><USER33> Where is the Haskell implementation? Are there public benchmarks of its asymptotic performance?

I'm pretty sure a worst-case linear sorting algorithm would have multiple NYT articles about it.

EDIT: The edits to the parent since I made this comment make it look a bit silly. It made more sense in the context I was initially responding to.

EDIT 2: Thanks for linking the paper. From there:

> As Proposition 1.1 and the corresponding well-known combinatorial lower > bound of Ω(n log n) (Knuth 1998, Section 5.3.1) for comparison-based sorting > show, we cannot accomplish efficient generic partitioning and linear-time > sorting by using black-box binary comparison functions as specifications of > equivalence or ordering relations. Instead, _we show how to construct > efficient discriminators by structural recursion on specifications defined > compositionally in an expressive domain-specific language for denoting > equivalence and ordering relations._

Which sounds like the advantage I speculated on in the grandchild below. I understand, now, and it's a nice result. Thanks for pointing it out.

<comment><USER32> [https://hackage.haskell.org/package/discrimination](https://hackage.haskell.org/package/discrimination)

Which, by the way, contains boatloads of novel work. It uses internal parallel execution to work around some of the problems with discrimination passes between two groups. Cutting edge stuff.

It's haskell. People like you have chips on their shoulder about "typing goo." I'm sure one day someone will write a golang implementation then go to some inventor-centric conference and be hailed as a genius tho, as per usual.

But, did you not know about Radix sort or American Flag sort? Counting sort is a common stupid silicon valley interview question, and has been in the ruby code katas since like 2003. So as keen as I am to see this work widely distributed, it's not totally unprecedented the way you make it out to be.

Hey, want to know another surprising one? With a proper pre-encoding you can compress an (extremely long) binary value and answer questions about how many bits in a given range have occured (a popcount) in constant time. You can also, by extension, figure out the offset of the Nth hot bit appears, in O(1) time. Pretty surprising, huh? Especially when you realize that the entire space of 32 bit integers is only 512mb as a fully realized bitfield, 32x smaller than actually storing one instance of every integer.

See here: [http://alexbowe.com/rrr/](http://alexbowe.com/rrr/)

<comment><USER33> > But, did you not know about Radix sort or American Flag sort?

Oh, is it that the types somehow mandate roughly evenly-sized ordered partitions at any sample size? If so, I get it, now. Thanks.

<comment><USER32> No I'm afraid they don't. I confess I regret this conversation. I've gotten very little out of it other than a confrontstonal attitude from you and a bunch of confusingly worded challenges.

<comment><USER33> If you have any suggestions on how I could have been clearer, I'd welcome them. I'm always glad to learn of flaws in my writing.

Can you get linear time even on Haskell's arbitrary-precision `Integer`s?

<comment><USER32> "typing goo". Generally complaining about the methods used by someone with a novel-peer reviewed result and demanding further and further reductive summarization on your terms, defining said terms with phrases like "typing goo" is bad form.

Further suggesting a lack of credibility by suggesting press coverage should be in place is confrontational.

I lost my temper after you did these things a few times. I apologize.

And yes, it can get O(n) performance with Arbitrary length integers. These analysis exclude underlying comparison times.

<comment><USER33> Thanks for the feedback.

> suggesting a lack of credibility by suggesting press coverage should be in > place is confrontational.

You edited out the context I was responding to, there. As I recall, you were originally explaining the "six years" in terms of it just being hard to get academic results into the mainstream. That seemed extremely unlikely for such a well-known result.

I'm not bothered by your invective or personal attacks, but editing your comments so that my responses are out of context is pretty disturbing.

<comment><USER32> I edited a cloned paragraph with a missing sentence that was accidentally added. I prepared that reply on mobile, and that can happen because you don't see scroll bars.

The literal and figurative content weren't changed. I just removed duplication and improve the flow of the post.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER34> "Breakthrough in Nuclear Fusion?" \- Professor Dennis Whyte. [https://youtu.be/KkpqA8yG9T4](https://youtu.be/KkpqA8yG9T4)

The title threw me off, but it's a very interesting and grounded talk about the different approaches to scaling fusion energy.

<comment><USER35> Looks like they're moving ahead with that approach: [http://news.mit.edu/2018/mit-newly-formed-company-launch- nov...](http://news.mit.edu/2018/mit-newly-formed-company-launch-novel- approach-fusion-power-0309)

</comment></comment><comment><USER36> I really enjoy this Noam Chomsky interview on automation and the structure of society, and find it highly relevant today.

[https://m.youtube.com/watch?v=h_x0Y3FqkEI](https://m.youtube.com/watch?v=h_x0Y3FqkEI)

I also enjoyed the documentary Black Power Mixtape 1967-1975, which can be found on torrent sites.

[https://en.m.wikipedia.org/wiki/The_Black_Power_Mixtape_1967...](https://en.m.wikipedia.org/wiki/The_Black_Power_Mixtape_1967–1975)

<comment><USER17> There is this youtube channel that regularly uploads short excerpts of Chomsky speaking on particular topics. Chomsky's Philosophy [https://www.youtube.com/user/chomskysphilosophy](https://www.youtube.com/user/chomskysphilosophy)

And this channel did a great compilation that introduces to all Chomsky's works. Table of contents is in the video description. Topics are: War, State Power, and American Exceptionalism. Capitalism, Neoliberalism, and Corporate Propaganda. Anarchism, Libertarian Socialism, and Classical Liberalism. Science, Philosophy, and Language. Noam Chomsky - The Essentials [https://www.youtube.com/watch?v=1umiaNjOinE](https://www.youtube.com/watch?v=1umiaNjOinE)

</comment></comment><comment><USER37> Elevator hacking - 2 hour long walkthrough of elevator security and building security. So much fun in that one. Search for it on Youtube.

<comment><USER38> [https://www.youtube.com/watch?v=oHf1vD5_b5I](https://www.youtube.com/watch?v=oHf1vD5_b5I)

<comment><USER37> Yes. That's the shorter censored versoion.

I recommend the full 2 hour version, with all the secrets: [https://youtu.be/ZUvGfuLlZus](https://youtu.be/ZUvGfuLlZus)

My second favourite talk: You spent all that money and your still got owned: [https://youtu.be/tJsNu0VRKYY](https://youtu.be/tJsNu0VRKYY)

I even watch these again and again.

</comment></comment></comment><comment><USER39> For that long time period you might want to watch "A Glorious Accident". Wim Kazyer sits down with Daniel Dennett, Freeman Dyson, Stephen Jay Gould, Oliver Sacks and Stephen Toulmin (and Rupert Sheldrake) to do one-on-one interviews. Then all of them sit down together for 3 1/2 hours at a round table and have a wide-ranging discussion. Wim Kazyer acts as a kind of moderator, Rupert Sheldrake, who has written books like "The Science Delusion" acts as a kind of antagonist.

</comment><comment><USER40> Talking Machines [https://www.thetalkingmachines.com/](https://www.thetalkingmachines.com/) Human Conversations about Machine Learning

Podcasts are better for flying. You can close your eyes, and if you don't stay awake, all the better.

</comment><comment><USER41> literally anything by rich hickey. especially his presentations on Datomic

</comment><comment><USER42> The Flight channel: [https://www.youtube.com/channel/UCXh6VKhioaeEaMQasii7IfQ/vid...](https://www.youtube.com/channel/UCXh6VKhioaeEaMQasii7IfQ/videos)

</comment><comment><USER43> Harvard digital photography course, all (12) lectures are on Youtube

[http://digitalphotography.exposed/#schedule](http://digitalphotography.exposed/#schedule)

</comment><comment><USER44> I always suggest this [https://www.infoq.com/presentations/Simple-Made- Easy](https://www.infoq.com/presentations/Simple-Made-Easy)

</comment><comment><USER45> Bracing for Impact: True Tales of Air Disasters and the People Who Survived Them (paperback version)

</comment><comment><USER46> It won't take up a whole long flight, but one of the best TED talks ever is Jill Bolte Taylor's talk. She's a neuro-anatomist who had a stroke and reports from the inside about what goes on, with significant implications for what kind of choices we make in life:

[https://www.ted.com/talks/jill_bolte_taylor_s_powerful_strok...](https://www.ted.com/talks/jill_bolte_taylor_s_powerful_stroke_of_insight)

</comment><comment><USER47> If you have an iPad, play FTL. It'll make the time zip by.

[http://subsetgames.com/ftl.html](http://subsetgames.com/ftl.html)

</comment><comment><USER48> Snakes on a Plane

</comment><comment><USER49> Wild, Wild Country on Netflix

</comment><comment><USER50> It might be a good opportunity to close your eyes and listen to many of the amazing (but nearly always way too long, IMHO) interviews available on developer podcasts like Software Engineering Daily, Microsoft Research, Hanselminutes, IndieHackers, etc. You get to listen to some great conversations and if you fall asleep, it's a bonus!

</comment><comment><USER51> I found Stephen Wolfram's talks about complexity and computation _extremely_ rewarding.

[https://youtu.be/cbu_bCQ2Lkg](https://youtu.be/cbu_bCQ2Lkg)

[https://youtu.be/_eC14GonZnU](https://youtu.be/_eC14GonZnU)

</comment><comment><USER52> Get the coursera app and download a week worth of courses.

</comment><comment><USER53> Richard Hamming "The Art of doing Science and Engineering" [https://www.youtube.com/playlist?list=PL2FF649D0C4407B30](https://www.youtube.com/playlist?list=PL2FF649D0C4407B30)

</comment><comment><USER54> Netflix now lets you download many (most?) of their films, I always get some documentaries and fictional shows of my destination. For France I watched a documentary on the national soccer team and a comedy special by Jacky Bloom.

</comment><comment><USER55> Hurtling through the air, surrounded by 200+ of the most fascinating and complex entities in the universe—fellow beings who share many of my hopes and fears—I always end up watching Batman vs Superman

</comment><comment><USER56> Jeremy Rifkin 3rd industrial age talk [https://youtu.be/QX3M8Ka9vUA](https://youtu.be/QX3M8Ka9vUA)

<comment><USER57> I think there was some good info in there but I also couldn't help thinking 'this guy is just lumping together all of the tech buzzwords of the last 2 years'.

<comment><USER56> This guy is a chief architect of both the EU and Chinese plans for the coming years. His words weigh more than the average speakers.

</comment></comment></comment><comment><USER58> The Ultimate Gameboy Talk from 33c3 is a good one if you haven't already watched it. Frankly a large amount of CCC videos would also fit the bill.

</comment><comment><USER59> Terence McKenna's lectures from the Earth Trust Foundation conference are really great, if you're into that sort of thing. There are four parts, totaling ~6.5 hours.

[https://archive.org/details/PsychedeliaRawArchivesOfTerenceM...](https://archive.org/details/PsychedeliaRawArchivesOfTerenceMckennaTalks/EarthTrust1.mp3)

</comment><comment><USER60> Split my time between documentaries that can be downloaded from Netflix, usually of middling quality, and whatever audiobook caught my attention in the days before flight.

Being a bigger guy, unless I get the upgrade I can't do anything with a keyboard so doing much productive is out of the question.

Recent highlights have been Sour Grapes and the Rotten series on Netflix, and Ray Dalits Principles on Audible.

</comment><comment><USER61> I usually read long articles I have saved in Pocket.

</comment><comment><USER62> I'm one of those people who are slightly susceptible to motion sickness, so I can't do anything that requires intense focus on an airplane.

My two go-tos are:

1) Noisli - white noise app to help me sleep.

2) BBC's Sherlock - the slickness of the production values keeps me engaged.

I find I'm more productive when I land if I don't try too hard to be productive on the flight.

</comment><comment><USER63> I like Hello Internet [https://www.youtube.com/channel/UCwez9XDNV_wS0WNDZteXjgw](https://www.youtube.com/channel/UCwez9XDNV_wS0WNDZteXjgw)

It's in the genre of "two dudes talking" podcasts, but technology and futurology are some of the topics they touch on often.

</comment><comment><USER64> I found these great brain anatomy lectures (WARNING, real brains!): [https://library.med.utah.edu/neurologicexam/html/brain- disse...](https://library.med.utah.edu/neurologicexam/html/brain- dissections.html)

</comment><comment><USER65> I like to listen to the "Coding Blocks" pad cast on long drives (most every weekend). They go over a book like "Clean Code"[1] a chapter at a time and distill the high points in an entertaining fashion.

[1] by Robert C. Martin (aka "uncle Bob")

<comment><USER66> I love the Coding Blocks podcast. It's one of my favorites. I also recommend Developer Tea.

<comment><USER65> Developer Tea eh? I'll check that out.

</comment></comment></comment><comment><USER67> All three parts of Adam Curtis's "All Watched Over By Machines Of Loving Grace"

</comment><comment><USER64> This talk, by a DARPA project manager on brain computer interfaces: [https://www.youtube.com/watch?v=PVuSHjeh1Os](https://www.youtube.com/watch?v=PVuSHjeh1Os)

</comment><comment><USER68> If you are interested in machine learning, I thought this was a very thought provoking lecture:

[https://youtu.be/N5oZIO8pE40](https://youtu.be/N5oZIO8pE40)

</comment><comment><USER69> Old episodes of the Computer Chronicles: [https://www.youtube.com/user/ComputerChroniclesYT](https://www.youtube.com/user/ComputerChroniclesYT)

</comment><comment><USER70> I am currently studying a language so long flights are just the perfect time to "quiz myself" in that language. Either by flash-card videos or un-subtitled dubs of a movie in that language

<comment><USER71> Which language exactly? I am currently studying kanji with anki during my commute.

<comment><USER70> I am studying mandarin, I could probably be using my time doing some writing in my commute. I use a plethora of apps / download youtube videos / rosetta stone

</comment></comment></comment><comment><USER72> This is a great resource: [https://github.com/JanVanRyswyck/awesome- talks](https://github.com/JanVanRyswyck/awesome-talks)

</comment><comment><USER73> One more flight, and I will be an expert in javascript. At least theorically, since I never wrote a single line of js code ...i can only read books while on flight, but not to code.

</comment><comment><USER74> My favorite video of all time: [https://www.youtube.com/watch?v=FITJMJjASUs](https://www.youtube.com/watch?v=FITJMJjASUs)

</comment><comment><USER75> Yehuda Katz - <USER2> [https://www.youtube.com/watch?v=yYihop9gHj4](https://www.youtube.com/watch?v=yYihop9gHj4)

</comment><comment><USER76> Watch something you normally wouldn't watch if you had a choice but might find interesting. It's a good way to force yourself out of your comfort zone.

</comment><comment><USER77> I would recommend the audio book of Thinking Fast and Slow. I have been listening to it during my commute and it is fascinating.

</comment><comment><USER78> Fight Club

</comment><comment><USER79> Grubby playing Warcraft 3! Let your brain rest

</comment><comment><USER80> Course in miracles:

[https://youtu.be/nrmFlDpeF-k](https://youtu.be/nrmFlDpeF-k)

</comment><comment><USER81> Final Destination

</comment><comment><USER82> The entire Bojack Horseman series on Netflix.

</comment><comment><USER83> To answer your question, Jordan Peterson's video lectures series, "Maps of Meaning" (You would need internet access to YouTube).

For long flights, I buy magazines in the airport shop (Wired and Scientific American). I also have my music collection on Spotify on my phone with wireless Bose headphones to listen to. Also, several books on Kindle on my phone (now reading "Sapiens: A Brief History of Humankind"). Don't forget a spare battery. Or, sometimes I check the plane's movie list for a new release I haven't seen and watch that.

</comment><comment><USER84> write a complete program with pen and paper

<comment><USER85> Reminded me of college, when we had to write several pages of assembly code (that was 6 years ago). Fun exercise, not so much when your grade depends on it.

<comment><USER86> You really wish for a paranthesis counting pen when you have to write pages and pages of LISP (I think it was Common Lisp in our case) on paper for an exam.

</comment></comment></comment><comment><USER87> A serie like The Handmaid's Tale.

</comment><comment><USER88> if you like interviews, then check out these podcasts:

How I built this

Masters of scale

Akimbo

Software engineering daily

</comment><comment><USER89> Airplane.

</comment><comment><USER90> Read

</comment><comment><USER91> I mostly use a very long form podcast like Joe Rogen, Sam Harris, or videos hosted on Internet Archives.

</comment><comment><USER92> read a book</comment>


AMD's aggressive pricing update on the EPYC 7371 - <AUTHOR> https://www.servethehome.com/amd-epyc-7371-pricing-update-an-insane-value/ <comment><USER2> _Now that AMD has closed the gap, and passed Intel's per core performance in the 16-core CPU market, it has a platform with more RAM capacity and more PCIe lanes along with more performance than the Intel Xeon Gold 6142M, at around a quarter of the price._

What does Intel have so far up its sleeve that AMD has to virtually give away its chips like this?

<comment><USER3> My background is in the FPGA industry which gives an absolutely fantastic example of this exact dynamic. FPGA companies produce a new generation of chips every 3-5 years or so. There are only two companies, with market share split 60:40. In the last 2 decades we've seen one produce a fantastic product and the other screw up, and then it flips, and then it flips back. Over that time the market share of these companies went 60:40, 62:38, 58:42, 60:40. The actual number of customers who switch to the better product is tiny. Why?

* All the existing knowledge in the company is about one platform, it's incredibly expensive to develop the skills on the other toolchain. Port all your software etc.

* The existing products are all from one vendor so you save loads of effort if all the products are basically similar.

* There are existing relationships with the company you're with.

* You know that if you do switch, all that cost of switching may result in only a few years of using the better product.

* The IP you're buying works better with the vendor you're currently on.

* You don't know what the real world performance will actually be for your application.

So yeah, you could move from Intel to AMD, but the chip is only a tiny part of that cost.

<comment><USER4> I almost know zero about FPGA market but it seems that it's infinitely harder to port programs between FPGAs than from CPUs.

Other than that thanks for the insights, interesting facts.

<comment><USER5> That is probably generally true but if you have some serious compute workloads that you specialized for your server's CPU it could be hard to port between AMD and Intel as well. When people have customized the code for particular cache sizes, SIMD instruction sets, core counts, throttling schemes, ram throughput, and branch predictors and so on.

<comment><USER4> It's quite true, but what I've heard from FPGA users is that toolchain and conventions are aliens to each others, I think these are both two different kinds of hell. I don't know what would cost more to a company.

<comment><USER6> When you deliver an FPGA you're normally also making your own hardware. That's a big part of the FPGA porting cost, you have to redesign your boards. I don't think the same is generally true in the x86 realm. Even if you're specifically using some feature of the intel micro architecture it's still essentially a code change rather than an full redesign.

</comment></comment></comment></comment></comment></comment><comment><USER7> not only amd is providing a good performance, value and features(omg, so many pci lanes), amd can actually deliver their chips without the wait.

the lead time for intel desktop models(i7-9700) is months. i know that enterprise vendors are also experiencing intel cpu shortages and long wait times. Some discussion [https://www.reddit.com/r/sysadmin/comments/9ea8y2/intel_cant...](https://www.reddit.com/r/sysadmin/comments/9ea8y2/intel_cant_supply_14nm_xeons_hpe_directly/)

</comment><comment><USER8> I can't wait for DO to offer EPYC, or hopefully Zen 2 as they are very close to launch. I need more CPU Core but instead most of the Cloud Vendor offer me 1 Core ( Actually 1 Thread ) and 2GB Memory. I would much rather see a 1:1 Core and Memory Config.

<comment><USER9> If you need that many cores why don't you get a dedicated server?

<comment><USER8> I don't need that many core as in 64 Core or 128vCPU. I just hope we get better pricing on Core Count. Now I do want a Dedicated Server but having Cloud / VM is much easier for Scaling. It would have been great if there are CloudVM provider that has Dedicated Server as Baseline, but so far only Vultr has it.

</comment></comment></comment><comment><USER10> Great point about core frequency being important for per-core licensing.

I wonder if any software with per-core licensing has tried to take a possibly 'fairer' approach, for example by summing the frequency of all cores? E.g. 4x cores at 2GHz is 8GHz?

It's not that straightforward a comparison, I know, just wondering if anyone has tried something different here.

<comment><USER11> SQL Server switched from CPU-based licensing to core-based licensing as of their 2012 version and they included a "core factor" that reduced licensing costs if you were running on AMD cores to 75% of the cost for Intel cores to account for AMD's then-lower per-core performance.

[http://download.microsoft.com/download/7/3/c/73cad4e0-d0b5-4...](http://download.microsoft.com/download/7/3/c/73cad4e0-d0b5-4be5-ab49-d5b886a5ae00/sql_server_2012_licensing_reference_guide.pdf)

<comment><USER12> Are you sure that lower core factor applies to EPYC ? The document [http://download.microsoft.com/download/4/4/5/445627B4-9AB0-4...](http://download.microsoft.com/download/4/4/5/445627B4-9AB0-4AED- BCCD-C7AC5ADAF6B2/CoreFactorTable_4_1_2014.pdf)

mentions only old Opterons.

<comment><USER11> That applied only to certain AMD CPUs that were available in 2012. It's just an example in which there was some effort to take a "fairer" approach to per- core licensing. It's especially notable since the EPYC line is evidence of the great strides that AMD has made since then when MS just gave you a blanket discount for running SQL Server on AMD cores.

</comment></comment></comment></comment><comment><USER13> Will be interesting to see the Intel response.

AMD is doing really well.

<comment><USER14> Nobody is selling them in quantities/systems that could in any way threaten Intel. AMD might have a great tech right now (and possibly even better with Zen 2), but it won't help them financially if nobody can buy them or only in overall inferior offerings to Intel ones. EPYC has still a lot to overcome in DC/server space. I am happy with my TR in a Deep Learning machine, all the PCIe lanes for multiple GPUs are giving me insane value, however server contracts are way more complicated than enthusiast space and Intel has a firm ground there.

<comment><USER15> Amazon, Microsoft, and Baidu would have to disagree with you, since they all have already done large Epyc implementations.

Also, I can go on CDW right now and get an Epyc server, no problem.

Availability isn't an issue for AMD.

<comment><USER14> ...and that's why AMD didn't miss revenue forecasts, right? EPYC was pretty anemic, I would have expected an explosion in sales with such a product, not the underwhelming sales performance it experienced. There are obviously other factors holding it back.

How do you know AWS/Azure etc. aren't using them just for price haggling with Intel, as it was done with AMD in the past all the time? The fact you can get EPYC servers doesn't mean they are wide-spread anyway.

<comment><USER15> >I would have expected an explosion in sales with such a product

That is just unrealistic expectations in the server space. These aren't consumer products where adoption is fast. Business don't upgrade their infrastructure as quickly as consumers, and when they do decided to upgrade it is months of planning. No company is going to jump ship to AMD when their current servers aren't fully depreciated by their accounting standards, and they still have several years left on their support contracts.

The EPYC sales will come, but not overnight.

EDIT: And I am not sure why are are so disappointed here. Epyc sales and adoption has been in-line with what AMD has given as guidance. Why would you expect adoption to wildly exceed AMD's own guidance? I think AMD had aggressive but realistic guidance and so far Epyc has been great success and will only continue to chip away at Intel. Also not sure what revenue miss you are referring to. Overall AMD beat their expected earnings per share by 1 cent last quarter. If there was a slight miss on the Epyc sales then they made up for it somewhere else, but it must not have been a very big miss otherwise they would have missed the EPS target.

</comment></comment></comment></comment></comment><comment><USER16> So what happened to intel that they're now no longer way ahead in both performance and manufacturing technology? They're being squeezed from all sides, and don't seem to be pulling ahead...

<comment><USER17> It's not so much Intel were ahead in terms of architecture design, but rather AMD was way behind. Bulldozer was a disaster, and Zen is a ground-up redesign that means AMD have a decent core at the level of Intel's again.

But AMD have two advantages Intel don't. Their Zen architecture is designed for multi-chip module scalability, so they can deliver higher core counts at much better yields (especially important on new process nodes!) and thus manufacturing costs than Intel's monolithic designs. And AMD uses 3rd-party fabs that, unlike Intel, are already doing great on the new process node.

2019 will be a reckoning for Intel.

<comment><USER18> > Bulldozer was a disaster

I dunno if it was as much a "disaster" as it was Intel's Sandy Bridge doing so well.

Intel's Sandy Bridge (2600k / 2700k) were HUGE improvements back in 2011. Bulldozer was a step-wider (the cores were roughly the same as K10 but you'd get 8-cores instead of 4), and Piledriver / Steamroller incrementally improved on the formula.

Bulldozer managed to increase core counts from ~4 (AMD Phenom) to 8 with Bulldozer. Sure, the 8 "cores" of Bulldozer shared a decoder and perhaps was more appropriately a 4-core with hyperthreading... but it was still a core-to- core improvement compared to K10.

[https://www.tomshardware.com/reviews/piledriver-k10-cpu- over...](https://www.tomshardware.com/reviews/piledriver-k10-cpu- overclocking,3584.html)

But the incremental upgrades to AMD's K10 were just no match for the 20%+ boosts that Intel was doing with their Sandy Bridge architecture. Ultimately, Intel's Hyperthreads (4c/8t) were roughly the same as AMD's "8 core (4-decoders)" setup... because Sandy Bridge was just so far ahead of the game.

<comment><USER17> Bulldozer was less performant than its predecessor once you factor in the process node. That's why it was a disaster: AMD managed to design a _worse_ processor and were stuck with it for years. They did make it less bad over time, but it still sucked.

<comment><USER19> This. Bulldozer was (ironically) AMD's Netburst moment. They made a speculative play ("modules" in the case of Bulldozer, long pipelines in the case of Netburst) to chase high core counts/clock speeds (respectively) and the technology didn't end up panning out like they expected on top of performing worse out-of-the-gate than the architectures they were meant to succeed.

The difference is that Intel had the cash and political clout to wait out Netburst and force the market to take it while Bulldozer nearly killed AMD (which only held on thanks to its GPU division, itself now in crisis due to lack of competitive products due to under-investment).

</comment></comment></comment></comment></comment><comment><USER20> AMD needs to go as low as possible, get a foothold in the server market then start increasing prices.

<comment><USER21> They already have a foothold in the server market: AWS just released new families of instance types based on AMD CPUs.

</comment></comment><comment><USER22> Great value here. All modules get full memory bandwidth versus the desktop chips. I'd get one for sure.

<comment><USER23> Small nitpick, but if I'm reading this correctly, the EPYC 7371 is a 16-core part. The Threadripper 2950x with 16 cores and the 2970wx with 24 cores both have full speed memory access for all cores. It is only the 32 core 2990wx that has half the cores running without direct access to memory. Do correct me if I'm wrong.

<comment><USER22> You could be right. I was under the impression that all mainstream parts even the threadripper ones were artificially handicapped.

<comment><USER24> EPYC supports up to 8 memory channels where as Threadripper only 4.

<comment><USER25> Sure, but that's 32-core EPYC (4 dies, each die has 2 channels). Does this 16-core part use 2 dies or 4 dies (with half of each die's cores crippled)?

<comment> Tuna-Fish All EPYC cpus support 8 memory channels, and the full set of PCI-E lanes. AMD has chosen to segment the market so that every CPU plugged into every existing SP3 MB can make full use of all it's features.

This CPU has 4 dies, each of which have 2 CCX, each of which have 2 cores and 8MB of cache.

</comment></comment></comment></comment></comment></comment><comment><USER26> If AMD came out with a platform with decent power/battery/performance and ECC memory available in a laptop it would go to the front of my next purchase list.

<comment><USER27> I have the thinkpad a485 with ryzen pro. It supports ECC but I haven't tried that yet.

I would hold out though. AMD is really bad at managing drivers for the video card. It's supposed to be fixed soon new, but there is also a new chip on the horizon.

Battery life is about 8 to 10 hours depending on what I'm doing.

</comment></comment><comment><USER28> For realtime audio processing Intel still bears the crown. Hopefully AMD will get closer so that Intel get on improving.

</comment><comment><USER29> > AMD EPYC 7371 Pricing Update [Is] An Insane Value

Off topic, but I can't take it anymore. Enough with "it's a good value". "A" good value? "Excellence deserves admiration" is _a_ good value. $1550 for an EPYC 7371 is just ... good value.

> AMD EPYC 7371 Pricing Update [Is] Insanely Good Value

<comment><USER23> And what would be an Insane value? Something like "I want to see my enemies driven before me, hear the lamentations of their women," something like that? :)

<comment><USER30> You'd have to ask Crazy Eddie [https://www.youtube.com/watch?v=4yYGoO5imyY](https://www.youtube.com/watch?v=4yYGoO5imyY)

<comment><USER31> Iirc, his prices were irs-trouble insane</comment></comment></comment></comment>


Kleiner Perkins Suffers Security Breach - <AUTHOR> http://www.nytimes.com/reuters/2014/08/21/business/21reuters-usa-crime-kleiner.html

<comment><USER2> "The suspect took a docking station, three Lenovo laptops, an Apple desktop computer and two monitors, and an Apple MacBook Air."

Sounds more like an opportunistic grab for hardware rather than software/data.

Assuming there was any stock crypto at all on the systems, I don't think they have much to worry about their secrets falling into the wrong hands.

<comment><USER3> Depends on the password you still have to worry about that being cracked by brute force.

[http://derflounder.wordpress.com/2011/11/23/using-the- comman...](http://derflounder.wordpress.com/2011/11/23/using-the-command-line- to-unlock-or-decrypt-your-filevault-2-encrypted-boot-drive/)

[http://derflounder.wordpress.com/2013/06/11/decrypting- filev...](http://derflounder.wordpress.com/2013/06/11/decrypting- filevault-2-on-mac-os-x-10-8-4-unlock-first-then-decrypt/)

<comment><USER2> Well let's hope the VC firm left sacrificial wallet.dats on the drives. When we see the *coins move, we'll know the drives were successfully decrypted. (Unlikely, but would be neat)

</comment></comment></comment><comment><USER4> The article doesn't mention whether the stolen machines' hard drives were encrypted or not, though I guess we can assume they weren't based on the level of concern being raised =/

<comment><USER5> You can't infer that. This story would be reported the same whether the drives were encrypted or not. Kleiner is just covering its ass by reporting the theft to the relevant authorities and affected customers.

So far, it's not Kleiner raising concern - they're just doing due diligence. It's the media raising concern. It makes a better story.

Kleiner haven't spoken about this to the media yet.</comment></comment>


The Growing Peril of Index Funds: Too Much Tech - <AUTHOR> https://www.wsj.com/articles/the-growing-peril-of-index-funds-too-much-tech-1514457003 <comment><USER2> The second edition of Jack Bogle's "Little Book of Common Sense Investing" came out this last October. It goes into detail in evaluating active investment such as what's being recommended in this article v.s. passive investment.

Plainly put, I've yet to see any compelling evidence that the supposed experts being quoted in this article can time the market as they're suggesting with their suggestion of getting out of tech stocks now.

The percentage of active management firms that can beat the index funds even before you factor in their high fees is so miniscule that random chance could account for those low numbers.

The market's almost definitely heading for a bit of a crash in a few years, but it's instructive to look at how passive v.s. active investing did in 2008 (detailed in this book) to see how much these supposed experts really know.

<comment><USER3> This article is talking about diversification to reduce risk, which is unrelated to "timing the market" and which can be consistent with passive investing.

<comment><USER2> Some investors described in the article moved from market cap weighted indexes to ones that were disproportionately weighted to disfavor tech stocks, they believe that they're overvalued and that they can get out earlier than everyone else.

That's trying to time the market.

The firms mentioned in the article will be happy to charge you much higher fees than Vanguard for questionable longer term returns. It's basically a PR piece for managed investment.

<comment><USER4> Even if the average returns and risk are identical for each security individually, you can minimise your portfolio risk by investing equally across many different industries. If a downturn were to occur, different securities in a single industry will be much more correlated, as compared to different securities across different industries.

<comment><USER2> Yes, that makes sense. But that's entirely unrelated to what I'm pointing out here.

(Some of) the article is making the argument that investors should decide for themselves how over- or undervalued certain segments of the economy are.

So on one hand you'd have a diverse market cap weighted index where you buy into stocks representing the proportional to their portion of the economy.

On the other hand you might think you have special knowledge to layer on top of that. Are banks undervalued? By how much? Let's say 10%. Then let's sell something else to buy 10% more banks, now what's 10% overvalued? Tech?

I've yet to see any sort of compelling data that this sort of managed investing is a good idea, and that's what it is.

Just because you're not buying TSLA and instead just disproportionately buying "car stuff", or not selling AAPL but just selling "tech stuff" you're still trying to pick stocks and trying to beat other stockpickers doing the same thing. You're just picking subsets of the economy instead of individual stocks.

<comment><USER5> I don't think it's a good idea to guess what's over/undervalued. But equal weighting different industries is not quite the same thing. No matter how you weight things (by capitalization, by company, by industry, whatever) you are making assumptions about correlations. No dogma can tell you which is right a priori, because correlations change, especially in crises when it matters most.

Also...the total market, cap weighted, may be the optimal way to invest the whole world's capital - but that doesn't mean that it's the optimal way to invest, say, $50K. If you have $100B, say, you _can 't_ put it in a stock that's currently valued at $1B. But if you have $1000, you don't have that constraint.

Regardless of _how_ the optimal portfolio may differ, when you're investing a relatively minute amount compared to the entire market, it's hard to imagine the optimum is not going to be different.

...You don't pilot your car under all the same constraints as an 18-wheeler, just because that's optimum for shipping large quantities.

<comment><USER6> >If you have $100B, say, you can't put it in a stock that's currently valued at $1B. But if you have $1000, you don't have that constraint.

Your argument is that there is a mispriced security somewhere that can be bought at a low price. There is currently only 1B$ of it available and so the professional manager with 100B$ to spend just doesn't bother to pick up that money. But you with just 1000$ can do it instead. What makes the professional manager pass up on that opportunity? He has at least as much money as you, why doesn't he invest at least that amount?

In reality that opportunity doesn't exist. Companies can't be consistently mispriced lower because there is too much demand for their stock. That's not how markets work for anything.

<comment><USER5> Every cap weighted fund has a cutoff where it omits companies that are too small. But even the stocks that are barely large enough to include don't contribute much to returns.

I don't think the small cap stock is "mispriced". Rather, it has a different value for different investors, and the market price is a compromise. That means different investors should probably have a different amount of it in a portfolio.

It's an abuse of theory to claim that since the market is efficient, you should ignore the things that make you different from the total market. For example, suppose you invest in tax-exempt investments when you are in a low tax bracket, or even when you are investing in a tax free account. Is that optimal because markets are efficient? Of course not. Because the value set by the market does not take into account the way in which you differ.

The reason to believe in index investing is because you understand your own lack of knowledge and are honest about it. That's a good thing, but it doesn't justify pretending you don't know things that you _do_ know. People seem to have the same issue with probability, I find.

<comment><USER2> I think you're arguing a point that isn't even being brought up in this thread or in the article.

Of course it's just fine to have an index fund that's weighted towards certain types of stock, e.g. there's the S&P 500, then various "woldwide" funds, EU- weighted funds etc. The risks & benefits of those are well understood. Nobody argues that different types of index funds shouldn't exist.

Similarly, there's funds that cater to specific regulations, e.g. investing heavily in "green" stocks which may be subsidized by the government, or avoiding certain taxes (e.g. lower turnover for lower capital gains).

Both of those are categorically different from supposing that you know better than other people that tech stock in 2017 is overvalued, and trying to move away from that in favor of something else. Now you're making an active investment move which history shows you're more likely to lose on as gain anything on.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> This is an inherent property of all market cap weighted indexes. An alternative approach is equal weighted indexes, which have historically outperformed market cap weighted indexes. There's no free lunch though: this is a case of your typical risk/reward tradeoff. Equal weighted funds are more risky (volatile), but have higher historic returns than market cap weighted funds.

Some of them feature equal weighting across industry sectors as well as amongst separate companies. Here's an example of one (also note the high fees): [https://www.guggenheiminvestments.com/etf/fund/rsp- guggenhei...](https://www.guggenheiminvestments.com/etf/fund/rsp-guggenheim- sp-500-equal-weight-etf)

I personally just stick with VTI and VXUS for stocks (excluding my Airbnb & other startup shares), but I also have about 50% of my net worth in crypto (and I remain bullish).

<comment><USER6> >Equal weighted funds are more risky (volatile), but have higher historic returns than market cap weighted funds.

But do they outperform after risk adjustment? I doubt it and if not you're better off just leveraging a bit to your desired level of risk. The point of passive investing isn't that the trading strategy can be automated, the point is to say "I want to grab exactly the average return of the market every year". Trying to do anything else is by definition not achievable by everyone so why do people think they should be the lucky ones? What specific advantage does a retail investor bring to merit that?

<comment><USER5> I don't think it makes sense to equate risk with volatility. Say I have $1M. If I buy a cap weighted S&P 500 fund, I have about $40,000 in Apple, because it's about 4%. If I then switch to an equal weighted S&P 500, that $40K will be split up between 20 stocks with about $2K each. That is clearly more diversified with respect to company specific risk. It may or may not be worth it given increased fees, but you can't just dogmatically say it's equivalent to cap-weighted plus leverage. It's qualitatively different.

<comment><USER6> >you can't just dogmatically say it's equivalent to cap-weighted plus leverage

I'm saying it's _strictly better_ to do cap-weighted plus leverage. That's what financial theory tells us anyway, that the best portfolio is whatever mix between risk free cash and the same mix as all the assets in the world. Deviating from that brings you farther away from the efficiency frontier. Now for this to hold EMH must hold and we know it's not true in the stronger forms which was why I was asking for a risk adjusted benchmark.

<comment><USER5> I don't think there's a theory that tells you risk is volatility. Isn't that more of an axiom? And I have the distinct impression it was chosen because it leads to tractable math. I'm not endorsing a specific alternative definition of risk, but I was suggesting risk is clearly multidimensional and intimately related to predicting what will be correlated in the future.

</comment></comment></comment></comment></comment><comment><USER8> Well you could always avoid the tech heavy indices, but then you're sort of actively managing your positions rather than letting it be purely passive.

Theoretically, is not another growing peril of indexing that it removes incentives for companies to behave well or outperform, since if they're part of an index their shares will be bought automatically by retirement plans and investors anyway, irregardless of performance or competency?

And yet another theoretical peril question would be, if everyone is indexing, it surely must lose it's efficacy because it is no longer efficient, will indexing then not underperform? I suspect if or when that happens, active management will regain interest.

Indexing by the masses is a fairly new trend, it will be interesting to see how the markets handle the behavior long term.

<comment><USER7> This is not quite correct. There are other weighting methods (specifically, equal weighting) which would prevent this situation, whilst still remaining passive.

<comment><USER6> I'd argue an equal weighted index is no longer passive. You're actively making a bet that smaller companies will outperform bigger companies. Someone else will be taking the other side of that bet. Me with my cap weighted index will get the average of the two and be actually passive.

<comment><USER5> But many investors are going to take the side of big companies, not because they are choosing the optimal approach, but because they _have_ to, because they have too much capital to invest in small companies. So it's not logical to assume their returns must be equal on average.

<comment><USER6> I didn't say their returns would be equal, I said my returns will always be equal to the average of the market exactly because I'm a passive investor. The sum of all bets is what I'll be making. The standard text on that:

[https://web.stanford.edu/~wfsharpe/art/active/active.htm](https://web.stanford.edu/~wfsharpe/art/active/active.htm)

As for there not being enough of the small companies available to do the equal weighted S&P500, the smallest companies in the index have 3.5B$ in market cap. So there's almost 2T$ of that mix available and there would be much more if that amount of capital suddenly decided to implement that strategy as the market caps of the bottom of the S&P500 index would certainly rise significantly.

</comment></comment></comment></comment></comment><comment><USER9> "Tech" in this case is an overly broad classification. Apple, Amazon, and Facebook are all very different businesses and should not have correlated performance.

<comment><USER10> Logistics, hardware, advertising. All very different for sure.

Every successful business has to be "tech" in this century. Tech is how you achieve high scale

<comment><USER11> Yeah, if "tech" just means "many employees are programmers" or "has a large online presence" it seems obvious that tech is going to become a larger portion of the market, and rebalancing to cut down the "overweight" tech in your portfolio runs directly counter to what passive cap-weighted investing is about -- having investments that follow what has won in the past and is expected to win in the future.

</comment></comment></comment><comment><USER12> This looks like a PR piece to make people afraid of investing in the S&P. Some people have suggested thought that since investing in index funds makes the companies that are apart of the index fund to not compete with each other due to their shareholders. I'm not sure if the previous statement is even valid or real.

<comment><USER13> There have a lot of hit jobs on index funds in the last few years, often published in the Wall Street Journal. Active fund managers are getting really scared that money continues pouring out of them and into Vanguard as it becomes common knowledge that actively managed funds almost never beat index funds net of fees.

[https://news.ycombinator.com/item?id=12368136#12368902](https://news.ycombinator.com/item?id=12368136#12368902) <\- I said this a year ago and it's going to continue

</comment></comment><comment><USER14> The growing peril is only for _long positions_. Peril for long positions is profit for short positions.

Seeing this article advising "reducing your exposure to tech by selling your tech stocks" assumes that you have no choice to participate other than _buying_. They are looking at half of the market(buy-side) and ignoring the equal-sized selling-side of the market. Stocks going down == Short the market. Stocks going up == long the market. The direction is unimportant, the volatility is absolutely important - and there is an awful lot of it(volatility) at the moment.

<comment><USER15> Are you suggesting an index fund that passively shorts everything? I don't think that's in my 401k.

(I use Hedgeable which is the most complex robo-advisor, but as they say they hedge long instead of using options. Please sign up, I'd be inconvenienced if they went out of business!)

<comment><USER14> I'm suggesting if the article has any conviction for moving out of tech, then the smart play is not to move out but to short. The language of "reduce exposure" is not what you should hear from your advisor/broker. You don't reduce exposure, you reduce _risk_ by opening short hedges or opening short outright. If these people are tied up long in QQQ(the article references this instrument, which is an amateur hour NQ100 ETF), then risk of tech bust is managed by shorting NQ as a hedge.

</comment></comment></comment><comment><USER16> [http://archive.is/qPANt](http://archive.is/qPANt)

</comment><comment><USER17> Balancing your portfolio with domestic (US) and international index funds would seem to take care of that since.

I'm currently 60/20/20 US/International/Bonds IIRC.

If the US market is mostly tech and tech bottoms out when I want to retire, I won't be completely wiped out.

I also plan to semi-retire early and work a lower stress job between 40/50 and 65 so I'll be less under the gun than someone who _needs_ sell stock and can't tighten their belt.

</comment><comment><USER18> Aren't there any broad non-tech index funds?

<comment><USER19> Looks like Pro Shares has S&P 500 ex Tech: [http://www.proshares.com/funds/spxt.html](http://www.proshares.com/funds/spxt.html)

<comment><USER20> ADV of 311 shares according to Yahoo. Do not buy.</comment></comment></comment>