Google's robots.txt parser is now open source - <AUTHOR> https://opensource.googleblog.com/2019/07/googles-robotstxt-parser-is-now-open.html <comment><USER2> Where was this 10 years ago when I was reverse engineering the Google robots.txt parser by feeding example robots.txt files and URLs into the Google webmaster tool? I actually went so far as to build a convoluted honeypot website and robots.txt to see what the Google crawler would do in the wild.

Having written the robots.txt parser at Blekko, I can tell you what standards there are incomplete and inconsistent.

Robots.txt files are usually written by hand using random text editors ("/n" vs "/r/n" vs a mix of both!) by people who have no idea what a programming language grammar is. Let alone follow BNF from the RFC. There are situations where adding a newline completely negates all your rules. Specifically, newlines between useragent lines nor between useragent lines and rules.

My first inclination was to build an RFC compliant parser and point to the standard if anyone complained. However, if you start looking at a cross section of robots.txt files, you see that very few are well formed.

With the addition of sitemaps, crawl-delay, and other non-standard syntax adopted by Google, Bing, and Yahoo (RIP). Clearly the RFC is just a starting point and what ends up on website can be broken and hard to interpret the author's meaning. For example, the Google parser allows for five possible spellings of DISALLOW, including DISALLAW.

If you read a few webmaster boards, you see that many website owners don't want a lesson in Backus–Naur form and are quick to get the torches and pitchforks if they feel some crawler is wasting their precious CPU cycles or cluttering up their log files. Having a robots.txt parser that "does what the webmaster intends" is critical. Sometimes, I couldn't figure out what some particular webmaster intended, let alone write a program that could. The only solution was to draft off of Google's de facto standard.

(To the webmaster with the broken robots.txt and links on every product page with a CGI arg with "&action=DELETE" in it, we're so sorry! but... why???)

Here's the Perl for the Blekko robots.txt parser. [https://github.com/randomstring/ParseRobotsTXT](https://github.com/randomstring/ParseRobotsTXT)

<comment><USER3> It's an easy fix if Google cared. Have an online tool that validates if the robots.txt is correct, and send out an announcement that files that don't meet spec will be penalized in terms of SEO.

<comment><USER4> That just punishes users by sinking relevant results for reasons users couldn't possibly care about.

<comment><USER5> I enjoy the hypocrisy of Google punishing sites that aren't "mobile-friendly," and then deliberately disabling ZOOMING on their own mobile sites.

</comment></comment></comment></comment><comment><USER6> I've been in disagreements with SEO people quite frequently about a "Noindex" directive for robots.txt. There seem to be a bunch of articles that are sent to me every time I question its existence[0][1]. Google's own documentation says that noindex should be in the meta HTML but the SEO people seem to trust these shady sites more.

I haven't read through all of the code but it assuming this is actually what's running on Google's scrapers this section [2] seems to be pretty conclusive evidence to me that this Noindex thing is bullshit.

[0] [https://www.deepcrawl.com/blog/best-practice/robots-txt- noin...](https://www.deepcrawl.com/blog/best-practice/robots-txt-noindex-the- best-kept-secret-in-seo/)

[1][https://www.stonetemple.com/does-google-respect-robots- txt-n...](https://www.stonetemple.com/does-google-respect-robots-txt-noindex- and-should-you-use-it/)

[2] [https://github.com/google/robotstxt/blob/59f3643d3a3ac88f613...](https://github.com/google/robotstxt/blob/59f3643d3a3ac88f613326dd4dfc8c9b9a545e45/robots.cc#L262-L276)

<comment><USER6> Google is also really generous with how they will let you spell "disallow": [https://github.com/google/robotstxt/blob/59f3643d3a3ac88f613...](https://github.com/google/robotstxt/blob/59f3643d3a3ac88f613326dd4dfc8c9b9a545e45/robots.cc#L691-L699)

:D

<comment><USER7> I'm not surprised. Some people think _humans_ read robots.txt and get super angry when the crawler doesn't understand.

<comment><USER8> I read robots.txt, but I'm not a massive corporation.

</comment></comment></comment></comment><comment><USER9> The interesting thing about robots.txt is that there really isn't a standard for it. This [0] is the closest thing to one and almost every modern website deviates from it.

For instance it explicitly says "To exclude all files except one: This is currently a bit awkward, as there is no "Allow" field."

And the behavior is so different between different parsers and website implementations that, for instance, the default parser in Python can't even successfully parse twitter.com's robots.txt file because of the newlines.

Most search engines obey it as a matter of principle but not all crawlers or archivers [1] do.

It's a good example of missing standards in the wild.

[0] [https://www.robotstxt.org/robotstxt.html](https://www.robotstxt.org/robotstxt.html)

[1] [https://blog.archive.org/2017/04/17/robots-txt-meant-for- sea...](https://blog.archive.org/2017/04/17/robots-txt-meant-for-search- engines-dont-work-well-for-web-archives/)

<comment><USER10> > The interesting thing about robots.txt is that there really isn't a standard > for it. This [0] is the closest thing to one and almost every modern website > deviates from it.

That is changing, and was announced today: [https://news.ycombinator.com/item?id=20326067](https://news.ycombinator.com/item?id=20326067)

<comment><USER9> Yeah, my first reaction to Google heading yet another standard was to cringe but this is one of the situations where I think it makes a lot of sense. They're dominant in the search industry and most other engines tend to take their cue so having them spearhead it seems like a good move.

</comment></comment></comment><comment><USER11> I absolutely understand why they did this, but I have to say I was disappointed to see only 7 commits at [https://github.com/google/robotstxt/commits/master](https://github.com/google/robotstxt/commits/master) dating back to June 25th.

When I read "This library has been around for 20 years and it contains pieces of code that were written in the 90's" my first thought was "that commit history must be FASCINATING".

<comment><USER12> I think it's pretty rare for a company to make internal commits public once making something open source.

<comment><USER13> yeah, otherwise you might see commit messages like "wtf is this shit" :)

</comment></comment></comment><comment><USER14> > This library has been around for 20 years and it contains pieces of code > that were written in the 90's.

Whilst I am sure there are good reasons for the omission, it would have been interesting to see the entirety of the commit history for this library.

<comment><USER15> From a archeological perspective very much.

From Google's perspective it's probably too much work. I would assume this was a part of the cralwer code and extracted over time into a library, while part of the monorepo, so changesets probably didn't only touch this code, but also other parts and this code probably depended on internal libraries (now it depends on Google's public abseil library) publishing all that needs lots of review (also considering names and other personal information in commit logs, TODO comments and their like)

<comment><USER16> Not only that, code libraries that weren't designed to be open source often have things in them that Google might want to show: codenames, profanity, calling out specific companies…

<comment><USER17> Also, even if it is authoritatively managed in git _now_ , the whole 20 year history certainly wasn't (since git is only 14 years old, and Google probably didn't adopt it on day one), and it's quite likely commit history wasn't converted,so it's quite possible Google couldn't easily make the whole history available when publishing it to GitHub even if they wanted to.

<comment><USER15> I assume the authoritative version is still in Google's Piper-based repo and previously was in perforce and I assume that was for a while ... so if there were interest Google's could dig deep. But I assume there are other projects where this is even more interesting. (how ranking changed over time; how storage formats for the index changed; ...)

</comment></comment></comment></comment></comment><comment><USER18> Code here: [https://github.com/google/robotstxt](https://github.com/google/robotstxt)

<comment><USER19> It's also the 5th link "open sourced" in the article.

</comment></comment><comment><USER20> Note that this is quite strict on what characters may be contained in a bots user agent. This is due to strictness in the REP standard.

[https://github.com/google/robotstxt/blob/master/robots_test....](https://github.com/google/robotstxt/blob/master/robots_test.cc#L152)

        // A user-agent line is expected to contain only [a-zA-Z_-] characters and must
        // not be empty. See REP I-D section "The user-agent line".
        // https://tools.ietf.org/html/draft-rep-wg-topic#section-2.2.1

So you may need to adjust your bot's UA for proper matching.

(Disclosure, I work at Google, though not on anything related to this.)

<comment><USER21> The strictness is in what may be listed in the robots txt, not the User-Agent header as sent by bots. the example given in the linked draft standard[0] makes this abundantly clear that it's on the bot to understand how to interpret the corresponding lines of robots.txt.

Of course, in practice robots.txt tend to look less like [1] and more like [2].

[0]: [https://tools.ietf.org/html/draft-rep-wg- topic#section-2.2.1](https://tools.ietf.org/html/draft-rep-wg- topic#section-2.2.1)

[1]: [https://github.com/robots.txt](https://github.com/robots.txt)

[2]: [https://wpengine.com/robots.txt](https://wpengine.com/robots.txt)

<comment><USER20> Sorry, I mean for matching, and I did try to imply it was a limitation of the standard and not the library. Though to avoid confusion, I do personally think keeping the user agent minimal is wise, since users might have difficulty guessing what value to use if it differs sufficiently from the real user agent that's sent.

</comment></comment></comment><comment><USER22> I wonder how much noindex contributes to lax security practices like storing sensitive user data on public pages and relying on not linking to the page to keep it private. I wonder how much is in the gap between "should be indexed" and "really ought to restrict access to authorized users only".

<comment><USER23> I am hoping not much, because that is beyond a horrible "security practice". I have seen some lazy shit out there, but this would take the cake.

<comment><USER22> If I recall correctly there was a large company several years ago who tried to prosecute a whitehat who discovered their user account pages included the users' e-mail addresses and that changing the address to that of a different user would drop you right into that user's page with all their personal information listed.

</comment></comment></comment><comment><USER24> > how should they deal with robots.txt files that are hundreds of megabytes > large?

What do huge robots.txt files like that contain? I tried a couple domains just now and the longest one I could find was GitHub's - [https://github.com/robots.txt](https://github.com/robots.txt) \- which is only about 30 kilobytes.

<comment><USER25> They enumerate every page on the site sometimes specifically for different crawlers.

Or they have a ton of auto generated pages they don't want crawled and call them out individually because they don't realize robots.txt supports globing.

<comment><USER26> Can you given an example in the wild?

<comment><USER25> I was actually trying to find an example when I made my initial comment, but was unable to. It's been a long time since I did web scraping. Since then there are a lot more frameworks that help you build a website (and a correspondingly sane robots.txt), so there may not be as many as before.

</comment></comment></comment></comment><comment><USER27> fun & useless little bit of trivia: Sci-Fi author [1] Charles Stross (who hangs around here) is the cause of the first robots.txt being invented.

[http://www.antipope.org/charlie/blog- static/2009/06/how_i_go...](http://www.antipope.org/charlie/blog- static/2009/06/how_i_got_here_in_the_end_part_3.html)

(reminds me how Y Combinator's co-founder Robert Morris has a bit of youthful notoriety from a less innocent program)

[1] and former code monkey from the dot-com era

</comment><comment><USER28> I guess lots of people misspell ~disalow~ disallow[1]

1\. [https://github.com/google/robotstxt/blob/master/robots.cc#L6...](https://github.com/google/robotstxt/blob/master/robots.cc#L691)

<comment><USER29> including yourself! Must be easy to do. :) [https://www.merriam- webster.com/dictionary/disallow](https://www.merriam- webster.com/dictionary/disallow)

<comment><USER28> Oh snap!

</comment></comment></comment><comment> noir-york I doubt there's any vulns in the code seeing as its job for th last 20 years has been to parse input from the wild west that is the internet, and survive.

But I'm sure someone out there will fuzz it...

<comment><USER30> I'd be surprised if google isn't fuzzing it with their (also open sourced) fuzzing tool.

<comment><USER31> I'd never heard of this, so looked into it:

[https://opensource.googleblog.com/2019/02/open-sourcing- clus...](https://opensource.googleblog.com/2019/02/open-sourcing- clusterfuzz.html)

Fascinating.

</comment></comment></comment><comment><USER32> can this been seen as a initiative to make google robots.txt parser the internet standard? every webmaster will want to be compliant with google corner cases...

<comment><USER33> That's probably the hidden agenda.

<comment><USER34> It's not even hidden, Google explicitly says that in the blog post.

</comment></comment></comment><comment><USER35> Anyone else witnessed this behavior? [https://stackoverflow.com/questions/4769140/robots-txt- user-...](https://stackoverflow.com/questions/4769140/robots-txt-user-agent- googlebot-disallow-google-still-indexing/52732538#52732538)

<comment><USER36> There is a difference between robots.txt blocking a page and noindexing a page.

Blocking in robots.txt will stop Googlebot downloading that page and looking at the contents, but the page may still make it into the index on the basis of links to that page making it seem relevant (it will appear in the search results without a description snippet and will include a note about why).

To have a page not appear in the index you need to use a 'noindex' directive [1] either in the file itself or in the HTTP headers. However, if the file is blocked in robots.txt then note Google cannot read that noindex directive.

Also, in the StackOverflow response you linked to that the user agent is listed just as 'Google', but it should be 'Googlebot' as per the 'User agent token (product token)' table column listed in [2].

Good luck! :)

[1] [https://support.google.com/webmasters/answer/93710?hl=en](https://support.google.com/webmasters/answer/93710?hl=en) [2] [https://support.google.com/webmasters/answer/1061943](https://support.google.com/webmasters/answer/1061943)

</comment></comment><comment><USER37> That's actually nice and straight forward and relatively simple. I had expected something over engineered with at least parts of the code dedicated on demonstrating how much smarter the code writer is than you. But it's not. Just a simple parser.

<comment><USER38> Looks like standard Google C++ coding style to me.

Honestly, excessive cleverness does not generally pass code review @ Google. Especially something that would get this many eyes.

<comment><USER39> Honestly I'm most surprised they haven't replaced all the c-isms already. Seeing raw char pointers and strbrk is...weird.

<comment><USER38> Yes as a Googler, I'd probably flag that in review.

But it being old and critical, I'd also be wary of major changes.

</comment></comment></comment></comment><comment><USER40> Seems strange to get excited about a robots.txt parser, but I feel oddly elated that Google decided to open source this. Would it be too much to hope that additional modules related to Search get released in the future? Google seems all too happy to play the "open" card except where it directly impacts their core business, so this is a good step in the right direction.

</comment><comment><USER41> Looking forward to the robots.txt linters created as wrappers around this (especially for VSCode).

</comment><comment><USER42> I find it really cool the code for this is so simple and clean.

</comment><comment><USER43> I don't understand the entire architecture behind search engines, but this seems like a pretty decent chunk of it.

What are the chances that Google is releasing this as a preemptive response to the likely impending antitrust action against them? It would allow the to respond to those allegations with something like, "all the technology we used to build a good search engine is out there. We can't help it if we're the most popular." (And they could say the same about most of their services: gmail, drive, etc.)

</comment><comment><USER44> So, is it premature to expect a Go package by Google as well?

There's already [https://github.com/temoto/robotstxt](https://github.com/temoto/robotstxt)

<comment><USER45> This is the sort of code you write a binding to and call it a day, since the entire point is to absolutely precisely match the behavior of this code, which is basically a specification-by-code. You can never be sure a re- implementation would be absolutely precisely the same in behavior, so it's not worth doing.

<comment><USER46> The c++ implementation is <1000 lines. Doesn't seem like a correct port would be particularly difficult, especially with a reasonably large test corpus.

<comment><USER45> Famous last words.

I mean, I get it; it feels that way to me intuitively too. But I'd still recommend against trying it, because I've learned the hard way the intuition here is, if not _wrong_ , at the very least _very badly underestimating the cost_ , especially in the "unknown unknown" department.

<comment><USER47> Do you even need to match Google's robots.txt parsing behavior? With less than 1000 lines you can be pretty sure they are not doing it right and are breaking plenty of people's assumptions about it. Either way you have to test it on real world data.

<comment><USER45> The point of this code release seems to be to release Google's precise logic. That you may incorporate it into something else is, IMHO, less interesting; we've got plenty of other solutions that "do robots.txt" well enough. If it was just about that, Google's release of this would not be worth anything. The point is so that non-Google parties can see exactly what Google is seeing in your robots.txt.

That's why I'm saying there's no point trying to re-implement this. If you were going to re-implement this, there's probably already a library that will work well enough for you. The value here is solely in being _exactly_ what Google uses; anything that is a "re-implementation" of this code but isn't _exactly_ what Google uses is missing the point.

If they formalize it into a spec, others may then implement the spec, but they can and should do that by _implementing the spec_ , not porting this code.

<comment><USER47> As I understand the point about Go complaint is to parse actual real world robots.txt. For which you don't need to behave exactly as this library does.

</comment></comment></comment></comment></comment></comment></comment><comment><USER48> Cool, thanks

</comment><comment><USER49> Is Golang significantly slower than c++ ? I thought Google had invented Golang to solve precisely these kinds of code for their internal use.

I had thought most of the systems code inside Google would be golang by now. is that not the case ? the code doesnt look too big - I dont think porting is the big issue.

<comment><USER50> > Is Golang significantly slower than c++ ?

Depends the context, but in general, yes. C++ is very close to C on this aspect, trading memory safety for performances.

Concerning google, as far as I know the codebase is mostly C++, Java, and python. Go will surely eat a bit of the Java and Python projects but it's unlikely to see C++ being replaced any time soon.

<comment><USER51> > Is Golang significantly slower than c++ ? > Depends the context, but in > general, yes.

I don't believe this is the case. Most optimized, natively compiled languages all perform similarly. Go, C, CPP, Rust, Nim, etc. I'm sure there are edge- cases where this isn't the case, but they all perform roughly the same.

The performance rift only starts when you introduce some form of a VM, and/or use an interpreted language. Even then, under certain workloads their optimizations can put them close to their native counter parts, but otherwise are generally slower.

The real reason Google didn't re-write this in Go is likely because the library is already finished, it works, a re-write would require more extensive testing, etc. Why spend precious man-hours on a needless re-write?

<comment><USER52> Golang and Java have similar performance characteristics. I would not put it in the same class as C/CPP/Rust.

[https://benchmarksgame- team.pages.debian.net/benchmarksgame/...](https://benchmarksgame- team.pages.debian.net/benchmarksgame/fastest/go.html)

<comment><USER53> Presumably less startup time and JIT.

<comment><USER52> Depends on if the Java code is ahead of time compiled or not. The AOT compiler is included in the latest JDKs.

<comment><USER54> _fwiw_ AoT with Substrate VM

[https://benchmarksgame- team.pages.debian.net/benchmarksgame/...](https://benchmarksgame- team.pages.debian.net/benchmarksgame/fastest/java-substratevm.html)

<comment><USER52> Nice, good to know the JIT is doing something. Will be interesting to track the performance of this over time.

<comment><USER54> And that's _graalvm-ce_ not the enterprise edition.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER55> "Because the REP was only a de-facto standard for the past 25 years, different implementers implement parsing of robots.txt slightly differently, leading to confusion. This project aims to fix that by releasing the parser that Google uses."

The amount of arrogance in this sentence is insane.

Because Google way is the only one true way?

<comment><USER56> In terms of "what should a robots.txt file look like to be parsed correctly," yes, because they're the ones who are going to be doing most of that parsing. Yes, ideally it would be an entirely independent standardization process, but it's not arrogant of them.

</comment></comment><comment><USER57> Never before has a company stood on such a mountain of open source code, achieved so much money with it and contributed _so_ _little_

No really. Microsoft? BSD TCP/IP stack for win95 maybe saved them but there was trumpet winsock and probably would have survived to writing their own on the next release.

Google doesn't get off the ground and has literally no products and no services without the GPL code that they fork, provide remote access to a process running their fork and contribute nothing back. Good end run around the spirit of the GPL there and that has made them a fortune (they have many fortunes, that's just one of them).

New projects from google? They're only open source if google really need them to be, like Go which would get nowhere if it wasn't and be very expensive for google to have to train all their engineers rather than pushing that cost back on their employees.

At least they don't go in for software patents, right? Oh, wait...

At least they have a motto of "Don't be evil" Which we pretty much all have personally but it's great a corporation backs it. Corporate restructurings happen, sure, oh wait, the motto is now gone. "Do the right thing" Well this is fine and google do it, for all values of right that equal "profitable to google and career enhancing for senior execs".

But this is great a robots.txt parser that's open source. Someone other than google could do something useful for the web with that like writing a validator, because google won't. Seemingly because it's not their definition of "do the right thing."

"Better than facebook, better than facebook, any criticism of google is by people who don't like google so invalid." Only with more words. Or none just one button. Go.

<comment><USER58> So you aren't wrong that google is built on the shoulders of giants, but I will point out that every single company today running their SaaS offering on top of linux/BSD is doing the exact same thing.

The only reason Linux is as mainstream as it is today, is exactly because of this freedom to leverage the code. You even point out that the cause for Golang's success is for precisely the same reason. Overall opensource isn't about making money, it has never been about making money. Its been about making an impact, and bettering the world around us all by giving a piece of technology to be freely used by everyone. There are a variety of opensource licenses that can/will protect your code from any/all closed source uses, for example AGPL explicitly states if your application so much as interacts with the code over a TCP connection or furthermore a single UDP packet it must be opensource as well. However you will rarely see libraries/applications using this license. Why you might ask? The answer is simple, it reduces the impact that code can have.

Really at the end of the day, it comes down to a choice of the developer(s), do you want to make money? i.e. go the Microsoft/Apple route? or do you want to make an impact? i.e. go the Linux/BSD route?

Let me ask one final question, which of the above operating systems do you think are more widely used, or have changed the world in a more dramatic manner?

<comment><USER57> I could care less about other companies that have existed for 5 minutes in the SaaS space in my comment that nobody has ever derived more value and given that, contributed less back.

Google is built on an end run around the spirit and intent of the GPL. "Don't distribute software, distribute thin client access to it! No GPL! Hurrah! Money!"

Decide for yourself what you think of that but it happened. Without it, no google.

But hey, list anyone you think derived more value and contributed less back. It's a reasonable thing to do. Doesn't affect criticism of google.</comment></comment></comment>


AskUbuntu: The following signatures were invalid: EXPKEYSIG 1397BC53640DB551 - <AUTHOR> https://askubuntu.com/questions/1133199/the-following-signatures-were-invalid-expkeysig-1397bc53640db551 <comment><USER2> >" _Apparently Google did not extend the validity of the signing cert... it was due to end today and so it did._ "[0]

It happens to all of the "too big to fails".

[0] - [https://askubuntu.com/questions/1133199/the-following- signat...](https://askubuntu.com/questions/1133199/the-following-signatures- were-invalid-expkeysig-1397bc53640db551/1133208#1133208)</comment>


IBM open sources Mac-at-IBM code - <AUTHOR> https://9to5mac.com/2018/10/23/ibm-open-sources-macibm-code/ <comment><USER2> Here's the link to the source code: [https://github.com/ibm/mac-ibm- enrollment-app](https://github.com/ibm/mac-ibm-enrollment-app)</comment>


The Tweetbot Roadmap — Tapbots Blog - <AUTHOR> http://tapbots.com/blog/news/the-tweetbot-roadmap <comment><USER2> Lack of push notifications natively is the primary reason why I haven't purchased tweetbot.

Isn't there some middleware they could use to solve this problem intelligently (like urban airship)?

<comment><USER3> They point out in the post that they cant justify the costs of using something like urban airship right now, based on the initial 'revenue' per sale. Theyre looking into maybe a subscription service:

        "Hopefully we can get the price per user down enough that it can just be a free
        feature for everyone. If not we may have to resort
        to a nominal yearly subscription fee for the service."

</comment></comment><comment><USER4> This is good. It helps to illustrate why you should buy apps from awesome teams, even if they're not _exactly_ everything you need. you're supporting the dev of those features in the future.</comment>


Grocery shopping via smartphone on South Korean subways - <AUTHOR> http://www.springwise.com/retail/homeplus/

<comment><USER2> Previously submitted (with discussion). <http://news.ycombinator.com/item?id=2856609>

Form a comment of cliff: <http://news.ycombinator.com/item?id=2857088>

It is fake / only a proof of concept: [http://english.hani.co.kr/arti/english_edition/e_business/48...](http://english.hani.co.kr/arti/english_edition/e_business/489884.html)

</comment><comment><USER3> I love the execution on this, turning what was otherwise a normal billboard into something that's much more interactive and useful. What stands out for me are:

\- They tried to duplicate the look and feel of being in the supermarket, allowing customers to navigate naturally in their browsing and shopping habits. Don't underestimate the power of routine/habit and creating experiences that are familiar/comfortable.

\- They also deliver the groceries so it completely replaces the need to make a trip to the grocery store which is a time-saver.

\- This works well in areas of high smartphone/internet adoption rate like South Korea and Japan.

I was thinking about whether or not this idea would work in San Francisco/Bay Area given the high mobile adoption rate and young tech-savvy crowd. I would focus exclusively on fresh produce given that these are often the most expensive in stores and its risk of spoilage leads many of us to not buy as much as we should.

Ask HN: Would you shop via a BART/Caltrain/Bus Ad using a smartphone app for groceries if the price as more competitive than Whole Foods and you got free delivery to your workplace/home?

</comment><comment><USER4> From what else I've seen about this, it's not actually a functioning store and was mostly a publicity stunt.

That aside, I'm curious about why they would have multiple slots on the same shelf for the same product. At least looking at the image in the article, it appears that some of the milk and various juices have >1 slot on the same shelf. Since there is no stocking issue and there need only be one QR code to scan per product, this seems like a waste of space where additional products could be displayed.

I suppose in this case, it was probably just a consequence of them using a photo of a real-life cooler. However it would be interesting to find out if it's more effective to retain that traditional in-store setup versus using a display that had only one image per item. Perhaps keeping the normal in-store look adds a certain level of comfort/familiarity that makes up for reduced product set.</comment>


POWER8 firmware release notes for Spectre/Meltdown - <AUTHOR> https://download4.boulder.ibm.com/sar/CMA/SFA/07d79/0/01SC860_138_056.html <comment><AUTHOR> What's interesting is that I noticed that the firmware notes indicate that CVE-2017-5753 and CVE-2017-5754.

It's not clear to me if IBM is acknowledging that POWER is affected by Meltdown (thought to be mostly/only Intel so far) or if this is an error in their notes.

<comment><USER2> Meltdown being Intel-specific has been parroted a lot, but the primary sources say:

\- The Meltdown paper speculated that the Meltdown class of Attack should/could work on AMD & ARM, even though they hadn't developed their PoC exploit for non-Intel CPUs

\- ARM published a note saying some of their cores are vulnerable to Meltdown.

\- AMD put out a press release saying AMD is only to Spectre, but there has been no technical/engineering explanation of the claim.</comment></comment>


NetbootCD: Install Ubuntu, Fedora, Debian & More From One CD  - <AUTHOR> http://www.makeuseof.com/tag/netbootcd-install-ubuntu-fedora-debian-cd-linux/ <comment><USER2> See also: NetBoot.Me: <http://www.netboot.me/>

</comment><comment><USER3> What's a CD?

<comment><AUTHOR> Round piece of plastic which can hold some information in electronic format. Also can be used as coaster for coffee cup.

<comment><USER4> With a little work from an old 'Make' article you can also use it as a target for shotgun practice---cheaper than clay pigeons and they have that 'ooh shiny thing' going for them.

</comment></comment></comment><comment><USER5> Hmm, burning a bootable ISO image to a USB stick seems more reasonable from time to time.

<comment><AUTHOR> Not all Linux distributions have ISOs ready for USB-usage. Some of them are still sticking to CD as media.

<comment><USER6> Is there anything that Unetbootin won't work on?

<comment><AUTHOR> Mageia

<comment><AUTHOR> I honestly tried it: [http://linuxblog.darkduck.com/2011/06/mageia-is-it-kind- of-m...](http://linuxblog.darkduck.com/2011/06/mageia-is-it-kind-of- magic.html)

</comment></comment></comment></comment></comment><comment><USER7> also yumi - <http://www.pendrivelinux.com/yumi-multiboot-usb-creator/> \- mentioned in the comments

<comment><USER8> YUMI is OK if you spend the time to download the distros ahead of time and store them all on your USB key. NetBoot is good if you want to get a distro you haven't downloaded yet. That way you have a lot more distros available than can fit on your thumb drive. The downside is that you have to download them every time you install them.

</comment></comment><comment><USER9> I expected to see Arch Linux in the list. Go figure.</comment>


"A little bit of slope makes up for a lot of y-intercept" - <AUTHOR> https://www.quora.com/What-are-the-most-profound-life-lessons-from-Stanford-Professor-John-Ousterhout?share=1 <comment><USER2> One of Ousterhout's examples of this principle is something that comes up frequently on HN:

 _" Before I came back to academia a couple of years ago I was out doing startups. What I noticed is that when people hire they are almost always hire based on experience. They're looking for somebody's resume trying to find the person who has already done the job they want them to do three times over. That's basically hiring based on Y-intercept.

Personally I don't think that's a very good way to hire. The people who are doing the same thing over and over again often get burnt out and typically the reason they're doing the same thing over and over again is they've maxed out. They can't do anything more than that. And, in fact, typically what happens when you level off is you level off slightly above your level of competence. So in fact you're not actually doing the current job all that well.

So what I would always hire on is based on aptitude, not on experience. You know, is this person ready to do the job? They may never have done it before and have no experience in this area, but are they a smart person who can figure things out? Are they a quick learner? And I've found that's a much better way to get really effective people."_

<comment><USER3> Intriguingly, that's the opposite of what's popular on HN. HN and /r/programming posters prefer the "Look at what I've done. Here are my projects" approach which is the y-intercept because it describes where they are. Asking them a question they've never heard before and observing their approach is a way (albeit imperfect) of seeing how fast one can deal with a novel problem.

<comment><USER4> "Look at what I've done. Here are my projects" is definitely not the Y-intercept "Look I went to <fancy school>" is the Y-intercept. With projects and experiences you can start to plot someone's work according to it's own Y-intercept and guess the slope. The hard part is assessing someone's slope when all you have is it's Y-intercept ( freshly out of school ).

Concerning asking tricky question, you can do it the Google way ( How many frogs does it take to build a space elevator ) but even they decided to stop these kind of questions or take a lot of time and find something your candidate can work on, even if the technology is known to him, seeing how fast he can appropriate himself the codebase is really a good way to judge your future new recruit potential

<comment><USER5> You could try to "guess the slope" if the slope were static, but it's not. Aptitude and track record are, in my experience, not correlated. I can get specific about this if you want, but I don't want to be tiresome about this subject (you can find similar comments in my history).

A short preemptive summary is: the last 10 amazing hires we made would not have been distinguishable from their online portfolios.

<comment><USER6> That sounds very interesting. Could you elaborate ? How did you figure out their aptitude if their track records weren't so different from the rest ?

<comment> barry-cotter They appear to be the only software company on Earth to do work sample tests. It's working out really well for them. The process is extremely well laid out here.

matasano.com/careers

We'll get on the phone and talk to you about the company and what our work looks like. At the end of this call you should have a good idea of what we do, how our hiring process works, and answers to questions about Matasano. Most importantly, you'll have a contact at Matasano to talk with and bounce questions off of through the duration of our process. We do 1-3 technical phone screens. You'll talk to a senior Matasano team member who will ask you about your technical background and talk you through scenarios and concepts from our day-to-day work. If you've been doing app security for 5 years, you'll be talking about your past projects; if you're a developer, you'll be talking about code. We do a web app challenge. Most software written within the last several years is web code. Everyone on our team needs to be able to deliver a solid web pen test. When you're ready, you'll be given an instance of a vulnerable web application and an hour or so to break it. We timebox challenges to avoid taking too much of your time. You're doing this on your own schedule, in your own comfortable setting. We do a custom protocol challenge. Every Matasano team member routinely runs into exotic network protocols. We'll throw something at you that you're unlikely to have worked with before and watch you reason your way through breaking it. This challenge seems to be everyone's favorite; candidates routinely tell us how they particularly enjoyed it. That's great! It's part of our day-to-day work here. Like the web challenge, it's timeboxed and you're doing it remote. We'll have you write a fuzzer. Everyone here writes fuzzers. We'll give you a file format. In the language of your choosing, you'll write a fuzzer for it. This gives us a chance to see how you code and to see what types of things you automate testing for. Like the other challenges, this one is time limited and you can do it remote. We've talked. We've done phone screens. We've answered questions. You've done challenges for us. At this point we both have a pretty good idea whether you'll be happy working with us. If that's the case, we'll bring you onsite for an in-person interview, which concludes our hiring process.

</comment></comment></comment></comment></comment></comment><comment><USER7> Good points. I would add to his last point though that hiring at least one person with a high y-intercept can increase the slope of your other hires significantly. (It's a lot easier to learn something quickly if you have someone around who already knows a lot about it.) And people who both know a lot and learn quickly are expensive and rare, so I expect the optimal hiring process would look for a large percentage of relatively inexperienced fast learners, along with a small (but non-zero) percentage of more experienced people, even if they aren't likely to develop as quickly. (Perhaps consultants would fit well in that role too.)

</comment><comment><USER8> [http://web.stanford.edu/~ouster/cgi- bin/sayings.php](http://web.stanford.edu/~ouster/cgi-bin/sayings.php)

</comment><comment><USER9> One hopes that the graph of life is linear and not a differential equation path-dependent on initial conditions...

</comment><comment><USER10> Ah, Ousterhout. Guess he's at Stanford now; he was a prof at Berkeley when I was there. He had an amazing knack for presenting CS in an entertaining (and, at times, hilarious) manner.

Best professor I ever had.

</comment><comment><USER11> It's a bit of a false dilemma to choose between experience or aptitude.

In the start-up period, though, aptitude is a better strategic choice in that you need people who can grow as quickly as the venture.

Later, you need the people with experience, even if they're a bit slow. They're the ones that have been bitten by the edge cases, the deadlocks, the XSS openings - and know to avoid them.

<comment><USER12> Really? I'd rather have the experienced but dull people to put together the minimal product first. Having them onboard gives me more time to find and hire the right aptitude people, and having them there _first_ keeps the inexperienced people from sticking in all the deadlocks and security holes in the first place.

Do people really want their minimally-viable product to be full of deadlocks? I personally do not. And I don't want to hire promising but new-to-the- industry people who have to check stackoverflow to remember the difference between a pointer and a reference, or who spend hours trying to debug why something in their python program is iterating over a string.

Bottom line, I guess I don't see how anyone can want to bring in aptitude in the absence of experience. People with both would be fine, or a combination of both types of people would be fine. Aptitude without experience is not going to work.

</comment></comment><comment><USER13> I happen to disagree regarding the hiring. I prefer to put more emphasis on the system itself, and not the developers, who should be replaceable. (I say this as a developer myself, who prefers to be replaceable.)

When a developer forgets to test their code before pushing it to production, we often blame the developer. But the real problem is lack of automated testing, lack of processes, and too much responsibility for the developer.

With a good system in place, you hire people who have all the prerequisite knowledge (the languages, patterns, experience with similar solutions to the ones they'll do, and preferably good team spirit that matches your culture). The rest can be learned on the job. But once again, focus on your onboarding materials!

In short -- you should always look to be optimizing the system. THAT is your "slope" if you will. Except it's not a slope, it's an exponent! Because it builds on itself week after week. And you don't risk that one developer somewhere messing up your code.

We say: people live lives, companies create products.

</comment><comment><USER14> Said another way: You are perfect exactly as you are, because in this moment, there is no other way you can possibly be.

However, in this moment, perfect as you are, you can still resolve to practice.

\- Paraphrased from "The Practicing Mind"

</comment><comment><USER15> Let's race 10 miles. I run each mile in 5:59 and you run each mile in 6:00. You get a mile head start.

Whoops—this fable is only true "eventually", if the slopes remain the same as t→∞.

<comment><USER16> I'm curious, do you think Ousterhout or the people quoting him are confused about this?

<comment><USER15> Probably everyone knows the math; I just think they're exaggerating the relevance of this model to real life.

I'm with the commenter (here) who said that real life is more likely to be an initial-value-dependent or path-dependent PDE. Someone else said having a high-IV person on your team will raise the slopes of everyone else which seems also right: the issues are multidimensional, not affine 1-D.

Ousterhout is taking sides (smarter > more experienced) which is fine; he can make that argument. But using y=mx+b as x→∞ doesn't count as an argument; it's rhetorical flair, not rhetorical substance. The substance of his reasoning seems to be "That's my opinion based on my experience in my past jobs".

<comment><USER16> I read it more as an illustrative example to open up the following discussion than a terribly serious model of real life.

He does riff back on it, but it's five minutes of perspective tossed at teenagers, not anything formal.

<comment><USER15> It seems like a way to lend scientificity to his opinion. There are other ways of saying smarts>experience. Invoking y=mx+b here seems deceptive.

<comment><USER17> Analogies are inexact by definition, which doesn't mean that the speaker is being intentionally deceptive by using one.

I don't think anyone took this beyond the anecdotal evidence provided. We can argue motive and hypotheticals all day but the fact remains that this is just a blurb thrown at fresh undergraduates.

It's a motivational speech (delivered via heresay on Quora) not a thesis.

</comment></comment></comment></comment></comment></comment><comment><USER18> Why do Quora articles continue to make the front page when we can't read them? I'm greeted with [http://i.imgur.com/WTeSt8s.png](http://i.imgur.com/WTeSt8s.png) No, I won't sign in with Google, nor Facebook.

How do the people upvoting this read the article? Do they sign in? What possible benefit is there to signing in? They're basically holding content hostage.

When I defended Scribd, people came out in droves to point out how wrong it was to hold unique content hostage. I'll admit, it made me rethink my position. But it's strange to see that Quora doesn't get the same stick.

EDIT: If the HN homepage had a popup saying "Login with Google or Facebook to read all of HN," would you tolerate it?

<comment><USER19> Speaking only for myself, I visit Quora so frequently that I pretty much always have a login cookie stored, so I never see the login page.

That said, given my very public and vocal arguments against "walled gardens" in the past, why do I tolerate Quora? For the same reasons I tolerate G+, Facebook, etc. I annoys me and tickles a sore spot regarding the way I want things to be, but the value of the content is sufficient for me to "grin and bear it". But if there were a really good Quora alternative that was more open, I'd work to promote and support it. I don't know of one offhand though, especially considering network effects (that is, the value of Quora isn't the Quora software, it's the people posting and answering).

<comment><USER20> I have a similar position. If the content is valuable enough then it's tolerable to have a gate. Quibb is another similar platform, but I don't know of anything with the quality of Quora answers - Stack Overflow is the closest, but it's very specific and Quora covers a lot of content the various Stack sites don't.

</comment></comment></comment><comment><USER21> Link without the annoying Quora-login pop-over: [https://www.quora.com/What- are-the-most-profound-life-lesson...](https://www.quora.com/What-are-the-most- profound-life-lessons-from-Stanford-Professor-John-Ousterhout?share=1)

</comment><comment><USER22> Actually, if you are going in the opposite direction, a little bit of slope makes up for a lot of negative Y-intercept.</comment>


Show HN: Jupyter4kids – Notebooks to help teach kids principles of programming - <AUTHOR> https://github.com/mikkokotila/jupyter4kids <comment><USER2> Very nice!

Please consider uploading it to

Http://notebooks.azure.com

It's a free service for running and sharing Jupyter notebooks. That way the can run it immediately w/on installing anything (has anaconda).

Lots of teachers are using it. For example:

Http://notebooks.azure.com/richie

Disclaimer: I work on this :)

<comment><AUTHOR> Thanks, this seems exactly what I was looking for! One question though, all demos I looked at had 'run' disabled. Is there some 'catch' to this?

<comment><USER3> I'll chime in here (Disclaimer, also dev for the Azure Notebooks team, etc etc)

If you want to alter/play around with the demo, you can "clone" the sample library, which will prompt you to create an account if you have not. This process is free/noncommittal/just requires an email address, but lets us create an env + storage for you to run and persist it in. Post-cloning, you can run any library you own like you would expect.

</comment></comment></comment><comment><USER4> Funny, this thursday, I just used jupyter notebooks to teach basic python to beginners who wanted to plot graphs and clean up a data file: [https://github.com/Hackerfarm/python- workshops](https://github.com/Hackerfarm/python-workshops)

1 to 3 are introductions to basic concepts. 4 is a bit more specific to the work we are doing here at the hackerfarm.

<comment><AUTHOR> Very cool! The first cell of your first notebook "reminds" me that we should have one part which just talks about using Jupyter in an effective way (tab completions / shortcuts / etc).

</comment></comment><comment><USER5> Please consider using Python 3 instead of Python 2.

In particular, in Python 3, the expression 2/3 no longer gives 0, but 0.666… Much more intuitive.

<comment><AUTHOR> Thanks, I think that is a good idea. Especially given we are talking about the future generation, and not dinosaurs like myself (attached to 2.7). Will make that change few parts down and then "upgrade" the early parts as well.

</comment></comment><comment><USER6> Quick question: who is the target audience? Have you tried showing it to them?

<comment><USER7> I looked in the notebooks and I don't think programming can be taught like this:

        if 1 is 1:
            print("hello world")

What is this "if 1 is 1" ? It makes no sense to ask this question, for a kid. It's not clear in English, and if you don't know programming syntax, it is quite mysterious.

We need some kind of game where kids can learn the purpose of if statements and other basic operations in a grounded way, not purely theoretical.

I think the best way to go about teaching programming to kids is to invent some interesting problems they can solve and let them play it out. We can't axiomatically introduce programming at that age.

<comment><AUTHOR> Thanks for your feedback.

The whole point of initially using non-sensical (and obvious) examples is 1) to make it as easy as possible to go beyond any doubt with basic numeric programming concepts 2) to allow sinking in the key difference between people and computers, for computers nothing at all is obvious, everything has to be tested. In the later parts, this will become obvious.

For the 'games' you are looking for, there are already many options like scratch. In this series, where these three parts are the start, the idea is to take people of all ages to learn advanced data science in a fun and simplistic way, along the way cementing solid understanding of fundamental numerical computing concepts.

Obviously the later parts will do exactly what you suggest, allow kids to tackle actual problems that are relevant to their lives and their families lives.

</comment></comment></comment><comment><USER8> The 'start learning' page tells the user to try `print("hello world")` but doesn't tell the user _where_ to put it. The page has even gone so far as to tell the user that it is _not_ to be put into an IDE or CLI prompt.

You might want to help the user install and open Jupyter before telling him what to type into it.

<comment><AUTHOR> Good point. I'll try to get get the notebooks to an interactive environment (maybe Azure as is suggested below unless there is some run restriction) so there is ability to run the codes right there and then. I'm surprised that there is not more competition in the free notebook run space.

Regarding Jupyter, I'll do a notebook that's just on Jupyter at some point, that covers general use cases, cell types, formatting, and shortcuts.

<comment><USER8> Why not save the user the trouble of establishing an Azure account and keep everything local?

<comment><AUTHOR> I also don't like the idea of having to create Microsoft account just to access a notebook, so if it turns out that is required for run access, then the other option is to self-host it instead. The problem with local is that if you have no experience with programming, or are on ipad, etc. etc. etc. having to start by going to terminal, installing jupyter is kind of drag. But I agree that local should be an option, and there needs to be a notebook that starts with how to do it, and then have a link to that notebook at the top of all other notebooks.</comment></comment></comment></comment>


We Don't Need to Work So Much (2015) - <AUTHOR> https://www.newyorker.com/news/daily-comment/you-really-dont-need-to-work-so-much <comment><USER2> As an European working with lots of American contractors in the tech field, I'm always impressed by their work ethics. The put in 10-12 hours, 6 days a week.

But from talking with them, it seems like their focus is on retirement - hopefully in their 50's. That's their grand plan, because that's when they'll start "living".

It's a bit of a contrast to us Scandinavians - as we really like to live in the present. No pie in the sky dreams.

Which is also why we guard our working rights and work-life balance like a national treasure.

<comment><USER3> I find it hard to empathize with FIRE people [0] because I like my job. But the few I've discussed it with absolutely _despise_ working. Like all- consuming, life-purpose-defining revulsion. And I think there's a feedback loop to it, because in trying to "get it over worth" they increase the intensity and make it worse for themselves. Then it becomes all the more important to get it over with quickly.

I wonder if the European attitude is just more similar to mine, being comfortable with and accepting of a job as a backdrop to build a life around, rather than something intolerable to escape from ASAP at any cost. Probably has to do with the jobs themselves being more comfortable.

[0] [https://www.investopedia.com/terms/f/financial- independence-...](https://www.investopedia.com/terms/f/financial-independence- retire-early-fire.asp)

<comment><USER4> You can enjoy your job and pursue FIRE. FIRE just gives you options. You can either stop working, do other work that doesn't pay as well or at all, or keep working and live at a much higher standard of living. Maybe the company you like working for only gives you a few weeks off per year and you want to take 1-2 months of unpaid time off. Maybe you don't want to care about layoffs or market changes or bad management. Maybe one day you wake up and decide you want to spend time with your kids and productive work can take a breather for a while.

It's foolish to focus just on enjoying your job right now, finances be damned, without planning for the future. For what it's worth, I think it's also foolish to despise your job every day for years and years just to retire a few years earlier. But that's not what FIRE is, that's just one group of people taking it to extremes, as is common in any endeavor. For most people, it just means diligently and consistently setting money aside that will give them a level of freedom that those who don't will never achieve.

<comment><USER3> Maybe we are talking past each other. By "FIRE" I mean the ability to live on investment income, exclusively and permanently, decades before a typical retirement age.

Retirement on a normal schedule and security for a few months of rest here and there are, I would say, baseline financial responsibility (if you can afford them). These are marginal sacrifices, not a lifestyle/subculture.

<comment><USER5> There is lean FIRE, fat FIRE, barista FIRE, only FI, and several other variations of FIRE

</comment></comment></comment></comment></comment><comment><USER6> The whole 'we should work less' meme strikes me as more about the meaninglessness of modern work, moreso than our desire to have more leisure time. In other words: people want more meaningful work, not merely to have less meaningless work. Maybe this is an American-centric idea, wherein your work is strongly tied to your purpose and identity, but I don't see how you can simply brush away an activity that takes up 1/3 of your time.

The example in the article (Amazon packing plants) is perhaps the most pointless Kafkasque job one can imagine: packing boxes of mostly unnecessary consumeristic products.

Speaking only personally, but I _want_ to be working so much that I have little leisure time, but only because the work I'm doing is fulfilling and fundamentally important. I have no desire to have more leisure time to consume content (which is what most people do: look up the hours spent watching TV/video per person, or social media usage), as I don't think consumption activities are ultimately fulfilling.

<comment><USER7> The pleasures of urban populations have become mainly passive: seeing cinemas, watching football matches, listening to the radio, and so on. This results from the fact that their active energies are fully taken up with work; if they had more leisure, they would again enjoy pleasures in which they took an active part.

\-- Bertrand Russell, 1932 [http://www.zpub.com/notes/idle.html](http://www.zpub.com/notes/idle.html)

<comment><USER8> We had a rapid check of this hypothesis with unemployment high in the US. People were sitting at home with nothing to do.

Perhaps the massive jump in active leisure will suddenly manifest but so far no such thing has come to be.

<comment><USER9> > Perhaps the massive jump in active leisure will suddenly manifest but so far > no such thing has come to be.

Have you tried to buy exercise equipment lately? What about hiking backpacks? Have you been to Lowe's? Have you heard that there are protests happening across the country?

It seems to me like there _has_ been a massive jump in active leisure.

<comment><USER10> Seems more likely to me that the people that were previously going to the gym decided to buy their own equipment. There's also something to be said about the quantity of exercise equipment inventory available at any given time.

<comment><USER9> Possibly, which is why I included three other examples.

</comment></comment></comment></comment></comment></comment><comment><USER11> I chose to work in the public sector of Denmark because I'm ideologically inclined that way. That was decades ago, today I would chose to work here because of how much less hours I work compared to my old university friends in the private sector. They earn more money than me, but except for one of them who founded and now manages a successful company, it's not that much more money. In fact when you calculate in all the extra hours they work, my extra weak of paid vacation, my better paternityleave, the two extra paid child's sick leave days I get (per time your child is sick), the two yearly extra vacation days (per child under the age of 7) and my pension, I probably only earn around 10-20% less than them, and I get to spend so much more time not working. Hell I even bumped my hours to 30 a week for a while when my daughter was born. That was expensive, but it's something you aren't offered in a lot of private sector jobs.

I can't begin to imagine how you guys in America get through life working so much. Let alone how on Earth you raise a family, maintain a relationship and have time for yourselves while you do it.

<comment><USER12> I feel like you are incredibly privileged that you can afford to to have a family while working so little hours in the public sector and I suspect your situation is not the norm in DK.

Friends of mine who moved to SE/NL/DE/DK are definitely working the full 40h/week and sometimes more since, as they say, living there is very expsensive.

From my experience, the Europeans who toot their _" life is so good here, why do other people work so much?"_ horn are privileged enough to have a hand-me- down property bought/inherited from their (grand)parents, and since rent/mortgage is the biggest expense here, with property prices rising 8 times faster than wages, that property ownership early in life, which they take for granted, gives them the clear advantage that allows them to not bother with the rat race and could even live comfortably on minimum wage.

That of course, is not the norm for everyone and especially not if you've just emigrated here, so anyone wishing to own their own property one day has no choice but to jump in the meat grinder and join the rat race.

<comment><USER13> For what it's worth I live in Denmark, working a private sector full time (37/week, usually less) job with a nice salary - though nothing out of the ordinary for my education and experience. Came from a middle class family: My mom is a nurse and my father is an engineer. No life-changing inheritance from grandparents (~$10.000).

My wife stays at home with our 2 year old daughter.

We own an apartment in Copenhagen that we bought some 5 years ago, though we're obviously still paying a mortgage on it (0% loans definitely made that easier!). We're paying it down aggressively, though still have quite a bit of cash left to invest.

From my experience it's about priorities. We rarely go out to dinner and don't own a car - hopefully never will. Rarely buy new clothes.

While it might not be the norm, it's not something that's completely unheard of.

One major factor though is probably student loans. When my wife and I left the university we both had savings of roughly $15.000. A year later, we bought the apartment. That would have been impossible if we had debt we needed to overcome first. If your friends moved here with significant debt, they're definitely playing catch-up compared to their peers.

<comment><USER14> It sounds like you are in a great situation, by why would you aggressively pay down a 0% interest loan?

<comment><USER12> Nobody likes to be in debt, the sooner you can get rid of it, the better.

<comment><USER15> Hmm. But if you were investing that money even in some sort of risk free investment like time deposit, you'd be earning free money.

<comment><USER16> There is no such thing as risk free investment.

<comment><USER17> There definitely are investments that have a better risk/reward ratio than carrying a 0% interest debt on a mortgage.

e.g. savings accounts with 1% interest in the EU which are government insured up to 100k. Plus mortgage debt tends to have fiscal benefits in most countries, too. That's a better idea than paying off a 0% mortgage. Especially past a certain point where there's plenty of equity in the home.

There's generally virtually no financial reason to pay off a 0% debt apart from some edge cases here and there. People do it for peace of mind, which is of course fine, but it only works because of financial illiteracy as it's not the most financially optimal decision to make. (even not the least risky).

<comment><USER18> The problem with "financially optimal" strategies is that tend to ignore the fact that income can be lumpy. If everything is steady-state then it's "optimal" (as in better returns) to run highly leveraged. The problem is that life for individuals is not steady-state: you might lose your job, the economy might crater, the currency might get devalued, the property/stock market crash, etc. A 0% loan on a residence can be quite a problem if your income is 0 for some reason. Unlike a CEO who still gets his golden parachute if his highly leveraged, "financially optimal" company encounters "unforseeable" "turbulence" and ends up bankrupt, a bankruptcy in Me Myself and I, Inc. could be disastrous. Like, family on the streets disastrous. Individuals opting for peace of mind are minimizing downside risk, not maximizing "upside potential". Financiers can afford to take risks that end up with them blowing up, but when it's your life that blows up if some problem comes, minimizing the downside is a pretty maximizing strategy. The fact is, we know there we be periods of difficulty, we just don't know what the specific difficulty will be in advance.

<comment><USER17> I've got no clue how that's really meaningful in this case. I just mentioned there's a 1% interest government-insured savings account available, i.e. if the bank goes bankrupt, you still get your money. The only instance where you don't get your money is when the government goes bankrupt and doesn't honour its obligations. In that scenario, your mortgage debt would be even more screwed.

Meanwhile, getting 1% return on a 30 year mortgage puts you in a much safer, financially cushier position, than having had 0% returns for that period in time. Would you rather have 40k saved + 10k interest saved up in savings saved up in a government-insured savings account when you lose your job, or would you rather have a 40k lower mortgage debt and slightly lower payments or a lower time-to-payoff? Particularly given that mortgage payments can be furloughed and negotiated in times of payment difficulty. The answer is very clear. Choosing 0% returns over 1% returns in this case is the riskier choice.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> My European take on this: while upper class salaries might be 2-3x above middle-class salaries they don't afford you a lifestyle that would be unrecognizable to a middle-class family. Maybe a nicer neighborhood, maybe more long-distance traveling, a good restaurant from time to time. But you're certainly not removed from the idea that even a smaller loss in income has a direct impact on your lifestyle. So the upper class is as afraid of a loss in income as the middle class. And while middle class jobs are often union protected and have a stable market of supply and demand, upper class jobs are often highly specialized and a transition might not be as easy after having lost a job

<comment><USER19> (Also europe.. ireland)

I would confirm this take. I would also say that "upper class" in modern european terms, refers to something different then in other times... or possibly in the US. We have a more cultural take on class. It's not just about income.

First, our cutoff for "elites" is a lot lower. This is sometimes reflected in proposed laws. For example, when France implemented a wealth tax, they set the cuttoff around €1m. Other laws and proposed laws around europe tend to have a similar take. In US equivalents, thresholds are proposed aropund $50m-$1bn.

Here in Ireland, the top thresholds of income tax are near (even below) median full time income... €44k. Also, our public housing system houses around 20% of the population. The quality of housing is better than the lower end of private rentals... which tend to house employees of Google FB, etc.

Meanwhile, there's a lot more emphasis on cultural divide here. In the UK, the "anywhere vs somewhere's" is a popular take on modern class division. Anywhere's being internationally oriented, college educated people. Somewhere's being locally oriented. It corresponds to a more evergreen take: professional class vs working class.

In both cases, the income gaps are smaller than the cultural gaps. A college graduate working in Google account management for €35k can earn half what a plumber earns, but the class divide is still what it is. This is actually a significant dynamic in Ireland now. We have a lot of middle-income white collar jobs at international tech companies, with questionable long term security. We have a lot of higher paying blue collar jobs, mostly construction adjacent.

Often, the economic differences between classes just comes down to neighborhoods, property wealth and accumulated family wealth... not necessarily income. Often, but not always. The lowest 25% income families tend to be distinctly, and proudly working class. The genuinely wealthy tend to be upper class.

Class is complicated in europe. It can be quasi-ethnic, with accents, body language and cultural subtleties playing a big role. It doesn't correspond neatly to income.

<comment><USER20> What makes you think it "corresponds neatly to income" in the US?

There's economic class, social class, and socio-economic class. These are three different things, and that's also true in the US.

<comment><USER19> i don't

</comment></comment></comment></comment><comment><USER21> Here's an observation: employees, companies & countries that are winning tend to work harder. Those that are merely coasting or declining - eg in a situation where there is nothing to win with more effort - work less hard.

Elon Musk - "no one changes the world working 40 hr weeks" - is trying to win in a global competition. Your uncle Bob who is Director of QA at Oracle? Probably not playing to win.

Countries like Japan or SKorea or China are trying to win. US is trying to win. Denmark or Italy are no longer trying to win a world-wide competition.

No one has shown that individuals, companies or countries can wim while working less than the competition.

<comment><USER22> I get what you mean about trying to win on the level of people and even companies. Winning or being in the game to win is motivating and when you see an excellent payoff curve where each hour is delivering value, working longer does not feel so bad.

When you use this logic for larger entities though, it falls down. Very few countries are in that kind of rapid growth mode for long as an entire country. Indeed, you've grouped some wildly different countries together there.

South Korea and Japan had periods of extremely rapid and impressive growth where it really did make sense to say that the entire country had organised themselves and sacrificed in order to win. Their goal was to grow their children's standard of living and they succeeded. Is aging, inward looking Japan on 2020 still playing to win? Or have they won enough to enjoy the fruits of that victory?

China is a country that is still on the rise.

Is "the US" trying to win at a national level? I don't think so. I think that parts of the country are full of people who are trying to win but on a national level? I think the best we can hope for is that the humiliatingly inept set of responses over the last six months are a galvanising moment for Americans on the order of Meji era Japan.

I also think that you've given Uncle Bob quite a good job for someone who's not too bothered about winning. Oracle might have a reputation as not such a great place to work, but I rather suspect that making it to their director of QA takes quite a lot of winning along the way!

If you want a tech example of not fussed about winning, maybe someone doing SAP implementation or running internal business logic as a 9-5 is a better example.

Also it's a bit odd comparing Italy and Denmark since I'd be hard pressed to come up with two countries that are so different!

Denmark's labour productivity is the highest in the EU after Ireland and Luxembourg, both of which have distorted figures because of financial concentration of assets there by American tech companies. Italy's is not much more than half that of Denmark.

<comment><USER23> >> the last six months are a galvanizing moment for Americans on the order of Meji era Japan.

Or how about Sputnik? Or the Great Depression, Civil War, etc. I like to think that the US has changed course when a course change was required. It often takes a crisis to trigger it.

</comment></comment></comment><comment><USER24> It's also the fact that many modern, well-paid jobs are actually somewhat satisfying. Humans get off on solving problems, as long as they are allowed autonomy, mastery, and purpose.

The things I do at work are the things I would do also in my spare time, if I had more of it. I like solving puzzles. So if I'm going to do this stuff anyway, why not get paid for it?[1]

Of course, I don't want to do it for more than around 40 hours[2] a week (mainly because I have other things I also want to do, and there's only so many hours in total to do all the things in) but I do want to do it for a significant portion of my day. I need that intellectual stimulation to be happy.

But I can imagine some people want to go "beyond the average" and really show their employers their commitment. In a world where people happily volunteer 40 hours a week to satisfy their inner desires, there's little room to do anything other than offer 60 hours to prove yourself better.[3]

[1]: Well, I know why. If your employer is bad you won't find work as fulfilling. So maybe my hypothesis doesn't hold up anyway.

[2]: Now that I have a young child at home, I'm starting to think about asking for less than 40 hours. There have simply become more other interesting things for me to do.

[3]: Well, you could work more efficiently instead, which would be my preferred solution. But I think a lot of people are stuck in a rut of average skill but want to present themselves as beyond average. At that point you cannot do "better", you can only do "more". (Which might lead to better, come to think of it.)

</comment><comment><USER25> This has always bothered me. In the US there is no way I have seen beyond running your own business to negotiate the time spent working. When I started in the work force I thought taking a smaller salary for 4 day work week would be a reasonable tradeoff but everyone thought I was nuts. I have tried over the years asking for more vacation or shorter weeks for a pay cut but workplaces are just not equipped to handle those sorts of requests. The system here does not value free time in the same way it does money.

<comment><USER26> There are exceptions. For example my current employer, a decent-size public company, allows employees to purchase up to one week of extra vacation time each year. (Straight time—equivalent to unpaid time off, but with the cost spread out over the year in the form of reduced pre-tax salary and no change in other benefits.) I've known several coworkers who did the 4-day-week thing for months at a time rather than taking long vacations. They are also open to negotiating shorter hours or alternative schedules.

</comment></comment><comment><USER27> This is made worse by those insufferable over-zealous team members with a i-can-wag-my-tail-harder-than-you attitude. They send emails from holiday expressing their concern over a change, which frankly no one gives a damn about, and moan their semi-brags on how they are dying under work because how random senior managers (who don't give a crap about them and are just playing them to get more work done) have "insisted on my involvement". These are the kind of rotten apples who don't have a life and drag those that do have/want one to the bottomless pit of overwork. /rant

<comment><USER28> As an alternative perspective, they'd make the exact same rant about how you're the rotten apple.

<comment><USER29> Toxic people will use whatever excuse is necessary to justify their behaviors.

If they want to wage slave, slave away. Leave the rest of us out of the theater production. Unless your job involves life safety, it doesn't matter all that much.

<comment><USER28> Again, they would also call you toxic for the inverse reasons.

Ie "if they don't want to work hard and build a career, so be it. Don't drag down the rest of us" etc.

Two sides to the story is what I'm getting at.

<comment><USER29> Not every side is valid, or carries the same validity, is my point.

</comment></comment></comment></comment></comment><comment><USER30> Recently I came across this video "The design for easy life": [https://www.youtube.com/watch?v=b2eoQyYoUww](https://www.youtube.com/watch?v=b2eoQyYoUww)

I live in Thailand and am very interested in working less for other people and more for myself. At first I though I'd need to invest maybe 500.000 USD and then perhaps I should be able to live on investments. But this video changed my mind, perhaps I should be able to get by with a lot less.

This Thai farmer claims to work about 1 hour a day and that 1 hour of daily work generates for him around 50 USD. More than enough to take care of his family.

Another interesting video from this farmer on "Sufficiency": [https://www.youtube.com/watch?v=LWwh5N0JRhk](https://www.youtube.com/watch?v=LWwh5N0JRhk)

<comment><USER31> I love that man. Brilliant mind. I have lived a Digital Nomad lifestyle for 2 years and when I was in homestays on the countryside, I always felt relaxed and blessed. Just some chickens roaming around, some fruit you could pick. Simple life.

Big polluted cities like Bangkok and Penang drove me nuts.

One owner of one of those simple homestay farms quit his farming activities and became a taxi driver, on Lombok (Indonesia). Lombok just got a brand-new airport and they expect more tourists coming in from Bali eventually. The man became a taxi driver to make more money for "a better life".

I get it. Everybody wants a shot to live "the dream" and you gotta start somewhere, but --> you don't know what you have til it's gone.

</comment></comment><comment><USER32> If you are on salary and you are putting in more than 40 hours a week, you are working to buy your boss his vacation home, not your own.

</comment><comment><USER7> The core idea of the article can be found in a paragraph somewhere in the middle of the piece

> What all of these explanations have in common is the idea that the answer > comes from examining workers' decisions and incentives. There's something > missing: the question of whether the American system, by its nature, resists > the possibility of too much leisure, even if that's what people actually > want, and even if they have the means to achieve it. In other words, the > long hours may be neither the product of what we really want nor the > oppression of workers by the ruling class, the old Marxist theory. They may > be the byproduct of systems and institutions that have taken on lives of > their own and serve no one's interests. That can happen if some industries > have simply become giant make-work projects that trap everyone within them.

Now. I strongly believe that software / IT industry is one of those make-work projects.

<comment><USER33> In America, too many benefits are tied to full time employment. The biggest one is health insurance. This creates a huge imbalance of power between employer and employee. The absolute worst time to lose (affordable) health insurance is when you become unemployed. It makes no sense! This doesn't even go into people or families with chronic illness.

In this way, in order to maintain these benefits, Americans must do what they feel whatever it takes to keep their jobs. That's the number 1 incentive.

This is _not_ the product of an institution that has taken on a life of its own. Universal healthcare as a majority of support among Americans (as long as you don't attach Obama's name to it, anyway), but neither party (especially not Republicans) have shown an interest in delivering this to its constituents.

</comment></comment><comment><USER34> In service sectors of the economy, work can be created out of thin air, without any proportional returns in the form of increased well-being of society.

For example, you may spend days improving every little detail of that customer email template, for marginal or no returns over the previous template.

</comment><comment><USER35> I'm from Germany and none of my friends work full-time anymore.

50% - 80% is the norm.

<comment><USER36> What is your approximate age group for your friends and are all your friends in the same industry? Just curious about what things are like.

<comment><USER35> 30-40

All different industries.

Healthcare, IT, agriculture, retail, magazines and books.

</comment></comment></comment><comment><USER37> I'd love to work 4 days a week. It just doesnt seem like it's possible. But having Wednesday's off and being able to actually take a week or more of vacation a few times a year as a manager would make my life so much better.

</comment><comment><USER38> It's a little disappointing they don't address the idea of peer competition.

We live in a time of insane material wealth, so we create artificial scarcities of necessities to ensure peers compete with each other.

The article does mention the substantial rises in productivity we've had, but this: "There's something missing: the question of whether the American system, by its nature, resists the possibility of too much leisure, even if that's what people actually want" merited far more exploration.

There's a horrible positive feedback loop going on.

Say 10 people want a home in an area that has, oh, 8 homes.

If somebody chooses to work long hours, read emails on weekends, etc. they have a better shot at getting promotions and being able to outbid their peers for those homes. If someone doesn't do this, they have a better chance at falling in to the 2 people who can't afford a home.

So the workaholics work and work and work, get the raise, and buy a home with a big mortgage attached. Maybe a $1.5 million dump in San Jose. Now they're a highly-leveraged housing speculator who desperately can't afford for house prices go down. And the leverage they used to buy the house is secured with the threat of homelessness and personal ruin.

So they oppose any and all new building ferociously and cruelly. Scarcity is the only reason their house even has value!

Similarly, _they_ had to work nights and weekends, so why shouldn't other people? As they rise up the ladder this is what they select for.

So we have house prices ratcheting upwards, people working themselves to the bone to get in to the club of "homeowners" (a dubious word when you owe 90% of the home's value to the bank), and a strong incentive to make sure it keeps going that way.

It's true that presenteeism and wasted time at work are an issue, but ultimately there is a point where more time working means producing more (say 20 hours vs 10), and in a sector where results are hard to quantify people go for the easy thing to measure - hours where you butt was in the seat.

I mean, houses are pretty cheap to build. Habitat for Humanity does it for $50k. Imagine if you could work part time and build your own home? It would threaten people whose power comes from housing scarcity! Why, what if it turns out there's no inherent reason their house should cost 25 times the median income? Can't be having that! Better make it illegal to build homes (or demolish existing low-density ones) anywhere near a job.

The only homes that get built near high paying jobs now are over the wailing shrieks of existing rent-seekers (aka homeowners). The person who owns a home in Berkeley can vote, because they live in the area, but the person who would _like_ to live in Berkeley but commutes from Stockton has no voice. I'm hopeful that remote work will finally destroy their power, though with a decrease in tech incomes as well.

<comment><USER39> Housing scarcity is more about the cost of the land than the house itself. You're right that that a house can be built for $50k (not by most modern standards, but it can be done). But if you wanted that $50k house in SF bay area, suddenly it's on a piece of land worth $2 million.

<comment><USER40> Yeah, but these days you can build cheap, decent quality apartment buildings with 10-12-20 floors. Suddenly that piece of land can afford much higher population density.

</comment></comment></comment><comment><USER41> If you look at the employment rate, you will see that only about 65% of working age people have a job. Any job.

[https://tradingeconomics.com/united-states/employment- rate](https://tradingeconomics.com/united-states/employment-rate)

Add in the huge number of OAPs and most developed nations have sub 50% of people working. A lot of the issues we have (from stress and mental health to inequality to the rise of extremism) can be linked to the fact that we have fewer and fewer people working harder and harder supporting more and more others.

We either need to find some ways to reverse this trend OR we need to embrace it. But if we are embracing it, we need to deal with the consequences (everything from access to healthcare to burnout to how we decide who has to work and how we compensate\reward them for doing so).

Right now, this is a massive social change that goes basically ignored. Whether people like Yang and Musk are right about automation and the rise of AI or not, change is upon us and we cannot keep ignoring it.

</comment><comment><USER42> I don't think it's a quantity problem, it's an existential problem.

 _Why_ are we doing what we're doing?

When the best answer anyone can come up with is 'so that my kids get to have a good life', it's not enough.

It's average people living in a system that's designed to make them replaceable and eventually get rid of needing them completely.

Not get rid of them so that they can enjoy the human achievement of no longer needing to work, no, get rid of them so that somebody rich can then hire you as a wage slave to do whatever pleases them that day.

I wonder if this is just inevitable - it would explain the existence of the pyramids - why else would anyone build them, but because there were so many idle hands and a few bored ego-maniacs? Unless of course, aliens :)

</comment><comment><USER43> In China, "996"(9am-9pm, 6 days a week)[0] is popular in tech companies, now more and more compnanies are adopting this work schedule, so employees have much fewer choices if they want a "normal" schedule. The administration did not enforece labour laws well in order to pursue economic interests. What makes it worse is that, as this work schedule becomes a trending, some companies and employees take it as granted.

[0][https://996.icu/#/en_US](https://996.icu/#/en_US)

</comment><comment><USER44> I assume working less while still making the same annually would lower suicide rate. One of the main reasons people commit suicide is financial problems. Added free time is never unwanted and can be substituted for whatever the person desires. I personally think we're burning out the younger generation with all the optimizations that have came with technology but at the same time continuing to expect more from people. I think we're missing out on what benefits come to society when people are less overworked.

</comment><comment><USER45> Well this didn't age well. We have now used the pandemic to force work into our homes, if anyone was even pretending it didn't follow them before all of this. That is of course only true for those who can still find work. Of course, the dynamic of job scarcity creates an even worse version of stockholm syndrome for those 'lucky' enough to continue to draw paychecks.

</comment><comment><USER46> And there is also this related article that was discussed a lot here on HN last year: [https://www.theatlantic.com/ideas/archive/2019/02/religion-w...](https://www.theatlantic.com/ideas/archive/2019/02/religion- workism-making-americans-miserable/583441/)

</comment><comment><USER47> I just refused a full-time offer because of this. Instead, I prefer to work few hours a day for my current contract and hope that things get better post- covid. Still, if anyone is looking for a Ruby contractor to give a hand in some project, reach out :)

<comment><USER48> Are you a freelancer?

In a couple of years I would like to start gradually scaling down the number of hours I work. But right now I am a salaried employee and I have no freelance experience, so I'm not sure whether that is the right avenue for me.

Any thoughts are appreciated.

<comment><USER47> My single advice would be to find a client (or clients) beforehand and start doing work for them before you drop your job.

</comment></comment></comment><comment><USER49> Maybe it's just me, but the idea of retiring early and then travel the world is really strange to me. I'd rather spend my life seeking and working on the next ideal job, and the only time that I'll skip it is when I believe it's impossible to get the ideal job. For now, if you give me a BI/DE job I'd be happily to work at least 60 hours and keep improving my shits. In short, I'd rather die working.

But again I'm not against FIRE, because most of the time people don't get to do their ideal job (or even the top 5 ideal jobs), and in their shoes I'd rather just grab enough $$ and retire early. The point is, sadly, in modern society, many people are not really getting the chance to contribute to the society in a way both he/she and the society are satisfied with. This is very sad. The idea of Communism, at least part of it, is to make sure that each person out there gets the chance to fulfill their dreams and do their best, and this is really attracting.

</comment><comment><USER50> The article starts with the assumption that the total amount of work in an economy is fixed and that the extra hours we make are make-work. I beg to differ. At least in high tech, we can improve a product or service as many times as we want. Each of these improvements can be from relatively minor to requiring years of research. The total work per year is how fast we move through these improvements. The free market forces this to be higher every year due to competition.

</comment><comment><USER51> Lmfao. How naive can people be?

Do you honestly think capitalists don't know that when people have free time they start thinking about life, and when people start thinking about life, bad shit happens to capitalists?

<comment><USER52> I was trying to find a rational and intellectual way of explaining this very stark reality that I've consistently run across but... for now I'll just upvote this.</comment></comment>


We can chip away at rape culture by teaching girls emotional self-defense - <AUTHOR> https://work.qz.com/1299969/empowerment-self-defense-classes-teach-girls-how-to-set-boundaries-and-say-no/ <comment><USER2> I'm all for teaching people to defend themselves, but why is the onus for fixing the problem put on the potential victim rather than on the perpetrator, or on the culture as a whole?</comment>


CHT Presents Humane: A New Agenda for Tech - <AUTHOR> https://vimeo.com/331897593 <comment><USER2> Truly a super cool video.

If you ever tried to think of how to explain a criticism of social media's harms, etc. And then thick of solutions - this video is for you.

Start at like 21:00 for "kids and social media" and like 24:20 for Tristan Harris's talk to start.</comment>