An inside look at Google's data-driven job interview process - <AUTHOR> http://www.washingtonpost.com/business/capitalbusiness/an-inside-look-at-googles-data-driven-job-interview-process/2013/09/03/648ea8b2-14bd-11e3-880b-7503237cc69d_story.html?wprss=rss_social-postbusinessonly&Post+generic=%3Ftid%3Dsm_twitter_washingtonpost <comment><USER3> I'd never be good enough to be a Googler. It's because of 2 things:

1) I graduated from college long time ago, and have worked in a BigCorp for 6+ years, where the toughest technical problem all center around CRUD, nothing that requires any deep algorithmic, machine learning, or NLP techniques (ala building a search engine).

2) During my free time, I hack on my startup, and although the problems are more challenging, I tend not to spend too much time trying to optimize things, or come up with the most efficient, elegant solutions because of time constraints. I just ask myself: is this good enough and move on. Because of this, I develop bad habits that would never be allowed in Google.

<comment><USER4> I'd much rather have someone on my team who knows how to get shit done. optimizing in a vacuum is easy.

<comment><USER5> When you're trying to build services for hundreds of millions of people, optimizing gets pretty key, and it's often the harder part than building a MVP or "getting shit done". Ask Twitter.

<comment><USER3> That's true. It's often the hardest part if your service isn't really anything technically groundbreaking (ie Twitter, Facebook as opposed to Google)

</comment></comment></comment></comment><comment><USER6> Whenever you see a selection system that is very specific and continues to build specific criteria - this happens in corporate policies, admissions, interviews, and also seen often in evolution - your suspicions should constantly be raised. The more specificity, the better systems get at optimizing for a few local maxima while potentially ignoring other (or most) maxima entirely.

For example we have a fairly "locally optimized" single column spine for something that stands upright, but if we were to design something that stands upright (such as a building) we would rarely use a single column.

Anyway, I wonder what Google does to mitigate such a threat from their specific systems described in the article.

Further I wonder if they collect any data that might tell them whether their process turns people away, because I've heard lots of horror stories (especially on HN) about the Google interview process. The article doesn't seem to mention whether or not the interviewees enjoy the process. I suspect nobody (at Washington Post or Google) bothered to ask them.

(Not that I can blame Google, I've never heard of a company asking recently hired employees about how they could make the interview process better. But I figured the WaPo would have interest in writing that story)

The article ends with:

> To make sure they don't miss out on top talent, Google employs a team of > full-time screeners to sift through applications. The company would not say > how many people are employed in these roles, but it said the group is > "sizeable."

I suspect at the end of the day their screeners and recruiters are about as "average" as other companies of their size. Numbers certainly wouldn't make up for quality. Just my anecdote, but here's the first part of the email a Google recruiter sent me:

> I'm a talent scout on the Engineering Staffing Team at Google. I came across > your details and feel that you could be the sort of person we are looking > for to work in a new role we're hiring for in our Mountain View HQ, called > 'Performance Engineer' for which we're looking for a candidate with a > combination of compiler, high performance software design and computer > architecture experience.

I'm not known for much, but for what I am it's exclusively _JavaScript._

<comment><USER7> Google does have another outlet, which is acquisitions. When a company is acquired, generally, a big chunk of the employees get hired without the normal interview process. Some of them are essentially on probation, and if they perform well, they are made permanent. Others end up not getting converted.

Google also hires contractors and interns, and some of those get 'converted' to full time employees, essentially with the contracting/intern period as one big job interview.

<comment><USER8> Whether the employees get interviewed at Google or not depends on the size of the acquisition, at least for engineering positions. That's largely for logistical reasons. The vast majority of acquisitions of software engineers involve interviewing.

</comment></comment></comment><comment><USER9> I laughed at this:

      > The firm recently generated buzz in the talent industry
      > when it said it had done away with the notorious brain
      > teaser component of its interviews after statistics
      > showed the ability to ace them had no correlation with
      > success at the company.

Brain teasers have (to my knowledge) never been used in Google interviews. They're an urban legend which used to be said about Microsoft, and will no doubt be said about whatever big companies come next.

Journalists: "$COMPANY gives all its new employees wedgies!"

$COMPANY: "We do not think giving employees wedgies is useful."

Journalists: "$COMPANY generates buzz by doing away with wedgies!"

<comment><USER10> It's been a long time since my interview (2005), but I had a couple of brain teasers during my my interview. Not as many as I had at Microsoft, but definitely some. I specifically remember being asked the Two Eggs question ([http://www.datagenetics.com/blog/july22012/](http://www.datagenetics.com/blog/july22012/)).

<comment><USER9> The two eggs question seems fair to ask an engineer. It's an optimization problem, and could just as easily be rephrased in real world terms (e.g. branching in a hard-realtime microcontroller). The solution can be arrived at through math and reasoning.

Typically, trick questions are things like "how many golf balls fit in a bus", "you've been shrunk and dropped in a blender" , or the infamous "why are manhole covers round". These supposedly test the candidate's creativity, but actually just test whether the candidate has read a book of riddles recently.

<comment><USER11> I disagree with your categorization of "how many golf balls fit in a bus" as a trick question. That's a normal estimation problem that's highly relevant to a software engineer's daily work. (Rephrase the question as "how many users can you serve with one CPU core". Capacity planning is an entire department's worth of work.)

<comment><USER12> Please tell me you guys do capacity planning with more than just vague numbers you pull out of your ass, a whiteboard, and some guy staring at you making subjective judgments of the number you come up with.

<comment><USER5> Yes, but we don't expect someone to use actual numbers in response to a hypothetical question in an interview. The question is intended to see if you could figure out _which_ numbers you'd really need to figure out the right estimate.

Often, the specific result of an estimate is less interesting than the user's understanding of what drives that estimate, because it determines if they understand the levers they can pull to change the outcome in the real world.

<comment><USER12> Sure, if you ask the actual question about capacity planning. The golf balls and bus question gets you where exactly?

<comment><USER11> Abstraction. I hear computer programming is pretty abstract.

<comment><USER12> Well that was a little condescending. If you think it's vitally important to screen people for their ability to solve Fermi problems, it's no skin off my back. I was just trying to have a conversation.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> _On average, Google takes about 45 days to hire._

This struck me as exceedingly slow (I know that my friend got an offer in 10 days start to finish, but he's probably an anomaly since he had multiple offers already).

Is this in line with other software companies this size? I also wonder if this varies with function.

 _That fifth person is a "shadow interviewer" who is simply training to conduct interviews for future job seekers, and that person's analysis isn't included in the decision-making process._

This is hilarious! It's just like the ETS (education testing services) that runs all the US based standardized tests.

<comment><USER12> 45 days seems exceedingly slow to me as well, based on my experience with another large tech company. Though I can't say for sure the end-to-end time of the whole process, we can generally get from an onsite interview loop to a hiring decision within 24 hours, and HR takes another day or two to generate the first verbal offer. So that's what, three days? As I recall it didn't take anywhere near 42 days to go from resume submission to onsite interview, either.

How on earth does any company take 45 days!?

<comment><USER5> It's not really slow if the "time to hire" is the time from an inbound resume, to an outbound written offer (or even longer, an accepted offer). (A verbal offer is great, but nobody generally accepts a verbal offer, they have to sign some kind of agreement to formally accept the job.)

The screening process takes a week or two (14 days) and then you have to schedule an on-site interview loop, which can often take a week or two lead time (since many interviewers have to travel to SF, and you have to schedule the interviewers as well) and that only leaves a week or two at best to issue a formal written offer. I've seen it take a lot longer than in total at many good companies.

<comment><USER12> > (A verbal offer is great, but nobody generally accepts a verbal offer, they > have to sign some kind of agreement to formally accept the job.)

Well, you'd want to call someone on the phone and get a verbal OK before sending them the offer letter, right? From verbal to written shouldn't be more than a 24 hour turnaround.

So that's the day of the interview, the day after the interview to make the decision, and three more days (so the rest of the week) to generate and send out a written offer that the candidate may or may not have already verbally okayed. That's 5 business days, or 5-7 calendar days.

> The screening process takes a week or two (14 days)

14 days to look at resumes and do a phone screen or two? Maybe at maximum, if you have to bounce the resume between teams to find the best fit.

> and then you have to schedule an on-site interview loop, which can often > take a week or two lead time (since many interviewers have to travel to SF, > and you have to schedule the interviewers as well)

Why would the interviewers have to travel? Don't you interview people on the same campus where most of the interviewers actually work? Why would it take _two weeks_ to book a couple of hours on k different people's calendars (hopefully most of them selected from a pool of size n >> k)? Sorry, I think that timeline is way too padded.

<comment><USER5> No, it can be a long time between a verbal and a written, because the verbal offer doesn't contain all the compensation details and requires final sign off from whoever makes those decisions. In almost all the corporate jobs I've gotten, I got a verbal offer relatively quickly, but the formal, _signed_ offer letter came a long time (anywhere from a week to a month) later because it required the signature of VP or someone similar.

Re travel: I meant that the candidate has to travel, and the interviews have to be scheduled with the pool of interviewers. The minimum time for this is generally a week, but usually it takes longer. Interviews are a significant / immovable commitment and interviewers are required to give notice if they can't make it several days ahead, which means they have to be scheduled well before that.

</comment></comment></comment></comment></comment><comment><USER14> I very much enjoyed the opportunity to interview at Google and to see some of the inside of the campus, but articles like this seem disingenuous to me.

My first interview on-site was delayed due to room reservation issues, so it started late, and this continued as a theme throughout the day.

That's anecdata, but I have to wonder whether whatever schema and/or processing pipeline are allegedly in place to remove human bias would be able to understand issues like this or of other kinds - i.e. environmental and/or human failures outside the scope of the process. These systems don't tend to critique and monitor the environment, just the data that is entered into them afterwards.

This plus the admission from a few Googlers that personal recommendations do really make a big difference make me a little jaded when reading articles like this - I'd much prefer to see admissions of honest weaknesses alongside all the positives, but I guess we're still some way from treating anything except severe security breaches that way in the software/marketing industries (this article is in the latter industry).

Edit: I feel like I should add a bit of context about why I feel the article is misleading, given that my post was inspired by personal experience.

The misleading aspect is that the article tends to portray the process as striving towards an unbiased science, whereas my perception is that bias is still part of the decision-making process (arguably for good -- personal recommendations can be very positive indicators, as long as they're not from an old-boys-style network), _and_ I feel that there is insufficient measurement and understanding of interview factors to make it a science (i.e. exhaustion/travel factors, cultural differences, personal factors, etc - <USER2> I don't think affected me, but are still a real part of interviewing).

NB: When I say old-boys network, I imply any kind of non-meritocracy which simply aims to get people 'in the door' without full vetting; I believe this is possible regardless of gender, but just that's the term I know to describe it

</comment><comment><USER15> Out of Google, Amazon, Microsoft and Facebook, Google was the only company that straight up declined to interview me because I did not meet GPA requirements (3.0 cumulative - I had a 3.4 at the time in CS but I hadn't done well in freshman chem or calc 3)

I like that they decided to stop asking brain teasers due to a lack of correlation between performance in them and performance once hired. Do they really think that cumulative GPA has a strong correlation on new-hire performance?

I certainly don't.

(I expect to get some push-back from you guys and I'm interested in the discussion to follow :))

<comment><USER11> Was this as an intern or full-time employee? If two classes pulled you down from 3.4 to 3.0, I have to wonder about the total number of credits.

Also, what was the exact wording of Google "straight up" declining to interview you?

</comment></comment><comment><USER16> For another look at data-driven recruiting, with actual numbers instead of fluff, you can check out [http://blog.alinelerner.com/lessons-from-a-years- worth-of-hi...](http://blog.alinelerner.com/lessons-from-a-years-worth-of- hiring-data/)

</comment><comment><USER17> Google does the same kind of software interviews as anywhere else. They aren't special. You code on a whiteboard and it's supposed to be compilable in C or Java. The process beyond that is just as subjective as anywhere else and is largely based on gut. They depend largely on employee references/friends.

The data they collect is really only to make the process more efficient, not more effective. They reward employees that process the most phone screens in a month. The strangest thing, in my opinion, is that the interviewers usually don't make a decision at all, they just give a rating. Then, a group of people who've never even met the candidate decide whether to hire them based on forms that were filled out.

<comment><USER18>
        | Google does the same kind of software interviews as anywhere else. [...]
        | You code on a whiteboard and it's supposed to be compilable in C or Java.

It may seem like a strange idea, but not all places interview like this. The company I currently work for doesn't do this and the one I will start working for soon doesn't interview like this.

The reasoning is that my job is not to stand in front of a white board and write syntactically correct code without the aid of an editor or compiler, so maybe there is a better way to screen candidates that directly test the skills they will use on the job.

<comment><USER19> There's a difference between syntactically-correct code (meaning every single paren/brace is balanced, no dropped semicolons, should compile exactly as written, etc.) and what I call "valid" code. Expecting a candidate to write the former without an editor and compiler is silly.

A good interviewer (and good hiring committee) wants to see that you can write "correct-ish" code: it looks like it would compile modulo a typo or niggling detail, but the algorithm, data structures, and control flow are clear and valid. A good interviewer will also tell you that up front, e.g. "I'm interested in your code, not your syntax". If they don't, ASK!

The problem is: not everyone is a good interviewer, and it's surprisingly hard to teach someone to BE a good interviewer.

<comment><USER18> I think I agree with all of that. Especially the "interviewing is hard" part. But what's wrong with having someone sit in front of a computer to write some code?

At the company I'm leaving, we ask candidates to do a small project as a first pass. If the code isn't awful, we call them in to pair program with us (we're a pairing shop) on the code they wrote to improve it.

The company I'm joining just has you come in and pair (they're also a pairing shop) on projects their team is actually working on for most of a day.

I find both to be pretty good approaches. Better than having someone write code in an unfamiliar setting (the white board) to solve problems which are often, but not always contrived (write quick sort, etc.), at least.

We've had pretty good success with identifying good candidates since we switched over to this model and the place where I'm starting is a consultancy which is pretty well-regarded in the startup community, including HN, so they've probably had reasonably good success identifying talent.

<comment><USER19> How big are these companies, how many people do they hire per week, and how many do they interview? Any number of perfectly reasonable interviewing strategies in smaller companies fall apart at big-company scale (1000+ resume submissions per DAY)

<comment><USER18> Fair point. The companies are small and medium-sized. I'm not convinced that those models can't scale, though. One large-ish company that I know of who still does something similar is ThoughtWorks. They have on the order of thousands of employees and their interview process consists of, among other things, a take-home code submission and on-site pair programming.

If you're getting 1000 resumes per day, I think you would be able to weed out a significant number of them just by giving them a coding assignment to work on. Churning out a resume is easy, but sitting down for a few hours and writing well-designed code takes effort.

<comment><USER19> They probably can scale up to a point. Thoughtworks has 2100 employees. Google has 45,000.

How would you grade/score ~1000 code submissions per day, though? You could conceivably do Coursera/Topcoder-type automated grading, but that can only get you so far and can't distinguish good code vs. bad code vs. "copied from Glassdoor" code. It might be useful as an initial filter, though. You'd have to constantly be implementing new questions w/ associated grading scripts, though, as the problems would inevitably leak.

<comment><USER18> Grading 1000 code submissions per day would be challenging. What is trying to suggest, though, was that many of those 1000 would not submit code.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> 'On average, Google takes about 45 days to hire.'

That's a ridiculously long time to keep a candidate waiting.

<comment><USER21> It does sound long, but maybe it's the time from first contact to decision. You schedule an interview a few weeks in advance, for example.

<comment><USER22> It's about 35 days too long from first contact, in my opinion, outside any legitimate logistical concerns (e.g. traveling from out of state).

</comment></comment></comment><comment><USER23> The 'hiring religion' of Paul English at Kayak is an interesting contrast to the Google process, especially with regard to English's "7 day rule":

[http://paulenglish.com/hiring.html](http://paulenglish.com/hiring.html)

</comment><comment><USER24> I've been in the recruitment pipeline at Google 3 times over the last ten years. Each of the latter two times they had no record of the previous recruitment. In the one case where I got to onsite interviews the first set of interviewers had somebody else's resume with my name on the top. It took me most of the morning (because its an all day process) to convince them that I was not actually the piece of paper they were waving at me.

</comment><comment><USER25> I'm curious about what is meant by the elimination of "brain teasers." Could it be that the brain teaser has just shifted to code questions?

Part of the challenge of a technical interview is to get at someone's coding ability without resorting to what are essentially brain teasers disguised as computer science questions - and I'd expect a lot of disagreement around where you draw that line.

Here are a few I've been asked:

\- Print out the fibonacci sequence recursively.

\- Print all permutations of a string (using recursion).

\- Swap two integers without creating a third integer.

\- Out of several million database entries, select a few at random to display to the user. Don't repeat any until they've all been displayed.

\- Implement mergesort, code a singleton, print a binary tree in order, add a branch, find a cycle in a linked list...

Which of these would you consider brain teasers, if any? I'd say the "swap two integers" is the closest... but if you're including a lot of these questions, are you still in brain teaser land?

I'd love to see some google data on what types of technical questions correlate with job performance, rather than simple "brain teasers".

</comment><comment><USER26> It's interesting that they mention speed when they're notorious for taking months to get back to their candidates. 45 days doesn't seem like a number to be all that proud of! I suppose the sheer volume of applications makes things harder, but surely companies like Microsoft and Facebook get lots of applicants too.

<comment><USER27> In my last interview, they took over 3 months to get back to me. In the end, I emailed them.. seems like my HR manager completely forgot about me. Nice.

</comment></comment><comment><USER27> It just sounds complicated for the sake of being complicated. I can't see how being impartial is really helpful here - surely they want people to form relationships and figure out whether someone can actually work on a specific project or problem?

<comment><USER9> Impartiality is very important when hiring skilled workers. If your hiring process is not impartial, then it's introducing undesirable bias into the hiring decision.

The classic example is sexism in orchestras. Orchestra interviews used to consist of the candidate sitting on a stage and performing their piece for a panel of judges. The gender ratio of performers was terribly skewed, even worse than the software development field is today. Orchestra managers excused this by saying that women were simply not as good -- after all, the judges scores don't lie!

But a funny thing happens if you put the performers behind a screen. Suddenly, all the factors of their gender and race and grooming go away, and new performers start being a lot more diverse. There actually was a bias, a serious one, and it caused orchestras to lose who knows how many excellent candidates.

The current state of the art for programming interviews at big companies is to have the candidate solve actual programming problems, including whiteboard coding. It's not practical to put the candidate behind a screen or modulate their voice, so splitting up the tasks of "interview candidate" and "review interview feedback" is the best that Google has figured out.

<comment><USER28> You must realize that your orchestra example doesn't apply here. What they did is clean-cut and simple removal of the gender bias. Provable.

Compare this with what google is doing. It would be the equivalent of having orchestra candidates write an "original" tune on paper in 20 minutes and then talk about select topics in music theory and then submit that to a figure skating judge panel.

</comment></comment></comment><comment><USER29> Haven't RTFA, but I hope this is a joke. For a company that boasts making all of it's decisions with solid scientific data, it's hiring process is an emotional ass grabbing parody of twelfth night.</comment>


Antitrust+? - <AUTHOR> http://parislemon.com/post/15627530949/antitrust <comment><USER2> Google's counterargument that Siegler rejects as "hollow" seems pretty valid to me. If these other companies choose not to give Google the data it would need to give them the same treatment, how can you say Google is dealing unfairly here? If Google offers them the same opportunity but they think the business upside of getting listed is less than the downside of sharing their data, then that's their call.

<comment><USER3> I agree that it seems valid. I also think that Google would most definitely use that data if they had it. Google used to show live twitter results in their SERPs; but stopped when Twitter took that data away if I remember correctly.

</comment></comment><comment><USER4> I reject the premise, namely, that Google is a monopoly supplier of Internet search. Bing, Yahoo! and DuckDuckGo are perfectly suitable search engines. I don't think any of them are necessarily as good as Google, but a) I use Bing all the time when I'm on IE9 and I hardly notice and b) that's a poor criterion for monopoly. As a bonus, the switching costs for changing your search provider are approximately zero.

<comment><USER2> Monopoly doesn't mean nothing else exists. At the time when Microsoft was abusing its monopoly with Windows in the '90s, _many_ other operating systems existed that people used, like Mac OS, NeXTStep, Linux, BSD, OS/2, BeOS, etc. It's just that Microsoft had enough users that they could parlay their OS monopoly into an everything-else monopoly just by cutting competitors out. Google seems to be in a similar position as far as market penetration goes (to counter your anecdote, I actually know people who visit Yahoo by typing "www.yahoo.com" into the Google search box), so the question here is basically whether Google is doing bad things with its hold over the market.

<comment><USER5> Agreed. However, like mfb mentioned: _> As a bonus, the switching costs for changing your search provider are approximately zero._ This means that if you don't like the services that google offers, you can simply choose not to use google, without _any_ overhead in time or money. In the case of an OS, it is much more difficult, first to install on your machine, get used to the UI etcetc. Microsoft could take advantage of this fact to force whatever software it wanted, and users essentially had no option but to comply.

<comment><USER2> That's true as well, but the accusation here is anticompetitive behavior — they're not complaining that Google is dominant in search, but that Google is taking unfair advantage of its dominance — so the fact that users can switch easily is really neither here nor there.

</comment></comment></comment></comment><comment><USER6> I never knew he was an expert on antitrust law!

At least it's expected from the likes of him, it's less excepted and much more disappointing when actual reporters writing for supposedly credible newspapers start speculating about the law or relying exclusively on quotes from "experts" which are paid by one interested party or the other.

IANAL either but I asked one, and she thinks that any antitrust issues with Google results are little more than the product of negative PR and don't have legal merit.

As for Search+ itself I think that G+ is just a foothold, this feature will become more useful and interesting when data from one's calendar, email, voice messages, saved maps, etc be included in the mix.</comment>


The Collison Brothers and Story Behind The Founding Of Stripe - <AUTHOR> http://startupgrind.com/2012/02/true-startup-story-the-collison-brothers-and-stripe-com/

<comment><USER2> From my Irish perspective, I'd be more interested to learn how they came from Limerick, went to two prestigious American universities, and ended up in Silicon Valley. I heard for example that they initially approached banks in Ireland with their idea, but could not get funding. I'd be fascinated to read about this if anyone has a link to share (this article really skims over the early details).

I'd also love to see the lads bring their service back to the EU at some stage, it's badly needed!

<comment><USER2> I found it myself, this gives a lot more detail: [http://www.siliconrepublic.com/start- ups/item/22016-exclusiv...](http://www.siliconrepublic.com/start- ups/item/22016-exclusive-interview-with-a/)

</comment></comment><comment><USER3> Does anybody else read these stories and think "crap... I'm 23 and I haven't received any awards, nor did I go to MIT..."?

I know it's not requisite in the startup world to go to a top school, but every time I read a story like this I have to mentally justify my education and qualifications. I feel I'm a decent hacker who can produce, but I feel my background pales in comparison to guys like these.

<comment><USER4> It's easy to feel that way, but everybody does (including people at MIT). You just have to do your best not to compare yourself to others and do what you can do well and to the fullest.

<comment><USER5> Don't compare yourself to a strawman stuffed with everyone else's achievements.

Everyone looks bad compared to the aggregate. Individual differences are more manageable.

Stripe's success has more to do with hard work than genius.

</comment></comment></comment><comment><USER6> I'm surprised it doesn't mention their previous startup Auctomatic which they sold for a few million when John was 17 and Patrick was 19.

<comment><USER7> "Patrick founded Auctomatic and joined Y Combinator as a Winter 2007 company. Its path would not last long. At the age of 19 and ten months after incorporating, the company was bought by Live Current Media for $5MM where Patrick became the Director of Product Engineering in 2008"</comment></comment>


Russian Scientists Discover Bacteria That Neutralizes Nuclear Waste - <AUTHOR> https://sputniknews.com/science/201810081068701682-nuclear-waste-neutralizing-bacteria/ <comment><USER2> Yogurt is a food that contains bacteria, so maybe these bacteria could later be harvested and used to make food.</comment>


Anatomy of a hack: 6 separate bugs needed to bring down Google browser - <AUTHOR> http://arstechnica.com/security/2012/05/anatomy-of-a-hack-6-separate-bugs-needed-to-bring-down-google-browser/

<comment><USER2> The scary part was the limitation that hackers weren't allowed to exploit Windows vulnerabilities. When you think of how many unpatched Windows 7 (or older) machines are out there it's no wonder botnets keep growing..

<comment><USER3> I thought the scary part was the pink my-little-pony wielding a battle-axe that Pinkie Pie chose as his ("his?" ... i assume) mascot. ;) But then, different things scare different people different ways, and maybe I've just become numb the Windows-scare already.

<comment><USER4> Actually, that "pink my-little-pony wielding a battle-axe"-pony is Pinkie Pie (<http://mlp.wikia.com/wiki/Pinkie_Pie>) ;)

She's known for being able to break the fourth wall (unlike the other ponies in FiM) and doing other "impossible" things, which I'm figuring the hacker may have connected to breaking out of a sandbox.

You may also want to see: [https://secure.wikimedia.org/wikipedia/en/wiki/My_Little_Pon...](https://secure.wikimedia.org/wikipedia/en/wiki/My_Little_Pony:_Friendship_Is_Magic_fandom)

</comment></comment></comment><comment><USER5> Where do many of these hackers come from? What is their background?

I ask, as wish to know at a high level what tools and code they are using to find and exploit Chrome.

</comment><comment><USER6> The article makes it sound like since "6 separate bugs" were needed to bring down Chrome, it proves that its a more secure browser (than presumably IE or Firefox). But in 10 days a teenager found 6 bugs in Chrome in 2012? That doesn't sound more secure.

<comment><USER7> It's probably easier to find bugs the deeper you get. There's the assumption that you're authorized, and fewer people get that deep so exploits come up less. It's far from ideal, but most of the issues are listed as low (2) or medium (2). Only one is listed as high (though the last is undisclosed).</comment></comment>


Barclays and Bank of America see looming oil crunch - <AUTHOR> http://www.telegraph.co.uk/finance/newsbysector/energy/oilandgas/7266837/Barclays-and-Bank-of-America-see-looming-oil-crunch.html

<comment><USER2> Peak oil compounded by exponential population growth spell the end of cheap oil, in the least.</comment>


Ask HN: Are any startups interested in Windows Phone/Blackberry? - <AUTHOR>

Reading the latest Who is hiring/Freelancer HN posts, I was a little bit surprised about the number of Windows Phone and Blackberry related offers. I know they're not as profitable/big market as Android or iOS, but I didn't expect so little attention (3 offers at most out of ~400).<p>I'm pretty curious about this, both personal and professionally (I'm a WP dev), so I wanted to ask if this is the actual scenario. Maybe there are startups working on Windows Phone / BB that don't think HN is the best way to find the proper dev, maybe no one is interested in such a little markets... I would like to know your opinion. <comment><USER2> I can't speak for the WP8.

As for the BlackBerry 10 environment there are many android applications but from what I am seeing the BlackBerry 10 Native application app land is empty and ripe for the taking.

The BES 10 environment applications are also something I am shocked to say that has no competition. There are many different niches that you can work on for BlackBerry 10.

SharePoint, Remote Desktop, Stock Data Applications, Alternative to BlackBerry Work Drives, Real File Manager

</comment><comment><USER3> My comment is going to be down voted into oblivion but I think startups ignore Windows Phone and BBZ10 at their own peril.

I currently own a WP8 and it's a lustworthy phone. I see a lot of opportunities for app developers. I'm hearing good things about the Z10 too and friends are buying it. Not many apps though. Not much competition it seems.

Reminds me of the saying _"In the land of the blind, the one-eyed man is king."_

<comment><USER4> Okay, perhaps they are nice devices. But, from a startup point of view, the fact that they are small current markets mean you are taking an extra gamble -- you are counting on all the factors that would normally go into making your product _plus_ the success of the platform. Unless there is some superlative feature of the platform that makes practical something central to your concept that is impractical on more-currently-popular platforms, it seems like an unnecessary risk.</comment></comment>


RESTful Realtime - <AUTHOR> http://code.realcrowd.com/restful-realtime/

<comment><USER2> Screw long polling. Use websockets.

<comment><USER3> That's basically what I was going to say. I've developed a realtime system for my company that uses a REST API on the backend, but we use websockets to push updates to the clients in realtime. It's so much faster (no polling and no overhead of establishing new connections). When there's an update we hit the API once to fetch the new data and then push it out to all clients subscribed to the given data set which massively reduces load on the server.

<comment><AUTHOR> In this approach we aren't re-establishing connections every time since the server side supports keep-alive and the clients are web browsers, which also support keep-alive.

We also hit the API once to push out changes to the subscribed clients through the connection proxy. And the next poll has relatively minimal overhead (obviously not as small as 0) to be queued back up.

In the end, though, this obviously isn't an implementation aimed at extreme efficiency. If that were the goal we'd take a much different approach.

It is about simplicity and semantic elegance.

</comment></comment></comment><comment><USER4> That's an interesting approach, but it's still pooling, although I wouldn't know better.

Does anyone know what the alternatives are? Can you avoid pooling and still speak HTTP?

<comment><AUTHOR> Yes, it's definitely still polling. It is polling about once per minute. However, at least it's only polling at the application protocol level, not the connection level. Browsers and the GRIP proxy support keep-alive so the same TCP connection is very likely to be re-used for quite a while.

HTTP by nature is request/response. To speak pure HTTP, there isn't a way around polling that I know of. Avoiding polling means implementing your own protocol over HTTP, or using WebSockets (see Comet or implementations in Socket.IO, Firebase, SignalR, etc).

<comment><USER5> There is actually a way to do streaming over HTTP without doing WebSockets. It's called "Server Sent Events" and is implemented in the EventSource API in Chrome, Firefox and Safari.

It's quite easy to fall back to long polling in older browsers and it's a very simple and straight forward format.

<comment><AUTHOR> Yes you can certainly stream and do message passing within HTTP (for some browsers) but you can't, for instance, return headers more than once without your own application protocol.

The most interesting bit about this architecture for me is the 1:1 mapping with a pure HTTP resource. It's a bit like an implicit subscription to a pub- sub enabled resource. In this implementation we are simply calling our API like we normally would (except on the stream domain) and waiting for an update.

</comment></comment></comment></comment><comment><USER6> As far as a REST API supporting etags and if-none-match headers, it's pretty unexpected that if I supply an if-none-match to the request, the server may keep the connection open for an arbitrary amount of time. I'd expect that behavior to be a side-channel of the normal 304 response. For example, it should only keep the connection open if I supply ?timeout=60 resource option or X-Modification-Timeout: 60 header.

</comment><comment><USER7> Your code samples are incredibly hard to read, you should use a different library to generate those. I don't know if it's just a Chrome thing, but they look like this to me:

[http://imageshar.es/52cf03f8f629007c1400043e](http://imageshar.es/52cf03f8f629007c1400043e)

<comment><USER8> This is a Chrome bug [https://code.google.com/p/chromium/issues/detail?id=236298](https://code.google.com/p/chromium/issues/detail?id=236298)

</comment></comment><comment><USER9> I'm curious if you guys did consider SignalR. Did you? If yes, yeah not use it?

<comment><AUTHOR> We did consider SignalR with the Service Bus backplane (we run on Azure). We also considered node.js with Socket.IO/Redis, Firebase, and a number of other realtime systems. But when we looked into it we realized we would be duplicating a lot of code on the realtime side that was already there on the RESTful API side. With this approach we're able to do realtime with minimal modifications to our server and client side implementations that deal with our simple use case of keeping a resource up to date. Because we use AngularJS, simply updating the resource on the client side re-binds everything.

<comment><USER3> In the realtime stuff I've built I publish update notifications on different channels to Redis whenever data is saved from a model. Then I have a Node.js server that listens for those update notifications, fetches the new update once, and then pushes it out to all connected clients. This reduces a LOT of load on the server because if we have 300 clients connected, it still results in only one request to the server to fetch the new data and update all clients.

Even if you don't want to use a pub/sub and publish instant notifications when data is updated you could implement polling in a Node.js server and just have all the clients connect to it with Socket.io. Then when the Node.js server sees an update in the data it can push it out to all the clients.

This strategy also has the advantage that you don't have to change anything on the client if you decide to switch to a pub/sub in the future for more real time updates.

<comment><AUTHOR> I've definitely done the separate channel thing in the past for update notifications and it does perform very well.

Our setup is a pub/sub system, where the "sub" happens implicitly with the request for a resource via the GRIP proxy and protocol and the "pub" happens when we update our models as well. The proxy acts a lot like node.js and Socket.IO would, but simply speaks pure HTTP on our RESTful resources instead of the Socket.IO protocol over their transports.

</comment></comment></comment></comment><comment><USER10> Have you looked into accepting Server Sent Events instead of long polling?

<comment><AUTHOR> We didn't see much of a need to expand beyond long polling, if for nothing more than simplicity's sake. Realtime is a small feature in small parts of the application right now. Keeping it very simple and coupled with our RESTful API let us roll it out quickly.

<comment><USER2> Main reason to avoid long polling is the awful scalability (compared to plain old socket programming).

<comment><AUTHOR> Processing the poll request certainly has a bit of a CPU/memory/network overhead. Though any sort of keep-alive would require something similar. I'm sure we'll get fancier with WebSockets when we have to rent too many new servers because of this stuff. The simplicity and elegance of overloading the RESTful HTTP system was the most appealing bit for me with this architectural experiment.

<comment><USER10> As I understand, Server Sent Events are plain old HTTP, unlike Web Sockets.</comment></comment></comment></comment></comment>


Nokia sues Apple over more iPad/iPhone tech (data compression, antenna design) - <AUTHOR> http://www.businessweek.com/news/2010-05-07/nokia-sues-apple-over-technology-used-in-iphone-ipad-update3-.html

<comment><USER2> "Nokia, which took mobile phones to the Internet more than 10 years ago with its keyboard-based Communicator, was slow to move to the touch screens featured on the bestselling iPhone"

Sounds to me like they lost their innovative touch and are morphing into a troll.

<comment><USER3> It's pretty much _never_ a good sign when a company files this sort of patent lawsuit.

<comment><USER2> Very true. I'd be interested in seeing some data about the life-expectancy of a business filing lawsuits like this, tho I think theres probably too many variables to accurately measure.

<comment><USER3> Indeed. E.g.:

Are you getting funded by Microsoft or another big company that's suing by proxy (SCO Group v. The World, 6 years and counting).

When does the company realize it's in big trouble and pull the lawsuit trigger? E.g. how much cash and continuing business does it have left?</comment></comment></comment></comment>


Elon Musk's Wild Ride - <AUTHOR> https://www.bloomberg.com/features/2016-elon-musk-companies/ <comment><USER2> I have a healthy dose of skepticism around the Musk personality cult that has developed. However, the two essential (and, it might be added, correct) points are:

> To the extent that the electric car is a reality, it's largely thanks to > Tesla selling tens of thousands of high-priced vehicles, pressuring BMW, > Audi, Mercedes, Volkswagen, and others into a response.

> Before SpaceX, it was considered daft for an individual to try and compete > against nation-states with decades of experience and billion-dollar budgets. > Now a handful of rocket companies have cropped up, seeking to undercut > SpaceX on price. As the price of getting to space has dropped, dozens of > other startups have come running with new kinds of satellites that promise > to change technology ranging from imaging to medicine.

Even _if_ he loses, we win.</comment>


Ask HN: Redash or Metabase or Superset or QuickSight? - <AUTHOR> I work for a small organisation that's looking for the right tool to visualise our analytics data.
I know selecting the right tool is highly subjective, but, what parameters should I use to judge/compare these tools? There's no nice online comparer for these tools, and every article I've tried reading ends up being quite opinionated.
HN, can you help me rate these, and other similar tools, on the following dimensions:
* Ease of use (for non-developers)
* Versatility (in terms of query diversity)
* Scalability (in terms of data size)
* Visualisation options
* Under-the-hood strength (for power-users)
* Any particular DB requirements
* Any particular DB Schema requirements
* any other that may be relevant...
Here's to hoping this becomes a comprehensive snapshot of analytics/BI frameworks as they are right now. <comment><USER2> I'm a UX designer on the Metabase core team, and I'll try to answer some of the bullet points about Metabase as honestly as I can from my perspective. Also happy to answer any other specific questions folks might have.

One of the main things we're pretty proud of about Metabase is the care we've tried to take with its design and usability. We often get positive feedback from users about the ease of using Metabase and its aesthetic appeal. Beyond that, we have a simple GUI querying interface that's great for users who don't know how to write SQL or native queries. Lastly, we've been doing some interesting things using ML to provide automatic insights called x-rays to show things like growth analysis for time series at the press of a button.

In terms of scaling, we have a wide range of companies using Metabase with either very small hobby DBs or huge Redshift instances, both with success. We have some work to do in making sure super massive DBs don't overly slow down certain parts of the app (like field mapping with SQL filters, or paginating long lists more efficiently). In terms of scaling in-app files or organizing things, Metabase lets you put your saved queries into collections, which you can set user group permissions on.

I think Metabase checks off most of the common visualization options: line, area, bar, row chart, pie/donut, scatter/bubble plot, region map, pin map, progress bar, and funnels. Two requests that we get often that we're working on are line+bar combo charts and cohort analysis tables (which can be accomplished with a pivot table, but without nice color formatting).

Under the hood, we've been trying to add a lot of power over the last year: data access controls, field value remapping, nested or sub-queries, SQL/native query variables, the ability to embed charts or dashboards in other applications, fine-grained scheduling controls for DB sync and analysis, and Google auth and LDAP login.

Not sure if you have any specific questions about DB or schema requirements, but I'll just say we currently support: Postgres, MySQL, Druid, SQL Server, Redshift, MongoDB, Google BigQuery, SQLite, H2, CrateDB, Oracle, Vertica, and Presto. There's also an experimental Google Analytics driver that's not 100% fully featured, but is still quite good (we use it to visualize a few of our own metrics).

Lastly, our team is quite active on our GH repo ([https://github.com/metabase/metabase/](https://github.com/metabase/metabase/)), doing our best to quickly respond to issues, requests, and PRs.

I hope that helps.</comment>


Ask HN: Sanity check wrt shares and exploitation - <AUTHOR> I need a sanity check to know whether I'm being screwed or not - I have no idea anymore, and have no one to talk to who'd really know how to advise me.
Some background on me - I don't have a fancy CS degree(do have business undergrad) and I don't live near any of the tech hotspots. I also have about 4 years experience as a full stack dev.
I have been working for a startup for about a year where we had a verbal agreement(I know, I know... but I was desperate for work at the time) that I would receive 3%(300M valuation at this stage) of the company in exchange for a regular, but only OK salary. However, no bonuses or overtime, because I'm a shareholder.
So, predictably, I have been working a lot of overtime - 12 hour days, almost every second weekend, ect. I'm also being treated like a regular employee, not a shareholder(not being kept in the loop about business regularly, given orders, and such).
But in our last shares-related conversation, it was revealed that I will, in fact, only get 3% of the allocated shares(50% allocated, 50% unallocated). This means that I only get 1.5% of issued shares, when considering future dilution possibilities.
So now I'm feeling... I dunno. I feel I'm being screwed, but not sure if I am. It was a sneaky move to not be upfront about what exactly I'm getting, but my desperation also lead me to not push for a clear written agreement. But maybe 1.5% of a growing 300M company is not bad for someone who was only looking for work?
Another thing to consider: the income projections indicate that profitability can only reasonably be expected in about 3 years time. This means that if I continue and accept my 1.5%, I'll be working at least 3 years with no overtime pay, no bonuses, and a doubtful prospect for increases before being eligible for distributions(if there would be any).
What would you do? Am I being screwed? Am I being stupid? Should I hold out and see how it goes? <comment><USER2> Did you at least get the 1.5% in writing?

<comment><USER3> > but my desperation also lead me to not push for a clear written agreement.

Sounds like not. A written document would be the Sanity check he is looking for.</comment></comment>


On the Law of Diminishing Specialization - <AUTHOR> http://calnewport.com/blog/2018/10/03/on-the-law-of-diminishing-specialization/ <comment><USER2> I'm surprised at the hostile reception to this post, because I think it's making a very good point.

The trend in the new economy is for a large percentage of people to be discarded like rubbish, while increasingly insane workloads are placed on the remainder. Not only is this bad for all the individuals involved and for the health of society, but OP makes the point that it's actually bad for the bottom line too. Executives are rewarded for doing things this way because it superficially looks like they are saving money, but the resulting productivity cut ends up reducing profitability. A more sane and equitable distribution of workload, would be better by every metric including profit.

<comment><USER3> And even if the workloads are doable in a standard workday, the context- switching involved removes any possibility of high-quality thinking.

</comment></comment><comment><USER4> This is a huge problem in government and industry research labs. Rather than have dedicated staff to do things like travel expenses and purchasing, they now have each expensive researcher do it themselves, slowly. Looks like money is being saved with less staff, but now researchers are less productive.

<comment> the-dude Sounds like DevOps to me.

<comment><USER5> No way.

The Dev/Ops interface is one with very little information hiding, little responsibility segregation, very little outcome independence, and nearly no knowledge differentiation.

The researcher/admin interface separates things with complete opposite qualities on each of those dimensions.

<comment><USER6> In all my experiences with devops, I've had to do their jobs for them in their entirety, but by proxy: "ok, first type 'sudo service https start'. Now type 'cd /var/log/httpd'. Then type..." Except I have to do it all up front, and if I miss an (obvious) step, somebody who ought to have no business yelling at me yells at me for providing incomplete instructions.

<comment><USER7> I thought DevOps folks are people who take an engineering mindset to operations, i.e. with both skills of software engineering and system administration? What you're describing sounds like someone with neither skillset.

</comment></comment></comment></comment></comment><comment><USER8> > Because the former [administrative workers] are cheaper to hire than the > latter [specialized workers], the result is the same work for less total > staffing costs.

I worked for a small company (5 or so employees). We had an administrator who was a former VP of a very large bank. I don't know how much we paid her, but she was worth more than any other employee, in my experience. She made that place run.

I suspect that one very good administrative worker is worth their weight in gold, and has the ability to make 10x-ers out of otherwise normal employees. I'd suggest paying for really good admins, rather than viewing them as low- skilled, low-wage commodity workers.

</comment><comment><USER9> I think there's a psychological facet at play as well.

It may well be statistically bad for the bottom line, but it's much _easier_ for the day-to-day. Planning things is _hard_ , decomposing work is harder still, and making sure that the things you build are architected and organized in such a way that they _can_ be planned and decomposed is the hardest yet; it's the same reason parallel programming is hard, I think (this, granted, all from a software perspective).

So even if yes, it may be biting people in the long term, the short-termism is that it's way easier to get things done because you (as the exec) just ask one person to do the entire project, and they maintain all that complexity in their head, never needing to translate it out into shallower, noisier channels, and the 'difficulty overhead' of sound planning, architecture, management, and communication is avoided.

The "I'll just do it myself" culture is really hard to escape from, too, because I think we _feel_ more productive in it, even if we aren't (compared to an organization that effectively staffs work out, and has the staff to accommodate). Intuitively, you're doing all this work and your fingers are constantly flying across the keyboard. You've got so many commits in today. You feel a lot more powerful than you do writing up issues and stories, and asking for statuses. And at the same time, it's a lot _easier_ than writing up issues and stories (ones that will be genuinely useful as planning and work items, at any rate). It's part of why hero-efforts remain pervasive, I think. And it's a kind of sunk-cost fallacy at the end of the day; you're so deep in it already that you may as well just put this one last little bit of effort in and finish it yourself, rather than waste all the time documenting and decomposing and planning so that you can better facilitate parallel, distributed work.

<comment><USER10> Picture a meeting where 6 people making 100+k are looking for a whiteboard pens that works/eraser/ etc * 5 meetings a day.

Paying somone minimum wage to just keep that from happening across say 15 conference rooms is a likely a net win even if they did nothing else. The issue is this is an invisible cost where cutting that person looks like real savings.

We have this idea that everyone needs to be actively working all day when there is such a huge discrepancy in pay scales that it's not that important.

<comment><USER11> This is startlingly real. Or, from my own day today, 8 people that earn 100K plus/year on a call where half of them never speak. How much did that call cost?

</comment></comment></comment><comment><USER12> It's an attractive concept but I have seen that adding more administrative folks does sometimes just add more bureaucracy.

People want to justify their purpose in an org, so they will create approval or audit processes where they aren't needed.

<comment><USER13> To me it always seemed obvious that adding more people decreases efficiency; it's just a matter of communcation:

With more people, it takes more effort for knowledge to spread, so you either spend that effort (which reduces efficiency), or you have people doing their jobs without information that could make them more efficient (which reduces efficiency).

I've heard arguments that adding administrative staff is worse than adding other sorts of staff, but that is not as immediately obvious to me.

<comment><USER14> It depends how you measure efficiency.

5 people can produce 10 widgets in a year.

10 people can product 15 widgets in a year.

Are the 10 people less efficient than the 5?

If you need 15 widgets, the team of 5 has 0% efficiency.

<comment><USER13> Oh, there are many reasons we have large organizations, and one of them is that you simply cannot do some things with a small team. I think you have to bend the definition of efficiency to it's breaking point to argue that producing 15 widgets with 10 people is more efficient than producing 10 widgets with 5 people though.

A Honda Fit is clearly a more fuel efficient car than an F1 racer. This remains true whether or not the problem you are solving is "win an F1 race," which would necessitate using the latter.

<comment><USER15> Actually an F1 car is extremely fuel efficient. In absolute terms it consumes more gas but considering the generated power they are probably one of the most efficient engines around.

</comment></comment></comment></comment></comment><comment><USER16> The thing I never understood was this:

Why don't more executives hire really good programmers and analysts to be their assistants? If a tech CEO is earning $5m a year who gives a damn if their assistant is earning $500k? Right, like why not just pay for the boost in automation and understanding of abstract concepts to maximize the communication-time to useful-actions ratio between assistant and CEO?

<comment><USER17> Tech CEOs that make that kind of money have whole armies of assistants already. But their problem isn't _technical_ , generally. They're worrying about finances, long term industry outlooks, and keeping Wall Street happy. The job of a "tech CEO" at a giant corporation is no different from the job of a "retail CEO" or "service CEO" or "manufacturing CEO".

You might be thinking of startup and small tech company CEOs that really understand the nuts and bolts of a software product, but they're not making $5M/year.

<comment><USER18> "Why don't more executives hire really good programmers and analysts to be their assistants?"

They do. Usually they have a title like "Chief of Staff" or "Speechwriter" or "Associate." My friend occupied just such a role after working in investment banking and private equity. He was paid in the low-mid six figures and ended up doing tons of research, analysis and writing for the CEO of one of the most successful private equity firms. The position was stepping stone to Harvard Business School and nobody stayed in the role for much more than a year or two, so they are constantly rotating in fresh, hungry and smart people.

I like the idea of adding automation to the mix, but realistically, if there is a task the CEO wants automated, he/she can tap any engineering manager and pull their team onto that project. The value add would be automation of tasks for the assistant rather than automation that benefits the CEO directly.

</comment></comment></comment><comment><USER19> I think this makes sense. It's consistent with the Mythical Man Month's concept of building a supporting cast around the specialist.

In a startup it may be faster to do everything yourself. Once you operate at scale, every hour of low value activity sucks away an hour of high value activity. I don't respect folks who are incompetent without their admins, but I also disrespect managers too stuck in the weeds to lead.

<comment><USER20> > I don't respect folks who are incompetent without their admins

Yet this is _exactly_ what the article is proposing.

If you don't do, say, Powerpoint, with enough frequency, you are effectively incompetent at it. Insourcing this to an admin practically guarantees you will be incompetent at it.

Steve Jobs had Apple create iPreach (Keynote) because Powerpoint wasted so much of his time ...

<comment><USER19> It's a matter of degree. There are things that someone can do from a skills standpoint, but shouldn't from a priority standpoint. (For many execs PowerPoint is in this bucket)

</comment></comment></comment><comment><USER21> For anyone interested in this topic, I highly suggest the book _Bullshit Jobs_ , which is basically a very in-depth (but entertaining!) ethnography of such over-specialized jobs.

<comment><USER22> I don't think that "bullshit jobs" has anything to do with any over- specialization, but instead with jobs that are actually pointless.

The article is talking about tasks that actually are necessary, because they are still being done. But instead of being done well, by a person specialized in that task, they're being done less well by people who are specialized in entirely different tasks, taking away the time they can spend on the tasks that they themselves are specialized in.

<comment><USER21> Sorry, have you read the book? (Not trying to be be condescending, actually curious.) I'm pretty sure it has everything to do with it, and more. The book even addresses arguments that such over-specialization is _necessary_ :

> In other words, the author claims that when we speak of "bullshit jobs," > we're really just talking about the postindustrial equivalent of factory- > line workers, those with the unenviable fate of having to carry out the > repetitive, mind-numbingly boring but still very necessary tasks required to > manage increasingly complicated processes of production. As robots replace > the factory workers, these are increasingly the only jobs left. (This > position is sometimes combined with a rather condescending argument about > self-importance: if so many people feel their jobs are useless, it's really > because today's educated workforce is full of philosophy or Renaissance > literature majors who believe they are cut out for better things. They > consider being a mere cog in administrative machinery beneath their > dignity.)

<comment><USER23> This comment is very confusing. The quote is from Graeber's dismissive summary of an article in The Economist [1]. Here, Graeber specifically rejects the notion that his book is an 'ethnography of such over-specialized jobs.' In Chapter 1 he gives the following explicit definition of a bullshit job:

> Final Working Definition: a bullshit job is a form of paid employment that > is so completely pointless, unnecessary, or pernicious that even the > employee cannot justify its existence even though, as part of the conditions > of employment, the employee feels obliged to pretend that this is not the > case.

[1]: [https://www.economist.com/free-exchange/2013/08/21/on- bullsh...](https://www.economist.com/free-exchange/2013/08/21/on-bullshit- jobs)

</comment></comment></comment></comment><comment><USER24> This kind of reads like the idea that "High level people shouldn't be doing work that is beneath them". Which is rather offensive.

Moreover, I'd guess the overhead of communication probably means it is more efficient for a marketing director to make his own presentation and charts. After all, he is the only one that knows what he wants to present, and what data should be in the charts. If I had to communicate that, I'd probably make a crude powerpoint and say "Like that!".

<comment><USER14> It's not offensive to put people to work in a way that generates the best results for the team.

[https://en.wikipedia.org/wiki/Comparative_advantage](https://en.wikipedia.org/wiki/Comparative_advantage)

<comment><USER24> True, but it is bad to say "I have a degree, therefore I feel that doing X is beneath me". Note that I distinguish that from "I cost the company a lot of money, so it would be more efficient to have someone else do X".

<comment><USER25> Indeed, it has to be said much more diplomatically, and/or carried out more strategically. The CEO doesn't say "I am a CEO, so doing X is beneath me." He simply gets X done for him, and that's that.</comment></comment></comment></comment>


Show HN: Publish a Markdown website in 5 minutes - <AUTHOR> http://www.markdownsites.com/ <comment><USER2> Congrats on the launch! I had an initial thought of coming up with a site like this, but decided that providing an open HTML platform was best (hence, [https://neocities.org](https://neocities.org)). But I'm happy someone finally scratched the itch.

Random thoughts:

You're going to want to use subdomains for the sites, otherwise all sites are sharing the same domain security model, and you're going to run into problems when (not _if_ , _when_ ) a site gets reported for abuse/phishing/spam/ect issues. Path origin security support is years out.

While you're on subdomains, get a wildcard SSL cert and default to it. Neocities started without SSL, and next month I'm paying the price for that sin - we're switching to default SSL and unfortunately it's likely going to break a lot of sites. It's not really a choice anymore, given the political climate and the upcoming pagerank dock from Google.

If the site explodes in popularity, I'm happy to give advice. There's a lot of problems I've solved relating to running a massive web hosting service for a lot of people. Some of it required months of research and some serious firehose drinking to figure out.

Lastly, don't be afraid to ask for money - at least $5/mo, if you go lower you're just crop sharing for the credit card companies. At a bare minimum, throw up a PayPal donation button. I've learned that if you provide a useful service, people will pay you, even if there's an excellent free option. I'd rather pay a small amount to help sustain a service than have it fall apart from lack of funds and/or dev interest. That's a good thing and it's worth something.

To illustrate the importance of at least a donation button: HN users donated over $4000 to us on launch, and I ran the site on that budget for almost two years. Site possibly wouldn't exist today without HN support. Sometimes the comment sections betray this (I'm including myself in this statistic), but kind hearted, wonderful people lurk here.

Good luck.

<comment><USER3> Sorry for the off-topic, but I have to share:

A year and a half ago, I was looking for a place that would allow me to host a single HTML page for free. A couple of shady free hosting sites later, I came across Neocities. It was a breeze to set up an account there and host a short story of mine in HTML format.

I found myself giving out the link to anyone who'd like to check it out, without needing to e-mail them a PDF (or even worse, a Word document). I like nice typography and readability so sending a simple text file was not the ideal option either. But a link to a page was perfect: it displays well on every device and the page can easily be updated. I wrote the whole thing in Markdown and hacked together a short script to generate HTML from it, which I could easily drag-n-drop to Neocities for quick iteration based on remote people's feedback.

I never wrote again, but if I did, I'd definitely use the same method. An open platform for HTML publishing is essential to keeping the spirit of the minimalist, hyperlink-driven Web alive.

So thank you for Neocities, and cheers!

</comment></comment><comment><USER4> My major critique is that the generated sites are needlessly javascript powered. Right when I saw the "Loading" I thought oh no how many assets are going to be downloaded... for just markdown content.

I guess I'm just an old web 1.0 guy but I just don't see and understand why everything must be done in a super dynamic way. Hell sometimes I like clicking on a link and watching the browser do the loading as it is confirmation for me the thing actually got loaded. Particularly when buying/shopping for stuff (ie traditional POST).

<comment><USER5> Aw...this makes me so sad.

I need to fix my brain. Every time I see "static sites", I think "oh great, no dynamic server or client code, just static HTML/CSS for fast loading".

If I'm going to have to undergo a "compile step" for my website, it's going to do something worthwhile, darn it.

<comment><USER6> I keep waiting for someone to write a program that consumes as input Markdown text and emits as output a pure HTML/CSS site literally like this [http://bettermotherfuckingwebsite.com/](http://bettermotherfuckingwebsite.com/) except with the text from the Markdown file instead of swear words. Maybe it could even accept several Markdown files and make them into blog posts or something.

If nobody else does I guess eventually I'll get fed up and build it myself :\

<comment><USER7> Plenty of static site generators, as they're called, exist to do this. Good jumping-off points would be investigating Jekyll (written in Ruby) and Hugo (written in Golang).

[https://github.com/jekyll/jekyll](https://github.com/jekyll/jekyll)

[https://github.com/spf13/hugo](https://github.com/spf13/hugo)

<comment><USER6> Yup but they're bloated.

<comment><USER8> I agree. In Hugo's case there wasn't even a way to generate a completely static website for the longest time (i.e. not a blog or anything like that) and I'm not even sure they've fixed it yet.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> I have many informational websites like [http://www.krister.ee](http://www.krister.ee) and sometimes I register domains on a whim when I get a really good idea for a website. I decided to create a simple tool to help manage these microsites and in addition to buying a domain I can immediately fill it with demo-content using markdown.

Before I get into feature creep (hero image, forms, analytics) I'd like to work on reaching an audience. Problem is, I'm not sure who would appreciate it. I know it's a useful tool so any tips on how to spread the word is greatly appreciated.

<comment><USER9> Your target market is somebody that wants to create _lots_ of minimalist websites quickly. (those creating one personal website whose priority is getting things done likely skip the "evaluate best alternatives" step and go straight to Tumblr/Wordpress or their favourite social network.)

Unfortunately, apart from _polymaths that just want to share their expertise on as many discrete subjects as possible_ which I'm not convinced is the biggest target market, it's probably blogspammers and astroturfers...

<comment><USER10> Also troll websites, for example, [http://www.makecppgreatagain.com](http://www.makecppgreatagain.com), or [http://opisaflag.com](http://opisaflag.com)

</comment></comment></comment><comment><USER11> This is excellent. There has been a lack of tools to solve this exact problem - wanting to get a basic site up very quickly and cheaply.

Previously the best alternative was a free Firebase/Google account, and then setting up a HTML/CSS site, and deploying it.

This is much faster.

Thank you for building it!

And please, don't be dissuaded by the other commenters. Sure, the market is much smaller than the market that Squarespace et al are targeting, but you're solving your own problem, and there are at least a few others that share that same problem!

Problem this solves: People who want to quickly register a domain and get a text landing page up.

Side note: I wish Medium.com could set up a specific blog post as the home page for a publication with a custom domain

Or I wish I could get a custom domain for Quip.com notes

<comment><USER12> Sounds like you're looking for something similar to what i just released today. It's a service called Idnty and let's you create a simple and clean personal page in no time. No custom domains yet but it's in the pipeline. Please have a look, [https://idnty.me](https://idnty.me)

<comment><USER13> Really cool. I don't want to create an account just to try it out though, maybe letting users play around until they are satisfied would give a better sign up conversion rate? (Although the video itself is great).

Just a thought, not sure if good or viable. But maybe let users buy their designs for self-hosting would be an idea.

<comment><USER12> Thanks for some really good feedback! Your idea about signup after trying it out is spot on. I will probably implement that straight away. Self-hosting is interesting as well, never thought of that, but might be an alternative way to go.

</comment></comment></comment></comment><comment><USER14> Making a modern, responsive, mobile-aware website seems like a huge learning curve right now.

It would be great if there was a tool that let you use simple markdown, took a few options, and generated beautiful static pages like e.g. Medium with a minimum of hassles.

I guess a lot of people use WordPress with themes, or Weebly, SquareSpace but it still seems pretty complicated.

<comment><USER15> If you want Medium, why not use Medium?

<comment><USER16> medium is blog focused, a medium post is a post on your account. this is a standard single webpage

</comment></comment></comment><comment><USER17> Reminds me of Aaron Swartz's Jottit application: [http://www.aaronsw.com/weblog/sweatsmall](http://www.aaronsw.com/weblog/sweatsmall)

<comment><USER18> While Jottit is great and representative of yet another cool thing Aaron contributed to the world, it occasionally goes down for days, or even weeks at a time.

<comment><USER17> Oh, yeah. It's also currently incredibly slow to load for me. I wouldn't recommend that anyone use it at this point. Just pointing out an example of a similar service for comparison. It doesn't do much, but the UI has some serious polish.

<comment><USER18> Current usability aside, like another poster said, thank you for sharing that, James.

</comment></comment></comment></comment><comment><USER19> Awesome project! Anything to get faster deployments out is a great tool.

Hate to be 'that guy', but it would be nice if you served this over HTTPS, given that you are accepting credit card details.

<comment><AUTHOR> for sure, top3 issue to work on.

</comment></comment><comment><USER20> If you want to roll your own, I have a Docker image that pulls Markdown out of git, runs Hugo over it and pushes the site to an S3 bucket. Also works with Bitbucket pipelines so you can get a site rebuild with a simple git push: [https://github.com/rabidgremlin/hugo-s3](https://github.com/rabidgremlin/hugo-s3)

<comment><USER21> Now I feel old.

I have a very small server running Apache, with mod_markdown installed. .md files are rendered as HTML by Apache, and there's a CSS file to tweak if I feel like it.

[https://github.com/hamano/apache-mod- markdown](https://github.com/hamano/apache-mod-markdown)

<comment><USER20> That's similar to how I used to do it it but I got tired of paying for servers.... plus this approach let me play some "buzzword bingo"

BTW my first Internet facing thing was a CGI-BIN Perl script delivering call records for telcos, way back in 1995. I've been cutting code for 35+ years. Never feel old!

<comment><USER21> I have several almost-zero-traffic sites running from an Odroid (ARM board) at home. The largest site has only 30k requests in the Apache log for October, but it's a photo gallery, so there's 150GB of pictures behind it. There's also a backup of my desktop, which is another 2TB or so.

I set its ancestor up when I was about 16, using Debian -- and invested a fair amount of time in it, learning a lot.

When I bought it in 2013, the cost of cloud storage for a few years was far greater than the cost of the board + a hard drive, but setting up the obscure hardware took more effort than I expected. Nevertheless, I have no plan to switch to cloud services. I like owning my own stuff, and keeping the flexibility to add whatever service I need on the network at home is very useful. For example, I can SSH into it, and control my fancy central heating system, and a while back I added a UPNP media server, which can stream to a Chromecast.

<comment><USER20> Nice!

</comment></comment></comment></comment></comment><comment><USER22> I run uBlock and got a message saying the site is down. Disabled it and got it to work.

I think one way to differentiate yourself in this kinda area is making it run without any backend tech like jekyll, python, etc. Make it run client-side only. Just my 2 cents.

<comment><AUTHOR> good point. it's something I myself would also like. I'll have a think about this.

I'll also have a look at ublock. I also use it, haven't had problems. Maybe you tried exactly when the site was down. Had overload problems for a few minutes.

<comment><USER23> It loaded fine for me, ublock in chrome on Linux. Nifty tool you've made - thanks for sharing :)

<comment><AUTHOR> thank you :)

</comment></comment></comment></comment><comment><USER24> Let me suggest Classless Themes[1][2] and its approach to theming that only switches a CSS file to transform the same HTML into multiple different themes. It's ideal for setups like that, where your user want themes, but you can't allow him to touch the HTML templates.

[1]: [https://websitesfortrello.github.io/classless/](https://websitesfortrello.github.io/classless/) [2]: [https://github.com/websitesfortrello/classless](https://github.com/websitesfortrello/classless)

</comment><comment><USER25> I like to make a markdown as well as a site. It's usually different each time, which keeps things interesting. This isn't a problem for me as the number of sites I make is limited by the number of domains I have.

</comment><comment><USER24> I'm interested in that domain part. How can I buy a domain through this thing? Will it own my domain or will I be able to move the domain and use it for different things in the future? How?

</comment><comment><USER26> Coming from wordpress to a website that uses markdown for posts I have run into a few problems or inconvenience that I haven't seen addressed.

There's a lot of boiler-plate in markdown posts. Author, date, etc. Wordpress does all this for you.

My editor doesn't know which categories, authors or tags already exists. I can't remember if I use the category "car" or "cars".

I haven't found a nice way of handling inline images.

<comment><USER27> That's not a markdown problem. Its a consequence of using markdown within a framework, generator, etc that uses frontmatter or an equivalent to define meta data.

Several frameworks and tools allow for markdown and static site rendering that also support an admin UI where one can define this data more easily.

<comment><USER28> _also support an admin UI where one can define this data more easily._

I'm looking for one such or a couple, can you enumerate?

<comment><USER27> Not all of these fit the bill exactly, but they all have their own advantages.

\- prose.io

\- siteleaf.com

\- statamic.com

\- getkirby.com

\- github.com/jekyll/jekyll-admin

\- github.com/netlify/netlify-cms

\- federalist.18f.gov

You can also always build your own on top of whatever generator you want to use (example: [http://willschenk.com/building-a-gui-for-managing- middleman-...](http://willschenk.com/building-a-gui-for-managing-middleman- blogs/)).

My favorite setup though is middleman + contentful + codeship. Static site generation happens with middleman. Content management happens with Contentful. Publishing happens with codeship.

</comment></comment></comment></comment><comment><USER29> FYI - When you check a site, there is a little blue pop up in the bottom right corner. It is hidden by the green "Online" chat bar.

<comment><AUTHOR> hmm, i'm not sure what blue popup you're talking about. Can you screenshot maybe?

<comment><USER30> It's the thing that tells you to save before you can check the availability of a domain

<comment><AUTHOR> riiiight.. fixed! thanks for the report. pretty nasty bug :)

</comment></comment></comment></comment><comment><USER31> There's no greater metaphor for the detachment of the tech-elite from the average user than the pervasiveness of Markdown. Its a poor answer to a problem that exists solely because of lack of coordination among the very tech-elite who rely on MD.

An open spec for efficient binary encoding of formatted text is the thing we've needed for 20 years - not more MD tools.

<comment><USER32> I have two questions: 1) What's the problem that's being solved? 2) And what's a good answer?

I'm genuinely curious because I've never considered Markdown anything more than a way to more easily render text -- which it seems to do fine. But, maybe I've just exposed myself as an average user..

<comment><USER31> 1) See here[1] - problem very specifically being solved is lack of readability of HTML (and HTML being the only open portable spec for formatted text).

2) A good answer is another, more efficient open spec for encoding formatted text - meaning I can use any app to format my text, then open it in another app and be sure I have full fidelity. This fidelity would be constrained by a requirement of HTML output.

[1]: [https://daringfireball.net/projects/markdown/syntax#philosop...](https://daringfireball.net/projects/markdown/syntax#philosophy)

<comment><USER33> Isn't the answer CommonMark?

</comment></comment></comment></comment><comment><USER34> Why does it flicker "Loading" before it displays the content? Can't something be cached somewhere to avoid that whenever I click something?

Not displaying any change until it's rendered might make it feel more seamless

</comment><comment><USER35> Great idea, but the design would have to be significantly more attractive for me to use it for most of the things that would immediately spring to mind.

That's a solvable problem, though - grab some top-flight themes with appropriate licenses and you're probably good to go!

<comment><AUTHOR> Could you perhaps point out some specific details that you find annoying?

</comment></comment><comment><USER36> mdwiki is the best take on this idea I've seen: [http://dynalon.github.io/mdwiki/#!index.md](http://dynalon.github.io/mdwiki/#!index.md)

</comment><comment><USER37> At SunSed we are working on v3 of our CMS.

V3 is very similar (in concept) to what you have built. I believe that there is huge market for tools designed only for developers.

Like Weebly but for developers.

<comment><AUTHOR> interesting, i'll keep an eye out for your release. Incredible UI design btw.

<comment><USER37> Thank you! :)

</comment></comment></comment><comment><USER38> Beautiful idea, you are heading in a crowded market and at the moment the app is not too differentiated, I hope you the best.

<comment><USER39> what other apps are there like this? i've wanted a "splat some markdown to the internet" app but have never found one.

<comment><USER40> Github pages?

</comment></comment></comment><comment><USER41>
      > Extension not supported

when trying to add my personal domain. What DNS extension is it trying to use?

</comment><comment><USER42> Is any group working on standardizing and implementing native support for markdown in the browser?

What are the major obstacles?

<comment><USER43> What is the value in that?

Markdown has two substantial benefits: Easy to store format that is easy to convert into HTML.

All native support would do is lot in a specific Markdown feature set, specific markdown styling, and specific Markdown syntax.

If you then tried to make a browser supported subset of Markdown that you then allowed to be customised, you'd wind up with so much goop that nobody would be happy.

Also, this: [https://xkcd.com/927/](https://xkcd.com/927/)

</comment></comment><comment><USER44> just a heads up, the site was marked by my company's net filter as containing malware.

<comment><AUTHOR> interesting. Do you know where your company get's its filter data from? I wouldn't be surprised if this domain was used dor something else before.

<comment><USER44> openDNS malware

</comment></comment></comment><comment><USER45> surge.sh

<comment><USER46> surge.sh + emacs org mode is how Ive taken to doing most of my static web hosting now; works like a charm

</comment></comment><comment> joe-mccann dillinger.io

<comment><AUTHOR> interesting. i'll try reach the author see how they are doing.

</comment></comment><comment><USER47> It's good.</comment>


Explorsquare - <AUTHOR> http://www.avc.com/a_vc/2011/04/explorsquare.html <comment><USER2> Not a whole lot of content here. Is anyone surprised that someone managed to use foursquare to find a nice restaurant?</comment>


Advice to developers on managing MySQL - <AUTHOR> http://insidechannelmeter.wordpress.com/2011/05/25/advice-to-developers-on-managing-mysql/

<comment><USER2> Great advice.

You can also easily alter old tables to use innodb if you're running into performance issues. I beleve the command is 'ALTER TABLE foo ENGINE = INNODB;'.</comment>


RFC8482 – Saying Goodbye to ANY - <AUTHOR> https://blog.cloudflare.com/rfc8482-saying-goodbye-to-any/ <comment><USER2> (Title of the submission should probably capitalize "Any" as "ANY".)

I liked the ANY type as a convenience and debugging tool, but I guess the disadvantages far outweigh that perceived convenience. And now that I read about the actual semantics of ANY with respect to caches, turns out I was using it wrong anyway!

<comment><USER3> In recent versions of BIND ANY queries still work pretty well as a debugging tool when `minimal-any` is turned on, because you get the traditional full-fat response over TCP, and dig automatically uses TCP for ANY queries.

I implemented `minimal-any` in BIND to reduce problems caused by large responses in DDoS attacks. It helps in situations when RRL is not quite enough.

<comment><USER2> Awesome, thanks a lot for that! I was indeed mostly using ANY through dig (as do most, I suspect), glad to know there's still a way to make it work!

</comment></comment></comment><comment><USER4> This is great. It's so important to try to push for improvements externally, rather than just sucking it up and implementing onerous workarounds.

It doesn't always work out, but this sounds like a good and reasonable outcome this time. (Though it's their blog, so of course it sounds that way!)

</comment><comment><USER5> Even though I know how it works, and why it works, I'm still kind of surprised that individual DNS records have individual TTLs. It feels unfamiliar/alien, compared to all the other hierarchy-of-caches systems I work with.

Every intuition I have about caching would lead me to imagine an alternative implementation of DNS where zones are documents, and intermediate resolvers retrieve and cache entire zone documents from their canonical hosts, in order to respond to queries for individual records from stub resolvers.

Is there some big use-case for the DNS records within a zone to have different TTLs? Would it really be so inefficient if the zone as a whole just had one TTL, which was set as the minimum of the TTLs required by any of its records?

Does DNS not have spacial locality to its access patterns? I.e., if an intermediate resolver retrieves the A records for a domain, does that not imply that it'll soon also want the MX records for that domain, and the TXT SPF records, and—essentially—all the records with some probability? Is there no justification for pre-fetching these?

And, if both of those statements were to be true—that DNS could get by just fine with one TTL per zone, and that DNS queries do have a spacially-local access pattern—then why would you want to design DNS in a way where intermediate resolvers retrieve individual records, rather than entire zone documents? Would a modern ground-up DNS architecture do things this way?

(I have the suspicion that the answer lies in security-through-obscurity, vis. the reason AXFRs are prohibited. So, as an addendum: I ask the same questions as above, but this time presuming your DNS daemon has Row-Based Access Control (RBAC), such that an intermediate resolver retrieving a zone document would only see the records its upstream resolver thinks that particular client should be allowed to see—e.g. enterprise Intranet clients get everything, while public Internet clients get a trimmed view. This implies secure DNS, but just take that as a given.)

<comment><USER6> > Does DNS not have spacial locality to its access patterns? I.e., if an > intermediate resolver retrieves the A records for a domain, does that not > imply that it'll soon also want the MX records for that domain, and the TXT > SPF records, and—essentially—all the records with some probability? Is there > no justification for pre-fetching these?

Probably not. If I want to visit your website, I might just need A and AAAA. That doesn't mean I will soon want to send you emails (MX) or look at your google site verification token (TXT).

> Is there some big use-case for the DNS records within a zone to have > different TTLs? Would it really be so inefficient if the zone as a whole > just had one TTL, which was set as the minimum of the TTLs required by any > of its records?

Definitely no. I tend to set low TTLs for A and AAAA records because in practice they do change very often. Depending on your setup, it possibly changes every deployment. But on the other hand MX is long-lived and I set it to many days. This is important security implications: if say, Gmail, has cached my MX records for many days, it will be very difficult for a DNS takeover attack to maliciously redirect my mail or other sort of DNS poisoning to do the same.

<comment><USER7> Fun fact: mailers will use the A record if a domain doesn't have an MX.

See RFC 5321

</comment></comment></comment><comment><USER8> Could someone explain why standards are "RFC"s? Can't anyone request comments for something? Why would a request for comments be treated as a standard?

<comment><USER9> not all RFCs are standards. RFCs that become one are labeled as STD and a number, but nobody uses those (afaik)

[https://www.ietf.org/rfc/std-index.txt](https://www.ietf.org/rfc/std- index.txt)

[https://en.m.wikipedia.org/wiki/Internet_Standard](https://en.m.wikipedia.org/wiki/Internet_Standard)

</comment></comment><comment><USER10> Why would it be harder to implement then simply iterating over the known types and catenating the results of those queries.

Say we have A, CNAME and MX type records in our system. If we see ANY, let's convert it up-front into into three queries: A, CNAME and MX. Combine the results and that's it.

ELIA5

<comment><USER11> Here's why: suppose some types are easier to answer than others. Maybe A and CNAME and AAAA are cached, MX is in the database but not cached, and NSEC3 requires computing a result. In order to correctly implement ANY (pre- RFC8482), you must convert it into _all_ of those, and wait for the slowest one. An implementation like Firefox that asks for ANY because it just wants A and AAAA is wasting both the client and the server's time.

However, also, pre-RFC8482 that's not what ANY is defined to mean. A non- authoritative caching resolver is permitted to respond with _only the records it has cached_ if something is in cache. So you're already getting a random subset of records. So the behavior is unhelpful for clients and also unhelpful for authoritative servers, and not in a way where the unhelpfulness trades off for helpfulness elsewhere.

<comment><USER10> That semantics is indeed useless.

And to begin with, applications should never be interested in all record pertaining to a domain; they want specific records. If I'm sending mail, I want a MX record for a domain. I don't want the SPF record unless I'm receiving mail which implicates that domain. If I'm making a HTTP connection, I don't want any of these; I want an IP address. So _no_ semantics of ANY is easily justifiable.

On the other hand, we can't claim that this semantics is hard to implement.

<comment><USER11> It's hard to implement _in a way that makes sense in the constraints of the world as it is_ (people asking for ANY when they have some meaning in mind other than the nonsensical specs) and especially in a way that doesn't enable DNS reflection attacks. In an ideal world, sure, you could enable ANY provided nobody either good or evil ever used it. And in fact Cloudflare says they did previously implement it just fine, by littering if statements everywhere.

</comment></comment></comment></comment><comment><USER12> I am happy to see it go. For years I've had to block it in iptables using:

        iptables -t raw -I PREROUTING -i eth0 -p udp -m udp --dport 53 -m string --hex-string "|0000ff0001|" --algo bm --from 40 --to 65535 -j DROP

Most bots are poorly coded, so most of them don't try the individual records when ANY times out. ANY is also a decent generalized amplification attack vector. There are plenty of other ways to do amplification attacks, but this one is quite brain dead.

Thankyou to the folks at CloudFlare for submitting this.

</comment><comment><USER13> But... Qmail! :)

<comment><USER14> What does qmail use it for?

<comment><USER15> Qmail used to do ANY.

Qmail, as opposed to other software, was sane enough to understand that lack of MX in ANY didn't mean it wasn't there. It meant it wasn't in a cache, so it would retry with just MX.

Our initial proposal (to do REFUSED for ANY) indeed had a chance to break some older qmail installations. This is why we engaged in a longer process and found acceptable solution - HINFO. HINFO is both backwards compatible (qmail will work fine) and solves our problems with ANY. Win-win.

<comment><USER16> Why did it do ANY?

<comment><USER17> > This is really just an old workaround that's no-longer needed. Let me > explain...

> Once upon a time, back in 1996, there was a really unfortunate bug in the > most popular DNS server software (BIND 4.9.3): it did not respond correctly > to "CNAME" requests (that is to say, requests for any CNAME data about a > particular domain name). This is critical information that an email server > needs to know to do its job. Thankfully, there was a way to work around the > problem: "ANY" requests. These requests ask the DNS server, essentially, for > any and ALL information it has about the domain name in question, including > CNAME information.

> These ANY queries have two big problems:

> As you might imagine, for big domains with lots of mirrors (e.g. gmail.com), > that's a lot of information, and so the response can be quite big. Big > responses pose two problems: first, it's a waste of bandwidth, and second, > it can expose a bug in qmail's handling of large DNS responses (see the next > patch).

> ANY queries are often not cached by relaying DNS proxies (for whatever > reason), and so ANY queries cause more traffic EVEN behind a caching DNS > proxy server.

</comment></comment></comment></comment></comment><comment><USER18> Holy crap I remember when RFCs had three digits!

</comment><comment><USER19> > This got us thinking: do we really need it? "ANY" is not a popular query > type - no legitimate software uses it (with the notable exception of qmail).

Cloudflare once again pushes for broad changes to the way the net operates that benefits themselves and other centralized corporate players without any benefit to individuals that actually use the net in the sense of using the internet and not just a web browser.

The kind of exploration and direct learning that was possible when I was a kid growing up 90s/00s is slowly being phased out as the money seeps in.

<comment><USER20> What exactly about the deprecation of ANY do you disagree with (other than Cloudflare proposed it and it used to exist)? Also the blog post and RFC seem to make convincing arguments for the general health of the internet, do you disagree with these and if so why?

<comment><USER19> I've always used it to explore the internet for fun as a person without any profit motive.

Cloudflare's argument's are all based around ANY making it slightly harder for them to make a profit. A small chance of DNS amplification attacks in DNS is what Cloudflare thinks about because it's their business. But there's no reason to believe this is more important than individual users wishing to see what servers are behind a domain.

Combined with GDPR killing off whois the internet is a much more boring, less transparent place.

<comment><USER21> Limiting ANY queries to just TCP would have been a reasonable middle ground, but now Cloudflare has a cleaner codebase and does not have to do any difficult thinking around what to do with synthetic rrsets in response to ANY queries. Good for Cloudflare!

<comment><USER15> > Limiting ANY queries to just TCP would have been a reasonable middle

This was proposed solution, the problem is that in case of attack, against a valid Authoritative service, launched via open resolvers, the open resolvers would just download Gigabits of ANY traffic with TCP. Read about this here: [https://fanf.livejournal.com/140566.html](https://fanf.livejournal.com/140566.html)

</comment></comment></comment></comment></comment><comment><USER22> I'm not convinced that ANY is that bad. They wrote some bad code, they have special goals and all of a sudden ANY is not good for you.

<comment><USER23> People are using it for reflected DDoS attacks. This has resulted in DNS providers scoping ANY to the point of breaking it. So even without removing it, it was horribly broken and returned inconsistent results.

In essence all this change does is remove the fiction of ANY, with or without RFC8482 ANY wasn't reliable enough for real usage.

<comment><USER24> Given how bulky ANY is, and the fact that we're changing the standard anyway, would not just restricting ANY to TCP queries "fix" the DDoS issue? The attempted TCP connection would not be formed (the victim would RST it).

(I agree with other issues pointed out by the article, and there are other reasons why, as a RR type, I would still axe ANY. But the functionality of being able to query all RRs on a server is often useful for debugging, though I think there are other practical ways to work around that. (Issue a query for many common RR types.))</comment></comment></comment>