Ask HN: How to deal with negative "tell me about a time" interview questions? - <AUTHOR> I have interviews next week that I'm very nervous about. I'm excited about the job and know I can do it well. However, I know for a fact that it'll be a very long day of "tell me about a time when you failed" and "what's the worst thing your previous manager would say about you". After a day of that, it'll take a while to recover from the bad feelings, especially if the result is the usual ghosting.<p>Any advice how to deal best with this awfulness? <comment><USER2> The goal is for you to tell them what you did and learned about from that negative experience.

tell me about a time when you failed.

I was asked if I could do ABC with a certain amount of time, and I said sure I would get it done, I was sure of A and C but I couldn't figure out B. so I set out to figure out B, I asked for help and no one knew how to solve B, so I went through XYZ process to figure out B, it took longer than I thought, I communicate that I would miss the deadline, I finally figured out B after taking a crash course, but I missed the deadline and was pretty embarrassed by that. I learned that I shouldn't promise to deliver on a hard deadline when there are unknowns, however to my surprise, the client was understanding since I communicated ahead of time that I might ship late. I also learned to keep an open communication and communicate progress and expectations while working on something.

These questions are not traps, they are asked to extract your experience. It's also a good way to demonstrate your communication skills through story telling, it needs to be succinct.

</comment><comment><USER3> I'm a bit surprised at the answers you are getting here. I think this touches on two subjects so I'll try to give two answers.

First of all, when asked about a time when, it's perfectly normal to spend some time to think about it. I always give them a truthful answer and God knows I have a lot previous fails and / or managerial arguments on my plate.

Seriously, "my old manager thought I was a dumb idiot with attitude issues" (hint: not true! or maybe it is).

What does this even tell them? That you're honest (probably) and either you're a dumb idiot with attitude issues or your manager was not a nice person.

Do they act on the assumption that your manager was right? This says a lot more about them than about you.

Remember, it's a two sided interview. If you can't be honest, if they rush to conclusions, if they put you on the spot, you'll be in for a frustrating employment.

Secondly, about your dealing with the awfulnes of past deeds, I'll just copy paste a nice little story here:

> A senior monk and a junior monk were traveling together. At one point, they > came to a river with a strong current. As the monks were preparing to cross > the river, they saw a very young and beautiful woman also attempting to > cross. The young woman asked if they could help her cross to the other side.

> The two monks glanced at one another because they had taken vows not to > touch a woman.

> Then, without a word, the older monk picked up the woman, carried her across > the river, placed her gently on the other side, and carried on his journey.

> The younger monk couldn't believe what had just happened. After rejoining > his companion, he was speechless, and an hour passed without a word between > them.

> Two more hours passed, then three, finally the younger monk could contain > himself any longer, and blurted out "As monks, we are not permitted a woman, > how could you then carry that woman on your shoulders?"

> The older monk looked at him and replied, "Brother, I set her down on the > other side of the river, why are you still carrying her?"

<comment><USER4> > why are you still carrying?

For me, it is because of the serious concern that I'm going to repeat my errors until I find a solution for how to avoid them... and for some longstanding personal problems I don't know how to start finding a solution.

</comment></comment><comment><USER5> Practice. You're telling a story, it can be prepared, you don't have to relive it every time.

Understand why they're asking. It's easy to talk about your success, and to idealize how we got to an outcome, but in reality it's more like: 1) search for something on StackOverflow, 2) figure out you should be searching for different terms, 3) try some off-the-shelf software recommended in a buried comment, 4) fail to compile it from source, 5) try a buried answer in a related question; etc. Those are all "failures" in that you weren't born knowing the correct answer, and being able to describe how you got to an outcome fully is valuable.

Ask them the value they're getting so you can characterize the experience appropriately. We were hiring a senior designer to work with a particularly internally contentious product and development team. We were concerned they hadn't discussed any grit during the interview process, only happy paths. We call them up and say, tell us about a time you fought with someone and lost. They respond, "uh, am I going to have to actually fight people in this job?" No, your PM might be a real pain in the ass and we can't seem to get them replaced so you're going to have to stick up for yourself, and we'd like to hear about a time you did that and it still didn't work out for you, and what happened next. And, they did, and we hired them, and they worked out well.

<comment><USER6> props to you for being honest with the candidate about what they should expect and letting them know that your team wanted to hire someone that would succeed in the role, knowing it's particular challenges.

</comment></comment><comment><USER7> Take a queue from politicians. Use the opportunity to tell one of your stories/talking points. You should have at least 3 fun/informative stories about yourself/life/experience prepared. Make sure not to answer the question directly and up front like "I failed when...". Instead imply your answer in the middle of the story.

</comment><comment><USER8> I feel these type of questions are hazing to make sure you prepared for the interview. In any case prepare and blast through them.

It's all about turning the negative into a positive.

The worst thing my manager said about me is my code is sometimes unclear. So I read code complete and now produce much better code (that's the outline but you need details when probed of course.)

My biggest weekness is perfectionism that leads to missing deadlines. Ive been working on this and now ive reduce the time to deliver and usually meet my estimates.

Etc.

<comment><USER9> _My biggest weekness is perfectionism_

I would never give that answer. It's as trite and predictable as college entrance essays about how a volunteer trip overseas was the most impactful thing in your life.

<comment><USER10> you can say essentially the same thing and re-word it in such a way that it is less trite:

"my biggest weakness is that i prioritise code quality, completeness and long- term maintainability above all other concerns, without understanding the broader engineering or business context where sometimes an ugly short-term hack or ignoring edge cases may be necessary to get a release out the door on schedule for a marketing campaign, or temporarily mitigate a production issue, or allow a project to be delivered on budget"

note: this is clearly still a weakness in an engineer, ideally you also need to talk through how you recognised this issue, what resources you used to change your behaviour and improve your performance in time or budget constrained situations, and what the result was...

<comment><USER9> My current reply would be two fold.

1\. "My biggest weakness is front end UI/UX. If you are expecting a great UI person, I'm not your guy."

The purpose of that answer is to both show self awareness and self select out of jobs I know that I won't be a good fit for.

2\. "When I was a team lead/architect at the company before my current one, I realized I had X, Y, Z weaknesses even though I accomplished what they hired me for before the company changed directions. I 'self demoted' and found a company where I could fill in those skills gaps and strengthen those areas. Now as I pursue new challenges, I'm sure I will discover new weaknesses, but just as before, I have a methodical method for learning new soft and hard skills by doing $X"

<comment><USER8> > 1\. "My biggest weakness is front end UI/UX. If you are expecting a great UI > person, I'm not your guy."

Clever, but I assume you've already selected out of such jobs before you hit the interview. So you are in an interview for a back end job.

Which means it's not much different from saying "My biggest weakness is Enterprise sales, if you need someone to land you that million dollar contract, I'm not your guy."

I think my perfectionism example is better because it is a genuine weakness. I have shown that it leads to things taking too long which is a real issue for companies, and also that the candidate is working on addressing it (so they want to improve).

> 2\. "When I was a team lead/architect at the company before my current one, > I realized I had X, Y, Z weaknesses even though I accomplished what they > hired me for before the company changed directions. I 'self demoted' and > found a company where I could fill in those skills gaps and strengthen those > areas. Now as I pursue new challenges, I'm sure I will discover new > weaknesses, but just as before, I have a methodical method for learning new > soft and hard skills by doing $X"

This is effective, as long as you can answer the follow up questions about why you had to leave that company and not move within it, etc.

<comment><USER9> _Clever, but I assume you 've already selected out of such jobs before you hit the interview. So you are in an interview for a back end job._

I always use my network of trusted external recruiters. You often see postings for "full stack developers", but once you actually talk to the recruiters, you find out that the hiring company would love that in theory, but they will take someone who is good on the back end. Also, I have some front end experience and I can usually get through an interview, but my fear is that I'll interview well enough to set false expectations.

 _This is effective, as long as you can answer the follow up questions about why you had to leave that company and not move within it, etc._

I kind of hinted at it when I said "the company changed direction". They decided that they "didn't want to be a software company" and they were moving toward trying to fit everything into none custom off the shelf solutions and hire contractors (like WorkDay contractors) to customize solutions. They wanted to turn us into a ticket resolving/report writing department.

Even while I was lead we were mostly using software developer contractors.

</comment></comment></comment></comment></comment></comment><comment><USER11> In my experience, its best to drive the discussion into technical areas that are so interesting that the interviewer runs out of time to ask their dumb canned questions, including the "tell me about a time" questions (and hopefully also the whiteboarding questions). For example, if there's a new or emerging tool or technology that you're familiar with but the interviewer doesn't know much about, try and drive the discussion in that area. They'll likely be very curious to know if the new tool/tech can be useful to them, and will want to know all about your experience with it, giving you a chance to brag about areas where you're strong (or at least, stronger than them).

</comment><comment><USER12> Look just tell them what they want to hear. Ask stupid questions, get stupid answers. They're probably not even listening.

They want to know if you're interested in self-improvement, and how well you get along with people.

> "what's the worst thing your previous manager would say about you"

I have too much free time because I get things done so fast, I am always asking for more work, or making code comments and improvements.

> "tell me about a time when you failed"

I was building a notification system for my previous company, and began using RabbitMQ. After my last manager realized what I was building he moved me on to the team to build out a core internal event driven architecture.

<comment><USER13> This is the worst possible advice.

It shows you are a narcissist, incapable of learning from experience, incapable of accurate self-analysis (identifying weaknesses or areas of improvement).

It also shows you struggle with soft skills, because you have completely misunderstood the point of these questions and are treating them as an opportunity to 'win' the question or 'beat' the interviewer. You are showing that appearing clever is more important than being smart.

You think the questions are beneath you, you think honesty is beneath you, and that interactions between people are defined only by the letter of the law and not its spirit. You know very well the intent of the questions, but you play games with your answers?

No one is looking for an answer here that says "I have no weaknesses and have never failed". Which would be a better and more honest answer than the "clever" answers you are suggesting.

You might as well be saying "I decline to answer because I am too smart to answer this question. Where's my signing bonus, will I be receiving it today, before the end of this interview? Because I have won, yes? I won the question by giving a non answer that makes me look great, so I am the cleverest! Therefore you must make me an offer immediately, yes? Because I am so clever! I beat the test! I beat YOU!"

a/k/a "no hire".

Follow this advice if you wish to remain unemployed or (arguably worse) hired by an idiot, into a team of fellow idiots. Which if you answer this way is where you belong.

<comment><USER12> literally the second thing I said summarized your entire rant.

Paraphrased again: > They want to know you are interested in self improvement

I am merely helping the guy out, to point out that the super negative question shouldn't get to his head.

You, my numeric friend, 4ec0755f5522, should try being less negative. You just made a lot of super negative assumptions:

> "idiot this, idiot that"

what's the saying, there's a million ways to skin a cat? Morbid as it may be, there's many ways to solve a problem and answer a question, and some ways may resonate better with some people. You have no idea what company or role the guy is applying for, or what they are looking for. Maybe they are looking for a super douche sales guy, in which case they need someone who is pure ego and arrogance. Everyone has a place in society.

Life isn't about being perfect all the time. Sometimes if someone is giving you a bs question, they might know it, and they might expect a bs answer back, simply because it's fun.

</comment></comment></comment><comment><USER14> 1 Everybody fails. Not everybody learns from failure.

2 A manager will say at least one negative thing about you. Not everyone grows from criticism.

Your goals: 1) You can recognize your failures, take responsibility for your failures, fix your failures, and learn from them

2) You can take constructive criticism, recognize that criticism is not personal, and use it to improve yourself.

The person you do NOT want to be is the one who blames others, sees not fault in themselves, and never grows.

</comment><comment><USER15> I have no idea how to deal with it but I sympathize with your problem. It almost feels like the interviewer is trying to tilt you just to measure how you will react. Like if you're on a date and the girl says "So bjourne, what is the worst thing your previous girlfriend would say about you?" You'd probably think that she is a psycho and stay away but if an recruiter asks it, it's ok.

<comment><USER6> Also, there are a number of responses you could say to your date that would not be okay in a job interview.

</comment></comment><comment><USER16> Don't take it too literally. If the worst isn't easy to talk about, choose your second worst.

When I get this question, my problem is I'm too optimistic (possibly righteous, narcissistic, idk?). I think this is because I go easy on myself and others for mistakes and just try to avoid creating or being in the situation.

</comment><comment><USER17> Answer in a way to turn things in your advantage.

"tell me about a time when you failed"

Failed to meet deadline (which you never do) that one time where you take initiative to rewrite spagetti legacy code from scratch. Later on everyone thanked you for that. But yes - first time it looked like a failure.

"what's the worst thing your previous manager would say about you"

Manager was pedantic asshole and insisted on doing thing his way. You did it your way and succeeded making manager unhappy because you took a risk and challenged the stigma jeopardizing his role.

</comment><comment><USER18> Sound genuine. Or better yet be genuine. Share openly. Everybody has setbacks at some point, so don't get defensive about it.

Remember in the long run, the hard times are best because that's when you grow.

</comment><comment><USER19> There is probably much better advice out there, but if they are throwing an oldie "time you failed" or "tell me about your weaknesses" question at me, I generally just reply with the (equally as old) "Once I failed to hit a deadline because Im a perfectionist and X wasn't quite right. I've since learned to better manage my efforts on key priority areas".

The "perfectionist" route is a fairly adaptable answer for any of the "state a negative about yourself" trap cards.

<comment><USER20> The general approach can be okay but don't call yourself a perfectionist at a job interview.

<comment><USER19> Hasn't had any negative impact in any of the interviews I have said it in after they ask for a weakness, but sure. What would you say instead?

</comment></comment></comment><comment><USER21> Just tell the truth, and don't complain about your old boss or your former colleagues.

</comment><comment><USER22> No comment beyond that's a terrible title that tells me nothing about what you're asking

<comment><AUTHOR> Fixed - thanks!</comment></comment>


Youtube-dl - <AUTHOR> https://rg3.github.io/youtube-dl/ <comment><USER2> It's very nice to see a project I started reach the front page of HN.

I remember starting the project around 2006. Back then, I had a dial-up connection and it wasn't easy for me to watch a video I liked a second time. It took ages. There were Greasemonkey scripts for Firefox that weren't working when I tried them, so I decided to start a new project in Python, using the standard urllib2. I made it command line because I thought it was a better approach for batch downloads and I had no experience writing GUI applications (and I still don't have much).

The first version was a pretty simple script that read the webpages and extracted the video URL from them. No objects or functions, just the straight work. I adapted the code for a few other websites and started adding some more features, giving birth to metacafe-dl and other projects.

The raise in popularity came in 2008, when Joe Barr (RIP) wrote an article about it for Linux.com.[1] It suddenly became much more popular and people started to request more features and support for many more sites.

So in 2008 the program was rewritten from scratch with support multiple video sites in mind, using a simple design (with some defects that I regret, but hey it works anyway!) that more or less survives until now. Naturally, I didn't change the name of the program. It would lose the bit of popularity it had. I should have named it something else from the start, but I didn't expect it to be so popular. One of these days we're going to be sued for trademark infringement.

In 2011 I stepped down as the maintainer due to lack of time, and the project is since then maintained by the amazing youtube-dl team which I always take an opportunity to thank for their great work.[2] The way I did this is simply by giving push access to my repository in Github. It's the best thing I did for the project bar none. Philipp Hagemeister[3] has been the head of the maintainers since then, but the second contributor, for example, was Filippo Valsorda[4], of Heartbleed tester[5] fame and now working for Cloudflare.

[1] [http://archive09.linux.com/articles/114161](http://archive09.linux.com/articles/114161) [2] [http://rg3.name/201408141628.html](http://rg3.name/201408141628.html) [3] [https://github.com/phihag](https://github.com/phihag) [4] [https://github.com/filosottile](https://github.com/filosottile) [5] [https://filippo.io/Heartbleed/](https://filippo.io/Heartbleed/)

<comment><USER3> Remarkably, YouTube makes scripting downloads very easy. The script below needs only sed and some http client and it has worked for years. I have only had to change it once when there was a change at YouTube; the change was very small.

       # this script uses sh, sed, awk, tr and some http client
       # here, some http client = tnftp
       # awk and tr are optional

       # wrapper for tnftp to accept urls from stdin
       ftp1(){
       while read a;do
       ftp ${@--4vdo-} "$a"
       done;}

       # uniq
       awk1(){ awk '!($0 in a){a[$0];print}' ;}

       # some url decoding
       f1(){
       sed '
       s,%3D,=,g;
       s,%3A,:,g;
       s,%2F,/,g;
       s,%3F,?,g;
       s/^M
       //g;
       #  ^ thats Ctrl-V then Ctrl-M in vi
       '
       }

       # remove redundant itags
       f0(){
       sed -e '
       s/&itag=5//;t1
       s/&itag=1[78]//;t1
       s/&itag=22//;t1
       s/&itag=3[4-8]//;t1
       s/&itag=4[3-6]//;t1
       s/&itag=1[346][0-9]//;t1
       ' -e :1
       }

       # separate urls
       f2(){
       sed '
       s,http,\
       &,g'
       }

       # remove unneeded lines
       f3(){
       sed '
       #/^http%3A%2F.*c.youtube.com/!d;
       /^http%3A%2F.*googlevideo.com/!d;
       /crossdomain.xml/d;
       s/%25/%/g;
       s,sig=,\&signature=,;
       s,\\u0026,\&,g;
       /&author=.*/d;
       '
       }

       # separate cgi arguments for debugging
       f4(){
       sed '
       s,%26,\
       ,g;
       s,&,\
       ,g;
       '
       }

       # remove more unneeded lines
       f5(){
       sed '
       /./!d;
       /quality=/d;
       /type=/d;
       /fallback_host=/d;
       /url=/d;
       /^http:/!s/^/\&/
       /^[^h].*:/d;
       /^http:.*doubleclick.net/d;
       /itag.*,/d;
       '
       }

       # print urls
       f6(){
       sed 's/^http:/\
       &/' | tr -d '\012' \
       |sed '
       s/http:/\
       &/g;
       '
       }

       f8(){
       sed 's/https:/http:/'
       }

       FTPUSERAGENT="like OSX"

       case $# in
       0)
       echo|$0 -h
        ;;
       [12345])
       case $1 in

       -h|--h)
       echo "url=http[s]://www.youtube.com/watch?v=..........."
       echo usage1: echo url\|$0 -F \(get itag-no\'s\)
       echo usage2: echo url\|$0 -g \(get download urls\)
       echo usage3: echo url\|$0 -fitag-no -4o video-file
       echo N.B. no space permitted after -f

        ;;
       -F)
       $0 -g \
       |tr '&' '\012' \
       |sed '
       /,/d;
       /itag=[0-9]/!d;
       s/itag=//;
       /^17$/s/$/ 3GP/;
       /^36$/s/$/ 3GP/;
       /^[56]$/s/$/ FLV/;
       /^3[45]$/s/$/ FLV/;
       /^18$/s/$/ MP4/;
       /^22$/s/$/ MP4/;
       /^3[78]$/s/$/ MP4/;
       /^8[2-5]$/s/$/ MP4/;
       s/.*?//;
       '|awk1
        ;;

       -g)
       while read a;do
       n=1
       while [ $n -le 10 ];do
       echo $a|f8|ftp1||
       echo $a|f8|ftp1 &&
       break
       n=$((n+1))
       done \
       |f2|f3|f1|f0|f4|f5|f6|f1|sed '/itag='"$2"'/!d'
       done
        ;;

       -f*)
       while read a;do
       n=1
       while [ $n -le 10 ];do
       echo $a|$0 -g ${1#-f} |ftp1 $2 $3 $4 $5 ||
       echo $a|$0 -g ${1#-f} |ftp1 $2 $3 $4 $5  &&
       break
       n=$((n+1))
       done
       done
        ;;

       esac
       esac

There are separate scripts for extracting www.youtube.com/watch?v=........... urls from web pages to feed to this script.

<comment><USER4> The problem is that this only works for some YouTube videos (for example it will fail for basically all VEVO videos), not to mention maintainability issues.

<comment><USER3> I had to look up what "VEVO" was. A joint venture of several major record labels and Google launched in 2009.

Personally I have no need for "VEVO" videos. Nor do I ever encounter VEVO youtube urls posted to websites, like HN. I wonder why?

As for maintainability, I beg to differ. The raison d'etre for this script arose out of frustration that early YouTube download solutions, e.g. gawk scripts, clive, etc., kept breaking whenever something at YouTube changed. I got tired of waiting for these programs to be fixed, if that ever happened.

I can fix this 164 line script faster if YouTube changes something than waiting for a third party to fix something they developed that is far more complex. Moreover, it does not rely on Python. Is there something wrong with DIY?

I see someone posted a link in this thread to another 208 line script, yget, that uses sed and awk. This further demonstrates the relative simplicity of downloading YouTube videos.

<comment><USER5> > _Personally I do not watch "VEVO" videos but I am curious what they are._ //

Go to youtube.com, you may need to scroll but most unlikely, bam! "VEVO" video with x-million views: it's a music video promotion brand.

Actually it's not globally promoted so outside of Western Europe and USA I'd guess you don't get VEVO vids so much?

According to [https://www.youtube.com/watch?v=5zs1ClgqhLw](https://www.youtube.com/watch?v=5zs1ClgqhLw) their 100th most viewed video has 200 million views. Top 10 are all above 600 million.

They're quite a big brand.

<comment><USER3> Interesting.

An alternative to goofing around on the youtube.com web site, scrolling constantly and getting hit with advertising and endless lists of "related" videos is to search and retrieve youtube urls from the command line via gdata.youtube.com.

</comment></comment></comment></comment></comment></comment><comment><USER6> Despite its name, youtube-dl doesn't just download from YouTube but from a ton a different sites as well [1]. The rate in which this project keeps up with changes is incredible.

[1]: [https://github.com/rg3/youtube- dl/tree/master/youtube_dl/ext...](https://github.com/rg3/youtube- dl/tree/master/youtube_dl/extractor)

<comment><USER7> It seems like quite a modern success story for the classic "Cathedral and the Bazaar" model of open source development structure and motivations.

As I recall, it was originally written by one person (Ricardo Garcia) in 2008 and worked only on YouTube using (by later standards) relatively simple heuristics to find the URL to extract the video. But it's catalyzed an explosion of interest in every aspect of the problem: tracking changes to the HTML of the video sites, adding support for more video sites, figuring out indirection and parsing through multiple pages and HTML objects, making the tool much more multiplatform and easier to install and update...

It's attracted hundreds of contributors (many of them motivated by a personal desire to be able to use the tool on a different site, or to fix a bug that was preventing them from downloading video in a particular rare case) and maintained an incredibly rapid pace of development.

<comment><USER8> I'm not sure if "leeching copyrighted content" was the kind of motivation that Eric Raymond had in mind for future open source projects when he wrote _Cathedral and the Bazaar_.

<comment><USER9> Yeah, because at the core of human civilization lies a respect for copyright, a BS notion that was developed for exploiting the restrictions of (analog) physical formats for profit...

<comment><USER10> Copyright law was developed in the 1700s precisely to prevent people from exploiting the limitations of physical formats for profit.

It's the opposite of what you've stated.

Authors, composers and publishers needed protection against cheap printing presses that would just print anything that was popular and flog it in the marketplaces.

<comment><USER9> > _Authors, composers and publishers needed protection against cheap printing > presses that would just print anything that was popular and flog it in the > marketplaces._

The limitations I mention are the difficulties and cost of the printing itself.

What authors wanted was to restrict who can print their work -- but it's not true that authors "needed protection" because printing presses started appearing.

That makes it sound like authors were paid for the work until those "cheap printer" pirates appeared. But on the contrary it was the invention of the printing presses themselves that gave authors an industry in the first place -- for millenia authors just wrote for free.

<comment><USER11> The reason the industry of paid authoring could develop is because of copyright. Without it, all the value of the new printing industry would have accrued to the printers, and none to the authors.

<comment><USER9> Yeah, and culture would be free.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> YT videos lag a lot when I stream them directly in VLC 2.2.0, (`vlc <youtube link>`) and some protected videos don't play at all, so I often use this:

        ytplay() { youtube-dl "$1" -o - | vlc - }

As a side benefit it of course also allows you to instantly watch stuff from all the other sites YT-DL supports :)

<comment><USER13> May I ask what the program "ytplay" is? Google doesn't guide me.

<comment><USER14> It's a function name that renekooi would have made up to run the command following it without having to type it all out every time.

</comment></comment></comment><comment><USER15> Quick protip for those wondering, the simple command to download an entire youtube channel is like so:

$ youtube-dl -citw ytuser:LastWeekTonight

I downloaded a channel with 121 videos, 4.4 gigs, took 26 minutes, so 2.8MB/s average. Curious if the Youtube people will shrug it off and free the beer or rate limit or more aggressively combat this.

Also, to get the total number of supported sites:

$ youtube-dl --extractor-descriptions|wc -l

466 (wow)

As this can run on anything with Python, I guess that includes Android[0], iOS[1], Windows Phone[2], heck even Blackberry[3]??

[0] [https://python-for-android.readthedocs.org/en/latest/](https://python- for-android.readthedocs.org/en/latest/)

[1] [https://code.msdn.microsoft.com/windowsapps/using-python- on-...](https://code.msdn.microsoft.com/windowsapps/using-python-on- windows-e914e5c5)

[2] [http://pythonforios.com/](http://pythonforios.com/)

[3] [http://forums.crackberry.com/blackberry-z10-f254/blackberry-...](http://forums.crackberry.com/blackberry-z10-f254/blackberry-10-os- has-built-python-intepreter-d-798856/)

Thanks pmoriarty for submitting this. Awesome and I'm just getting started poking around with it. Makes me really want to learn Python, seems that's what all the fun stuff[4] is coded in.

[4] [http://motoma.io/pyloris/](http://motoma.io/pyloris/) :)

<comment><USER16> Downloading an entire channel is exactly how I downloaded the How to Start a Startup class from youtube. For the interested:

      $ youtube-dl -citw ytuser:UCxIJaCMEptJjxmmQgGFsnCg

<comment><USER15> fredoliveira: Eyeballing those lectures, which are enormous, youtube-dl is helping you more easily teach people to make their lives and careers better, instead of just downloading Taylor Swift songs. So youtube-dl is indeed being used for Good.

</comment></comment></comment><comment><USER17> This may sound surprising but via youtube-dl I bought more music than before. If I find a new band that I might like, I search for Youtube videos first. The non-official videos often show just the cover of the CD or some useless slide show, so I extract the audio to have it in my playlist. Once I am decided that I like the music I had over to Bandcamp or Amazon to buy the mp3s. As an example: I lately bought four digital cds from progmetal act Redemption because someone upped their cd 'This Mortal Coil' to Youtube.

</comment><comment><USER18> A very nice utility to have installed.

It can also convert a video to an mp3:

youtube-dl --extract-audio --audio-format mp3 [https://www.youtube.com/watch?v=OKbtC223e30](https://www.youtube.com/watch?v=OKbtC223e30)

<comment><USER19>
      > youtube-dl -F OKbtC223e30
      139         m4a       audio only  DASH audio   49k , audio@ 48k (22050Hz), 525.88KiB (worst)
      171         webm      audio only  DASH audio  129k , audio@128k (44100Hz), 1.27MiB
      140         m4a       audio only  DASH audio  129k , audio@128k (44100Hz), 1.36MiB
      172         webm      audio only  DASH audio  176k , audio@256k (44100Hz), 1.78MiB
      141         m4a       audio only  DASH audio  255k , audio@256k (44100Hz), 2.71MiB

Just use youtube-dl -f 141 to direct download just the audio from YouTube. I always use this to download songs.

<comment><USER4> While -f 141 is of course perfectly fine, may I suggest -f bestaudio ? That should work fine for non-YouTube sites (soundcloud or so), and will get you a better version should YouTube one day add it. If you really prefer 141, you can also use -f 141/bestaudio to fall back to bestaudio if 141 is unavailable.

<comment><USER19> That'll come in handy. I always used -F to find out what is highest # for downloading audio.

</comment></comment></comment></comment><comment><USER20> This is the best video downloader. It has downloaded videos from every site I've thrown at it. It even downloaded videos from Comedy Central!

Edit: [https://rg3.github.io/youtube- dl/supportedsites.html](https://rg3.github.io/youtube-dl/supportedsites.html)

</comment><comment><USER21> I use this all the time and It's really great.

I have messed around with one sketchy youtube downloader or another for years until I found this.

Oh, and it's on brew!

</comment><comment><USER22> I've used this quite extensively. It's less critical for YouTube now that almost all YouTube videos work with the HTML5 player, but it helped quite a bit when every other video required Flash. Still necessary for many third- party sites as well.

It'll also download an entire playlist, and add sequential numbers at the beginning (with the -A option).

<comment><USER23> I've also used it for downloading all possible formats for a video when debugging problems that only seem to be there in some resolutions.

<comment><USER22> Yeah, the format enumeration works quite well too. I've used it to download the "original" format for videos available in higher-than-1080p resolutions, as well as using the --extract-audio option.

</comment></comment></comment><comment><USER19> No one pointed out that it has a do-whatever-you-want license. This is the thing which bothers me most. There are gazillions of shitty youtube downloaders (paid, free and adware supported) out there that people still use and the code is being powered by the work of open source developers.

<comment><USER24> I don't think the license would change anything, even if they chose something like AGPL. The sites would either ignore it (hard to prove infringement) or put a small link with their source. They don't rely on being proprietary anyway.

<comment><USER19> Well it'll atleast do something to limit creations and who knows if the copyright attribution was placed on many projects, youtube-dl would have received more attention, I'm surprised it wasn't already posted to HN.

It's very nice though for legitimate developers.

</comment></comment></comment><comment><USER25> I use this to replace noisy audio on a smartphone recording of dancing lessons with a high-quality version from a youtube video, automatically: [http://youtu.be/AVIHpaNQLS0](http://youtu.be/AVIHpaNQLS0)

<comment><USER26> Wow that's quite an impressive tool!

I mean I understand that all the separate steps are stuff we've seen is easily possible nowadays, but putting it together in a single UI makes it (roughly) 3000x as useful!

I want to try it, anyone had luck compiling it for Linux?

 _EDIT /update:_ Well I gave it a try, grabbed QtCreator, loaded the project, not much luck. Some issues with the "phonon" library, it seems. I'm not very good with getting C++ stuff to work when it gives build errors. I did spend about half an hour fiddling and googling error messages, but now it's time to give up, sorry :)

I'm writing this update to let you know that one of the errors I did manage to fix, is that Windows has case-insensitive paths/filenames, while Linux does not. Apparently the path for the phonon library is lowercase, so you should `#include <phonon>` lowercase. I'll try to leave a Github issue about this.

That didn't help much (complaints about the State enum in soundfix.h) which I tried to fix by also putting `#include <phonon>` in the soundfix.h. I'm not sure if that was right _at all_ , but it did seem to fix that particular problem. As a result I was greeted with a whole bunch of other (I think unrelated?) errors about some types not being strictly compatible or something. That is where I check out until I know more about C++, decided it had been long enough, and just writing you a little message to let you know how it went.

However, it made me install and try QtCreator, something that I was meaning to do anyway. So that's a win :)

<comment><USER25> I'll try compiling it on qt5/linux as well, thanks for tracking down the phonon issues!

</comment></comment></comment><comment><USER27> There's also a GUI wrapper for it: [https://github.com/MrS0m30n3/youtube-dl- gui/](https://github.com/MrS0m30n3/youtube-dl-gui/)

And a php web app that uses youtube-dl as a a backend: [https://github.com/Rudloff/alltube/](https://github.com/Rudloff/alltube/)

</comment><comment><USER28> Youtube-dl is the biggest pre-built thing I use in GifMachine[0] after ffmpeg, and I've used it in innumerable projects since then. I love youtube-dl, it's fantastic!

[0] - [http://gifmachine.xwl.me/](http://gifmachine.xwl.me/)

<comment><USER29> Nice gifmachine - Thank you!

</comment></comment><comment><USER30> Is there a plugin version of this program, which would dynamically change any (supported) flash video reference to an HTML5 video tag? That way I can get rid of flash completely.

<comment><USER31> Don't know about this program, but I use this on OS X Safari:

[http://www.verticalforest.com/youtube5-extension/](http://www.verticalforest.com/youtube5-extension/)

Helped to get rid of Flash, although I have Chrome in my programs folder for sites that are still flash only.

</comment></comment><comment><USER32> I've setup this function in my bashrc to check for the latest build if the new one is more than 24 hours old and then run it with arguments supplied:

function youtube-dl {

            exe=$HOME/bin/youtube-dl
            link=https://rg3.github.io/youtube-dl/download.html
            url=$(curl -s $link |grep ">sig<" |head -1|sed -e 's/href="/|/g' -e 's/">/|/g'|cut -d"|" -f2)

            fetch=N
            if [ -s "$exe" ] ; then
                    ts=$(date "+%s")
                    yts=$(stat -c "%Y" $exe)
                    [ $(( ($ts-$yts)/(60*60) )) -gt 24 ] && fetch=Y
            else
                    fetch=Y
            fi

            if [ "$fetch" = "Y" ] ; then
                    url=$(curl -s $url |grep ">sig<" |head -1|sed -e 's/href="/|/g' -e 's/">/|/g'|cut -d"|" -f2)
                    echo "Fetching [$url] and deploying to [$exe]"
                    curl -s $url -o $exe
                    chmod a+x $exe
            fi
            [ -z "$@" ] && $exe --help || $exe $@
    }

<comment><USER33> sounds like a job for a package manager ;-)

</comment></comment><comment><USER34> youtube-dl gets updated very frequently and the version that comes with your distribution (ie, ubuntu) usually is out of date and doesn't work often on most sites. So its better to download from the source and update it using "youtube-dl -U"

<comment><USER22> That's true for the version in Debian stable as well. The package in unstable tends to pick up upstream updates pretty quickly, though, especially when they're needed to fix site support.

</comment></comment><comment><USER35> The amount of time it takes to keep up with all of the changes big sites make is impressive.

At some point I decided to write something similar in Ruby ( [https://github.com/rb2k/viddl-rb](https://github.com/rb2k/viddl-rb) ) and I'm kind of ashamed of how broken things are from time to time.

Video hosting sites don't have APIs and reverse engineering the sources for the videos is like shooting at a moving target.

So kudos for leading that project :)

</comment><comment><USER13> OK. I just realized this does something really cool. I've been troubled with 1080p videos as they no longer contain audio. They're separated and YT uses DASH to join the audio+video stream.

youtube-dl seems to solve this: [https://github.com/rg3/youtube- dl/issues/2165](https://github.com/rg3/youtube-dl/issues/2165)

</comment><comment><USER36> I just use quvi for that.

Example:

        quvi -vm --format $format "$url" --exec 'wget %u -O %t.%e'

instead of $format put any format that the video supports. To query them, use:

        quvi --query-formats "$url"

So it's going to be something like:

        quvi -vm --format fmt43_360p "$url" --exec 'wget %u -O %t.%e'

And to extract audio from the result you can use:

        avconv -i something.webm -vn -acodec copy something.ogg

Youtube however is switching away from fixed video files to separate streams to be used with MSE. You can note that higher resolution video is not available the old way. So downloading that won't be so straightforward.

<comment><USER37> youtube-dl supports muxing the separate video and audio streams from YouTube. You just need a recent ffmpeg/avconv and youtube-dl -f bestvideo+bestaudio.

<comment><USER13> This is way cool as YouTube separates video and audio streams for 1080p (and up) and then joins them with DASH.

</comment></comment></comment><comment><USER38> If you don't want to install anything, just use [http://savedeo.com](http://savedeo.com) for almost any video or [http://auderio.com](http://auderio.com) for Music from Youtube

<comment><USER39> Thanks, been using listentoyoutube.com but auderio.com is definitely better (less annoying popup stuff + required clicks)

<comment><USER38> And way faster :)

</comment></comment></comment><comment><USER40> How does it compare to get_flash_videos, which is available in Ubuntu repository? [https://code.google.com/p/get-flash- videos/](https://code.google.com/p/get-flash-videos/)

<comment><USER41> get-flash-videos supports Hulu; I've been meaning to try it out [1] because flash is Satan's anus. But I'd still keep youtube-dl for downloading & converting YouTube videos.

[1] [https://gitorious.org/get-flash-videos- plugins/pages/Hulu](https://gitorious.org/get-flash-videos-plugins/pages/Hulu)

</comment></comment><comment><USER42> Is there a way of bypassing geoblocking via youtube-dl directly?

<comment><USER43> Well, you can get a cheap VPS in the US, or UK, or wherever the content is available, and run youtube-dl there. You can also use a proxy[1]. Unfortunately, I think socks proxies are not supported yet[1].

[1] [https://rg3.github.io/youtube- dl/documentation.html#d5](https://rg3.github.io/youtube- dl/documentation.html#d5)

[2] [https://github.com/rg3/youtube- dl/issues/402](https://github.com/rg3/youtube-dl/issues/402)

</comment></comment><comment><USER44> There are lots of projects to do this, and this one has been around for quite some time. Is there any context as to why to post this now?

<comment><USER45> If it's on the front page, some people have upvoted it.

Take a look at the "new" section!

</comment></comment><comment><USER46> Till now I didn't know that other Sites are supported :D wow really impressive site support! Thx for maintaining this nice tool :)

</comment><comment><USER47> I've used this one for so many things and it's awesome.

I've downloaded a whole youtube channel, downloaded videos as mp3's and what not.

I have couple of aliases set in my .zshrc too :)

mp3dl() { youtube-dl --extract-audio --audio-format mp3 $1 }

root@haseebr7 ~ mp3dl <youtube_video_url>

and bam i have the mp3 download. I no longer have to visit shitty ad infested websites to do these kind of things.

<comment><USER48> I do the same, but have ethe -K option also whch keeps the original DLd video.

</comment></comment><comment><USER49> It seems like this functionality should be part of YouTube itself, at least as an option settable by the uploader.

</comment><comment><USER50> youtube-dl rocks. One of my favourite features: youtube-dl -F

</comment><comment><USER51> Yget is an alternative I've found to be more reliable. It's just for youtube though, and doesn't support all the things (such as bypassing age restrictions).

[http://repo.hu/projects/yget/](http://repo.hu/projects/yget/)

<comment><USER26> How is it more reliable?

<comment><USER51> Back when I used youtube-dl, it seemed like Google changed something about Youtube every couple months and youtube-dl would break. Yget survived many more of these changes.

Getting youtube-dl to run has been a pain for some reason, for example I always seem not to have the right version of Python available. Yget doesn't depend on such volatile tools.

</comment></comment></comment><comment><USER52> This appears to be the only option for Free Software access to soundcloud.com.

<comment><USER53> It's easy to just throw the streams of Soundcloud songs against whatever music player you have at hand. MPD even has a playlist plugin, which correctly handles basic soundcloud.com urls and handles all the API stuff for you.

</comment></comment><comment><USER54> On OS X, you can use afplay(/usr/bin/afplay) to play those downloaded videos' music(in a headless player). This is pretty useful if you listen to youtube music at work.

</comment><comment><USER55> If I can lazyweb on this for a moment, is there a good script out there that can use this to sync/download your entire watch later queue?

<comment><USER4> youtube-dl can do that out of the box, try

youtube-dl --username pimlottc :ytwatchlater

If you get a problem, please file a bug report at [https://yt- dl.org/bug](https://yt-dl.org/bug) . Thanks!

</comment></comment><comment><USER56> I really like this tool and it has been very reliable for me so far. Thanks for sharing the code.

</comment><comment><USER57> I'm just using [https://github.com/np1/mps- youtube](https://github.com/np1/mps-youtube)

</comment><comment><USER42> Oh man, I had that ages ago and used it frequently! It stopped working, if this is working again that's awesome :-)

<comment><USER9> You should update it often, as internal formats, urls, options etc of the sites change.

</comment></comment><comment><USER58> What's so special about this?

<comment><USER59> Probably lots of hackers want to be freed from the bondage of youtube.com while still retaining access to the wealth of content available there.

<comment><USER58> ? hence the "so special" in my question. There are plenty of these tools already and it's been so for long time. As a hacker you should know. So again, what's so special about this?

</comment></comment></comment><comment><USER60> It would be nice if it supported Netflix. There's some difficulties with this, but a discussion about it here: [https://github.com/rg3/youtube- dl/issues/1564](https://github.com/rg3/youtube-dl/issues/1564)

</comment><comment><USER61> I just use chrome extension</comment>


Show HN: A script to check if Benford's law applies to GitHub stats (it does!) - <AUTHOR> https://github.com/chpmrc/benfordslaw <comment><USER2> I'm curious about what an analysis of the COVID stats would show.

<comment><AUTHOR> Oh man...I'll check!

<comment><USER2> And...?</comment></comment></comment>


Why not keep it simple? - <AUTHOR> http://aflanagan.com/why-not-keep-it-simple/

<comment><USER2> I think the answer is that they don't want a crappy-looking, buggy product/service that doesn't do much.

Keeping it simple is great, and suggested, but making it useful and nice looking is also very important. Are you sure that it failed because it wasn't simple enough? Maybe it just wasn't filling the right need?

<comment><AUTHOR> Oh no, I'm not saying it failed because it wasn't simple enough. Not at all. It failed because we were unable to find a reliable revenue model for our users.

I'm also not saying that you shouldn't work for a nice looking site and a non- buggy service, just that those things can be continually improved as you go. You don't need them for launch.</comment></comment>


Big Data Is Useless Without Science - <AUTHOR> http://kaleidoscope.kontagent.com/2011/11/09/big-data-is-useless-without-science/

<comment><USER2> The author makes a good point. Unfortunately, many companies are led by people who just don't get data and its analysis and potential for driving change.

Many executives got to where they are by either lingering around for a long time and waiting for competent people to leave or by mastering the skills their company needed 10 years ago, but which are now irrelevant.

This is leading to ample opportunities for startups and growth companies to beat the living daylights out of bigger competitors. No amount of MBA-speak and powerpoint presentations can help you understand what to do with the data deluge. These leaders can try to hire smart analysts, but how are they going to be able to differentiate between data scientists who can make a real contribution and those who can't?

A lot of bigger more established companies won't make the cut, no matter how much data they accumulate.

There's ample opportunity to disrupt established markets with data driven strategies,but this disruption probably won't come from larger more established companies.

</comment><comment><USER3> While I agree that "science" (as defined in the article) is a very good way to make use of data, I do not think the data without the science is useless.

For one, if you collect the data now you can always apply the "science" later --as long as you have the data and can query it, you're always ready. This alone means that collecting data now could be useful even if you have no idea of what to do with it. Storage is relatively cheap, and you--or somebody else --might come up with a clever way to analyze it in the future.

Additionally, I think the author underestimates the potential of machine learning. I am by no means an expert in the field (I'm taking an AI class-- that has to count for something! ;) I hope) but even the simple techniques we've covered can get some interesting information without much domain knowledge involved. As ML evolves, I suspect there will be more and more technology that can find interesting trends and relationships in data regardless of what the data is actually modelling. The article does mention ML a bit, but I think it will be much more significant in the near future.

Ultimately, figuring out what questions to ask about data and harnessing human curiosity will give you much more than just hoarding data. I just think it's possible that sufficiently abstract and generic approaches in the near future will make it easier to get similar--or perhaps orthogonal--results using techniques from ML and AI. I firmly believe that hoarding the data without doing anything to it is still much better than not collecting it at all.

<comment><USER4> I agree with you that collecting data can be a good thing, and it can be analyzed at some later point. But in the case of online world, the lifetime of data is very small(this might not be for other industries). Trends change very rapidly, and using an old data for making inference might not be profitable if not a loss.

<comment><USER5> Actually that's exactly the opposite of true. In fact you -need- old data to validate that your mathematical model actually works by applying it to whatever historical timespan of data you are trying to derive intelligence from. At least if any sort of unit of time is applicable to any of your models.

Of course just sitting on the data doesn't do anyone any good at all..and randomly picking at stuff isn't likely to do much either unless you go in knowing what you want to find.

</comment></comment></comment><comment><USER5> I think his main point was that people are already hoarding but - like most knowledge - don't know what they're missing/clueless about without some good statistical analysts.

My employer has a whole department of these analysts and after working with them closely it is obvious to me that they are the most important / valuable to ourselves and customers asset in the whole company. Engineers can pick up on some of these things but these guys aren't just picking stuff out of their ass or anything. I'd have to agree with the author that any company storing or collecting any significant amount of data should give this stuff some serious thought.</comment>


The Surveillance Engine: How the NSA Built Its Own Secret Google - <AUTHOR> https://firstlook.org/theintercept/article/2014/08/25/icreach-nsa-cia-secret-google-crisscross-proton/

<comment><USER2> The downvoting on this one is disgusting. I suppose a lot of the "dudes" who built these privacy invading infrastructures lurk here and have downvoting rights.

<comment><AUTHOR> I had suspicions of foul play involving surveillance-related stories getting buried in the past, but after a discussion with dang, I think it's quite plausibly just a difficult topic to moderate with the tools that are available on HN. [0]

If no automatic moderation was applied, the front page would be filled with surveillance stories--many of which contain no real new information. So there's an automatic penalty. And it appears there's not a moderator who's following the surveillance story closely, so it can be difficult for them to determine what's newsworthy.

The system is clearly somewhat broken, but I don't think it's due to bad intent. You can help by flagging stories that add no value, and you can email email@example.com if you see a major story that is being penalized unfairly [1].

I think the mods are trying, but the system here wasn't really built to gracefully handle a topic that the community is so passionate about like this. Or even to communicate what's going on with moderation. I know a lot of us here naturally speculate about systems and like to get into them and improve them, but I'm fairly convinced that the best approach, at least at the moment and outside of new moderation tools, is an email to the moderators when stories shouldn't be penalized.

[0] [https://news.ycombinator.com/item?id=8059068](https://news.ycombinator.com/item?id=8059068)

[1] [https://news.ycombinator.com/item?id=8065601](https://news.ycombinator.com/item?id=8065601)</comment></comment>


Pardon Plea by Adolf Eichmann, Nazi War Criminal, Is Made Public - <AUTHOR> http://www.nytimes.com/2016/01/28/world/middleeast/israel-adolf-eichmann-holocaust.html <comment><AUTHOR> I submitted this because, ever since reading Arendt's Eichmann in Jerusalem, I've always thought about not only the tasks that I do, but who I do them for. Eichmann was a very talented engineer, but he used his talents to support one of the most brutal regimes in modern history.

I think it's particularly relevant for programmers today. Many of us are tempted by pay or prestige or excitement to do morally dubious tasks. And even though it feels safe to do things like collect data for the NSA or help build weapons for the US military, that history is filled with examples of people who "just did their job," but later learned a very harsh lesson that they should have considered more carefully who they were doing it for.</comment>


Minimalism: another boring product wealthy people can buy - <AUTHOR> https://www.theguardian.com/lifeandstyle/2017/mar/04/minimalism-conspicuous-consumption-class <comment><USER2> "there is a very strong capitalist-critical argument to be made about buying in more intentional and ethical ways, but color me shocked that very few of these minimalist troubadours ever really take things to an economic or class- based argument."

As long as they look critically at their consumerism and are voting with their wallet for ethical consumer goods, let them have their Instagram pictures. Even if they don't care about consumerism and are doing it for aesthetics, it shows people that you can think different about your consumption habits.

</comment><comment><USER3> Yes, this type of minimalism does exist, but the author seems to be discounting the possibility of a genuine focus on what matters to a person regardless of financial status.</comment>


Why CommonMark Isn't the Solution - <AUTHOR> https://ariabuckles.github.io/blog/common-markdown/ <comment><USER2> Ok, I had challenges with statements like this in the article:

 _All of this means that it is not possible to write a simple rule for when a paragraph (or other block element) ends._

Except there is a very simple rule, block elements are terminated with a blank line. The argument that changing

       line 1
       ---  < this is inserted between two blocks
       line 2

Really changes the 'line1' from a block element to a header element (its no longer followed by a block line) which has not a lot to do with parsing but more with lexical analysis. And then later the author reveals what is perhaps their actual thesis statement _" Markdown became so popular because it was simple and somewhat extensible."_ which I would agree with if the period had landed three words earlier in the sentence. :-)

CM is a nice spec for a "markdown like system" :-) with many of the ambiguities rubbed off the edges. Add tables and I'll totally switch from multi-markdown to this more 'elucidated' specification.

I'm really surprised that so many people are hung up on Atwood et alia trying to bring some explicitness here. I've been using Markdown(tm) derived systems for a few years now and have felt the pain of many of their points directly.

<comment><USER3> I think the explicitness is the one aspect which has been widely well- received. We all know the pain of an inexplicit Markdown, and CM did a fantastic job of addressing that.

Much of the frustration, at least from my eyes, is because CM attempts to incorporate too much of the mess that Md once introduced. I understand they did this in the interest of keeping compatibility with most "Markdown" documents. But they had the opportunity to fix things up, and really I wish they did.

The quintessential example here is the inclusion of HTML. By allowing html to be mixed in, Gruber really allowed a lot of flexibility and versatility, but that only makes sense if the content could be trusted and that it will only ever appear on the web. Tables are a great example of reasonable html usage- they are supported in both Md and CM. There are a lot more examples of unreasonable HTML usage, which I'd rather not enumerate.

These days, with the proliferation of Md as a widespread content language for public discussion across mediums, HTML doesn't belong. There are other cruft and ambiguities that CM could have cleaned up easily. CommonMark had the chance to improve this situation, but they went to great lengths to embrace it. If they had considered themselves as a distinct dialect instead of "the" standard, they may have gone a different route and written something more modern and clear.

</comment></comment><comment><USER4> The project is no called longer "Common Markdown". It's CommonMark [1].

From Atwood's announcement yesterday:

 _Edit: after a long and thoughtful email from John Gruber – which is greatly appreciated – he indicated that no form of the word "Markdown" is acceptable to him in this case. We are now using the name CommonMark._ [2]

Anyway, the reason the specification is complicated is because creating a regular grammar from a rich irregular grammar requires a lot of work. Things like reading ahead to distinguish between:

         paragraph 1
         %%%
         paragraph 2

and

         paragraph 1
         ---
         paragraph 2

The goal is to help companies like StackExchange and Github get out of the markup language business by solving a set problems that aren't really on the radar of individual bloggers using markdown, but that very much impact websites at scale.

The article is a case in point - StackOverflow can't be efficient if it's pushing an entire parser into the browser. It needs to degrade gracefully when javascript is off.

In general, the world will be better off when the canonical markdown documentation is not written with a tiny font on a grey background. Perhaps we'll even see markdown [or CommonMark] begin replacing RTF and BBCode more places.

[1]: [http://commonmark.org/](http://commonmark.org/)

[2]: [http://blog.codinghorror.com/standard-markdown-is-now- common...](http://blog.codinghorror.com/standard-markdown-is-now-common- markdown/)

<comment><USER5> Well, I guess now we've learned what it takes to get John Gruber to respond to an e-mail.

<comment><USER6> "Are the Yankees the best team in baseball?

We used to think so."

[http://commonmark.org](http://commonmark.org)

</comment></comment></comment><comment><USER7> There is another evaluation of Common/Standard Markdown by the author of vfmd: [http://roopc.net/posts/2014/eval-stmd](http://roopc.net/posts/2014/eval-stmd)

<comment><USER8> That's a very good writeup, and his points are important for anyone to consider before proclaiming support for the CommonMarkdown effort. I wish there were way more discussions like this of the technical merits of CommonMarkdown than the name.

While I'm very supportive of some kind of sane Markdown specification, it's pretty disappointing that those with the biggest audience to push a specification have created such an apparently poor one. And after 2 years! Spec-by-examples, especially for an inherently ambiguous and forgiving syntax like Markdown, should be known to be a bad idea.

</comment></comment><comment><USER9> I've tried to use Markdown for very simple formatting (to have two consecutive paragraph have a wrapper style, e.g. to have a note style) - and failed. I've tried various parsers, wrote my code hacks (also called: extensions), regex replacers whatnot, just to conclude, that even if I am able to do that, the live preview will suck.

Fast-forward today: I've started to use Asciidoc with the Atom editor and its Asciidoc live preview. I'm using asciidoctor to generate docbook, and xslt to get it formatted in the way I want. I feel productive to write the content, and I can control every aspect of the output, in the very specific way I see fit.

A simple blog post with a bunch of paragraphs and bold styles may not be worth it, but anything more complex will favor asciidoc.

<comment><USER10> Out of curiosity, why not write DocBook directly using a WYSIWYM editor -- for example OxygenXML? Edit: I don't think it's worth using for blog posts and such, but for anything that you plan to publish -- papers and books -- it works very well.

<comment><USER9> For me it is easier to version-control and review changes in asciidoc than in docbook.

<comment><USER10> Do you review your own changes, or someone else's? Asking because the former is unusual, but the latter works much better with OxygenXML's built-in change tracking. It's fantastic for working with editors (in combination with, say, Subversion).

<comment><USER9> Both. Also, I have sometimes a doc edited on google docs, then copied over to local file and formatted in asciidoc. After that we put it back to the google doc in asciidoc markup, and subsequent comparison and merging becomes a way lot simpler.

</comment></comment></comment></comment></comment><comment><USER11> Can't people use "Isn't perfect" instead of "Isn't the solution"? Or does that highlight too much why the title of this article is stupid?

Not that it's not a good article - it is. But damn do I hate when people go about yelling "This isn't the solution! You're doing it wrong!". Urgh.

<comment><USER12> I would agree with you if the article hadn't ended with a link to what the author considers the solution.

</comment></comment><comment><USER13> Good article. I also agree with his approach of taking Common Markdown and breaking it up into a core, and then extensions. That means parsers can be standards compliant while working in environments and situations where certain features don't make sense (e.g. something that takes in markdown and outputs to an programmable LED sign, won't need URLs, HTML, and so on just formatting).

I wish Hacker News had markdown.

Off topic question: Why has rich text fallen so far out of fashion? I really like Markdown, but why is it that Markdown is now so popular and richtext boxes that can display a realtime preview (e.g. bolding text, italics, etc) aren't. What is it markdown offers over that? Is it just transportability?

<comment><USER14> Interesting question.

One nice feature is that markdown makes text annotations explicit and obvious. There's no hidden styling. Empty lines don't have a font size. Its obvious when a bolded region doesn't bold the spaces between words. In the Rich Hickey[1] sense, markdown is much simpler than rich text editing because all you have to worry about is the semantics of your text (this is a heading) and not how its actually styled.

Weirdly, its kind of a huge throwback to LaTeX. Thinking of markdown as a "modern, simplified LaTeX for the web" seriously hits the mark.

[1] [http://www.infoq.com/presentations/Simple-Made- Easy](http://www.infoq.com/presentations/Simple-Made-Easy)

<comment><USER15> Agreed. I would summarize that as: it doesn't have _hidden state_. Which is nice.

</comment></comment></comment><comment><USER16> So why did the common folks not get together with the vdmd folk(s)?

<comment><USER17> According to the vfmd spec history[1] the first version appeared in july 2013. According to the other project[2], they started "about two years" ago. It would be impossible for the former to discover the latter, given that the work was behind closed doors. The people in the latter group probably did not bother to look at alternatives while they were pursuing their own solution.

[1] [https://github.com/vfmd/vfmd-spec/commits/gh- pages?page=7](https://github.com/vfmd/vfmd-spec/commits/gh-pages?page=7) [2] [http://blog.codinghorror.com/standard-markdown-is-now- common...](http://blog.codinghorror.com/standard-markdown-is-now-common- markdown/)

</comment></comment><comment><USER18> Is it "Common Markdown" or "Standard Markdown"?

<comment><USER18> Nevermind: [http://blog.codinghorror.com/standard-markdown-is-now- common...](http://blog.codinghorror.com/standard-markdown-is-now-common- markdown/)

<comment><USER19> wow, gruber is more of an ass than I thought. His modus operandi is blow off private communications and have a public tantrum

       We've been working on the Standard Markdown project for about two years now.
       As we got closer to being ready for public feedback, we emailed John Gruber,
       the original creator of Markdown, two weeks ago (On August 19th, to be
       precise) with a link to the Standard Markdown spec, asking him for his
       feedback. Since John MacFarlane was the primary author of most of the work,
       we suggested that he be the one to reach out.

       We then waited two weeks for a response.

[...]

and again:

       Starting with the name. In his email John graciously indicated that he would
       "probably" approve a name like "Strict Markdown" or "Pedantic Markdown".
       Given the very public earlier miscommunication about naming, that
       consideration is appreciated.

       We replied with the following suggestions:

       Compatible Markdown
       Regular Markdown
       Community Markdown
       Common Markdown
       Uniform Markdown
       Vanilla Markdown
       We haven't heard back after replying last night, and I'm not sure we ever
       will,

<comment><USER20> Gruber's often an insufferable Apple mouthpiece who thinks way too highly of himself, and has been for the last ten years that I've paid attention. That doesn't keep me from reading everything in the RSS feed, though. Hell, I've even spent considerable sums of money to advertise on his podcast. But this exchange doesn't surprise me.

What _does_ surprise me is how Marco was happy to be one of Gruber's flying monkeys on Twitter. Maybe Marco has an interest in the fight that is unknown to me, dunno.

Anyway, I don't care one way or the other. If forced to care, I'd even side with Gruber on the principles. But don't be shocked that some parties handled this in a less than graceful manner.

<comment><USER21> No, he's just very particular ([https://www.google.com/?gws_rd=ssl#q=john+gruber+%22moto+270...](https://www.google.com/?gws_rd=ssl#q=john+gruber+%22moto+270%22)) about people respecting other people's names for their work.

Stay tuned though. My sources tell me he's going do a really clever thing with a dollar sign in the middle of "Microsoft" next week.

<comment><USER22> what's the story behind "Moto 270"?

<comment><USER21> Motorola makes a watch, the Moto 360, the name being an obvious hat tip to being round. There's a bit of black plastic along the bottom though, so the actual screen takes up a bit less than the full area of the circle. And Gruber certainly seems to think he's very clever in consistently calling it the "Moto 270".

Maybe he has a good argument that they can't call their thing Markdown, but certainly if he starts whinging that they aren't properly capitalizing his trademark, then I think he's behaving in a hypocritical fashion when he quite clearly takes other people's trademarks so casually.

Also, I think it's just as childish as seeing Micro$oft back in the day, but I guess that's a matter of taste.

<comment><USER22> Wow, I really don't understand how this man has such a following. Everything I hear about him is so petty (not to mention that everything about his blog is written to somehow conclude that Apple is some sort of god, even when the source is completely the opposite)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER23> If you're writing a parser - or defining a "standard" \- without a formal grammar, you're doing it wrong.

<comment><USER24> Are they working towards building a formal grammar? I thought so from yesterday's discussion, but I don't know how hard would be to get a sane BNF for markdown

<comment><USER25> They don't have to use BNF - they can use any reasonable grammar notation for the spec, and let implementors work out what tech to use.

</comment></comment></comment><comment><USER26> [Off topic]

      <script>
      var CONTENT = '' +
      "Why Common Markdown isn't the Solution\n" +
      ...

That's probably the most roundabout way to put an article on the web. You are inlining Markdown in an HTML document. Then the browser has to parse and execute 4.5 MB (!) of JavaScript and then it can finally turn that Markdown string into HTML.

HTML documents are a pretty good choice for, y'know, HTML.

Well, if your goal was to increase the page weight from 10kB to 2.2Mb, you surely succeeded.

<comment><AUTHOR> Oh, I'm quite aware :). It's an effort of extreme dogfooding of [https://github.com/khan/perseus](https://github.com/khan/perseus). This post doesn't use anything interesting, but if I wanted to post about some of the interactive math content we're doing there, I could actually embed it. It is also useful to learn how annoying it is to embed (answer: much more annoying than I'd like).

<comment><USER27> Say hi to spicyj for us!

<comment><USER28> Hi.

</comment></comment></comment></comment><comment><USER29> REST:SOAP::RSS:Atom::Markdown:CommonMarkdown

I suspect this pattern will continue into perpetuity because such a large audience will never come to grips with the fact that "better" isn't always better.

<comment><USER30> Do people use RSS anymore? All the stuff I use is Atom.

<comment><USER30> Answering my own question: [http://en.wikipedia.org/wiki/Atom_(standard)#Barriers_to_ado...](http://en.wikipedia.org/wiki/Atom_\(standard\)#Barriers_to_adoption)

Sad.</comment></comment></comment>


Show HN: My friends and I started a data science consulting firm after our PhDs - <AUTHOR> http://www.strong.io <comment><USER2> Can you share some insights...what made you guys decide to go into consulting?How did you land your first client? Any lessons learned? Best of luck.

<comment><AUTHOR> We were all drawn to applying the statistical, experimental, and algorithmic approaches we learned in graduate school (and in our spare time) to a range of problems in industry. Every project has a big learning component that keeps things exciting and fresh.

Our first handful of clients all came from our professional network. I've been a developer and consultant for a long time (shockingly, over half my life!) and so, despite selling my companies and heading to graduate school, I had a bit of a network of other founders who knew me and were supportive of the new venture.

A few lessons learned, in brief: (1) Try really, really hard to be specific about what you offer (even when in reality you offer a lot of different things), (2) Write great proposals -- they become the project bible and really help streamline client conversations, and (3) Understanding a client's data and business always takes longer than you'd think.

</comment></comment><comment><USER3> Hey guys! Thanks for sharing this. I'm actually in the process of starting something similar. What is your approach to ensuring data security for your clients? That's one of the biggest open questions we're always improving.

<comment><AUTHOR> Data security is hugely important. Here are a couple things we do to deal with it: (1) We cleanse the data of Personal Identifiable Information (PII) as quickly as possible (i.e., before it touches our servers), and (2) We host our databases behind secure networks and follow best practices with regards to authentication, encryption, etc.</comment></comment>


Trump Invites Conservative Activists To Discuss Big Tech Bias At The White House - <AUTHOR> https://www.washingtonpost.com/news/powerpost/paloma/the-technology-202/2019/07/02/the-technology-202-trump-invites-conservative-tech-critics-to-white-house-for-social-media-summit/5d1a7d40a7a0a47d87c56f90/ <comment><USER2> Title has been editorialized. Original title: _The Technology 202: Trump invites conservative tech critics to White House for 'social media summit'_</comment>


I just spoke to 80 first graders about programming for career day. - <AUTHOR> http://jimmyjacobson.github.io/careerday

<comment><AUTHOR> I should write up my notes from talking to them. I explained that programming was like speaking a language computers understood and used Spanish and English as examples.

I defined code as the words computers understand.

I defined a program as a story made up of code that a computer knows how to read.

A couple of the great questions I got were "Do you like your job?" and "Do you like typing all day?"

<comment><USER2> Did any of them get the Super Mario reference? Why didn't you go with something more current as your theme?

<comment><AUTHOR> I wanted to leverage the parallax scrolling background with Reveal and make it fun.

And as soon as I plugged in the projector the entire classroom erupted with shouts of "Mario!!!!"

<comment><USER2> That's great! I actually didn't expect them to recognize the 8bit graphics.

What really prompted me to ask was the "The princess is in another castle" reference at the end.</comment></comment></comment></comment>


Testing, testing, testing. It prints - <AUTHOR> http://design.canonical.com/2012/03/printer-test-page/ <comment><USER2> In lieu of a standard test page, I print the google home page.

Assuming it's not a doodle, the google logo + search box has all the ink colors for most printers, and is very low coverage.

</comment><comment><USER3> Wow, whether through legacy support, or they just don't care, when's the last time you heard Microsoft or Apple talking about the print diagnostic page? Perhaps Ubuntu will finally crack the Linux as a truly (your grandmother can use it) alternative desktop OS.

<comment><USER4> Maybe MS and Apple think other things are much more worth talking about. I would concur, personally.

</comment></comment><comment><USER5> It is astonishing that they took _four years_ to fix their incredibly wasteful custom test page.

<comment><USER6> I assume their todo list resembles mine, but more complex.

I've got a lot of stuff (116 todos) as line items. Some of that is important, priority 1 stuff.

A lot of it it is not, and will be gotten around to as time permits.

Changing a test print-out would seem to me to be one of those low-priority fix it when you have time, deals.

<comment><USER5> Or they could just revert it to the CUPS default page.</comment></comment></comment>