Who Goes Nazi? (1941) - <AUTHOR> https://harpers.org/archive/1941/08/who-goes-nazi/ <comment><USER2> I get what the author is getting at. Not sure I agree. Many years ago, I read "Ordinary Men: Reserve Police Battalion 101 and the Final Solution in Poland", which analyzes interviews with the actual men performing the slaughter of innocents at the end of the war.

The conclusion that book drew was different than this article, and makes more sense to me — that it's not "mean people 'go nazi'", but ordinary people who 'go nazi', given a system which relieves them of responsibility for their actions. "Just following orders" was the typical & honest answer, if I recall. The men didn't feel they bore a moral duty to disobey, because these things just had to be done, or so they were told.

So, if I can riff a bit on the article's themes... maybe mean people will hand you the gun, but _anyone_ will pull the trigger, if they're told to do so.

<comment><USER3> This dynamic is significant in the culture wars of today.

As part of some work in architecture for some public sector projects, I read "The Nazi Census" which was a description of the technologies and techniques of the 1938 german census which was a basis for the NSDAPs brutal bureaucracy. ([https://www.amazon.com/Nazi-Census-Identification-Control- Po...](https://www.amazon.com/Nazi-Census-Identification-Control- Politics/dp/1592132596))

One of the interesting parts was adding "unused," fields to the Hollerith punch cards for "future use," much like we use extensibility fields in data models today. The book says many of the people recruited to administer it were promoted from the ranks of the disaffected, often far above their level to ensure their loyalty. It was a technique used by the NSDAP, Stalin, and Mao, where they put country "peasant" types in administrative roles over towns and cities to exploit rural resentment of city dwellers.

As a result, you can "steelman" the sentiments behind many conservative arguments by summarizing them as questioning the wisdom of handing reins of unimaginably powerful institutions and technologies to people who identify as victims with an implied entitlement to revenge, and who are not bound by the ethical frameworks of the deposed - the ones assumed when those techs and institutions were built. It at least provides a logic beyond evil and hatred.

Regardless of whether it's accurate in the context, it's a heuristic for reasoning about the motives and quality of an argument.

<comment><USER4> What do you mean by "the ones assumed when those techs and institutions were built"? Both Nazi and Communists institutions and techs were built for that exact purpose.

<comment><USER5> If you mean designed and built, that's a rather harsh judgement of eg IBM - well beyond "war profiteer". Perhaps _supplied_ for that purpose though.

[https://www.theguardian.com/world/2002/mar/29/humanities.hig...](https://www.theguardian.com/world/2002/mar/29/humanities.highereducation)

Somewhat chilling to contemplate what purpose the current "Watson" could be put to, though:

"When the Nazis invaded Poland (...) IBM New York established a special new subsidiary called Watson Business Machines," after its then- president, Thomas Watson."

"Hello, Watson - is our genocide still on track?"

</comment></comment></comment></comment><comment><USER6> The Nazis in the article remind me of the political class in Croatia, where I live.

Once socialist one party rule collapsed, the whole ruling class divided itself into political parties and participated in the whole political spectrum, like nothing happened. Some of them became ardent anti-communists, despite being active parts of the old regime.

After ten years of center-right rule, some even switched sides and former nationalists (and before that communists) rebranded themselves as Christian democrats, Europeans, clean shaven centrists.

Some people are simply made for all regimes. They're the Nazis the author talks about.

<comment><USER7> I think it's that people tend to just go with their group. And when good people don't rise to oppose evil it snowballs into tragedy.

</comment></comment><comment><USER8> There is something ugly about this piece, about the idea that you can see into someone's soul and just know that they'd be a Nazi if they could. To be honest, the tone of it reminds me of the stereotypical pre-Anschluss German who privately scrutinizes the faces of party guests for hints of Jewish or Roma heritage, and flatters himself for his keen eye in discerning their inferiority.

For what it's worth, when the article's author Dorothy Thompson was in her twenties, she was a fiery activist for women's suffrage, which was deeply intertwined with the temperance movement of that era. (Many temperance groups shared the same leadership as women's suffrage groups, and both shared a core idea that women were inherently less coarse and crude than men and that the burden fell on women to transform the debased, violent world men had created.)

Thompson toured New York state giving barnburner speeches promoting women's suffrage, and promoting temperance figures as well, like Dr. Anna Howard Shaw, a popular keynote speaker for the New York Woman Suffrage Party and prominent member of the Women's Christian Temperance Union. The W.C.T.U. is the political organization most responsible for bringing us the 18th Amendment prohibiting the production, sale, and transport of alcoholic beverages in the United States. The Volstead Act passed to enforce Prohibition ultimately resulted in the deaths of thousands of U.S. citizens who were gunned down in the crossfire of speakeasy raids, or who drank liquor poisoned deliberately by the Feds.

Not quite Nazism, but government agents murderously enforcing philosophies of moral purity are perhaps not so far away, either. All this was going on in 1919; Thompson moved to Europe in 1920, the year Prohibition came into effect, leaving behind the violence and chaos she'd helped bring to power.

It's doubtful she wanted things to turn out quite the way they did. She meant to support only the good people and the good idea of women's suffrage, yet somehow some bad people and bad ideas came along for the ride. Indeed, some of the good people with the good ideas were also the bad people with bad ideas.

There might be a better moral in this than the notion that Nazis are a separate breed of people we can know by second sight.

</comment><comment><USER9> Discussed in 2016:

[https://news.ycombinator.com/item?id=11053415](https://news.ycombinator.com/item?id=11053415)

[https://news.ycombinator.com/item?id=11155824](https://news.ycombinator.com/item?id=11155824)

Dorothy Thompson was one of if not the most famous American journalist of the 1930s and 40s. She's largely forgotten now, but is sure to be rediscovered.

[https://en.wikipedia.org/wiki/Dorothy_Thompson](https://en.wikipedia.org/wiki/Dorothy_Thompson)

<comment><USER10> It's amazing reading comments that sound like they are about Trump, but are actually written about Obama.

<comment><USER11> There's only one comment in those about Obama, and that's turning a thread about Trump into one about Obama. (No one seriously thinks of Obama as being the "strongman" archetype)

OTOH, there is the typical anti-Semitic "Nazis were just a response to Commununists, and it was the Jews fault" comment. See [https://news.ycombinator.com/item?id=11156462](https://news.ycombinator.com/item?id=11156462)

</comment></comment></comment><comment><USER12> If anyone has ever read Speer's memoirs, the striking thing is just how ordinary all of the Nazi big-baddies were.

He'll be recounting some story of Bormann or Goering, and you'll be thinking to yourself, "That's just like Joe in HR!" or "Matt does that all the time!"

Take the ordinary schmuck in middle-management, give him absolute power, then BAM--nazi.

[https://archive.org/stream/Inside_the_Third_Reich_Albert_Spe...](https://archive.org/stream/Inside_the_Third_Reich_Albert_Speer/Inside_the_Third_Reich_Albert_Speer_djvu.txt)

</comment><comment><USER13> The second paragraph's mention of Jewish Nazis actually is correct, [https://en.wikipedia.org/wiki/Association_of_German_National...](https://en.wikipedia.org/wiki/Association_of_German_National_Jews). Almost anyone can become a Nazi. Unfortunately, it turns out that being a Nazi, does not save you from the Nazis.

<comment><USER14> "[…] it turns out that being a Nazi, does not save you from the Nazis."

That depended on your level of devotion, I suppose. See e.g. Emil Maurice, he was a Jew and a founding member of the Schutzstaffel (SS member #2).

"Hitler compelled Himmler to make an exception for Maurice and his brothers, who were informally declared 'Honorary Aryans' and allowed to stay in the SS."

More: [https://en.wikipedia.org/wiki/Emil_Maurice](https://en.wikipedia.org/wiki/Emil_Maurice)

<comment><USER15> _See e.g. Emil Maurice, he was a Jew_

One of his great-grandfathers was Jewish; he was not.

<comment><USER4> Yeah, these are less puzzling when you realize that nazi defined Jewishness by blood and thus many assimilated people who considered themselves Germans and believed same things as other Germans got catch in net too.

<comment><USER16> Which was oddly less strict than in the American South where one drop of Black blood was enough to make you not White. As far as determining blood mixing in the Reich the Mischling Test was used and if you had 3 grandparents who were Jewish that made you Jewish.

[https://en.m.wikipedia.org/wiki/One- drop_rule](https://en.m.wikipedia.org/wiki/One-drop_rule) [https://en.m.wikipedia.org/wiki/Mischling_Test](https://en.m.wikipedia.org/wiki/Mischling_Test)

I totally was reading this thread last night and didn't refresh before posting, I didn't see the post below.

</comment></comment></comment></comment></comment><comment><USER17> I posted this article several years ago. In my mind, the interesting question isn't who goes Nazi; but rather, would I go Nazi and what circumstances would I go Nazi?

Its easy to assume that you are the good guy in the movie. Twist the plot a bit, and do you become the bad guy?

</comment><comment><USER18> Who becomes a cog in the surveillance capitalistic machinery?

Me, it turns out. Is it because I am unhappy and insecurity? Yeah maybe I suppose. It's not that I'm seeking revenge or domination. More that I just don't have enough fucks to give left over to care about anyone else. Staying alive takes full concentration.

<comment><USER19> Now that you recognize it, you can escape being part of that machinery. It will require you to nurture your conscience, though.

</comment></comment><comment><USER20> I believe this ignores the component of social cohesion. In rapidly changing and uncertain times, the lure of steadfast resolve and order can be psychologically powerful to many types of people. Fascist ideologies of all types promise this. And if the atrocities of the state are out-of-sight out- of-mind for the majority of folks, sadly, many find them easy to ignore as long as _their_ world is "in order".

<comment><USER21> To add to this: it is incredibly difficult to risk yourself and your family to protect strangers, no matter what the numbers are. That's why the act of defying the Nazis in WW2 was particularly heroic: the alternative was to just go along, and be safe and comfortable. Heroism in warfare counts for less, since the alternative is often just a different kind of death.

</comment></comment><comment><USER22> This is all well and good, but to recast this to 2018, you need to factor in the people who can't recognize fascistic tendencies unless they come with a swastika on their sleeve.

You see it in both the obsession to label anything and anyone a nazi or nazi- adjacent, as well as to turn a blind eye to the real abusive power wielded in the name of anti-fascism.

The ends do not justify the means, and you don't get to paint your team's actions as merely "consequences" like it's some sort of divine and just punishment. You can't cast yourself as a powerless observer when you're actively applying leverage in collusion with institutions.

Consistent rules, actual accountability and real responsibility need to apply to everyone if this is to get better.

</comment><comment><USER23> The author, Dorothy Thompson, was a key journalist and personality who encouraged the U.S. entry into WW2. The book "Those Angry Days" by Lynne Olson describes the struggle between the interventionists and the isolationists.

</comment><comment><USER24> I highly recommend the excellent

"They Thought They Were Free"

For perspective on what it feels like to become a nazi.

[https://www.amazon.com/They-Thought-Were-Free- Germans/dp/022...](https://www.amazon.com/They-Thought-Were-Free- Germans/dp/0226511928)

</comment><comment><USER25> It's a spot on essay on the kind of the original Nazi movement (and others since), but not applicable to the modern Trump situation.

First, because the Nazi analogy is far-fetched -- from the Wall, to the travel ban, to throwing out immigrants, and on to war, he is continuing more or less the same policies as Obama. Who didn't do that great on institutional racism either (remember "black lives matter"? The hurt that caused this and the killer cop business had continued as usual under his watch. Same for things like surveillance, hawkish attacks in foreign policy, etc. Heck, even Obamacare was "inspired" from Mitt Romney -- and nothing like e.g. British or Canadian healthcare).

But even more so, because while the Nazi party was THE establishment, Trump is a small anomaly, with most of the establishment figures (all "good society" and the prestigious press) against him. You can't build a career on being for Trump today (you'll regret very soon), whereas you could for being pro Nazi in the thirties. And many of the author's analogies are based on those aspects.

</comment><comment><USER26> An incognito window works for this one, and the first page is viewable at [https://outline.com/36bqXA](https://outline.com/36bqXA).

</comment><comment><USER27> Great read. The central concepts apply to modern day subculture movements.

</comment><comment><USER28> _It's an amusing game. Try it at the next big party you go to._

<comment><USER9> Could you please stop posting unsubstantive comments to Hacker News?

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

<comment><USER28> It is a quote from the article, hence why it is italics. How is this unsubstantive? It's the last lines of the article, it practically answers the title question. It's satire and a polemic.

<comment><USER9> I agree that mitigates it somewhat, but the comment was still unsubstantive, no? It added no information.

<comment><USER28> Not even slightly. The definition of substantive: _having a firm basis in reality and therefore important_

Quotes, citations, showing one's work are all inherently substantive.

To take something seriously as Nazism in the context of a game like a puzzle to be solved, it's brilliant and absurd, and literary cleverness, and worth repeating as is. And this wedge fits within the timing of the article, itself wedged between rampant Nazism in the U.S. but before the U.S. had entered WWII. I doubt this activity could be called a game during or after the war, no matter the satirical hints.

[https://www.theatlantic.com/video/index/542499/marshall- curr...](https://www.theatlantic.com/video/index/542499/marshall-curry-nazi- rally-madison-square-garden-1939/)

Adding no information? That's the vast majority of comments on HN including my own, which don't get either an up or down. But fine, you can say it did not enrich the conversation.

<comment><USER9> When we say "substantive" in HN comments, we mean adding relevant information.

</comment></comment></comment></comment></comment></comment><comment><USER29> I feel like there is much more of propaganda than insight in this piece. For one, because it looks specifically at Nazi-ism, and collaboration, rather than at fascism from the home front.

Secondly, it reeks of 'The good old boy' and the idea that a good man could never be deceived. It promotes a sense that, if you are simply American enough, you can escape the feverish grip of nationalism or public panic.

<comment><USER30> I didn't get that from the article at all.

I think this sentence in the last section of the article is the central point:

> Kind, good, happy, gentlemanly, secure people never go Nazi.

That Nazism is a product of cruelty, unhappiness, and insecurity.

Fascism is about dominance. About being a part of the group to which the other is subordinate. Which is something that appeals to people who are deeply bitter, delight in suffering (of their own or others), or lack a sense of self-identity.

<comment><USER31> > Fascism is about dominance. About being a part of the group to which the > other is subordinate.

This is utterly correct.

> Which is something that appeals to people who are deeply bitter, delight in > suffering (of their own or others), or lack a sense of self-identity.

This is absurd, and does not follow from the first part.

Here's the truth (and you do get this from the article) : anyone who rises a lot in the world, or has that as an ambition, in either popularity or money, would "go nazi". That is very different from actively persecuting people, and that should be clearly understood.

I feel like dropping another 10 points on this site, so may I just make the point. Someone who was born a man (or woman) of some privilege (not necessarily much, but some) and does not really seek to advance, someone who gets his life pre-planned for them, and follows the plan. "Dad was a doctor, and I will be too". Those are the people that do not go Nazi, that, no matter what, will never join.

In other words : rich republicans would be the bastion against Nazism. They would be the people that, no matter what efforts are done, cannot be converted. Doctor families. Lawyer "dynasties".

The Bay Area ... would be a hotbed of Nazism in America. No doubt about it. Nazism has everything that SF wants : loads of young people. Support from universities and "follows science" (read the newspapers from the time). It sings the praises of the poor, gives a clear reason for the poor getting repressed (rich jewish bankers), Nazism hates the status-quo and wants to change it at all costs (even though most people don't nearly realize just how big those costs were going to get. Please keep that thoroughly in mind before judging people).

<comment><USER28> Charlottesville. A rich Republican president says 'Some Very Fine People on Both Sides' about people who self described themselves as Nazis and chanted Nazi slogans. Yes many other Republicans distanced themselves, but nevertheless he has not poisoned the well too much for them, because he's still useful otherwise.

Birtherism to me is Nazism. It is his original sin entering the political scene, and the party did not do anywhere near enough to repudiate it. The passivity was tacit acceptance of it. That's not a bastion.

</comment></comment></comment></comment><comment><USER32> What an awful article. The author starts by ascribing a biological basis to a person's likelihood to be a Nazi. In that respect, she's no better than them.

National-socialism was a collectivist ideology. It was grouping people according to a hierarchy of "races", in other words judging individuals according to their ancestry. It was fundamentally horrible not because of its form of collectivism, but because it was both collectivistic and totalitarian.

Human action is driven by ideas and personal choices, not by collective biological origins or urges, nor by "feeding", nor by "physical training."

If you want a deeper, intellectual, practical understanding of what made Nazi Germany possible read "The Ominous Parallels" (1982) [1] or an extract titled "The Cause of Hitler's Germany" [2].

[1] [https://www.amazon.com/dp/B00Y30U3XM](https://www.amazon.com/dp/B00Y30U3XM)

[2] [https://www.amazon.com/dp/B00INIYHQO](https://www.amazon.com/dp/B00INIYHQO)

<comment><USER33> It literally starts by saying it's not a racial thing.

<comment><USER32> First paragraph: "I have come to know the types: the _born_ Nazis, ..." [emphasis added]

<comment><USER34> You need to fully read the article before you make a really strong accusation like that.

"I think young D over there is the only born Nazi in the room. Young D is the spoiled only son of a doting mother. He has never been crossed in his life. He spends his time at the game of seeing what he can get away with. He is constantly arrested for speeding and his mother pays the fines."

If you actually read the article in its entirety, it's pretty clear from both her misuse of the words 'born' and 'biological factors' that Dorothy Thompson, unlike Nazis, is refering environmental factors and not genetic ones. Misuse of those words are the only flaws I see in what's otherwise a good article that feels sadly still relevant today.

<comment><USER32> It's interesting that you assume, incorrectly, that I did not read the article in its entirety.

So the explanation for the true meaning of her article is that she somehow "misused" the words _born_ and _biological_ throughout her arguments? and that what fundamentally causes someone to become a Nazi is their environment?

Whether your interpretation of her arguments is correct or not, I hold that individuals who promote national-socialism (i.e. Nazism) are entirely responsible for choosing evil ideas (not helpless pawns of their childhood feeding, physical training, or as you assert "environmental factors").

<comment><USER34> Ok, if you read the article then you've missed the context. She's pretty clear as to what she's meant by born which I've quoted above i.e born with a silver spoon. She's also made it clear as to what she meant by 'biological factors' which others have already commented on as well as quoted, which is definitely not how people today would interpret it.</comment></comment></comment></comment></comment></comment>


Ask HN: Help choose a static site generator - <AUTHOR>

I'm technically proficient but haven't programmed in a long while (used to be a programmer). I haven't learned Python, Ruby, JavaScript, etc. I do know HTML somewhat well and can manage CSS a bit.
I'd like to use a static site generator for a small/medium site and have read about them, including Jekyll, Pelican, Octopress, Middleman, Punch, Wintersmith, Frog, Hugo and some others.
I narrowed down to Wintersmith because of its perceived flexibility and then learned about Frog and Hugo. I tend to struggle with such decisions and don't want to spend a lot of time on one without knowing more about others. I'm sure many will say that there's it's a personal thing, but I would like to seek opinions from people who have used different site generators. I did try a bit of Middleman and Punch but didn't like either much.
Considering:
1. Wintersmith (offers a plugin architecture)
2. Frog (Racket looks different, LISPy and maybe something cool to learn if I need to)
3. Hugo (says that site generation is very quick)
4. Anything else...?
My requirements:
1. Easy to learn templating language (preferably closer to HTML than something that's translated to HTML).
2. Allow meta tags for individual pages/posts so that SEO information can be easily included.
3. Includes Twitter Bootstrap templates or has such templates available since I struggle with design.
4. Should not just be focused on blogs. It should allow individual pages (unrelated articles, sections) as well as blog posts (with categories and/or tags).
5. Good defaults so that I can focus on writing stuff and not spend a lot of time on learning about the generator (at least initially).
6. Easy setup and simplicity are preferred.
7. Optional: Support for formats beyond Markdown (or different flavors of Markdown) would be nice. <comment><USER2> I think the Jekyll Bootstrap project[0] might be helpful, or maybe the follow up project Ruhoh[1].

[0] [http://jekyllbootstrap.com/](http://jekyllbootstrap.com/) [1] [http://ruhoh.com/](http://ruhoh.com/)

</comment><comment><USER3> Hugo meets all of your requirements.

It averages ~1ms per page created (hundreds of times faster than most alternatives).

It also boast the easiest installation of all SSGs. Simply download the executable and run. No package managers or dependencies needed.

Hugo is not focused on blogs, but has all the features you would need for any website. Sections & flexible taxonomies are built in. Hugo has support for as many different content types you want.

Hugo defaults are sane and no configuration (other than baseUrl) is needed for most sites.

Learn more at [http://hugo.spf13.com](http://hugo.spf13.com)

</comment><comment><USER4> Docpad [http://docpad.org/docs/intro](http://docpad.org/docs/intro) \- meets all your requirements and then some more.

<comment><USER5> I also use docpad, its up to the task if you pick the correct starting skelleton

</comment></comment><comment><USER6> This site[0] bills itself as "The definitive listing of Static Site Generators — all 242 of them!"

There, that should make your life much harder! :)

[0] [http://staticsitegenerators.net/](http://staticsitegenerators.net/)

</comment><comment><USER7> I went through the same phase, Don't over think it. I would recommend just using Jekyll and starting to code. (or jekyll bootstrap if you don't want to customize your blog too much)

</comment><comment><USER8> I played with a few of these and being proficient in Ruby, I went with Middleman. I liked WinterSmith but the source being in CoffeeScript turned me off slightly.

</comment><comment><USER9> Highly recommend Middleman or Jekyll. Find an open sourced Jekyll theme you like and build off that. Tumblr can also get the job done

</comment><comment><USER10> Nanoc would be another you can try [http://nanoc.ws/](http://nanoc.ws/)

</comment><comment><USER11> [http://www.statamic.com](http://www.statamic.com) is a great option

</comment><comment><USER12> You could try Mixture.io has everything you listed and more

</comment><comment><USER13> perhaps not exactly what you asked for , but my sideproject might be useful. it spits out a static template that looks quite nice.

www.landinggear.me

</comment><comment><USER14> I can recommend jekyll.</comment>


Dropbox Passwords - <AUTHOR> https://www.dropbox.com/features/security/passwords <comment><USER2> > Who can get Dropbox Passwords?

> Passwords is currently available to Dropbox Plus and Dropbox Professional > users.

Dropbox could've done better by offering this to the users on the free tier too. Segment the features but let everyone have access to a password manager with auto fill options to gain a larger user base and market share in password management.</comment>


Why are we using black box models in AI when we don't need to? (2019) - <AUTHOR> https://hdsr.mitpress.mit.edu/pub/f9kuryi8 <comment><USER3> I think it's important to note that _human_ pattern recognition is basically black-box as well.

We can't "explain" how we recognize a particular person's face, for example.

Robust pattern recognition based on 100+'s of factors is just inherently black box.

Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization afterwards, but psychologists have shown that this is often a bunch of post-hoc fiction.

Black box doesn't necessarily mean bad. I think the relevant question is: how do we ensure machine learning is trained and verified in ways that don't encode bias, and only used with "intended" inputs, so a model isn't being trusted to make predictions on things it wasn't trained for?

And also: when do we want to decide to use alternatives like decision trees instead -- because despite being less accurate, they can have designers who can be held legally accountable and liable, which can be more important in certain situations?

<comment><USER4> > I think it's important to note that human pattern recognition is basically > black-box as well.

Agreed. But as you note, even though humans are basically black boxes we can ask them questions in order to find out how they came to a particular conclusion. (How reliable the answers to these questions are is of course a different matter.)

So maybe we don't necessarily need fully interpretable models but simply a way to ask black-box models specific questions about their state, e.g., "To what degree does a person's age influence the output?".

<comment><USER5> > But as you note, even though humans are basically black boxes we can ask > them questions in order to find out how they came to a particular > conclusion.

No, you can't. If somebody treats you with suspicion, it's because of a combination of their news intake, their culture, local events, what their friends and family would think, the way you present yourself, and many other factors. You can always ask somebody to state their reason as a simple "if- then" statement, and they can make one up on the spot, but it'll be so oversimplified that it's basically a lie.

> So maybe we don't necessarily need fully interpretable models but simply a > way to ask black-box models specific questions about their state, e.g., "To > what degree does a person's age influence the output?".

You can already do that. Just change that number in the input and see how the output changes. To that extent, even the most black box AI model is more transparent than human decision making.

<comment><USER4> > You can always ask somebody to state their reason as a simple "if-then" > statement, and they can make one up on the spot, but it'll be so > oversimplified that it's basically a lie.

Well, I guess it depends on how self-aware a person is. I think the biggest danger is trying to rationally explain your decision when in fact it was based mostly on your feelings, in which case I agree that the explanation is "basically a lie". One needs to be honest when something is not based on a fact but on a feeling to prevent pointless discussions. (If I hold an opinion based on a feeling then you cannot convince me that I am wrong by giving me facts.)

> You can already do that. Just change that number in the input and see how > the output changes.

Makes sense. But I guess transparent models would still be generally preferable because you can fully understand how the output is produced, whereas in black-box models you might have to ask quite a lot of questions to get a feeling for it, but even then you can't be sure that you have a full understanding of it.

</comment></comment></comment></comment><comment><USER6> They never got to the results of the competition and how the interpretable model did!

From [https://www.fico.com/en/newsroom/fico-announces-winners- of-i...](https://www.fico.com/en/newsroom/fico-announces-winners-of-inaugural- xml-challenge): "The team representing Duke University, which included Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang and Tong Wang, received the FICO Recognition Award acknowledging their submission for going above and beyond expectations with a fully transparent global model and a user-friendly dashboard to allow users to explore the global model and its explanations. The Duke team took home $3,000."

Cynthia Rudin is one of the article's authors.

<comment><USER7> But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts

<comment><USER8> Their model didn't win. IBM's model won, based on actual metrics around useful insights.

<comment><USER9> The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good.

IBM turned the model/paper into a toolkit: [https://www.ibm.com/blogs/research/2019/08/ai- explainability...](https://www.ibm.com/blogs/research/2019/08/ai- explainability-360/) Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are.

And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.

<comment><USER10> Reading "subjective" to mean "nonexistent" is a potentially big mistake.

Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria _is_ the definite goal.

In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goal _posts_ , but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai.

Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.

<comment><USER11> All objective criteria are chosen, directly or indirectly, based on subjective criteria.

<comment><USER10> Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020

</comment></comment></comment></comment></comment></comment></comment><comment> 1e-9 The main advantage of a blackbox ML solution is shorter development time to a useful performance level. Creating a transparent, explainable solution typically takes more time, more work, and a higher level of expertise to get to the same performance level. If the problem is complicated and the cost of a mistake is low, then your best approach today is likely to be blackbox. If the cost of a mistake is high, you should not even be considering a blackbox approach (at least not without transparent safeguards). There is a large gray area between these two extremes that requires good engineering judgement to choose well.

I think we definitely need more R&D dedicated to creating easier-to-use, lower-cost approaches to transparent, explainable ML. There is way too much effort devoted to blackbox R&D today. Ultimately, transparent, explainable ML should almost always beat blackbox ML due to a better ability to find and resolve hidden problems, such as dataset biases, that may be holding back performance.

<comment><USER12> > The main advantage of a blackbox ML solution is shorter development time to > a useful performance level.

I think that's kinda true but kinda false. Its true that deep learning often makes feature engineering moot. however, a lot of deep learning projects takes machine learning engineers and applied scientists along with a host of other support engineers + hardware costs ( I seen people at work say: I only used a 16 GPU instances over a couple weeks of training )

meanwhile, I consider gradient boosting fairly interpretable and they can get pretty close results with a lot less tweaking and training time. If you want to go full non-black box, logistic regression with l1 penalization and only a little bit of feature engineering often does really well, probably a lot less development time / cost compared to those high cost PHD research scientists.

<comment><USER13> gradient boosted trees are black box

<comment><USER12> I would disagree with that statement. There are many ways at getting to what the model is doing, feature importances, SHAP, etc. It's not as clear as logistic regression or a single decision tree, but its not a black box either.

</comment></comment></comment></comment><comment><USER14> My understanding is that interpretable models, especially for neural networks, are very far and away from state of the art performance. Work that e.g. tries to approximate neural nets with decision trees have yet to be applied to very large models [1].

Even in computer vision, which is where I think they've been most successful, the visualization techniques used seem more suggestive, then explanatory [2].

[1] [http://www.shallowmind.co/jekyll/pixyll/2017/12/30/tree- regu...](http://www.shallowmind.co/jekyll/pixyll/2017/12/30/tree- regularization/) [2] [https://distill.pub/2019/activation- atlas/](https://distill.pub/2019/activation-atlas/)

<comment><USER15> I think it's a real open question whether the interpretable models are actually worse, or merely worse in competition/benchmark problem sets. The more deep models I build, the more I'm convinced that behind every inscrutable parameter hides a certain amount of overfitting, with maybe a few notable exceptions. E.g., can you build a decision tree that's not obviously overfit but is also susceptible to adversarial perturbations?

<comment><USER16> It's also a real open question whether any of the interpretable models are actually interpretable, or even if they are in any well-defined sense more interpretable than "black box" alternatives.

In practice the answer is a massive "no" so far. Some of the least interpretable models I've had the misfortune to deal with in practice are misspecified linear regression models, especially when non-linearities in the true covariate relationships causes linear models to give wildly misleading statistical significance outputs and classical model fitting leads to estimating coefficients _of the wrong sign._

Real interpretability is not a property of the mechanism of the model, but rather consistent understanding of the data generating process.

Unfortunately, people like to conflate the mechanism of the model for some notion of "explainability" because it's politically convenient and susceptible to arguments from authority (if you control the subjective standards of "explainability").

If your model does not adequately predict the data generating process, then your model absolutely does not explain it or articulate its inner working.

<comment><USER15> _> If your model does not adequately predict the data generating process, then your model absolutely does not explain it or articulate its inner working._

That's a very dynamicist viewpoint. I don't necessarily disagree.

However, in what sense to the prototypical deep learning models predict the data generating process?

I tend to agree that a lot of work with "interpretable" in the title is horseshit and misses the forest for the trees.

</comment></comment></comment></comment><comment><USER17> I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree.

Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?"

I tend to agree with the articles premise that any model being used in critical decision making should at the very least have a list of parameters and the weight given to each, no matter how long and complicated such a list might be.

I think eventually this will be the end result of most critical implementations of machine learning applications as they make their way through the courts, as I can't imagine a judge accepting the argument "The machine made a mistake, we are not sure why or the reason why is proprietary, but we are not responsible because the machine made the error".

<comment><USER17> Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected.

It makes me think about the mess the derivatives markets made in 2008

<comment><USER18> It's not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint.

Humans are far more 'black box' than neutral nets, we just happen to be able to construct plausible explanations in parallel.

<comment><USER19> No, they were impenetrable from the customer perspective, but either the human making the decision or the software being used can answer. With a black box model, nobody can answer.

<comment><USER18> From the post I responded to:

> the scary thing is when decisions are being made that will affect a persons > life but will never make it to the courts. Like the example of predicting > loan defaults or parole releases, those decisions are made with little to no > explination.

I was pointing out that it's already like this. Whether it's a black box ML model or a capricious bank employee makes no difference if there's no transparency either way.

</comment></comment></comment></comment></comment><comment><USER20> I am worried about the recent trend of "ethical AI", "interpretable models" etc. IMO it attracts people that can't come with SOTA advances in real problems and its their "easier, vague target" to hit and finish their PhDs while getting published in top journals. Those same people will likely at some point call for a strict regulation of AI using their underwhelming models to keep their advantage, faking results of their interpretable models, then acting as arbiters and judges of the work of others, preventing future advancements of the field.

<comment><USER15> _> IMO it attracts people that can't come with SOTA advances in real problems and its their "easier, vague target" to hit and finish their PhDs while getting published in top journals._

I'm also pretty wary of interpretability/explainability research in AI. Work on robustness and safety tends to be a bit better (those communities at least mathematically characterize their goals and contributions, and propose reasonable benchmarks).

But I'm also skeptical of a lot of modern deep learning research in general.

In particular, your critique goes both directions.

If I had a penny for every dissertation in the past few years that boiled down to "I built an absurdly over-fit/wrongly-fit model in domain D and claimed it beats SoTA in that domain. Unfortunately, I never took a course about D and ignored or wildly misused that domain's competitions/benchmarks. No one in that community took my amazing work seriously, so I submitted to NeurIPS/AAAI/ICML/IJCAI/... instead. On the Nth resubmission I got some reviewers who don't know anything about D but lose their minds over anything with the word deep (conv, residual, variational, adversarial, ... depending on the year) in the title. So, now I have a PhD in 'AI for D' but everyone doing research in D rolls their eyes at my work."

 _> Those same people will likely at some point call for a strict regulation of AI..._

The most effectual calls for regulation of the software industry will not come from technologists. The call will come from politicians in the vein of, e.g., Josh Hawley or Elizabeth Warren. Those politicians have very specific goals and motivations which do not align with those of researchers doing interpretability/explainability research. If the tech industry is regulated, it's extremely unlikely that those regulations will be based upon proposals from STEM PhDs. At least in the USA.

 _> faking results of their interpretable models_

Jumping from "this work is probably not valuable" to "this entire research community are a bunch of fraudsters" is a pretty big jump. Do you have any evidence of this happening?

<comment><USER13> > If I had a penny for every dissertation in the past few years that boiled > down to...

This is very, very accurate. On the other hand, I oftentimes see field- specific papers from field experts with little ML experience using very basic and unnecessary ML techniques, which are then blown out of the water when serious DL researchers give the problem a shot.

One field that comes to mind where I have really noticed this problem is genomics.

</comment></comment></comment><comment><USER21> Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough."

For-instance when a model starts being over fitted, one can say it's pretty much converging to something akin to a LUT (Look up table). While over fitting is generally undesirable it still might be interesting. Principally, if someone could figure out how basically it is indexing the data. Perhaps you could create a simpler hash function or find some rules for creating useful hash functions for that type of data.

<comment><USER22> The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

<comment><USER23> I'm confused by what you're saying... by data that hasn't seen before, do you mean new variables which need to be hypothesis tested? Or just a larger sample?

I also don't see why you can't create a "function" hash or otherwise, that works for 1000s of dimensions. The model itself, if it was perfect would be an equation... the number of dimensions really doesn't matter that much.

Sorry, I'm just real confused by your statement.

<comment><USER24> Say you're building a self-driving car. You need to take an image of what the car sees, and figure out whether/where there is a pedestrian/car/bicycle/etc in the image. So your input/variable is the image itself.

Almost every image the car sees is going to be different. I'd guess that's why GP said that it doesn't make sense to use a hashing function - there's little value in mapping inputs to results, because your input images are pretty much always going to be different. So pretty much every time you look up an image in your hashmap, you wouldn't find a match.

That's the point of ML models - find patterns in data so that when you see a new example, you can predict what you're seeing based on what you've seen in the past.

<comment><USER25> It seems like you might be deep in the ML rabbit hole. Zoom out a bit. A hash function is "just a function." Every image labeling black box can be though of as a hash from images to label vectors.

OP's comment effective asks whether there is another, more grokkable function that maps/hashes inputs to the same labels.

Granted, that question boils down to "can we create human-understandable models?" which is the whole point of this discussion.

It's a good question, though. If we had black-box-like spaghetti code performing the same task, I predict that the comments here would be very different.

<comment><USER24> Thanks for explaining that perspective. I was seeing things a little too narrow-mindedly for sure.

Based on your phrasing of the issue, it seems like we could think of the problem as: can we reduce the number of parameters in an ML model to the point where humans can understand all of the parameters? That's related to an active research area - minimizing model size. ML models can have billions of parameters. It's unfeasible for a human to evaluate all of them. Research shows that (sometimes) you can reduce the number of parameters in a model 97% without hurting it's accuracy [0]. But, 3% of a billion parameters is still way too many for people to evaluate. So I think the answer so far is no, we can't create human-understandable models that perform as effectively as black boxes.

[0] [https://openreview.net/forum?id=rJl-b3RcF7](https://openreview.net/forum?id=rJl-b3RcF7)

</comment></comment></comment></comment></comment></comment><comment><USER26> Interpretability is probably one of the most misunderstood topics in AI. Interpretability is fundamentally not a math or statistics challenge, it's a Human Computer Interaction (HCI) problem and needs to be studied in a similar manner, with user studies. The fundemental goal of interpretability is to help users achieve some concrete goal such as detecting bias, identifying bad models, and debugging. This is similar how we can better design keyboards to help users achieve faster typing speed. Unfortunately, most interpretability studies don't actually bother testing if their methods actually work and I am highly skeptical that things like "white box models" or salience maps or attention weights or whatever latest methods can actually achieve the aims that interpretability proponents claim they do. I would love to see a simple randomized user study that would emperically show a concrete improvement in something like bias detection or error detection.

<comment><USER27> > nterpretability is fundamentally not a math or statistics challenge

No, I keep on seeing AI/ML/DS people keep on downplaying statistic.

Statistic interpret things. The majority of the models out there have a one to one, predictor to response, holding all other predictor constant (linear regression, logistic regression, arima, anova, etc..). Statistic inference is a thing. Inference is interpreting. Descriptive statistic is interpreting. Parsimonious is a thing. Experimental design is a thing. Degree of freedom is a thing in statistic.

If you want interpretability do statistic. One of it's tenant is to quantify and live with uncertainty not fitting a curve and lots of coefficient to just predict. Not just classification.

It's a reason why biostat or econometric is a thing. Statistic.

Even the blog cited statistic papers even though it barely mention statistic models in it. ~~And Rudin is a statistician and contributed a lot in statistic.~~ Wrong person (I'm thinking of Rubin for casuality and missingness)

This is not a tribal fight between statistic and ML. This is pointing out that ignoring statistic is a detriment to AI/ML/DS as a field.

I predict that 2020 to 2030 statistic will be coming to AI/ML much more so regardless how much people downplay statistic.

~~Seeing on Dr. Rudin is coming over.~~ I've seen other statistician too. Dr. Loh works took decision tree and added ANOVA and Chisqaure to build parsimonious decision tree.

</comment></comment><comment><USER28> I've happen to be in that workshop room and had chosen the 2% robot, not the 15% surgeon. If I remember correctly, the point of the question was determining willingness of society to use black box models, not challenging the need for models interpretability.

Interpretability obviously doesn't hurt accuracy. But it is costly to engineer. And not always possible to make. Not always possible, because human capacity (and willingness to put in the effort) into understanding the explanation is limited.

<comment><USER29> >> Interpretability obviously doesn't hurt accuracy.

Why do you say this? From what i have seen it certainly can and does. For some industries finding a trade off where the magic is.

[https://link.springer.com/article/10.1007/s10664-018-9638-1](https://link.springer.com/article/10.1007/s10664-018-9638-1)

[https://www.oreilly.com/ideas/predictive-modeling- striking-a...](https://www.oreilly.com/ideas/predictive-modeling-striking-a- balance-between-accuracy-and-interpretability)

<comment><USER28> You can take nearly any black box model and add _some_ interpretability into it, without modifying this model. No modifications to the model, so accuracy stays the same.

</comment></comment></comment><comment><USER23> > [...] these black box models are created directly from data by an algorithm, > meaning that humans, even those who design them, cannot understand how > variables are being combined to make predictions.

So uhhh, isn't this like not science? Like my biggest problem with "machine learning" is people assume the data they have can correctly answer the question they want to answer.

If your data is off, your model might be incredibly close for a huge portion of the population (or for the biased sample you unknowingly have), but then wildly off somewhere else and you won't know until it's too late because it's not science (e.g. racist AI making racist predictions).

A model cannot be accurate if it doesn't have enough information (like predicting crime, or the stock market). There are an insane amount of statistical tests to detect bullshit, and yet we're pushing those and hypothesis testing right out the window when we create models we don't understand.

Like I just don't get how some folks say "look what my AI knows" and assume it's even remotely right without understanding the underlying system of equations and dataset. You honestly don't even know if the answer you're receiving back is to the question you're asking, it might just be strongly correlated bullshit that's undetectable to you the ignorant wizard.

I find it pretty hard to believe we can model the physical forces inside a fucking neutron star (holla at strange matter) but literally no one in the world could pen an equation (model) on how to spot a fucking cat or a hotdog without AI? Of course someone could, it would just feel unrewarding to invest that much time into doing it correctly.

I guess I can sum this up with, I wish people looked at AI more as a tool to help guide our intuition helping us solve problems we already have well defined knowledge (and data) of, and not as an means to an end itself.

<comment><USER30> > So uhhh, isn't this like not science?

Very little of technology has anything to do with validating hypotheses.

> meaning that humans, even those who design them, cannot understand how > variables are being combined to make predictions.

The _intention_ is to not rely on the explanation to evaluate the effectiveness of the model. This does not preclude any of the infinite narratives that might explain the model.

This is fundamentally a cost saving mechanism to avoid hiring engineers to code heuristics useful to business. There is nothing related to science here at all. A "black box" model is fashionable to those who prefer to observe and not create meaning, even if the observed meaning is deeply flawed from a human perspective. After all, people spend money based on less all the time.

<comment><USER29> At our company, we are working with "mechanistic" or mathmatical models rather than stastical approches. I have observed :

1\. It is hard than it should be to explain the concept to people (particularly VCs) 2 . people struggle to understand that a mechanistic model could have more utlity than a machine learning black box 3\. people think you are doing something wrong if you are not using a neural network 4\. The less people understand about neural networks, the more they seem to believe they are appropriate for all predictive / modelling problems 5\. There is generally quite a low understanding of scientific method in the startup / VC space (speaking as someone who has worked in and around academia for years) vs how "scientific" people believe they are because it sounds good to be data driven and scientific about running startups and funding them.

<comment><USER31> Do you mean like symbolic regression or algorithmic information theory based stuff?

If so, I'd love to get in touch, shoot me an email

<comment><USER29> We are using biophysics based approaches. Will send an email over :)

<comment><USER32> Send me one, too, if you don't mind. I like collecting these techniques to give to researchers or practitioners wanting to try new things.

</comment></comment></comment></comment></comment></comment><comment><USER8> This article seems to misrepresent a number of important issues, and as a result significantly overstates their claims. I'll pick just one illustrative (but important) example:

> " _For instance, when ProPublica journalists tried to explain what was in > the proprietary COMPAS model for recidivism prediction, they seem to have > mistakenly assumed that if one could create a linear model that approximated > COMPAS and depended on race, age, and criminal history, that COMPAS itself > must depend on race. However, when one approximates COMPAS using a nonlinear > model, the explicit dependence on race vanishes, leaving dependence on race > only through age and criminal history. This is an example of how an > incorrect explanation of a black box can spiral out of control._ "

The concern about the strong relationship between race and COMPAS predictions is not largely based on a concern about whether there is an _explicit dependence_ in the model. The concern is whether there's a relationship either explicitly _or implicitly_. And in particular, whether such a relationship results in _unfair_ outcomes. The findings of the ProPublica study ([https://www.propublica.org/article/how-we-analyzed-the- compa...](https://www.propublica.org/article/how-we-analyzed-the-compas- recidivism-algorithm)) strongly suggested this was the case:

"\- Black defendants were often predicted to be at a higher risk of recidivism than they actually were. Our analysis found that black defendants who did not recidivate over a two-year period were nearly twice as likely to be misclassified as higher risk compared to their white counterparts (45 percent vs. 23 percent).

\- White defendants were often predicted to be less risky than they were. Our analysis found that white defendants who re-offended within the next two years were mistakenly labeled low risk almost twice as often as black re-offenders (48 percent vs. 28 percent).

\- The analysis also showed that even when controlling for prior crimes, future recidivism, age, and gender, black defendants were 45 percent more likely to be assigned higher risk scores than white defendants."

I understand the desire of the MIT researchers to promote the value of their work, but in this case they appear to be doing so in a potentially damaging way.

<comment><USER33> No, they're not misrepresenting this at all. ProPublica's article [https://www.propublica.org/article/machine-bias-risk- assessm...](https://www.propublica.org/article/machine-bias-risk-assessments- in-criminal-sentencing) was pushing the claim that somehow, the COMPAS black box was implicitly deducing defendants' race from the "137 questions" input to it and labelling them likely reoffenders based on it in a way that was indepdendent of the key factors known to affect reoffending rates, such as age and gender. The paper in question seems to demonstrate the exact opposite is true: ProPublica were inadvertently using race as an imperfect proxy for age at sentencing, which is what the COMPAS algorithm really cared about, because their attempt to control for age at sentencing didn't work. After controlling for the actual age factor, COMPAS results didn't have any relationship to race anymore. (It's not even a weird weighting factor: predicted reoffending risk falls off rapidly with increasing age at first, then more slowly in a smooth fashion. It's just not linear.)

Now, it's of course possible to argue that judging reoffending risk based on age is in fact unfair and racist because it has disproportionate impact on certain racial groups, even though it's strongly predictive across all racial groups. That's not the argument ProPublica made, though. Their argument was about the supposed perils of black boxes, and they kind of acknowledged that age probably wasn't a racist criteria - or at least that it would be a lot harder to justify calling it one - by attempting to strip out its effects in the first place. It's also a different kind of argument entirely, one that revolves not around whether the algorithm is somehow treating people differently based on their detected race - because it isn't - but around what it means for a decision like this to be fair in the first place.

<comment><USER8> Whether a variable is latent or explicit isn't really relevant to the question of algorithmic fairness.

The link I provided gives the actual details of the method and findings; this is probably a more useful source for the details. The claim that the actual source of the difference is 'age' doesn't really make sense. There isn't enough of a difference in the number of young people between black and white populations to result in the differences found in the analysis.

(I do agree that the actual attempt to control for age was poorly done; it really shouldn't have been done at all, since it had nothing useful to add to the analysis or results.)

PS: It's 'COMPAS', not 'COMPASS'.

</comment></comment></comment><comment><USER34> It's not that accuracy will _always_ be sacrificed if one wants an explainable model. The point is: if interpretability is an important constraint, it could prevent improvements on accuracy.

Sometimes, the best interpretable model is as good as a black box, and that's great.

When this is not the case, the trade-off is that one should see what's more important for the actual problem. Perhaps interpretability is not a big deal.

Another solution is to try to extract interpretability from the more accurate black box model with something like SHAP.

<comment><USER29> This is a great point. There is a general lack of understanding about what it means for models to be interpretable & explainable. These words get thrown around often by people who don't understand the definition, and also the trade off with accuracy.

Some papers i found interesting on the subject:

[https://arxiv.org/abs/1606.03490](https://arxiv.org/abs/1606.03490)

[https://arxiv.org/abs/1707.03886](https://arxiv.org/abs/1707.03886)

[https://arxiv.org/abs/1806.07552](https://arxiv.org/abs/1806.07552)

[https://arxiv.org/abs/1702.08608](https://arxiv.org/abs/1702.08608) (i found this was a good sumary of the issues)

</comment></comment><comment><USER10> Ultimately, a lot comes down to the accessibility of tools and know how. The more widespread and general a technique or tool, the more widespread and general it becomes... because people already have it and know how to apply it.

OTOH, these authors (and old school heavyweights like David Ferucci) are not wrong. Interpretability, explainability & interoperability with human intelligence is not something to just give up on.

I like "challenges" as ways of exploring these areas. Good luck to the authors. Female team, btw.

</comment><comment><USER35> A couple salient quotes from "Interpreting AI Is More Than Black And White" in Forbes [1]:

"No matter the definition, developing an AI system to be interpretable is typically challenging and ambiguous. It is often the case that a model or algorithm is too complex to understand or describe because its purpose is to model a complex hypothesis or navigate a high-dimensional space, a catch-22. Not to mention what is interpretable in one application may be useless in another."

"Underspecified and misaligned notions of interpretation impede progress towards the rigorous development of understandable, transparent, trusted AI systems."

[1] [https://www.forbes.com/sites/alexanderlavin/2019/06/17/beyon...](https://www.forbes.com/sites/alexanderlavin/2019/06/17/beyond- black-box-ai)

</comment><comment><USER36> The interest in black-box models is partly for the reasons given, that it is presumed that their inexplanability makes them more powerful, but also because business people are sick of dealing with programmers and analysts as a class, people with arcane knowledge limited replacability, and a penchant for blowing deadlines.

The dream of machine learning is to fire the programmer and break up the last trade (baring doctors and lawyers).

If it was just business people being alienated, I'd say fuck it they deserve it. But the fact is everyone, and many of the programmers too, are alienated too when much of the code makes sense to no one. I would rather fix programming to be comprehensible to more people even though it will take down the walls between programmer and non-programmer.

</comment><comment><USER37> This is a pointless article. Obviously everyone would prefer fully interpretable models if it was possible to get comparable performance with the leading black box/deep learning approaches. No one _chooses_ to make black box, non-interpretable models, it's simply the fact that the models that give the best performance tend to be black box. There is a lot of work happening on trying to interpret deep learning models, because no one prefers non- interpretable models and being unable to pin point and explain why some prediction was wrong.

<comment><USER38> The article explains that their fully interpretable model actually got within 1% accuracy of the best black box model. Which was within cross validation error. That was at a data science competition whose explicit goal was to encourage people to explain black box models. They also give other examples of cases where people have used black box models unnecessarily.

<comment><USER37> > The article explains that their fully interpretable model actually got > within 1% accuracy of the best black box model. Which was within cross > validation error. That was at a data science competition whose explicit goal > was to encourage people to explain black box models.

That's awesome, but this was a single model for a single application being measured against presumably by a limited test criteria/benchmarks. The point still stands that the recent narrow AI "renaissance" is largely due to deep learning, which is inherently black box. There is a lot of work going on in making deep learning more interpretable precisely because it's so prominent today and because its lack of interpretability is a huge con.

Machine vision for example has come a long way due to deep learning. A lot of autonomous car companies are relying on it for perception. All of these companies hate the fact that there is no way to tell when the classifier will fail or why it'd fail. When using deep learning in finance setting, its lack of interpretability is a huge downside.

Despite this deep learning is being used because it provides an appreciable improvement in performance over the alternatives. Certainly there could be alternatives to deep learning that may perform just as well or even better. But finding those alternatives for the vast number of applications deep learning is being used for today is much easier said than done.

</comment></comment></comment><comment><USER39> When using ML models to deal with natural science such as physics, working well+being interpretable actually implies new discovery. Working well only, on the other hand, cannot win you a Nobel Prize.

</comment><comment><USER40> > there are actually a lot of applications where people do not try to > construct an interpretable model, because they might believe that for a > complex data set, an interpretable model could not possibly be as accurate > as a black box. Or perhaps they want to preserve the model as proprietary.

I think they missed an important reason here. It's why I decided to study AI and Machine Learning: The ability to "just throw some data at it and see if it figures it out" is just so _exciting_ , mysterious and intriguing.

I remember actually being a bit disappointed when I learned how (classic, 3 layer, early 2000s) neural networks worked. That it was just something super simple with derivatives, and resulted in something that seemed a lot like a more complex form of statistical regression. Kind of took the magic out of it for me, a little (didn't stop me of course, it was and still is an exciting field of research).

I know it's not a _good_ reason, but surely I'm not the only one who thinks it's extremely _cool_ that one is even able to build a black box that does useful things with what you feed it, but you don't know how it works, and yet you built it.

It's just something mystifying, and I believe there is also a tinge of fear for that disappointment, to figure it out and find out it's not really doing as clever things as you hoped it might.

</comment><comment><USER41> The convenience to be able to say: "we didn't take the decision, the algorithm did" is the holy grail of non accountability.

I wouldn't be surprised if someone, especially in business, is actively opposing transparent AI.

</comment><comment><USER42> Interpretable models are an exciting new entry to the ML toolkit. While they are likely sufficient for everyday ML tasks, they might not be expressive enough vs deep networks to tackle complex tasks like fraud and anti-money laundering classification. As with any tool, interpretable vs black models have optimal target applications that data scientist can apply them for. As interpretable models get wider tooling for training, expect them to play a larger role in simpler ML tasks. Black box models are however here to stay and will always have their place with their versatility.

</comment><comment><USER43> I don't know how many of you are familiar with Kaggle competitions, but the winning scores on the leaderboards can come down to 0.0001 points of accuracy or less. Winning something like that with an explainable model is like having one hand tied behind your back.

If tiny fractions of a point matter (the rare case), then making the model explainable is adding a ton of complexity for little practical gain. It sounds nice, but the numbers are unforgiving

</comment><comment><USER44> What a dumb question is it? The same logic is behind sports and nobel price awards.

It's easier to say to people who make financial decision that this is great artificial intelligence that will solve all your problems then explain mathematical equations behind it that 90% people won't understand. The more complicated the stuff is the higher probability it will be sold. That's the standard in IT world from quite some time.

</comment><comment><USER45> We have all these tools to build neural networks and we use them to peform feats where a simple C4.5 would have sufficed.

To continue innovation in the algorithmic field, I think we need a branch of scientific research or a competition where researchers are constrained to 100MHz CPU clocks and no GPU cheats when actually running their algorithms (not when loading and parsing data)

</comment><comment><USER46> I feel like we're ignoring the obvious answer: because most people in AI right now are overwhelmed by the necessary mathematical background and comfort necessary to produce first-principles generative innovative algorithms and models, so all they're left with is following the latest trend, which is locked on deep learning right now.

</comment><comment><USER47> While I can definitely see the value of black box models in engineering and commercial applications (including clinical ones), I am quite troubled by the new hype of applying to deep learning to so many basic scientific problems. The 'deep' models do not tend to result in deeper insights - <USER2> idealistically ought to be the goal.

</comment><comment><USER48> When I want to predict something, I need the highest accuracy, and I am not interested in interpretability of an individual parameter.

and interpretability is only a half measure. If i really want to understand the model, I will conduct causal analysis with randomized controlled trial and will find the root cause and impact of the factor of my interest.

</comment><comment><USER49> I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability.

Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met.

This logic is perhaps most evident in google/facebooks content moderation dilemmas, where both companies refuse to define any sort of non-vague actionable standards about what they censor (or fail to promote) as 1) they can't provide them and 2) they don't want to be held responsible for what they actually are.

Then, as outrage has grown over facebook/google's terrible content moderation and censorship policies, and the need for accountability has grown, both companies have been forced to hire ever more moderators and censors, because the technologies as claimed just don't work well enough.

<comment><USER18> Well maybe also because black box models can be applied to a wider range of systems and can help to understand general properties of a class of algorithms. I'm sure it's mostly all a conspiracy, though.

</comment></comment><comment><USER50> It's true our own cognition is also a black box, e.g. how do we recognize? It's as much a mystery to introspection as digestion. But the goal of science is to understand. Whereas with DL, we're just chimps in the dirt playing with sticks.

</comment><comment><USER51> I think there is a lot of scope for interpretation in black box models. E.g. when doing transfer learning with a frozen pretrained model it should explain the new classifications in terms of pretrained classes.

</comment><comment><USER52> Can you train a white box solution with only a tagged data set? It seems obvious to me why we have black box solutions even if we could have spent man hours engineering an algorithm.

<comment><USER53> You actually can - for example [https://arxiv.org/abs/1806.10574](https://arxiv.org/abs/1806.10574)

</comment></comment><comment><USER54> There are probably dozens of holes one can drill into this sensationally titled click-baity study. It's surprising what gets through Nature's review system these days.

<comment><USER55> And yet you provide none. What do you base this on?

</comment></comment><comment><USER56> Off topic, apologies, but why are we putting (2019) in the title when this article is barely two months old?

Makes is seem like it's an out-of-date article. Is it?

</comment><comment><USER30> Absolution of liability. Why else use the term "AI" to describe your heuristic?

</comment><comment><USER57> For god sakes. Enough of this black-box—we don't understand how this works narrative. Neural networks are JUST computational circuits. Unfortunately there's been a lot of laziness in cleaning those circuits up (W_ijk=O) to better reveal the topological circuitry.

</comment><comment><USER16> "Explainability" is a subjective idea that allows people to act as gatekeepers about the use of a model through arbitrary politics.

"The Mythos of Model Interpretability" is good reading on this.

[https://arxiv.org/pdf/1606.03490.pdf](https://arxiv.org/pdf/1606.03490.pdf)

</comment><comment><USER58> Human cognition itself is a black box model. We routinely make snap decisions first and then justify them after the fact. Anything that requires serious consideration is darn near insurmountable for a human mind in the presence of incomplete information, emotions, and conflicting motivations.

So while I'd like neural nets to be more interpretable, to me it'd take a distant second place. The first would be to get models that actually work better than humans for practical tasks, even under limited circumstances.

<comment><USER59> I share this sentiment.

It occurs to me that human decisions making can be either explained a very high level, or a very low level(neurons firing). But the magic in between is too complicated to draw direct lines between stimuli and decisions.

AI seems to be the same way. We have statistical models that explain how most everything works at a low level. That a simple math formula can approximate just about anything. How we arrive at weights that separate data sets in different dimensions and etc. For convolution networks we also understand how layer "decisions" flow through to other layers. But, it's too complicated to look at an image and explain exactly how the input pixels will result in a classification output.

I'm not sure how much I should care about that.. Seems like a problem for mathematicians/statisticians. Having the doctor explain himself gives me no more true insight into how his brain works than having somebody explain at a high level how the robot was trained and reacts in different situations.

</comment></comment><comment><USER60> why include 2019 in the title of this post? was it automatically added (is that a bug)?

<comment><USER61> There's a (human-enforced) HN rule to do that for old articles, so they're not mistaken for news. I'm not sure what the standard for "old" is, but it isn't 2019 anymore, so I guess it's technically correct.

<comment><USER60> 2 months old is usually not considered old news on this site... I think that their algo is buggy on year-change

<comment><USER61> As far as I know, there's no algorithm doing it. People put it in the title themselves, or mods do if they forgot.</comment></comment></comment></comment>


Ask HN: Good engineer, bad grades in undergrad. How do I get back into academia? - <AUTHOR> Dear HN,
I just turned 24 and am having a quarter-life crisis.
I graduated over 3 years ago with very average grades from a decent university by doing badly even in the subjects that I knew I was great at because:
1. I wasn't good at writing timed exams (I just cannot think under massive stress) 2. On-and-off phases of depression
Fortunately I am a very good engineer and I did a bunch of open source work while at univ and after graduating have been almost exclusively writing FOSS for the past 3 years. And not simple CRUD apps, but difficult engineering problems.
But here's the thing:
I enjoy writing software, but I don't see an exciting future for myself doing the kind of work I do. I have a great aptitude for Math/CS and I want to do something that is more exploratory research (currently obsessed with ML and specifically Neural Networks). I want go back to academia because 3 years of working in the industry doesn't give me the kind of satisfaction that university research would. I want to broaden my horizons; I'm also getting older and with each passing year it will get more difficult for me to resume academics. I keep flirting with interesting courses on Coursera etc, but deep down I know it's not working because it genuinely doesn't compare to what I would get in a proper degree program. Ultimately I want to get a PhD and I'm willing to work hard for that.
But it looks like all the good places want someone with at least a 7.5 CGPA (on a 10-point scale), and recommendations from professors. Mine is only 6.5 and I expect to get recommendations from (high profile) industry figures.
I'm Indian, but based in Germany, and looking for univs in Western Europe. I don't believe I could ever afford American ones.
So I'm asking for help, HN:
1. Do you know if admissions committees can make exceptions for people with demonstrated open source work? 2. Any professors/academics here who may be able to help me out? <comment><USER2> >I don't believe I could ever afford American ones.

Most PhD programs at American universities cost nothing for most students, i.e., if the university accepts you into a doctoral program it almost always comes with full tuition remission (i.e., you pay nothing) and a modest stipend (i.e., they pay you, somewhere between $10,000 and $25,000 per year). The stipend can be in the form of a teaching or research assistantship, in which case you will work 10-20 hours per week helping a professor, or in the form of a fellowship which requires no work (but fellowships are highly competitive and you likely would not receive one).

Does that change if you're an international student? I honestly don't know, but it would surprise me.

<comment><USER3> The same funding applies to international students, but the admissions criteria is a little steeper for foreigners. You also need very good GRE/ english proficiency test scores to get in as an international student.

</comment></comment><comment><USER4> A PhD especially in ML is not about Software Engineering or learning how to write code. Learn how to do research, write a proposal, survey the breadth of the field. Find interesting problems. Attempt to get to the stage where you either publish or are able to have an interesting conversation with your professor.

There are multiple ways of doing this.

1\. Do this solo.

2\. Work in a research lab as an intern.

2\. The other possibility is to do a Masters and work with a Professor.

Getting into a PhD has very little to do with your GRE, undergrad grades or where you went to school from. These are merely supposed to be proxies for your ability to focus on doing research. I would argue that you should spend sometime doing research to understand whether you actually like doing research.

</comment><comment><USER5> I will tell you this... it is a real issue. I have a similar issue in life as I get bored with a teacher standing in front of me talking theory, and I too deal with the depression...

My advice to you would be looking into certs... I have been doing alot of self education and I believe certs will give you not only the proof that you know a topic but the feeling of completeness.

I am 23 I will be 24 this month on the 28th, I have been actively programming since I was 12 and I have been doing it for a living since before I graduated high school in 2010. I recently began taking courses at my local community college to bring up my GPA which was a 1.9 when I left due to some of the same reasons you didn't do so well. I lived to prove that you can be successful without a degree, my family told me I would never amount to anything without a college degree, and I am making around 125K in Detroit, MI where the median salary is 35K.

With all of that being said, just take the small steps in the right direction, I know I have and its a good feeling.

I hope this helps.

<comment><USER6> >certs will give you not only the proof that you know a topic but the feeling of completeness.

Do people care about certifications outside of IT/Sysadmin jobs? I don't think CS research groups and/or professors even look at them.

</comment></comment><comment><USER7> I am not sure what the situation is like in Europe, but (coming from the US) I think that your conception of what the life of an academic is like is probably tinted by rose-colored glasses.

At a good university, you will be expected to serve on numerous soul-sucking committees, your students will be sucking up to you so that you will hook them up with good recommendations and/or good jobs (and you better deliver), and your dean will be riding your ass to get grants, publish, and get rated highly on poorly designed course surveys.

At a less-than-good university, you will still have the course survey problems, but you will largely have a student body that is best described as apathetic and untalented (there will be a few good students, but they will be the exception rather than the rule). You will also have a larger teaching load. Good luck getting a good research grant -- I hope you have a great professional network. Expect your research to be reviewed a bit more strictly, because no respectable journal wants to publish an also-ran.

Now that the reality check is complete, let me make some pro-active suggestions:

\- If you are any good at all at ML, there are many organizations with interesting problems who will snap you up in two seconds flat. You just need to be good at ML and have a way to demonstrate that.

\- As far as learning goes, find the people you want to work with and figure out a way to work with them. There are a handful of researchers who have a surplus of talented people who want to work with them, but most researchers are starved to find people who are both talented and passionate about their subject. If you can be that person, they will find a way to make it work. It might be on a grant, it might be as a grad student, it might be something else -- just roll with it.

\- If you can't make Coursera work for you, then I truly wonder about your motivation. What do you think you will get from a "proper degree program"? Most of your professors (at good universities, at least) will not want to hold your hand -- quite the contrary. From my perspective, the best grad students go to professors for suggested readings and to answer questions when they come to a (relatively rare) road block in their autonomous learning. Outside of that, peer networks are peer networks.

I think this is enough for now -- feel free to ask follow-up questions.

As a side note, I get the sense that some of your issues/challenges are being fed from the psychological side (e.g., what does it take for you to be "satisfied"). I suggest you talk to a professional if possible. The road to self-actualization is often riddled with potholes.

</comment><comment><USER8> This is math/CS. You don't need a particle accelerator to generate results, just a pencil, paper, and a computer. Do research, get published. That's the currency that matters in academia, and that is what will make a professor ask you to be in their program.

If your FOSS work has publishable results, publish it.

<comment><USER9> I think you are massively underestimating the difficulties to get published in a decent journal.

<comment><USER10> At this point peerj will be fine. Peer reviewed is what matters at this point. Also find a Prof whose work you like and are interested in. See if you can use some of their published work to do something neat.

</comment></comment></comment><comment><USER11> Did you get a master degree already? If not, it should be fairly easy to get into a German university to get that and go on to do a PhD. Have you already tried to apply? Which university? If you can't get into a top university, try _any_ university and move to a different one for your PhD. The best German universities will likely accept you for your PhD, if you do really well while getting your master degree, no matter where you got it.

</comment><comment><USER12> Being good at Math and writing open source software plus having an intense interest in ML, you might try contacting a few professors working in the field you are interested in and volunteer to write up some software in exchange for some help.

You can contact them online, but try contacting 10 or so, not just one and getting disappointed if you get a bad response.

</comment><comment><USER13> Take a look at some problems on Kaggle. Enter a competition, see how you do, publish your results or at least use them in your application package. You can also find startups in this space (some are in Paris). You might also find a job at Google/Facebook. Then you might be able to get into grad school as well. Try all the options.

</comment><comment><USER14> I'm in a similar boat. My plan is to learn ML through application. I'm going to start doing exercises on Kaggle.

I'm also going to start focusing on the computing/architecture side once I get the theory side down.

I wonder how many other folks out there are in similar boats? We should band together lol.

</comment><comment><USER15> Hmm, at the risk of self promoting, I would be happy to have a chat with you. I like helping students out and do it professionally as well. You can read more and if you would like to chat, please check out the contact page. My website is www.scholarstrategy.com

</comment><comment><USER16> How good is your German? German universities are practically free, so maybe you could join an undergrad program to get inside the network?

I know a friend who did this for phd.

</comment><comment><USER17> Just apply. See what happens.

</comment><comment><USER18> you should come to india first, we have many jobs for expert coders here, u could land up getting u a self quite many opportunities if u have good coding skills...

If I would have been a company owner and if u have had that skills i would have taken u even if u had that level of skills \\\</comment>


Show HN: completeme, 't' in Github for Bash - <AUTHOR> https://pypi.python.org/pypi/completeme

<comment><USER2> Very cool, I love using this feature on github.

Looks like it looks for github repos, and will search up the tree and back down from there. However, this seems to mess up for me. If I go into a sub- directory in a repo and then try to open a file in the parent directory via `ctl+t`, then it tries to open that filename in the current directory, not from the parent (I'm using emacs as $EDITOR).

One other little thing, is there a way to get curses to cleanup better after itself? My terminal ends up with a relatively large blank spot after finding something, viewing it, and then closing emacs.

<comment><USER2> A little more info—I'm using iTerm2 on Mac OS 10.8.2

Also, it'd be really nice if the open action ended up in my bash history, e.g., open something via `completeme`, close it, and then use ctrl+r or the up arrow to get back to it and open it again.

One more—running the escape command (ctrl+g) seems to exit emacs? (using emacs 24.2.1)

<comment><AUTHOR> Replying to everything:

\- Good catch on the subdirectories within a git repository. Currently, we'll open the file you end up on with $EDITOR, which isn't necessarily correct relative to your cwd. I'll fix that. \- Re: curses cleaning up after itself, I haven't seen that in xterm or terminator. Are you sure that's completeme's fault? Perhaps it's iTerm or emacs? I've never used iTerm and don't have access to a Mac. It could have something to do with this function not getting called: [https://github.com/mattspitz/completeme/blob/80328799a000180...](https://github.com/mattspitz/completeme/blob/80328799a000180bdc1bf64ddfb188161f217914/completeme/__init__.py#L49) \- I originally had this writing to your bash history but removed it because it wasn't really executed by the user. This may end up being obviated by: <https://github.com/mattspitz/completeme/pull/1> I'd like to have Return open your editor (with history) and something else (Tab?) dump back into your prompt. \- I don't know what could cause your misbehavior in emacs with Ctrl+G. I'm using 24.3.1 and can't reproduce it.

</comment></comment></comment><comment><USER3> I've no idea how I'm supposed to install it. Didn't find instructions anywhere.

<comment><AUTHOR> Use pip, <http://www.pip-installer.org/en/1.3.X/>. Sorry for my assumption.

</comment></comment><comment><USER4> Eh, SublimeText does this already. Cool to have it in Bash though.

<comment><AUTHOR> That may be where GitHub got it from.

I don't use SublimeText because it's not installed on all the machines I login to. Also, GUI windows are a pain.

<comment><USER5> And TextMate did it before Sublime :)

<comment><USER6> And Vim, :find, before TextMate.

<comment><USER5> that's really not the same, no incremental matching or feedback, no fuzzy matching.

</comment></comment></comment></comment></comment><comment><USER7> This removes a major bash painpoint for me. Wicked awesome.

<comment><AUTHOR> Thanks!

</comment></comment><comment><USER8> Wishlist #1 in a pull request :)

<comment><AUTHOR> You are awesome.

</comment></comment><comment><USER9> You opted for JSON for configuring your project. I've seen other people do that, but the problem is most interpreters don't support comments (because comments arn't part of json) and even if you have really_verbose_names_with_underlines_for_spaces, sometimes you just can't get away without comments.

YAML, I think, is better suited for configuration files.

Disclaimer: This is a _minor_ criticism.

<comment><AUTHOR> Thanks for the thought. I wanted to use something like YAML but I didn't want to introduce an unnecessary dependency.

Upon further review, I probably could have used a vanilla .ini file and ConfigParser (<http://docs.python.org/2/library/configparser.html>) It's a built-in module and .ini supports comments.

<comment><USER10> (not OP) I personally like using regular python for python project config files (like django does.) If you want to keep it simple, it will be simple, if you want to do something fancy you have the whole language.</comment></comment></comment>


Cashless Casinos - <AUTHOR> https://www.billiondollarstartupideas.com/ideas/cashless-casinos <comment><USER2> The USA IRS requires casinos to report winnings over $1,200. Right now they only do that for slot machines because there is a discrete payout that's on record. It's not done for table games since they're all working in scrip. (and slot machines are as well, as the article mentions)

If a casino could track all income and payouts for every player with a centralized system, would the government compel casinos to send the IRS reports at the end of a gambler's trip if the gambler won >$1,200 over the duration? Something tells me they would.

Casinos certainly don't want to handle cash either. But they also don't want to scare away gamblers that will be taxed on every aggregate win.</comment>