Text Embedding Models Contain Bias - <AUTHOR> https://developers.googleblog.com/2018/04/text-embedding-models-contain-bias.html <comment><USER2> This is a genuine, difficult problem. It's so easy to join up on your political team of choice and scream about it, and all this makes any real attempt to solve it so much harder to talk about in public or collaborate on. In fact, there's practically guaranteed to be some greyed out text in the discussion here.

So some of these associations simply reflect the way-the-world-was or the way- the-world-is - like associating "woman" with "housewife". That's a whole debate in itself.

But some of these can be accidental. Suppose a runaway success novel/tv/film franchise has "Bob" as the evil bad guy. Reams of fanfictions are written with "Bob" doing horrible things. People endlessly talk about how bad "Bob" is on twitter. Even the New York times writes about Bob latest depredations, when he plays off current events.

Your name is Bob. Suddenly all the AI's in the world associate your name with evil, death, killing, lying, stealing, fraud, and incest. AI's silently, slightly ding your essays, loan applications, uber driver applications, and everything you write online. And no one believes it's really happening. Or the powers that be think it's just a little accidental damage because the AI overall is still, overall doing a great job of sentiment analysis and fraud detection.

<comment><USER3> With current technology the problem of Bob (or Adolph or Mohamed) becoming associated with evil is insolvable by the fact that current deep learning systems are fundamentally characterized by not being able to distinguish causation from correlation.

The only solution I can see is forcing any company that imposes _life-defining actions_ on people (credit bureaus, banks, parole boards, personnel offices, etc) to use only rules based on objective criteria and to prohibit systems based on a "lasagna" of ad-hoc data like present day AI systems. Indeed, if one looks at these in the light of day, one would have to describe such system as _fundamentally evil_ , the definition of "playing games with people's lives." (just look at the racist parole-granting software, etc).

<comment><USER4> > The only solution I can see is forcing any company that imposes _life- > defining_ actions on people (credit bureaus, banks, parole boards, personnel > offices, etc) to use only rules based on objective criteria and to prohibit > systems based on a "lasagna" of ad-hoc data like present day AI systems.

That is probably the exact opposite of what you really want. If the problem is that someone's name is Bob and the AI thinks Bobs are evil, what you want is for there to be 100,000 other factors for Bob to show the system that it isn't so. As many factors as possible, so that the one it gets wrong will have a very low weight.

Even the objective criteria will have biases. There is a significant racial disparity in prior criminal convictions, income, credit history and nearly every other "objective" factor. The more factors you bring in, the more opportunities someone in a given demographic has to prove they still deserve a chance.

<comment><USER5> > That is probably the exact opposite of what you really want.

No, it really isn't. In an ideal world, the reasons behind a decision are transparent, auditable, understandable, and appealable. Machine learning is none of those.

<comment><USER6> In an ideal world, society would compensate you when they would like you to assume a risk below its market price, rather than forcing you to pretend not to notice the risk.

<comment><USER7> But there isn't actually a risk associated with an Adolf. Its an inefficiency born out of an incorrect belief by the whole of (or at least most of) society. The correct solution is not to price in the incorrect assumption, but to not make the incorrect assumption.

In other words, by offering Adolf's below-market rates, you're exploiting a market inefficiency at no additional risk. This is an ideal world as you describe it. It's capitalism at it's finest!

</comment></comment></comment></comment></comment></comment><comment><USER8> > But what if we found that while Model C performs the best overall, it's also > most likely to assign a more positive sentiment to the sentence "The main > character is a man" than to the sentence "The main character is a woman"?

As I understand the problem, they are saying that statistically, the statement about the main character being male is a bit more likely to be positive than if the same thing is said about a woman. If that is statistically true and you are trying to create a model to determine the level of positive sentiment in a review, then that may be a legitimate indicator of how people categorize things. If the goal is to try to "fix" how people talk and write, I'm not sure ignoring statistical patterns in the way we talk is really the right approach.

<comment><USER9> Actually it's any name, even the name of the reviewer, as the test they conducted showed.

the issue is that humans don't understand multi-variable statistical analysis, whether in the form of ANOVA or machine learning training, so they try to pack everything down into two or sometimes three variables of output.

And that's fine if you are pulling from a population that's homogenous. But if there are two or more discrete subpopulations, you want to control for them or represent them separately, not just ignore them or pretend they reflect the information you want.

Anyway if the reviewer name is enough to throw the results, it may suggest that guys praise more movies, rather than movies with male character's get more praise.

I just looked at the first page of 10 star reviews of battlefield Earth and none seemed to be female, just saying.

<comment><USER10> If the multiple sub populations exist, that'll be reflected in the data, as long as your dataset is good enough.

<comment><USER9> Maybe? But even something as simple as an analysis of human height has genetic/ethnic, nutritional, age and gender components, not to mention historical differences. If your input is name, and your output is predicted height, knowing and testing for bias in your data sources is definitely important.

I think where people get caught up is that they don't see the world at large as biased, because they view their understandings as essentially correct. For example, we expect judges to rule fairly on every case, right?

To pick a non controversial issue, likelihood of parole is apparently effected by how recently the judge ate [https://www.economist.com/node/18557594](https://www.economist.com/node/18557594)

Now, parole data accurately reflects how people were paroled, so predicting likelihood of clemency requests is a perfectly valid use of that data. If you were doing machine learning to try to help people get paroled, you'd want to leave that bias in as a predictor, because it's unfair but real.

But you'd probably want to adjust that data to correct for the recently having eaten bias if you were writing a system for parole recommendations for new judges based on past judicial decisions.

You wouldn't want people to be more likely to be denied just because they came before a judge before lunch. And if you don't test for a bias like that, how would you be able to tell that the machine learning algorithm had it? And it wouldn't even need to be direct... seeing people A-Z in court could mean a bias based on name.

Long story short, bias is a real issue, and you need to be aware of it and test for it, not assume that your input data isn't effected by human error.

</comment></comment></comment></comment><comment><USER11> I'm glad that Google is part of this conversation, and they're now applying tests for bias to new models that they release. (Some of their old models are pretty awful.)

If you want to see a further example, in the form of a Jupyter notebook demonstrating how extremely straightforward NLP leads to a racist model, here's a tutorial I wrote a while ago [1]:

[1] [http://blog.conceptnet.io/posts/2017/how-to-make-a-racist- ai...](http://blog.conceptnet.io/posts/2017/how-to-make-a-racist-ai-without- really-trying/)

<comment><USER12> For those who aren't aware, @rspeer has been taking this problem seriously for years.

His ConceptNet NumberBatch embeddings[1] are one of the few pre-built releases which attempt to fix this.

[1] [https://github.com/commonsense/conceptnet- numberbatch](https://github.com/commonsense/conceptnet-numberbatch)

</comment></comment><comment><USER13> I imagine that the model would also score "he was murdered" higher than "she was murdered". Models reflect their inputs, and it happens that yes, murder victims are disproportionately likely to be male and nurses are disproportionately likely to be female.

Is there a problem we should address here? Absolutely -- but the problem is that men keep on getting murdered, not that the model recognizes truths with which we are uncomfortable.

<comment><USER14> Biased models can cause ethical and legal problems. While your specific example is not a huge deal, the article gives the example of making hiring decisions in part based on sentiment analysis of candidates' text reviews. In this context, an engineer has responsibility to ensure that the model has no gender, race, or age bias towards candidates' names.

For a real life example, in 2017 Google was more likely to filter the comment "I am a woman" than "I am a man": [https://www.engadget.com/2017/09/01/google- perspective-comme...](https://www.engadget.com/2017/09/01/google-perspective- comment-ranking-system/)

Or consider the impact of any bias in AI for criminal sentencing recommendations: [https://www.wired.com/2017/04/courts-using-ai-sentence- crimi...](https://www.wired.com/2017/04/courts-using-ai-sentence-criminals- must-stop-now/)

<comment><USER15> If the biased models replace human decision making, then it just has to be shown that the models are less biased than humans which may not be that high of a bar to pass.

<comment><USER14> US law prohibits businesses from discrimination. If you're sued, you can't argue that you only discriminated the average amount.

</comment></comment></comment></comment><comment><USER16> Be careful what you wish for.

Under this definition of 'bias', an unbiased model would, say, spit out equal associations between any occupation and any gender/sex/age/race/religion label.

We should probably ask ourselves whether that's a strictly desirable outcome, since by definition the 'biased' model has a higher predictive value. How much accuracy are we willing to sacrifice for the sake of erasing inconvenient facts about either our world, or our current models of the world?

<comment><USER17> Predictive value isn't an end in itself.

<comment><USER18> Hit the nail on the nose.

Sure, you can build a discriminating classifier or generative model that is the most accurate, correctly identifying / emulating the reality of our world down to 5 nines; and nobody says you shouldn't be able to identify or quantify all of these "inconvenient" associations. The trouble is always when you intend to make decisions based on your framework -- the decision to offer somebody a loan, the choice of language your chatbot uses, etc. -- and that is where fairness ought to be paramount.

And yes, if you manage to build a discriminating model that sneaks in protected classes through indirect causal effects with no attempt to suppress them, it would yield your insurance agency higher returns over time, and just because that's the way the world is, currently. But all this would achieve is perpetuating the current status quo, placing short-term gain ahead of long- term equality. You might be doing right by yourself, but that still makes you morally impugnable.

One shouldn't need to wait for an overreaching law prohibiting such indirection, to do the right thing.

<comment><USER19> If a person is loan-worthy, offering them a loan is good business sense. If others are denying loans for no good reason, that is a business opportunity. It doesn't require benevolence to offer people loans that they are qualified for.

How far do you expect systems to go to ignore real-world associations? What if it is coming down to personal safety? Would you object to a self-driving car that routes itself around more dangerous neighborhoods? What about a model that predicts that unknown men on the street at night are more dangerous than unknown women?

<comment><USER20> > If others are denying loans for no good reason, that is a business > opportunity. It doesn't require benevolence to offer people loans that they > are qualified for.

Afaik the black families who were segregated out of their neighborhoods back in the 1940s, 1950s and going into the 1960s were never provided with another comparable "business opportunity" that could have "righted" their situation, they had to settle with living in what turned out to become "ghettos" (for lack of a better word).

> Would you object to a self-driving car that routes itself around more > dangerous neighborhoods? What about a model that predicts that unknown men > on the street at night are more dangerous than unknown women?

Following the same line of thought, would you be ok with an AI system giving a person named Deion or Jayla or Latisha a higher interest rate on their mortgage (and so, potentially, driving them out of certain markets) compared to the interest rate offered to persons named Chad or Emma or Sophia?

<comment><USER19> The shameful history of "redlining" (housing segregation) was driven by policy, not market forces. Black people weren't being judged non-credit- worthy, they were being explicitly prohibited from borrowing and buying by FHA policy and housing covenants. So this history, while awful and something that we should not repeat, does not speak to my point that making rational decisions about credit-worthiness is good business sense.

> Following the same line of thought, would you be ok with an AI system...

You didn't answer my question. I asked it because I want to know if the people most vocally arguing that de-biasing is a moral imperative will admit even one case where there might be a compelling reason to see the world as it is, even if that association is considered "problematic".

If people will admit this, then we can argue over where the line should be. But many don't appear to admit that a line even exists.

I will admit that the line exists, and that an incident like this is a clear example where removing hurtful associations is proper. This was a case where a ML model reinforced a loaded and racist stereotype, and the harm of removing that from the model is almost zero: [https://www.theverge.com/2015/7/1/8880363/google- apologizes-...](https://www.theverge.com/2015/7/1/8880363/google-apologizes- photos-app-tags-two-black-people-gorillas)

<comment><USER20> > You didn't answer my question. I asked it because I want to know if the > people most vocally arguing that de-biasing is a moral imperative will admit > even one case where there might be a compelling reason to see the world as > it is, even if that association is considered "problematic".

I didn't find those questions to be that smart, to be honest, they're more on the scare-mongering side, and I find that a smart question should have half of the answer included in it and a scare-mongering question doesn't look like it contains anything smart (at least to me). But if you really want to know my answer is "yes" to all of your questions. To get into more details: I grew up as a kid into a middle-class-ish family and back then I had no issues going to the "ghetto"/dangerous area of the town I grew up in (I grew up in Eastern Europe, so that the "dangerous" area was populated by the local gipsy community instead of the AfroAmerican/Latino communities now associated with the "dangerous" areas of US cities). I turned up fine.

> If people will admit this, then we can argue over where the line should be. > But many don't appear to admit that a line even exists.

I know of that line, I was just trying to say that further reinforcing it using ML techniques will only aggravate things at a societal level (so that "dangerous" areas will become even more "dangerous"). At least when the discrimination happens out of our (us, humans) own volition there are ways to fix it, but once we "outsource" our racist-tendencies to ML-like tools the voice inside of us telling us that this is all wrong will become even weaker. After all, the algorithms/machines are more "right" then us, humans, that's what we like to think.

<comment><USER19> If we're trading anecdotes, I grew up in a small town where I didn't worry about where I went. Then as an adult in a big city I was mugged at knife-point for obliviously walking through a bad part of town at night. If you want to dismiss people's concerns for their own safety as "scaremongering", don't be surprised when people don't find much use for your ideology.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> This sounds a lot like how people get TSA redress numbers when their info falsely flags them as suspicious ([https://www.dhs.gov/redress-control- numbers](https://www.dhs.gov/redress-control-numbers)). Or how Barack Obama suffered innuendo around the middle name "Hussein." Mistaken identity or unfortunate associations are as old as humanity. AI systems (and non-AI systems) need ways to deal with these problems, but we also have a lot of experience about how to do that.

<comment><USER21> The problem is when we hand AI full reigns over a system that should have humans with the final say, because doing it with humans is "too hard" or "doesn't scale." At least with humans you have auditable decision making and recourse via the legal system, even if it can be hard to fight. AI are essentially black boxes that unconsciously learn the biases of the datasets they are given.

When AI starts determining rather consequential things like how long to send someone to prison for, that's a problem. [https://www.nytimes.com/2017/05/01/us/politics/sent-to- priso...](https://www.nytimes.com/2017/05/01/us/politics/sent-to-prison-by-a- software-programs-secret-algorithms.html)

<comment><USER22> > AI are essentially black boxes that unconsciously learn the biases of the > datasets they are given.

Please don't confuse "AI" with the current state-of-the-art of the latest deep learning model. Many of us researchers are working on interpretability and understanding of causality.

Calling it "AI" makes it appear final, as if in 50 years all machine learning- based decision systems will behave exactly as they do now, without nuance.

<comment><USER21> Calling currently existing models AI is no different than calling a Model T a car. It being more primitive than what may exist in the future is of little consequence to the public dealing with the consequences of AI today.

AI can only learn from the data it has, so it will always carry some sort of bias, because it is impossible to collect the nuance of every last bit of context into a digestible data format. At best it's an advisor, but it should never be a decision-maker.

<comment><USER23> Very different in fact. Model T had most big level components of a modern car in place. Rubbered wheel. Protection from elements. Combustion engine. Transmission.

In comparison, current automated decision making systems are at the stage where we do not even know what the components are. And calling them intelligent is insulting.

</comment></comment></comment></comment></comment><comment><USER24> This is an example of sacrificing the scientific method to make results more politically correct. We've come full circle.

<comment><USER25> >This is an example of sacrificing the scientific method to make results more politically correct. We've come full circle.

Wrongthink is nothing new, comrade ajwnwnkwos, and is as prevalent as ever.

The output of science has been suppressed throughout history where it didn't fit the narrative of the day. At one period, that was by the Church. Another, it was at the hands of the government. Today, mainly by a self-censoring, everything-must-be-pleasant-and-entertaining society that is very highly prone to fits of outrage.

Inconvenient facts are, after all, inconvenient.

<comment><USER26> I never actually understood the difference between self-censoring and just deciding to not be offensive, and why deciding not to be offensive is such an unscientific thing. If you can objectively observe that saying something doesn't accomplish your goals(assuming your goals is to make friends and get along with others) and seems to offend or harm people around you, what are the consequences to simply not saying things?

In the specific context of science I'm unsure what self-censoring would even look like. Not only would such conclusions be a relatively small subset (hard to discuss the self-censorship of a graph algorithm) but there is plenty of attention(therefore funding) on both sides to uncover some truth about any political subject.

<comment><USER27> Because sometimes to reach the truth, you must risk being offensive.

</comment></comment></comment></comment><comment><USER28> In Which We Define 'Biased' As Meaning 'Not Conforming To Our Ideas Of How The World Should Be'. Because it's unthinkable that movies with male main characters could actually just be better than movies with female main characters.

(I'm not saying they are, mind you - but when we analyze sentiment in a large dataset and reach a result like that, the first question to ask should be "is that result accurate?" not "how do we tune out this problematic result?")

<comment><USER29> > the first question to ask should be "is that result accurate?" not "how do > we tune out this problematic result?"

That depends on the product that you want to make. If you're a company that wants to sail on the status quo and just make money, by all means be amoral.

On the other hand, technology now allows us to analyze and steer social structure at scale. We could reproduce past inequalities in the name of efficiency, or we could think a bit longer about our axis of optimization.

"But cultural relativism!" — I agree. It's a tough problem, but we don't all of the sudden need a definitive generalized social plan. Biases can be tackled one algorithm at a time.

<comment><USER30> Maybe a useful way to think about this is Positive/Normative distinction.

Science is "amoral" in your sense, it's trying to describe how things really are, and make predictions. So is the stock market: you're rewarded for correct predictions, whether or not others regard those outcomes as desirable.

Of course we are also moral beings, interested in changing things for the better. And we have varying and contradictory ideas about what better means.

As soon as you say "steer social structure at scale" the question is: who gets to steer? If this were obvious, then we wouldn't need democracy, we could just all work together not to "reproduce past inequalities"... but we can't.

</comment></comment></comment><comment><USER15> The politicization of bias in this realm is unproductive. We know where it leads. There will be 'committees' of people who are not trained in statistical thinking, who haven't even taken a statistics class, who are not qualified to make any statements and who's interests are not in line with making the best product acting as nuisances and aggressors to people who are do have the expertise and who strive to make the best product.

The best course of action is to treat this bias like normal bias with non- politicized, inanimate objects. How would data scientists that encountered similar bias while running machine learning models of the motion of waves act?

<comment><USER31> The article was written by trained experts in statistical thinking. It's worth a read.

</comment></comment><comment><USER9> I'm reminded of [https://whitecollar.thenewinquiry.com](https://whitecollar.thenewinquiry.com)

It turns out that white collar crime is predominantly committed by white men. A system trained to detect white collar crime using, say, enron emails, might suggest a white guy's emails over someone whose name doesn't sound like an enron employee, or who shared pictures of their cat.

I mean, I suppose you can argue that hey, maybe that bias is usually correct. Maybe it usually is the white guy. But personally, I'd probably control for things conflated with gender or race and then look for indicators that differentiate between criminals and innocent people. You will probably have a lower AUC, but better differentiation between criminals and innocent people is what matters.

</comment><comment><USER32> I guess the question posed in the article is an interesting one and is an interesting discussion re how to deal with bias and building counter-biases into algorithms without the editorial decision asking being clear.

Movie reviews are editorial content. Measuring that content is a difficult problem in this type of context... Are the best reviewers people who dislike movies with female leads? Are you going into a back catalog of movie reviews from an age where societal expectations were different? Are popular genres skewing the result?

You could have a curation issue as well — if the female lead movies are dominated by "Hallmark Channel" fare, algorithm C has a point!

</comment><comment><USER33> Here is a video that explains this blog well. [https://youtu.be/59bMh59JQDo](https://youtu.be/59bMh59JQDo)

The unnerving part for me is this "eliminate negative associations/bias". Okay, how about we learn the truth, and then address that outside _in real life_ and keep the computer doing what it's good at ... showing us the data.

<comment><USER31> > Okay, how about we learn the truth, and then address that outside in real > life

Aside from the aspect you overlooked -- that our online lives _are_ real life, and that AI models are (over)used to make _decisions_ , not merely show us the data -- your idea is quite what the article recommends:

> . As with Tia, Tamera has several choices she can make. She could simply > accept these biases as is and do nothing, though at least now she won't be > caught off-guard if users complain. > She could make changes in the user > interface, for example by having it present two gendered responses instead > of just one, though she might not want to do that if the input message has a > gendered pronoun (e.g., "Will she be there today?").

She could try retraining the embedding model using a bias mitigation technique (e.g., as in Bolukbasi et al.) and examining how this affects downstream performance, or she might mitigate bias in the classifier directly when training her classifier (e.g., as in Dixon et al. [1], Beutel et al. [10], or Zhang et al. [11]).

No matter what she decides to do, it's important that Tamera has done this type of analysis so that she's aware of what her product does and can make informed decisions.

</comment></comment><comment><USER34> OK, factual data fed to some AI-based algorithms produces results that are not so politically correct as some people would like it to be. Is this a problem with the AI algorithms, or with the people?

</comment><comment><USER35> I think the core issue here is that people want two things. On one hand we want our models to accurately describe reality, not an idea of what reality should be. On the other hand, we don't want ML to freeze society and culture in their current state, but to help decide on and drive social change. The tension between these goals arises when models trained today are used to make decisions tomorrow.

One way to resolve the tension might be to add time dimension and historical training data. The models might then be able to return in addition to any prediction variable p also its time derivative dp/dt. For example, a model might then return results such as: "movies with female main character: lower sentiment, trending up; movies with male main character: higher sentiment, trending down".

</comment><comment><USER36> It's hard to be believe these days, but once upon a time language models were written by hand. Imagine you hand-wrote such a model and put it to use in (for the sake of example) a psychological evaluation application. Now imagine that after years of use you discover that your model systematically marks african americans as less psychologically fit than white americans. Who would be to blame? Naturally, you would. Your actions led to a biased model being used to unjustly and arbitrarily harm innocent people, and your leadership would be right to call into question every decision your application ever made.

Now imagine the same scenario except your app was trained on data instead of hand-written. Make no mistake, the answer to the question of who's to blame is exactly the same: the developer. The response should be exactly the same: a complete loss of confidence in the model.

I'm appalled that this needs to be said, but reading this comments section I'm afraid it does: _Machine learning models are inference and pattern recognition devices, not scientific tools_. They don't magically reveal hidden patterns in the world, they repeat the patterns that the developers train them on. If you trained a machine learning model to perform psychological evaluations [1] or sentence convicts [2] or recognize faces [3], and your model is biased in a way that is unnecessary and unjust, your model is bad you should be held accountable for its failures.

[1] [https://affect.media.mit.edu/projects.php?id=4079](https://affect.media.mit.edu/projects.php?id=4079)

[2] [https://www.nytimes.com/2017/05/01/us/politics/sent-to- priso...](https://www.nytimes.com/2017/05/01/us/politics/sent-to-prison-by-a- software-programs-secret-algorithms.html)

[3] [https://www.wnycstudios.org/story/deep-problem-deep- learning...](https://www.wnycstudios.org/story/deep-problem-deep-learning/)

<comment><USER37> What do you do when your classification is correlated to race through no fault of yours? For example, I might successfully predict credit score from backyard size, and end up with a correlation to race that doesn't have anything to do with me.

I don't think the blame always lays with the algorithm, especially when it doesn't have access to race as an input (this is a reasonable expectation). I can score students with a simple algorithm based on what they write on their math tests, and even that's going to correlate with race. In that case the blame pretty clearly lies in the process that produced the reality that's being measured, not the measurement technique itself.

Let's say that black people default on their loans more often then white people. Is it better to criticize the math that discovered that fact, or the root cause that made it true to begin with?

<comment><USER36> > Is it better to criticize the math that discovered that fact, or the root > cause that made it true to begin with?

The only place where math was involved was the guts of the training stage of the model. A crucial stage to be sure, but one that's bookended in the front by problem definition, data selection, model selection, and a design for an evaluation process, and behind by the execution of that evaluation and the decision to launch the model. Literally every other stage of this process is driven by human decisions.

I'll say it again, because apparently the point didn't sink in the first time around: _Machine learning models are inference and pattern recognition devices, not scientific tools._ The fact that it's inhuman and unthinking mathematics that produced a biased model offers no ethical or legal cover to the people who decide to put that model into use.

The decision to apply the model is key here. Contrast two applications: one that takes in a patient's information and diagnosis to compute a dosage for a drug, and one that takes in a potential tenant's request and produces a rent/no rent decision. In the first, there are cases in which the bias is admissible if not necessary, e.g. [1]. However, the legitimacy of that model's application comes not from the supposed objectivity of the model's findings but from volumes of peer (i.e. human) reviewed research. In the second, there is no legal way in which this model can be applied, and I struggle to imagine a moral one. I can't imagine any court of law taking "the machine made me do it" as a defense in an FHA case.

[1] [https://www.nytimes.com/2005/06/24/health/fda-approves-a- hea...](https://www.nytimes.com/2005/06/24/health/fda-approves-a-heart-drug- for-africanamericans.html)

<comment><USER30> _pattern recognition devices, not scientific tools_

Isn't science all about pattern recognition? If the pattern exists, in the real world, then a good theory is one which encodes this.

What you're asking for in a "moral" way of applying the model is that our actions should abide by your moral preferences. Perhaps even universal preferences. But it seems useful to me to keep logically separate these ideas about how we ought to do things. They don't flow naturally from observations of how things are.

The example of adjusting drug & dosage based on race is a good one. The science backing this is exactly the same kind of statistical correlation as backs the rental decision. The training input is what race some test patients ticked on a form, and their tick mark sure as hell isn't the causal factor... that's some gene which is correlated, maybe, or some diet difference, or what's on TV, who knows. Nevertheless the correlation is there, as far as we can tell: the peer-reviewed science process isn't infallible. The reason we're OK with using this information is, I guess, that it aims to improve things for the patient. (Not every single patient, only statistically.) We make a moral judgement that this is more important than a landlord's wish to avoid bad tenants (again statistically).

</comment></comment></comment></comment><comment><USER38> "Pray, Mr. Babbage, if you put into the machine biased models, will unbiased answers come out?"

</comment><comment><USER39> I'm reminded of a much more obvious example.

[https://www.theverge.com/2016/3/24/11297050/tay-microsoft- ch...](https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot- racist)

I see a lot of comments about how its somehow sinister to want your model to be better than the lowest common denominator and that is pretty damn ridiculous.

</comment><comment><USER40> I'll tell you more: _human judgment_ contains bias. You can't possibly think logically about every single judgment, particularly when information is incomplete, which it is in the overwhelming majority of cases. It is not a given, to me, that on average AI does any worse than the human population outside the "woke" segment. Or even _within_ that segment considered in isolation.

<comment><USER41> If you put a human in charge of a judgement call, mostly there are mechanisms to monitor them, a requirement that they give reasons for their decisions, and mechanisms to appeal those judgements.

We aren't used to having to do that with decisions made by computers.

It's not that AI makes better or worse decisions, it's the way we treat those decisions.

<comment><USER30> "a requirement that they give reasons for their decisions"

But this is mostly a sham. People lie about their reasons for doing things, even to themselves. Especially when they know what reasons are publicly acceptable.

Maybe the major difference is that it's much easier to run experiments on computers. I mean people try this on humans but it's very hard to do realistically: most of those studies where you submit 1000 CVs with varying details are garbage, because they can only access an unrealistic part of the process (I mean who ever got a job without networking? etc). Whereas on almost any computer system you feed it completely realistic fake data.

</comment></comment></comment><comment><USER42> Key takeaway: It is important to be aware of bias in ML models. Some biases may correctly model the reality of the world, and some may show the bias in the underlying dataset or in what the model has focused on in the data. The goal is not to "unbias" everything, as people seem to be focusing on, but rather to determine if the bias is appropriate given the context.

<comment><USER43> Damn, I just wrote almost exactly these words before scrolling down to your comment. Could have saved myself a few minutes by upvoting yours instead.

</comment></comment><comment><USER44> >> Normally, we'd simply choose Model C. But what if we found that while Model C performs the best overall, it's also most likely to assign a more positive sentiment to the sentence "The main character is a man" than to the sentence "The main character is a woman"? Would we reconsider?

It seems like you have discovered that movie reviewers tend to review movies with am male main character more highly than movies with a female main character; what you need to consider is that while this may tell you something about movie reviewers, it doesn't necessarily tell you anything about the quality of the movie.

</comment><comment><USER45> Despite the controversy surrounding "debiasing" classifier outputs, I think further research in this area is still of merit. This area of research would help us understand and build transformations over latent / high level representation space, a general use case applicable to all fields interacting with machine learning.

</comment><comment><USER46> There's a lot of complaints here about "erasing reality" and other hyperbolic talk, but these models are trying to make predictions within a particular user's context.

It's just inappropriate to apply some global biases for a particular user, and avoiding that can result in a better user experience.

<comment><USER19> I think you make an interesting point: if we use context to train a model to reflect _local biases_ instead of _global biases_ , will that be more just and/or lead to better user experience?

It seems related to the question of whether Google results should be tailored to you. If I Google "did Russia interfere in the election", should Google tailor the results so I always see articles that reinforce my world view?

If we go that route, I think we take the path of Stephen Colbert's concept of "Truthiness", where we judge something as true because it "feels" true. Users will definitely be happier if everything they see reinforces their existing world view. So companies will be incentivized to accommodate this desire. But does this actually lead to a more just society?

</comment></comment><comment><USER47> FYI, a good talk about this from 33C3:

[https://media.ccc.de/v/33c3-8026-a_story_of_discrimination_a...](https://media.ccc.de/v/33c3-8026-a_story_of_discrimination_and_unfairness)

</comment><comment><USER48> I wish people who write articles about "bias" would explain what they mean by "bias". I've seen hundreds of these articles. I'm still waiting for a usable definition.

</comment><comment> cup-of-tea If the models _reflect_ bias then surely that's a good thing.

It's funny. I like programming because a computer can't lie and doesn't make mistakes. I guess some people don't like that.

<comment><USER49> The goal of ml isn't to accurately model training data. The goal is to make something useful. Correctly doing the former can hinder the latter.

</comment></comment><comment><USER50> The only thing that can be biased is the training data.

The model is simply a statistical breakdown of the training data.

</comment><comment><USER51> Google uses their AI systems for profiling where the sex of the people being profiled is a crucial piece of information and acts as a predictor for interests which is then further used in targeting ads. As long as it makes Google money, it's not a problem. This bias actually accurately reflects reality and the advertisers know that, otherwise they wouldn't be paying Google for targeted ads.</comment>


5G Phone Won't Hurt You. But Russia Wants You to Think Otherwise - <AUTHOR> https://www.nytimes.com/2019/05/12/science/5g-phone-safety-health-russia.html <comment><USER2> [https://news.ycombinator.com/item?id=19892993](https://news.ycombinator.com/item?id=19892993)</comment>


What job profiles are suitable for a generalist IndieHacker? - <AUTHOR> I have created many projects as a freelancer as well as for myself (eg. Check saleztalk.com - sold this site).
As an Indiemaker, I have to focus on all aspects of product and so I am a generalist.
Most of the job opportunities are for specialists.
So where does an Indiemaker fits in? <comment><USER2> My two cents as I've been a generalist for 10 years now. I actually see a lot of hybrid areas where sales and programming meet and they practically beg for those who can walk that line.

Possibly a technical project manager at a startup? Since you have knowledge of the technical scope, you would be very valuable to speak to everyone on the team, set realistic expectations, etc, especially for clients that your company would take on.

Also, if you're looking for a bit more of a step down from the technical side, there's an interesting ad in the side of "evangelism" that is looking for a generalist as well for a pretty cool company. [https://keybase.io/jobs#evangelist](https://keybase.io/jobs#evangelist)

I think something within the realm of those two, and you'll likely evolve within the company to whichever role comes most naturally.

</comment><comment><USER3> Small companies would fit the best, that need all their bases covered but can't afford the costs of hiring an army specialists (nor do they really need them at an early stage). Especially those outside of the general internet/computer software market but understand they need to take a harder look into tech and UX to solve their business issues

Depending on how well your freelance or indie hacker career went, it may be very difficult to find job opportunities with startups that can exceed what you already had. But they are out there.

I think you already know how to position yourself – technical person that can solve business problems – you just have to throw out more nets to find the opportunities that actually fit.

</comment><comment><USER4> BTW: the site is broken and configured to leak error data, so the web.config needs to be changed to remoteonly custom errors (or something like that). Better let your buyer know.

</comment><comment><USER5> Is the question meant for the [https://www.indiehackers.com/](https://www.indiehackers.com/) forum?

<comment><AUTHOR> I posted there. The question is meant for IndieHacker community. The answer lies probably with senior management and HR guys.

</comment></comment><comment><USER6> Small companies often need generalists. I know that's what we prefer at my company.

</comment><comment><USER7> What exactly is an "Indiemaker"?

</comment><comment><USER8> Hmm, why not just stick to being an indiehacker then? Or stick to being a freelancer?

To be honest, I am curious myself...

<comment><AUTHOR> A regular job bring some steady flow of income and energy back in life.

If you have been a freelancer and a IndieHacker, I am sure you understand that it gets really stressful emotionally and financially.

<comment><USER8> You are right.

</comment></comment></comment><comment><USER9> would be interested in hearing about your experience building and selling saleztalk. have you written anything up about it?

<comment><AUTHOR> I was able to sell it via Flippa for $1300. I put up the listing and within 7 days, the deal was closed.</comment></comment>


"On Seeing A's and Seeing As" by Hofstadter - <AUTHOR> http://www.stanford.edu/group/SHR/4-2/text/hofstadter.html

<comment><USER2> In Zen, it is said that "aggregates" (things) have no "self nature", that all things arise in "codependent origination" -- you can't get a driver in a universe without cars, nor a car without drivers. One seeks not the "car-ness" in cars, but their context. The "seeing as" notion amounts to the same rejection of Platonic ideals.</comment>


PayPal's Beautiful Demonstration of Extended Validation FUD - Digit-Al https://www.troyhunt.com/paypals-beautiful-demonstration-of-extended-validation-fud/ <comment><AUTHOR> I run infra for a payments company and switched our main website from EV to DV last month silently. This is the landing website, not the application (which had always been DV). There have been ZERO people who have noticed, including peers in my Infra or Security teams.

<comment><USER2> It's obvious that EV does nothing for end users these days, which is why I think these "EVs are useless" articles from Troy and others are kind of silly, like dunking over a person with two broken legs.

I think the far more interesting question is _why_ EV certs do nothing for end users, and why browser makers have not introduced any new features to replace what EV certs were intended to do: help end users understand which websites they can trust on first visit.

The entire Internet industry has absolutely punted on this problem. It's a hard problem, but no one is trying to solve it!

And it is absolutely striking to compare against how well this problem has been solved in real life, at least in the U.S. You can walk into any retail establishment in the nation and reasonably expect to have a safe, productive interaction with a proprietor you don't know--even on the first visit.

It's the exact opposite on the Internet, and it's insane that everyone just accepts that.

<comment><USER3> > this problem has been solved in real life, at least in the U.S.

I haven't spent enough time in the US to have an opinion about it, but in many places this is not true. For example, taxi services in many countries seem to pretty commonly be ripe for dodgy behaviours with little accountability.

It's really about that accountability - what are you going to do against an internet entity which scammed you? How effective is that action going to be?

<comment><USER2> > It's really about that accountability - what are you going to do against an > internet entity which scammed you? How effective is that action going to be?

Accountability was supposed to be the extra value of OV and EV certificates.

If you look at the cert for [https://www.walmart.com/](https://www.walmart.com/), you see the following Subject fields:

        Common Name = www.walmart.com
        Organization = "Wal-Mart Stores, Inc."
        Locality = Bentonville
        State = Arkansas
        Country = US

The certificate authority (GlobalSign in this case) is attesting that this is accurate identification information for this company--that you can use this information to locate legal records for the company and contact it, sue it, investigate it, prosecute it, etc. DV certs do not have the bottom 4 fields.

That's what the famous "green banner" in the address bar was supposed to convey--that this website has a paper trail that can be used to impose accountability.

That's NOT a guarantee that the website will treat you well. It is a guarantee that you can try to do something if it doesn't. Even if a corporation is only established temporarily for the purposes of getting an EV cert to run a scam, that still leaves a legal paper trail. DV certs do not.

And this is a standards-based distributed system. It doesn't rely on one company like Google or Microsoft, or one particular browser, or even one particular CA. If Wal-Mart doesn't like GlobalSign anymore, they can go to another CA like Digicert. If you don't like Chrome, you can use Firefox--they can both read this cert information equally well.

IMO it is really a shame that the Internet industry was not able to lean into this concept of accountability. For example, why don't browsers surface this company contact info directly to consumers? The way that a consumer can easily find the street address for a store they visit? Instead it's buried 4 clicks deep in a complicated cert menu. Why don't they supplement it with a link to the corporate record of the company? Or to the Dun and Bradstreet listin? Or to the police department website in the registered locality? (Bentonville, AR in the case of Walmart) Or to the Better Business Bureau in that state?

It just seems like there was been the opposite of innovation into this concept of using certs to provide IRL accountability for websites.

<comment><USER4> For accountability: Knowing who to sue doesn't really get my $50 back from some jerk. The effort put into small claims makes it a net loss. My best option is a chargeback, and the cert doesn't do anything.

For reputation: I have no idea if that's the company name of Wal-Mart. It's barely better than verifying the domain name.

<comment><USER2> If you walked into a store on Main Street and got scammed for $50, would you just throw up your hands? Or would you call the cops? If you did call the cops, they would have the address of the store to start their investigation. And aside from maybe getting you your $50 back, the state might prosecute the scammer for committing a crime.

EV and OV certs are intended to provide similar information. If you report a scam by a website with an EV or OV cert, the police can use the information in the cert to start their investigations. There's more to accountability than small claims court.

Setting aside the cert issue, it's just striking how many people take it for granted that there are no consequences and no recourse for online crimes. How did we come to accept this as normal, as a society?

<comment><USER4> In person I'd try some things. Online I really doubt the cops are going to care about $50.

> How did we come to accept this as normal, as a society?

Probably because it's so much harder to pin down specific actors. That cert only does so much.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> I'm surprised no one has mentioned the main practical benefit of EV certs: the fact that a green cert in the browser ensures that there is no man-in-the- middle network proxy/appliance intercepting the traffic even if a trusted certificate for the proxy/appliance (or bad guy) has been placed in your machine's cert store. Browsers will only honor the EV setting in the cert if it was signed by a hardwired (and much shorter) list of CAs.

As long as you, the user, trust that the browser itself hasn't been modified (for example if you downloaded it yourself), then it's a nice reassurance when using a browser in a not totally trusted environment.

<comment><USER6> Browsers are moving away from showing differences between EV and DV certs nowadays though. And anyone can register a company name and get an EV cert - that doesn't make it trusted. [https://stripe.ian.sh/](https://stripe.ian.sh/) is a prime example of this.

HTTPS is a nice reassurance, sure, but an EV cert isn't.

<comment><USER7> > [https://stripe.ian.sh/](https://stripe.ian.sh/) is a prime example of this.

It used to be. All his EV certs were revoked.

<comment><USER6> Sure, but my point doesn't change. [https://scotthelme.co.uk/the-power-to- revoke-lies-with-the-c...](https://scotthelme.co.uk/the-power-to-revoke-lies- with-the-ca/) is a good write-up about that.

</comment></comment></comment></comment><comment><USER8> "PayPal really doesn't care that the world's most popular browser no longer displays the EV visual indicator."

OT but hopefully interesting: Paypal doesn't care about many things. Like keeping their callback JVMs up2date, or changing their UAs from defaults to disguise their enabling tech.

2019.05.20 xxxx|126.96.36.199|xxxx|POST xxxx/paypal/callback HTTP/1.1|200|954|-|Java/1.8.0_60|xxx|xxxxxxx 2

<comment><USER9> I'm really happy that Paypal ops is not upgrading anything just for the sake of upgrading it. At that scale, changes should be approached with extreme caution.

<comment><USER10> Well, perhaps they should upgrade it for the sake of the ~170 CVEs that have been published against Java 1.8.0_60? [https://www.cvedetails.com/vulnerability- list.php?vendor_id=...](https://www.cvedetails.com/vulnerability- list.php?vendor_id=93&product_id=19116&version_id=&page=4&hasexp=0&opdos=0&opec=0&opov=0&opcsrf=0&opgpriv=0&opsqli=0&opxss=0&opdirt=0&opmemc=0&ophttprs=0&opbyp=0&opfileinc=0&opginf=0&cvssscoremin=0&cvssscoremax=0&year=0&month=0&cweid=0&order=1&trc=597&sha=52d5e2189c154fbfa1e3d4ba0a13fc6b00642bc4)

Sure, a lot of those probably don't apply to how they're using Java, but I'd bet at least a couple do.

<comment><USER11> Vendors often backport CVEs without changing version numbers.

<comment><USER3> Or review the details and decide that the specific issue doesn't apply to them. This actually makes sense in larger environments.

</comment></comment></comment></comment></comment><comment><USER12> Firefox 66 does not show EV indicators, either, any more.

But I would also like to point out that PayPal is a known bad player when it comes to phishing, so no one really should give a damn what they do, and if you do the opposite of what PayPal does in security you are probably doing better than them.

<comment><USER13> Interesting because I've got Firefox 67 and Paypal.com does show EV indicator. Actually I see the same in Google Chrome 74.

<comment><USER14> I'm seeing the EV indicator with Chrome 74.0.3729.169 on Mac OSX Mojave 10.14 (18A391)

Edit: I also see the EV cert on Firefox 66.0.5 and 67.

I can't really tell if safari has an EV indicator or not? I really don't use it much but it looks like this [https://i.imgur.com/9asQkbN.png](https://i.imgur.com/9asQkbN.png) Safari version 12.0 (14606.1.36.1.9)

<comment><USER15> On MacOS, Safari uses the green (as shown in your png) for EV certs. HN does not appear to have an EV cert, as it's a gray padlock (not green). And, Paypal does appear with the EV green on my Mac as well.

</comment></comment></comment></comment><comment><USER16> Tangential question: neither McDiarmid's "Kill Sticky Headers" [0] nor its improved version [1] work on this website. Does anyone know how the bookmarklet can be changed to work here?

[0] [https://alisdair.mcdiarmid.org/kill-sticky- headers](https://alisdair.mcdiarmid.org/kill-sticky-headers)

[1] [https://news.ycombinator.com/item?id=19962875](https://news.ycombinator.com/item?id=19962875)

<comment><USER17> Is the bookmarklet being blocked by a CSP? There should be an error in the console about that (on chrome, at least)

<comment><USER16> Indeed it is. So I guess there is no way around it?

<comment><USER18> A browser extension. Most can run in a context above CSP.

<comment><USER17> Yeah, interestingly enough the CSP spec says that this behavior is wrong, but browser makers don't seem to care.

[https://bugzilla.mozilla.org/show_bug.cgi?id=866522](https://bugzilla.mozilla.org/show_bug.cgi?id=866522)

[https://bugs.chromium.org/p/chromium/issues/detail?id=233903](https://bugs.chromium.org/p/chromium/issues/detail?id=233903)

<comment><USER19> We do in fact care. I put up a fix on [https://bugzilla.mozilla.org/show_bug.cgi?id=1478037](https://bugzilla.mozilla.org/show_bug.cgi?id=1478037) just yesterday which will make [https://alisdair.mcdiarmid.org/kill-sticky- headers](https://alisdair.mcdiarmid.org/kill-sticky-headers) work in Firefox no matter what the CSP says. I fully expect to get that landed sometime in the next few weeks.

Making CSP not affect bookmarklets at all (as opposed to not affecting whether they execute) is a different situation entirely, because bookmarklets by definition execute in the JS environment of the page (unlike extensions, say), so it's not really possible to tell apart things that the bookmarklet does from things the page does. For example, what should happen if the bookmarklet calls a page-defined function that then does a subresource load? What if a bookmarklet defines a function to do a subresource load that the page later calls? Note that from the point of view of the JavaScript engine the functions the bookmarklet defines and the ones the page defines are indistinguishable from each other...

<comment><USER17> Sorry, I was a bit frustrated with how long that (and similar bugs in other browsers) have been open without resolution.

<comment><USER19> I can understand that, for sure!

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER20> On the HN thread on Troy's last post about this, I said [1]:

 _" Big sites can get by with DV because people trust big sites by fiat, just by mental associations they already have to a URL. There's no benefit to Facebook having an EV cert, because literally everyone who'd want to visit Facebook knows Facebook's URL. User error about entering credentials on the wrong site -- accidentally due to typosquatting, or through leading such as phishing -- is better mitigated in other ways: multi-factor authentication (especially unproxiable such as U2F); not by making the high-profile site pay thousands of dollars for a text string in green, when there's users who fall victim to phishing from bizarre domains too."_

Ultimately, this is a bad example to show that EV is pointless. The biggest benefit of EV is as a flawed signal of legitimacy [1] for sites whose URLs aren't widely known and get a fair amount of first-time visitors: web presence for real-life service businesses, specialized payment portals accessed through redirects, and the like.

This is because people's mental model of the trust that EV confers is broken. People typically care about whether the site they arrived at was the one they were intending to visit, which the computer can't possibly know without additional input, but EV has attained a role of serving as a flawed signal of such, because the browser bar said something that doesn't look alarmingly different.

EV formalizes the vetting between legal entity and domain name, so it translates okay to entities that are firmly anchored in meatspace. But all of this chaining is trust in people's heads is done by names and strings, and experiments like stripe.ian.sh prove [2] why it's fallible. Nonetheless, EV effectively allows one extra indirection between (1) the name of the business as people refer and recognize it, and (2) the domain name that's likely correct, than DV does -- and some operators and some visitors benefit from this indirection, when the URL doesn't roll off the tongue.

[1] [https://news.ycombinator.com/item?id=18010961#18011914](https://news.ycombinator.com/item?id=18010961#18011914) [2] [https://news.ycombinator.com/item?id=15904513#15909273](https://news.ycombinator.com/item?id=15904513#15909273)

<comment><USER21> > literally everyone who'd want to visit Facebook knows Facebook's URL

I'm not sure I'd agree with this. I don't think most people look at or really understand URLs or domain names. I think people assume they are a lot "fuzzier" than they are, so "facebookapp.com" or "facebook.foobar.com" or anything else would be assumed to be "Facebook" by most people.

As technical people it's easy for us to assume things that feel basic to us are at least understood but I don't think it's the case. My parents do most of their shopping online, and when I was growing up we always had computers around, but I don't think either one of them really understands URLs or things like file hierarchies, or windowing systems on desktops for example.

<comment><USER22> I have seen multiple people, mostly elderly, who will go to whatever search engine that loads by default in their browser, search for Facebook, and click on the link. Nothing close to everybody has Facebook's URL memorized.

<comment><USER21> Not everyone knows what a URL is, or understands the difference between a search box and a URL text box.

</comment></comment></comment></comment><comment><USER23> I'm one of those who doesn't pay any attention to those EV cards at all. I know I probably should, just out of completeness, but I don't. In fact, with Firefox anyway, that spot where the EV indicator goes is used for enough weird little things that I've sorta developed a blindness to anything that appears there (I'm not saying that's a good thing!)

However, if I'm going to a site that really matters, I'm not clicking on some link on a web page or email somewhere. I'm using the link I've bookmarked. So, ignoring the EV cert is probably not putting me at too much risk.

<comment><USER24> I (used to) implement EV certs among other duties and _I_ don't pay attention to them either.

Step One: Create Doofenshmirtz Evil LLC.

Step Two: Register HugsAndTrustBanking.com to LLC

Step Three: Get the legal services of Dewy Cheetum & Howe to issue a letter stating Evil LLC is your legitimate business.

Step Three: Show CA "proof" of ownership for Doofenshmirtz Evil LLC and HugsAndTrustBanking.com.

Step Four: Get genuine EV cert for your shady scam operation.

Step Five: Profit.

EV certs are dumb, you can trust me on that[0].

[0][https://i.imgur.com/1dbJUQ9.jpg](https://i.imgur.com/1dbJUQ9.jpg)

<comment><USER25> Steps 1-3 generate quite a paper trail for you and step 3 generates a paper trail for Dewy Cheetum & Howe.

Without an EV Certificate, you can just buy a domain name and use LetsEncrypt to generate a DV certificate which is a much smaller paper trail.

</comment></comment></comment><comment><USER26> "PayPal really doesn't care that the world's most popular browser no longer displays the EV visual indicator."

Funny story - I was about to rant that yeah, it probably wasn't by design because Paypal still as of 2019 didn't offer 2FA which, for a payment company, was quite disturbing...

well, turns out I was wrong, they now do. I just hadn't checked in a while, and of course received no email about it when they did turn it on... or maybe just missed it. Typically something you should advertise on each login if it's not activated, but hey, who cares, it's just money after all.

<comment><USER27> PayPal was very early in the 2FA movement, hardware security keys were available starting in ~2008 ([https://systembash.com/using-the-paypal- verisign-security-ke...](https://systembash.com/using-the-paypal-verisign- security-key-with-openid-for-two-factor-authentication/)). If you're referring to app based TOTP that is a relatively new option.

I was prompted to change to TOTP when I logged in.

</comment></comment><comment><USER28> > Note: yes, I know there can be regulatory requirements for EV in some > jurisdictions, but let's not confuse that with it actually doing anything > useful

That exactly. Similarly confused about customers who think OV certificates are better than DV - as far as I can tell no end user knows how to tell those apart in their browser (no 'green' locks...)

</comment><comment><USER29> Speaking of FUD:

> "the true meaning of SSL certificates"

The certificate fields for organisation, city, etc have existed and were expected to be verified since SSL was created in the nineties. Anyone who has ever made a CSR knows this.

Rolling back from checking the organisation to domain-only validation in 2003 was GeoTrust trying to increase profit margins. Again, Troy should already know this and it's surprising he doesn't seem to.

Oddly enough Troy also Tweets from a verified account, which seems somewhat unusual since a well known username should be enough according to his logic against verifying websites.

Heads up: I run CertSimple, a startup that does faster, simpler verification for EV.

<comment><USER30> I don't want this to end badly for you (via CertSimple), but as you perhaps come to anticipate my primary concern is always preventing Future Harm. EV hasn't been effective for that purpose.

The article I've just read never uses the phrase "the true meaning of SSL certificates" that I can see, if that was removed I apologise but otherwise I think I have to assume you're putting words in Troy's mouth.

"The certificate fields for organisation, city, etc have existed and were expected to be verified since SSL was created in the nineties. Anyone who has ever made a CSR knows this."

Anyone who understands what's actually going on here knows that these are part of the X.500 directory system and are present in Netscape's SSL because it leverages that systems's X.509 certificate format. In 1999 PKIX (RFC 2459) proposes how to use this system sensibly with the Internet and the modern Web PKI largely falls out of that and its successor documents.

Prior to the CA/B Forum (and the creation of Extended Validation) the only promises you had about what, if anything, in certificates you relied on had been "verified" and to what extent, was written in the legal documents of the issuing CA. In most cases they disclaimed all or almost all responsibility to the extent possible. Their methods were... unsound.

Even today, when if you run Firefox (or more or less, Android) you can actually trust that someone cares whether the validation was done properly, it's more slip-shod than any of us should want. What has Certinomis been up to for the past few years? What are all these certificates doing with ST=Some- State (yes, literally the words "Some-State", because that's the default in OpenSSL)? or L=Default City (again, the default in OpenSSL)?

I think we can reasonably conclude that the reason nobody engaged with my questions about those is that the answers would be embarrassing and they're hoping that if they stay quiet nobody will follow up by asking why they're filling this crap out (to make money) if they can't validate it properly...

<comment><USER29> The full sentence from the article is:

> Frankly, I think this is more a symptom of people coming to grips with the > true meaning of SSL (or TLS)

I did add 'certificates' so it read better (the system is PKI not SSL) but yes it's from the article.

Yep, they're x509v3 fields. Netscape was under no obligation to include them - they simply could have included CN only if they thought people didn't want to know the organisation they were communicating with.

However Organisation was included by Netscape and it was verified until GeoTrust invented DV (and later other CAs copied it to compete with the low margins of not checking identity. Totally agreed about the poor quality of verification during the 90s though (fax on company letterhead etc).

The one thing that hasn't changed in 20 years is that people still want to know who they're communicating with online.

<comment><USER29> (DC would also provide a tree structure if Netscape had wanted to use domain names as the exclusive form of identification - but they didn't. The objection was to verify enough information for commerce, which domain names clearly do not)

</comment></comment></comment></comment><comment><USER31> How long ago was it that users didn't notice or care if the URL was [http://](http://) rather than [https://](https://)? Yet no-one was declaring HTTPS dead on those grounds.

<comment><USER32> Apples and encrypted oranges. There's no actual technical difference between EV and DV for the end user, much unlike https vs http...

Why does my comment have to be spelled out anyway? I'm having a hard time believing you don't know that or haven't realized it.

<comment><USER31> > Apples and encrypted oranges. There's no actual technical difference between > EV and DV for the end user, much unlike https vs http...

What does it matter to the end user whether the difference is technical, legal, or procedural? The user just wants to be able to tell the difference between a legit site and a phishing spoof.

> Why does my comment have to be spelled out anyway? I'm having a hard time > believing you don't know that or haven't realized it.

The article points out that users and paypal didn't care that their domain wasn't using EV, as if this were a slam-dunk argument against it. But that argument would go through just the same for a domain that wasn't using HTTPS at all.

<comment><USER32> The difference matters because http vs. https makes a difference regardless of what the user or HN commenters think, whereas EV relies entirely on the user acting in a way that is entirely theoretical and not seen in practice.</comment></comment></comment></comment>


Google's Intelligence Designer - <AUTHOR> http://www.technologyreview.com/news/532876/googles-intelligence-designer/ <comment><USER2> These MIT Tech Review articles, alas, emphasize hype: "No one had ever demonstrated software that could learn to master such a complex task from scratch," and "But until DeepMind's Atari demo, no one had built a system capable of learning anything nearly as complex as how to play a computer game, says Hassabis."

I think the article must have overlooked significant activity in training learning systems to play games well. The glaring omission for me was Neurogammon (1987), later TD-Gammon (1992), developed by Gerry Tesauro and colleagues ([http://en.wikipedia.org/wiki/TD- Gammon](http://en.wikipedia.org/wiki/TD-Gammon)).

Neurogammon was, at the time, a sensation at the same conference the article coyly refers to as "a leading research conference on machine learning." The paper has almost 1000 citations. A curious omission.

<comment><USER3> Aren't these quite different tasks, though? There's a big difference in 'learning to play a specific game well' vs. 'learning to play arbitrary games'; such a big difference that I think they're entirely different disciplines. Correct me if I'm wrong, but the software in the research you reference was given the ruleset to the game, right? And DeepMind's software is not given that information, I think. I doubt they intentionally omitted that work, I think it's more likely they didn't consider it relevant enough.

<comment><USER2> Thanks for the correction. The "arbitrary" qualifier is not in TFA, but (as, indeed, you said) that's the point of the demo, e.g.: [https://www.youtube.com/watch?v=EfGD2qveGdQ](https://www.youtube.com/watch?v=EfGD2qveGdQ) Note that they're using just the video signal from the game as input.

It's really a sad comment on the state of reporting at _MIT Tech Review_ that you learn more about the tech from a youtube video than from an article.

(My complaint is not with the DeepMind people, it's with the article, which should put the work in context.)

<comment><USER4> > It's really a sad comment on the state of reporting at _MIT Tech Review_ > that..

I feel compelled to point out that the only connection between the "MIT" tech review and MIT is that the magazine licenses the name from the alumni association. It's how the alumni associations funds itself and every MIT grad gets a lifetime subscription to a version of the magazine with the alumni notes bound into the back. I doubt many of us read it. I don't know how many people _other_ than MIT grads read it, but I would imagine vanishingly few.

A friend of mine calls it "the magazine of things that will never happen" which I think is dead on. It's a shame because the editor, Jason Pontin, as actually a good guy so it's surprising that the magazine continued to suck after he took it over.

There are many reasons to criticize MIT (don't I know it!) but you can't judge the institute by this magazine.

<comment><USER5> I'm going to disagree a bit here. Tech Review does tend to focus on the possibilities of technology and to highlight potentially exciting research. Almost by definition, a lot of this stuff is never going to amount to anything commercially interesting. I suppose that TR could insert more implicit or explicit disclaimers to that effect but I find it a good source for insights into what's going on in the labs.

Personally, I think that Jason has brought a lot of positive changes to a magazine that, for a long time, tended toward a technology policy wonkish orientation.

So I think it's fair that a lot of what's written about "will never happen." But I'm not sure that's really avoidable if you cover cutting-edge research.

</comment></comment></comment></comment></comment><comment><USER6> FYI this is also the guy that made Elon Musk fear strong AI. Elon Musk invested in DeepMind in the early days just to see where AI is going.

</comment><comment><USER7> For a brief, horrifying moment, I thought this was the name of a product.

What a time to be alive.

</comment><comment><USER8> Machine learning folks don't know the history of CS or AI, so they've reinvented neural networks as "deep learning"?

Or, industry types are looking for the next big thing, after "big data," and have rebranded neural networks as "deep learning"?

I don't mean to be too cynical, but I still don't understand if "deep learning" represents any meaningful advance besides the ML and EE communities finding the benefits of a certain amount of structure, which is already well established in other lines of research.

<comment><USER9> Deep learning is not just neural networks, but rather the application of these in deep (i.e. many-layered) architectures, broadly speaking.

This enables hierarchical learning of increasingly complex concepts – building new concepts upon less complex concepts from previous layers. Deep architectures are thus able to learn high abstractions, as in [1], for instance.

If you have not yet done so, I would strongly urge you to read some papers on the subject from the last decade (e.g. Hinton, Bengio or LeCun), or even just skim through the Wikipedia entry [2].

[1] [http://www.technologyreview.com/view/532886/how-google- trans...](http://www.technologyreview.com/view/532886/how-google-translates- pictures-into-words-using-vector-space-mathematics/)

[2] [http://en.wikipedia.org/wiki/Deep_learning](http://en.wikipedia.org/wiki/Deep_learning)

<comment><USER10> Deep learning is a large scale application of Restricted Boltzmann Machines, of which Hinton (among others) was a pioneer. But that was in the 80s, not in the 2000s.

[http://en.wikipedia.org/wiki/Restricted_Boltzmann_machine](http://en.wikipedia.org/wiki/Restricted_Boltzmann_machine)

<comment><USER9> I don't believe the term "deep learning" is restricted to RBMs only – at least that's not the way I've seen the term used in literature (e.g. Deep Convolutional Neural Networks, various deep Autoencoders, etc.).

<comment><USER10> Convolutional networks were also developed in the 80s as well as backpropagating algorithms (autoencoders). The way i see it used, "deep" usually means many layers, indicating a difference in quantity, not in quality.

Point is, the science was there since the 80s, and not much has changed.

<comment><USER9> Sure, but these types of deep architectures haven't really been practical until relatively recently.

Well, then we're in agreement about the meaning of the term. Deep Learning, then, would be Machine Learning using any of these deep architectures – be they Restricted Boltzmann Machines, or otherwise.

</comment></comment></comment></comment></comment></comment><comment><USER11> This guy vs Shingy, AOL's "Digital Prophet".</comment>


Show HN: A browser extension that makes switching between tabs easier - <AUTHOR> https://chrome.google.com/webstore/detail/popup-tab-switcher/cehdjppppegalmaffcdffkkpmoflfhkc <comment><USER2> Nice work with this extension, I like that you've made it quite configurable. I personally don't like the auto-switch or scroll effect, but it's good that you added the ability to turn these off using 0 values.

Just a couple of things that I'd suggest to add some improvements to the extension:

\- One thing that I'd suggest is adding the ability to Alt + Y, but keep the popup open. This means that you could then do something like navigating the list with tab/hkjl/arrow keys. Without this functionality, the extension basically works the same as pressing Ctrl + Tab, but with a rendered list instead.

\- Another thing that would be cool to add is a search function, so that I can Alt + Y, type "hn" or "ycom" and the list would be filtered to show news.ycombinator.com ( also matches the HN ) in the tab title.

Both of these are already available in the vimium extension [1], but would be kind of neat to just have a standalone extension for this functionality.

[1] [https://github.com/philc/vimium](https://github.com/philc/vimium)

<comment><AUTHOR> Thank you for your ideas! I have a plan for adding the search and the ability to focus on the popup and I want to merge them together. I think that Alt+Shift+Y is not very helpful when you press it the first time, because it just selects the oldest of the visited tabs. And want it to focus on a special search item. Here is an image of what I want to do [https://imgur.com/a/qlfVT0M](https://imgur.com/a/qlfVT0M). So, what do you think?

</comment></comment><comment><AUTHOR> Hello HN!

The Popup Tab Switcher is an attempt to bring the power of your system app switcher to a browser. The extension shows you a popup with last active tabs when you press its shortcut (Alt + Y by default). You cycle through them by holding Alt and pressing Y. By releasing the Alt key you switch to the chosen tab. This is like switching between apps in Windows (Alt + Tab) and in macOS (Cmd + Tab).

Also, when you close an active tab, you will be placed to the previously active one. It is more helpful than the default Chrome's behaviour, which activates the nearest tab.

You can delegate the default switching between tabs (Ctrl + Tab) to the extension, see how [https://github.com/dvdvdmt/popup-tab-switcher#replace- defaul...](https://github.com/dvdvdmt/popup-tab-switcher#replace-default-tab- switching-behaviour)

Thanks for you time, any feedback is welcome!

</comment><comment><USER3> Wow, that's an amazingly polished extension! I couldn't help but sniff out the [1] github link, and noticed you were using vue.js, I used vue too with my extension [2] Newsit, check it out to see how you can deploy to both Chrome and Firefox (as the extension APIs are _almost_ standardised).

The only issue I found, was that the popup switcher doesn't show up on the new tab page, which makes it a little confusing.

[1] [https://github.com/dvdvdmt/popup-tab- switcher](https://github.com/dvdvdmt/popup-tab-switcher)

[2] [https://github.com/benwinding/newsit/](https://github.com/benwinding/newsit/)

<comment><USER4> The New Tab page is a Chrome-specific page (chrome://newtab). Extensions cannot run on these browser-specific pages, and are essentially disabled. You can "forcibly" allow extensions to run on these pages by enabling this flag: chrome://flags/#extensions-on-chrome-urls

<comment><AUTHOR> Thanks for sharing the info about this flag, I'll definitely experiment with it.

<comment><USER4> No problem, and great extension! :)

</comment></comment></comment></comment><comment><USER5> Is there a FF version tho?

<comment><USER6> Firefox has this built-in, no? You can quick-switch to tabs by using sigils in the AwesomeBar, and you can click the drop-down button in the tab bar to get a list of open tabs.</comment></comment>


List of eponymous laws - <AUTHOR> http://en.wikipedia.org/wiki/List_of_eponymous_laws

<comment><USER2> I am now addicted to this page. Thanks for sharing.</comment>


Ask HN: How to gain experience in Ops - <AUTHOR> I have lots of 'traditional' systems administration experience.  By that I mean windows domain environments, networking, a little Linux experience, etc.  Basically anything a non-technical small to medium business would need.  I have a little experience in programming, but not a huge amount.  How can I get experience in cloud computing or infrastructure? It seems like in 10 years I will not have a job if I don't adapt.  I don't live in a place that has a huge tech culture or technology job market. <comment><USER2> I am working on an open source project around Ops (Chef, Mesos, Docker etc...).

To me, I have learned the most while pairing with someone who had MUCH more experience than me.

So it's time for me to pass that karma along and help others. If you want to pair on this project, I'll be happy to.

Email is on my profile. Invitation open.

Edit: Just to clarify here. This is NOT an invitation for services. These are completely free of charge pairing sessions.

</comment><comment><USER3> I'm working on a tutorial on automated configuration management and I would love some feedback from users coming from a Windows background. Can you shoot me an email at firstname.lastname@example.com ?</comment>


Firefox to disable extensions in private browsing - <AUTHOR> https://twitter.com/FirefoxNightly/status/1103593021122248707 <comment><USER2> But why Firefox is doing this? What is the reason?

<comment><USER3> Browsers have been split on this: E.g. Chrome (and I think Edge) requires you to explicitly opt-in each extension for private mode too.

Generally, the reasoning is that private browsing promises to leave as little record as possible of what you do using it, and browser extensions can (not even necessarily maliciously) break that promise. Hiding private windows from extension by default means only those extension the user explicitly allowed access can do that.</comment></comment>


Jubilee – A Ruby Rack server with Vert.x built in - <AUTHOR> https://github.com/isaiah/jubilee

<comment><USER2> This can actually be huge if well-executed. The vert.x ruby examples have had no rails afaik.

Look forward to playing with this.</comment>


"an ex administrator has deleted all customer data and wiped most servers" - <AUTHOR> http://verelox.com/ <comment><USER2> for posterity: [http://web.archive.org/web/20170609142424/http://verelox.com...](http://web.archive.org/web/20170609142424/http://verelox.com/)</comment>