Debunking the duct tape programmer - <AUTHOR> http://jeffreypalermo.com/blog/debunking-the-duct-tape-programmer/ <comment><USER2> The article misses the point that Spolsky makes: some people are smart enough that they can foresee _where_ the duct tape can be applied (and, by extension, where it would be catastrophic to do so).

This is, of course, a far cry from the person who does everything with duct tape, because it's simpler to do so. And it's also a far cry from the person who always follows his/her standard approach, because nobody ever got fired for [using the safe-but-costly approach].

In essence, the duct tape programmer has a safety margin between the smartest he could do, and what he actually does, and uses this safety margin (1) to make the software actually work, because he can avoid stupid errors, and debug the subtle issues that come up, and (2) to make the software ship on time. Not using your most-advanced-yet-poorly-understood techniques does not mean you'd stick to the stone age approach, but that you're perfectly fine with using stone age tools in addition to the fancy-schmancy newfangled stuff where it makes sense.

I, for one, won't Party with Palermo on this. I'm more cautious of people making their money with agile training than with people selling bug tracking systems.

<comment><USER3> I agree with you, but the point is damn well hidden behind one of the crappiest analogies of all time: using duct tape is an indication that you're great engineer - but _only_ if you are one of the naturally talented "pretty boys" who know what they're doing.

</comment></comment><comment><USER4> "Analysis Paralysis" often has a root cause: incompetence.

A pal of mine was a consultant for a logistics company serving a large country that's north of the U.S. This company was developing a package tracking system in Java, back around 2000 or so, when "Java" was often a synonym for "Project Failure."

Now, when the whole thing started, I told him that I could knock off a prototype in a week in PHP. Nope, these guys decided to spend five months developing a framework for their application, and then they were going to spend the next month developing the actual app.

Well, six months later, I hear that the predictable outcome had happened: management was shocked to discover that the application took 30 seconds to serve up a page view.

My pal sent me a copy of the app and it was an absolute rat's nest: the kind of project where you use 1500 lines of code to say what you could say in 15. The sheer bulk of the code made it difficult to maintain: and it had poor performance written into it's DNA. The real problem was that this wan an IBM shop that was developing its first webapp, developing its first Java app and... nobody knew what they were doing.

\-----

Don't get me wrong. There's nothing wrong with front-loading your design and analysis work... IF YOU KNOW WHAT YOU ARE DOING! In the hands of people who have a holistic view of the application and toolset it's based on, front- loading is a beautiful tool that can make implementation and testing a breeze.

However, because it delays feedback, prolonged analysis doesn't (necessarily) test the ideas behind a design. Bad ideas can get in, and stay in -- nothing challenges the arrogance of the uninformed.

If, on the other hand, you are humble, and "know what you don't know", a rapid prototype can be a powerful way to get some experience, test your ideas, and, who knows, maybe even ship something.

<comment><USER5> I would argue a different root cause for "Analysis Paralysis": fear.

Most people who analyze things to death fear making decisions that result in a mistake (subsequently resulting in job loss). The costlier a decision, the more likely the paralysis will occur.

And quite often, these people are technically competent.

</comment></comment><comment><USER6> _never take software advice from a bug tracking system salesman_

Beautiful.

<comment><USER7> Spolsky seems to be a less fashionable idol than he was when I started to become interested in startups. In the context of programming AND startups, his opinions have much more weight to me than XP proponents'.

Joel has created a successful company that is apparently fun to work in, have shipped several working and commercially successful products and made a service for human-kind by designing and successfully launching programming Q&A site that doesn't suck. Plus his writings inspired a generation of programmers to start companies, just like PG's writings did later.

Contrast this with Extreme Programming that OP embraces as a third option compared to duct tape programming vs analysis-paralysis. I was very excited about XP when I was young and inexperienced programmer, but nowadays XP sounds much more snake-oil salesmanship to me than Joel's opinions. Did original XP gang work in startup like environments, or were they actually solving problems (payroll systems) that they and many others had already solved? And Kent Beck seems to be still working on jUnit (and trying to find a way to monetize it), not shipping new products.

My kudos goes to Joel and duct tape programmers around the world.

<comment><USER6> Joel seems to talk a lot of sense but this is tempered for me by two things. 1) no-one I know uses any of his products and 2) I have a hard time believing he really is able to attract the top talent to work on a project management tool written in VB.

<comment><USER8> It's written in ASP.NET, not VB.

<comment><USER9> Really? Since when?

<http://www.joelonsoftware.com/items/2006/09/01.html>: "FogBugz is written in Wasabi, a very advanced, functional-programming dialect of Basic"

<http://www.joelonsoftware.com/items/2006/09/01b.html>: "And since FogBugz goes back many years and was originally written in VBScript, Wasabi is 100% backwards-compatible with VBScript." (granted, VB != VBScript)

<http://fogcreek.com/Jobs/Dev.html>: "Today we happen to use Wasabi, JavaScript, xhtml and CSS, and C++ to build FogBugz"

<comment><USER8> I'm totally wrong. I meant to write .NET (not ASP.NET) and .NET isn't even a language, so I'm doubly wrong. It's probably been four years since I've touched the code :\

It isn't written in VB and never was. In 2000 it was written in classic ASP (vbscript) and c++. Probably a few years after that it was converted to Wasabi, which at the time was very close to VBScript, but is now equivalent to C#. For the last few years it is shipped as a .NET product on both windows and unix, and the /Dev.html page you cited is accurate. (The articles you cited from four years ago aren't accurate anymore).

<http://xkcd.com/386/></comment></comment></comment></comment></comment></comment>


Knuddels: Chat platform must pay after hacker attack fine - <AUTHOR> http://www.tellerreport.com/tech/--knuddels--chat-platform-must-pay-after-hacker-attack-fine-.S1xs7l4Am.html <comment><USER2> I see this is based on a Spiegel article. Can someone link to the full ruling? One thing I am curious about is how the number was derived.

Regardless, even if you disagree with the scope of the law (which I do but not the intent of course), it is a very welcoming sign to see some actual enforcement happening. An under/subjectively enforced law of this size is much worse than a reasonably enforced one.

<comment><USER3> Here's the Data Protection Officer's press release (in German): [https://www.baden-wuerttemberg.datenschutz.de/lfdi-baden- wue...](https://www.baden-wuerttemberg.datenschutz.de/lfdi-baden-wuerttemberg- verhaengt-sein-erstes-bussgeld-in-deutschland-nach-der-ds-gvo/) \- the relevant paragraph is towards the end. The tl;dr is that the fine is rather low because they were very cooperative and quick to follow suggestions for improvement, and have additional improvements planned. Also, they likely couldn't afford more, and the goal of GDPR fines is not only to be effective and a deterrent, but also proportionate. The DPO says that including the cost of IT security measures taken and planned, the total expense for Knuddels is a six-digit figure.

<comment><USER4> I think the cost of complying with the law is just cost of conducting a business lawfully. So, those parts should not be spoken of as if they were part of the damages.

<comment><USER3> Well obviously they should have improved their security practices years ago. Nonetheless, they incurred the costs now, as a result of self-reporting a data breach (which is mandatory). That's the number that's relevant as a deterrent for others, so I think it's fine to report it this way.

</comment></comment></comment></comment><comment><USER5> Full list of 5000+ websites that store their passwords in plain text: [https://github.com/plaintextoffenders/plaintextoffenders/blo...](https://github.com/plaintextoffenders/plaintextoffenders/blob/master/offenders.csv)

<comment><USER6> That list is very out of date. One of my clients appears on there and when we took over in 2012 we encrypted all their user credentials.

<comment><USER7> Their FAQ [1, 2] suggests that using an encrypted password still warrants an entry.

[1] [http://plaintextoffenders.com/faq/devs](http://plaintextoffenders.com/faq/devs)

[2] [http://plaintextoffenders.com/faq/non- devs](http://plaintextoffenders.com/faq/non-devs)

</comment></comment></comment><comment><USER8> _" Knuddels is safer than ever."_

Corporate speak is just so funny. The bar for "safer than ever" is pretty low when your dev team hasn't heard of password hashing.

<comment><USER9> The passwords to login where actually hashed. But they stored another copy in plaintext on purpose, to censor the users password if they wrote it into chat...

<comment><USER8> _" Passwords were hashed as a hash in 2016, but the unchanged version of the passwords has been retained, so users can not filter their own password via our platform via a filter"_

[https://www.archynety.com/tech/why-knuddels-saved-his- passwo...](https://www.archynety.com/tech/why-knuddels-saved-his-passwords-in- plain-text/)

Which sounds odd. You could just hash/compare filter words.

I'm guessing similar issues too, like "no salt" or "same salt for all passwords".

<comment><USER10> If you do password hashing properly, using a key derivation function, you shouldn't be able to do that filtering efficiently at all.

<comment><USER8> It just has to run once, at filter creation time. Disallow creation of the filter if your password is in it.

Note the quoted reason from Knuddles is different from what others are saying the reason is: _" so users can not filter their own password via our platform via a filter"_

Edit: Apparently, the posted articles on this are misquoting things. Here's the original company response: [https://forum.knuddels.de/ubbthreads.php?ubb=showflat&Number...](https://forum.knuddels.de/ubbthreads.php?ubb=showflat&Number=2916515#Post2916515)

It does appear they were screening all chat text for the user's password after all.

</comment></comment></comment></comment></comment><comment><USER11> This is actually a cool idea - paying idiot tax

<comment><USER12> In my experience, this is mostly what the GDPR is. There is no excuse for storing plaintext passwords in 2014+ and 20k is a fair fine for a mid-size company.

<comment><USER13> €20k doesn't seem much to me. Cheaper than taking on a security consultant.

Not that you need a security consultant to know passwords shouldn't be stored (at all, nevermind plaintext).

If they're doing that then they're likely being sloppy elsewhere, and by only paying €20k across the last n years they might have saved a €million.

If your company is in the same boat probably worth not bothering to get any security issues addressed. Why address security, just pay the much smaller fine if you ever get caught ...

I couldn't find Knuddels annual profit but they appear to have a dozen staff, which suggests to me the fine is too small.

<comment><USER3> Well, the fine is only 20k€ because they were very cooperative, quick to fix the worst issues, and promised to continue improving their security further. According to the Data Protection Officer's statement their total expenses were in the six figures. They also explicitly state that the fine wasn't higher as not to place a disproportionate burden on the company's finances, which probably means that they wouldn't have been able to afford a significantly higher fine. Contrary to the fear-mongering on this site, the purpose of GDPR isn't to fine companies out of existence.

<comment><USER13> Yes, I saw that elsewhere when I'd finished writing.

More cooperative still would be doing the changes before you're caught.

If you could skip your tax bill for a few years, but get a much smaller fine if you cooperated when caught then you'd be silly to actually pay.

In short, in terms of _pour encourage les autres_ this fails badly IMO.

<comment><USER14> > If you could skip your tax bill for a few years, but get a much smaller fine > if you cooperated when caught then you'd be silly to actually pay.

If you do the work ahead of time, you pay the cost of doing the work. If you wait for the fine, you pay the cost of doing the work plus the fine. It doesn't take a lot of fine to make doing the work to begin with worth it -- basically just accounting for chance of getting away with it and time value of money, which goes _down_ as the government gets better at catching more people quicker, as should be their primary goal for something like this.

<comment><USER13> Maybe, but there's much more work that needs doing to secure PII than just not having plaintext passwords. So, they can seemingly avoid doing all that work too, and maintaining those systems (with staffing costs). And you get a leg-up over the competition who can't use the cash that they put in to security.

That means those with poor security regimes may "win" because the costs of poor PII hygiene are externalised.

It would certainly be nice to imagine all the 2 million UK corporations are addressing PII security rather than hiding and hoping not to get a fine ...

<comment><USER14> > Maybe, but there's much more work that needs doing to secure PII than just > not having plaintext passwords. So, they can seemingly avoid doing all that > work too, and maintaining those systems (with staffing costs).

If the regulators catch someone breaking a rule like this, the consequence should obviously involve an audit that looks for other violations and requires them to fix those too.

But even if it didn't, your conclusion wouldn't follow, because they would still have no incentive to fix the other problems unless they expected to get caught for not fixing them. But if they did expect to get caught then the numerous predicted small fines would be a sufficient deterrent.

> It would certainly be nice to imagine all the 2 million UK corporations are > addressing PII security rather than hiding and hoping not to get a fine

It's the hiding and hoping not to get a fine that's the reason large fines don't work. Higher penalties can't deter someone who doesn't expect to be caught.

What works is smaller penalties with vigorous enforcement.

<comment><USER13> Thanks for expounding your position, I still disagree however.

The analysis is similar to a parking fine, if the fine is €1 but parking is €2 per hour then people will chance it.

If the fine is having your car towed and €200 then people will be damned sure not to go even a minute over their paid time.

<comment><USER14> Parking fines aren't designed for deterrence, they're designed for revenue generation. If parking is €2 and the fine is €1 on top of the parking cost if you get caught (plus €5 worth of inconvenience doing fine paperwork), and there is a 90% chance of getting caught, nobody parks illegally -- and therefore there is no fine revenue.

But if you make it a $200 fine with a one in a thousand chance of getting caught, then it's profitable, because then many people rationally take the risk and become a source of citation revenue. But the violation rate is higher, so if that was your goal, it fails -- unless you're still doing vigorous enforcement, in which case high fines are once again unnecessary.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Maybe [https://www.plus.net/](https://www.plus.net/) should be next. Reported few times without any result.

<comment><USER16> Report for what? Storing plain text passwords?

<comment><USER15> Reported to them that it's not a good idea.

</comment></comment></comment><comment><USER17> Seems like they got punished for informing the government about the hack, obviously gives the next company that gets hacked a reason to try to hide it.

<comment><USER18> The fine was set low for cooperating with the DPO. If you hide it and someone leaks it then the agency could ask for much more.

</comment></comment><comment><USER19> I read the headline as saying that Knuddels was a platform for chatting about GDPR, which made it all seem very ironic.

edit: title has been changed, nevermind

</comment><comment><USER20> Funny to speak about security and a GDPR fine for a website that hasn't HTTPS activated...

<comment><USER21> I don't see much reason why their website should have HTTPS though.

There are no input fields, no requests sent with personal information at all etc.

Everything that's questionable already comes over HTTPS on their site though, like Facebook content etc.

</comment></comment><comment><USER22> Attack, succeed and blackmail could become a business. "If you don't pay me X we'll report you under GDPR and you'll have to pay much more."

<comment><USER23> "If you don't pay me X we'll report you under criminal law and you'll have to pay much more."

"If you don't pay me X we'll report you under environmental protection law and you'll have to pay much more."

"If you don't pay me X we'll report you under labour regulations law and you'll have to pay much more."

How would GDPR be special?

<comment><USER24> In the extreme large fines.

<comment><USER3> They paid a 20.000€ fine. How is that extremely large?

<comment><USER24> I wasn't talking about this specific case. The regulation allows larger fines than that, and some people fear that. I myself disagree with them, btw.

</comment></comment></comment></comment></comment><comment><USER25> laughably little though (20k)

</comment><comment><USER26> According to the link: [https://www.baden-wuerttemberg.datenschutz.de/lfdi- baden-wue...](https://www.baden-wuerttemberg.datenschutz.de/lfdi-baden- wuerttemberg-verhaengt-sein-erstes-bussgeld-in-deutschland-nach-der-ds-gvo/)

They were doing this so they could filter out the passwords from chats (i.e. to make it so users can't give out their passwords to other users). Not saying this justifies it, but it's interesting.

<comment><USER27> It's possible to do that without storing the passwords in plain text though! Run each word of the chat though the same hash+salt mechanism and compare to what you have stored.

<comment><USER28> Assuming they're using a suitable hashing algorithm for passwords (ie, Argon2, bcrypt, scrypt, PBKDF2), this approach would be prohibitively expensive, especially for a chat platform, with presumably lots of messages.

Also, you probably can't just try hashing each word, since there could be whitespace and punctuation in the password text, so I think you'd have to hash all possible substrings of each message to be able to reliably catch passwords.

Obviously, though, they shouldn't have been storing them in plaintext.

<comment><USER29> Store the length L of the password, its salted hash H, and its bytes, XOR-ed, X.

For every message typed, compute a running XOR of each sequence of L bytes (2 XOR's per character, so as good as free). Whenever it equals X (about once every 64 letters or so, because typical text doesn't use all bits in each byte equally), compute the salted hash of the last L characters, and compare with H.

Unicode and Unicode normalization will complicate that, but I think it should be fast enough for a chat.

You probably can also improve on that factor 32 by storing multiple XOR-like (but slightly more computationally expensive) hashes and computing multiple running totals.

Given that this is to protect users from falling for scammers who claim they need their password to help them, you may be able to run it on the user's machine.

I fear, however, that a scammer will just ask them to type their password with a space inserted, spell it in the NATO spelling alphabet, or whatever. If you fall for a scammer, that won't stop you from giving them your password.

<comment><USER28> I did think of similar approaches, but anything I could think of that helps you to quickly determine if a given string contains the password also helps an attacker if the passwords and salts are compromised.

In the suggested case, storing the length of the password alone massively reduces the search space, and storing the XOR (of the plaintext with the hash, I think you're suggesting?) negates the value of using a hashing algorithm suitable for passwords, since the point is that checking if a password matches a hash is an expensive operation.

</comment></comment></comment></comment></comment><comment><USER30> Who gets the money, after those who administer it take there cut

<comment><USER31> The state, Baden-Württemberg.

</comment></comment><comment><USER32> GDPR is not supposed to be enforced after a 2 year grace period ?

<comment><USER33> That 2 year grace period ended in May 2018</comment></comment>


Why Economic Models are Always Wrong - <AUTHOR> http://www.scientificamerican.com/article.cfm?id=finance-why-economic-models-are-always-wrong <comment><USER2> I think what the author is describing is simple overfitting.

<http://en.wikipedia.org/wiki/Overfitting>

It is quite a newbie mistake for a scientist to be surprised by it. It affects every kind of modelling.

I thought maybe this article would talk about why economic models are worst than other kinds of models. There are issues that arise when applying scientific models to the economy caused by the fact that when even good models are used to predict markets, the use of the models themselves to do trading, distorts the markets. When multiple parties use good models to compete in markets, they distort the markets in such a way that destroys the predictive power of the models.

There is a great explanation by Glen Whitman of Agoraphilia, that uses grocery line wait time predictions as a metaphor for this:

<http://agoraphilia.blogspot.com/2005/03/doing-lines.html>

See also:

<http://lesswrong.com/lw/yv/markets_are_antiinductive/>

<http://en.wikipedia.org/wiki/Efficient-market_hypothesis>

<comment><USER3> Alternatively, it may be simple information theory: A model that takes in 100 bits of specification simply can not correctly describe a process that has 10,000 bit's worth of degrees of freedom. And that's _before_ we talk about iteration over time, and before we get to the final killer you mention, which is when the models are ruined by their own application to the domain.

I think radical underspecification is much more likely than overspecification, really.

(Since I encounter this a lot, let me pre-answer one question in advance, which is "What if only 300 bits really matter and the rest don't matter as much?" and the answer is that the term bit in information theory encompasses that idea already. If you have ten "bits", but they tend to be highly correlated together such that they are usually all 0 or all 1, you in fact _don't_ have ten bits in information theory. Ten bits are, by definition, ten fully-independent true or false values. Bits-in-memory are not the same as information-theory-bits. A real system with 10,000 bits can not, pretty much by definition, be modeled by 100 bits. If it could, it would be a system with only 100 bits in the first place. Information theory cares about the true degrees of freedom available, not about your particular representation of the system.)

<comment><USER4> Here's the thing: you're both right. It's both radically underspecified and overfitted. The information-theoretic argument demonstrate that a model cannot exactly match the reality unless it's as complex as the reality.

This article speaks of the separate problem that economic models are not evaluated in any sort of experiments, and thus are prone to overfitting. This makes them unlikely to even approximate well.

Consider a basic multilayer perceptron-style neural network. Overfitting is a well-understood problem in training an MLP. We work around it by training on a part of the data, and then measuring its accuracy on another part -- much as Carter did in his analysis. If the accuracy is poor, something is adjusted: the size of the hidden layer can be increased, the training set expanded, the duration of the training increased or decreased, or the MLP model discarded entirely.

If increase of the training set or reduction of the duration improves accuracy against the test set, this means we had an overfitting problem.

<comment><USER5> "It's both radically underspecified and overfitted."

He used a perfect model (of a hypothetical world) which had exactly the right parameters, and then he calibrated it using exactly correct data.

So I don't see how this could be underspecified or overfitted. Can you please explain?

"The information-theoretic argument demonstrate that a model cannot exactly match the reality unless it's as complex as the reality."

In this case he defined his model to be reality.

<comment><USER4> Those particular statements referred to some representative economic model, not the experiment in question. In the experiment in question, the model is fully specified by definition.

As far as overfitting goes, that applies when you have a parameterized general model and need to discover the correct parameters. You probably won't get the exact correct parameters; instead, you'll (hopefully) get parameters that approximate reality well.

More closely matching the training data can actually make it a worse approximation in the general case.

</comment></comment></comment></comment></comment><comment><USER6> This is known to anyone who's ever monkeyed with any type of machine learning: genetic algorithms, Bayesian filters, anything.

I agree with many of the commenters in this article. This should be common knowledge.

I also, like many commenters, couldn't help but think of model-based climate predictions.

<comment><USER7> The problem is that you're comparing statistical methods with process-based methods. The mathematically inclined tend to have a reflex to approach modeling wiht this sort of black box methods. The thing is that for modeling processes like geomorphology, hydrology but also less quantitative processes like quality of life in urban environments, black box methods cannot be verified nor reasoned about - with issues like overfitting etc. becoming a problem.

On the other hand, you can model by building conceptual models, calibrating them by hand (using computer methods for the number crunching only) and reasoning about divergences between model results and observed data rather than computing them away with raw power. This is what modeling should be about - a tool for understanding.

(this topic is dear to my heart - I have had this discussion so often. Models are not crystal balls, they are tools for understanding processes. Which is why I am so desperate when another economist, mathematician or computer scientist stands up and wants to model processes that require understanding with their barbaric brute force statistical methods to not have to study things that are outside of their comfort zone. When all you have is a hammer etc.)

</comment></comment><comment><USER8> This article is more about how multiple sets of parameters can fit the same data equally well. This is why economists draw a distinction between calibration and estimation. If a parameter is "identified" in some estimation procedure, they mean they have an experiment or quasi-experiment that gives them a credible CI for the true parameter.

</comment><comment><USER9> I think that with economic models used for trading there is also another big problem: Their application changes the model itself. So, even if you had a perfect model for the market without you applying your model, as soon as you start applying it, the market changes... and this is also true for all the other quants who do the same with their models.

IMHO, it was much better when most stock market decisions were mostly based on "fundamentals". Because that way the market was incentivising sound business decisions.

<comment><USER10> 'it was much better when most stock market decisions were mostly based on "fundamentals"'

I don't recall such a period. Is there a particular interval you're thinking of?

<comment><USER11> I've thought there was more opportunity in fundamentals up until Warren Buffet and Ben Graham's the intelligent investor became well known. More people tried to use these methods, thereby increasing demand and decreasing the upside on securities that meet Graham and Buffets criteria. The stock market today is very different from when they got going, although long term I don't know that anything has fundamentally changed, even before robot traders there had always been random and unexplainable noise.

<comment><USER12> Wait, if there was more "opportunity in fundamentals" back then it would mean that stocks were further away from their fundamentals, right? That's pretty much the opposite of what the OP is complaining about.

<comment><USER13> This was probably the most insightful thing I've seen all week. Thanks for making me smile.

</comment></comment></comment></comment></comment><comment><USER14> Great discussion! The author doesn't seem to introduce the concept of training/testing datasets which absolutely critical to obtaining any reasonable model. So I don't buy the author's thesis that economic models are always wrong.

The solution to the hypothetical problem posed in the article is to separate the historical dataset into training and testing groups. The models should be generated while only 'seeing' the training data. You will, as the author mentioned, get many models that appear to fit the data. Most of these models will be garbage.

The fun part is when the testing data is introduced against the many models generated above. Most of the models will completely bomb, but a handful may actually predict the previously 'unseen' testing data with high accuracy. Those few models which pass the testing stage are the ones worth their salt.

Due to the self-aware nature of the markets, successful models probably will not be true indefinitely, but it's very possible they may be true long enough to be profitable. The less known your successful models are, the longer they will be successful predictors of the market. Hence why successful quant funds are notoriously secretive with their approaches. Open source would never work in finance.

<comment><USER15> "training/testing datasets which absolutely critical to obtaining any reasonable model"

This is partly correct but, in general, too strong.

Am I commenting on the OP? Not really!

Why too strong? Because it assumes too little and sometimes more information is available and with the extra information a 'testing data set' may not be needed.

Why are 'testing data sets' important? If about all you have to go on is the 'historical data' and then are just searching for a 'model' based mostly just on what 'fits' the data, then, sure, a 'testing data set' will likely be just crucial. One way to get such a 'testing data set' is to partition the 'historical data' into two parts, use the first to 'fit' a model and the second to 'test' the fit. Of course, there are still risks: If fit 10,000 models, find 10 that fit well and test each of the 10 with the 'testing data set' and accept the model that fits the testing data the best, then still may have some problems from a 'generalized version of overfitting'! As I recall, there has been some mathematical statistics to address this issue.

Where can get by without a 'testing data set'? Broadly if know more than the meager assumptions common in 'machine learning' or 'curve fitting'.

What more can be known? In principle the variety is large.

Examples? Sure: Broadly just simple, old 'regression analysis', looked at as statistical estimation, makes a long list of quite detailed assumptions. E.g., we assume that there is a model the works and that we know in good detail the form of that model. We assume a lot about the 'historical data' we have, E.g., we assume 'homoscadasticity' and mean zero, independent and identically distributed (i.i.d.) Gaussian for the errors. We make some assumptions about dimensionality (e.g., to get around 'overfitting'). Then the usual derivations give minimum variance, unbiased estimates of the unknown parameters and more, all without any use of 'testing data'. "Look Ma, no testing data required!".

"Yes, son, but as your father kept telling you, a LOT of assumptions are required, and the assumptions are not all easy to verify. Or the regression derivations are a nice logical trip from island A to island B we would like to get to but we don't always know how to get to island A.".

Other examples? Sure: Calculate the trajectory of a space craft doing 'slingshots' in the inner solar system and then reaching, say, Saturn. We start with Newton's second law, his law of gravity, maybe a little about the solar wind, a lot of details about the orbits of the planets, and do some good numerical work with an initial value problem of an ordinary differential equation. We build a 'model' but don't really 'fit for parameters' or use 'historical data' and have no real use for 'testing data'. Why? Because we believe in Newton's laws and our numerical work. A 'model'? Yes. Fitting 'parameters'? No,

Can there be a connection between space craft trajectories and economic models? Sure: Bring more assumptions than just curve fitting. An example is to bring, essentially, accounting. So, then can get a Leontief input/output model. We bring basically just accounting data and not other historical data, do no real 'parameter' estimation, and use no 'testing' data. If the input data is noisy, then, sure, so will be the output and we might do some work with confidence intervals. Still we don't check with 'testing data'.

More examples? Sure: The broad field, with many techniques, of distribution- free statistical hypothesis testing is based on historical data and some assumptions and really needs no testing data. What is obtained is much like a 'model' where can plug in new data and get the intended results. The assumptions are typically that the data is i.i.d.

Net, a lot can be done beyond the common approach of machine learning curve fitting.

<comment><USER16> As usual, an excellent summary. Economic models based on low level data (essentially better "instrumentation", capturing bank transactions, some contracts, individual spending etc.) might be quite useful at least for short term prediction. Perhaps old ideas of "optimal control" can be to some extent realised.

</comment></comment></comment><comment><USER17> Every model is "wrong", by definition of it being a "model" and not "reality". It's one of the few mind opening things I've learnt at university.

That's not a problem if you take it as an incentive to improve how much you know about the real world. It's a problem when you put the model before the people, and say that "models got us in trouble because of calibration problems".

An economic crisis is not an unavoidable natural disaster, it's people screwing up other people.

<comment><USER18> Simple and to the point. Couldn't agree more.

</comment></comment><comment><USER19> This article is avoiding terminology, data and any specifics on the problem that it renders it useless.

You might be fooled it says something useful if you don't know what a 'model' means in any science.

So what is the point of the article? The author is trying to sell you his book where he most probably makes people who don't know anything about economics feel good or push an ideological agenda.

<comment><USER20> It's not so much useless, it's just far more general than the author probably intends.

All his arguments apply equally well to any scientific models which require fitting, in geophysics (as he acknowledges), atmosphere/ocean science, climate modelling, most of biology, ecology, etc.

Why he singled out economics is beyond me.

</comment></comment><comment><USER21> The author is only partially right. The mistake is in defining a closed system that is in fact not closed, and then curve fitting.

For instance a great part of growth in the last 100 years has been from man's ability to harness energy from fossil fuels. If your time line is narrow enough, you can disregard the point that fossil fuels is not unlimited, and project continued rise in extraction.

Another example is the baby boom, and the introduction of women into the paid work force which led to continued rise in property prices.

One more is the introduction of laws which suddenly compel people to invest in the stockmarket. It leads to short term asset inflation but generally makes worse investment all round.

That said, it is fitting that an economy is well modelled using the principles of hydraulics. See <http://en.wikipedia.org/wiki/MONIAC_Computer>

</comment><comment><USER22> The problems with economic model or most modeling are not the methods. It's usually dealing with the quality of the features or parameters. Even in a much simpler problem, no matter how good the methods, if you don't have the right params, your model will suck. And with economic models, it's dealing with a open world system with ever changing params, the challenge is not on the methods, but how to discover quality parameters/features. And that require not just the skills of modelers but many other disciplines.

</comment><comment><USER23> _Financial-risk models got us in trouble before the 2008 crash_

Is this accurate? I remember reading that all the alarms were going off, they were just ignored or the models were "adjusted".

</comment><comment><USER24> Carter's papers on the subject:

Ballester, P. J., & Carter, J. N. (2006). Characterising the parameter space of a highly nonlinear inverse problem. Inverse Problems in Science and Engineering, 14(2), 171-191. doi:10.1080/17415970500258162.

Ballester, P., & Carter, J. (2007). A parallel real-coded genetic algorithm for history matching and its application to a real petroleum reservoir. Journal of Petroleum Science and Engineering, 59(3-4), 157-168. doi:10.1016/j.petrol.2007.03.012.

</comment><comment><USER25> A great quote from George E P Box:

All models are wrong. Some are useful.

<comment><USER26> So are those types of models useful? The ones that need to be adjusted constantly?

</comment></comment><comment><USER27> Actually, even correctly parametrized, any predictive model will suffer from the paradox of the oracle : if you have a "oracle" capable of anticipating the decision of an actor, and that this actor knows about the prediction, this actor can make the prediction false.

In economy, some actors have an interest in faking the prediction, even if it is costly for them : it is often valuable to be unpredictable.

</comment><comment><USER28> Is this really surprising? I would have thought this would be self-evident as these kinds of models would seem to be highly chaotic.

It's really no different than the meteorology simulations in the 60's that first discovered the butterfly effect.

[http://en.wikipedia.org/wiki/Butterfly_effect#Origin_of_the_...](http://en.wikipedia.org/wiki/Butterfly_effect#Origin_of_the_concept_and_the_term)

<comment><USER29> I think the butterfly has gone extinct when it was replaced with CO2.

</comment></comment><comment><USER30> A "scientist" finds by cross-validation that his model is over fitting the data. Luckily it wasn't published by a reputable source of science journalism.

<http://en.wikipedia.org/wiki/Cross-validation_(statistics)>

Also who the heck is Wilmott? He just pops up in the last paragraph with no introduction.

<comment><USER31> He's pretty well know in the Quant community : <http://wilmott.com/about.cfm> , but I also had to double-check the article to see where he was introduced. I guess there was some heavy-handed editing.

</comment></comment><comment><USER4> Macroeconomics resembles a science in exactly two ways: it looks at history, and it makes predictions (or prescribes courses of action; these are equivalent).

Greek mythology resembled a science in those same two ways.

<comment><USER13> _Economics ... makes predictions (or prescribes courses of action; these are equivalent)._

This is absolutely false, except at the micro level. In terms of _policy_ , economics can only inform us of the relative costs of various alternatives. It cannot tell us which alternative is _right_.

Consider the question of free trade. Virtually every economist agrees that free trade improves total efficiency (viz the Law of Comparative Advantage). However, at the margins, it may harm some individuals. Economics cannot tell us if it is morally right to incur those individualized harms in order to improve the lot of the whole, nor what if anything we should do to make whole those who were affected.

Economics makes predictions, giving us insights. Our morals are then needed to prescribe courses of action.

<comment><USER4> You are right. To prescribe a course of action is to make a prediction (that this action will lead to results in some way superior to alternative courses). The reverse, as you correctly point out, is not necessarily the case.

</comment></comment></comment><comment><USER32> Isn't this just an instance of a chaotic system, in which the parameter settings that almost match the historical data will inevitably diverge because of sensitive dependence on initial conditions.

</comment><comment><USER33> From scientist to scientist a little secret: All models are always wrong! If the model would be correct it would be as detailed as reality and thus also as useless.

</comment><comment><USER34> This is perhaps naive, but why are the parameters to a model not considered as part of the model as a whole?

<comment><USER35> In the end it's a matter of convention. If you think about Newton's "model" of gravity, for example, you'll notice that the formula that describes the gravitational force can be plausibly explained based on intuition. However, the gravitational constant (i.e. the parameter) has no explanation. It just is.

Of course, a great deal of physics is ultimately about trying to make the parameters go away by explaining them using more fundamental models. But at any given level of abstraction, you'll have parts of the model that are reasoned intuitively, and parts of the model that just are the way they are, for no good particular reason other than that's what you happen to get by measuring.

</comment></comment><comment><USER36> Economic models are wrong because [https://secure.wikimedia.org/wikipedia/en/wiki/Economic_mobi...](https://secure.wikimedia.org/wikipedia/en/wiki/Economic_mobility) and [https://secure.wikimedia.org/wikipedia/en/wiki/Social_mobili...](https://secure.wikimedia.org/wikipedia/en/wiki/Social_mobility) are mutually exclusive

</comment><comment><USER37> Can you really compare Economic models to Physics models without discussing the simplifications necessary to create an Economic model?

<comment><USER12> You usually have to make a large number of simplifications to create a Physics model too. The question is how those simplifications change the accuracy of the model.

"Essentially, all models are wrong, but some are useful" -George E.P. Box

<comment><USER38> We can generalize to more than just physics and economics... As the quote says, ALL models are wrong (no matter what field). If you have a model that is "right" then it isn't a model, is it?</comment></comment></comment>


Coding Horror: A Question of Programming Ethics - <AUTHOR> http://www.codinghorror.com/blog/archives/001072.html <comment><USER2> big name sites should stop asking for people's username and password from other sites.

<comment><USER3> I'd say, ALL sites should stop asking for username and password from other sites (which they don't control[1]).

It's bad when you get screwed, for obvious reasons. It's also bad when you don't get screwed because it reinforces your false sense of security in an inherently insecure practice.

[1] I don't mind as much about situations like logging in to Youtube with your Google credentials since I know google controls both sites.</comment></comment>


Mathematician Says Big Data Is Causing a 'Silent Financial Crisis' - <AUTHOR> http://time.com/4471451/cathy-oneil-math-destruction/ <comment><USER2> There is a lot of interesting research on the societal impact and the (potential) problems caused by the use of large-scale data analysis and artificial intelligence, e.g. by Kate Crawford at MSFT Research.

Last year I gave a talk on this at 32C3 where I tried to elucidate some of the problems that can pop up when using data analysis to automate things that were previously done by humans:

[https://www.youtube.com/watch?v=iRY9IceaVig](https://www.youtube.com/watch?v=iRY9IceaVig)

Personally I see a lot of potential in data analysis & artificial intelligence, but like all technologies they pose significant risks as well. What we really need therefore (IMHO) is to teach ethics to people that work with data, and establish organizations and methods that ensure data analysis isn't used to do harm (deliberately or not).

<comment><USER3>
        > What we really need therefore (IMHO) is to teach ethics to people

The issue with almost all organized teaching (as opposed to when you don't even know you are doing it, e.g. to your children, every day) is that it only targets conscious thinking. Do you already see the problem with teaching "ethics" (and empathy)? It's the wrong brain area. People will ace the tests but how they actually act won't change. Another big reason for why teaching ethics (the way it is usually done) is that much of behavior is environment- driven. So you can teach someone all you want, if you then place them in a cut-throat competitive "result-driven" environment you can see empathy and ethics quietly sneak out the back door.

<comment><USER4> Being deliberately obtuse goes a long way to delay a logical ending. Flashback to 2004 and the same type of debate/affirmations were posited to Dr Mandelbrot's apocalyptic warnings[0][1] of the imminent subprime crisis.

Instead of learning lessons, the players continue to 'print money out of thin air'[2] and deny any fault for the last collapse. The rationale seems to be, 2007 was an extreme culmination of events that wouldn't happen again in millions of years. Problem is, in a universe of infinite variables, an infinite number of combinations exist and could manifest anytime a given set of vulnerabilities and unanticipated outlier events present themselves concurrently.

[0] The (Mis)Behavior Of Markets 2004

[http://www.goodreads.com/book/show/665134.The_Mis_Behavior_o...](http://www.goodreads.com/book/show/665134.The_Mis_Behavior_of_Markets)

[1] 2006 reminder

[http://www.ft.com/cms/s/2/5372968a-ba82-11da-980d-0000779e23...](http://www.ft.com/cms/s/2/5372968a-ba82-11da-980d-0000779e2340.html)

[2]The Quants

Great Monday Morning QB analysis on what went wrong & why it will be repeated(hint:greed +ability +hubris)

[http://www.goodreads.com/book/show/7495395-the- quants](http://www.goodreads.com/book/show/7495395-the-quants)

edit: fixed links

</comment></comment></comment><comment><USER5> > [algorithms] decide who gets access to credit and who pays higher insurance > premiums, as well as who will receive online advertising for luxury handbags > versus who'll be targeted by predatory ads for for-profit universities.

As opposed to what? Subjective decisions made by humans based on biases and personal preferences with no semblance of reason or fairness?

> for example, who lives in an area targeted by crime fighting algorithms that > add more police to his neighborhood because of higher violent crime rates > will necessarily be more likely to be targeted for any petty violation, > which adds to a digital profile that could subsequently limit his credit, > his job prospects, and so on.

So having more cops in a high-crime neighborhood is somehow a bad thing, because the presence of more police actually confirms the fact that there is more of a need of police in this neighborhood?

> this technology was actually siloing people into online gated communities > where they no longer had to even acknowledge the existence of the poor,

So targeted advertising to the needs of people is bad because... I can't even make sense of this one.

<comment><USER6> > the presence of more police actually confirms the fact that there is more of > a need of police in this neighborhood?

If we were to enforce all laws we would need massively more police in every neighborhood. When police are sent to flood a poor neighborhood they end up ticketing poor residents for a bunch of "violations" that they would also find if they went to the rich neighborhoods. But if they ever actually did issue all those ticky-tack tickets in the rich neighborhoods residents would be up in arms contacting their city council members and it would end quickly. In the poor neighborhoods nobody ever hears about it and the residents that can least afford it take the financial hits.

<comment><USER7> > they end up ticketing poor residents for a bunch of > "violations" that they > would also find if they went > to the rich neighborhoods.

I'm not sure what data you have to support this. But check out violent crime rates. Larceny-thefts. Burglary/Property theft. Motor Vehicle Theft. Aggrevated assult. Robbery (mugging, stickups. etc).

These very popular crimes aren't so popular in nicer neighborhoods. Rich neighborhoods have their own problems, but let's not appeal to the spirit of lawlessness (cops are bad, rules are bad, unfairly arrested, etc.)

</comment></comment></comment><comment><USER8> > add more police to his neighborhood because of higher violent crime rates > will necessarily be more likely to be targeted for any petty violation

He'll also be less likely to be the victim of violent crime in an area with a proven record for violent crime. For the price of avoiding 'petty' crimes, it seems like a good trade-off.

<comment><USER9> That's a solid theoretical argument. However, it is contradicted by the empirical evidence.

What usually happens is that the police issue fines so frequently that they become considered an occupying force by the populace. In turn, the populace withholds information from the police, and both violent crimes and petty violations rise.

Police statistics make quantitative conclusions difficult, since they are so often fudged for political purposes. They rise when the police need more funding, and fall when they need political capital. Murder rates are generally considered the only reasonably accurate figures.

On the ground, however, the reality is very clear.

<comment><USER10> I would suggest, however, that this is not an argument for less police but instead an argument for fewer laws that are only infrequently and inconsistently enforced. It is these laws that give police the opportunity to "issue fines so frequently" and to give affected populations (rightfully or not) the impression that they are singled out. (Laws against loitering or the consumption of alcohol in public come into mind, to a lesser degree probably also the criminalization of marijuana consumption.)

<comment><USER11> Right... but those laws only ever get revoked when they hit the rich with them. And for every one you revoke there's ten more wealthy/connected people with sob stories that could have been prevented if X were illegal who are contacting their elected representative.

I agree with you on principal but IMO society on Mars will be having this debate (I hope I'm wrong).

</comment></comment></comment></comment><comment><USER12> An interesting point touched on in the article is the extent to which feedback from a predictive algorithm contributes to its later predictions being biased. If police are targeting a specific neighborhood because it is flagged as being a high crime area, then there is likely to be a higher arrest rate for innocent people too, i.e. a higher false positive rate. Unless there is a good way of differentiating false and true positives (which seems unlikely in a justice system heavily focused on plea bargains), then a cycle of over- policing may ensue. A good predictive algorithm would need to account for this type of temporal autocorrelation.

<comment><USER13> One could legitimately say that learning of patterns is the process of bias creation.

</comment></comment><comment><USER14> On a related note, a Princeton professor and some colleagues just published a blog post and a research paper documenting that "language contains human biases, and so will machines trained on language corpora:" [https://news.ycombinator.com/item?id=12356111](https://news.ycombinator.com/item?id=12356111)

</comment><comment><USER15> A wide array of concerns, but I stopped reading when the article explains that algorithms are going to keep people from being aware that poor people exist. If the only awareness someone has of poverty is from the Internet ads or any other content, then they really don't know anything about it at all. This sounds more like the people that protested computers in the 60's because they are used for war.

<comment><USER16> You should probably keep reading.

</comment></comment><comment><USER17> I've always taken it for granted that discrimination and inequality would be the natural result of any optimization/ML algorithm, especially when maximizing for profit or any other utilitarian goal. I wonder why people would be surprised by this logical outcome given that it has been long known that statistical discrimination is profitable, and companies rarely have incentives to improve equality and diversity.

</comment><comment><USER18> > Unlike the WMDs that were never found in Iraq

They _were_ found: [http://www.military.com/daily-news/2015/04/03/army-seeks- to-...](http://www.military.com/daily-news/2015/04/03/army-seeks-to-identify- troops-veterans-exposed-to-chemical.html)

<comment><USER13> We knew that WMD were in Iraq, prior to finding any, because the WMD came from outside Iraq.

</comment></comment><comment><USER19> Big data has the potential to create either unprecedented equality or inequality. Transparency is a key word in this post (I often Ctrl+F to see how many matches for "transparen" I see in this type of posts).

When it comes to software, one policy to make things more transparent is open- sourcing. The question is opening what and to what degree. I feel strongly that organizations today could and should be much more open and are instead absurdly opaque.

It's our responsibility as a society to talk about transparency as key for trust and equality.

<comment><USER20> Transparency of pricing models does not immediately result in better outcomes for consumers. In some cases, it may actually result in _worse_ outcomes.

Consider when there are few companies within an industry, like rare metal mining. If they chose to use the same, open-source pricing model, and any changes to that pricing model would be reflected by all companies, the optimal pricing model would be to price at the monopoly level -- to raise the price to where it would be if there were no competition and a single firm in the marketplace.

In the case of lending, however, different firms have different levels of risk acceptance. Many firms (big banks) avoid high-risk borrowers. A few firms (pay-day lenders) are willing to lend to those borrowers, but only at a high interest rate. Somewhere in the middle is a collection of scrupulous lending opportunities.

Any regulation that would restrict the acceptable interest rates to a narrow band would reduce not just the (unscrupulous) loan sharks but also the banks lending to high-risk customers.

This would necessarily result in _loss of borrowing ability_ to those labeled "high-risk".

Is this a desirable outcome? Maybe, maybe not. But it should certainly be discussed.

<comment><USER19> It's true openness isn't free of side effects. I think a mining company that has exclusive possession over a public resource should be rewarded with the value creation (mining) and not by possession (natural resources). A government should designate the natural resources to public welfare and is ultimately achieved by openness. A resource greedy mining company would rather dispense with transparency in this case.

Regarding the borrower problem. This is a common fear of revealing one weaknesses by openness. I believe society will invent new alternatives to support the new needs. Again, it seems openness is positive overall, despite new challenges it presents.

</comment></comment></comment><comment><USER21> Disallowing the use of race as a signal in machine learning / ad targeting could help intercultural awareness. Here's the last paragraph of the article:

"""

O'Neil also proposes updating existing civic rights oriented legislation to make it clear that it encompasses computerized algorithms. One thing that her book has already made quite clear – far from being coolly scientific, Big Data comes with all the biases of its creators. It's time to stop pretending that people wielding the most numbers necessarily have the right answers.

<comment><USER22> _Disallowing the use of race as a signal in machine learning_

That sounds like a great idea, but it isn't as simple as it appears. Most systems won't have "race" directly encoded as a feature, but that is insufficient.

See slide 22 from [1], further in depth discussion discussion from [2] or just this feature (which many systems could automatically discover):

      Feature6578 = Loc=EastOakland && Income<10k

I don't know what the solution to this is, but it's a pretty hard problem, and even the best intention are insufficient.

Delip proposes in [1] to introduce a "fairness constraint" where the probability of a favourable outcome in the majority class is the same as in a protected class. This sounds sensible, but make optimising systems much harder.

[1] [http://deliprao.com/archives/129](http://deliprao.com/archives/129)

[2] [https://www.chrisstucchio.com/blog/2016/alien_intelligences_...](https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html)

<comment><USER23> The problem is deciding which features are reasonable to include. Yes, location and income can probably predict race to a certain precision, and if you're building a race-detector-by-proxy, you'd want to include those. But what if you're evaluating eligibility for a mortgage? Certainly location and income are critical. A black person in East Oakland with an income below $10k _is_ going to struggle with a mortgage, but so is a white person with those characteristics (and a black person in Connecticut making $250k is going to do fine and so is a white person). But if more black people than white legitimately are in a situation where they might struggle to service a mortgage, more black people than white would get flagged as such by an objective algorithm. Does that make the algorithm racist? The slide seems to imply so (although without speaker-notes it's difficult to ascertain).

The flip-side is that under a human-biased system, the first white person might get the mortgage based on being white, and the second black person might be denied. That would be reversed under an algorithmic system only considering legitimate factors, and surely that's progress?

<comment><USER24> The problem is actually different.

Race is often very predictive. Blacks don't perform worse simply because they have income < $10k. Blacks perform worse on many measures even if you hold all that stuff equal. Here are a few of the many studies reproducing this:

[http://ftp.iza.org/dp8733.pdf](http://ftp.iza.org/dp8733.pdf) [http://www.mindingthecampus.org/2010/09/the_underperformance...](http://www.mindingthecampus.org/2010/09/the_underperformance_problem/) [https://randomcriticalanalysis.wordpress.com/2015/05/16/on-c...](https://randomcriticalanalysis.wordpress.com/2015/05/16/on- concentrated-poverty-and-its-effects-on-academic-outcomes/) [https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t...](https://randomcriticalanalysis.wordpress.com/2015/11/22/on- the-relationship-between-negative-home-owner-equity-and-racial- demographics/#my_analysis)

If you were right, then the machine learning algorithm would quickly determine that race doesn't matter. Redundant encoding, direct encoding, etc would be irrelevant - the algorithm would ignore it as noise.

The problem is that the algorithm is supposed to uncover hidden patterns that predict loan default. But then there is one _specific_ hidden pattern that is socially and legally taboo to detect, yet also highly predictive.

(Incidentally, if you can solve this problem in lending, it's easily a unicorn startup.)

</comment></comment></comment></comment><comment><USER25> I'm surprised the term "disparate impact" hasn't come up. I'm too well versed in anti-discrimination law, but it seems like this is the term for exactly what they're trying to avoid.

[https://en.wikipedia.org/wiki/Disparate_impact](https://en.wikipedia.org/wiki/Disparate_impact)

</comment><comment><USER26> Jerr Thorpe actually has a tool called floodwatch[1] that keeps an eye on your targeted ads and shares at the meta level to try and hold advertisers to the rules. Sadly for it to work you have to browse the internet with no ad- blockers.

[1] [https://floodwatch.o-c-r.org/](https://floodwatch.o-c-r.org/)

</comment><comment><USER27> I find it insulting when statements start with "Mathematician says", "Experts say". Whoever does this thinks little of his readers. They count on that prefix to make their story more convincing. Sadly, it probably works.

</comment><comment><USER28> "she shows how the algorithms [...] are based on exactly the sort of shallow and volatile type of data sets that informed those faulty mortgage models in the run up to 2008" so the fault is of the technology, right?

<comment><USER29> Well there is a paradox here, if the models were faulty, then why was every bank scrambling to offload the mortgages to someone else? If the models said the mortgages were sound, why didn't the banks want to hold onto them as a cash cow? Could it be the data spoke but no-one wanted to hear what it said?

<comment><USER30> Depends on which model you mean.

The banks were holding higher tranches of the securitised mortgage pools. While those pools had traditional levels of defaults, the tranches had their expected value. As the defaults accelerated beyond historic levels, they plummeted in value. In both cases the same model said the instrument had/didn't have value. As these instruments are fair value accounted the fact that they were still bringing in cash didn't help any.

However, as the mortgage market started to go south, the default models underestimated the level of defaults, particularly in the equity release market. This underestimation was endemic in the industry at the time, from the banks to the regulators and ratings agencies.

There is, of course, a very strong argument to say that the ignorance was willful, everyone was on quite a nicer earner and reacted late to the evidence as it started appearing.

So there wasn't really a paradox. The banks were holding tranches while the expected defaults were within historic levels. Once it became clear that that wasn't a good predictor anymore, they tried to limit the effect but it turned out that so many participants were in the same boat that there weren't enough buyers.

I have to say I'm not entirely sure what this has to do with ad analytics but I'm not writing a book on the subject so maybe that's not too surprising.

</comment></comment></comment><comment><USER31> Just like programs only know to do what they're coded to do, algorithms only learn what's already in their dataset. It's up to data scientists and analysts to scrub their dataset of potentially discriminatory data. If you're contractually obligated to do otherwise, you should either find another job or do the right thing anyway. If you choose the latter and someone takes action against you, then you should take it to court.

I was in a situation like this in my own work and I regret not standing up for the right thing now...

</comment><comment><USER32> The article emphasizes the fact that data driven decisions amplify not only efficiencies of the modern society, but also its inefficiencies. Which is probably true for any technology.

</comment><comment><USER16> If we train a mortgage engine on Redlining [0] data, it will then red line. Algorithms are not somehow "fair" because there isn't directly a human in the path. They are as "human as we are" but only proxy and at much lower fidelity.

Not to Godwin the thread ... but big data has been used for immoral purposes since well, big data [1].

The even larger danger, is that people who don't know better, will simply "train a model" and having no overt ill-will, will still create a biased algorithm.

[0] [http://www.encyclopedia.chicagohistory.org/pages/1050.html](http://www.encyclopedia.chicagohistory.org/pages/1050.html)

[1] [https://en.wikipedia.org/wiki/IBM_and_the_Holocaust](https://en.wikipedia.org/wiki/IBM_and_the_Holocaust)

<comment><USER24> This is simply not true. If we train a mortgage engine on redlining data, it will only reproduce the red line if it's predictive. I.e., the algorithm will only reproduce the red line _if the red line is statistically valid_.

I discuss this in detail here, with numerical examples: [https://www.chrisstucchio.com/blog/2016/alien_intelligences_...](https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html)

The fact is that machine learning takes biased inputs and produces unbiased outputs, and this is a pretty normal occurrence. If I tell you that my algorithm to target advertisements treated `displayedInterest x isMobile` with 15% more weight than `displayedInterest x isDesktop` because it corrected for bias induced by latency on mobile connections, you'd think nothing of it.

<comment><USER33> > The fact is that machine learning takes biased inputs and produces unbiased > outputs

That is not necessarily true, because it assumes that the biased inputs have no influence on the outputs.

Suppose the training data is a bank that has historically given higher- interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often.

Cause and effect are very hard for an algorithm to distinguish when it's the clustered _input_ variables that are linked together for some opaque reason. The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other.

Sure, more training data could help the algorithm distinguish between the two, but there may not _be_ enough training data that shows the inverse -- low- interest loans given to minorities -- if the historical data is biased enough.

<comment><USER24> _Suppose the training data is a bank that has historically given higher- interest loans to minorities. Those loans will default more often, and so the algorithm could "correctly" conclude that minorities default more often._

But if the relationship is actually `defaultProb = a x interest_rate + b`, then the algorithm will reflect that. I discuss this case explicitly in the blog post I linked to in the section "What if black people don't perform as well?", and explicitly give an example of a linear model NOT doing what you claim it will do.

 _The fact is, both statements in this example would be true: "higher interest is correlated with more defaults" and "minority borrowers are correlated with defaults." The algorithm doesn't know one from the other._

No, it wouldn't because you'd also have cases of white people with high interest who defaulted. The model using race as a predictive variable would fail to fit those data points. I explicitly discuss this case in my blog post.

You are correct that in some cases you don't have enough training data. But in these cases, there is no particular reason for bias to have a particular sign. Again, as I discuss in the blog post I linked, why would Captain Kirk be biased against "black on left guy" vs "black on right guy"?

Insufficient training data causes _variance_ , not _bias_ , and could just as easily result in bias _in favor_ of blacks. (In fact, I link to a number of examples where it does.)

<comment><USER12> > Insufficient training data causes variance, not bias

Key point.

If one were to compare the two high-interest groups, a small sample for one group would be reflected in a large confidence interval around its mean, a greater overlap in the CIs of both group means, and less confidence that a real difference exists.

</comment></comment></comment></comment></comment><comment><USER34> Let's blame the math. It couldn't possibly be the fault of businesses targeting the poor.

</comment><comment><USER35> I found this article problematic. First the assertion that "big data comes with all the biases of its creators". Isn't this exactly what big data (which I believe is a term used here to reference some sort of predictive model or machine learning) is supposed to avoid - ie. Making decisions based on evidence and not bias?

Maybe it's the cynic in me but someone who worked in a hedge fund, started yet another advertising tech company and now releases a book about the evils of "big data" abusing the poor... guess I'm biased.

<comment><USER36> Big data somehow being unbiased and impartial is a myth. The algorithms are still a product of humans. One notable example is a Google algorithm for labeling objects in photos that tagged black people as gorillas [1]. Now I don't have particular insight into the development process behind this algorithm, but this result may have occurred because the training data used did not include enough photos of black people.

Another example (from an old version of Picasa -- I'm not trying to pick on Google but I have firsthand experience of this) is when a friend of mine had photos of two different Asian people in his collection, but Picasa always recognized person B as being person A, presumably because it couldn't distinguish features unique to the faces of Asian people.

[1] [http://www.cnet.com/news/google-apologizes-for-algorithm- mis...](http://www.cnet.com/news/google-apologizes-for-algorithm-mistakenly- calling-black-people-gorillas/)

<comment><USER24> What evidence do you have that one specific classification error resulting in a tweetstorm was the result of bias at all? I've been mislabeled as a tank in at least one image processing demo (admittedly, this was well before the era of deep learning). Image classification errors happen.

Also, it's a bit strange that you suggest ML algos are racist against Asians. You do know that Asians are wildly overrepresented among people building such algos, right?

Consider the possibility that some classification problems are actually more difficult than others. As one example, darker objects are harder to distinguish than bright ones and there are very clear mathematical reasons for that. That's true even in fields where questions of "racism" don't make sense, e.g. in MRI (magnetic resonance imaging is non-optical, but contrast still matters): [https://www.chrisstucchio.com/pubs/wf_segment.pdf](https://www.chrisstucchio.com/pubs/wf_segment.pdf)

<comment><USER36> I did not make a blanket statement that all ML algos are racist against Asians. My point was to convey that if the training data do not accurately represent the population, the algorithm will carry the biases of that inaccurate representation: the algorithm is only as good as the data it is trained upon (for supervised learning).

As a sibling comment stated, humans are deeply involved in all aspects of designing and interpreting an algorithm's results. To the extent that we are flawed, so will our algorithms be.

Another example I heard recently from a conference was that of analyzing blighted homes in areas. The underlying data was generated by human inspectors who turned out to have given passes to violations in more affluent areas and fines in less affluent areas. The fines compounded the problem of blight as the residents could not afford to pay them and ended up causing a feedback loop of blight increasing in neighborhoods identified as blighted.

I am frankly confused as to what is so controversial about the statement that algorithms created by humans are not free from human biases. I apologize if I have only added confusion rather than a productive analysis.

<comment><USER24> _I did not make a blanket statement that all ML algos are racist against Asians. My point was to convey that if the training data do not accurately represent the population, the algorithm will carry the biases of that inaccurate representation: the algorithm is only as good as the data it is trained upon (for supervised learning)._

But that's not true. Inadequate training data will cause _variance_ , not _bias_. This variance can have any direction.

In a linear prediction scenario (i.e. any go/no go decision process), which exactly what Cathy O'Neil is criticizing, the effect could just as easily be "more loans for black people" as "less loans for black people".

In a multidimensional predictor, it does directly decrease accuracy but that's all. "Black guy -> gorilla" is just as likely as "yummyfajitas -> tank".

Also, realistically, if we want to think about your black guy -> gorilla example, most likely if insufficient training data was the problem, then it was the number of gorillas that was too few. Google has vastly more black humans in their training set than gorillas - a quick google search suggests there are only 100,000 gorillas in the world, most of whom are probably unphotographed.

 _As a sibling comment stated, humans are deeply involved in all aspects of designing and interpreting an algorithm 's results. To the extent that we are flawed, so will our algorithms be._

Sure, algorithms are flawed. But this _does not imply that bias must be present_ proportionally to ours. Insofar as we do good statistics, human bias is mitigated and can even go away. Science works.

As a classical example, consider Morton's analysis of human skulls. Morton was a super racist guy trying to prove how white people are superior. But he did an unbiased analysis and accurately measured skull volume - filling a skull with buckshot is a pretty unbiased process.

[http://journals.plos.org/plosbiology/article?id=10.1371/jour...](http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001071)

 _I am frankly confused as to what is so controversial about the statement that algorithms created by humans are not free from human biases._

What's controversial is that this is the idea of "original sin" but applied to science. And most of the proposed mechanisms _are simply mathematically wrong_. Also many of the examples cited by proponents of this original sin concept are also wrong.

<comment><USER37> Thanks for the interesting article. Although, from my reading, it seems to support the importance of both data handling and statistics in avoiding bias. That is, a bias in the dataset (or which subset is analyzed) would subsequently bias the statistical results (Box 1 and 2).

Regarding big data and ML, I guess the question is then what kind of biases exist in data collection, something that seems dependent on the details and history each particular dataset.

</comment></comment></comment></comment></comment></comment><comment><USER30> In other news, it has been discovered that a Time article was written in English. A language that has previously been used for hate speech.

The only possible conclusion is that we should ban Time magazine. Or some such nonsense.

</comment><comment><USER38> The irony is that Times is partnered with advertisers torun these exact algorithms on the readers of this very article. In fact, they have anti- adblocking scripts to prevent circumvention, which unfortunately breaks reader mode.</comment>


Jeb Bush suggests he would back mandatory backdoors in encryption if elected - <AUTHOR> http://www.dailydot.com/politics/jeb-bush-debate-encryption-backdoors/ <comment><USER2> Hey I figure Congress, the Senate, the Supreme Court and the Cabinet can all start using backdoored encryption.

Let's see how it works for them.

</comment><comment><USER3> All that does is makes U.S. encryption the worst available and hands a free kick to other countries.</comment>


Yo Startup House Headquarters - <AUTHOR> http://techcrunch.com/2014/06/24/the-house-of-yo/

<comment><USER2> I upvoted this because it's objective and new information. I know a lot of people here hate Yo because of the underlying phenomenon they represent, but isn't it at least a little interesting that someone in a studio apartment can command so much respect? It seems like the internet has rewritten most of the world's social rules, and we have to acknowledge that, on average, we seem to be moving closer to a meritocracy. I'm keenly aware of how strange it sounds to call an insider-driven industry (VC funding) a meritocracy, but if you compare the current state of affairs to any point in history prior to 20 years ago, people care far more about what you have (users) than who are you are or what your pedigree is.</comment>


Is There Anything Good About Men? [Roy F. Baumeister] - <AUTHOR> http://www.denisdutton.com/baumeister.htm <comment><USER2> If there were nothing good about men, they wouldn't exist.

Edit: There are vertebrate species with no males. Some have males only when conditions are suitably favorable. Given that there are men, there must be selection pressure.

Edit: I wonder why this submission was flagged? Too contentious?

<comment><USER3> > _Edit: I wonder why this submission was flagged? Too contentious?_

That, as well as the clickbait title, plus it's missing a (2007) tag. When it was submitted back in '07 the article received several hundred points and a lively discussion. [https://news.ycombinator.com/item?id=589346](https://news.ycombinator.com/item?id=589346)

I was hoping there would be interesting discussion today but this time users are willing to flag it and move on, no time to leave a comment.

<comment><USER2> Thanks. I didn't notice either. I just lucked out, catching it in time to comment ;)</comment></comment></comment>


Ask HN: How do Hacker News articles become "Dead"? - <AUTHOR> I'm using the public BigQuery Hacker News data to track popular keywords and I'm thinking that filtering out "dead" articles would be helpful, but I want to understand how articles become "dead" first. Does anyone know what the process is? <comment><USER2> The submitter can delete it, or several users can flag it.

<comment><AUTHOR> Thanks!</comment></comment>


Can a supercomputer solve the eurozone crisis? - <AUTHOR> http://business.financialpost.com/2011/11/24/can-a-supercomputer-solve-the-eurozone-crisis/

<comment><USER2> I am not saying computer modeling can give a reliable answer to this problem, but comparison to LTCM is not fair. The reason why that fund failed so badly was that they applied their models to situations where it was not meant to be applied. As they got bigger, they ran out of arbitrage opportunities, this led to an increased use of leveraging, which ultimately made their fall so spectacular. Leveraging introduces asymmetric risk in your exposure. The original arbitrage models w/o leveraging was not the reason for failure.

I do agree with "... and computer models of financial events are far from infallible, regardless of the amount of money and mental resources used to create them.", but what else do we have? The gut feeling of people who may have a conflict of interest?</comment>


Self-hosted vs outsourcing - <AUTHOR> http://blog.puncht.com/2009/06/10/self-vs-outsourcing/

<comment><USER2> It's a great theory as long as all those outsourced sites: stay in business, offer you a way to backup your information, don't act like jerks (ex: suddenly killing your account for unknown and unanswerable reasons), don't suddenly change business models and start charging you a lot for what was cheap or something for what was free.

Perhaps there's room in the market for all-in-one local solutions (even if it's the freely-produced-out-of-love market). I'm thinking of something like Wamp for Windows (a complete LAMP stack that you can install from one exe and gives you a simple control panel for the whole thing). Maybe a Linux distro that installs (or gives you options during install) all the packages you might want for a web business - source control, bug tracking, blog, wiki, web server, db, scripting languages, etc) plus configures firewall, backup systems, etc (or guides you through setting up something like that). Or maybe it's not a distro but a HOWTO.

Or maybe that sort of thing already exists and someone wants to link to it :)

<comment><USER3> GitHub Firewall Install is a turnkey, local GitHub. I'm not into hosting commercial software in private repos on GitHub though. It's periodically down, and that stuff needs to work. And GitHub FI looks pricey.

I think I'm better off with a VPS, gitosis, and local backups. It's cheap, it works, and it doesn't matter that I don't have all the cool social networking aspects of GitHub.

<comment><USER4> It is occasionally down but the whole point of git is that it's distributed, if you need to deploy when github is down you just do it from your local version.

<comment><USER5> Assuming you don't rely on submodules for your deploy :)

</comment></comment></comment></comment><comment><USER6> Where is the discussion of the whole risk/reward aspect. When I was freelancing I had dreams of building up an entire service agency that I would, some day, sell for millions. I knew the risks of failure were greater but the upside was golden. After 5 years reality struck and with a growing family I tabled my dream I went for a solid corporate job and some stability.

The person in this story took the risk (although her reward is different) and failed. C'est la vie.</comment>