What it's like to uncover a $1 billion fraud - <AUTHOR> http://stockfraudstory.blogspot.com/ <comment><USER2> "You can't cheat an honest man. He has to have larceny in his heart in the first place." -- W.C. Fields

In the US, the investor is supposed to be the first line of defense against stock fraud schemes. Unfortunately, the laws were written to help out the lower and middle classes from unscrupulous stock scams, not the wealthy "sophisticated" investors.

SO here's a tip: if you get scammed, don't count on the government to help you. They've got bigger fish to fry. You need to do your own due diligence, get your own lawyers/accountants/etc to look things over, and verify every claim and every representation that someone makes to you. If anything is even a little bit off DONT INVEST.

I was approached to invest money in two separate projects but ultimately didn't because there wasn't any way to ensure that I was going to see a good use of the money. That's how it is: if your only connection to someone is the use of your money, you better make sure that you understand how and when you make money on the deal. Otherwise, you're as good as robbed.

Note that this applies to ALL INVESTMENTS. If the company goes under for good reasons or fraudulent ones, the result is the same. CAVEAT EMPTOR.

<comment><USER3> _"If anything is even a little bit off DONT INVEST."_

This is so true. I remember be really pressed hard to invest in some natural gas wells where each well was "owned" by the investors who were paid out fractionally based on the production, the more wells you were in to the more you got paid. Someone I know was getting $3000 - $5000 checks a month from this operation and I was quite curious. But given the scope of things and the kinds of returns they were promising I was amazed that such a deal would even be around by the time people got to me. So I asked for a picture of the gas works. Which was provided but didn't match where they said it was (go go Google Earth!) and once I tracked down where they said things were going on, it was near the edge of the live fire missile range which a) meant you couldn't go out there to take pictures, and b) the US Bureau of Land Management wasn't about to be writing gas leases on land you might get bombed on. Sure enough, a call into the local sheriff left no doubt. No wells, no drilling, anywhere. And he got another data point for his investigation (I wasn't the first person who called). About 9 months later the whole thing vanished. I mean literally everyone in charge just disappeared. Sad really.

</comment></comment><comment><AUTHOR> Appreciate the comments, particularly those that provided specific organizations and links related to investigative journalism, white collar crime etc. I also hope the information was helpful to those who are involved in forming their own organizations, dealing with investors, etc.

Some additional points addressing various comments and themes:

>I'm glad there is additional dialogue about investor due diligence. Others are absolutely right that if you're going to invest in a company it is up to you to do your own due diligence. Go visit the company, speak with customers, get references.

>I lost no money in this process and was simply looking out for the interests of a friend and investor.

>The original individual referenced counter sued the State of Idaho for defamation, after the State originally filed suit against him years ago. Obviously the State has far greater resources than I do. As much as I'd like to throw up names, links etc. there is a very real downside and unfortunately is not something I'm willing to do at this time publicly. If some enterprising investigative journalist wanted to take the reins I'd be open to helping them find out information on their own.

>This was posted on Blogger because I didn't want it distracting from other things I'm working on. I also don't think my personal site would have been able to handle the traffic.

>There are legitimate examples of reverse mergers and stocks that aren't traded on major exchanges. However, the smaller the stock the more chance for fraud, fewer checks etc.

>International incorporation is a regular practice and employed by companies such as Apple and Facebook. However the Cayman Islands are simultaneously a hot bed for fraudulent activity (interestingly as are Vancouver, B.C. and Boca Raton, Florida).

[http://www.businessinsider.com/facebook-funneled-nearly- half...](http://www.businessinsider.com/facebook-funneled-nearly-half-a- billion-pounds-in-the-cayman-islands-last-year-2012-12)

As for Unicorns and sniff tests, if you don't believe it, that's ok. I wouldn't have believed it either. It's good you're exercising your right to question things which was the basis of the post.

</comment><comment><USER4> This doesn't pass the sniff test. It might be true, but without names, dates, or companies for verification - it might as well be fiction.

In Unicorns we trust; all others must bring data.

</comment><comment><USER5> You said you contacted the SEC and the FBI, did you consider the Secret Service? They investigate a lot of organized crime when it relates to finance.

Their "dual mission"[1]:

The mission of the United States Secret Service is to safeguard the nation's financial infrastructure and payment systems to preserve the integrity of the economy, and to protect national leaders, visiting heads of state and government, designated sites and National Special Security Events.

[1]<http://www.secretservice.gov/mission.shtml>

</comment><comment><USER6> As the author is establishing background for the story, which took place from 2004-2008, he writes "'White collar crime', and the attention that it receives today, did not exist."

I'm going to give the author the benefit of the doubt and assume that he was being facetious. I suggest that he reword that phrase.

Google ngram of the phrase "white collar crime" [http://books.google.com/ngrams/graph?content=white+collar+cr...](http://books.google.com/ngrams/graph?content=white+collar+crime&year_start=1800&year_end=2008&corpus=15&smoothing=0&share=)

The OED lists uses of the phrase "white collar" from 1919 onwards.

<comment><AUTHOR> The point was not to establish whether "white collar crime" existed already but rather to emphasize how little attention was being paid to it at that point in time.

For example, since then inter-agency task forces have been established, media has begun covering the topics, books have been written and movies produced. <http://en.wikipedia.org/wiki/Inside_Job_(film)>

The general awareness is much higher (despite little action taken).

<comment><USER7> Embezzlement is 'white collar crime' and has happened for a long time (and been the focus of movies). Cheque fraud is also 'white collar crime' and Catch Me If You Can came out in 2002. You're just associating more public scrutiny (and therefore more government focus) on banks / investing / Wall Street with a greater focus on "white collar crime."

</comment></comment></comment><comment><USER8> "Ultimately my investor has never been paid back, and the individual I met as a possible board member is alive and well, wheeling and dealing in the San Francisco Bay Area."

So, who is he?

<comment><USER9> It sounds to me like someone trying to get feedback from HN on his script. Is is believable? Sure. Would it make a good movie? Not in my opinion because there are actual stories with hard facts and names that are just as interesting, but more so because they actually happened.

<comment><USER3> I like this explanation. If you were going to write a financial thriller, just what would it take to be believable, and when does it jump the rails? Can the reader believe high level CIA officials were duped? How about a complete lack of interest in oversight bodies? What level of response keeps the tension high and doesn't make it boring?

I wonder if you could a/b test this with two stories and adwords or something. Completely automated story generation starting with basic plot. That would be a cool startup.

</comment></comment></comment><comment><USER10> _I contacted people at the Wall Street Journal, New York Times and even had an in person meeting with someone at the offices of Fortune._

Why not _4 Corners_ or _Panorama_? Investigative journalism might not have a capitalist business model anymore, but socialism still gets the job done. Especially for stories that will embarrass rich Americans ;-)

</comment><comment><USER11> That would be much more useful if we had some names to go with that story.

<comment><USER12> Strongly agree. If he names names, something good might come of this now that it has HN's attention for the day. And if he was going to journalists, he had a willingness to name names at some point in the past. I understand that these guys are organized crime, but without names no action can be taken. With the right push, Anonymous can/will dox the hell out of these guys given enough details. OP has to judge for himself whether (a) these are really bad guys and (b) he has the courage to call down the Internet upon them.

(To preempt: yes, Anonymous is not always good. No, I don't believe that Internet justice is always bad, or that it would be bad in this case, especially as all formal avenues appear to have been exhausted and the perpetrator is scot free.)

</comment></comment><comment><USER13> Do we have anything like independent verification of this story? I don't find it implausible, but a single entry purpose-built blog telling an anonymized tale, is worth the cost of entry.

</comment><comment><USER14> Great read.

This part left me a bit shocked:

 _His response, "And you found all of this information just by looking on the internet? Incredible." Let me repeat that, this is the former Deputy Director of the CIA._

I hope to hell that the guy was a just Washington bureaucrat and it's not indicative of the CIA's thinking.

<comment><USER15> The article doesn't specify _when_ he held that title. Of the dozen or so living former Deputy Directors of the CIA[1] there are only a few that I would expect to _not_ to still be surprised at how much you can find out over the Internet. Heck, I'm still in my twenties and I'm impressed at how easy it was to narrow it down to one likely candidate.

[1] [http://en.wikipedia.org/wiki/Deputy_Director_of_Central_Inte...](http://en.wikipedia.org/wiki/Deputy_Director_of_Central_Intelligence)

<comment><USER14> Good catch. I never thought it might have been a deputy director from that long ago. In that case, it makes a lot more sense that he wouldn't know.

However, don't you think that someone who is involved in intelligence gathering would have _some_ inkling of the important of the internet in intelligence?

</comment></comment></comment><comment><USER16> This story doesn't sound too surprising, as these kind of scumbags are out there, but it would be helpful to know the names of those involved.

For example, Carl Freer or Rich Jenkins of Gizmondo/GetFugu/Serge fame. see <http://carlfreerscam.wordpress.com/>.

Sadly, if these folks put as much effort into building a real company as they did their criminal enterprises, they might actually achieve legitimate success and not have to run from the FBI.

</comment><comment><USER17> You either keep doing what you do and keep watching your ass at the same time or just change your point of view. Watching your own ass usually is much less attractive activity. It just doesn't smells right, no matter whom you're trying to hunt down.

</comment><comment><USER18> This sounds like a dispute from a failed exit being aired out online. In particular, the unilateral and evidence-free labelling of the offending party as an "organised crime group" smells of vindication (when my phone got hacked it was clearly terrorism, so I'm sympathetic).

If you find yourself in a similar situation contact your lawyer. You, as a stockholder, can sue. The courts are the best place to adjudicate this. _Conditional upon advice from the law firm_ contact the SEC, the state financial regulators, e.g. DFS in New York, and the state DOJs. This is _not_ to elicit enforcement action, simply to put a time-stamp on the complaint.

> _The con artist would then "help" the company execute a Reverse > Merger...[then dumped his] stock prematurely or used insider information to > sell off. This ultimately caused the stock to crash and the public market to > be stuck with the loss. Meanwhile you, the unknowing founder, probably end > up in jail._

This is not evidence of pump and dump. There was a dump which _may_ have violated lock-up terms, fair disclosure, or insider trading rules. But I don't see a pump, just a reverse merger and liquidation by an investor. The classic pump and dump involves acquiring a position in an illiquid stock, spreading false information to get investors to bid up its price, selling out of the position at a profit, and leaving the market to figure out it overvalued the stock.

Also, you will not go to jail. You can be incredibly stupid, down to hiring a chimpanzee chairman who moonlights as a special advisor to the board to take your company public via a reverse merger (he read about it on StockTwits) who liquidates his investment on the day the lock-up ends, thereby hosing every other stockholder, and still be well within the law, which punishes fraud, not naïveté.

> _Here were a few reasons why it didn't get covered: ...b) Too difficult to > prove_

The threshold for the Wall Street Journal, Bloomberg, et al to report purported fraud isn't astronomical. They can claim first amendment protection and like to break this kind of information for short sellers.

> _The international holding company was actually an organized crime group, > with offices in Barbados and the Cayman islands committing fraud in the US._

This line set off alarms because I've experienced, first hand, how difficult it is to get information about a Cayman holding company or trust in person, let alone remotely, let alone before 2008.

> _In order to silence the class action lawsuit from Texas, the fraud > portfolio company ultimately filed for bankruptcy and settled with the > investors._

If this is true, the class action settlement is illegal - you are not allowed to take payouts from a fraudulent scheme. Further, it is unlikely that the victims of a fraud would settle via a class action lawsuit (versus privately and bilaterally), let alone settle at all. The former deputy director of the CIA could probably manage a phone call to a DOJ.

<comment><USER19> Per the OP (<http://news.ycombinator.com/item?id=5237895>), he did not lose any money in this. It was a cautionary tale.

<comment><USER18> One can take sides without losing money. In any case, the evidence presented still does not sustain the conclusion. There are also technical mis- statements. I still believe the more likely explanation is a well-meaning but naïve individual researching a colleague's issue, coming to a faulty but understandable conclusion, and then getting ignored by those who could have pointed out potential leaps in the logic.

<comment><AUTHOR> _> The classic pump and dump involves acquiring a position in an illiquid stock, spreading false information to get investors to bid up its price…_

This happened. The company paid exorbitant fees for PR and news services to hype the stock with announcements that were't real in the build up period.

 _> This line set off alarms because I've experienced, first hand, how difficult it is to get information about a Cayman holding company or trust in person, let alone remotely, let alone before 2008._

I didn't interface with any of the Cayman organizations directly. What made it difficult? Just paper trails, continual mailed notices etc?

 _> If this is true, the class action settlement is illegal - you are not allowed to take payouts from a fraudulent scheme._

Not a lawyer, however the company shut down just before the SEC investigation was complete, investors settled the lawsuit before fraud was proven, and evidence dismissed before it could show up in public records…it is very possible.

As a high frequency trader I'm sure you're very familiar and aware of how plausible and possible these things are.

</comment></comment></comment></comment><comment><USER20> Wow... What incredible story. Unbelievable!</comment>


Get whole year of Skype Premium for free. - <AUTHOR> https://collaboration.skype.com/promotion/

<comment><USER2> Free group video calls: [https://jitsi.org/](https://jitsi.org/)

Free group screen sharing: [https://jitsi.org/](https://jitsi.org/)

No advertising: [https://jitsi.org/](https://jitsi.org/)

Live chat customer support: [http://webchat.freenode.net/?channels=jitsi&uio=d4](http://webchat.freenode.net/?channels=jitsi&uio=d4)

<comment><USER3> wow, looks great! I am looking forward to WebRTC support (it is on their roadmap for Q2 2014 as it seems).

</comment></comment><comment><USER4> Seems like a promotion for this:

The Skype Collaboration Project: [http://collaboration.skype.com/](http://collaboration.skype.com/)

<comment><USER5> it sounds as if AOL mail offered one year of increased storage when Google introduced Gmail.

Another commenter mentioned jitsi which is free. Then, there is Google Hangouts. Why does Skype think it can get away with charging money for Skype to Skype calls, even multiparty?

</comment></comment><comment><USER6> Thanks but no thanks.</comment>


A Detroit Policeman Invented the Modern Traffic Signal - <AUTHOR> https://en.wikipedia.org/wiki/William_Potts_(inventor) <comment><AUTHOR> My late father knew the traffic signal inventor, William Potter's, son.</comment>


The Internet Needs Decentralized Content Distribution - <AUTHOR> https://medium.com/futuresin/the-internet-needs-decentralized-content-distribution-d0a805b26f6 <comment><USER2> An interesting read, and I agree that we need to de-couple "news" from "advertising" but the question still remains - "How will content creators make money?" Since, unfortunately, financial gain still seems to be what's driving journalism and news reporting. I don't know how to fix that, or if blockchain (which is ultimately what the article is about) is going to be _the_ solution, but it might be part of the solution.

</comment><comment><AUTHOR> Amazon and Google as centralized entities will only get stronger, decentralization on the other hand has an off-chance of going entirely extinct.

<comment><USER3> Email newsletters are decentralized, unlikely to go extinct, and compatible with many payment mechanisms.</comment></comment>


The "Lost" Steve Jobs Speech from 1983: Foreshadowing Wireless Networking & iPad - <AUTHOR> http://lifelibertytech.com/2012/10/02/the-lost-steve-jobs-speech-from-1983-foreshadowing-wireless-networking-the-ipad-and-the-app-store <comment><USER3> I think that an important distinction has to be made about this kind of people. Unlike many journalists, and 'futurists' out there, trying to predict what the future will be like, people like Jobs and Gates[1] do not make bold predictions, just for the sake of making them. Instead they have a vision and they work to make that vision real.

That's the reason of why, when they speak, sometimes the things they tell may seem to be a little bit odd at the time, but eventually that things became real.

There are a few examples of this in the interview made by AllThingsD to Gates and Jobs together in 2007. [1] At 35:30 Gates said that he was sure the tablet market was about to explode (while jobs was looking at him with a funny poker face), and later, he basically described Kinnect, a few years before Project Natal was made public.

If Google haven't revealed Glass or their autonomous cars, and Larry or Sergey came out speaking about driver-less cars, and wearable devices, some would think that they are dreaming, but the truth is that they are describing their vision, and they are working to make it real.

It's like what jobs said at min 43, when asked about 5 years from 2007 (yeah, right now!). And he said: "I don't know, " [...] "but something comes along, gets really popular, people love it, get used to it, you want it on there".

They are not wizards, they happen to know what currently exists, and what's being developed. So, instead of making weird assumptions (like: "ok, assuming that we will develop a material 1000 times stronger, 100 times lighter and 10 times cheaper, we'll have X in ten years"), they rationalize about what's the logical evolution of the technology, based on what they know.

[1] [http://allthingsd.com/video/?video_id=60C4F9FA-9AD5-4D04-8BB...](http://allthingsd.com/video/?video_id=60C4F9FA-9AD5-4D04-8BB6-015AEBB1C052) , Min 28 and forward.

<comment><USER4> Exactly! As Alan Kay said "The best way to predict the future is to invent it".

I don't find it is so important that people like Jobs, Gates, Brin, etc. have sometimes correctly predicted the future. What is important is that they executed, and turned these predictions to reality. Even more. They turned these predictions to everyday reality for most people, instead of a select few.

It annoys me when these people are compared to people who just throw a million ideas around and then, when one of these ideas becomes reality, they claim that they are so good at predicting the future.

<comment><USER3> That quote came to my mind, while I was writing my comment, and I think it's magnificent. I love it, because it basically tells you: Don't lay there just dreaming about the future, go out and build the future!

</comment></comment></comment><comment><USER5> This tape really is worth listening to for anyone who was not involved with computers in 1983. Jobs clearly sees what is happening in the home computer revolution. He's not alone in that, many of us at that time realized that the computer was going to take over everywhere.

Around that time I worked on a project for a company that made machines to stick labels on product packets. Those machines were electromechanical and worked but were quite inflexible. I was asked to prototype a computer control of one of the machines and wrote the code by hand and demonstrated that not only was computer control cheaper and smaller it was way more flexible. For example, the electromechanical machines were unable to detect when there was a missing sticker on the roll of stickers to be applied to the products. It was trivial with computer control (and an IR detector) to add that functionality.

The machine was a 6502-based box whose interface was a hex keypad and a 6 digit seven segment display. You typed your program in in hex and ran out, but it and enough I/O channels to run the machine.

Here is is... The KIM-1: <http://en.wikipedia.org/wiki/KIM-1>

</comment><comment><USER6> _He confidently talks about the personal computer being a new medium of communication. Again, this is before networking was commonplace or there was any inkling of the Internet going mainstream. Yet he specifically talks about early e-mail systems and how it is re-shaping communication. He matter-of- factly states that when we have portable computers with radio links, people could be walking around anywhere and pick up their e-mail. Again, this is 1983, at least 20 years before the era of mobile computing._

Meanwhile, and also in 1983, Steve Roberts was traveling across the US with his computer-on-a-bike and checking his email across CompuServ using satellite links, so actually _doing_ it: <http://microship.com/resources/technomadic- tools.html>

<comment><USER7> I seem to remember that the satellite links came later, in 1983 the Winnebiko only had CB radio for communication (no email) and an audio-casette deck to store what he was writing on his stops.

Winnebiko 2 added packet radio for email in 1986-88, but I don't think the satellite links came until Behemoth which was in the 1990s.

Neat presentation he did describing it at Xerox PARC in 1989. <http://youtu.be/tDaz8vaKzdQ>

</comment></comment><comment><USER8> I thank Mozilla almighty for Firefox' 'Inspect Element' feature allowing me to block that intrusive element full of social networking cruft that was obscuring much of the content. If it wasn't for that, I wouldn't attempt to read this article at all.

Presumably some browser/window size configurations wouldn't have a chunk of article text blocked by an obnoxious window demanding I shill this article on a bunch of websites I never visit, so I figure it's bad website design rather than an intentional retro throwback to 1998-style popup-heavy browsing, but still - if you made that site, and you're reading this, make it not happen anymore.

<comment><USER9> Great tip.

For others, in Firefox, under OSX:

Tools -> Web Developer -> Inspect

Then hold down option, select the box, then click on the downarrow "delete node" and the box will disappear.

</comment></comment><comment><USER10> We had everything he said back then - it was just rather bulky. It's not a visionary statement - it's logic.

I wish everyone would stop harping on about Jobs - he was a marketing expert not a visionary.

<comment><USER11> oh boy aren't you smart. the old "just marketing" troll.

      all you need to make an mp3 player successful is marketing.
      all you need to make a smartphone successful is marketing.
      all you need to make notebooks successful is marketing.

my god, it is so simple!

fire all the designers, engineers, coders, all the fucking nerds and replace them with marketing people! brilliant. this is exactly how you become one of the most valuable companies on this planet.

quick, call HP, Dell and Nokia to tell them about this grandiose insight!

<comment><USER12> Bit harsh, but I sympathise with your frustration. The "only marketing" trope is so shallow that it is annoying how it survives.

<comment><USER10> In the interest of backing up my statement and sticking to my guns:

Take the marketing and hype away from Apple, and what is it? A Foxconn rebranding outfit, a 1990s style UNIX outfit which ships proprietary hardware and a media pushing company with a less than 7% market share of the IT industry with many rising competitors. That is it.

That is no different than Sony 10 years ago. And look where they are now.

Hype cannot be maintained forever. When you can no longer outdo yourself, it's gone. This is happening rapidly. People used to wait on Apple news announcements. I mean hell even I did as they usually stepped up the game. But what have they delivered of note recently: absolutely sod all. Lumias are starting to tread on their innovation territory, Surface is less than a month away, Google has taken over the appliance model and the mobile market is dominated by South Korean giant Samsung.

They're losing the marketing battle as that's all they had and they're suing everyone on the way down.

My main problem is that people worship Jobs as some kind of idol. I think he was an extremely bad role model and a warning rather than a sign. Sure he knocked up Apple's market cap, which is his only real acheivement, but it in the process he shafted just about everyone on the way, destroyed the perception of an open computing model, lied persistently about bad products and ripped off other people's work persistently.

The guy was technically speaking in every way an utter psychopathic arrogant asshole with no remnant of humanity. But a good marketer.

I'm fed up with reading all the tripe about him.

<comment><USER13> Your position is absurd and contradictory.

The hype and popularity exists because Apple has produced very compelling products. The iPod and the iPhone were both leaps and bounds ahead of the competition at their launch. It's also clear that Jobs was the driving force behind their vision.

You say that "even [you]" waited on Apple announcements because "they usually stepped up the game" which translates into the fact that Apple made better/interesting/more innovative products. Note you didn't say "even I waited on Apple news announcements because their marketing was so amazing."

You say that Lumias are treading on their "innovation" territory. Not their marketing territory, their innovation territory.

If it was all marketing, they would still be winning by your argument. Has their marketing dramatically changed in the past few years?

The only rational way to reconcile this is by you saying "oh, by marketing, I mean the whole product package, user interface, software, and hardware design". In which case, you just are simply defining marketing wrong.

<comment><USER10> The iPod and iPhone were not leaps and bounds ahead. That is a common misconception which is powered by the marketing hype. There were other products out there which were far superior. Archos produced better music players and Nokia produced far better phones under the Symbian banner.

The differentiator was the marketing hype.

They stepped up the game by delivering on day zero which made people hang on them. That is still marketing.

Lumia marketing is horrible, but they are producing hype via innovation. Apple don't do that any more. They have nothing to deliver any more.

Their marketing has changed from "new product" to "new incremental improvement" i.e. the hype is dying.

Marketing here is purely spin.

<comment><USER14> I'll bite on the Symbian claim.

Saying that the iPhone is not leaps and bounds better than anything else in the market at the time is disingenuous - if you used both the N95's browser (with the crummy joystick-controlled mouse) and the iPhone's multitouch browser, you'd know there is no comparison.

I'll go further and say the Nokia phones at the time were much more marketing- driven. They had huge checklists of features _so that marketing could say they have "more features"_ (e.g. irDA), but the core experience was poor (unresponsive and confusing UI, bad input methods) it didn't matter.

Did Nokia make better phones? In one sense, yes - they had longer battery lives, better cameras, they worked much better as phones, for chrissake. But, in the most important way, the iPhone blew everything out of the water - <USER2> phone people would prefer to use. I preferred a usable browsing and mapping experience to the jack-of-all-trades and master of none approach of Nokia phones, and apparently, so did the market.

<comment><USER10> Well you kind of justified my point:

The iPhone is a crappy phone with crappy battery life and has a crappy camera. None of this is desirable nor unique (O2 XDA kind of nailed all of these in 2002).

Apple however made it successful through marketing.

And don't mention usable mapping after the last week or so :)

<comment><USER15> It's got a good web browser, good media playback, good apps and games, and good display. The camera isn't crappy either IMO.

<comment><USER16> on launch the iPhone didn't have a great camera. (Or apps)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Wow that was visionary! </sarcasm> if it wasnt for the fact that Alan Kay had invented the dynabook concept in 1968 and had cardboard prototype of an ipad in 1972. And he went and helped create that vision in xerox parc.

and later went to work for apple were jobs was able to copy more of his ideas.

So yeah what a prediction say in 1983 what Alan Kay invented in 1968.

<http://en.wikipedia.org/wiki/Dynabook>

<comment><USER18> You sound bitter. So other people have thought of and built parts of what jobs' team brought together, ergo Jobs was not a visionary? You can hate Apple and SJ's personality as much as you want, but don't let it blind you. Lisa and Mac invented more over the Xerox UI than you think, plus they got the license from Xerox in a fair deal against shares.

<comment><USER17> You mean the macintosh project that Steve Jobs took away from its original inventor Jeff Raskin?

Im not bitter it just bothersme the fanboyism that is so prevalent in Sillicon Valley...if we want to improve our craft we need to know it's history.

<comment><USER19> Except that Raskin's vision for the Mac was completely different from the direction Steve took it in. That's not speculation, we know what Raskin wanted the Mac to be because he founded a separate company and implemented it as SwyftWare and later the Canon Cat. Raskin's plans for the 'Mac' interface didn't even use a GUI, it was entirely text based.

</comment></comment></comment></comment><comment><USER20> Wow. He was just 28 when this was recorded, I think.

</comment><comment><USER21> Repost of <http://news.ycombinator.com/item?id=4605117>

</comment><comment><USER20> On a side note, is digitizing and distributing this recording violating some copyright?

</comment><comment><USER22> I came here, thinking it would be another typical Jobs speech, and was expecting mindless Apple fanboys to praise everything about it.

But 5 minutes in, I realize this speech is actually filled with content, and good content at that.

</comment><comment><USER23> To everyone saying Steve Jobs is just a "marketer" obviously doesn't know what it takes to run a company, let alone his full genius. Now watch as Apple starts falling down.

<comment><USER24> Steve Jobs was obviously holding Apple back, only just recently did we finally get a 4 inch screen, 4G LTE, and (keep in mind he publicly decried this form factor) an iPad mini 7ish inch tablet

long live tim cook

<comment><USER24> my source: [http://liliputing.com/2010/10/apple-ceo-steve-jobs- bashes-7-...](http://liliputing.com/2010/10/apple-ceo-steve-jobs- bashes-7-inch-tablets.html)

<comment><USER25> And jobsy bashed tablets not long before he released the iPad. What jobsy said publicly was not a good basis for what he believed.

</comment></comment></comment></comment><comment><USER26> Almost as if he was born with a single vision that he implemented all the way through his life.

</comment><comment><USER27> Is it me or is there a recent trend of more and more apple/steve jobs/ios threads? There seems to be so many of them, when ios6 came out there were quite literally a wall of related threads on the front page.

Is it because most people these days use macbooks for development?

</comment><comment><USER28> I highly recommend listening to this.

</comment><comment><USER29> Startup = Vision

</comment><comment><USER30> And Sci-Fi has done far more for tech innovation than Steve Jobs. Too bad the authors never get any credit.

</comment><comment><USER31> This article is bunk. I'd advise the author to stay far away from fortune tellers and palm readers.</comment>


Ask HN: What stack should I use for my mobile app? - <AUTHOR> I am about to start development on a mobile app prototype, which, if we can secure funding, would be used by hundreds of thousands of people daily.
I would like to build the prototype on the same stack as I'll use for the final product to start getting my head around architecture choices and third party libraries to include.
I'm predominantly a web developer, having experience in Ruby on Rails and Django on the backend side of things and having worked with React, Cordova (Phonegap) and Angular.  I've built some mobile apps using these technologies, as well as a small mobile app using Kivy.
The app will largely be informational with some data capture and lots of updates pushed to users, probably several times a week.
So based on my experience, I was thinking of going with Django Rest Framework and React Native, as I've built an app with this before a few years ago.  But I remember dealing with the state on the UI side being a bit of a pain and was wondering if maybe going with Django, GraphQL (Apollo) and React Native wouldn't be better (even though I don't know GraphQL yet).
Do I have better options?  Would my suggested stack be able to eventually handle the workload?  Obviously the API will have to sit on some solid cloud infrastructure as well. <comment><USER2> In the last 6 months, I've built 3 mobile apps using React Native with Django Rest Framework. I can't be more satisfied.

We tried GraphQL, it added a lot of complexity on the backend that you don't really need

We like to keep things simple, so we avoided state management libraries and all we needed was [https://github.com/async-library/react- async](https://github.com/async-library/react-async)

We also liked onesignal to manage push notifications

<comment><AUTHOR> Something crucial I should have mentioned is that many of the users will likely have poor internet quality. Not sure if this will end up affecting my need for state management.

<comment><USER3> I kind of disagree with OP about GraphQL adding too much complexity on the backend. Depending on what kind of queries you define, creating efficient resolvers that don't hit the database excessively can be complex, but in most cases I find GraphQL very easy to work with.

I've used Django Rest Framework as well, and although I would agree there's a bit less overhead (though I'd argue not that much: you still should be defining serializers for models like you would be defining types in GraphQL, there are just a lot of nice little helper methods and conventions for DRF that make it a bit faster than setting up GraphQL with Django), one of the main selling points of GraphQL is the ability to use Apollo on the frontend. Because of Apollo's caching system, you don't really need to create any custom state management related to fetching and storing queries from the server. You might need state management for other purposes, and you'll still need to tell it to do things like update lists when deleting items/when a mutation might invalidate other cached query results, but it handles a lot right out of the box.

The biggest headaches related to SPA state management I've run into before Apollo were always about fetching data efficiently and invalidating results from the server, which Apollo does really well. You don't need to worry about "am I fetching X somewhere else an extra time" or "did I remember to update Y in all the right places in the store when I fetched new data"; you can just declare the data you're component needs, and Apollo will either get it from the frontend cache, or if it can't find it there, the server.

If your app was more about user input than ingesting information, I might recommend checking out pouchdb. I just started using it for an app that's almost entirely based on user input that I wanted to work during long stretches of being offline, and it's been working pretty well so far (although I don't feel I've done enough with it to really endorse it fully yet/I have yet to see how the project pans out).

But since it sounds like your app is mostly about displaying information coming from the server to your users, and you want to avoid hitting the wire for data as much as possible, I think a combination of React Native and Apollo is probably a good fit. I'm sure there are other solutions, but it's the one I'm most comfortable with/what I'd probably jump to in your situation.

<comment><AUTHOR> GraphQL and Apollo looked attractive to me for all the reasons you stated, but I was just worried that the learning curve and making it play nicely with the rest of the stack (that I know quite well already) would be too steep to justify using for the prototype. I'm thinking of writing the prototype just using DRF and React Native but during each step of the process basically looking into how that step would be done with GraphQL.

<comment><USER3> I was familiar with GraphQL before integrating it into a Django project, so my experience may differ, but I had a good experience using graphene: [https://docs.graphene- python.org/projects/django/en/latest/t...](https://docs.graphene- python.org/projects/django/en/latest/tutorial-plain/)

Since efficient querying is a concern, I highly recommend just biting the bullet. It's extremely easy to underestimate the complexity of caching state on the frontend and to end up accumulating an unmaintainable frontend state management situation. It's totally possible to do what OP suggested without running in to those kinds of issues, but typically that involves more frequent fetching from the server.

It depends how quickly you need to make the prototype/whether you'd be willing to refactor it/how efficient the querying really needs to be/whether you think you can keep state management simple and structured enough to avoid it ballooning/how quickly you think you can grok the basics of GraphQL/etc. Apollo doesn't really care what your schema looks like, unlike relay, so the learning curve isn't as steep. When I picked up GraphQL/Apollo for the first time the terminology felt pretty obtuse, but once the basic idea clicked it felt pretty natural. Most of the devs I've worked with have had similar experiences.

<comment><AUTHOR> I'd like to get the prototype out quickly, and then there will probably be quite a bit of time before the final product is launched.

Things like this worry me about getting up and running with GraphQL though, especially "Common GraphQL problems: Server/Client Data Mismatch":

[https://www.freecodecamp.org/news/five-common-problems-in- gr...](https://www.freecodecamp.org/news/five-common-problems-in-graphql-apps- and-how-to-fix-them-ac74d37a293c/)

<comment><USER3> Out of all those problems I'd say the most difficult one to tackle is related to performance, but if it's an api that only you are hitting, you can usually avoid doing anything too costly.

The recommended solution of using graphql on the server for the server/client data mismatch problem is a good one, but it's not really applicable in your case. That's geared towards people using both javascript and nodejs. If you're using javascript on the frontend and python on the backend, you can't share code/there's inevitably going to be a mismatch. A mismatch isn't always a bad thing, either; often you don't need to expose all of the database models to the client, and want a lighter weight abstraction.

</comment></comment></comment></comment></comment></comment></comment><comment><USER4> If you are a Rails developer, this Turbolinks presentation makes a strong case to be used for mobile apps. [https://m.youtube.com/watch?v=SWEts0rlezA](https://m.youtube.com/watch?v=SWEts0rlezA)

It seems like there is a strong productivity gain from using rails here since it's extremely productive and you can make simple wrapper apps to release to the Play and App Store that really just wrap the web view.

The biggest gain is obviously reusing most of the web app across three different platforms, and the ability to add native navigation and interaction as needed, later. You also avoid needing a big javascript library.

<comment><USER5> I've looked at this before, but it seems to require you use remote forms. And when you use remote forms there's no out of the box solution for displaying errors. Am I missing something here? Were you able to successfully build a Turbolinks app off a Rails app?

</comment></comment><comment><USER6> Why not node.js instead of Django? In general I feel node.js community is more active in terms of web development. And you don't need to do context switch between node and python. I would go with GraphQL, the part that I like the most is the schema. You don't need to maintain doc now. There is also a playground, you can do integration test easily during development.

If you feel bad about all the boilerplate code you need to write in server side. There are many node libs/blackbox services that can help you, such as prisma, hasura. You can also take a look at a lightweight library that makes your life easier when building GraphQL server: [https://github.com/charlie0077/graphql-server- crud/](https://github.com/charlie0077/graphql-server-crud/)

I am the author of that library.

</comment><comment><USER7> Django is fine, if a little heavy but it will provide a good restful API. For the app - I'd really recommend dipping into Flutter. The language is simple, tooling is great and you can get up and building rapidly.

<comment><AUTHOR> I'm keen to try out Flutter as I've heard good things, but I'm weary of delving into an entirely new language for this project if the languages I know already offer a viable (and popular) mobile app dev stack. Unless it really makes state handling on the UI mega simple, which is my biggest problem with the JS library/framework solutions I've worked with before.

<comment><USER8> Try out both and check them out. The dev experience on Flutter is in my opinion far superior to React Native. It _just works_ whereas with RN I spent so much time getting things to work correctly, even with Expo. I am not sure why it's so terrible given that create-react-app "just works" almost as well as Flutter.

</comment></comment></comment><comment><USER9> React Native and Firebase

or, if you want to be more independent

React Native and FeathersJS

<comment><AUTHOR> What are the benefits of FeatherJS and Firebase over using the backends that I'm already familiar with?</comment></comment>


Uber Admits That Driver Arrested For Killing 6-Year-Old Worked For Them - <AUTHOR> http://sfappeal.com/2014/01/uber-distances-itself-from-driver-arrested-for-allegedly-striking-and-killing-6-year-old

<comment><USER2> Shouldn't he be considered innocent until proven guilty in a court of law?

<comment><USER3> It only says he was arrested, not that he was guilty. The process is arrest, plea...

</comment></comment><comment><AUTHOR> Uber statement: [http://blog.uber.com/2014/01/01/statement-on-new-years- eve-a...](http://blog.uber.com/2014/01/01/statement-on-new-years-eve- accident/)

Local coverage: [http://sfist.com/2014/01/01/7-year- old_girl_struck_and_kille...](http://sfist.com/2014/01/01/7-year- old_girl_struck_and_killed_i.php)

 _> In addition, the driver was a non-taxi, for-pay driver. Certainly this incident raises questions regarding the driver training and selection process for the increasing number of ride share drivers on our city streets. While emerging driver service apps fulfill an important need for flexible on-demand transportation in San Francisco, we must examine a universal baseline of scrutiny and safety oversight for these services._

Techcrunch examines the question of insurance company liability given the claim that the Uber driver was not "doing a trip": [http://techcrunch.com/2014/01/02/should-car-services- provide...](http://techcrunch.com/2014/01/02/should-car-services-provide- insurance-whenever-their-driver-app-is-open/)

 _> People have scrutinized Uber's approach to insurance in the past. But this terrible accident on New Year's Eve brings a specific question into focus. Should app-based taxi and ridesharing should provide insurance to cover drivers while they're between rides but actively looking to pick up passengers?_</comment>


SSH Brute Force – The 10 Year Old Attack That Still Persists - <AUTHOR> http://blog.sucuri.net/2013/07/ssh-brute-force-the-10-year-old-attack-that-still-persists.html <comment><USER2> "As for protecting your self against these, you need to use strong and good passwords..."

I went back to the top to see when this was posted...today. I'm no expert on securing a server, but I thought the common thinking now was to just turn off password auth. The post itself says that all it takes is one weak password to be compromised.

Using keys is ever-so-slightly less convenient at setup, but negligibly so. Works on any device I have. Is there a reason one would continue to use passwords?

<comment><USER3> >Using keys is ever-so-slightly less convenient at setup

And it's easier once it's setup. I don't have punch in a password any time I need to make a server change.

<comment><USER4> Don't forget that makes you more vulnerable to malware on the client, which now has easy access to your keys, and your known_hosts file (which gives the attacker a convenient list of servers to attempt to log into using those keys.)

<comment><USER5> Yes, this is why having a passphrase on your key is recommended. The idea is not to use keys _instead_ of passwords, but _in addition to_ : the password protects the private key locally, which is used to authenticate to the server. However that seems to be uncommon practice.

</comment></comment></comment></comment><comment><USER6> There are lots of good guides online on how to protect yourself from this (disable password-based auth, or use 2-factor auth, Fail2ban etc.), but there's one piece of advice that crops up a lot that is probably a bad idea, and that's changing the SSH port. This can give (at best) a false sense of security and (at worst) actually reduce security if you choose a port number above 1024, because then malware could pose as SSH without requiring root access and then steal your password once you've entered it.

Just thought it was worth mentioning because I recently went through all this myself to try and secure a Raspberry Pi and it was news to me...

<comment><USER7> _steal your password_

i had always assumed that the protocol somehow protected your password, but reading the spec at [http://www.ietf.org/rfc/rfc4252.txt](http://www.ietf.org/rfc/rfc4252.txt) that's not the case (and i've pretty much convinced myself it's not possible to do better).

learn something new every day...

[apart from the reference above, someone else here claims to be logging these things.]

<comment><USER8> With "password" authentication method, the password is not transmitted to the remote server. (AFAIK)

However, all modern sshds (AFAIK, at least OpenSSH) default to (publickey, keyboard-interactive). As the name says, keyboard-interactive sends your keystrokes, one at a time (bar buffering) to the remote server.

This is required, for example, when using PAM, as PAM requires the actual authentication key.

<comment><USER7> about password auth - you're wrong (AFAIK). i even gave a reference in my original post. didn't you bother to check it?

<comment><USER8> I am wrong. No.

</comment></comment></comment></comment></comment><comment><USER9> What sane person would allow password authentication on their servers? Turn it _off_.

</comment><comment><USER10> I am very surprised that the OP, and nobody in this thread, has mentioned port knocking.

You are absolutely better off / more secure, with port knocking enabled. The scans never touch your sshd, because your server does not even answer port 22. As far as the outside world is concerned, you _don 't_ run an sshd.

On my own systems, I set up port knocking, I delete the allow rule for the IPs I knocked from every 24 hours (so fresh knocking is needed every day, and I don't leave a trail of "open" IPs as I travel the world) and my .login script spits back at me the current days list of "knocking IPs" so I can immediately note if someone else is knocking.

<comment><USER11> The problem is that it's harder to setup and is less convenient to use. Can you point to any guides that make it simple to set up port knocking?

Would it fairly secure to setup a VPN service, connect via VPN, and then SSH in over the VPN connection?

<comment><USER12> Its easier to go the VPN route than port knocking, but the vpn then ssh is too tinfoil hat for even my taste.

As someone else mentioned, just run it on another port than 22, like 53 or 109, and youll get 99% less brute forcing attempts, and dont use passwords as those in the list, and disable user root to authenticate at all.

If you're still paranoid about the minor brute forcing attempts left then add the iptables rule to limit connection attempts to 1/min. You're still paranoid? For what really? Port knocking wont help out at this level of paranoia. Hrm, security awareness. Its just not worth the hassle.

If you already have OpenVPN setup, then its easy to put the ssh to listen only on the tun interface the vpn server has opened.

</comment></comment></comment><comment><USER13> I usually just limit SSH connections to one of my own static IP addresses, as well as disable password authentication and implement PSK.

At first I was shocked by the amount of intrusion attempts my servers were logging, but as time goes on I see that no attempts have yet been successful, so these sorts of intrusion attempts have probably made me complacent. (never a good thing)

That said, I found the article to be lacking in listing mitigation alternatives, such as those posted here in the comments.

</comment><comment><USER14> It's amazing how quickly the brute force attempts start rolling in if you turn on SSH with password authentication. Last year I had it on for a bit on my home computer and logged thousands of attempts with the username 'initech'. Thank god I went to work for Intertrode! [http://i.imgur.com/6lqXa.png](http://i.imgur.com/6lqXa.png)

</comment><comment><USER15> seriously, password-based auth... have ppl not heard of public key authentication?

allowing password-based auth in sshd is plain stupid. _always_ use pubkey auth, it's a 1-line change in /etc/ssh/sshd_config.

<comment><USER16> Can they really bruteforce a 12 characters password? I don't think so...

What happen when you lose your key? How do you backup your key?

<comment><USER17> The same way you back up all your other important and private files? That is, frequently and securely.

Also note that you can have multiple keys, potentially one per device (useful in case a device could get stolen).

</comment></comment></comment><comment><USER8>
      ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh limit: up to 1/min burst 2 mode srcip

Have fun brute forcing at 1 connection per minute.

(Oh, and PasswordAuthentication is off too.)

<comment><USER18> Can you paste the config line that adds this to your iptables?

<comment><USER8>
      -A PORTS -p tcp -m tcp --dport 22 -m hashlimit --hashlimit-upto 1/min --hashlimit-burst 2 --hashlimit-mode srcip --hashlimit-name ip4-ssh-brute -j ACCEPT

</comment></comment></comment><comment><USER19> What sane person exposes SSH externally?

\- OpenVPN takes only minutes to set up

\- There are easy to use GUI clients for all major desktops (Windows, Mac OS X, Linux) and phones (Android, iOS)

\- It supports running with zero privileges (chroot+setuid) since it only needs to forward packets to /dev/(tun|tap), which does not require privileges after the device has been opened.

\- It limits your compromise exposure to only one edge gateway (you can put any number of machines behind a single dedicated gateway). If the gateway is pwned, you still have the defense-in-depth of internal SSH, internal encrypted comms, etc.

\- Supports HMAC validation of incoming data to avoid any complex processing (and thus exposure of bugs) of packets from users that don't have the shared HMAC key. This means that _only_ the HMAC code is exposed to untrusted users.

\- Supports public key +/ password authentication.

There's _no_ reason to expose SSH to the public internet when you can run a gateway that handles _only_ VPN traffic and greatly limits your attack surface. Defense in depth!

<comment><USER20> You would be surprised. But to say the truth, I trust the OpenSSH code a lot more than any VPN software that you can install to prevent direct access to it.

<comment><USER21> It's fairly common actually, at least around 12,860,698 devices have their SSH open to the public ([http://www.shodanhq.com/?q=port:22](http://www.shodanhq.com/?q=port:22)).</comment></comment></comment>


Ask HN: Why was there so much outrage over SOPA compared to NN? - <AUTHOR> I remember when SOPA was announced there was so much outrage everywhere. Reddit shutdown for a day(iirc?), fb wall was filled with SOPA related posts, even HN had a black bar (again iirc).
But this time except one or two posts on Reddit homepage it seems not many are outraged. So just wondering whats the reason behind it.
1. Everyone is just tired of protesting? 2. With trump at helm there is not much hope? 3. SOPA was more important than NN? 4. It's Thanksgiving and their plan is working? 5. It's not that urgent just yet?
6. I'm creating a strawman? <comment><USER2> Likely a combination of the reasons you listed but given that past protests against ending net neutrality were quite extensive, I'd say protest fatigue is a large part of it. And not just fatigue over the ending of NN rules seeming to be inevitable due to attacks on it seeming to never end but also protest fatigue in general. Protesting just about anything in order to gain social currency has burnt people out, and now we're all left vulnerable due to protest energy having been spent on issues such as 'air conditioning is sexist'. People are tired of being asked to be outraged all the time, which makes it an excellent time for pushing through what would otherwise be unpopular changes.

<comment><AUTHOR> Yes, I too thought the same that's why I listed it a #1 I guess. Kind of the best strategy by govt for anything. Instead of saying No and facing a backlash, just keep delaying it (which makes people think they've won at least at that time) but then keep bringing it up over and over.

I think people get bored and run out of steam by 2nd or 3rd time.</comment></comment>


Yahoo to ignore IE10's "Do Not Track" - <AUTHOR> http://www.ypolicyblog.com/policyblog/2012/10/26/dnt/ <comment><USER2> I think it is perfectly reasonable to assume that users intend to not be tracked by the very large number of third parties they are involuntarily exposed to on the web.

Yahoo are resorting to this whole buzzword-laden meaningless rhetoric around ~user experience~ and ~value proposition~. That just reinforces the impression that the only reason anyone was prepared to go along with DNT was that they assumed that 99% of users weren't going to be in a position to express their ~user intent~ to not be tracked. Since, you know, most people have better things to do than to learn how to teach their computer about obvious preferences like "please don't spy on me".

Microsoft is simply making the benefits of the DNT scheme more accessible to its users. It's pretty telling that Yahoo is already backpedaling from respecting the users' intent, faced with the possibility that more than an insignificant fraction of users might actually be enabled to benefit from DNT by this decision.

(Edit: Personally I think rather than squabbling about DNT, browser vendors should be taking much more aggressive, technical steps to make tracking users harder, instead of having a default configuration that stops just short of transmitting the user's SSN via request header. Disabling features like user agent and referer headers for and quickly discarding cookies from untrusted (by individual user "intent", not based on SSL certs or anything) hosts would be a start.)

<comment><USER3> > the benefits of the DNT scheme

The benefit of the DNT scheme was to kill the lie that most users don't care. If 99% of users take positive action to change a default and say "Don't track me", it's believable. If a browser vendor says this, it's not.

Bear in mind that Do Not Track has _zero_ technical merit; it's equivalent to the "evil bit" prank RFC. Any merit it has must be political.

The value in DNT was going to be that we could convince advertisers that normal users do, in fact, care, and do, in fact, not want to be tracked. IE's decision is squandering what DNT attempts to communicate, and squandering that value. And so when you see advertisers _and_ web server developers rejecting IE 10's DNT indicator, that doesn't mean that the advertisers or web server developers are bad people -- that just means that you lost the politics.

<comment><USER4> That puts Microsoft in a bind. Sensible defaults are important; if you can guess what users want most of the time, then you should just do that.

In their shoes I would have done some focus groups, spending an afternoon with people and really educating them on the details of tracking, and what the pros and cons are for them. If at the end of it most typical users would have turned it on, then this would have been the right default.

After all, if places like Yahoo don't like it, they could ask people to turn it off. If Yahoo's right, then presumably most people would turn DNT off, or make an exception for them. But I suspect Yahoo knows that people don't want to be tracked, and that a lot of their profit comes from keeping their users in the dark.

<comment><USER3> > Sensible defaults are important; if you can guess what users want most of > the time, then you should just do that.

That is a good general rule. In the case of DNT, the header was formulated specifically with the intent that the default would be off, regardless of what you expect the user to want, so that turning it on communicates individual user intent. This is a reason to ignore the general rule in this specific case.

A good related example would be license agreements. Most users want to ignore them entirely. Focus groups would indicate skipping them. But if you make a click-through license agreement invisible, while that's a better UX, the agreement is now completely legally invalid. In order for the agreement to be valid, you need the user to have an opportunity to read it (even if focus groups indicate nobody does).

And while you expect 100% of your users to accept the agreement, the default needs to be "No, I do not accept".

</comment></comment></comment></comment><comment><USER5> > Recently, Microsoft unilaterally decided to turn on DNT in Internet Explorer > 10 by default, rather than at users' direction.

> It basically means that the DNT signal from IE10 doesn't express user > intent.

Blatantly false. Not only are you presented with the option to turn off DNT on first use (that takes up the entire screen), but I'd imagine users would choose to have advertisers track them about 1-10% of the time if made to choose. So a default On setting does represent the consumer to a degree that you can't ignore.

<comment><USER6> I also did a double-take on that first sentence you quoted. I'm pretty sure I read earlier that they will be prompting first-time users on what their preference will be, and that the "default" basically just means which position that switch will be in when the choice is presented. However, I did some quick fact-checking and I couldn't find anything to back that up; if you don't mind, where did you find that IE10 presents that option?

<comment><USER2> I had the same reaction. I found [1], which seems consistent with what I remembered.

[1]: [http://blogs.technet.com/b/microsoft_on_the_issues/archive/2...](http://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/08/07/do- not-track-in-the-windows-8-set-up-experience.aspx)

</comment></comment></comment><comment><USER7> In all the comments here, I can't find anybody who thinks that Yahoo is doing the right thing here. Well, I do. I think what Yahoo is doing is the right thing for them, for their users, and for the web.

If the web is going to be ad supported, then its going to have to be targeted advertising or its going to be both shit and annoying. Remember "punch the monkey", or ads that took over the entire screen? Now, through tracking, we are able to get really really good ads - things you might even be interested to see and buy.

If DNT was supported by everybody and on by default, that's the end of online advertising in its current form. So we can choose from the following options: ignore DNT, ignore DNT for IE10, or go back to non-targeted advertising.

Let's assume the last of those, which leads us to the following options: revert to shit ads, make users pay for content directly, or pack up your content-producing company and go home. None of these are best for the users or the web.

The DNT founders know this - that's why it was default null in the spec and in Firefox. IE10 is doing this deliberately even though they know it can't work, and there are choices here: they are trying to improve the world but are incredibly wonderfully naive, they want to undermine Google, or they want to undermine DNT. I'd love to believe its the first, but no-one has ever claimed that about MS.

<comment><USER8> Or we could have non-intrusive ads and non-targeted ads, there's nothing wrong with that.

<comment><USER7> Except that no-one would click them. We barely click on ads tailors to our behaviour - we'll never click on ads that aren't (and we never did).

<comment><USER8> Personally I'm fine with that.

<comment><USER7> Unfortunately, if nobody pays for the content, it will go away. That's probably bad (lets leave aside discussions of how high the quality on ad- supported content is, and presume that there are people who like to read it).

<comment><USER8> That seems like the 'too big to fail' argument once used in favor of the established banks, albeit simplified.

Advertisers and bankers may be unpleasant allies and we've never wondered what life would be like without them, so let's try it and find out.

</comment></comment></comment></comment></comment></comment><comment><USER9> I know people celebrated Microsoft's decision to do this in IE10, but this is what many of us were saying would happen. The relationship with the Do Not Track flag was always tenuous so flagrantly ignoring the spec (which indicates that default on is wrong) was simply going to cause companies to ignore the flag completely.

Regardless, this whole thing is silliness in the extreme. I wonder if this means yahoo is going to start allowing requests with the evil bit set as well :).

(<http://en.wikipedia.org/wiki/Evil_bit>)

 __EDIT __: also, didn't IE revert this change?

[http://blogs.technet.com/b/microsoft_on_the_issues/archive/2...](http://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/08/07/do- not-track-in-the-windows-8-set-up-experience.aspx)

Well ... Sort of

> DNT fits naturally into this process. Customers will receive prominent > notice that their selection of Express Settings turns DNT "on." In addition, > by using the Customize approach, users will be able to independently turn > "on" and "off" a number of settings, including the setting for the DNT > signal.

<comment><USER10> Advertising companies can be shameless. What makes you think they ever would have respected DNT even if was opt-out?

<comment><USER11> They support the NAI which enables you to turn off various tracking bits. Let's not go around shitting on a whole type of business wholesale because of a few bad actors, hm?

<comment><USER10> You say "they" like all advertisers support this initiative. They don't. I was careful not to include all.

I can assure you that not everyone will follow it. Google is a part of NAI and they were found guilty by the FTC to be circumventing Safari preferences.

<comment><USER11> And you say "they" like all advertisers are skeevy and wouldn't honor the DNT flag.

</comment></comment></comment></comment></comment><comment><USER12> Maybe Yahoo should respect IE10's DNT defaults, but display huge modal screens that tell the user "Your browser vendor is inhibiting our value proposition. Please allow us to track your behavior for maximum value extraction."

<comment><USER13> They should give you a medal. That's how you create upward revenue stream dynamics!

</comment></comment><comment><USER14> Took me a second to figure out this whole DNT business.

So basically it is just an HTTP header your browser sends to the server that tells it not to track. Seems kind of like the wrong way to do it. If I was some nefarious website wouldn't I have straight up ignore it? There isn't any incentive for me to not track a user. In fact aren't a lot of companies around advertising based on the fact that you CAN track users?

More info here: <http://donottrack.us/>

<comment><USER15> The thing about DNT is that the advertising industry (not malicious/shady websites) are supportive of it as a voluntary standard that they will comply with it. All the major players are saying they won't play nicely with IE10 due to the default on flag instead of a default null (no intent expressed) flag.

<comment><USER16> In other words meaning they're not going to support it...

"We're only going to support it if it's not turned on by default" is not really supporting DNT. It really shows they're business model depends on people being computer illiterate. And now that DNT is on by default they're revolting because they know few people will ever go out of their way to opt-in and get tracked.

<comment><USER15> > "We're only going to support it if it's not turned on by default"

No, they are only going to support it if it is _actually_ the users intent, not a vendors intent. This is completely reasonable and the actions of MS are undermining the efforts to get this voluntary standard going. Keep in mind, DNT is completely voluntary.

<comment><USER17> What method are they using to determine if it's _actually_ the user's intent?

<comment><USER18> You have to explicitly set it on, that means the user had to take a specific action, and that means there was an intent to do it.

</comment></comment></comment></comment></comment></comment><comment><USER19> They key thing to understand is that if IE10 did not have DNT enabled, that the default setting would be _just as arbitrary_ and would still therefore not "map to user intent" in their words. There has to be a default in one direction or the other.

That, and many users will use IE10 knowing that it ships with DNT pre-enabled. To ignore this is totally immoral and unethical. This is totally shameful.

<comment><USER15> >They key thing to understand is that if IE10 did not have DNT enabled, that the default setting would be _just as arbitrary_ and would still therefore not "map to user intent" in their words. There has to be a default in one direction or the other.

The default is to send a null value in the header meaning that no intent has been expressed, that's not arbitrary.

> That, and many users will use IE10 knowing that it ships with DNT pre- > enabled. To ignore this is totally immoral and unethical. This is totally > shameful.

Most users will have no idea what DNT is, nor will they bother to switch the flag. MS setting it to do not track by default undermines the effort being put into the standard. No one is going to comply with a voluntary standard if one of the largest browser vendors turns it on by default.

<comment><USER19> > The default is to send a null value in the header meaning that no intent has > been expressed, that's not arbitrary.

My statement that there has to be a default in "one direction or the other" is clearly wrong, as you've indicated. I would still argue that a null value is arbitrary and does not map to the user's wishes. My opinion is that having it 'on' is no more or less arbitrary than having no wishes expressed.

<comment><USER15> > I would still argue that a null value is arbitrary and does not map to the > user's wishes.

It's not arbitrary though, it clearly spells out that the browser vendor doesn't know what the users intent it.

<comment><USER19> Again, fair point - but then by definition the second clause of my statement is correct - the user's wishes are unknown, so any direction taken is then... arbitrary? :)

At any rate, you seem well-versed in this area, so let me ask you a question: what is the difference the website's behavior between a "null" and a "track me, please" value in the header?

<comment><USER15> It's not binary, it's ternary. 1 opt out, 0 opt in, null unknown choice.

In practice the difference between null and don't track me please are probably non existent. However, in the future if this were to take off it's possible that someone would come up with creative benefits/uses for tracking that provide incentive for users to be tracked.

edit: Don't think I addressed your question. The difference between null or tracking ok and don't track are likely just to be generic advertisements shown to a user instead of targeted ads, the prevention of some back end selling of user interaction data and some other things that most people have no idea is going on. If you can't track users across sites you lose some of your ability to build up profiles for them. Advertisers will argue that the ads will have significantly less value without those secondary or tertiary ways to monetize eyeballs and I suppose you could see some decline in what advertisers are willing to pay for impressions. Ultimately I've been removed from advertising for a few years so I'm not entirely sure how much it will make a difference.

<comment><USER19> This is really helpful - thanks.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> > In our view, this degrades the experience for the majority of users and > makes it hard to deliver on our value proposition to them.

I know Yahoo! has to maintain their business which depends on things like ads and content delivery; but to say it with such sterile marketing jargon just makes me nauseous.

How about you guys do what everyone else has had to do since the beginning; create something awesome and let people use it with a minimal barrier to entry. Right now, Yahoo! is like a giant skyscraper tenented only by iPhone case kiosks.

<comment><USER4> The problem is that Yahoo's customers aren't their users. Yahoo's users are the veal calves. They don't seem to understand what to do with businesses where the customers _are_ the users. Flickr, for example.

</comment></comment><comment><USER20> Issue aside, I'm curious about this site itself. The first thing I noticed is that they're running a pretty old version of WordPress. They're on 3.0.3 (December 2010) when the latest version is 3.4.2. For security purposes, I'm surprised they don't stay on top of that.

Also I was really confused if this was an official Yahoo site. No real mention anywhere on it. After some quick digging, it appears to be. But I'm surprised it's not hosted under the Yahoo.com domain somewhere.

</comment><comment><USER21> The only reason Microsoft is making DNT the default is because it will directly impact Google's bottom line (and Microsoft loses money in its online division, so they won't hurt as much). Since when did Microsoft really start caring about the users?

</comment><comment><USER22> _I left this as a comment on their blog which I assume will never be approved:_

      "We fundamentally believe that the online
      experience is better when it is personalized"

Um, doing so is not impeded by DNT as that does not relate to ads. That a bit of a white-lie to imply that it is they way you've worded that first paragraph.

      "It basically means that the DNT signal
      from IE10 doesn't express user intent."

Actually I think it does - you think your average person on the street wants targeted ads? Seriously - who is writing this.

      "In principle, we support "Do Not Track" (DNT)"

In principle China, Syria, Iran etc support Human Rights...

      "Ultimately, we believe that DNT must map to
      user intent — not to the intent of one
      browser creator, plug-in writer, or third-
      party software service."

Again - seriously - what reality are you apart of?

I had hoped for a Yahoo turn-around of sorts, I really did. You've lost me.

<comment><USER23> Both side are playing an Orwellian game, pretending they are concerned with users intentions & desires when in reality both are assuming that users have no intents and desires coherent enough to act on.

They are fighting over the default because they know that in 95% of cases, the default is all that matters.

</comment></comment><comment><USER24> This is surprising, given that Microsoft is the provider of ads on Yahoo Search (and perhaps other properties).

</comment><comment><USER25> It's obvious that Microsoft doesn't have as much to lose with a DNT default setting in IE10 when their profit center is tied to Windows and business software. They have MSN.com and Bing but both are money losers and at this point, hurting Google and Yahoo might be a better strategy even if it means "cutting off your nose to smite your face". They also hope that this will stem the defects from IE and help sell more Windows 8 upgrades and Surface tablets in general.

In regards to how people feel about tracking, people are always asked in isolation about tracking and of course everyone say's they hate it and they don't want to be tracked. The better question is asking them if they will accept the alternative. Scared of Facebook? Put a $5 fee per month to replace lost ad revenues and users will depart en masse. The reality is a big chunk of the internet and the services people rely on, love, and use daily are ad supported and if given the alternative of a free ad supported model vs. a paid model, they would choose ad supported.

</comment><comment><USER26> To be honest, i actually don't care if yahoo tracks me on yahoo properties - in fact, i _expect_ them to. What i DO NOT want is for them to track me across the entire internet through injected javascript, iframes and dedicated tracking domains that serve same-origin analytic scripts from hundreds of sites - that is unethical.

Currently using adblock plus, noscript and ghostery on my FF setup with specific additional controls in ABE for Twitter and FB domains.

<comment><USER27> it's also a very useful feature of the internet, that browser's like safari 6 have broken. there are other uses to third party cookies besides tracking - a/b testing. many web properties are shared between multiple top level domains. ecommerce sites will do this for example to sell different product lines, but they might want to have similar user preferences shared between the top level domains... you can't do that if you kill third party cookies... or block the same features that we'd use in ad networks to track you... IE10 isn't so bad it's just saying don't do this... so - we'll happily ignore it. safari 6 is the real story that hasn't gotten enough attention in comparison... thankful it's not as widely used or is it... iphone,ipad...

</comment></comment><comment><USER28> Am I mistaken or did the FTC not just fine Google $22.5 million over the exact same behavior for a considerably smaller share of web users.

If I'm Microsoft I am making a very public appeal to the FTC over this Monday morning. $22.5 million is a lot more money to Y! than Google. And as wary as consumers are of Microsoft long term they've got to degrade the Google/Firefox brand to start gaining any traction. Why not go full bore on tracking/creepiness?

<comment><USER29> The FTC reached a settlement with Google this year for a $22 mil fine because Google violated a previous consent order over Buzz.

Yahoo is not under an existing consent order with the FTC. For the most part, the FTC does not get fines unless a company violates an existing consent order.

Furthermore, Google didn't get fined because it circumvented Safari's privacy settings, but rather, because it lied about the extent to which users could opt out of tracking.

I've not seen anything that suggests that Yahoo is lying about the extent to which they will respect or ignore the DNT header.

The FTC's deception powers aren't going to be of much use against Yahoo here.

(Disclosure: I worked at the FTC between 2009-2010, and worked on the investigations of Facebook, MySpace and Twitter).

</comment></comment><comment><USER30> No comments on the post? Hmm... I'm curious what they are getting but haven't approved.</comment>


The sound of electricity - <AUTHOR> http://www.noisemademedoit.com/the-sound-of-electricity/

<comment><USER2> Thats fantastic - finaly abstract art I can understand and apprecieate.</comment>


CloudFlare's ScrapeShield spots single line of text copy/pasted from my blog - <AUTHOR> http://blog.jgc.org/2012/08/cloudflares-scrapeshield-spots-single.html

<comment><USER2> I didn't know about the wheel morse code - that's pretty awesome. I'd loved some technical detail about what exactly the beacons are? Javascript, images, special characters...? The CloudFlare page isn't exactly informative in that regard.</comment>


Not in Our Genes - <AUTHOR> http://www.bryanappleyard.com/not-in-our-genes/

<comment><USER2> This article reaches a mostly correct conclusion but not from the strongest line of reasoning. I am blessed with the opportunity to attend the weekly behavioral genetics seminar ("journal club") at my alma mater university each week during the school year. That journal club includes many of the pioneers of the Minnesota Twin Family Study. A new book about the research conducted here has just been published by Harvard University Press

<http://www.hup.harvard.edu/catalog.php?isbn=9780674055469>

so readers all over the English-speaking world will get a lot of the inside story of what has been discovered in research on monozygotic ("identical") twins reared apart. In the local journal club I have learned about professional articles that dispel common misconceptions about human behavioral genetics.

Turkheimer, E. (2012). Genome wide association studies of behavior are social science. In K. S. Plaisance & T.A.C. Reydon (Eds.) Philosophy of Behavioral Biology (pp. 43-64). New York, NY: Springer.

[http://people.virginia.edu/~ent3c/papers2/Turkheimer%20GWAS%...](http://people.virginia.edu/~ent3c/papers2/Turkheimer%20GWAS%20EWAS%20Final.pdf)

"If the history of empirical psychology has taught researchers anything, it is that correlations between causally distant variables cannot be counted on to lead to coherent etiological models."

Johnson, W., Turkheimer, E., Gottesman, I. I., & Bouchard, T. J. (2009). Beyond heritability: Twin studies in behavioral research. Current Directions in Psychological Science, 18, 217-220. [I am personally acquainted with three of the four co-authors of this paper, one of whom regularly exchanges links with me by email.]

[http://people.virginia.edu/~ent3c/papers2/Articles%20for%20O...](http://people.virginia.edu/~ent3c/papers2/Articles%20for%20Online%20CV/Johnson%20\(2009\).pdf)

"Moreover, even highly heritable traits can be strongly manipulated by the environment, so heritability has little if anything to do with controllability. For example, height is on the order of 90% heritable, yet North and South Koreans, who come from the same genetic background, presently differ in average height by a full 6 inches (Pak, 2004; Schwekendiek, 2008)."

Turkheimer, E. (2008, Spring). A better way to use twins for developmental research. LIFE Newsletter, 2, 1-5.

[http://people.virginia.edu/~ent3c/papers2/Articles%20for%20O...](http://people.virginia.edu/~ent3c/papers2/Articles%20for%20Online%20CV/Turkheimer%20\(2008\).pdf)

"Unfortunately, that fundamental intuition is wrong. Heritability isn't an index of how genetic a trait is. A great deal of time has been wasted in the effort of measuring the heritability of traits in the false expectation that somehow the genetic nature of psychological phenomena would be revealed. There are many reasons for making this strong statement, but the most important of them harkens back to the description of heritability as an effect size."

To sum up, the submitted article is correct insofar as it asserts that genes are not destiny in human behavior. Monozygotic twins are from time to time quite discordant even in characteristics that are deemed "highly heritable" with reference to the experience of other twin pairs. The shuffle of genes that you or I receive from parents at the beginning of life does not determine our behavior, but simply establishes a biological substrate for the "reaction surface" that each of us present to our individual environments. Other factors that are properly called "environmental," both at the societal and individual level of analysis, can and do overcome the odds influenced by individual genomes.

</comment><comment><USER3> If you don't have a background in this area and are interested in learning a little more about epigenetics, there is a good NOVA program on it: <http://www.pbs.org/wgbh/nova/genes/></comment>


Successful machine learning models: lessons learned at Booking.com - <AUTHOR> https://blog.acolyer.org/2019/10/07/150-successful-machine-learning-models/ <comment><USER2> "Content Overload: Accommodations have very rich content, e.g.descriptions, pictures, reviews and ratings."

Laughed at that one. Booking.com is so full of dark patterns that I dread using it.

<comment><USER3> What do you mean by "dark patterns"? I'm not familiar with that term

<comment><USER4> Deceiving, tricking and pressuring users into taking actions.

For example, LinkedIn having a flow that has an e-mail and password box, which will get a less attentive user to just re-enter their LinkedIn credentials. But it's actually a phishing form for your e-mail, so if your LinkedIn and e-mail password is the same, you have now "consented" to have your address book scraped and your contacts spammed.

Or, in the case of Booking.com:

* Every step has items designed to pressure you to book NOW because it'll be too late otherwise:

\- "booked x times in the last x hours" on the listing, or

\- "Only 1 room left!" (they now add "on our site" after they lost a consumer protection lawsuit)

\- Showing booked-out listings "You missed it"

\- Various notifications like "last booked X minutes ago" and "limited supply" popping in while you're scrolling to raise the pressure

* Misleading or deceptive claims

\- "Jackpot, this is the cheapest price you've seen" (emphasis should be on "you've seen", this will be shown even if you look at overpriced properties)

\- They seem to have stopped the "one person looking at this property" thing (to make you think that it may be gone if you don't book now - that one person is you), probably after being forced to do so by court

\- a misleading rating system (the lowest possible rating is 2.5/10, and you rate category-by-category, which means that if the staff is friendly and the hotel is in a good location etc. but the rats and cockroaches ate your luggage while you slept, that's an 8/10 property - in practice, you should assume that anything below 8 is not good, below 7.5 is bad, below 7 is catastrophic, below 6 you may not survive)

\- I'd also assume that they mess with the reviews in various ways, like showing mostly positive ones etc., but I haven't verified that.

Overall, I like to compare the booking experience with a drill sergeant yelling into your ear to convert (book) right now, NOW, DO IT, NOW, YOU MAGGOT! They seem to have improved significantly over previous experiences with them, probably due to a combination of me getting used to ignore the yelling, or because they realized that such a bad experience pushes customers away, or because their practices got banned one by one.

It's a shame, because other than the drill sergeant, their site is great.

<comment><USER5> For a while, somebody (not me) in the infrastructure department was maintaining a greasemonkey (I know) script that would remove the urgency messaging elements from the site. They used it both for themselves and to make a point about how much more pleasant the site was without them.

</comment></comment></comment></comment><comment><USER6> Experimental design is just a t-test? At least accord to that picture it seems that way. There are no ANOVA or interaction test?

Do websites usually just use t-test only? Like adding one feature at a time?

<comment><USER7> It's even worst than that. Most of the time the validity of the t-test that they are running is questionable. They are technically running an online t-test and as soon as they find significance they stop. This is fundamentally wrong, and not conclusive at all.

<comment><USER5> A few years ago when I was still working there, involved in the experimentation tooling among other things, we largely excised that particular behavior. What are you basing your assertion on?

</comment></comment></comment><comment><USER8> As an aside,

> developing an organisational capability to design, build, and deploy > successful machine learned models in user-facing contexts is, in my opinion, > as fundamental to an organisation's competitiveness

You hear that, right? In 2019 already you have to have AI and do it well to be competitive. I just wanted to point out how _cyberpunk_ that is.

<comment><USER9> I believe booking.com ran on perl for a very long time. Maybe still does. ~relevant quotes from [https://github.com/globalcitizen/taoup](https://github.com/globalcitizen/taoup) ...

 _In #devops is turtle all way down but at bottom is perl script._ \- @devops_borat

 _Comedy: You, trying to launch a startup from scratch using Java. Tragedy: Me, trying to debug 27k lines of legacy Perl that brings $113MM /yr_ \- @NeckbeardHacker

<comment><USER10> It still does. Booking and ZipRecruiter are easily the two largest employers of Perl programmers.

<comment><USER11> Booking has been around forever but I thought the later was a newish company? Did they consciously pick Perl in 2010 when they launched? I guess that makes sense if that's the founders' background and their business model is practically (crawling and) extracting and reporting job postings from all over the web.

</comment></comment></comment></comment><comment><USER12> > Model performance is not the same as business performance

This is interesting. Sometimes some people from business side consider that AI is the solution to all problems (as if there was just one catch-them-all AI solution) and some academic people think that the top-performance model for some classification task is the must-go, and all they forget that the goal is to earn money.

<comment><USER13> That was an interesting result from the original Netflix challenge.

First of all it turned out that the winner wasn't actually all that useful for various reasons such as computational intensity.

But, more interestingly, it also turned out that the goals of the model--"best" recommendations--isn't actually the goal of Netflix at all which is much more interested in customer retention and similar metrics. The two things may be correlated but they're certainly not the same thing.

I don't remember all the details but I thought it was a really good insight at the time.

<comment><AUTHOR> The big problem for Netflix was that their data was all from DVD rentals, but by the end of the contest, their business model was very streaming oriented. As you might imagine, people have beliefs about what they'll want to watch in a few days that don't exactly match up with what they want to watch now. That difference killed the model.

<comment><USER14> another problem was that their "user" was actually a household, with a range of (sometimes conflicting) likes and interests - which could result in strange recommendations. They finally added profiles to fix that data problem.

</comment></comment></comment></comment><comment><USER15> Nice to see that I am not only one hating booking.com with a passion.

But what really amazes me is the market failure that hotels and other accomodation providers can't come up with a co-op booking site. I am sure there are issues that are difficult to solve from competition point of view, but are they _really_ so difficult to solve that the rent seeking fees of current booking sites are justified?

<comment><USER16> Yes it is. You are maybe not considering the challenge of putting all those accommodations across the globe in one place. Accommodation providers don't care about how they fill up their rooms as long as they get filled. Booking, contrary to many other tech companies, is very successful financially with 14.5b of revenues and something like 25+% EBIT. I find hard to believe that a coop is going to be able to build such service worldwide.

</comment></comment><comment><USER17> >> _Booking.com go to some lengths to minimise the latency introduced by models, including horizontally scaled distributed copies of models, a in-house developed custom linear prediction engine, favouring models with fewer parameters, batching requests, and pre-computation and /or caching._

Any idea what these are ? especially the pre-computation/caching and batching. I'm not able to see what advantage does batching bring...or how you can really cache a prediction request

<comment><USER18> Here is an overly simple example:

Pre-compute the recommended hotels for my top users every night. Now when that user comes back, they see a slightly stale recommendation, but it's lightning fast.

You can also pre-compute and cache some of the inputs cheer model, like maybe a vector representation of the description of a hotel.

</comment></comment><comment><USER19> > Once deployed, beyond the immediate business benefit they often go on to > become a foundation for further product development.

This is one of the reasons I am a big believer in having a system to track model research and deployment lineage. (I personally use Domino Data Lab for this. I also work for Domino, but use it in my own modeling work and that of others I mentor.) No matter which system you use to track lineage, I've found it important to have a strict history of retraining, versioning, and experimentation. When models are used in downstream systems from the one they were originally intended, it becomes even more critical to able to explain and reproduce the 'research' that led up to deployment.

</comment><comment><USER20> I'm glad they highlighted inference latency. This is a big issue that I've started running into at scale.

<comment><USER21> Coming from the math side, I don't really get this. Isn't all of the latency introduced by the learning side of things? Shouldn't the answer side be entirely decoupled from the learning, and simply be plugging data into an equation with a bunch of constant parameters (with values discovered by your learning system, updated at a less than realtime frequency)?

<comment><USER22> The "equation with a bunch of constant parameters" generated by an ML model can be huge, with thousands of inputs or more. Evaluating that equation for a specific observation can require a huge number of computations, which is why there's a boom in ML inference hardware right now.

<comment><USER21> Yeah, I don't buy that. It's still just a matrix multiplication (for the linear bits). Incredibly fast. Besides, the old physics rule of thumb is that any real world equation with a bunch of parameters only has 5-7 that actually matter, and only 3 that matter a lot. Everything else can be set to zero without noticeable change in the result.

<comment><USER20> If you're making decisions that involve multiple variables you may be doing hundreds to thousands of inferences for a single page load. Keeping latency under 50ms becomes a real challenge.

<comment><USER21> But it comes down to this doesn't it:

x1 * a + x2 * b + x3 * c + ... + x1000 * zzz + ...

If a, b, c ... zzz, are all fixed constants already discovered by your learning algorithm. That's a _very_ fast calculation, and doesn't take anything like 50ms.

Also, in the real world, you can establish a significance cutoff for a lot of these constants and get something like this as your final equation:

x13 * m + x523 * cdf + x777 * wdc + x893 * ydz

<comment><USER23> The inputs to those functions might be coming from external data sources, or aggregated. These have a cost too. But mostly, it just adds up. At a _thousand_ features, you have a 0.05ms budget for each. Without taking into account network latency since you won't be running those models inside the application server.

<comment><USER21> So why not load the calculated constants to the application server to reduce network latency?

And the learning side of things should have culled that list of thousand features down to a list of 5 - 10 that mattered.

It really sounds like the off-the-shelf stuff isn't built for efficiency.

<comment><USER24> Nobody's saying this is an impossible problem. The paper shows how much additional work is required beyond a traditional data science workflow.

The team behind the paper built a model that had good performance on training data. They're a smart lot so they knew they needed to cross-validate. The results held up in cross-validation! Hooray, the model works! ...right?

That's as far as a lot of data scientists go. This paper points out that you need to have a model that does (at least) three things: 1\. Generates good scores with training and testing data 2\. Outperforms existing models in the real world 3\. Runs really really quickly There are a lot of data scientists who have no idea how to do #2 and #3. This paper says "These parts are really important!!!"

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> Any other good resources on production level deep learning practices?</comment>