What the Bagel Man Saw - An Accidental Glimpse at Human Nature - <AUTHOR> http://www.stephenjdubner.com/journalism/bagelman.html <comment><USER2> "Paul F. has noted a strong correlation between high payment rates and an office where people seem to like their boss and their work."

I wonder if this could be a way for companies to figure out (surreptitiously) how managers are doing: provide such a bagel service to each department, and see where the theft rate is the highest. Of course, it has to be done completely incognito, otherwise the managers would make sure that the cash box is stuff full of cash....

<comment><USER3> Given that he also noticed a correlation with weather, overall economic conditions, global events and myriad other factors, you'd need to work very very hard to isolate only what was down to the manager.

Plus most people respond primarily to their direct manager and most direct managers have relatively few reports (typically less than 20). Statistically that's going to make it hard to find significant findings given the variation that is normally seen.

Or think of a team which, for no reason of the manager's making had one inherently dishonest employee who steals all the time. That manager is on -5% before he does a thing - a big disadvantage given the variations being discussed.

EDIT: Not to mention the fact that this is one of his "intuitive conclusions" - which I'm reading as he has no solid data to support the level of liking / not liking their boss, or possibly whether it was even true.

<comment><USER4> If you provide the service to a few teams/companies, you can rule out.the noise

<comment><USER3> Given that he found a correlation between company size and industry you'd need to find a few companies that matched in those regards too as well as geography (weather) and other things.

Oh, and this wasn't something he had any data to support, it was just something he felt was probably true.

Speaking as a manager, I'd like not to be fired on the hunch of a bagel salesman / economist if that's OK.

<comment><USER4> He could use a dozen managers from the same company.

Also managers who doesn't get lame math jokes makes people steal more bagels. Beware.

</comment></comment></comment></comment></comment><comment><USER5> This was a very interesting article. I always find experiments involving honor systems very fascinating. The bus system in Germany and some parts of the Boston T (the trolleys when they are overground) are basically honor systems. In Bonn, Germany people are very particular about paying the fare. In Boston, not so much.

<comment><USER6> Rome has(had?) an honour system for public transport too. We didn't realise that no one ever pays unless they see inspectors getting on the bus. Lots of people would leap up and punch their biglietti when we punched ours because they assumed we had seen some inspectors. I couldn't work out what was going on until I read about it in the Time Out guide which explained that almost no one pays :-)

London had exactly the same system and the same fare evasion problem on the high traffic bendy bus routes. These routes were always overcrowded probably because of the number of unrecorded passengers using them. They withdrew these buses and replaced them with ones which take longer to board partly because of the fare evasion and partly because a sort of bizarre bus xenophobia (hatred of a foreign european style of bus) inspired by the Evening Standard which resulted in an election promise by mayor Boris to get rid of them.

<comment><USER7> I can't speak for their motivation, but the what the Evening Standard (and Metro et al) printed was not related to the origins of the bus. They claimed they were dangerous to other road users. TfL disputes this.

Solving the overcrowding issue is more complicated than building higher capacity buses - you have to think about the speed of boarding, route layout, speed of navigating London's congested streets, fare evasion and more. In some places, the bus stops allow for two bendy buses or 3-4 regular buses to board at once, which means faster boarding on regular buses during peak times.

The new Routemasters, with 3 doors and 2 staircases, might prove to reduce overcrowding despite their lower capacity.

<comment><USER6> There was plenty of research behind introducing bendy buses. The main argument being that they transit the route faster overall because they have a much shorter dwell time at stops. The introduction of bendy buses was a rational evidence based decision, the campaign against them was mostly emotional hubris, which resulted in them being replaced early at vast expense to the taxpayer. The main reasonable complaint against them was that you were more likely to have to stand up, but you often couldn't even get on the bus (e.g's routes 73,38) before they were introduced so I don't know what's worse.

</comment></comment></comment></comment><comment><USER8> > could any man resist the temptation of evil if he knew his acts could not be > witnessed? ... Paul F. ... knows that the answer, at least 89 percent of the > time, is yes.

Coincides remarkably with what a former fraud investigator once told me, that 11% of the population is sociopaths.

</comment><comment><USER9> I wonder how the payment rates would change if he simply mounted a non- functional wireless camera plugged into a socket pointing at the bagel location?

(ie, hinting to people that they are _not_ invisible as in the story of the Ring of Gyges)

<comment><USER10> _I wonder how the payment rates would change if he simply mounted a ... camera_

An interesting question. I wouldn't be surprised if the camera made it worse. My guess is that people, when trusted, hold themselves to a higher standard. But with a camera present, they're allowed to turn off their own consciences. So people would be ingenious about palming a bagel so the camera wouldn't see, and think that if the camera's not seeing it, they're home free.

<comment><USER11> This has actually been studied:

[http://www.staff.ncl.ac.uk/melissa.bateson/Bateson_etal_2006...](http://www.staff.ncl.ac.uk/melissa.bateson/Bateson_etal_2006.pdf)

> We examined the effect of an image of a pair of eyes on contributions to an > honesty box used to collect money for drinks in a university coffee room. > People paid nearly three times as much for their drinks when eyes were > displayed rather than a control image. This finding provides the first > evidence from a naturalistic setting of the importance of cues of being > watched, and hence reputational concerns, on human cooperative behaviour.

<comment><USER12> This is even more amazing, because the people _know_ they're not being watched. I wonder how ingrained the image of eyes is? Could you achieve the same effect with a picture of a CCTV camera?

<comment><USER13> that is a really interesting idea. With a fake camera people can see what they are supposed to avoid. With a picture of a camera, where they are unclear exactly where the camera is, I wonder how that changes things.

</comment></comment></comment></comment></comment><comment><USER14> Execs and higher paid people are less likely to carry cash on them as lower paid people. I wonder if this make theft, not making it any more ethical, but more likely to occur in moments of weakness. And what about the Whimpy effect of 'gladly pay you Tuesday' for bagel today. Are there make up payments being made?

</comment><comment><USER15> I've read this story before and wondered then if he would have made a cardboard cutout of himself smiling with outstretched hands and put it behind the bagels (so it looked like he was offering them) if his payment rate would have gone up.

</comment><comment><USER16> Nice story, but how do we know it's real? There's no data and all the names have been changed.

</comment><comment><USER17> What makes me sad in reading this article is that the man set out to respect people's good nature with his honor system, but (at least in the telling) became a bit of a nag, obsessed with measuring and haranguing cheats.

</comment><comment><USER18> I wonder if the theft rate is differs between donuts and bagels. Are people who choose healthier food more honest?

<comment><USER19> I think it would surprise you which has more calories, a plain bagel or a glazed yeast donut.

<comment><USER20> about the same number of calories, (224 calories in a donut, 207 in a bagel) but those calories come from sugar and fat in the donut (4 times the sugar and 12 times the amount of fat).

Sources:

<http://www.wolframalpha.com/input/?i=calories+in+a+donut> <http://www.wolframalpha.com/input/?i=calories+in+a+bagel>

<comment><USER19> That's a rather small bagel (75g). The issue here is mass. Bagels are _much_ denser than donuts. Calorie Count has 285 calories in a medium, plain toasted bagel (100g) and 239 calories in a 60g medium glaze yeast donut.

[http://caloriecount.about.com/calories-bagels- toasted-i18002...](http://caloriecount.about.com/calories-bagels- toasted-i18002?size_grams=99.0)

</comment></comment></comment></comment><comment><USER21> Wow, I read this article and immediately thought: this is the future of digital media economies.

It's obvious that in the long run, it will be impossible to force people to pay for digital media. The only alternative is coercive, expensive, leak-prone control of the internet by governments, and I just don't see that as feasible in the long-term.

So, asking people to please be honest is basically the only alternative.

Look at Louis CK's experiment, or the Humble Indie Bundle, or Radiohead's In Rainbows, all those years ago. What were they doing? They were leaving out a bunch of bagels, and a box with a slot in it, and saying "Hey, we're not going to watch you, but we'd really appreciate it if you paid for your bagel."

Some further thoughts:

1) With digital bagel selling, the bagels are infinite, and the potential office size is the entire world.

2) With digital bagel selling, a recommended price can be asked for the bagel, but customers are encouraged to pay as much as they feel willing to give.

3) Similarly, digital bagel sellers such as Louis CK have used guilt-tripping to attempt to raise payment rates. Saying "I worked hard on this, I'm not asking for that much, I would really appreciate it if you supported me" - making an appeal to people's sense of fairness and generosity - must increase payment rates (/decrease theft)

4) What other factors might influence payment rates in digital selling, and might it be possible to account for these factors - to adjust selling techniques depending on the "digital weather"? For instance, given enough data, could you correlate payment rates to internet activity - maybe someone who looks at lots of pictures of Lolcats is more likely to pay - or someone who has read lots of depressing news articles is less likely to pay - or maybe someone literally browsing in a place where the weather is shit is less likely to pay? Could you tweak it so that someone is most likely to see an ad/link for your digital bagels when the digital weather is optimal?

5) By always starting selling as digital bagels, sellers can prevent the worst excesses of the torrent cycle. If you try and lock digital bagels into the traditional payment model, they will almost certainly instantly appear in torrents, at which point, all potential buyers will be faced with a choice - buy at the set price from the standard outlet or sneak around to the backdoor torrent outlet. By setting up a digital bagel shop, you can let those who would not pay anyway do so, immediately, making it less likely for anyone to set up a backdoor torrent. Why torrent a digital bagel when it can be downloaded for free, in a pristine copy, from the official outlet? Preventing the runaway torrent cycle means that all customers, payers and non-payers, will be exposed to the optimum conditions for inducing payment - resulting in the highest potential returns.

6) How do we calculate a reasonable return for digital bagel selling? Beyond paying for hosting and bandwidth, there is no cost for selling/giving away 10,000 digital bagels vs 100,000 digital bagels. How much should creators of media be paid - how do we calculate how much they deserve to be paid?

Much of this I am sure is what the people at Humble Indie Bundle spend a lot of time thinking about. But it is clear to me that the SOPA-path is not a tenable future for the internet (my native New Zealand has unfortunately already gone down this path). Worldwide free or next-to-free circulation of cultural goods is a net good for society. More people reading more, watching more, thinking more, means potentially more people more educated and aware than ever before - and people creating more fantastic things than ever before.

But we are being dragged away from this path by those who reaped fat rewards from the old system, when commodity sold was linked to cost of production, and physical formats meant IP enforcement was still feasible. The stalwarts of the old system are not willing to consider the reformulations of basic economic strategy necessary to take advantage of the new situation.

Hopefully we end up with a free internet, where everyone sells their own digital bagels, and people are on the whole honest and generous! Fingers crossed.

</comment><comment><USER22> I remember reading this in Freakonomics but with a bit more embellishment. I remember that when the payment rates started to sag, he would put a picture of eyes by the money box, and that often helped the rate rise again.

A bored office worker could do a bit of his own experimentation in this vein -- how many donuts do people buy when there's a fake rubber cockroach near the box? A picture of a severely obese person, or a very fit person, or a salad?

</comment><comment><USER22> I noticed there is quite a bit of speeding in this article which is concerned with small crimes. Presumably that's OK because there isn't victim involved.

<comment><USER23> Speeding isn't a criminal offense. It's an infraction of the vehicle code.

<comment><USER24> Whether speeding is criminal depends on where you are, and the speed you are travelling. One might make a similar argument that an 'honor box' is a 'suggested donation', and that one is simply choosing not to donate. You could probably even argue that it _must_ be considered a voluntary donation as evidenced by the failure to collect the required sales tax.

I think 'funkah' has a point. There are lots of things happening in this article, and it's fair to look at the situation from a different perspective than the primary. As a California business owner, I didn't notice the speeding, but did wonder about whether or not DC required him to have health and vending permits for each of his locations, and whether the legality of his business should affect our conclusions: is it more or less ethically acceptable to 'steal' a donut from an 'illegal' business?

<comment><USER23> But if you consider the honor box a suggested donation box, is that a business, still?

The sales tax is likely included in the dollar amount. Like when you are quoted 3.99/gallon of fuel. That price includes a number of taxes built in.

We (well, specifically, I) don't know if DC requires permits for re-selling food-stuff. We do not know if he is on the level, were they to require such. It probably depends on how much revenue is produced by each location. Just as sidewalk Lemonade stands don't require permits, in most locales. Anyhow, all that is speculative, unless you have facts.

Ethics. I think you are getting into relative ethics. If someone else goes against ethics, does it grant me a waiver to act unethically?

<comment><USER24> _if you consider the honor box a suggested donation box, is that a business_

A fine question, and an ongoing local issue near me: <http://online.wsj.com/article/SB123422026431565295.html> A local Thai temple has a mandatory voluntary donation scheme that they use to skirt a lot of laws. I love their food, but am against their sham.

 _The sales tax is likely included in the dollar amount._

Could be, but I'd bet that he's technically non-compliant. The rules are mighty complex. Our next-door neighbor restaurant was recently fined heavily for considering their sandwich purchases "take out" while providing unofficial side-walk seating.

 _We (well, specifically, I) don't know if DC requires permits for re-selling food-stuff._

If we are to consider it a "sale", then I think he would require a Vending Machine License: [http://dc.gov/DC/DCRA/For+Business/Apply+for+a+Business+Lice...](http://dc.gov/DC/DCRA/For+Business/Apply+for+a+Business+License/Retail+Food+License+Information/Get+a+Food+Vending+Machine+License)

 _It probably depends on how much revenue is produced by each location._

I deal with many such laws in California, and have not found many such exclusions. Rather, like speeding, the behaviour is technically illegal but regulated with only selective enforcement. I don't like selective enforcement.

 _Just as sidewalk Lemonade stands don't require permits, in most locales._

I think this is again mostly selective enforcement. It's been a big issue lately: [http://www.forbes.com/sites/erikkain/2011/08/03/the- inexplic...](http://www.forbes.com/sites/erikkain/2011/08/03/the-inexplicable- war-on-lemonade-stands/)

 _Anyhow, all that is speculative, unless you have facts._

True. I can really only speak only to my experience running a food business in California, where my opinion is that it is impossible to run a business without breaking numerous laws. Heck, I just learned last month that all of my employees are working illegally: [http://sanfrancisco.grubstreet.com/2011/06/so_has_anyone_in_...](http://sanfrancisco.grubstreet.com/2011/06/so_has_anyone_in_california_he.html)

 _If someone else goes against ethics, does it grant me a waiver to act unethically?_

This is indeed the real question. It certainly doesn't grant an unconditional waiver, but I think there can be mitigating factors. Would it matter that the Bagel Guy has the permission of the business (he does)? What if he was representing it as a charity: [http://bulkvending.blogspot.com/2008/04/making- money-from-ch...](http://bulkvending.blogspot.com/2008/04/making-money-from- charity-boxes.html)

I think the question is ethics. While the article was entertaining, it didn't delve deeply enough into the situation to be informative on ethics. I post to HN hoping that discussion might be able to happen here...</comment></comment></comment></comment></comment>


John Skipper Details His ESPN Exit and a Cocaine Extortion Plot - <AUTHOR> https://www.hollywoodreporter.com/features/john-skipper-details-espn-exit-cocaine-extortion-plot-1094657 <comment><AUTHOR> The interview is far less sensational, and far more personal and insightful than the title implies.</comment>


Timeline for Logic, λ-Calculus, and Programming Language Theory (2012) [pdf] - <AUTHOR> http://fm.csl.sri.com/SSFT15/Timeline.pages.pdf <comment><USER2> To those interested in the history of logic and computation, and the relationship between them (and algebra) -- which, BTW, was not at all accidental or surprising but completely by design -- I've compiled an anthology of mostly primary sources starting with Aristotle and going up to the 1950s: [https://pron.github.io/computation-logic- algebra](https://pron.github.io/computation-logic-algebra)

<comment><USER3> Over the course of a few weeks, I've been going through your "History of Computation, Logic and Algebra". I just wanted to say, brilliant work, thank you! The way you present the history is really insightful in understanding the current of philosophy and ideas that led up to our modern conception and use of computation.

</comment></comment><comment><USER4> I've taken some graduate courses in programming languages where I've learned about the lambda calculus and about type theory, and I really appreciate this post since this provides a nice timeline of how programming language theory has evolved. One topic I'm very interested in is how computation can be expressed in many different ways and the tradeoffs behind various expressions.

I apologize if this is off-topic, but I wonder if there has been work done on algorithm analysis using the lambda calculus? The texts that I've read on algorithms describe the algorithms using procedural code, and then the analysis is done on the amount of procedures that get called. However, I would imagine that the description would be different for non-procedural styles of programming. I've heard of a book called "Functional Data Structures" but I haven't read it yet. I'm wondering if there has been work done on algorithm analysis at the lambda calculus level.

<comment><USER5> (On my phone walking so very quick and dirty post) purely functional DSs in general will cost you an extra O(log n) cause of their underlying tree representation for immutability (immutability achieved by copying paths in the tree to retain old data). Also some DSs are very difficult to express in a purely functional way.

<comment><USER6> While true on the first part, these days many cases don't have insane primary efficiency needs that would be affected by that log(n). As always though, you choose the tools based on the job at hand, not the tools. In my experience, functional philosophy often is about the gained mental and consistency advantages achieved by the features offered.

<comment><USER5> Don't get me wrong, you are preaching to the choir (I write Haskell at work), just thought it's worth mentioning this to the OP. In practice sophisticated compilers like GHC produce insanely fast binaries and is getting better and better.

</comment></comment></comment></comment><comment><USER7> Philip Wadler showcases and comments on some of this history in his conference talk "Propositions as Types" [https://www.youtube.com/watch?v=IOiZatlZtGU](https://www.youtube.com/watch?v=IOiZatlZtGU)

<comment><USER8> This is talk is responsible for my outrage whenever people say math is invented.

</comment></comment><comment><USER9> Homotopy Type Theory doesn't really contribute much to logic (it's basically just type theory with a few extra rules) or programming languages (computer- assisted proofs are easier in HoTT, but that's about it).

HoTT is a foundations of mathematics thing (think ZF/ZFC), so it's a bit weird to see it on the list. But it's kind of hot right now, so you see it just about everywhere.

<comment><AUTHOR> It connects logic, topology and type systems. How is this not a big deal? It's also computation friendly.

<comment><USER9> Logic and type systems have been connected since Frege (he coined "functional" and "non-functional" types[1]). Computation friendly is interesting, but not really revolutionary. Topology I'm not an expert in so I can't comment.

[1] On Function and Concept, 1891; [http://fitelson.org/proseminar/frege_fac.pdf](http://fitelson.org/proseminar/frege_fac.pdf)

</comment></comment></comment><comment><USER10> It is missing Plankalkül by Zuse [https://en.wikipedia.org/wiki/Plankalk%C3%BCl](https://en.wikipedia.org/wiki/Plankalk%C3%BCl) which heavily influenced ALGOL

</comment><comment><USER11> This seems like a good a thread as any to ask my perennial question:

Is there some sort of logic or formalism that would let me algebraically represent a program like bubble sort, and then algebraically simplify it to a quick sort?

</comment><comment><USER12> Any timeline of logic that fails to include Charles Sanders Peirce[1] is woefully incomplete. He discovered the existential and universal quantifiers and did seminal work on the relational calculus, to name just a few items. If you haven't heard of him I strongly recommend at least reading the article I linked. He's a great American genius who is virtually forgotten.

[1] [https://en.wikipedia.org/wiki/Charles_Sanders_Peirce](https://en.wikipedia.org/wiki/Charles_Sanders_Peirce)

</comment><comment><USER13> Is there any good book that provides an overview and reasoning behind all those concepts in this timeline?

<comment><USER14> Not that I'm aware of.

If you produced a mind-map to show their inter-relatedness, It's quite a dense network.

A quick sketch of some of the nodes and vertices might start to look like:

Logic <\--> Gödel's theorems

Gödel's theorems <\--> Proof theory

Proof Theory <\--> Linear Logic

Computability <\---> Turing Machines

Computability <\---> Lambda Calculus

Combinatory Logic <\--> Lambda Calculus

Type Theory <\--> Lambda Calculus

Homotopy Type Theory <\--> Type Theory

Pi Calculus <\--> Process Calculi/Distributed Systems

Category Theory <\--> Everything...

And that's far from complete. I imagine synthesising all the concepts into a single book, with a cohesive narrative would be quite hard, without some deeper unifying theory, uniting all concepts. (Category theory and HTT may be the best contenders).

In this case, a Wiki might best capture the semantics of structure. Although I'd love to be proved wrong and find such a book!

<comment><USER15> ncatlab.org</comment></comment></comment>


Sources say China used iPhone hacks to target Uyghur Muslims - <AUTHOR> https://techcrunch.com/2019/08/31/china-google-iphone-uyghur/ <comment><USER2> To add a bit of context, China forces Android users in Xinjiang to install surveillance apps:

\- [https://www.vice.com/en_us/article/ne94dg/jingwang-app-no- en...](https://www.vice.com/en_us/article/ne94dg/jingwang-app-no-encryption- china-force-install-urumqi-xinjiang)

\- [https://www.vice.com/en_us/article/7xgame/at-chinese- border-...](https://www.vice.com/en_us/article/7xgame/at-chinese-border- tourists-forced-to-install-a-text-stealing-piece-of-malware)

Such apps wouldn't work on iOS, due to its tighter security. But apparently if you're China that's not much of a deterrent, because you can just buy a rootkit to get around such limitations.

<comment><USER3> supposedly Fuchsia closes that hole

</comment></comment><comment> word-reader So we now know that drive-by rooting of even "locked-down" and "secure" user devices like the iPhone is possible and in active use by state actors. What are the odds that other governments, even the US (which has the additional advantage of being able to gag Apple and Google), are doing this? Seems like it would become a more important way to do surveillance as more data and networks become encrypted.

</comment><comment><USER4> It's telling that Google wasn't willing to name China publicly and that someone (most likely someone at Project Zero) had to leak it to this reporter off-the-record. Maybe they still have some hope of re-entering China?

<comment> word-reader It didn't have to be someone at Project Zero. Anyone with a bunch of network logs could have searched for some of the fingerprints or URLs that were publicly available and deduced the target from associated traffic.

</comment></comment><comment><USER5> This is semi off topic, but why (from a mainstream Chinese perspective) is China targeting Uyghur Muslims? Is there just a general ideology of enforced conformity or is this a special case?

<comment><USER6> Uighur's and Tibetans have historically not had significant ties with mainland China. The Uighurs are Turkic and have closer cultural ties to the likes of Kazakhstan. The Tibetans are more closely tied to India/Nepal.* After the Mao revolution, the Chinese annexed these territories and started a huge bout of propaganda claiming these regions were always part of China. The propaganda has worked on the mainland Chinese, but obviously, the Uighurs and Tibetans don't buy it because they know their own history. There's been some recent unrest in Uighur autonomous region, so China is starting around round of indoctrination and police state tactics.

*I know this is a controversial claim especially for people of Chinese origin. So if you doubt this, a couple of facts worth pointing out are the following. The Dalai Lama's government in exile is headquartered in Dharamshala in Northern India. Loads of Tibetan kings have had Nepali wives. Tibetan is most closely related to languages spoken in Ladakh, Sikkim, Bhutan and Nepal -- two of these are Indian states and the other two are effectively Indian protectorates. Some of the most important "gurus" in Tibetan Buddhism (e.g. Rinpoche aka Padmasambhava) were Indian. The Tibetan script is based on Brahmi which is an ancient Indian script that gave rise to all modern Indian scripts such as Devanagari (used for Hindi/Marathi etc.), Gurmukhi (used for Punjabi), Tamil, Kannada, Bengali, Assamese, etc.

<comment><USER7> Tibet has been close to China, as opposed to India for more than a thousand years.

Sikkim is in India because of the British, not because it is "Indian" (so to speak)... In fact it was in China's sphere of influence until the British took it over.

<comment><USER6> > _Tibet has been close to China, as opposed to India for more than a thousand > years._

That is just laughably false. If you live in the US, I suggest visiting a Tibetan store to disabuse yourself of this communist propaganda. (There are tons of Tibetan stores in Berkeley, CA for instance.) You'll find them selling many items that are instantly recognizable to Indians that Chinese people wouldn't have a clue about. Things like Ganesha figurines, Rudrakhsha malas and kurtis.

An easier option is to try asking an actual Tibetan person. They'll tell you the same thing that I'm saying.

<comment><USER7> I am not looking for political battles here or to spread propaganda from either side. I am not sure that a Tibetan store in Berkeley CA is a neutral and accurate representation of Tibet's history.

The starting point is that geography makes access to the Tibet region much easier from the East and North than from the South. So-called "Greater Tibet" extends East and North East quite beyond the current Tibet Autonomous Region in China.

Perhaps as a consequence, Tibetan are ethnically closer to East Asians than to South Asians (I think that is visually obvious). Their language is not related to Indian languages and neither is traditional clothing.

Historically Tibet has had close contacts with China and was periodically under Chinese vassality/rule since at least the Tang Dynasty in the 7th century.

To claim that Tibet "has historically not had significant ties with China" is simply not accurate.

Likewise, Xinjiang/Uyghurs have had ties with China for even longer because of the silk road, with China establishing a protectorate over the area in the Han Dynasty about 2,000 years ago.

<comment><USER6> > * I am not looking for political battles here or to spread propaganda from > either side. I am not sure that a Tibetan store in Berkeley CA is a neutral > and accurate representation of Tibet's history.*

What do you think is an accurate representation of Tibetan history? I've also been to Dharamshala and spoken to Tibetans in exile. They also echoed similar views, which I'm guess you will now discard as being unrepresentative of Tibetan history.

> _The starting point is that geography makes access to the Tibet region much > easier from the East and North than from the South. So-called "Greater > Tibet" extends East and North East quite beyond the current Tibet Autonomous > Region in China._

Greater Tibet is a made-up concept. It includes places like Arunachal Pradesh where Hindus outnumber Buddhists, and Hindus themselves are outnumbered by Christians. The same width Ladakh. It's just communist propaganda to justify a Chinese land grab.

> _Perhaps as a consequence, Tibetan are ethnically closer to East Asians than > to South Asians (I think that is visually obvious)._

Thank you for your opinion, but I'm not interested in 18th century racist tropes.

> _Their language is not related to Indian languages and neither is > traditional clothing._

I like how you dismiss the fact that their script is directly derived from Brahmi but consider their language very important. For the record though, plenty of languages related to Tibetan are spoken in the north and north east of India.

Also enlighten us on what you think traditional Indian clothing is, and what traditional Tibetan clothing is.

> _Historically Tibet has had close contacts with China and was periodically > under Chinese vassality /rule since at least the Tang Dynasty in the 7th > century._

Yes, the Chinese have repeatedly tried to invade Tibet. If you think that is a claim for Chinese culture being close to Tibetan culture, I don't know what to tell you! Do you also think that French culture is similar to Arab culture because the "Holy" Roman emperors repeatedly tried to invade Jerusalem?</comment></comment></comment></comment></comment></comment>


The Cost of JavaScript in 2018 - <AUTHOR> https://medium.com/@addyosmani/the-cost-of-javascript-in-2018-7d8950fbb5d4 <comment><USER2> I blame frameworks and the over-reliance on them.

People poke fun at StackOverflow over everyone answering every JavaScript question with "Use jQuery/React/$FRAMEWORK_OF_THE_WEEK", but its a real problem.

I have a browser game I've been writing, and I've just been using pure JavaScript with no frameworks. It's currently sitting at 28K for the JS+HTML+CSS, and that's without minification. Even when it's done, it'll be under 100K.

If I were to listen to anyone's suggestions for using a framework, I'd already be at 84K from jQuery alone.

"Oh, you want to do X? Well, that normally takes 4 lines of JavaScript code, but you can include this framework that turns it into a single function call!"

<comment><USER3> Unfortunately the realities of supporting a biz app leaves you with little choice. Sure you can do pure JavaScript, and we continually try. However at some point when that customer running IE9 calls you about another UI issue, you have to admit that you're just reinventing the wheel. jQuery and such frameworks have a place because they abstract a lot of conditional statements to handle different browser quirks away from you. The framework supports certain browsers and for the most part you can expect that the behavior will be relatively consistent across those browsers.

Meanwhile I can't even get consistent event firing order between IE, Edge, and Chrome for key and mouse events on inputs.

</comment></comment><comment><USER4> This article is great. I am honestly surprised that more articles on this topic were not written 2-4 years ago (maybe there were some but I missed them). JavaScript-heavy single page web apps have been around for a while now and JavaScript performance must have been a serious concern for a lot of those applications.

</comment><comment><USER5> "For mobile, aim for a JS budget of < 170KB minified/compressed. Uncompressed this is still ~0.7MB of code."

Almost a meg of code is a target to "aim" for? Please burn the DOM to the ground now, thank you.

</comment><comment><USER6> If js cannot properly run on a smartphone or other limited devices, it should not be used anywhere.

If you don't understand how v8 works, don't use js.</comment>


Show HN: A status dashboard for non-automated systems - <AUTHOR> https://devrig.com <comment><AUTHOR> For the curious, this was built using Firebase on the backend and React on the frontend.</comment>


MIT-related work that has transformed computer science - <AUTHOR> https://www.csail.mit.edu/node/2223 <comment><USER2> I think many of these claims as "MIT's contribution to transform Computer Science" is unjustified. Particularly, wasn't Bob Kahn already out of MIT when Vint Cerf and him created TCP/IP? What was MIT's role here? Just that they paid salary to Bob Kahn in the distant past?

On the other hand, when I think of MIT, the most important contribution to Computer Science that comes to my mind is Seth Gilbert and Nancy Lynch's proof to Brewer's Conjecture, famously called CAP theorem. This proof is so profound, so important to Computer Science and the way we build large-scale computer systems today. And d'uh, it has been left out.

This looks like pure marketing to me. And in some sense misleading.

<comment><USER3> Yeah, and if I remember correctly, RMS's work on GNU was not affiliated with MIT and he did it on top of his actual responsibilities in the AI Lab. It's like saying your high school is a great place to be educated because you became famous down the road.

</comment></comment><comment><USER4> The World Wide Web was INVENTED at CERN in Europe, I thought.

[http://en.wikipedia.org/wiki/History_of_the_World_Wide_Web](http://en.wikipedia.org/wiki/History_of_the_World_Wide_Web)

After that Tim Berners-Lee went to the MIT and founded the W3C.

<comment><USER5> You mean the botched SGML clone sent over RPC clicked together in NextStep?

<comment><USER5> dear downvoter: I am happy to counter each and every argument you may have, in case you should have any.

Otherwise forget the PR and dig deeper than usual.

ps: Actually, this MIT piece is like the CERN web pieces, run by the PR department of said institutions.

Such accomplishments are rarely if ever obtained by a single lab, or institution for that matter. They are rather embodiments of the spirit of the times, the effort of many of a given era implemented with the technology then available (see Otlet). The only question is, who gets to win the (often propaganda) fight for the claim and who gets to write history.

<comment><USER6> You're most likely being downvoted for tone and for lack of any support for your argument. HTML was based on SGMLguid... with one exception.

 _The only radical change was the addition of the all important anchor ( <a>) link, without which the WWW wouldn't have taken off._ [1]

Minor detail.

[1] [http://infomesh.net/html/history/early/](http://infomesh.net/html/history/early/)

<comment><USER5> prewar networked knowledge-base, pre-Bush:

[https://archive.org/details/paulotlet](https://archive.org/details/paulotlet)

or google youtu.be Paul Otlet, visioning a web in 1934

biased incomplete list of link implementations pre HTML:

[https://en.wikipedia.org/wiki/Hypertext#Implementations](https://en.wikipedia.org/wiki/Hypertext#Implementations)

The reason why WWW has taken off is very well described in:

High Stakes, No Prisoners: A Winner's Tale of Greed and Glory in the Internet Wars.

and has very little, if anything to do with the <a> incarnation, which as you say is a minor detail.

</comment></comment></comment></comment></comment><comment><USER6> Most of this work is publicly funded. They whole VC industry is built on it. It should return more of its profits to the public, just as it would to early- stage investors.

<comment><USER7> The industries have returned value to the public in the form of taxes, jobs, and improved quality of life. It's already a two way street.

<comment><USER6> Ok, I'll bite. Next time you invest in a startup, or if you ever were to, you agree to forego equity. Instead take a cut of the sales tax, a job, and the opportunity to purchase the product. Fair deal?

<comment><USER7> That's basically what I do as a tax payer anyways...especially when it comes to investing in basic research, I can live with that.

<comment><USER6> The question is, would an angel investor take that deal? The answer is obvious: no way.

An analogy would be if we all belonged to a VC fund, and the fund manager took a deal where instead of equity they take a cut of future sales tax. Imagine that. No way you'd consider that a fair deal, even if the startup produced some nice products and job opportunities.

You'd want equity if it was your personal investment. Why should it be any different for taxpayer funds?

<comment><USER7> Individuals are generally selfish and don't think very well about collective society good, but ever since the Neolithic revolution we've been pretty good about collective investment and risk.

Asking for direct obvious returns from research funds means much useful research won't get done, and researchers will focus on more incremental surer bets (like industrial research or startups). We'd just completely kill our society's competitiveness and might as well start learning Chinese to better welcome our Han overlords.

<comment><USER6> The entire U.S. system of public high tech investment is mostly done under the pretext of military spending. The military has specific objectives and is looking for clear ROI, but that doesn't impair their ability to invest in core science. They just take a longer term view.

Licensing core tech (instead of giving it away) is not the same as asking for direct returns. It's just giving the taxpayers their fair share of the eventual returns on their investment, instead of letting VCs collect it all.

<comment><USER7> The whole space program was about developing ICBMs. WII led to huge technological advances (computers for one), etc...

I'm not sure what the mix today is between military-oriented and social- oriented spending is. The fed obviously give lots of money to lots of research universities, medical oriented research with no obvious military applications coming up at the top of list.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> The title is misleading. It makes you think all this research happened at MIT.

<comment><USER8> For example: #12 The PC (1973), this is Butler Lampson's work on Alto while at PARC. MIT (the university) had nothing to do with it. Also, #17 TCP/IP (1977), Bob Kahn was at MIT for 2 years or so a decade before working on TCP/IP.

Including such things takes away attention from "pure" MIT inventions e.g., RSA (all of that work was done by Ron Rivest while at MIT).

<comment><USER9> That's a fair point, and we edited the title accordingly.

</comment></comment></comment><comment><USER10> my favorite is the GUI. While many credit XEROX with the GUI, most of the ideas came out of the quite amazing Sketchpad by Ivan Sutherland. As an MIT student, this list makes awesome reading!

Here is the relevant snippet:

Nearly 50 years before the iPad, an MIT PhD student had already come up with the idea of directly interfacing with a computer screen. Ivan Sutherland's "Sketchpad" allowed users to draw geometric shapes with a touch-pen, pioneering the practice of "computer-assisted drafting" that has proven vital for architects, planners, and now even toddlers.

<comment><USER7> The work demonstrated by Engelbart in his MOD was actually done at SRI, which then inspired XEROX.

When Alan Kay started his PhD at U of Utah, he was given Sutherland's dissertation to start off with (where Sutherland was a professor). That eventually led to his ground breaking Dynapad concept.

Anyways, back then, everyone knew everyone.

</comment></comment><comment><USER11> I believe that SICP should also be counted as one itself.

</comment><comment><USER12> "50 ways that MIT has transformed computer science"

This is really misleading and makes MIT look bad. It's just spin.

MIT has done plenty of great work; there is no need to try to take credit for things that they really can't lay claim to.

Many examples elsewhere in the comments, but I'll throw in that Ethernet was not invented at MIT. I know the main title now says "MIT-related", but the article is not so modest.

</comment><comment><USER13> #24 caught my attention with the poem/song about the algorithm behind the Spanning Tree Protocol: [https://www.youtube.com/watch?v=iE_AbM8ZykI](https://www.youtube.com/watch?v=iE_AbM8ZykI)

Somewhere in between music school and programming for a living, I had the superficially nonsensical epiphany that algorithms and data structures were essentially music... just another language for expressing ideas about combinations and time. It's interesting to see this other perspective... an algorithm expressed as a song for the purpose of clearly communicating how it works.

The interview with Radia Perlman is a good read too: [http://www.theatlantic.com/technology/archive/2014/03/radia-...](http://www.theatlantic.com/technology/archive/2014/03/radia- perlman-dont-call-me-the-mother-of-the-internet/284146/)

</comment><comment><USER14> The title should be "50 ways that MIT has transformed computer industry". Not too much are about "computer science". To say that MIT has transformed computer industry, it should include include someone like Leslie Lamport.

</comment><comment><USER15> So, Lisp machines get a mention, but Lisp itself does not.

Who writes these things?

<comment><USER16> Well Lisp itself is a Stanford invention. Scheme is an MIT invention. I agree that Scheme deserves a mention though, even though it isn't widely popular, it certainly transformed how people thought about education and things like garbage collection.

<comment><USER4> Sure not.

'Lisp was invented by John McCarthy in 1958 while he was at the Massachusetts Institute of Technology (MIT). '

</comment></comment></comment><comment><USER17> Dan may not be happy - [http://en.wikipedia.org/wiki/Dan_Bricklin](http://en.wikipedia.org/wiki/Dan_Bricklin)

<comment><USER18> Also because spelling is apparently not one of MIT's 50 achievments...

<comment><USER18> And not mine either :-)

</comment></comment></comment><comment><USER19> Some PR?

When I think MIT I think of Aaron Swartz.

<comment><USER20> Because inaction in response to federal charges being brought against an unconnected individual outweighs more than 100 years of academic contribution.

<comment><USER21> That isn't how it works. You don't get to bank being a not-immoral so you can use it later.

<comment><USER22> even if it did , MIT would likely still be working on the morality-debt incurred during their involvements with various wars.

(along with every other famous tech school)

[http://web.mit.edu/physics/about/history/1940-1945.html](http://web.mit.edu/physics/about/history/1940-1945.html)

<comment><USER21> I think this actually played into Swartz case. That possibly MIT didn't want waken a moral/ethical discussion around technology which is exactly what Aaron was trying to do. His message wasn't dangerous, but the conclusion of it means that the the army of engineers who work on MIT's weapons programs could disrupt a huge cash cow.

</comment></comment></comment></comment></comment><comment><USER23> Clearly one of them is not writing click bait headlines:

50? Ain't nobody got time for that...

</comment><comment><USER24> Is there a school that's done more for computer science than MIT? I can think of a few contenders - and obviously this is largely subjective - but when I think computer science in the U.S. I think MIT, Stanford and Cal-Poly in that order.

<comment><USER25> ...you think Cal-Poly over Carnegie Mellon and Berkeley? Really?

<comment><USER26> probably went to cal-poly lol.

<comment><USER24> I'm not familiar with the lol campus.

</comment></comment></comment></comment><comment><USER27> It's interesting that they include the FSF, Internet Archive, Creative Commons, and Open Courseware among their "computer science" contributions. These are more political than scientific, although some of them did end up contributing massively to the development of computer science.

Now let's talk about some ways in which MIT _hindered_ the development of computer science, including political hindrances.

1\. Complicity in the persecution of Aaron Swartz (2011)

2\. Anyone else?</comment>


Stack Ranking is Not the Cause of Microsoft's Troubles - <AUTHOR> http://hackingdistributed.com/2013/08/24/stack-ranking-did-not-kill-microsoft/

<comment><USER2> Why is the author comparing apples to oranges?</comment>


Everyone Hates Ticketmaster — But No One Can Take It Down - <AUTHOR> http://www.wired.com/magazine/2010/11/mf_ticketmaster/all/1 <comment><USER2> Ticketmaster's entire purpose is to be hated. It allows performers and venues to charge closer to market-clearing prices, while directing customer anger away from them.

</comment><comment><USER3> A friend's startup is doing well in this space: <http://showclix.com/>

Lynsie was on Mixergy, to talk about it: <http://mixergy.com/lynsie-camuso- showclix-interview/>

<comment><USER4> History tells us that if they get big enough, Ticketmaster will make them an offer they can't refuse.

<comment><USER5> And if their balls are big enough, they'll refuse and proceed to take down Ticketmaster.

<comment><USER6> Here's hoping they have big balls.

<comment><USER5> Here's hoping that their investors do too.

</comment></comment></comment></comment></comment><comment><USER7> One can hate Ticketmaster for their consumer experience. From the business perspective though, the company does (or has done) a remarkable amount of right things. Most important takeaways from this article:

* Gawda and Leffler are obviously hackers. Pretty good ones: Out-program the competition and get more performance (transactions) out of a smaller computer system than their larger competitor who has a much larger (and more expensive) system.

* CEO Rosen has realized that "ticketing isn't about the bands or the fans, it's about the venues". Thus, they have protected the essence of their business, their customer - the venues - by signing long term contracts with them. Bands and fans come and go, venues are here to stay (and be served).

* They have installed right CEOs throughout the decades: In the beginning the already mentioned Rosen who has set the right agenda, "cutthroat" Azoff who has led them thru the major merger and DOJ negotiations and Hubbard who says that he "can't turn on a dime like a startup" but is willing to change when major change is necessary.

* Never change the running system. Yes it isn't the system that fans (and bands) want but it is able to stand their rush like no other. Great quote from Andrew Dreskin, Ticketfly's CEO and cofounder "A large-scale concert on sale is, in essence, a denial-of-service attack".

* I, once again, am reminded of the following principle: It's not the innovation that wins, it's the leverage. If the innovation IS the leverage then innovate. If it's not, what is it? In the beginning, Ticketmaster was innovative with (ironically) their computer system. While they still use it as leverage, they in the mean time have another, much bigger one: Ties with major venues.

</comment><comment><USER8> I'm not sure people would hate Ticketmaster if their service charges were hidden in the ticket price. That they break it out and show you how much their "service" is costing you is what brings on the hate. We're talking 50% of the ticket sometimes. ($30 ticket plus venue $8 plus service charge $7).

Thankfully, I don't spend much time with Ticketmaster. I go to more indie productions and they almost never use TM. I prefer smaller, more intimate venues. Arena shows? Acoustics are horrible. I also can't stand the scalpers that inevitably come with any big show (thank you American Express and Front of the Line for saving me on a few "Big" shows I've gone to).

<comment><USER9> Here in Seattle, Ticketmaster is in a lot of smaller venues. e.g. indie art band Cloud Cult was here recently at The Showbox, and that (and all Showbox shows) was a TM event. Point is, that and many other similar events in Seattle are hardly arena shows.

</comment></comment><comment><USER10> It's hard to see how someone can develop tech that can compare with the level of reliability that comes from 30 years of debugging.

For small shows, sure. But for the massive tours with hojillions of fans all trying to buy tickets at the same time? Hard to see how someone gets across that gap.

</comment><comment><USER11> Two tickets, one credit card transaction, but two service charges applied. I hate Ticketmaster

</comment><comment><USER12> I always wonder if they wouldn't be hated if they just charged the venues requiring it to be absorbed into ticket sans label. Instead of being listed as a "surcharge." Nobody hates mastercard or visa (outside of the payments industry).

</comment><comment><USER13> they built their system in 1976, and the article seems to suggest it's still running the same codebase?!

Can anyone comment on what it might be written in, and why their infrastructure seems to hold up so well to such high traffic spikes?</comment>


RedMonk Top Languages over Time: January 2019 - 52-6F-62 https://redmonk.com/rstephens/2019/03/20/redmonk-top-20-languages-over-time-january-2019/ <comment><AUTHOR> I think it's amazing how little the rankings have changed over the past few years.</comment>


$1,000 Personal Genome Coming: Are We Ready? - <AUTHOR> http://www.webmd.com/news/20100429/1000-dollar-personal-genome-coming-are-we-ready <comment><USER2> Of course we're ready. Not knowing about a disease risk doesn't make it go away. Yes, some people will be saddened by new knowledge of their genome. But in the cost-benefit analysis, that downside is _overwhelmed by the massive improvement in medical care_. Customized medicine. Preventative care. Screening embryos for diseases. Etcetera etcetera. So many lives will be saved. So many lives will be improved. So what if some irrational people feel bad?

<comment><USER3> Who's going to pay for the curing of these diseases? Insurance companies? I'm sure the more popular diseases will become cheap enough for anyone to afford, but who's to say insurance companies won't deny you coverage to an uncommon disease based on the information your genome gives?

<comment><USER2> There's already a law preventing insurance companies and employers from discriminating based on genetics: [http://en.wikipedia.org/wiki/Genetic_Information_Nondiscrimi...](http://en.wikipedia.org/wiki/Genetic_Information_Nondiscrimination_Act)

</comment></comment></comment><comment><USER4> The benefits are huge of the knowledge learned with knowing your genome. My fear is that it takes us one step closer to the society in Gattaca. I envision that it will start in the sports world where athlete's genomes will be used to determine their athletic potential. Its amazing that the cost is so low, but I can't help but ponder the possible negatives.

<comment><USER5> Everybody talks about the society in Gattaca as if it's a bad thing, but what was really that bad about it? It kinda sucked for those who were born unenhanced, but give it another fifty years and there probably won't be any of those any more (they'll eventually figure out you can just freeze your sperm and have a vasectomy at puberty).

For everybody else, life might not have been perfect, but it was a lot better in many ways than life is today. It wasn't shown on screen, but they'd presumably gotten rid of an awful lot of horrible genetic diseases, and probably eradicated poverty by ensuring everybody has at least the minimum intelligence level required to make a decent living.

Oh, and also: everyone was pretty.

<comment><USER6> > and probably eradicated poverty by ensuring everybody has at least the > minimum intelligence level required to make a decent living

So all we have to do to eliminate poverty is to get rid of the stupid people?

<comment><USER5> That's my hypothesis, anyway.</comment></comment></comment></comment>


What Online Dating Was Like in the 1960s - <AUTHOR> http://fivethirtyeight.com/features/what-online-dating-was-like-in-the-1960s/

<comment><USER2> Isn't anyone going to post a comment describing their experiences with this?</comment>


A censorship resistant deadman's switch - <AUTHOR> https://killcord.io <comment><USER2> This is interesting, but runs contrary to my understanding of how Etherium works. I'm clearly missing something, any chance you (or anyone else) could elaborate more?

My understanding was that the decentralization of Etherium would mean that everyone watching the contract would need a copy of the decryption key. If that's the case, what prevents someone from publishing keys early? Or is it that the key isn't stored in Etherium, and Etherium is only being used as the consent to publish?

If the key is being stored somewhere else and just waiting for the contract to validate, how do we prevent a censor from just attacking that system?

If the key is being stored somewhere else and just waiting for the contract to validate, why not also store the contract on the same machine and do checkins directly into that? Would that be significantly less secure/reliable?

<comment><USER3> Killcord treats ethereum as a project backend API. The smart contract is pretty simple in construction by design. Writes are restricted to one of two accounts (the owner account and the publisher account) and the publisher account is further restricted to only allow writes to the publishedKey variable in the contract. Reads are open to the public.

As stated in other responses, the decryption key is stored own trusted systems that run the owner or publisher killcord projects.

As for attacking the system this is something to think about. So why did I choose Ethereum for this?

Why Ethereum - The contract code (backend API) and variable state are written to the block chain, so the availability are dictated by the network itself which is made of around 20K nodes (give or take). Of course, as others have mentioned the other aspect of this is internet access for the publisher and project owner.

For the publisher, this can be accommodated by running the publisher in a geographically distributed set of trusted systems. What do I mean by trusted systems? These are systems that meet your risk profile. The code can run on AWS Lambda in multiple regions, or on a raspberry pi, or in a datacenter in iceland, the more, the merrier.

For the owner... If you are cut off from checking in, the system assumes something bad is afoot. This is why its important that anything put in killcord is something you really want to publicly disclose. Killcord should really only be a system that runs on your behalf in the case that you go MIA and you feel that is a threat to the data being otherwise released.

Hope this helps clear things up a bit?

<comment><USER4> Are you using only a single decryption key?

If so, you could switch into M of N scheme - far more secure, and thanks to Ethereum, the coordination between key keepers would be really simple.

(Kind of what we did with Orisi.org years ago)

</comment></comment></comment><comment><USER3> Hey Gang. Author of killcord here. I'm honored and humbled this was submitted to HN and I'll be reading through the comments to answer questions and respond to feedback. I started this project after a thought experiment in using newer decentralized tech for internet activism.

<comment><USER5> Neat project! I thought up a trustless scheme for this a while back, but it's beyond my means to implement:

You can encrypt an entire circuit with homomorphic encryption, which users can run without decrypting its internal state. Construct a device like so:

Inputs: 1\. Ethereum block 2\. Previous run-state (encrypted) or zeros.

Outputs: 1\. Next run-state (encrypted) 2\. Decryption key (if triggered) or zeros (if not.)

Internal state: 0\. Hash difficulty range 1\. Hash of previous block seen 2\. Pubkey to scan for 3\. Counter of # blocks seen without a tx signed by pubkey.

If you feed the device more than 1 week of blocks without a tx from pubkey, the accumulator hits zero and it spits out the secret.

An attacker would have to mine 1 week of blocks at hash power IS.0 in order to trick the device into spilling its guts. If you die, and don't send txs for a week, anyone with the device can play a week of blocks into it and the secret will pop out.

Unfortunately, homomorphic encryption is still too slow for this to be quite feasible. Food for thought though! And you can build this today with SGX, if you trust that.

<comment><USER3> neat. Yeah, I picked symmetric encryption for the payload due to its relative simplicity, speed, and resiliency.

</comment></comment></comment><comment><USER6> Censorship resistant, until someone takes down the "publisher tool meant to run autonomously on a trusted system".

<comment> w-m And a landing page that omits this fact (but contains download and instructions for a command line tool). If you're thinking "wait, I can't put a self-publishing secret on the Ethereum blockchain, how does this even work?", the landing page leaves you hanging.

<comment><USER3> This is true.

I'm working with a friend who is a copy writer to help make the landing page clearer and more helpful.

</comment></comment></comment><comment><USER7> Given that the trusted party is required for this to work, is there any point at all in having it depend on the Etherium blockchain, other than perhaps a weak form of anonymity network?

<comment><USER3> The purpose of killcord + ethereum for public disclosures is that leaning on ethereum as an API backend ties itself to the fact that taking down the entire ethereum network is difficult and running your own backend resiliently is hard.

That being said, I'm working on the concept of "providers" so that storage, payload, and backend can be plugable and you'll be able to use whatever backend you are comfortable with.

</comment></comment><comment><USER8> As far as I can tell, Ethereum isn't actually doing anything interesting here - it's just being used to transmit pings to the server, which could just as easily be done with, for example, tcp/ip.

</comment><comment><USER9> Anyone who would think of using it you need to consider at least 2 threat models.

1) The key castodian can decrypt your Information either willingly or through coercion. If you use the same key to sign and encrypt the message or if you do not sign it then they may also be able to impersonate you.

2) A third party who would gain from the information being disclosed can force its release through a denial attack.

Never use a deadman switch as a bargaining or as an insurance policy if you do not intend the information to be released to the public and if you are not comfortable with the information being released the moment the switch is set up rather than when it would be activated.

The only manner in which this or any simmilar setup does not expose you to additional risk is if you only use it to ensure the release of said information in a timely manner and there is no adversarial motive to release it sooner.

@the creators you might want to look at the possibility of implementing [https://en.m.wikipedia.org/wiki/Chaffing_and_winnowing](https://en.m.wikipedia.org/wiki/Chaffing_and_winnowing) over a blockchain.

</comment><comment><USER10> There is a lot of hate for the trusted party set up of this, which seems reasonable.

It seems like you could create a dead man's switch using arbitrary participants. You distribute a secret to every participant and then to attempt to activate the dead man's switch they raise k to the power s mod p and pass it to the next participant. As long as you act as a participant each time and raise the passed value to some invalid s then the answer that is arrived at won't be the final secret.

As long as you participate every round the wrong answer will be arrived at, but as soon as you don't participate the right answer will be arrived at.

Any singular party refusing to cooperate would destroy the deadman's switch so malicious activation would be tough.

Designing it so it can tolerate failures would be the hard part.

EDIT: I am wrong, this isn't that great. It's really hard to hide information that can be recovered without a secret being revealed.

<comment><USER2> So, sort of like a secret generating linked list, where one node (you) are a bad actor?

What prevents the participant right before you from simply circumventing you or secretly passing to the next participant directly?

It also seems that once someone receives the correct answer for their step in the chain, they no longer need anyone beneath them?

(A) -> (B) -> (C) -> (you) -> (D)

Once C has participated in this one time, why do they need A or B?

<comment><USER10> Good point. You'd likely want to also encode something that opaque to who exactly has participated, only really show whether this is the last step and a way for individuals to tell if they have already added their secret.

The really bad part would be that if the poisoner happens to be the last step then the final step would produce the secret before handing it to be poisoned.

<comment><USER11> I built exactly what you've described, using semi-homomorphic encryption (addition of integers, used plainly as we were under the noise threshold of participants). Luckily for me though, I got to punt on some of the really hard questions of trust — the nodes that were communicating are adversarial, but the outside "organising" network was the government and "us" (company I worked for). It's a really fun problem. I highly recommend taking a crack at it, or even just reading the literature regarding digital voting — you need to prove that one vote was cast for a given person, and no more, without ever tying back any specific vote to said person, and with a huge range of attack vectors!

<comment><USER12> Was this a traceable ring signature[1], or something different?

[1] [https://en.wikipedia.org/wiki/Ring_signature#Applications_an...](https://en.wikipedia.org/wiki/Ring_signature#Applications_and_modifications)

</comment></comment></comment></comment></comment><comment><USER13> So a lot of these comments seem to be criticisms of potential vulnerabilities (which is par for hacker news really). I'm curious if there are better alternatives out there that aren't vulnerable to the same issues, like a single point of failure or attack?

<comment><USER14> You could do secret splitting:

[http://www.moserware.com/2011/11/life-death-and-splitting- se...](http://www.moserware.com/2011/11/life-death-and-splitting-secrets.html)

It's vulnerable in that whichever threshold N that you choose allows for N participants to conspire to publish ahead of time, or M - N to conspire not to publish after the fact.

<comment><USER15> interesting. i hadn't seen this although I implemented something effectively the same, except that all keys (which could be any number ≥ 2) be combined to reveal the secret (or any information about it).

</comment></comment></comment><comment> robert-wallis What if the miners deny check-in transactions to force the killcord to execute?

<comment><USER16> You're boned. Then most systems including Ethereum are based on the assumption that the miners aren't majority controlled by an adversary. That may or may not be a sound assumption.

</comment></comment><comment><USER17> Have any legal systems weighed in on a dead man's switch?

I get the premise, where typically it's illegal to take an action that releases confidential or censored information.

But, to governments, especially ones that want to keep information secret or censored, I'm not sure that negating that sequence and failing to stop the release of information (that you willingly put in a dead man's switch) will get you out of trouble.

Unless you're dead of course. But, I've seen this process promoted for living people to release information and I'm not sure it's any better than just posting the content anonymously, but with the added risk of accidentally releasing the information.

</comment><comment><USER18> Better and simpler solution: Create a Bitcoin address and send one Satoshi to yourself every month.

When the transactions stop, people know you are dead.

This way you need no trusted third party, no special software, no special contract.

<comment><USER5> You're describing a different problem. Killcord solves the _Insurance Policy problem:_

Suppose you're a whistleblower, who exfiltrated gigabytes of unredacted data from the NSA. So far you've leaked only redacted excerpts, but the NSA might kill you to stop your leaking.

However, the NSA really doesn't want the whole archive leaked, or it would blow their agents' covers.

So, you put the whole archive up on the net, encrypted, and set up Killcord to decrypt unless you keep checking in. This keeps you alive, since the NSA knows it'll leak if you're dead.

<comment><USER3> yep. this falls in line with my design thinking on this.

</comment></comment></comment><comment><USER19> Why is everyone suddenly spelling Ethereum with an '-ium'?

</comment><comment><USER20> Does this take into account network congestion and such?

<comment><USER3> This is left up to the killcord project owner to set the publisher threshold. If the project owner is concerned about congestion the owner should increase the time allotted to the threshold.

</comment></comment><comment><USER21> if someone puts a gun and steals your private key he can continue checking in after he kills you right?

<comment><USER22> No, if you have ordered keys and only you know the order, there is no way to do it unless you give the order, and even then there's no way to confirm the order is correct without trying it.

The way around this is to threaten not to kill the target, but rather kill their whole family or those they care about viciously and painfully, and be ready to do it, if the order is wrong and there is an automated leak.

<comment><USER23> Well sure, just like you could give them the wrong private key.

I always find these arguments against coercion attacks unconvincing. "Well, they can force you to give them information A, but for some reason not force you to give them information B." No, they'll put you in jail and force you to give them all the information needed to send check-ins, period.</comment></comment></comment>


The BuzzFeed wizard who changed media as we know it - <AUTHOR> http://www.marketplace.org/topics/tech/buzzfeed-wizard-who-changed-media-we-know-it

<comment><AUTHOR> >"Data should not dictate your strategy," Nguyen says, "But you should understand what data tells you and also what its limits are."

Listened to this last night and found it revelatory. All the 'data driven' talk misses the point. Yeah, sure, data is nice and should inform decisions, but at the end of the day, ethical, good business will drive the data.</comment>


Tech Giants battle it out in Starcraft 2 with Day9 - <AUTHOR> http://ahgl.tv

<comment><USER2> Looking at the photos I don't see any Indian engineers considering our IT ecosystem consists a large percentage of them.

<comment><USER3> I don't know this guy, but he looks Indian?

<http://afterhoursgaming.tv/teams/facebook/ankur-pansari/>

<comment><USER4> Yup, I'm Indian :) I wasn't in the team photo because I work from a different office.</comment></comment></comment>


The Tragic Triumph Of The MBAs - <AUTHOR> http://techcrunch.com/2011/09/03/the-tragic-triumph-of-the-mbas/

<comment><USER2> Excellent article. Jon Evans laid it out the way its suppossed to be. Salsforce is an incredible company with a huge future, but saying that CEOs will suffer the same fate as violent, tyrannical dictators is ridiculous.

For one thing, these large firms have a natural system of checks and balances (see relation between CEO, board of directors, and shareholders). In almost all cases (assuming the proper precautions have been taken), a CEO can't just run amok being wild against the wishes of everybody else involved. These issues are typically resolved at the early, early stages with mild disagreements and a back and forth. More often than not, a CEO will step down due to this (let's see this happen in Syria or Libya..)

Secondly, these CEOs have and maintain power because people have given them power. People enjoy the products/services that their firms produce and continue to purchase them. Nobody is coercing users to hand over their money to these firms. This is a fundamental, excruciatingly important difference between the state and the firm.

I could go on for days, but I'll wait for somebody else to chime in.</comment>


Bookstaber: Risk from High Freq. & Algorithmic Trading not as Big as Many Think - <AUTHOR> http://seekingalpha.com/article/158962-risk-from-high-frequency-and-algorithmic-trading-not-as-big-as-many-think

<comment><USER2> The level of uninformed fear surrounding this topic doesn't bode well for driverless cars. Computers trading stocks can't even kill people.</comment>


Amazon S3: Versioning Beta Design - <AUTHOR> http://doc.s3.amazonaws.com/betadesign/Versioning.html

<comment><USER2> potential to use this instead of a dropbox account? the code could be substantially easier to write.

<comment><USER3> i wish i had more upvotes...

<comment><USER2> no worries, going to start hacking at it shortly. I'll keep you posted :)

</comment></comment></comment><comment><USER4> What open source tools are you using to backup to S3 from linux? I imagine the good ones will add in support for S3 versioning.

<comment><USER5> s3cmd is a command line tool interfacing to S3, makes scripting your backups downright simple. <http://s3tools.org/s3cmd>

Another is the Boto Python library (more or less a wrapper for all AWS stuff). <http://code.google.com/p/boto/>

Hopefully both of these will get version control support real soon.

</comment></comment><comment><USER6> How is this different than simply adding a version suffix to each file name?

<comment><USER5> By appending version suffixes by your self, all your clients/apps must be made aware of the current version suffixe and coordinate it (say you have multiple files with different version), can get real messy. And with that method, there is no way to mimic deletes either.

I think the main goal of S3 is to provide a dropbox like feature. You can simply PUT/GET/DELETE/COPY just like before and not worry about version control... and s3 just transparently handles it for you

</comment></comment><comment><USER7> Does it do deduplication, or do I have to pay for too much storage when a large file changes just a little each day?

This is why I use JungleDisk.

<comment><USER8> I'm wondering the same thing too - how did they build this ?

ZFS (after pool ver. 21 I suppose) has deduplication - are they building on something similar ?</comment></comment>


NeXTSTEP on the HP 712 Part 2: Getting Software - <AUTHOR> https://blog.pizzabox.computer/posts/hp712-nextstep-part-2/ <comment><USER2> One interesting thing about the 712 "Gecko" was that it had a bi-endian CPU. So it could run HPUX in big endian, and Windows NT in little endian.

<comment><USER3> ARM also allows you to change the endianness.

ARMv7: [http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc....](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.kui0100a/armasm_cjacabbf.htm)

ARMv8: [https://developer.arm.com/docs/den0024/latest/armv8-register...](https://developer.arm.com/docs/den0024/latest/armv8-registers/endianness)

<comment><USER4> IIRC POWER can also do that.

<comment><USER5> PowerPC too. Most of them anyways, with the notable exception of the 970 ("G5")

<comment><USER6> The 970 was such a pain because of this, Virtual PC needed it to emulate the x86. I remember a massive delay.

</comment></comment></comment></comment></comment><comment><USER7> What a great site. So many memories!

Prior to my time there, my university issued DEC alphastations to CS students. By my time they had moved on to Windows 2000/XP + Linux. So one saturday I popped into the university auction and picked up an DEC Alphastation, a NextStation turbocolor and a couple of Sparcstations. About $5-20 a piece. I was super tempted to get a SGI Origin. No one was bidding on it but the logistics of moving and powering a supercomputer was out of my scope. I power up the Alphastation and imagine my luck, it's my professor's old workstation!

</comment><comment><USER8> Their collection is coming along nicely.

Right after NextStep, they _really_ need to get Apollo Domain/OS (not HP-UX) on the HP 9000/425e. :) Apollo innovated a lot, and rethought or created the network OS, display system, SCM and CI/build servers, etc. There are ideas you won't see elsewhere, or that only reappeared decades later.

If they don't have SunOS 4 with SunView on one of those SPARCstations, that's another neat one to have in the collection. Architecture-wise, a Sun-3 (m68k-based rather than SPARC) would also be nice, and were often used with "shoebox" drives, to complement the pizza box. There were also some neat Sun-386i models, though those were minitowers, not pizza boxes.

Since the theme is "Pizza Box computer", DG actually advertised the less-known m88k-based AViiON as "Mainframe in a Pizza Box", IIRC. If you've read _Soul of a New Machine_ , "AViiON" looks like "Nova II".

</comment><comment><USER9> The last release of NeXTSTEP was in the 90s. It's fascinating how many of the concepts that are still the same, or similar. DNS, TCP/IP, NFS, FTP, resolv.conf, tar, ICMP... Some of them are outdated, but still. The Unix paradigm has held up well IMO.

<comment><USER10> I don't see most of these being examples of "the Unix paradigm".

FTP, being from April 1971 ([https://tools.ietf.org/html/rfc114](https://tools.ietf.org/html/rfc114)), predates Unix 'escaping' from Bell labs.

TCP/IP, reading [http://elk.informatik.hs-augsburg.de/tmp/cdrom- oss/CerfHowIn...](http://elk.informatik.hs-augsburg.de/tmp/cdrom- oss/CerfHowInternetCame2B.html), also didn't start on Unix ("The significant growth in Internet products didn't come until 1985 or so, where we started seeing UNIX and local area networks joining up.")

BIND was fast to arrive on the scene but DNS, I think, was designed to be platform-agnostic.

<comment><USER11> And most of the internet protocols use CRLF, not Unix LF.

</comment></comment></comment><comment><USER12> Is the author, Sophie Haskins, here on HN ?

I went through pizzabox.computer and the personal blog, etc., but found no way to contact other than a twitter handle that hasn't updated in about two years ...

If you're here, I would like to donate funds to you, and these pizzabox endevours, with no strings attached. You can email email@example.com to reach me.

<comment><USER13> She's active on twitter here: [https://twitter.com/sophaskins](https://twitter.com/sophaskins)

</comment></comment><comment><USER14> Does anyone know where HPUX can be found on the net? I have a few old boxen I want to spin up, for fun.

<comment><USER15> Almost everything can be found here:

[https://archive.org/details/hpunix](https://archive.org/details/hpunix)

<comment><USER14> Awwww yeah! This is amazing, cheers.

<comment><USER15> Yes, also make sure you check out this guy's other UNIX related stuff on archive.org

</comment></comment></comment></comment><comment><USER16> What a great blog. Ended up on the SGI Indy page, and omg — that startup chime. Never knew.

</comment><comment><USER17> I've long wondered how hard it is to port from Next to OS X

<comment><USER18> ReDoomEd is a Mac/Linux/BSD port of DoomEd, id Software's Doom level editor for NeXTSTEP: [http://twilightedge.com/mac/redoomed](http://twilightedge.com/mac/redoomed)

About 95-99% of the program logic remained the same, however, there's naming differences (functions, classes, selectors), type changes (NeXTSTEP APIs used C strings & floats, Cocoa uses NSStrings & CGFloats), and missing functionality that had to be reimplemented (Display PostScript functions, Storage class).

I originally expected to have to recreate the NeXTSTEP UI resources by hand in Cocoa, as Xcode's Interface Builder can't read NeXTSTEP nib files. However, Xcode's IB does read OpenStep nib files (as of Xcode 2.5 & earlier - not sure if recent versions can still do this), and OpenStep's Interface Builder can read NeXTSTEP nibs, so it just took opening & saving the NeXTSTEP nibs to OpenStep nibs in an OpenStep VM.

<comment><USER17> Thats really cool!

Its interesting to know how possible (and easy) it is.

Thanks for your work, and pointing this out.

</comment></comment></comment><comment><USER19> I still have an old Sun SPARCstation IPC somewhere at home. I haven't turned it on for over 15 years so I have no idea if it's still works. Good times.

</comment><comment><USER20> Ah, NetInfo. So many memories! The HP was a nice development machine, definitely faster than the slabs.

</comment><comment><USER21> Can you build modern Nethack releases on that?

</comment><comment><USER22> I had one of these with NeXTSTEP but a while after a RAM upgrade it let its smoke out. I wasn't ever able to get it all back in so I threw the whole thing away.

That was like 20+ years ago.</comment>


Use deception to protect your software from piracy - <AUTHOR> http://www.bureau14.fr/blogea/2010/01/security-through-deception/

<comment><USER2> Question 1: Are you making enough money from the legitimate users? If so, forget about piracy. Some people will always steal, just because they think it's fun. Don't waste your time on these people; spend your time adding value for the people that actually pay for your software. If your anti-pirate protection is buggy, you will piss off the people who are already paying for your software.

As an example, take the movie industry. Their "war on piracy" has ensured that I will never pay for (or watch) a major movie again. When you pirate a movie, it's HD, and it plays on any device. You don't need a special cable, you don't need an HDCP-capable surge protector (or whatever). You can fast-forward whenever you want, and you can't be forced to watch ads or the "don't pirate this" propaganda. If you have a fast connection, you can usually watch the movie as it downloads. All in all, a great user experience.

When you pay for a movie, though, all that is out the window. You have to watch ads. You can't have HD unless you have a special cable and a special monitor. You can't watch the movie on a portable device. You can't download the movie; you have to walk to the store, pick something up, talk to a clerk, and carry the movie home. You have to buy a lot of stuff you don't need, you get less features, and you have to pay $20 for the privilege.

The movie industry thinks these piracy countermeasures discourage piracy, but actually, they make not pirating the product extremely unappealing. You would have to be insane not to pirate the movie.

Don't do this to your software.

<comment><AUTHOR> I really agree with you about the movie industry and also the music industry. That's because they didn't analyze that copying a movie or lending it to friends is actually a feature.

There's also much more to music and movies than just the product. It's Art, it's a way of communication. It conveys cultural and ideological values. People can have a deep and emotional link with a movie or music. Adding "don't pirate this" movies and making it hard to share what you love with your friends is really killing what your business is about.

Software is different. ISV are actually hurt by piracy. Platforms have died because of Piracy (Atari, Amiga), and most importantly it's possible to protect a little bit from piracy without annoying your users. I think asking for a serial number once or calling home from time to time is really reasonable. It doesn't affect the user experience and protects both you and your customers.

Where you are right is when you say you must evaluate how much piracy costs you before doing anything. Copying your software must not be "trivial". You want to avoid the situation where it's easier to find a pirate copy than an original one.

But spending months adding anti-reverse engineerings features is just plain stupid.

<comment><USER2> I don't think ISVs are hurt by piracy, they are hurt by people not buying their products. Sure, with piracy, it's easier to not buy your product. But I doubt anyone has ever gotten to the serial number screen and said, "oh, this needs a serial number? I guess I will pay $1000 for a real copy instead of looking for a crack." No. The people that pay for software already made the decision to pay you.

As a practical matter, we use a lot of "ISV" software at work. None of it requires registration or activation or phoning home. If it did, we wouldn't be allowed to purchase it. (This is at a company of 300,000 people, and I think most companies of this size have similar restrictions.)

</comment></comment></comment><comment><USER3> This seems like a fast track towards building a really bad reputation for your software. In many markets the pirates will be the most vocal users, and this could paint your software in a horrible light. How do users know if the crash is caused by your copy protection, or by just plain bad software?

And that's not even getting started on the people-try-before-they-buy-with- piracy argument, which is controversial enough that I don't want to go there.

<comment><USER4> I second that. You want to cripple pirated version of your software in specific, obvious, maybe even fun way.

The Sims had such protection. If game was incompetently pirated then player could not kill cockroaches in the game. It was specific enough so when people googled for some info about cockroaches they were finding out that the cause is badly pirated version. Some of them might have bought legal version. Others just looked around for more thoroughly pirated version. But every one knew game is good and solid.

</comment></comment><comment><USER5> "It must be so much easier to buy the software from you than getting the illegal copy that your customers will quickly dismiss the latter."

Yes THIS! Exactly.

Perfect example in my book is Steam. Steam has singlehandedly made me a consistent game purveyor in my off-work hours. Provide me with games from a trustworthy source where I don't have to get up off my ass to go buy it from a kid behind a counter - offer a service that's reliable, fast, keeps my games up to date and lets me do away with a gigantic binder of 200 cd's & cdkeys and I'll happily dish out the money you deserve.

<comment><USER6> <pedantic>

purvey

1 : to supply (as provisions) usually as a matter of business

2 : peddle

</pedantic>

<comment><USER7> Is the close pedantic tag meant to suggest that you are not normally a pedant? I find it hard to envision somebody who is not posting such a comment.

</querulous>

<comment><USER8> the OP's use of the word 'purvey' was exactly backwards. if i had made such a mistake, i'd want to know about it.

<comment><USER9> It's not the message that's annoying, but the format:

FYI: You flipped it, "A purveyor of goods or services is a person or company that provides them" not the other way around.

That communicates the same thing and takes up less space, and is easier to understand. And by starting with FYI: most people can simply skip it.

<comment><USER8> okay, that's fair. but i thought billybob's presentation was pretty good! it struck me as ironic-slash-cheeky, which is a quality i see in a lot of the best technical people.

</comment></comment></comment></comment></comment></comment><comment><USER10> This is certainly a bad idea.

The last thing you want to do is get into a battle with crackers. They have far more time and expertise at cracking software than you do at stopping them.

For this to work, the intentional bug would have be small enough not to warrant "fixing" by the cracker. In other words, it's unlikely to make a difference.

It's better to think about _why_ people pirate software. A large proportion don't have the means or inclination to buy, so let's disregard them completely. The remaining group _could_ buy, but for whatever reason, do not.

If piracy is easier than purchasing the software legitimately, you've got a problem. Ideally your store should be one page, and require as little information to be entered as possible. Multiple payment methods minimise the chance that a customer won't be able to complete the transaction. PayPal makes impulse purchases easier (assuming you're at that price point).

To sum up: You're probably not losing many real sales to piracy. In the cases where you are, you won't restore them by adding layers of "protection", but rather by looking at why someone might chose to pirate in the first place, and fixing it.

<comment><USER11> Instead of dismissing it out-of-hand, perhaps suggest that developers find metrics for measuring piracy and apply the appropriate amount of protection to their problem.

Disclaimer: one of Root Labs' areas of business is exactly that.

</comment></comment><comment><USER12> This reminds me of the Gamasutra article about Spyro the Dragon a few years back: <http://news.ycombinator.com/item?id=1031510>

Randomly removing essential game items required for progression. We'll that's just cruel :-)

</comment><comment><USER13> What legal implications does this have? You shouldn't be liable for someone illegally using your software. If you store their information they may have some claim against you for locking up their data.

<comment><USER14> (IANAL) In California, it's illegal to place (injuring) booby-traps, because it's effectively punishing somebody for a crime without a trial and that violates due process of law. However, refusing someone use of a stolen good isn't the same as punishing them for stealing it. I doubt this has come to trial yet, but a reasonable line to draw would be that bad side effects can castrate the stolen software as much as you want, but cannot extend any further than the intended use of the software itself. Having stolen software not work is totally fine; damaging the host computer in some way or leaking private information is probably a good way to get yourself sued.

<comment><USER15> > I damaging the host computer in some way or leaking private information is > probably a good way to get yourself sued.

The intent is to not damage anything, but rather to only crash the program itself. This might lead the user to visit a support forum where they will be kindly adivised to purchase a legitimate copy of the software.

However, if I understand correctly, the program is crashed when it overwrites its own memory. That means that at some point it could end up executing random code. Although very unlikely, it is possible that it could execute the equivalent of "remove users home directory" code. A more likely, but less 'punishing' outcome could be the execution of a tight infinite loop. That could be interpreted as a 'denial of service' and thus full under the booby- trap law.

</comment></comment></comment><comment><USER16> If major corporations aren't dick enough to do this to their users, i don't think many of us have the luxury.

</comment><comment><USER17> Does this mean I should write my future software in Whitespace ( [http://en.wikipedia.org/wiki/Whitespace_(programming_languag...](http://en.wikipedia.org/wiki/Whitespace_\(programming_language\)) )?</comment>


Language Models, Word2Vec, and Efficient Softmax Approximations - <AUTHOR> http://rohanvarma.me/Word2Vec/ <comment><USER2> How can this be used for full-text search, e.g. with Lucene? The first step in indexing a document for full-text search is reducing each word to its base form, and similarly for a search string. While it's not a difficult problem in English, in some languages (e.g. Herew) it's notoriously hard to figure out the base form of a word and further disambiguate its meaning, as the only way to do so is based on context. So how can you easily build a stemmer/lemmatizer on top of these instruments to perform such task?

<comment><USER3> Run Doc2Vec (or Word2Vec) on a large corpus of text or download pretrained vectors. To compute a document vector, take a linear combination of the word vectors in the document according to TFIDF. Now that you have vectors for each document, you need to create a fast index with a library called "Annoy". It can do very fast similarity search in vector space for millions of documents. I think this approach works faster than grep and doesn't need to bother with stemming. It will automatically know that "machine learning" and "neural nets" are related, so it does a kind of fuzzy search.

<comment><USER4> If you wanted it to know that "machine learning" and "neural networks" were related, wouldn't you need to do some type of entity extraction first, since Word2vec is run on tokens?

<comment><USER5> The original word2vec source code comes with a probabilistic phrase detection tool. Keyword: word2phrase.

<comment><USER4> Good to know, thanks!

</comment></comment></comment></comment></comment><comment><USER6> Nice write-up! I also found this one useful in terms of high-level implementation: [http://adventuresinmachinelearning.com/word2vec-keras- tutori...](http://adventuresinmachinelearning.com/word2vec-keras-tutorial/)

Initially it was not obvious to me that the dot-product was even part of the model. In hindsight it's intuitive: a pair of high similarity vectors will have a high dot-product, which yields a high sigmoid activation. This also motivates the use of cosine similarity, which is just a normalized dot- product. Likely obvious to some but this eluded me the first few days I studied this model.

<comment><AUTHOR> Thanks for the link! It actually provides a really clear and intuitive explanation of the notion of similarity.</comment></comment>


Small files are killing your query performance. Make bigger files instead - <AUTHOR> https://www.upsolver.com/blog/small-file-problem-hdfs-s3#utm_source=hackernews&utm_medium=referral <comment><USER2> A similar problem was faced by INN which stored each article in its own file (like Maildir) - life got pretty crappy on big transit boxes even with directory hashing. Diablo solved it by allocating huge files and storing articles in there sequentially with a pointer stored elsewhere (like an mbox file.)</comment>