Has Chomsky been blown out of the water? [pdf] - <AUTHOR> http://www.covingtoninnovations.com/michael/blog/1609/160916-Chomsky.pdf <comment><USER3> This is like asking "Has Einstein been blown out of the water?" in the context of physics.

The answer is that such a thing would be impossible, because even though his theories and models are imperfect (as all theories and models are), they were so influential that they fundamentally changed his field.

Chomsky had a similar effect on linguistics, and even if much of his work ends up being rejected, his influence will have changed the field of linguistics forever. The theories that supersede Chomsky's will be developed within the paradigm he created.

<comment><USER4> Linguist here. It really isn't. There are plenty of competing and very compelling theories in e.g. big parts of Europe. Especially Functional Grammar and its different branches. The only relation they really have to UG is that they're not it.

At my university no one in the linguistics department actually subscribed to UG or any derivatives thereof but in the English department they did. I've observed sort of the same divide in countries. In English speaking counties UG seems to be the prevailing theory while it's much less so in non-English speaking countries.

Unfortunately people are extremely religious about their preferred theory of grammar, and it's hard to have a useful discussion about it when people are unable to remove themselves from the subject and discuss it objectively.

<comment><USER5> I studied theoretical linguistics through the graduate level and it wasn't until rather late in my schooling that I even began to hear of competing frameworks such as HPSG or LFG, and much of it in passing. There was no mention of them in the coursework, and the tendency of faculty to sell modern linguistics as a hard science gave the impression that we were learning a standard approach that had a provable empirical edge and broad scientific consensus. Otherwise, why would it be called "Extended Standard Theory"?

While I know that part of the pedagogy was giving students a grounding in dominant strains of formalism (in North American circles anyway), at times I felt a bit cheated out of a broader perspective.

Of course in practice many of the competing frameworks are weakly equivalent in terms of their descriptive power and the differences come down more to things like biological plausibility, computational complexity, how strongly you interpret the claims of UG, whether you have any interest in building tractable computational implementations, and so on. Or how you feel about the proliferation of empty functional categories, which was what bothered me the most from an empirical point of view.

</comment></comment></comment><comment><USER6> If you want to prove a scientist wrong, present evidence to the contrary rather than personal innuendos and hypotheses / observations without proof.

<comment><USER7> This is a good criticism of a number of Chomsky's polemical papers.

<comment><USER8> "A number", as is usual with Chomsky, being a bunch of stuff you're not going to quote, because you'd likely end up eating all of it. There's gotta be a nest somewhere.

</comment></comment></comment><comment><USER9> The manner in which children of diverse cultures and varied circumstances acquire language is extremely uniform. Children of accomplished academics in Cambridge pick up English at the same rate at which children of illiterate farmers in Benin pick up Gbe. They will hit the same milestones at the same age.

We also know that a stroke can seriously impair a person's ability to use language, even as their memory and powers of abstract reasoning are otherwise in tact.

The only way to make sense of that that I know of is to suppose that language engages some sort of specialized structures within the brain.

If such structures exist and are implicated in the acquisition of language by children across different cultures, then language must not be infinitely plastic. There must be certain structural properties that languages share that allow them to be acquired in such a regular, programmatic fashion.

Is this conclusion unwarranted, unreasonable, false, or dubious? Are there other ways of making sense of these phenomena?

<comment><USER10> >The manner in which children of diverse cultures and varied circumstances acquire language is extremely uniform. Children of accomplished academics in Cambridge pick up English at the same rate at which children of illiterate farmers in Benin pick up Gbe. They will hit the same milestones at the same age.

Do you actually know that? Or are you supposing it because it makes sense? Given what I know about the differences in educational outcomes between socioeconomic groups in the US, I wouldn't be surprised if the rate of language acquisition depended on the number and variety of words that a child hears. It may very well be the case that the child of the Oxbridge academics _does_ pick up language more quickly simply because he or she is exposed to more of it.

<comment><USER11> Yes. Language acquisition follows a standard sequence with specific milestones universal to all languages. For example, the period around 6-12 months is critical for phonetic learning. Before 6 months, infants are able to discriminate between phonetic units present in any language, but after 12 months they are better at making distinctions that occur in their native language and worse at making distinctions that don't occur in their environment. For example, Japanese infants are just as good as American infants at distinguishing between r and l at birth, but lose some of this ability between 8-10 months.

(Chapter "Language", Principles of Neural Science, 5th ed.)

</comment></comment></comment><comment><USER12> If you read a bit further (pages 2 and after) it's actually not all innuendo. In fact some of it is actually laudatory about his contributions. It's basically a very brief summary of Chomsky's ideas and the impact he had on linguistics as a profession. Namely using trees to model syntactical forms and his hypothesis on universal grammar. I don't know what it is about HN that makes people refuse to read past the first little thing they object to and then bloviate about it.

<comment><USER13> > I don't know what it is about HN that makes people refuse to read past the > first little thing they object to and then bloviate about it.

That's hardly unique to HN. Following Chomsky, I'd argue that it's actually a universal behavior, which merely finds (slight) variation in expression according to the medium at hand. It even has a Latin name - argumentum ad logicam - and it's closely related to cherry picking except that it's applied to parts of the argument instead of to data. In fact, I just did it to you when I picked on the inessential "about HN" part of your sentence (which reads fine without it), and if you find that annoying then perhaps you should consider that you also did it when you jumped into a response about innuendo without addressing the more substantive discussion that had already occurred.

</comment></comment><comment><USER14> It's funny, in the 90s at college I knew some people in the Linguistics dept and even back then, people were saying, Chomsky's deep grammar is obviously nonsense but we have to learn it anyway because it's in the exam.

</comment><comment><USER15> Off topic, but as long as we have a Chomsky discussion - where would be a good place to start reading about his theories on the evolution of language?

More on topic, the dude is a giant, and the linked pdf not worth the electrons it's printed on.

<comment><USER16> This paper is a good place to start lol. The paper begins with a summary of Chomsky's major contributions, and is fairly readable.

The answer is no, Chomsky will probably never be blown out of the water. His hypothesis is evolving, though.

<comment><USER7> He has one hypothesis?

</comment></comment></comment><comment><USER17> It's interesting to think that connectionist theories of language are literally shipping on Google Translate and people still argue about Chomsky. Many of the cognitive scientists were poking about with neural nets too - <USER2> they were getting into huge arguments with Chomsky and like folks since the 80's.

<comment><USER7> To be fair, due to the nature of neural networks, there are a great number of domains where they can be applied in some way without that fact actually explaining anything about how the domain works.

For example, TD-Gammon plays a mean game of Backgammon, but this does not mean that Backgammon reduces to connectionism. TD-Gammon applies neural networks in a specific way with other algorithms involved, and then there are still underlying facts about what it takes to win at Backgammon which do not necessarily mandate anything that resembles connectionism.

</comment></comment><comment><USER18> This article shows practically no evidence of its main thesis. To substantiate that Chomsky has been "blown out of the water", he mentions 3 things:

1) "[...] greater awareness of the diversity of human languages, and especially the discovery by Daniel Everett of a Brazilian indigenous language, Pirahã, that apparently has no subordinate clauses"

That's the one and only evidence the author shows for this point. No examples on how the language works or how it stands in contrast to Universal Grammar, and no other mention of any other language that might be contradicting UG as well (even though the author implies that there are other languages that stand in contradiction to UG). Later on though the author says "It is not even undisputed that Pirahã lacks subordinate clauses" so the one and only evidence is still a maybe.

2)"The second is advances in cognitive psychology. [...] Nowadays we know that the human mind has complex, powerful, abstract capabilities in many areas, not just language, and new possibilities are opening up for explaining language from general mechanisms of thinking and learning."

This is not an argument and no evidence has been shown for this point.

3)"The third factor is the increasing use of data-driven methods in computational linguistics [...] In a data-driven approach, structure arises because sentences obviously consist of parts that can be recognized before recognizing the whole. [...] This predicts that the grammar of uncommon words will be simple and regular, but common words will have more quirks because we have more examples from which to learn them. That is exactly what we observe in language, and it's a source of great clutter in a grammar constructed on Chomskyan principles."

As far as I know, the fact that sentences have parts that can be individually recognized before recognizing the whole, is not contradicting UG at all. To the best of my knowledge, UG only says that there is a structure, not how the structure comes about. So the only point left is the fact that common words have more varied usage and are harder to pin down as a particular specific element of the structure. I don't see how that relates to UG quite frankly. Maybe someone can enlighten me on this.

I have no particular love for UG, I just know of it on a very general sense, but if I was a teacher and a student gave me this article as a report, I would've failed him just for the lack of evidence.

</comment><comment><USER19>
      (I don't hear good things about it from political scientists.)

Immediately closes tab. I assume the rest of this is garbage innuendos, too?

<comment><USER20> He just couldn't help himself. Just had to throw that in, even after acknowledging he is not familiar with it.

It's an appropriately weak start to a bland document.

</comment></comment><comment><USER21> > Chomsky

Heyyy it's actually about Chomsky qua linguist instead of qua political whatever!

> It would be premature to conclude that universal grammar has been proven not > to exist.

> But universal grammar has lost some credibility. Suppose the whole notion is > eventually abandoned. What will be left?

> Second, psycholinguistics is going to be big business.

> In short, Chomskyanism may be on the wane,

Welp. This is some Fox News "I'm just saying" bullshit right here.

edit:

> Third, opportunities to discover more about language are continuing to open > out before us.

Language attrition (and the dearth of linguists willing / able to work in the field to produce ethnographies of dying languages) are _decreasing_ our opportunities to learn about human language.

</comment><comment><USER22> Looks like some very important people ordered Chomsky's discreditation campaign, because that guy had enough balls to reveal something about them to general public in his book "How the World Works".

<comment><USER23> [http://literarylilt.blogspot.com.au/2012/02/noam-chomskys- ho...](http://literarylilt.blogspot.com.au/2012/02/noam-chomskys-how-world- works.html)

</comment></comment><comment><USER24> Maybe people will stop listening to his political opinions just because he used to be credible on some other unrelated topic.

</comment><comment><USER25> Today I learned about Betteridge's Law [0], which seems apt here. It states that when a news headline asks a question, the answer is always "no." When it applies to a paper, I'm disinclined to take the paper seriously...

[0] -[https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...](https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines)

<comment><USER26> This gets posted on EVERY submission ending in a question mark. It was old 2 years ago. Now it's just plain trite, pointless, tiresome and distracting.

<comment><USER27> I self-aggrandizingly call this tome's law:

[https://news.ycombinator.com/item?id=9077549](https://news.ycombinator.com/item?id=9077549)

<comment><USER8> Why not skip to the end? When enough articles (ending in a question mark or not) are discussed the probability of Betteridge's law (as well as anything else) being mentioned approaches 1. There everybody goes, forever, we are completely done with that class of stuff now. You're welcome :P</comment></comment></comment></comment>


Apple rolls out a new App Store developer site - <AUTHOR> http://techcrunch.com/2016/04/18/apple-rolls-out-a-new-app-store-developer-site-with-guides-and-videos-for-growing-app-businesses-2/ <comment><USER4> [https://developer.apple.com/app- store/review/guidelines/](https://developer.apple.com/app- store/review/guidelines/)

OK, this is not going to be a popular opinion but this is terrible, harsh and condescending language. Only apple can get away with this.

Phrases like 'think about the children', 'amateur hour', 'run to the press' simply don't help. I know how to take care of my kids, thank you. And the lesson I am giving them is not to get into walled gardens where someone else controls what my children can see and do. Calling things amateur is sad - <USER2> are they to decide? Let the market decide, don't decide for the market. Did these guys see the airbnb video the other day where Brian said everyone said the idea was totally stupid. 'Lasting entertainment', yeah right.

I don't know about you guys but this is as close to big brother speak as I have seen.

(I don't own any apple product and never will because I cannot imagine buying a device where I cannot install my own software without purchasing Apple laptops. Seriously? This is progress?)

<comment><USER5> "Calling things amateur is sad - who are they to decide? Let the market decide, don't decide for the market"

It's their market. That's who they are to decide. And they're right. Having a bunch of crappy, shoddy apps hurts those who actually put time, effort, and yes, money into apps with the hopes of making a return.

Seriously, by that token, any grocery store should be forced to carry my shittily packaged and shitty tasting food.

"I don't know about you guys but this is as close to big brother speak as I have seen."

No, it isn't. It's telling developers not to try and submit just any old crap and hope it sticks. Quite frankly, Apple has plenty of developers and apps; they can afford to be picky.

"(I don't own any apple product and never will because I cannot imagine buying a device where I cannot install my own software without purchasing Apple laptops. Seriously? This is progress?)"

Nobody cares.

<comment><USER4> > Seriously, by that token, any grocery store should be forced to carry my > shittily packaged and shitty tasting food.

If we had just 2 main grocery stores and each store acts in such an authoritarian fashion, then it's the duty of the people in know how to educate people about the situation. Over a course of 10 years, you will only get only "chips" and "biscuits" approved by them because they decided what is good for you. In fact, once biscuits becomes profitable enough, they will kick out existing players and make their own since they control the market (and Apple has much history of just merging existing products and calling it their own).

> Nobody cares.

Sure, if the above reality seems appealing to you, go ahead and not care. But many of us do and we are doing our bit to change it.

<comment><USER5> The "nobody cares" part was directed at your chest thumping bravado of "I Don't Have Apple Products", like you deserve some kind of cookie for saying so.

</comment></comment></comment></comment><comment><USER6> Also, they have recently overhauled the Developer Member Center, which instantly breaks all of the tutorials on the internet on how to set up that certificate crap for push notifications, dev deploy and prod deploy.

Android is "obtain a key from GCM, then ionic build android (and distribute the apk to whomever you want, as long as the target device has Play Services)"... iOS is "okay, generate a CSR here, (insert 20 steps), still not finished", and to make stuff worse it's a true PITA to debug push notifications.

Apple's developer workflows suck, not to mention the ridiculous 100-test- devices-per-account policy (huge company and all agencies are invited to their account => huge chaos).

<comment> k-mcgrady >> "iOS is "okay, generate a CSR here, (insert 20 steps), still not finished""

A few years ago maybe. Now Xcode handles most of this stuff for you automatically. There are still some situations when the member centre is queried but for the most part Xcode seems generate everything you need when you try to build & run for the first time.

<comment><USER7> Xcode handles the simple case automatically and screws up everything else making the "Fix Issues" button a bit of a gamble, honestly.

</comment></comment></comment><comment><USER8> Their guidelines page ([https://developer.apple.com/app- store/review/guidelines/](https://developer.apple.com/app- store/review/guidelines/)) has some pretty curt language.

Interesting how terse and forward they are here as compared to other Apple docs. Reads as if a jaded reviewer wrote it :)

<comment><USER9> >Apps or metadata that mentions the name of any other mobile platform will be rejected

So if I have a cross platform messaging app, I can't say, "syncs with android and Windows phone devices". Really?

>Apps cannot use Push Notifications to send advertising, promotions, or direct marketing of any kind

Ahem Lyft.

<comment><USER10> > So if I have a cross platform messaging app, I can't say, "syncs with > android and Windows phone devices". Really?

You can't say it in the application or its metadata (e.g. appstore listing), no.

> Ahem Lyft.

Report them?

<comment><USER11> How? With a bad review, or some other method? I didn't see anything on the App Store page, but I might have been looking in the wrong places.

</comment></comment></comment></comment><comment><USER12> They seem to have increased restrictions on purchases outside Apple's control (section on 'Purchasing and currencies').

        Subscriptions Outside of an App

        Subscribers to magazines, newspapers, books, audio, music, video
        and cloud storage services who were acquired outside of your app
        can read or play content through the app. However, you may not
        provide external links in your app that allow users to purchase
        subscriptions outside of the app.

Sure, they've created the OS, but this is starting to feel a bit like a troll guarding a bridge, forcing every one to pay up to walk across.

<comment><USER13> This is not new. This was put in place in 2011. This is why the Kindle app (for example) does not allow you to buy books in the app, and the Amazon app can order physical goods but not downloads.

[http://www.macrumors.com/2011/06/09/apple-reverses-course- on...](http://www.macrumors.com/2011/06/09/apple-reverses-course-on-in-app- subscriptions/)

</comment></comment><comment><USER14> > Apps larger than 100MB in size will not download over cellular networks > (this is automatically prohibited by the App Store)

I have a 10GB data plan, why is this still a thing?! At least give me an option to turn it off buried deep in the settings somewhere. I wish we had something like `defaults` on iOS.

<comment><USER6> > I have a 10GB data plan, why is this still a thing?!

Where are you living and holy fuck how much are you paying?

Not everyone lives in Finland, unfortunately. It's a small number of countries where 100MB won't eat up like half your monthly data allowance and it's good that Apple prevents morons with unneccessarily huge apps (FACEBOOK!!!) to screw their customers.

<comment><USER15> _Not everyone lives in Finland, unfortunately._

Finland? T-Mo in the U. S. (you know, the country that is _sooo_ much worse than Europe when it comes to mobile?), two lines with combined 14Gb data, $120/month. Not quite 10Gb per phone, but close enough for our use. We had the unlimited data, but we didn't use enough to justify the extra $30/month.

<comment><USER16> Here in Germany it's more common to have 1 GB for $10 per month. So more or less the same price you pay per GB, but not many people can justify paying >$100 per month for a mobile plan. I also have to manage with 1 GB data per month, simply because I don't have to pay more than maybe $10-15 per month.

<comment><USER15> Keeping in mind that the $120 we pay includes unlimited text and voice calls. We don't need the 14Gb, as we rarely go over 1Gb, but the difference is minimal enough that I pay the difference from a cheaper plan "just in case".

</comment></comment></comment></comment></comment><comment><USER17> > If your App is rejected, we have a Review Board that you can appeal to. If > you run to the press and trash us, it never helps.

Except it does, until the US and EU Antitrust commissions make you guys stop this shit.

<comment><USER18> What "(anti)trust"? This is a single company.

<comment><USER17> Their actions are anticompetitive towards developers and app publishers. These are individuals and companies, both valid entities in an antitrust complaint.

<comment><USER18> > _Their actions are anticompetitive towards developers and app publishers._

On their own marketplace. That they created themselves and can control as they like, and which is not even a monopoly (though that's also legal to be).

Not on the overall market.

That's like saying a supermarket can't decide what they sell or not, and whether they'll promote a particular product or not.

<comment><USER17> Correct, and all of the app developers and users are agreeing to the contract.

This has absolutely zero bearing on the establishment upholding the conditions of those contracts. Especially with said contract's arbitrary, vague and opaque, enforcement.

You heard it hear first. App Stores are an antitrust violation waiting to happen.

<comment><USER18> That's not what antitrust laws are about.

See here: [https://www.quora.com/Why-was-Microsoft-punished-for- violati...](https://www.quora.com/Why-was-Microsoft-punished-for-violating- antitrust-laws-while-Apple-which-is-even-worse-now-is-not-punished)

<comment><USER17> They engage in anticompetitive practices, the antitrust commissions will be the ones to levy sanctions whether they are in line with case law or set their own precedent.

<comment><USER18> I don't think antitrust means what you think it means...

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> I'd say that so far, Apple has gotten a lot of good apps _in spite of their App developer /app submission infrastructure_, rather than because of it. It's basically been an obstacle rather than a help.

Is this a sign that their Jobsian approach to third-party developers is changing?

<comment><USER20> I'm not going to argue that there aren't crap apps on Apple's app store, since there definitely are, but if it were easier to let apps onto the appstore, wouldn't there be even more crap apps? I doubt a loosening of restrictions would mean a massive surge of quality apps. Sure, it might mean some good apps that had been rejected or had avoided the appstore because of restrictions could get on, but the number of those quality apps would be minuscule compared to the crap app surge.

<comment><USER19> There's already insane amounts of crap apps in the Apple app store.

What Apple should build is robust quality app discovery, filtering out the crap.

What I'm seeing at the moment (seven years after the launch of the app store ) is a software architecture that appears to rely on the app store submission process being a guarantee of quality. What a joke...

</comment></comment></comment><comment><USER21> Their whole thing about sex in the second paragraph—anyone have any idea, given this, why if I type "sex" in to the app store search I get a good amount of apps blatantly about sex? First one is Sex Positions 3D.

<comment><USER22> Guess they're wanting to either halt future apps of that kind, or have a reason for removing them in the future.

<comment><USER23> They have had big clean outs before, where they removed lots of apps from the store, when they were trying to clean up the image of the store.

</comment></comment></comment><comment><USER12> Perhaps the URL should be updated to [https://developer.apple.com/app- store/](https://developer.apple.com/app-store/) (instead of techcrunch)

</comment><comment><USER24> Shouldn't they improve the terrible mess that the app store is, instead of "educating" developers?

<comment><USER25> I think a vague awareness has seeped through the walls of Cupertino now that the App Store has turned into the software equivalent of grey goo.

I wouldn't be surprised to see an announcement of a big clear out at WWDC - <USER3> not this year, then certainly next.

</comment></comment><comment><USER26> Apple recently rejected a tvOS app of mine due to the 2.12 no long lasting value item.

I sort of anticipated that and while for some users the amount of content and long lastingness is sufficient in that build I think the features I had initially planned for the first update will really make a better 1.0 with appeal to more users.

When I first browsed the Apple TV app store I was really disappointed by the vast amount of crap, so from a consumer's perspective I'm glad they started taking their guidelines serious now.

</comment><comment><USER27> no pictures of an apple fruit "3\. Variations, Takeoffs or Abbreviations: You may not use an image of a real apple or other variation of the Apple logo for any purpose. Third parties cannot use a variation," ([http://www.apple.com/legal/intellectual- property/guidelinesf...](http://www.apple.com/legal/intellectual- property/guidelinesfor3rdparties.html))

</comment><comment><USER22> Pay to be promoted in search? Ugh. How about offer a portion of credits out of the 30% cut to go toward search/category promotions. Or lower the cut taken. Or offer a choice between the two, plus an additional means for buying ads. But digging further into the 70% devs are pocketing before taxes? Lame. And paid promotion seems like it's already going to make the App Store even more awful for users and smaller developers.

<comment><USER28> 30% sounds a lot, especially before tax.

but if we take a lesson from the games industry, we know that 30% is what valve takes.. and what Sony/Microsoft takes.

Not to mention the up front cost of the verification (which is a significant amount of money honestly, Apple's is minuscule by comparison).

But for that money Sony/Microsoft/Valve will not promote your product, they will simply be available for the platform.

I'm not sure how the Play store works of course, but my point is that marketing budget and release "tax" are separate.

</comment></comment><comment><USER29> Looks like only certain parts have been updated. The Certificates/Provisioning section seems to be the same, login page remains the same, iTunes Connect remains the same (which admittedly received a facelift last year).</comment>


NASA Twins Study: How stressful will a trip to Mars be on the human body? - happy-go-lucky https://www.nasa.gov/feature/how-stressful-will-a-trip-to-mars-be-on-the-human-body-we-now-have-a-peek-into-what-the-nasa <comment><AUTHOR> Hey everyone! I work in a lab that studies the effects of interplanetary space radiation (NASA Grant), and we were actually discussing this study yesterday. While the data from this study is a wonderful first step forward, there has been a decent amount of criticism, because n is essentially 1 (with 1 control). This makes it very difficult to separate the inter-twin effects from the space effects.

If there is anything that you feel that we should be focusing on or any aspects of space radiation that haven't been properly explored, we're always looking for new ideas!

<comment><USER2> If I am alone with this thought, please let me know, but:

Isn't the amount of research, meticulate care and millions upon millions spent making space travel as safe as possible becoming ridiculous?

About 150,000 people die each day. That's about 50 million a year. Test pilots used to die in crashes all the time.

I'd wager there are plenty of aspiring Astronauts that would be okay with an earlier death (due to radiation exposure, bone degredation, whatever) for having had the chance to go to Mars.

We won't really know anything beyond speculation until we actually send someone anyway.

<comment><AUTHOR> While "as safe as possible" is certainly the goal, most of the research is centered around first clearing the low bar of "safe enough to even make it there and then do some science". "Earlier death" might mean "on the way to mars", which would end up being a much bigger waste of money and effort (to say nothing of the political capital). Those are the questions we're trying to answer right now.

The other side of these things is that the research being done is not single- purpose. The lunar missions were an amazing human achievement, but their lasting legacy can be seen much more clearly on earth. Our radiation work has homeworld implications for understanding radiotherapy. Heavy ion therapy is a rapidly emerging technology that has the potential to make some previously- mortal cancers tractable, and understanding its full biological effects is critical. By exploring new worlds, we hope to be able to improve the lives on our own as well.

<comment><USER2> While thinking about it:

My claim was really totally unsubstantiated, because I have no idea how much is spent on human safety. I just assumed because I read about that regularily.

It might very well be that the actual investment is miniscule when compared to the technology spending.

Your other points are definitely valid as well.

<comment><AUTHOR> Your point is still important, whether or not it is valid, because it represents a not-uncommon sentiment in the public. It's up to many researchers to try to make the results and potential of their work accessible to the public. The best mentors I've seen are always able to transfer their own excitement for their work to others.

</comment></comment></comment></comment></comment><comment><USER3> I wonder did NASA consider the study design in which both astronauts actually follow the same regime - the same food, the same exercise profile, sleep paterns, social exposure etc. That is, complete mirroring of the life style except for the location.

I imagine this must be _very hard_ to accomplish but I wonder how meaningfull is the study otherwise.

<comment><USER4> It depends on what you're trying to figure out. If you're trying to figure out _just_ the effects of location (which pretty much means zero G and extra radiation) then you're right, you need to control all the other factors. (You could even isolate the zero G effects by subjecting the control subject to extra radiation, though that might not fly with the ethics boards.) But if you're trying to figure out the sum total effects of space flight, _including_ all the ancillary effects (like decreased social interaction) then it makes sense to let the control subject just live a normal life.

</comment></comment><comment><USER5> What is the purpose of having a control person on the ground, can't the control be the same person before they went to space? I don't suppose a person changes so much in one year that aging would be a significant factor.

<comment><USER6> You might miss things like this:

> telomerase activity (the enzyme that repairs the telomeres and lengthens > them) increased in both twins in November, which may be related to a > significant, stressful family event happening around that time

</comment></comment><comment><USER7> Sample size = 1. While this study might be an indication of the kinds of things to look for, the results are not significant.</comment>


Facebook Starts Mandatory Profile Linkage to Public Pages - <AUTHOR> http://blog.louisgray.com/2010/04/facebook-starts-mandatory-profile.html

<comment><USER2> This title is completely misleading. There is nothing mandatory about Facebook's prompts for you to link items in your profile to public pages. I was able to click 'ask me later' and the prompt went away. Next time it shows up I'll wind up looking into how to disable it from ever showing up again.

<comment><USER3> If it's not mandatory, it's at least a strong push, and if you don't comply, your profile could be made empty.

<comment><USER4> _if you don't comply, your profile could be made empty._

I just did it. It makes the profile mostly empty. If you don't comply, it will delete your hometown, current city, employer, schools, and interests, It leaves family connections, quotes and contact info.

However, fB keeps that stuff behind the scenes so you can re add it later...

</comment></comment></comment><comment><USER5> If you haven't seen the prompt yet, had a lot of stuff on your profile, and want a majority of it deleted: facebook doesn't provide an "uncheck all" button.

When I saw it, it was in its own window, not a modal - so I don't know if this will work or not. Feel free to try. But this'll uncheck all of them for you so you can selectively choose which ones to make public (instead of which ones to delete):

javascript:(function(){var c = document.getElementsByTagName("input"); for(var i = 0; i < c.length; i++) { if(c[i].checked) { c[i].checked = false; } } })();

<comment><USER6> They do provide an uncheck all button. Check the "check all" checkbox, then uncheck it. That checkbox drives all the others.

<comment><USER5> Urgh. What a weird paradigm.

</comment></comment></comment><comment><USER7> I'm guessing that a large part of this is about SEO.

<comment><USER8> Rather, it's about data-mining for advertisers.

<comment><USER9> hmm certainly.. thats def the idea behind it

</comment></comment></comment><comment><USER10> They still try to increase PageRank while they already have more visits than Google.

In other words: they don't rest on their laurels.

</comment><comment><USER11> Does anyone know how to get my facebook page for the Happy Hour Finder to be the default for people who like happy hours?

</comment><comment><USER12> this is annoying. they make it mandatory and the only way to get out of it is to reload the page and hope it doesnt show up again. a) i dont really want to do this b) if i ever do, i just dont have the patience to pick+choose that im linking the right stuff.

<comment><USER2> What about the experience gives you the impression it is 'mandatory'? It's a great example of opt-in. Granted, there is no 'don't ever show me this box again' checkbox, but if Facebook would have taken this approach with their Open Graph initiative they wouldn't be having the same level of criticism laid into them by privacy groups, congress, their users, etc.

<comment><USER13> Try editing your profile if you don't think it's mandatory.

</comment></comment></comment><comment><USER9> im a firm believer in avoiding to provid any content on your profile that may be easy to datamine, therefore no comma separated interests. i usually just ask people to ask me if they really wanna know.</comment>


$13.3 Million For Startup That Wants To Kill Snail Mail - <AUTHOR> http://www.techcrunch.com/2008/01/08/133-million-for-startup-that-wants-to-kill-snail-mail/ <comment><USER2> <http://EPost.ca> is the same, but run by the Canadian gov't.

</comment><comment><USER3> A friend of mine and I had an idea for this in 2001, and even made it into an incubator. But everything folded shortly thereafter due to the bubble bursting. The company probably deserved to die.

The main market is traveling salesmen and businessmen who are away from home for extended periods of time. If you're in China 3 weeks every quarter, for instance, a service like this is immensely valuable. But it became obvious to us that it was a niche product, and while cool, wouldn't really fly. And it is definitely not worth a $13.3 million investment.

</comment><comment><USER4> Read the comments - the founder left a couple of very nice replies at the bottom of the page.

It seems like a lot of people (Arrington especially) don't bother to understand the idea or the market.

To me it looks like a fine business, and the success so far speaks for itself.

</comment><comment><USER5> This is difficult to believe. People are worried about privacy with GMail or with giving Mint their bank info - but are ok with some company looking at their postal mail? Bank statements, tax docs, SS statements...? Seriously?

<comment><USER6> You would probably want to send the sensitive stuff to your own address and not them.

</comment></comment><comment><USER7> This exact startup idea is the canary in the mine for investment bubbles :-)

</comment><comment><USER6> That's a pretty cool idea.</comment>


Building a Search Engine from Scratch - <AUTHOR> https://0x65.dev/blog/2019-12-06/building-a-search-engine-from-scratch.html <comment><USER3> That Cliqz is trying to actually build a new search stack is commendable. This is way more exciting to me than DuckDuckGo and other services that just package up Bing search results under different branding.

I'm skeptical that they'll be successful, but I wish them the best. They should market (and engineer) strongly on privacy since that's where Google is weak.

<comment><USER4> « _DuckDuckGo and other services that just package up Bing search results_ »

Had to look that up. I found [https://help.duckduckgo.com/results/sources/](https://help.duckduckgo.com/results/sources/) Bing is just one of "hundreds of vertical sources delivering" results to DuckDuckGo.

<comment><USER5> Bing is the primary source though.

<comment><USER6> I'm not sure that's true. If you search the same query on both Bing and DDG, you'll see pretty different results.

<comment><USER7> I played with Yahoo BOSS a lot as a undergrad and I could tweak it to get "better" results than Yahoo Search for certain queries. That Bing and DDG have different results doesn't really say anything.

</comment></comment></comment></comment></comment><comment><USER8> Cliqz still hasn't given a good answer as to why they use what's seemingly a typo in 'analysis' to get around user tracker blockers.

[https://anolysis.privacy.cliqz.com/](https://anolysis.privacy.cliqz.com/)

As you can see in this subthread, they claim it's "anonymized" analytics,

[https://news.ycombinator.com/item?id=21718694](https://news.ycombinator.com/item?id=21718694)

which, as 99% of research suggests, isn't anonymous at all:

[https://www.fastcompany.com/90278465/sorry-your-data-can- sti...](https://www.fastcompany.com/90278465/sorry-your-data-can-still-be- identified-even-its-anonymized)

<comment><USER9> [Disclaimer: I work at Cliqz]

Hi, I read the thread and thought the answer was good enough, but it seems that you are not yet convinced. Let me try:

1) Here there is a list of publications regarding privacy by Cliqz (including published scientific papers). It should have fairly easy to find it using a search engine :-) [https://0x65.dev/pages/dissemination- cliqz.html](https://0x65.dev/pages/dissemination-cliqz.html) Hopefully, the paper will convince you that Cliqz privacy commitment is serious.

2) Feel free to monitor your own traffic to see whether or not we are tracking you.

3) Honestly, if someone tells you that anolysis means anonymous + analysis, why do you not believe it? It does not take long to find references of the name on the source code. On a separate note, as a company (Cliqz) that offers anti-tracking and ad-blocking, I can tell you that blocklists are a bit more sophisticated than that.

Hope that this will address your concerns,

[comment edited: why do you _not_ believe it?]

<comment><USER8> A quick skimming of those PDFs found no mentions of "anolysis." Your colleague claimed that papers were going to be released "soon" on it. It's been at least two years since you started using it, so why hasn't there been yet?

 _It does not take long to find references of the name on the source code._

No references in any of your published source code, and your search engine isn't free software:

[https://github.com/search?q=org%3Acliqz- oss+anolysis](https://github.com/search?q=org%3Acliqz-oss+anolysis)

 _Honestly, if someone tells you that anolysis means anonymous + analysis, why do you not believe it?_

Cliqz has done unsavory things in the past (like the Firefox fiasco a few years back, for example, which I can't fault Cliqz entirely for: Mozilla is just as guilty).

 _On a separate note, as a company (Cliqz) that offers anti-tracking and ad- blocking, I can tell you that blocklists are a bit more sophisticated than that._

"anolysis" gets around both uBlock Origin and uMatrix, despite both of them automatically blacklisting any URL with "analytics" in it, as an example. Getting around the most popular content filterers on the internet is a pretty strong signal.

<comment><USER10> > No references in any of your published source code, and your search engine > isn't free software:

You looked at the wrong tab, check in "Code" to find "code" related to Anolysis: [https://github.com/search?q=org%3Acliqz- oss+anolysis&type=Co...](https://github.com/search?q=org%3Acliqz- oss+anolysis&type=Code)

> Cliqz has done unsavory things in the past (like the Firefox fiasco a few > years back, for example, which I can't fault Cliqz entirely for: Mozilla is > just as guilty).

Not sure how this is Cliqz' fuck-up. We are not hiding anything. On the contrary we are very transparent and detailed about how everything we do is designed to not track users. All of this is on our new tech blog: [https://0x65.dev](https://0x65.dev), feel free to have a look between two comments on HN and give us some feedback!

> "anolysis" gets around both uBlock Origin and uMatrix, despite both of them > automatically blacklisting any URL with "analytics" in it, as an example. > Getting around the most popular content filterers on the internet is a > pretty strong signal.

It's not called "getting around it" when there is no tracking or ads going on (if you want to see how smart the "most popular content filterers are" check out this link and see that the image is blocked because it contains the substring: "analytics": [https://whotracks.me/blog/private_analytics.html](https://whotracks.me/blog/private_analytics.html). Wicked smart!).

Anolysis is not a typo, it's a project name, people tend to do that when they care and spend a lot of time on projects: give them names. So, at the risk of repeating myself, Anolysis = Analysis + Anonymous (at the time we thought it was a pretty neat name!).

Anolysis does not operate outside of Cliqz products (no websites analytics here and we do not rely on a third-party, we built it in-house for this reason) and we put a lot of work into it to make sure it does not use a unique ID (like virtually every other analytics out there) but allows to by-design not track any single user (in fact the system does not even have the concept of a user). Sure, we did not write extensively about it but I guess we have to start somewhere (in December we are writing on 24 different things we do, we will be sure to consider Anolysis as a good candidate for a technical blog post in the future).

What you attribute to malice is simply a lack of time, as you probably noticed Cliqz is working on solving _a lot_ of very hard problems (search, browsers, antitracking, adblocking, privacy-preserving telemetry and so much more) and writing a paper about the new system you designed and implemented is not always the priority :)

</comment></comment></comment></comment><comment><USER11> I don't see how this will ultimately be successful.

They're basically reverse engineering Google by looking at user logs.

Google will always have a leg up here because they have all the Google data.

And even if it does work for a while, there still needs to be the original signal to copy. Someone will have to crawl the web and index content.

I'm super eager to find new approaches to search, but another Google clone is not that.

<comment><USER12> The article describes techniques used by all search engines, not just Google. Search engines have existed before Google, and despite Google's monopoly on search, "Google clone" is a poor term to describe all search engines when alternatives with unique features (e.g. DuckDuckGo) exist.

<comment><USER11> They are literally rebuilding Google by tracking how users use Google and rebuilding the SERPs.

If that's not a Google clone I don't know what is.

<comment><USER13> All search engines have search-engine results pages.

They're looking at how users use Google Search because the data's there. They're making a competitor to Google Search. That doesn't mean they're rebuilding Google Search's SERPs, or making a Google Search "clone"; I've got results from Cliqz for queries I'm confident have never been put into Google before, meaning it's functioning as an independent search engine.

<comment><USER14> >They're looking at how users use Google Search because the data's there

This. Having worked in past life for one of their competitors, can confirm - what users click on (in SERP) is one of the most powerful signals for ranking. And who got (almost) all the clicks in the world? Google!

That's why it's so damn hard to beat them. It's the unreasonable effectiveness of data: more data (which they have almost all of) usually beats a smarter algorithm, and with 20 years R&D, theirs is surely not dumb.

Do the clicks belong to users or Google, that's an interesting question, though.

</comment></comment></comment></comment></comment><comment><USER15> I'd never heard of Cliqz before, but just did a couple of test searches, and I'm honestly super impressed with the results. I found the result relevancy seemed to be closer to Google than DDG/Bing

<comment><USER16> Maybe so, but honestly, the name is putting me off more than anything. It just doesn't sound professional, and causes me to perceive it as shady, even if it's not. The name also makes it sound like it's more about marketing "clicks" to advertisers than providing good results. None of that is necessarily true, but it's the impression the name gives. It needs to re-brand.

<comment><USER17> It's the one time where I can honestly say: The discussion we're having within Cliqz about our brand name are even more heated and controversial than here on Hacker News (and Reddit for that matter) ... but then there is this saying: "Every brand name is shit until you surpass one billion users - than it becomes brilliant". More seriously: we do think about it a lot - happy to get ideas.

<comment><USER18> How are there even two sides to the discussion internally? I'd sooner cut off my left nut with a spoon than trust any company named 'Cliqz.'

You might as well name yourself _" Nigerian Princes Inc."_

<comment><USER19> So what you are saying is that you trust Google more than a German/European company, where the laws are way more strict.

Enjoy 1 nut.

( Extreme comment fyi )

<comment><USER18> I don't have to chose between the two. I trust neither.

</comment></comment></comment></comment></comment></comment><comment><USER20> I really just wish exact match search still worked. But now, words are all vectorized as every search engine tries to determine "my intent", resulting in a deluge of fuzzy matches.

Maybe I'm old school, but I don't want software that fixes my spelling mistakes. I want software that fails when I make a mistake.

</comment><comment><USER21> Honestly, what would be interesting is if there was an open source database of crawled webpages, available for anyone to search / use with their own algorithms. That would make it possible for... a lot of things, really.

I feel like the web parsing / indexing, perhaps rather than the search algorithm itself, is the hardest part of rolling a new search engine (largely due to the associated computing and storage costs).

<comment><USER22> There is [https://commoncrawl.org/](https://commoncrawl.org/), but it would be really cool if there were a more well-lit path towards building the rest of a simple search engine. For example, another commentator wanted "like Google, but without the spelling correction", well, spin up one of these and just stub out the spelling module :)

<comment><USER21> Thanks for the tip! I'll have to play around with that sometime.

</comment></comment></comment><comment><USER23> Interesting approach, certainly very different than what I'm used to as someone working on search.

Will probably keep an eye on this blog.

</comment><comment><USER24> If you're listening Cliqz, I (and I believe others) would pay $50-100 a month for a search engine that gave me the ability to blacklist sites. I think this alone could solve the biggest problem I have with google which is the reversion to the (what appears to be regressive) mean of the internet today.

<comment><USER25> There is a browser extension called personal blocklist for this purpose.

</comment></comment><comment><USER5> One immediate observation is non English porn filters / safe search do not work properly. I see NSFW results mixed with others for benign words.

</comment><comment><USER26> How do search engines stay out of trouble with things like copyright trolls and FBI-operated honeypots for child porn? If I had a private Web crawler, I'd be terrified to run it.

</comment><comment><USER27> Also having built a search engine from scratch, we use a similar method: [https://insideropinion.com/](https://insideropinion.com/)

In our case the "queries" are also the index creation components. Every time someone discusses something, we are indexing it, so you can search media, documents, people from context. We hint at how this works here: [https://austingwalters.com/fast-full-text-search-in- postgres...](https://austingwalters.com/fast-full-text-search-in-postgresql/)

The downside of our approach is it needs lots of conversation data. From their TLDR version:

"""

\- Our model of a web page is based on queries only. These queries could either be observed in the query logs or could be synthetic, i.e. we generate them. In other words, during the recall phase, we do not try to match query words directly with the content of the page. This is a crucial differentiating factor – it is the reason we are able to build a search engine with dramatically less resources in comparison to our competitors.

\- Given a query, we first look for similar queries using a multitude of keyword and word vector based matching techniques.

\- We pick the most similar queries and fetch the pages associated with them.

\- At this point, we start considering the content of the page. We utilize it for feature extraction during ranking, filtering and dynamic snippet generation.

"""

It appears 0x65 has similarly figured this out, the name of the game is forming proper search queries. In their case, their results would be good as soon as they start indexing and create synthetic queries. IMO might be better for documents and what not.

Either way, interesting to compare notes! Kudos to the work.

<comment><USER28> I remember reading your article on FTS in postgres, great stuff. Was wondering what strategies you might be using to perform counts on your data?

I'm trying to implement a faceted search in postgres and currently using window functions to count subcategories (a la [http://akorotkov.github.io/blog/2016/06/17/faceted- search/](http://akorotkov.github.io/blog/2016/06/17/faceted-search/)), but not sure if it's the most efficient.

<comment><USER27> Depends what you want to do, I created a "estimate_count" function to make it much much faster:

"SELECT planrows FROM estimate_row('SELECT COUNT(*) ON table WHERE XXX')"

<comment><USER28> That's actually brilliant.

If you're ever looking for something to write about for a new blog post, I would love to learn more about how you implemented that estimate_count function.

Thanks for the tip in the right direction tho!

</comment></comment></comment></comment><comment><USER29> We've had at least 2 posts from cliqz in the past few days. I have genuine issues with my short-term memory after having recovered from a coma so I don't know whether this is a glitch where I keep seeing the same posts or they keep getting reposted.

<comment><USER30> Your short-term memory is fine in this case, and I hope that is part of a complete recovery.

They're doing an "Advent Calendar" series where they're posting one a day:

[https://news.ycombinator.com/item?id=21676252](https://news.ycombinator.com/item?id=21676252)

[https://news.ycombinator.com/item?id=21684708](https://news.ycombinator.com/item?id=21684708)

[https://news.ycombinator.com/item?id=21694980](https://news.ycombinator.com/item?id=21694980)

and [https://news.ycombinator.com/item?id=21716860](https://news.ycombinator.com/item?id=21716860) (not even a day ago).

This is a problem for HN because users here are not used to this sort of repetition—indeed, we moderate HN explicitly to dampen repetition, because the point of the site is curiosity and curiosity withers under it ([https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...](https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20withers&sort=byDate&type=comment)). The more these posts show up with what for HN is a crazy frequency, the more likely users here are to experience it as a barrage and start to complain.

On the other hand, these articles are well-crafted, contain a lot of information, and would normally be fine HN submissions. The topic of building a new search engine is intrinsically interesting. It also resonates with a lot of themes that get discussed a lot on HN (concerns about big tech and so on). So this is a different situation than the usual marketing onslaughts that HN gets subjected to, where the content is crappy, users flag it away, and moderators squash what users missed.

I'm not sure what to do about this yet.

</comment></comment><comment><USER31> >> "Our model of a web page is based on queries only. ... This is a crucial differentiating factor – it is the reason we are able to build a search engine with dramatically less resources in comparison to our competitors."

>> "The total size of our index currently is around 50 TB."

Could you share your current index size (number of pages, size of raw text) to put those 50 TB into perspective in order to get an idea how much less resources in comparison to your competitors you need? This would help to compare your approach to Elasticsearch, Solr, Lucene

</comment><comment><USER32> But is Cliqz a search engine, or is it a web browser with a search bar set to a default search engine (or a Firefox extension that accomplishes the same thing).

Cliqz bought Ghostery to acquire a pool of privacy-conscious users. The goal is to show them ads. Not sure how excited they will be about that.

If Cliqz really is a search engine, can a user submit a query to the database using her own choice of tcp/http client. It looks like submitting requires first downloading and installing software from Cliqz.

<comment><USER33> [https://beta.cliqz.com](https://beta.cliqz.com)

<comment><USER17> Or via Tor browser as Onion service: search4tor7txuze.onion (quite unique - <USER2> available it will show onion services instead of www domains).

</comment></comment></comment><comment><USER34> Do you use Common Crawl? It seems like a pretty big corpus, reasonably up-to- date and free. It seems like a good way to supplement the page data from the browser extension.

<comment><USER35> From the article:

"It may seem like Common Crawl would suffice for this purpose, but it has poor coverage outside of the US and its update frequency is not realistic for use in a search engine."

</comment></comment><comment><USER36> 50TB on localhost is awesome! I wonder what the amount of new content created was between 2018 and 2019.

<comment><USER37> But I doubt you really need it all in RAM for a small localized single user search engine. In that case you back most of the index with NVMe which at the prices I saw on black friday is probably less than $10k using consumer grade QLC flash drives when combined with cheap x1 pcie expansion boards/etc.

The 10PB of disk is also quite reachable given its possible to buy bulk 10T disks at $150 each.

Bottom line, I did some of these calculations a couple years ago because I was interested in a topic based search engine that only indexed for certain topics and basically tossed any crawler results that didn't appear to fit the subject matter.

So, while the web is a lot bigger than when google started, storage and compute is also a lot cheaper. A web search engine that specialized in say cooking recipes might be entirely doable on a fairly limited budget.

</comment></comment><comment><USER38> Can someone comment on how to use knowledge graphs for search? I have seen some applications in NLP but I am curious how it can tie in with traditional search.

<comment><USER39> AFAIK, in Google's case instant answer cards come from the Google's knowledge graph, not search results. I.e. if you see some info rendered on top of search results or on the side, most likely it's from the knowledge graph.

</comment></comment><comment><USER40> Definitely interesting.

But...my searching has gotten to the point where well over half the time I am no longer looking at Google's conventional search results.

<comment><USER41> could you explain that a bit more? What are you using instead?

<comment><USER40> Just all of Google's "magic box" results. And a lot of searching is in the Chrome/Firefox/Safari address/search box.

</comment></comment></comment><comment><USER42> Strangely, cliqz.com was blocked by my pihole.

<comment><USER43> Hi,

Thank you for bringing this up. Although, this is not relevant in the context of the blog post, we are on that list by mistake. We do NOT collect any personal data in our browser: (more details e.g., here: [https://0x65.dev/blog/2019-12-02/is-data-collection- evil.htm...](https://0x65.dev/blog/2019-12-02/is-data-collection-evil.html) and [https://0x65.dev/blog/2019-12-03/human-web-collecting- data-i...](https://0x65.dev/blog/2019-12-03/human-web-collecting-data-in-a- socially-responsible-manner.html)) and we go a long way to make sure not even implicit indentifiers go through. We believe we ended up on that list for a bad Firefox experiment and we will reach out to the maintainers, make our case.

Disclaimer: I work for Cliqz.

</comment></comment><comment><USER44> [https://github.com/Qwant/](https://github.com/Qwant/)

</comment><comment><USER45> google's suggested autocorrect is one it's most impressive features; idk I'd say the relevance of the search results almost comes in a near second to that.

so make a competitive "suggested autocorrect" solution and then I think you'd have a stew going.

<comment><USER46> There is an autocorrect feature and it's even described in the article. Or you mean something else?

<comment><USER25> They mean auto complete.

<comment><USER46> Here is the quote from article "This not only involves some normalization, but also expansions and spell corrections, if necessary. "

</comment></comment></comment></comment><comment><USER47> Might put a link to actual search engine on the article. Good article though !

<comment><USER46> [https://beta.cliqz.com/](https://beta.cliqz.com/) is EP for search engine

</comment></comment><comment><USER48> Is the Human Web dataset available for download?

<comment><USER49> Data release, it's not possible, but if people want to come and do experiments on the data or try to test it for privacy, we are more than welcome to host them. There is no formal process in any way, best effort, we have done several times in the past. If you are very interested contact us and we will see if we can accommodate you.

[Disclaimer: I work at Cliqz]

<comment><USER50> (Disclaimer: I work at Cliqz) Extending on that, let me elaborate why we cannot open the data, not even a subset of it. We had the discussion in the past, but for two reasons it is not an option.

Although it is anonymous data - currently we are not aware of any de- anonymization attacks - it is still data that came from real persons. We have a responsibility: once the data is out, we have to guarantee that no-one will ever be able to identity a single person in the data. Take also in account that attackers can combine multiple data sets (Background Knowledge Attacks); that even includes data sets that will be published (or leaked) in the future.

You should never be too confident when it comes to security, neither should you underestimate the creativity of attackers. What we can do - and did in the past - is to simulate the scenario in a controlled environment by hiring pen testing companies. If they would find an attack, they will not use that knowledge to harm the persons behind the identities that they could reveal.

That is the main reason. We don't want to end up in a situation as AOL or Netflix when they published their data. By the way, Netflix is an example of a background attack where they needed to combine data sources.

There is also another argument. Skeptics will most likely remain skeptics, as we cannot proof that we did not filter out data before publishing. In other words, there is nothing to gain for us, we can only loose. Trust is important, but for building trust, it is better to be transparent about the data that gets sent on the client. You can verify that part yourself and do not have to rely on trust alone. That is the core idea behind our privacy by design approach.

Those are the arguments that I'm aware of why we will not open the data. However, getting access in controlled environments is possible. If you doing security/privacy research, you can reach out to us. In my opinion, having more people that will try to find flaws in our heuristics is useful. That gives us a chance to fix it before it can be used for attacks.

One notable exception: [https://whotracks.me](https://whotracks.me) is built from Human Web and all its underlying data can be freely downloaded. We know that it has been already used for research.</comment></comment></comment>


S3concurrent, yet another S3 syncing project - <AUTHOR> http://www.winstonchen.co/2015/09/introducing-s3concurrent-yet-another-s3.html <comment><USER2> How does its performance compare to aws s3 sync? Note that you can adjust max_concurrent_requests which defaults to 10.

<comment><USER3> I'm always surprised by the number of "download from s3" tools there are. AWS- CLI does a really quite nice job of it.

<comment><USER2> To be fair, here is a tool that has features no other tool I know: [https://github.com/rlmcpherson/s3gof3r](https://github.com/rlmcpherson/s3gof3r) It can saturate your bandwidth and you can pipe to/from it. It's great to deal with hundreds GB's archive on the fly. "Examples: $ tar -cf - /foo_dir/ | gof3r put -b my_s3_bucket -k bar_dir/s3_object $ gof3r get -b my_s3_bucket -k bar_dir/s3_object | tar -x

It's written in golang. So it's a simple binary.</comment></comment></comment>


Ask HN: Who is the equivalent of 37signals in Korea? - <AUTHOR>

Do they exist yet?  My aunt runs a design firm in Seoul.  She tells me the business climate there is a lot less friendly to freelancers. <comment><USER2> In what aspect do you seek similarity? Project management tools, company philosophy, company with a book or two, company with a design blog?

37signals rightly makes designers addicted to their culture, but if you ask me, I don't like their products.

I bet there are tons of companies in Korea whose products I would not like.</comment>


First Pass at Palin Emails by Opani - <AUTHOR> http://opani.com/help/sarah-palin-email

<comment><USER2> Here's a first pass at auto-discovering topics in that text using LDA (Latent Dirichlet Allocation) in R: <http://opani.com/ryan/sarah-palin-email- topics/148845806827/>

The biggest deficiency now is the stopwords list is not working 100% so mundane topics are creeping in. I'll have limited time to play with it in the next few days, so feel free to make some cool discoveries. :)

</comment><comment><USER3> Awesome work, but I find it strange that all of the emails in the example post are @yahoo.com.

<comment><USER4> This has been explained in the news by a number of journalists -- I believe it is because, Palin used her yahoo email account to discuss "off-the-record" thoughts with various members of her administrative staff. However, her dumb ass didn't realize that by sending from a yahoo account to staff email accounts, makes those received emails subpoena-able for public review. So the folks requesting the release of email records, were also able to get inbound email records from staffers received from Palin's yahoo as well as emails sent from staffer personal email accounts that were inbound to Palin's non-personal account. What is missing from the entire data set would be emails from Palin's yahoo account to staff personal accounts and vice versa.

As one journalist I saw on the news discussing this issue, what's interesting is the stuff that falls between the lines, i.e. redacted or missing from certain conversations that might be referred to.</comment></comment>


London tops list of most expensive cities in which to live and work - <AUTHOR> http://www.independent.co.uk/news/business/news/london-rio-hong-kong-sydney-new-york-expensive-cities-live-work-rent-a6905136.html <comment><USER2> I guess that this will have very bad consequences. For example, is very dificult for London to attract talent. People prefer Berlin or Barcelona, rent is cheaper.</comment>


How to be an Angel Investor...and make money (Don Dodge) - <AUTHOR> http://dondodge.typepad.com/the_next_big_thing/2010/06/how-to-be-an-angel-investorand-make-money.html

<comment><USER2> Philip Greenspun supplies his own (non-PollyAnna) perspective: [http://blogs.law.harvard.edu/philg/2010/06/01/boston- angel-i...](http://blogs.law.harvard.edu/philg/2010/06/01/boston-angel- investors/)</comment>


Ask HN: Where to learn about driving engineering change in large organizations? - <AUTHOR> I've helped establish best practices and healthy engineering cultures in greenfield scenarios, when I was the first technical hire in an early stage environment. I've also come into a small company with significant technical debt and introduced changes to the engineering practice in ways that were embraced by the legacy team. But I've never had the chance to reform engineering practices and culture at a very large organization, I'm considering it as a path for professional growth, and I'm looking for ways to learn more about doing it before I dive into the deep end.
I'm talking about a software company with dozens or even hundreds of engineers, with large amounts of technical debt, that has a giant, monolithic codebase that's highly-coupled and poorly tested, a release process that is slow and brittle. And in this scenario, I either lack the authority to dictate global change or I think that an authoritarian approach would be inappropriate and ineffective, or I have no authority at all and have to drive change locally, from the bottom up.
If I didn't know a lot about what best practices were, I'd be reading things like Clean Code. If I didn't know a lot about strategies for approaching technical debt, I'd be reading things like  Working Effective with Legacy Code. If I didn't know a lot about leading a small team in general, I'd be reading things like Radical Candor. But none of those quite capture the problem I'm describing.
Can HN recommend any books, blogs, videos, podcasts, personalities I should be looking at? <comment><USER2> > I'm talking about a software company with dozens or even hundreds of > engineers, with large amounts of technical debt, that has a giant, > monolithic codebase that's highly-coupled and poorly tested, a release > process that is slow and brittle.

I think you may have a flawed view of large software organizations. While I'm sure some companies exist that match the above description, I've more typically seen numerous siloed teams, with their own codebases, with both loose and tight coupling to other teams, with a variety of processes, and therefore a variety of quality around the teams.

So it is not as simple as just picking a universal pain point and transitioning to a new process. It is about understanding the drivers and history that got each team to where they are, picking apart which of those drivers is still relevant, and looking for shared pain points between teams that are now open to change due to changes in the business since things were set up.

Because of this, I'd focus less on the technical side, and more on communications and business, as those are the rabbit holes you'll need to go down to build consensus from multiple teams to mutually move to a better place.

</comment><comment><USER3> For personalities try Jason Friedman. Martin Fowler's Refactoring is a nice read too.

But personally, I'd like to ask what business goal are we looking to solve? Because regardless of how murky a codebase can be, if that codebase keeps hitting a business goal, then we really can't drive a change in practices.

So, we first present business goals. Here's a few examples:

If the business goal is to give better estimates for fulfilling custom features faster, then testing would be the first in the list. Not even the release process. Tests lessens cognitive load for developers when they're asked "how long can we make this feature?". They'll have an idea that it'll probably break a bunch of services, etc. a.) If you're in leadership, ask for tests when people make a PR. b.) If you're not in leadership, look for the smallest part of the monolith. Write 1 integration test, add a CI integration, put it in a PR. Co-workers would probably like that.

If the business goal is to give better SLA's (turn around time for addressing breaking bugs, scaling better), then the release process would come first.

Regarding technical debt: One way to address technical debt is to slowly address it by folding in the effort of cleaning and refactoring inside a feature request. ex: making a new form about 2FA takes 3 days, but you can say 7 and explain that we'll setup a CI test build, then hit and address some debt regarding User Authentication along the way.

It'll introduce as a "Clean As You Go" culture in the legacy engineering team, which will pay off in the long-run instead of having a huge amount of time just dedicated to addressing technical debt.

The danger of separating "address technical debt" as a separate effort is it sometimes results into some unnecessary abstractions and over-refactoring: i.e. trying to make certain things too generic, bordering on building a framework.

But then again, if the business goal was to make a framework for a business goal (maybe something the bizdev can sell), then it can definitely be made as a separate effort. Otherwise, it'll be seen in the bizdev as someone just "playing" around with tech.

</comment><comment><USER4> I'd suggest you start with identifying and quantifying the areas of improvement. Tools to analyze code quality, unit test coverage and defect leakage could give a sense of what the problem areas are. From there, you can build a consensus on the to-be state. This apprach would shine light on the relevant problems and would reduce resistance.

</comment><comment><USER5> This isn't quite what you're looking for, but you may find some inspiration in The Phoenix Project (Gene Kim, George Spafford, Kevin Behr).</comment>


Parametric Polymorphism aka Generics in Scala [video] - <AUTHOR> https://www.youtube.com/watch?v=ufm73YTP4Hw&feature=youtu.be <comment><USER2> We've updated the link from [https://youtu.be/ufm73YTP4Hw](https://youtu.be/ufm73YTP4Hw), which is banned because the guidelines ask us to submit original sources.

<comment><USER3> @agilesteel: Just to be more clear, don't use the "minimized" version youtube usually provides and links to the youtu.be domain, remember to always post here the version that links to the full www.youtube.com domain.

[https://youtu.be/ufm73YTP4Hw](https://youtu.be/ufm73YTP4Hw) ==> [https://www.youtube.com/watch?v=ufm73YTP4Hw](https://www.youtube.com/watch?v=ufm73YTP4Hw)</comment></comment>


Piano Media wants national paywalls all over Europe - <AUTHOR> http://www.niemanlab.org/2012/01/piano-media-wants-national-paywalls-all-over-europe/

<comment><AUTHOR> While the headline – 'national paywalls'?!?! – is almost certain to provoke misunderstanding and negative reactions, there's actually something interesting here.

It's a multi-site all-you-can-view subscription, with the revenues partitioned by traffic and success in signing up new users. So it's in the same vein as once-discussed-but-now-gone 'Contenture'.

There needs to be something other than AdSense/display-advertising to support quality online content; simple plans like this could be a help. Though I suspect this service could tend over time to become concentrated into one or a few giant roll-up providers.

To that end, Google has a little-discussed similar offering as well: 'OnePass' <http://www.google.com/landing/onepass/></comment>


XBill – prevent Bill from installing "Wingdows", a game - <AUTHOR> http://xbill.org/ <comment><USER2> While you're at it.

[http://www.bloodbathsoftworks.com/xylemon/projects/xlennart....](http://www.bloodbathsoftworks.com/xylemon/projects/xlennart.php)

"XLennart is an xBill modification about a hacker named, "Lennart" who has created the ultimate computer virus that is cleverly disguised as a popular init system. XLennart is commentary on a certain Linux/Unix topic, but I'll let you figure out which one."</comment>


The Enemy Within - <AUTHOR> http://www.theatlantic.com/magazine/archive/2010/06/the-enemy-within/8098/ <comment><USER2> "If the right order were given, and all these computers worked together in one concerted effort, a botnet with that much computing power could crack many codes, break into and plunder just about any protected database in the world"

Uh, no, it couldn't. A big botnet does not magically give you access to well- protected computers or networks. See <http://en.wikipedia.org/wiki/Brute_force_attack> for some perspective.

I don't care how big Conficker is, it's not going to break AES-256 any time soon (or really any large key based on an algorithm without a weakness).

Similarly, Conficker isn't going to be able to get into protected internal networks with a good firewall simply by virtue of having lots of machines.

<comment><USER3> Naturally, it wouldn't necessarily be able to enter _any given_ secure network, database or machine.

But it quite possibly could enter _quite a lot of them_ ... if the conficker masters are indeed as cryptographically sophisticated as the article describes.

Just imagine the attack on Google supplemented by a botnet. Modern attackers already have many vectors with which to attack private networks. Having a botnet can only give an attacker more opportunities.

So I think it's entirely correct to say the virus is worrisome at the least.

<comment><USER4> Are the Conficker masters really as cryptographically sophisticated as the article describes? They implemented an encryption algorithm that had been submitted as a SHA-3 candidate, _complete with its flaws_ , then updated it to patch the flaw only after the author of the algorithm corrected his own work.

Similarly, they exploited a flaw in Windows only after Microsoft released a security update describing the bug.

The Conficker creators are certainly paying attention and taking advantage of the right opportunities, but they're not quite the super-genius polymaths that the article is making them out to be.

</comment></comment></comment><comment> brown9-2 Previous HN discussion of the same article: <http://news.ycombinator.com/item?id=1350320>

<comment><USER5> Thanks for this. I went through @tptacek's comment, but I don't see any of the quotes in the article. Am I missing something?

<comment> brown9-2 They're all there, check the print version: [http://www.theatlantic.com/magazine/print/2010/06/the- enemy-...](http://www.theatlantic.com/magazine/print/2010/06/the-enemy- within/8098/)

</comment></comment></comment><comment><USER6> Anyone know a good, friendly, easy-to-deploy conficker killer? I think I have it on an old windows machine, and all the AV products are so fucking STOOPID that they do shit like refuse to install in safe mode.

</comment><comment><USER7> Just a nitpick:

"Ficker" is not "motherfucker" in german, it would loosely translate to "fucker", but probably wouldn't be used as a word.

There's no sensible translation for "motherfucker" in German. Literally it would translate to "Mutterficker", but would never be used as a word.

Else then that small nitpick: Interesting read.

</comment><comment><USER8> If I seriously wanted to crack something difficult, I wouldn't recruit more computers; I'd recruit more great people. A concerted effort by a small group of talented people, joined by the Internet is more effective than brute force will be, especially if the first plan is to brute force current encryption.

<comment><USER9> Millions of cores would be useful to a handful of "bright" people. It would not hinder them at least.

</comment></comment><comment><USER10> See also: <http://mtc.sri.com/Conficker/> <http://mtc.sri.com/Conficker/addendumC/>

</comment><comment><USER11> That was a very engaging read.

<comment><USER12> Yeah nice layman's description of the technical stuff without too many drastic oversimplifications or misrepresentations.</comment></comment>


Show HN: Hosted IRC for Teams (my first startup) - <AUTHOR> http://teamrelaychat.nl/

<comment><USER2> There is an opportunity for IRC and XMPP protocols to co-exist in the same group chat and solve customer needs better.

One of our customers is a YC company, that moved their group chat from a IRC based system to our XMPP based group chat product[1], because IRC did not work well for them. After they started using us, we figured that some people still preferred IRC and some wanted the ease of use of XMPP. The solution was a simple bot that will bridge the messages between the IRC channel and the group chat. This is now working out very well for them.

They now have benefits of both IRC and XMPP on their website. I'll be happy to talk more details and possibly collaborate. My email in profile.

[1] - <https://groups.gaglers.com/>

<comment><USER3> You can also have an irc interface to xmpp group chats using bitlbee.

</comment></comment><comment><USER4> Perhaps there is something to learn from grove.io (I'm not saying it's a bad idea, only that you should learn more about what didn't work for them -- which you presumably already have)

<https://grove.io/blog/grove-shutting-down-october-13>

<comment><USER5> Pownce, Convore, Grove, all shared similar fates, I think its more indicative of the founder not the business.

<comment><USER6> Personal opinions of said founder aside, all three were bringing arguably trivial products to market where there already was a large presence of two or more players dominating (Twitter/Facebook/status.net for Pownce, HipChat/Campfire for Grove) and with a lack of game-changing innovation to separate them.

Makes less sense to blame the people, rather than lay blame to what they did (or did not) do.

<comment><USER5> > bringing arguably trivial products to market ... with a lack of game- > changing innovation to separate them.

It sounds to me like you are judging the founder. Not that I would disagree with you.

Don't get me wrong, I think Leah is awesome. She just builds up companies for soft landings/aquihires and pockets some money along the way. There is no reason a viable company couldn't exist on the grove model.

</comment></comment></comment></comment><comment><USER7> I need it to be dead simple for people to join the chat, using existing chat clients. i.e. whatever they have installed, I can't ask them to install something new.

What I mean is this: A lot of people I want to invite into chat, it's an experiment for them and they aren't really committed.

Oh, I have to install IRC software? Sorry, too much work.

Oh, I can't access it from AIM/gchat/etc? Sorry, too much work.

These are the people I want to onboard, so they can see the value in participating in my chat room.

What is the tool that allows me to attract anyone to the chat room, without them installing new software?

<comment><USER8> Would the best solution be a web-based client? I have XChat on all my Windows machines and use Colloquy on my Macbook so I never worry about IRC support. But it's obvious that not everybody uses IRC on a daily basis.

I would wonder if it's possible to support people using their own desktop client, while others alternatively use a webpage on the site which connects into the same IRC channel.

</comment></comment><comment><USER9> I don't want IRC. I want HipChat (inline images, ability to directly upload & share media, persistence, good searching, integrated video, HTTP API) but with non-shit native clients. IRC is a fucking dinosaur, I feel something like this doesn't have the capability to really improve the situation.

<comment><USER10> Agree - for me, the persistence is the most important, something which IRC doesn't offer. I'm not always on, and in a business setting it's crucial to be available for messages at all times, even if I'm going to actually read the messages later. I can also find out what the others have been discussing.

<comment><USER5> Check out IRCCloud, they keep a client connected on your behalf 24/7 then let you connect to it when you open the web app.

</comment></comment></comment><comment><USER11> Why would I pay for IRC?

<comment><USER12> I know right? Am I missing something here? What does this server offer _extra_ that would explain the hefty $20p/mo pricing? I can just open a free private IRC channel on any server and use that?

<comment><USER13> Describe "on any server". Like a $20/m AWS micro?

<comment><USER14> On any IRC server. There's absolutely no reason to host your own.

</comment></comment></comment></comment><comment><USER15> This looks cool. Nice job.

I like the IRC style of chatrooms as it's familiar to me, but as others have already stated, I would prefer a service that provides additional features (ala HipChat or Campfire) such as file sharing, searchable history, and allows attendees to jump on a video call. Bonus points if you allow users to join over the phone as well.

We use Google Hangouts for video conferencing with my team of 3 (all remote), but we have to keep a campfire session open for sharing files and links relevant to the discussion. Even with this set up I'm missing a way to join our ad-hoc conference calls over the phone when I'm away from my computer.

</comment><comment><USER16> Hey, this looks like it would go well with what I'm currently developing: <http://www.getinstabot.com/>. I'll give it a shot, congrats!

<comment><AUTHOR> That looks great. How does it compare to, say Hubot? That's what is running on TRC.

<comment><USER16> Well, the idea is that it will provide more functionality without the hassle of hosting it yourself. As to the specific commands, Hubot has more, at the moment, but Instabot's list is growing.

<comment><USER17> Perhaps creating a thin layer of middleware so that your work supports hubot scripts is a good idea, then!

<comment><USER16> Yep, but hubot scripts are already very short and easy to implement, so it shouldn't be much trouble either way!

</comment></comment></comment></comment></comment><comment><USER18> It's a bad idea to use "I'll" and "Forever" in the same 6" radius of each other.

<comment><USER19> "I'll publish the deploy scripts and help you migrate."

This might actually be useful sooner rather than later.

</comment></comment><comment><AUTHOR> I'm in here and on the demo server to answer questions and take feedback.

Word of advice: don't all pick demo1.

<comment><USER20> If you control the IRC daemon is there any reason we can't have full server- side logging of all channels and 1:1 interaction in a similar way to HipChat, et al?

</comment></comment><comment><USER21> How does it compare to grove for example?

<comment><AUTHOR> Good question. They are obviously similar in that they are both IRC and more. The key is in the 'more' part.

One thing I'm working on is file sharing. This already works in the web client. Desktop is going to be tricky, involving DCC probably.

Currently my search/history story is not so strong, but that's also being worked on: <https://github.com/znc/znc/pull/325>

</comment></comment><comment><USER22> Congrats! It'd be cool if I could collect uploaded files and logs into S3.

<comment><AUTHOR> You can! Well, sortof.

On the web client you can upload files to S3. Logs are stored on local disk though.

Web interface for logs is being worked on: <https://github.com/znc/znc/pull/325>

</comment></comment><comment><USER23> Why is it soo expensive? Otherwise relatively good idea.

<comment><AUTHOR> Because you get a full VPS, maintained by me.

At 10 people, it's already cheaper than other services, which usually charge a few bucks per user.

<comment><USER24> Your pricing is what failed it for me. I can get a 512MB VPS from DigitalOcean for $5/month.

I love this idea, though. I just feel that your approach of giving a full VPS is a bad idea. People who want a full VPS will buy a full VPS for much cheaper than the 20GBP you're offering it for.

What you need to do is multi-tenant your hosts, and isolate them from each other. Then you'll get costs savings that you can pass on to your customer; which will put you in line competitively with the likes of HipChat and Campfire.

Great idea, can't wait to see what comes of it!

<comment><USER25> The cheapest Campfire plan is $12 per month and the second cheapest is $24. HipChat is $2 per user per month. 20 euros per month with no user limit is entirely in line with this pricing.

</comment></comment></comment></comment><comment><USER26> Can I search chat history with this tool?

</comment><comment><USER27> What are the advantages of this over say a hosted jabber or lync system?

also what authentication options are available I can't seem to find a mention on the site

</comment><comment><USER28> Personal projects are not startups. I don't know where this falls in, but it bugs me when I see something where someone maybe spent a couple of weeks on it and call it a startup. Have you talked to investors? Have you quit your day job? Are you seeing startup like weekly growth?

<comment><AUTHOR> I'm pretty serious about this.

<comment><USER13> If I may ask, why? I don't personally think this is going to be successful, what with Hipchat and Campfire in the same space (which are very very popular) and grove.io having a good name for it and competing directly against you.

Edit: I think you also misunderstand the value proposition for your customers: you mentioned above that you should buy because you get "a full VPS, maintained by me", which is very odd.

<comment><USER29> There is also Flowdock[0], which wants to compete pretty aggressively here (disclaimer, I work for the company that bought Flowdock)

[0] <http://www.flowdock.com>

<comment><USER13> Yes indeed. I have a FlowDock account and my company (<https://circleci.com>) integrates with you guys.

I'm curious though: how big are you guys? I only mentioned HipChat and Campfire as they appear to be pervasive, and while I've been hearing more and more people mention Flowdock, it doesn't seem like you're an 800 pound gorilla that he should be afraid to compete against (yet!).

<comment><USER29> I don't know how much I can really say to be honest, but I do think Flowdock will compete more in the future.

<comment><USER13> Cool, looking forward to seeing it!</comment></comment></comment></comment></comment></comment></comment>


Gender Diversity - <AUTHOR> http://avc.com/2017/03/gender-diversity <comment><USER2> I don't know what to think anymore.

[http://imgur.com/a/4gr8Y](http://imgur.com/a/4gr8Y)

<comment><USER2> To be clear, I support all causes. But at what point are we reaching hypocrisy? Everything is being turned the other way and the worst part is, it's seemingly becoming ok to do so.</comment></comment>


Put your website to sleep - <AUTHOR> https://www.nightnight.xn--q9jyb4c <comment><USER2> Very cool idea! It could also somehow contributes reducing bandwidth usage on your website.

</comment><comment><AUTHOR> I'm curious how we feel as tech creators.

Do we have a responsibility to our users for their sleep?

</comment><comment><USER3> This is nice.</comment>


Advice about a questionable startup - <AUTHOR> I was asked to help a group that is developing software/hardware for the blind community. I vetted their legal paperwork, rewrote it, got them their first investor funding, and introduced+organized with nonprofits, universities, medical centers, and industry contacts (none of which they had). I set my own projects aside in exchange for an agreed amount for 1-2 months.
I only managed to have them meet with the blind community on 1 occasion and they avoided others. They also avoided many of the other contacts and ignored them.
Shortly after they cashed their first check, they split, didn't compensate me for the time I'd set my own project aside, I had to smooth things over with the owner/investor they had been staying with, and they appear to have ignored many of the groups that I'd had them talk to, testing, and further talks I'd put in place.
As it turns out, they are back at it again.
How should I deal with this group? I'm somewhat tormented by the fact that I recommended them to a number of parties in the disabled community that have been taken advantage of in the past. And, I worry that this company will continue doing pretty much the same thing. <comment><USER2> It's obvious you're disappointed with how things are proceeding. In these situations, I think it's customary that you sit down and look at what was agreed on, on paper and in email.

So which agreements were violated? And what is described as the consequences for that?

For example, you say you set your own projects aside in exchange for an agreed amount. So I assume you sent the bill for that? Did they pay it?

<comment><AUTHOR> They did not pay the full amount of what I charged them. I sent an invoice, naturally.

I am somewhat uncertain as to how to prevent them from further taking advantage of the community that they were claiming to help as well. I know there will be potential legal ramifications there depending on how I approach it.

</comment></comment><comment><USER3> For me anything associated with nonprofits is a "bad smell"</comment>


Could Russian submarines cut off the internet? - <AUTHOR> http://www.bbc.co.uk/news/technology-34639148 <comment><USER2> Who knows, but the USS Jimmy Carter certainly can...</comment>


Ask HN: Where Are the Protest Discussions? - <AUTHOR> I understand that HN tends towards apolitical content, and the benefits of that. But on a day where protests and riots are filling the streets, the silence here is eerie.
Should we not be discussing actions that we can take to join the people on the streets?
Protests are a surveillance issue. Protests elevate the voice of otherwise unheard people. These are problems which members here seem to care about regularly.
Where are these voices now? <comment><USER2> There has been lots of related discussion in the past few days: [https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=tru...](https://hn.algolia.com/?dateRange=pastWeek&page=0&prefix=true&query=Minneapolis&sort=byPopularity&type=story)

If you want to have threads like that every day, you'll probably have to look for a more specialized site that is more tolerant of repetition.

</comment><comment><USER3> Ah, you must be new here. This site is the epitome of "don't bring politics into tech".

It's a policy that really needs an overhaul for the modern world.

<comment><USER4> That is far from true and always has been (or at least since 2008). We briefly tested a 'no politics' rule in late 2016 but that was purely as an experiment just to see what would happen, and we quickly reverted it.

The issue for HN is not that politics are off topic or "don't bring politics into tech", as if these two things could be kept distinct to begin with. It's that internet forums have a strong tendency to devolve into flamewar. Political flamewar will quickly take over the site if allowed to. That would destroy HN for its mandate, which is to gratify intellectual curiosity ([https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)). So we can't just not moderate such threads.

Excluding politics is not an option either, for a bunch of reasons. One is that many of the topics that get discussed here have a strong political overlap. It wouldn't serve intellectual curiosity to exclude those.

Therefore we need a more complex approach that includes some political topics and discussions while excluding others. I've written many explanations of exactly what our approach is and how we've worked it out over the years: [https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...](https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20political%20overlap&sort=byDate&type=comment)

Some good threads to start with might be [https://news.ycombinator.com/item?id=21607844](https://news.ycombinator.com/item?id=21607844) and [https://news.ycombinator.com/item?id=22902396](https://news.ycombinator.com/item?id=22902396). Also [https://news.ycombinator.com/item?id=17014869](https://news.ycombinator.com/item?id=17014869), which shows how far back political discussion goes on HN, as well as the argument _about_ politics on HN.

If you (or anyone) take a look at those past explanations and still have a question that isn't answered there, I'd very much like to know what it is. It's been a while since I've heard a question that isn't covered by the approach we've worked out, which makes me feel like that approach is the right one for this site. There are a _lot_ of constraints on this problem, given HN's intended purpose, the dynamics of the internet, the political world, and current social trends. It's a bit of a miracle that any solution exists at all, and I have a hunch that the one we've found may literally be the only one possible.

</comment></comment><comment><USER4> See [https://news.ycombinator.com/item?id=23361912](https://news.ycombinator.com/item?id=23361912) for one answer to a similar question.</comment>


Music Science, for People Who Don't Like Music - <AUTHOR> https://whatismusic.info/blog/MusicScienceForPeopleWhoDontLikeMusic.html <comment><USER2> >Nobody actually knows why they like listening to music.

So much for glittering generalities.

"Music is a higher revelation than all wisdom and philosophy." \- Beethoven

</comment><comment><USER3> Please don't format your website to only work on phones. Some of us use desktop computers with monitors. I see your tiny column on the left, at least center it.

Some of this is feels like hogwash. For instance, it says "anything" can be music. This seems incorrect. Music has a pattern and is composed of sounds.

Was this written by someone who doesn't like music? IE their brain is wired different from most of the population. You can't really understand something you can't comprehend.

<comment><USER4> An alternative viewpoint: music is anything presented. See John cage's 4'33". No sounds or patterns to be found.

<comment><USER3> That is not music. That is an audio track.</comment></comment></comment>


Sector/Sphere: High Performance Distributed File System and Parallel Data Engine - <AUTHOR> http://sector.sourceforge.net/

<comment><USER2> Sector is also a member of the Open Cloud Consortium:

<http://opencloudconsortium.org/software.html>

</comment><comment><USER3> One caveat: Sector/Sphere is not used by anyone (notable) in the industry. I hope they become a viable alternative to Hadoop as they do have some strong points on paper.

OTOH, an industry maven told me recently that "they don't listen" (to industry feedback), which I hope, is not entirely true.

</comment><comment><USER4> A lot of interesting and bold claims. Has anyone played with this product? Does it actually work well?

<comment><USER5> I'll be spending some time on this.

Glusterfs, Hadoop, Pohmelfs, there are a lot of interesting developments in this sphere.</comment></comment>


Show HN: An Angular port of ng-FitText.js - <AUTHOR> https://github.com/ngTrumbitta/ngx-fit-text <comment><AUTHOR> Feedback, code reviews, issues, and pull requests are very welcome.</comment>