BBC To Delete 172 Websites Due to Budget Cuts, Geek Saves Them for $3.99 - <AUTHOR> http://www.readwriteweb.com/archives/bbc_to_delete_172_websites_due_to_budget_cuts_geek.php

<comment><USER3> Horrible reporting.

The cost to the BBC to keep those pages on its servers is more than $3.99. Just because one guy downloaded all of the pages, compressed them, and then seeded them on bittorrent for a cost of $3.99 to him doesn't mean jack.

Notice that he's not hosting the content in any easily readable form. No, he decided to put that burden on everyone by putting it up on bittorrent. Why isn't he hosting the content? Because hosting a heavily trafficked site ain't cheap.

<comment><USER4> The fact that it'd cost the BBC millions to maintain those pages means jack.

People were going to lose access to that information, and now they aren't. That's really all that matters. With torrents, the burden is on "everyone" only if they really want to keep it up.

edit: Reasoning for downvotes would be appreciated. The BBC has a long history of recklessly losing valuable data. See kgtm's link to previous yc post on this topic.

edit #2: Oh, okay, I see what's going on. When I said "means jack", I didn't mean to imply that the BBC should spend inordinate amounts to keep the data up. What I meant was: Yes, it might have cost them to do it, but look, like what this guy did, there are other ways to keep the data archived. Hell, I'm sure some people would even settle for a CD archive or something.

<comment><USER5> "means jack"

Actually it doesn't mean jack. that's my money they'd be using, and from a quick scan of those websites, i'm happy for them to disappear.

For some reason, many people seem to have decided that all data is important and there's an almost fetishistic devotion to saving everything that can be saved. Not all information needs to be preserved for all time.

<comment><USER6> > Not all information needs to be preserved for all time.

I disagree, and strongly. Future historians will probably disagree with you too. Even the most inane TV shows can prove to be extremely valuable when trying to decipher the culture of various countries 300 or 3000 years in the past. Imagine reading a trashy romance novel from 1000 BC. You'd learn a lot about the people who lived in that time.

You might counter this by saying that with all the information out there now, we only need to keep the "good stuff". But who decides what's good, right now? How do you know that what you decide is good will always be seen as good? Your personal opinions on the content disappearing are totally irrelevant when perhaps millions of people have seen it or have been affected by it.

All information that is created should be saved, especially since we're _able_ to. Imagine the ancient Romans deciding to burn a bunch of books because it cost too much to hire guards for the library. Ouch.

<comment><USER7> Mind, I don't think future historians will weep too much about losing some promotional websites for such actual content.

<comment><USER7> Is the downvote above because someone disagrees, or because the downvoter is just completely unaware of the nature of the sites being taken down?

Most of these sites are promo fluff for BBC TV shows. Some are cancelled, some are for news show with little more "content" than "This show next comes on --- on BBC -".

Most of these sites would be dismissed as "marketing" if the BBC were a for- profit company. Marketing for defunct products. The rest is material that's being legitimately archived elsewhere.

<comment><USER6> (OT: I didn't downvote you above; in fact, I can't, since you replied to my comment I think. However, note that many of us frequently accidentally downvote instead of upvote and can't undo it, especially on mobile touch- browsers if we're not zoomed-in enough for a big arrow)

<comment><USER7> I've occasionally done that myself, but I then post a note that I meant to upvote.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I can't believe this passes for reporting. Did anyone contact the BBC for their estimate of the amount they'll save and how?

Also, just because the info isn't public does not mean the BBC doesn't have it stored in their own digital archives.

<comment><USER9> The BBC has stored at least some of if in an archive in cooperation with the British Library. I've posted a link to an earlier article; you can find it by browsing some of the "TLDs" they're removing.

(BTW, did anyone notice that the BBC calls these sites "TLDs", and that they took a beating for that by people who assumed they were incorrectly referring to "Top Level Domains"? But, TLD can mean "Top Level Directory" too, which is entirely accurate. Most amusing...)

Anyway, I think it's good that we're developing a culture where some people care, and make sure that things get archived, even if it's done sometimes unnecessarily, and almost always only gets a degraded copy (ie, a web rip without streaming videos, and without structured data). I only hope that these vigilante actions don't lead companies to _not_ pre-announce massive data erasures.

</comment></comment><comment><USER10> Just a note for those wondering about the content of the archive.

Having downloaded it, it's around 2GB compressed. It contains images, but most pages are nonfunctional, due to links being specified from the site root /. To view it properly you need to place the folders at the root of your web server or of your hard drive.

<comment><USER11> That does not really tell me what is in the pages though. What are the 175 sites that would have been foreve lost?

<comment><USER12> Load the torrent into your client and it will show you the file structure. You then select the file(s) you want to download.

</comment></comment></comment><comment><USER13> _The BBC needs to make visible cuts in places where the British (anti-BBC) press accuse them of providing services that they believe should be provided by private companies.If the site doesn't vanish the press wouldn't see it as a real cut, would they?_ [1]

[1] sambeau, <http://news.ycombinator.com/item?id=2188870>

<comment><USER14> Well said that man! :)

Also, keeping these sites (without actively updating them, monitoring comments etc) would essentially cost the BBC nothing, especially the low-traffic ones. BBC Online is a very lean organisation.

see: <http://news.ycombinator.com/item?id=2189564>

</comment></comment><comment><USER7> The BBC makes a show of "cutting costs" by closing down promotional sites for a pile of mostly-cancelled TV shows and handing off some sites that might actually count as "content" to the British Library. (Meanwhile, they leave up stuff like this because people actually watch _Doctor Who_ : <http://www.badwolf.org.uk>)

 _Why is this a story_ , even in web circles? Why all the hyped-up outrage?

Are people associated with the BBC shilling this?

<comment><USER15> Not necessarily. But the BBC are. The current UK government has asked all departments to cut spending by more than ever in history, even more than when the empire nearly went bankrupt after the napoleonic war. The BBC whilst nominally independent are funded through a state tax and are a state-owned chartered corporation. The current tories have always been treated with hostility by the BBC as the tories tend to want to reduce the BBC's funding, and the Lib Dems never really got much of a word in edgeways. Consequently the BBC (with it's massive media power) is pushing this to make it very visible that they're doing something about cutting costs.

</comment></comment><comment><USER16> The real issue is "Cool URIs don't change" vs "The BBC is hideously bloated". Keeping these sites online isn't so terribly expensive. Keeping a few hundred extra lines in your .htaccess isn't expensive. I've not seen a decent estimate of the cost savings, and as far as I know one doesn't exist. The material is being archived inside the BBC, as well as in torrents like this one. But the URLs are being broken, for no reason besides sending a message.

</comment><comment><USER17> If the BBC is unable to host these sites because of budgetary issues, is the BBC also unable to pursue copyright infringement of these same sites if a third party were to set up the site/s in a different country?

<comment><USER18> I could easily imagine one of the problems could be the BBC _not knowing_ what rights they have to the content. Likely some of it will be stuff they only have limited rights to; donating it to an external project could plausibly get them into trouble. And hiring somebody to check the contracts over 172 websites won't be cheap.

</comment></comment><comment><USER19> Did any of you actually see the content?

I downloaded one of the sites in the torrent titled Zombies. It's about a British girl who organizes a community effort to make a Zombie Movie. Sure the site was ripped but not the video content. Nothing on the page is worth seeing other than the video. Once the BBC turns off access to that stream the archive is virtually useless. I'm going to checkout more ripped sites; my gut says they're probably video heavy too.

<comment><USER19> I've looked at it a bit more. All the smaller sites are just place holders for lesser important topics or for streaming BBC shows. Sure the torrent saves some user comments (not all) and images (not all, some links are broken) but there's not really a lot of meat since that content is periphery.

I'd say that most of the valuable content is still stored on the BBC servers, in the DB where it's being served from right now. I'm sure they're going to repurpose the good stuff if they haven't already. I have an acquaintance who works in the web department at the CBC. They've implemented a pretty neat CMS to repurpose their older media to work with their latest site. I can see the BBC doing the same.

</comment></comment><comment><USER20> I don't understand is situation, but it sounds like a case of the Government removing a high visibility public resource as a sort of protest to budget cuts. Eg: <http://i.imgur.com/Vdk6D.jpg>

Even if that is the case, I'd sure like to see a breakdown of expense when a cut like this is made.

<comment><USER21> Almost all of the sites being archived or deleted (yes, archived - <USER2> reporting that these are all up for deletion was overly simplistic and incorrect) are for programmes or programme strands that are no longer being broadcast, in some cases for many many years.

From what I've seen reported, the only site of any significance which is being removed is 'WW2 People's War', which asked people to put their and their family's personal stories of the second world war into a central archive (<http://www.bbc.co.uk/ww2peopleswar/>). I believe that this was already being archived by the British Library, so won't be lost.

The overall decision to cut 50% of the 'TLDs' is definitely political, to placate an administration that's beholden to the Murdoch media empire, but the choice of which ones to kill seems mostly pragmatic.

</comment></comment><comment><USER18> Does anybody have a good idea of how much it _does_ cost for a big organisation to keep this kind of content online?

Intuitively, it seems that the answer should be 'not much' -- perhaps on the order of a few thousand dollars/year. A couple of servers, bandwidth, and a sysadmin checking in now and again to apply security patches &c.

But here (also with e.g. yahoo closing geocities), it's argued that the cost of keeping them up is much, much higher. Where does the expense come from?

</comment><comment><USER22> Am I missing something? Where is the price of $3.99 derived from? Cost of creating the spider script? Labor cost of creating the BT seed? This article say nothing about this.

<comment><USER23> I think it's the (monthly?) price of the seedbox.

</comment></comment><comment><USER24> It's funny, the activist seems to think the deletion is about appeasing political masters, but in doing this, they actually lend credibility to the opposite point: Just leaving the pages in places is unlikely to cost the BBC a lot. Is it possible that the BBC is cutting in the most visible way possible, in order to make the cuts seem much more extensive and painful than they actually are in order to scare the government away from further cuts?

</comment><comment><USER25> Here's the actual guy who did this. <http://22.214.171.124/>

</comment><comment><USER26> I'm sure someone will take the abandoned content, slap it on a server with some Adsense and call it a day.

</comment><comment><USER27> We've graduated from the grammatically-incorrect singular _they_ to _s/he_? Really?

<comment><USER28> Worse still, _they_ (being singular where gender is indertiminate, or you seek to be neutral) is not grammatically-incorrect, and has a long history in the English language. It's a bit like the split infinitive, which is often considered to be incorrect but isn't.

The author: they really should have known better than "s/he", to boldy write for RWW.

Edit: Added a sourcewith examples - <http://www.crossmyt.com/hc/linghebr/austheir.html> and the Wikipedia page seems to have some discussion about it too - <http://en.wikipedia.org/wiki/Singular_they>

<comment><USER29> Should a singular _they_ be used exactly like any other singular pronoun:

      "I will say of that person: they laughs a lot."
      (Compare: "I will say of that person: she laughs a lot.")

      "They is the person you should talk to."
      (Compare: "He is the person you should talk to.")

Or should it instead be:

      "I will say of that person: they laugh a lot."

      "They are the person you should talk to."

? And why?

<comment><USER30> Presumably if you're referring to a specific individual, you already know the gender so using "they" in that situation would appear to be incorrect.

However, in the general sense I would argue that the conjugation of verbs follows a pattern but that pattern doesn't have to be singular vs. plural. Given "he has" or "she has" one might expect a singular "they" to follow the pattern "they has". However, "I" is singular yet it uses "I have", so it's not unexpected for the singular "they" to use "they have". In other words, the conjugation of "they" is the same regardless of whether it's singular or plural.

<comment><USER31> they is used in a bunch of situations where the person prefers to be non gender specific. It only feels wrong for the first little bit, but hang out with enough people that have complex gender identities and you will get used to it.

</comment></comment></comment></comment></comment><comment><USER32> It's not the hosting that costs much usually, it's developers + editors + managers + their managers. It all adds up very quickly.

<comment><USER33> How much do you pay editors and managers and developers to keep 12 year old sites sitting on a webserver?</comment></comment>


Show HN: TimeForZen, my first "learning to code" solo project - <AUTHOR> http://www.timeforzen.com

<comment><USER2> Fantastic. I will definitely use this. Very relaxing.

UI could use some polish. What if you had a "big" featured track at the top and then more options below?

<http://imgur.com/5IBlu> (MS Paint copy and paste)

The columns below would be channels/categories.

<comment><AUTHOR> lol Thanks.

The mockup actually gives me some ideas. When I get enough tracks (like 50+) I think I'm going to take your advice and put a featured track at the very top.

As well as changing all the images to lazy load as people scroll down. That way the site loads up faster initially.

</comment></comment><comment><USER3> This is pretty nice!

I think it would be pretty cool if when you selected a track the beautiful picture that accompanies the audio goes to full screen. Maybe the black instruction bar could display until the track loads and just fade leaving the photo and a pause/close window icon somewhere discreet like in the top right hand corner!

 _Listens to the waves_

<comment><AUTHOR> I was originally going to do it that way. Have a full screen background, but by loading that up it would slow down the loading of the song. I think I'll revisit the concept since you like the idea. I'll probably give it a try in the next build. Thanks.

</comment></comment><comment><USER4> I can't get it to work, it just sits at the loading screen for me. Using Firefox on Vista.

<comment><AUTHOR> Thank you so much, I was afraid of this. I'm using tinybox2 from scriptiny.com and the glitch might be from his script.

What version of firefox do you have?

<comment><USER4> 11.0, the latest version. Chrome works just dandy though.

<comment><AUTHOR> Jesus Christ. OK check this out. Apparently there's a weird voodoo glitch between Firefox and amazon s3 bucket policies (which are a nightmare for beginners) that was preventing the audio from streaming only on Firefox browsers...

After 7 hours of hair pulling, I fixed it. But I've lost my bucket polices that prevent hot-linking the audio. And I've also decided to abandon Amazon S3. I'm making the switch tonight. It's such a pain in the ass It's just not worth it. So much trouble just to get the simplest things done when I could of spent 1 minute with an FTP client and a .htaccess file and gotten the same result. I've had so many problems with numerous Amazon services, from unboxed, to video on demand, to S3 numerous times, to amazon mp3. I am disappointed and frustrated beyond comprehension. Strike 10 Amazon. (literally this is the 10th time I've had problems) You're out. Maybe no one else has had problems but I personally am going to avoid Amazon services. I've experienced 10 glitches over the last year that have affected my purchases or websites and I've seen enough.

<comment><USER4> Still broken. Maybe its me and not your site.</comment></comment></comment></comment></comment>


Sexism and Computer Science: Where Are My Sister Coders? (2014) - <AUTHOR> http://moduscreate.com/sexism-and-computer-science-where-are-my-sister-coders/

<comment><USER2> In academia and industry, I've only seen one case of sexism, and it was from a visiting professor (from Turkey) and he was quickly removed.

I have seen women that shouldn't be in CS, not because they are women but because they were only there because they had heard that CS was a high paying field. Then they washed out because they didn't have the interest nor mindset to learn the subject. But so what? I've seen men fail out the same way.

The opposite is also true, I've seen women and men alike excel. In today's technology world, if you've got the skills, then you get the respect.

I will not hire a woman coder just because she is a woman, and the same is true for a man. I want to see a demonstrated mastery of the requisite skills.

If I have a daughter in the future, should I push her to be a coder just because there are fewer women coders overall? No. That, to me, is just sexist because it does not take my (hypothetical) daughter's interests or talents into account. If she likes it, then by all means do it and be the best! But I would not discourage my daughter from a woman-domineered field just because there are a lot of women in that field already, especially if it something that she is good at and that she enjoys!

The same would be said to my son. I would not discourage him from CS just because there are a lot of men in that field already. Neither would I encourage him into a woman-dominated field, just to fill an arbitrary quota. Let him do what he is good at and enjoys!

</comment><comment><USER3> That graph is misleading. You'd think that the number of women in CS was decreasing reading that when the reality is that they've gained, they've just gained slower than men.

Using a % graphic (men:women) is a very political choice. It was picked to give the reader the impression that women were running away from CS and the situation was getting more and more dire. Contrast that with this article's data:

[http://www.computerworld.com/article/2474991/it- careers/wome...](http://www.computerworld.com/article/2474991/it- careers/women-computer-science-grads--the-bump-before-the-decline.html)

Things could definitely be better. But at least the graphs in the above article give you a more realistic picture of things, rather than a misleading one.

</comment><comment><USER4> There is definitely some amount of sexism in corporate and academic CS, but there is an entrepreneur that doesn't need funding and is entirely anonymous.</comment>


Indiana Pi Bill - <AUTHOR> https://en.wikipedia.org/wiki/Indiana_Pi_Bill <comment><USER2> I believe numberphile (a youtube channel) made a video about this some time ago. Look that up if you prefer watching a video to reading wikipedia articles.</comment>


Show HN: SongDrop.me – A Better Way to Discover New Music - <AUTHOR> http://songdrop.me/

<comment><USER2> You probably already know this, but another music discovery service called "Songdrop" already exists and has been around for a few years - [https://songdrop.com/](https://songdrop.com/)

Might want to rebrand!

<comment><AUTHOR> Thank you for checking it out! We are aware and plan on rebranding soon. We are currently still in the MVP phase so the domain name is more of a short- term thing. We do plan on adding much more to the service.</comment></comment>


Coronavirus burial pits in Iran so vast that they're visible from space - <AUTHOR> https://www.washingtonpost.com/graphics/2020/world/iran-coronavirus-outbreak-graves/ <comment><USER2> They're visible in a satellite image. I'd hardly say a football field size ditch is visible from the ISS with the naked eye, or even on an airplane without a zoom lens.

<comment><USER3> Agreed. My back yard is 'visible from space'. I desperately want a sociable annotation tagging system so I can flag hyperbolic nonsense like this and then filter out anything with lots of unwanted tags.

I am sick of hyperbole and cliches in reporting. They make the internet and society worse.</comment></comment>


The problem with Sony's two-front HD war (Blu-ray and streaming media) - <AUTHOR> http://www.thestandard.com/news/2008/07/01/sony-s-blu-ray-will-keep-them-winning-new-format-war

<comment><USER2> The problem is to many people value bling over any actual useful features. Here's to Apple TV bringing rentals direct to your TV.</comment>


What It's Like to Get Doxed for Taking a Bike Ride - <AUTHOR> https://nymag.com/intelligencer/2020/06/what-its-like-to-get-doxed-for-taking-a-bike-ride.html <comment><AUTHOR> "To his surprise, the app that he used to record his regular rides from Bethesda into Georgetown via the Capital Crescent Trail shared that information publicly, not just with his network of friends and followers. Someone had located a record of his ride on the path on June 2, matched it to the location of the assault from the video, matched his profile picture — white guy, aviator-style sunglasses, helmet obscuring much of his head — to the man in the video, and shared the hunch publicly."

</comment><comment><USER2> Per tfa, the guy was not doxxed as much as he was misidetified. Most 'doxxing' is just public information that the 'victim' mistakenly thought was private. Address, name, even SSN are not private.

</comment><comment><USER3> Reminds me of the time that reddit caught the Boston Bomber...

</comment><comment><USER4> Doxing is the modern day equivalent of street lynching or witch hunting. The only way to stop it is to hold people accountable; so in this case the person who started Tweeting about it. 20y in prison should do it.</comment>


Ask HN: The experts told us soft-launch == app death. Does it? - <AUTHOR>

We have worked long and hard developing our mobile app. We think that it's great and has true value for users. We had planned to soft-launch and make it better as we learn from users. But every single app expert we had talked to has told us our success will be as good as the amount of noise we make when we launch. Death, they say, is a place in the long tail of the app store. What say you? <comment><USER2> You can find plenty of exceptions, but what the 'app experts' are telling you is generally correct, at least about the Apple ecosystem. Bunching up a lot of downloads into a small time period right after launch maximizes your chance of showing up anywhere users might actually browse to and discover serendipitously.

The Android marketplaces are a bit better for iterative learning. Although iOS then Android is the dominant pattern, you wouldn't be the first company to launch quietly on Android, iterate, and then come to iOS with a product informed by what they've learned from their Android launch.

</comment><comment><USER3> I'm not a mobile developer, so take what I say with a grain of salt, but as a user, I don't care about an app's age or whether it had a soft launch. Most apps are little more than opportunities to provide a bad user experience to an impressive Web-based service. As long as your service is on point and your app on my device is better than just using your mobile website, do whatever.

<comment><AUTHOR> So, would you be as interested in an app even if it didn't have a lot of media references to show (articles in newspapers, tech site mentions and so on)?

<comment><USER4> Consumer, yeah. B2B, not so much. The soft launch stuff is irrelevant (sounds made-up too). Marketing is an ongoing process. Those dudes need be thinking of a second campaign, instead of giving up once they see the churn after the hype.

<comment><USER5> Your reply makes sense.

I just wanted to add that I think I see what the OP meant by "soft launch." I think the OP asked a good question, because most startup business guides say launch something minimal and iterate. Whereas, app-store gurus seem to place a heavy emphasis on the initial bump. (That is, they hint that you either succeed on the initial push or die.)

OP: Thanks for asking the question.

<comment><USER4> I see what you mean now. Odd that they think that way though. Sure, there's more clutter, and if your app contributes to that, I suppose the 1st bump is all you get, but if you have a quality product, the sales process is the same as in any other channel.

</comment></comment></comment></comment></comment><comment><USER6> I've had no hype, probably because I'm boostrapped, but I'm in this for the long haul. What would be the point of giving up six weeks in because I only have X users and the experts say I should have 1000X? The experts are not financing me. If they were, then I could afford to create hype.</comment>


A 75-Year Harvard Study Finds What It Takes To Live A Happy Life - <AUTHOR> http://www.businessinsider.com/grant-study-reveals-what-makes-us-happy-2013-4 <comment><USER2> I hate it when these things pop up because they turn problems into self- fulling prophecies.

I hate being around my mother. I've never had a relationship with a girl in my entire life. I drown my sorrows into a mug of beer every friday. I barely get by as is - and have to work my ass off harder than everyone around me to get anything done.

And now I come across this article that says it's likely I'll make 6 figs less than some other guy because of these variables that are hopelessly outside of my control. How fucking depressing.

<comment><USER3> Sorry, but it's rather funny that what you deem depressing is not making 6 figs rather than being a lonely alcoholic. Maybe some pondering about happiness would do you some good?

<comment><USER4> Besides, honestly everything is futile anyway. So you can go play Frisbee golf with a group of friends, or stay at home in the dark drunk. At the end of the next century it doesn't matter, you'll both be dead. Happiness is fleeting and an illusion when seen. Get use to it.

<comment><USER5> That's just a lie. Ah, the beautiful false equivalence... you will be nothing in 100 years, but you're something now. You can choose to be nothing now as well if you want, except that you're not, because you're something while you're here.

You've been given about a half-century on this planet; you're one of the luckiest groups of atoms in the entire universe to have agglomerated into a self-replicating sentient body that is on the brink of _voluntarily_ expanding into a next level of its evolution at a more and more rapid pace.

At the end of the next century, the human race will probably be, as a whole, more aware and more advanced than ever before. And you, of all the agglomerations of atoms in the universe, get to be a part of it. You get to contribute; you get to be a cell in this incredible human race. And even better, you get to exist at a time when all the humans on earth are connecting to each other and spreading information and knowledge faster and better than ever before. We're becoming a human organism, we're cells in a great life- form, neurons in a giant brain; we're going to evolve as a planet now, faster than biology could ever dream.

Happiness is love, and love is the glue that binds us together like neurons, each knowing if only instinctively how important the others are to the great network we're in. You can isolate yourself and think that you don't matter and everything is futile, but if you do that, then and only then will everything actually be futile. Otherwise, you are very real, the time you have is real, your life is real, what you choose to do with it is real, and the results will—in some small or maybe big way—evolve the human race.

To me, that matters.

</comment></comment></comment></comment><comment><USER6> This kind of correlational study is awash in survivorship bias, and may not even apply to age cohorts born in a different society in a different era. Most likely, the study results neither show what is sufficient for a happy life for all nor what is necessary for a happy life for anyone. It would take an experimental study design with a study population more representative of the broad range of present-day people (not just Harvard men) to find information like that.

<http://www.econstor.eu/bitstream/10419/43616/1/626014360.pdf>

AFTER EDIT:

A second-level comment below asks:

 _Weren't women allowed in Harvard in 1938?_

No. Women have been able to attend Radcliffe College (very near to Harvard) throughout my lifetime and even before, but women being admitted as undergraduate students to Harvard only happened in the 1970s.

[http://www.college.harvard.edu/icb/icb.do?keyword=k61161&...](http://www.college.harvard.edu/icb/icb.do?keyword=k61161&tabgroupid=icb.tabgroup85886)

(In other words, the girl I know from high school in Minnesota who attended Harvard was one of the very first to do so.)

<comment><USER7> Agreed; this study seems to only find correlations between parenting and happiness/success in life.

What if some children grow up in certain ways regardless of how they're raised, and this is what causes them to have shitty relationships with their parents? Maybe some kids don't get good parenting simply because they became little shits? And then this carried on into their lives?

<comment><USER8> > became little shits

no child just becomes a "little shit". It takes a lot of time and effort to create one.

Children have some certain negative aspects in them and have some positive aspects in them. Parenting is all about making good aspects stronger and negative aspects weaker.

Recently, my child started yelling like crazy. She will just start screaming at a restaurant thinking it's fun (3 year old). Or just yell at 3 am. I could yell back and make this negative aspect stronger, instead I enrolled her to the music classes to channel all this yelling energy into something good. Guess what, she yells less.

It's all about the parents.

<comment><USER5> It really is.

There are people who believe people are predestined and struck in stone, and that there are good people and bad people, and that if someone is bad or lazy or has bad characteristics there's nothing that can be done, it's just who they are.

Those people are wrong and make horrible parents.

<comment><USER7> The world isn't black and white. We're born with predispositions. Upbringing has some influence. But consider that all the parenting in the world can't make a retarded kid smart. How do you know there aren't other things in our brains that are difficult or can't be controlled?

<comment><USER5> I absolutely agree, I think there definitely are.

I'm not saying there are only 2 kinds of people, but that there's sort of a divide. This divide appears to be nearly dichotomous, and it seems to follow some pretty fundamental beliefs about humanity and our place in the world. It's no doubt a stochastic process leading to such a divide, and it's no doubt continuous, but there's a line where if you're on one side, you sort of think it's okay to use some negative reinforcement with people and that they need that to grow stronger (a self-reinforcing idea), and if you're on the other side you sort of think it's not really okay to punish (also a self-reinforcing idea), and the self-reinforcing nature of those ideas leads to their internal reinforcement to become stronger, such that we really do get clustering of individuals who all hold this strong belief about people, and reinforce each other.

There are very few people in the middle of this, and while the child and the personality is not a dichotomy per se, the belief about punishment vs. reward; negative vs. positive reinforcement—is quite dichotomous, not because the world is black and white, but because of the effect of internal reinforcement, groupthink, and time. People gravitate to one side or the other, and having a middle way on these sorts of issues is not attractive to the human psyche or, as it turns out, the way ideas naturally propagate and grow within people.

Politics works precisely the same way, and indeed shares many of the roots of this very opinion on humanity: does punishment work better, or does reward work better? Do we leave the poor to fend for themselves, or do we proactively help them due to their circumstances? It comes down to the way people believe humans work, and again, these ideas gravitate naturally toward opposite ends.

It's quite sad, because reality really is somewhere in the middle, and an effective strategy would take into account the reality of human behavior in addition to the side effects of negative reinforcement and the benefits of positive reinforcement and result in a well-balanced type of policy or parenting style that's scientifically sound and based on truth. But that's not how we work with our offspring; we do what we believe is right and try to do better than our parents, and maybe if we read here or there that this or that is good for the kid, we'll take it into consideration. I truly believe most parents are trying to do what's best for their child, if they can, and it's ignorance and false beliefs about people in general that leads them to make mistakes. I think that's the gist of it.

So you're right, the world isn't black and white, I appreciate you opening my eyes to the truth of the middle way and the truth behind the incomprehensible layers of complexity. It's just fascinating to me how people form ideas, and especially how dichotomous ideas take hold and self-reinforce, because I really think they do. I think it's one of the great problems we could solve in society in the next few dozen years with great benefit, but there's probably not anyone with the right mindset to study it correctly.

<comment><USER8> Very interesting post. I definitely agree with you.

There are two additional points I'd like to make.

1\. >does punishment work better, or does reward work better?

There are always two "variables" when dealing with a situation: a) person you're dealing with b) situation you're dealing with For situation A that involves Henry, you know that punhishment works better. For situation A that involves Jonathan you will know that reward works better. For situation B it might be just opposite for the given persons. So, you just adjust to get results. It's _not_ like you are in the punishing camp and won't reward in another situation involving the same person or different one. It's all about results sometimes. You want to get your stuff done or your kid do what you want, right? Not just to punish or reward. That would be really hollow personality type. At least in my book.

2\. >It's just fascinating to me how people form ideas, and especially how dichotomous ideas take hold and self-reinforce, because I really think they do. I think it's one of the great problems we could solve in society in the next few dozen years with great benefit, but there's probably not anyone with the right mindset to study it correctly.

Well, this problem has been "solved" by politicians who are currently running show in the West. Ever heard of the term technocrat when talking about politicians? Left more efficient with the social issues? Let's have left there. Right more efficient with the economy? Let's do what right advocates there. What you described is what technocrat politicians believe in and do. And is the reason why our societies and economies are so messed up ;-)

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Ack. Reliable or not, I'm sure it was meant as a feel-good piece, but for people like me the quote:-

"Vaillant's key takeaway, in his own words: "The seventy-five years and twenty million dollars expended on the Grant Study points … to a straightforward five-word conclusion: 'Happiness is love. Full stop.'"

Is actually very depressing. I am 5'5" so have this (if you've not encountered it you are unlikely to be aware) INSANE disadvantage when it comes to dating - women can't stand short men. Seems almost certain to be a genetic thing.

Basically, I've given up. There are major issues in the family too, so I am half-estranged from many close members of the family.

So yeah, hopefully it's just _false_ feel-good fluff.

<comment><USER10> I'm 5'4", chubby, prematurely balding, and under 30, and I've been in long term relationships with several awesome women.

There are certainly disadvantages to being short, probably especially in the dating realm. Big whoop. That's not what's preventing you from dating anyone.

<comment><USER9> I never said _preventing_. It's a massive disadvantage. I could (+ might) write a longer piece about the whole subject as there is more to this than height but it's quite surprising what a difference it makes.

If you can overcome that disadvantage, all power to you, but it doesn't change the state of affairs.

I refer you to this graph - <http://cdn.okcimg.com/blog/lies/MessagesPerWeek.png>

It's a rough analysis but shows a clear female preference based on height. Certainly an indicator.

</comment></comment></comment><comment><USER11> Doesn't "Harvard undergraduate" give a significant bias to the whole study? I'm not disagreeing with their claims, but claiming universality doesn't seem appropriate.

<comment><USER12> Cool study, but this is obligatory:

        Roger Meyers, head of Itchy&Scratchy studios: Allright,
        heads up you bunch of low- lifes.  This is Abraham Simpson,
        and from now on you guys are going to take all of your cues
        from him.  He's got something you couldn't get at your
        expensive, ivy-league schools: life experience.

        One of the post-college kids: Actually, I wrote my
        thesis on life experience.

</comment></comment><comment><USER13> Strong relationships make you happy. Not that suprising - we are social creatures after all.

Those interested in learning more should read the original Grant study synopsis in the Atlantic [http://www.theatlantic.com/magazine/archive/2009/06/what- mak...](http://www.theatlantic.com/magazine/archive/2009/06/what-makes-us- happy/307439/)

<comment><USER14> HN discussion of the Atlantic's Grant Study synopsis:

<https://news.ycombinator.com/item?id=605207>

</comment></comment><comment><USER15> The big takeaway seems to be that as long as you don't become an alcoholic and are able to adequately maintain relationships you'll probably be ok.

</comment><comment><USER16> The title really has to be changed to "A 75-Year Harvard Study on Men Finds What It Takes To Live A Happy Life". I clicked and was disappointed.

<comment><USER17> I wonder why limit the study to men. Weren't women allowed in Harvard in 1938?

<comment><USER18> No, actually, they went to Radcliffe. Harvard did not admit women to undergraduate programs until 1973.

</comment></comment></comment><comment><USER19> Paul Tough's book, "How Children Succeed" [1] has a great section on how mothering style has a large effect on offspring throughout their lives:

"Parents and other caregivers who are able to form close, nurturing relationships with their children can foster resilience in them that protects them from many of the worst effects of a harsh early environment. This message can sound a bit warm and fuzzy, but it is rooted in cold, hard science. The effect of good parenting is not just emotional or psychological, the neuroscientists say; it is biochemical.

The researcher who has done the most to expand our understanding of the relationship between parenting and stress is a neuroscientist at McGill University named Michael Meaney. Like many in the field, Meaney does much of his research with rats, as rats and humans have similar brain architecture. At any given time, the Meaney lab houses hundreds of rats. They live in Plexiglas cages, and usually each cage holds a mother rat, called a dam, and her small brood of baby rats, called pups.

Scientists in rat labs are always picking up baby rats to examine them or weigh them, and one day about ten years ago, researchers in Meaney's lab noticed a curious thing: When they put the pups back in the cages after handling them, some dams would scurry over and spend a few minutes licking and grooming their pups. Others would just ignore them. When the researchers examined the rat pups, they discovered that this seemingly insignificant practice had a distinct physiological effect. When a lab assistant handled a rat pup, researchers found, it produced anxiety, a flood of stress hormones, in the pup. The dam's licking and grooming counteracted that anxiety and calmed down that surge of hormones.

Meaney and his researchers were intrigued, and they wanted to learn more about how licking and grooming worked and what kind of effect it had on the pups. So they kept watching the rats, spending long days and nights with their faces pressed up against the Plexiglas, and after many weeks of careful observation, they made an additional discovery: different mother rats had different patterns of licking and grooming, even in the absence of their pups' being handled. So Meaney's team undertook a new experiment, with a new set of dams, to try to quantify these patterns. This time, they didn't handle any of the pups. They just closely observed each cage, an hour at a time, eight sessions a day, for the first ten days of the pups' lives. Researchers counted every instance of maternal licking and grooming. And after ten days, they divided the dams into two categories: the ones that licked and groomed a lot, which they labeled high LG, and the ones that licked and groomed a little, which they labeled low LG.

...

The researchers ran test after test, and on each one, the high-LG offspring excelled: They were better at mazes. They were more social. They were more curious. They were less aggressive. They had more self-control. They were healthier. They lived longer. Meaney and his researchers were astounded. What seemed like a tiny variation in early mothering style, so small that decades of researchers hadn't noticed it, created huge behavioral differences in mature rats, months after the licking and grooming had taken place. And the effect wasn't just behavioral; it was biological too. When Meaney's researchers examined the brains of the adult rats, they found significant differences in the stress-response systems of the high—LG and low-LG rats, including big variations in the size and shape and complexity of the parts of the brain that regulated stress."

[1] [http://www.amazon.com/How-Children-Succeed-Curiosity- Charact...](http://www.amazon.com/How-Children-Succeed-Curiosity- Character/dp/1452658145)

<comment><USER20> What can the child do once he starts to ask himself what the hell is wrong with him, and little by little, starts to associate some wrongs with his childhood? Is it possible to revert bad parenting?

<comment><USER21> Albert Ellis has already proven the answer to this is "yes". In short, we are the ones typically responsible for our own neurosis as we maintain flawed thinking or understandings of reality. People with loving parents have turned out screwed up kids, just the same.

<comment><USER5> "People with loving parents have turned out screwed up kids, just the same."

This simply does not matter if there is a statistically valid conclusion saying the opposite.

There will always be outliers and anecdotes. You should be looking at the statistical trend, not individual cases.

<comment><USER21> I'd say it's valid if science says otherwise

<comment><USER5> Now you're just being silly.

</comment></comment></comment></comment></comment></comment><comment><USER18> Link is an annoying wrapper around [http://www.theatlantic.com/magazine/archive/2013/05/thanks-m...](http://www.theatlantic.com/magazine/archive/2013/05/thanks- mom/309287/)

</comment><comment><USER22> How on earth can we usefully generalize from the experiences of white men born ~1918 who attended Harvard? I think longitudinal studies are massively important, and I'm all in favor of them, things like:

"Alcoholism was the main cause of divorce between the Grant Study men and their wives"

Don't really give us that much of a clue about how alcoholism (for instance) affects everyone else. I suspect it's similar; but this doesn't give us any data on anyone who isn't in a very very specific group.

</comment><comment><USER23> Being a researcher myself, I can only guess the answer must be "long-term funding"? :)

</comment><comment><USER24> "For instance, the 58 men who scored highest on measurements of "warm relationships" earned an average of $141,000 a year more at their peak salaries (usually between ages 55 and 60) than the 31 men who scored lowest".

Why can't they give an percentage, do these people who scored high earn 40 % more or just 5 % ?

I'm too lazy too check the average salary of Harvard undergraduate men at age 60.

</comment><comment><USER25> Science. Correlation != Causation. In other words happy people are more likely to remember their relationships with others in a positive light. I used to think I had a bad childhood, I wasn't happy. Then I was happy and would rave about the wonderful childhood I had. Now I'm kinda in the middle and am satisfied with my childhood. Your current attitude affects your view of your own past.

<comment><USER26> This is a longitudinal study. They were surveyed every two years.

<comment><USER25> My comment still applies. People with happy lives will see their earlier years in a warmer light than those without happy lives.

<comment><USER5> I think the point is that they actually surveyed them every two years, so they weren't asking them to look back on their lives, they actually had that information while they were experiencing it.

There may be other biases and errors, but I don't think your argument is valid given the method.

<comment><USER25> They started surveying them when they were at Harvard from what I understand. That is still quite a bit removed from their childhoods. In any case the point about correlation != causation stands. The information is interesting, but I find that too often with these studies a correlation is taken. A conclusion is then made on the basis of the correlation, and then the conclusion is stated as fact. Anecdotal evidence follows and that's the end of it. The thing that irritates me the most is that it's just taken as truth and rarely if ever do people investigate further. I'd just love to have a discussion about the data and the possible implications and other explanations for the same data with judgement left up to the reader.

<comment><USER5> I think you'll always get that in the pop psych pieces; they'll always _always_ try to reach some eye-catching conclusion title to pull people in. I haven't read the original paper, but I highly doubt they made a case for causation unless the statistics backed it up; and if the stats did back it up, then the science does in fact say causation and not just correlation. It would be interesting to check.

</comment></comment></comment></comment></comment></comment><comment><USER27> If you really want to know what it takes to live a happy life, read the book Flow [1] by Mihaly Csikszentmihalyi. It's the best book I've ever read on happiness and reading it has positively changed my life like nothing else.

[1] [http://www.amazon.com/Flow-Psychology-Experience-Mihaly- Csik...](http://www.amazon.com/Flow-Psychology-Experience-Mihaly- Csikszentmihalyi/dp/0061339202)

<comment><USER28> I love this book and it likewise had a great effect on my life. That said, it's mostly affected me in career choices. It doesn't have much to say about relationships.

<comment><USER27> Actually, it has a lot to say about relationships. If you are unhappy about yourself, you'll have a tough time being happy with someone else. Therefore, it is very important to work on yourself first, on your own issues, to improve your own confidence and self-respect. Only then can you expect to be happy around other people.

</comment></comment></comment><comment><USER29> These are the kind of studies that make me think social-psychology is just a psuedo-science. If you want to understand human-nature, read Shakespeare over Freud. If you want to influence people, read Dale Carnegie over B.F. Skinner.

I'll gladly eat crow if this study results in a 6% increase in loving relationships or a 5% decrease in alcoholism...

<comment><USER30> Not actively disagreeing, but would you care to elaborate on why you think social-psychology is a psuedo-science?

<comment><USER29> In his book ([http://www.amazon.com/Statistical-Models-Causal-Inference- Di...](http://www.amazon.com/Statistical-Models-Causal-Inference- Dialogue/dp/0521123909/ref=sr_1_5?s=books&ie=UTF8&qid=1368755594&sr=1-5&keywords=causality)) Professor Freedman says one of the reasons the social sciences have had virtually no beneficial impact on society (as opposed to the incredible progress in the hard sciences) is that their "studies" have unsupported mathematics. They make too many assumptions, and their methodologies don't support their conclusions. For example, you can do a study on the top reasons teens use drugs, and in a decade those reasons have completely changed because of the shift in culture.

The social sciences don't work well with the scientific method. You can't randomly select people, and you can't get them to make long-term changes to their lifestyle. Without experimental evidence, you are left with observational data, which is a nightmare to find causation instead of just correlation.

<comment><USER31> You are conflating multiple issues here: whether social science studies are valid, whether the effects of social science are beneficial to society, whether passing some "scientific method test" is a requirement for an approach to be "good."

To take one example (of many), all of consumer marketing is an outgrowth of social science - nowadays consumer marketing, behavioral psychology, and behavioral economics are overlapping fields which happen to go by different names.

We might debate whether marketing has been "beneficial," but I don't think you can reasonably argue that all marketing doesn't work just because the math behind it isn't as rigorous as quantum mechanics. (Not that we completely understand quantum mechanics.)

<comment><USER29> Consumer marketing would be a good example of something that has no long-term benefit because society changes too often. Marketing techniques that work in one decade won't work in the next. Because of this, marketing isn't cumulative. Our knowledge of the hard sciences is always improving, building on the work of the previous generations.

Yes, there are legitimate fields that don't have rigorous math behind them, but the social sciences try to use studies and numbers as the foundation of their theories. They want to be treated like biology, but their studies are not up to par.

</comment></comment></comment></comment></comment><comment><USER32> They posed this question to the now over-aged Professor who started this study as a Bachelor's project.

He replied: "Tenure."

</comment><comment><USER33> Related: according to this study in the UK alcohol is the most destructive drug: [http://www.fcaglp.unlp.edu.ar/~mmiller/espanol/Variedades,%2...](http://www.fcaglp.unlp.edu.ar/~mmiller/espanol/Variedades,%20politica/drogas_Journal.pdf) [PDF]

</comment><comment><USER34> So... High earners scored higher on "warmest relationships". Which is cause, vs effect here?

</comment><comment><USER35> Here's to another round of correlation/causation errors. Brace yourself.

</comment><comment><USER36> Linking to Business Insider for science? Seriously? Business insider is just barely on this side of tabloid.

</comment><comment><USER37> I was often asked "what do you believe then? " after I told people I don't believe in any religion. My answer is rather simple: I don't need to live on any belief. I live on knowledge. I know the following for a fact: love makes me happy, hatred makes me unhappy. And I want to be happy. That's enough.

Now I can say it's not just knowledge, it's science.

<comment><USER8> Science says believing (i.e. religion) makes people's lives happier, longer, more productive. So, why you don't believe in any religion? Science says it's good for you. Paradox perhaps?

<comment><USER37> I was merely pointing out that now science also prove that love makes you happy. I didn't say that science says you shouldn't believe in religion. So what paradox are you talking about?

BTW, I don't know about the science you mentioned, sounds fascinating. Does that definition of believing also include believing in Nazi, believing in Communism? Did it include the consequence of all the wars and terrorism driven religious belief (or any other none scientific "believing")?

<comment><USER8> and I merely pointed out that science doesn't have all the answers. That's why science found out that to be happy you need to believe in religion.

<http://en.wikipedia.org/wiki/Religion_and_happiness>

The paradox is that you seem to embrace science but ignore some of its findings in your personal life. (i.e. you are not religious).

</comment></comment></comment></comment><comment><USER38> Which has now been completely invalidated by the internet changing how we all live our lives.</comment>


Ask HN: How do think tanks make money? - <AUTHOR> <comment><USER2> I worked in that space for seven years. To put it plainly, we were paid to create ideas by entities and individuals who wanted a certain frame or perspective to emerge or persist in the wild.

<comment><USER3> i understand generally what you're talking about, but could you elaborate or provide an example? i've always been kind of curious, myself

<comment><USER4> CEO selects appropriate thinktank, says, "I run an oil and gas company. I need you to produce articles that are critical of global warming. Here's one million dollars."

Thinktank goes and hires a couple of global warming "skeptics" for $50K/year. They produce writings and speakings, which the thinktank personnel shop around to appropriate outlets for minimal fees. Op-Eds are distributed to newspapers. Pre-edited 5-minute segments are distributed to TV news programs. The media outlets are happy to receive ready-to-publish content for cheap.

You the viewer receive all this information against global warming from a "neutral, non-partisan" thinktank. You are likely to believe it. The name of the CEO's oil and gas corporation is never mentioned.

Money laundering is taking money from illegal sources and turning it into clean, usable money. Thinktanks are engaged in the practice of idea laundering.

<comment><AUTHOR> Aren't these just PR firms then?

<comment><USER5> Think tanks: are they just PR firms? This person with important title from important-sounding organization says yes. Critics like this other guy say no. The more important person says the critics are wrong. The only thing that is certain is that the debate about whether think tanks are just PR firms continues.

<comment><USER6> Tom Logan from the Institute For Studies says that Think Tanks are an "Invaluable source of impartial opinion that help reflect the thoughts, movements and science that drives the world." Ribena de-Farquhar-Toss from the Policy Research (PR) Think Tank disagrees, suggesting that Think Tanks form "part of the process by which corporations and governments brain wash us into spending money we haven't earned on shit we don't need."

</comment></comment></comment></comment></comment></comment><comment><USER7> I was part of getting something similar off the ground the other year (Web Ecology Project, www.webecologyproject.org).

From what I've seen a Think Tank basically acts like an academic research group, but does commissioned research when possible. Getting started is a bit of chicken and egg thing, but once you're rolling you're kicking out research which seems to entice private companies into wanting to know more or use your tools.

With Web Ecology Project, we started off doing research on Twitter stuff, happened to grab every tweet out there on the Iran Election in 2009, self published a paper, and then boom. Tons of people wanted our data, expertise, analysis, etc. We weren't ready for this business-like shift, so we somewhat self-imploded for being able to run with it business-wise, but we're still doing research.

<comment><USER2> Its funny I had the EXACT opposite experience. I realized I wanted to make the products and left to look for opportunities to do so. But I think the point is that people use think tanks for self-affirmation. Whether its research, talking points, connections - my boss once said - it's better to be used than useless.

</comment></comment><comment><USER8> Most of their work is commissioned, that is, some(one/group/thing) comes to them and ask for their opinion, or they sell topical reports once in a while.

Most think tanks have a bias, and have their "findings" used as justifications.

<comment><AUTHOR> Which makes me inherently think to never trust anything they say.

</comment></comment><comment><USER9> Some do a lot more than just commissioned research. It's a very social business and one that, well beyond simple commissioned reports, is used by customers to maintain asymmetric advantages in information awareness and social access. (Some of the best established tanks are staffed in part by elites who, interestingly enough, achieve some notoriety outside of work for throwing some of the most famous elite dinner parties and such.)

Besides commissioned reports:

Expensive periodicals: informing an elite of little known new developments in a field of interest. For example, one article might be a survey of recent research in microprocessor fabrication written for an audience of investment bankers and CEOs.

Expensive membership libraries: Over time, some tanks accumulate extensive libraries of reports, many of which remain useful long after they're written. You can buy one-off copies or buy memberships.

Elite Conferences: Some run invitation-only conferences.

Public speaking, article publishing: self explanatory.

Lobbying: self explanatory.

Being a social butterfly: This requires some explanation because generally money does not change hands. In some cases, the reason the tank is able to do the kind of research is because it "knows all the right people" (and knows the diplomacy of how to speak with them without raising too many legal or social problems). Thus, a report might just come from looking up lots of open source stuff and synthesizing ... or it might come in part from having a lot of conversations with a related elite (who, normally, are barred or just would not speak directly with one another on the matter). As the diplomat, spreading around selected information while protecting sources, etc., they can help elites in industry and government to be "situationally aware" in ways that the general public can not.

Nobody pays them for that but the social butterfly role helps to reinforce the "brand" of the more serious tanks, and so when people are ready to spend on something, they have some reason to think of the tank.

I imagine that these days there are innovations I haven't seen yet (but that must be there) like "hidden" social networks, and such.

</comment><comment><USER10> From my experience at a political think tank, money comes from big donors who want intellectual support of their pre-conceived positions. There is some "independent research" that goes on along the edges of the "accepted stances," but when a big donor wants an article slanted this way or that, they get it.

</comment><comment><USER11> From what I can deduce, they get paid to cheerlead or spread FUD by parties with a stake in their area of expertise.

</comment><comment><USER12> Generally they don't; their overheads are covered by those with a vested interest in supporting their continued existence.

Usually they exist to analyse details and implications of policy decisions proposed or opposed by individuals and organisations that fund them.

</comment><comment><USER13> In my (limited) exposure to the think tank "industry", it seems that many make a fair chunk of their bottom line through memberships; companies buy memberships to receive publications & analysis, newsletters, networking opportunities and a voice in any policy processes the think tank can get involved in.

Some member companies will pay, partially or fully, for their memberships with services rendered; which means the think tanks have ready access to good and plentiful legal representation & financial advice/management, which must help lower costs quite a lot.

</comment><comment><USER14> I wonder if it would be possible to create a Think Tank that actually produces ideas, instead of laundering wishful thinking? This could be constructed as a consortium into which companies pay money. The output would be in the form of reports. The idea is to give the money going in a bit of "Platonic Amnesia."

This is a function that Universities used to perform.

<comment><USER15> They could give you money, or they could give that other think tank money that produces reports in their favor...

<comment><USER14> In this case, those are exactly the customers you don't want and who don't want you.

<comment><USER15> That's the existing market. It also makes sense, businesses generally don't behave altruistically. Perhaps approaching businesses at all is a mistake, maybe wealthy individuals, social-angels per se.

<comment><USER14> It's not that businesses don't behave altruistically. It's that humans are usually too short sighted to be willing to pay for honest research of that depth.

<comment><USER15> So, you've ruled out humanity as potential clients.

<comment><USER14> Only most of them.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> think tank == lobbying

</comment><comment><USER17> Catch 22?

</comment><comment><USER18> Think Tanks are paid to consider all possible and plausible eventualities, forecast outcomes and correlate dependencies for their given client or scenario.</comment>


32c3: live recordings (unedited) - <AUTHOR> https://streaming.media.ccc.de/32c3/relive/ <comment><AUTHOR> Skip about 15 minutes to get to the talk.</comment>


SHA-1 collider: Make your own colliding PDFs - <AUTHOR> https://alf.nu/SHA1 <comment><USER2> This was a good explanation of what's happening here from a previous thread: [https://news.ycombinator.com/item?id=13715761](https://news.ycombinator.com/item?id=13715761)

The key is that essentially all of the data for both images are in both PDFs, so the PDFs are almost identical except for a ~128 byte block that "selects" the image and provides the necessary bytes to cause a collision.

Here's an diff of the 2 PDFs from when I tried it earlier: [https://imgur.com/a/8O58Q](https://imgur.com/a/8O58Q)

Not to say that there isn't still something exploitable here, but I don't think it means that you can just create collisions from arbitrary PDFs.

edit: Here's a diff of shattered-1.pdf released by Google vs. one of the PDFs from this tool. The first ~550 bytes are identical.

[https://imgur.com/a/vVrrQ](https://imgur.com/a/vVrrQ)

<comment><USER3> I didn't get a chance to make this point in that other thread, because the thread [1] of its follow-ups quickly morphed from promising [2] to meandering, but the combination of lax formats (PDF and JPEG in this instance) makes this style of collision particularly reductive, and in a sense, a cheapshot, if still devastatingly damaging given both PDF's and JPEG's ubiquity -- both separately and together -- in document storage and archival.

This shows the importance of techniques like canonicalization and determinism, which ensure that given a particular knowledge set, that result could only have been arrived at given exactly one input. For general-purpose programming languages like PostScript, of which PDF is a subset, this is essentially an unfulfillable requirement, as any number of input "source code" can produce observationally "same" results. Constrained formats, and formats where the set of 'essential information' can be canonicalized into a particular representation should be the norm, rather than the exotic exception, especially in situations where minute inessential differences can be cascaded to drastically alter the result.

[1] [https://news.ycombinator.com/item?id=13715761](https://news.ycombinator.com/item?id=13715761) [2] [https://news.ycombinator.com/item?id=13718772](https://news.ycombinator.com/item?id=13718772)

<comment><USER4> > Constrained formats, and formats where the set of 'essential information' > can be canonicalized into a particular representation should be the norm, > rather than the exotic exception, especially in situations where minute > inessential differences can be cascaded to drastically alter the result.

That might be very challenging in practice, because a more expressive language directly allows a more compressed/efficient encoding of the same information, but at the cost of being more difficult (or impossible) to create a canonical representation.

Also, data formats that are purposefully redundant for error tolerance all basically have the property that readers should be tolerant of non-canonical forms. If we want to redundantly represent some bytes redundantly in case of data loss, then there _must_ be multiple representations of those bytes that are all acceptable for the reader for this to work.

Video and image formats use multiple encodings to give encoders the room to make time-space trade-offs.

<comment><USER5> I agree, for anything that a human is supposed to see with the eyes, there are always different representations that look the "same" enough to be indistinguishable.

People should be aware of it, not believe in a non-existing world where it isn't so.

</comment></comment></comment></comment><comment><USER6> Shameless plug: I built my own version of this to collide arbitrary PDFs: [https://github.com/nneonneo/sha1collider](https://github.com/nneonneo/sha1collider)

My version is similar to this, but removes the 64kb JPEG limit and allows for colliding multi-page PDFs.

<comment><USER7> Excellent! This ought to be the top comment; have you submitted it separately? I was considering implemented the same thing, since there wasn't any reason for those limitations. The Google researchers purposely designed this collision to be highly reusable, probably to encourage everyone to generate lots of fun examples which will be spread widely and break systems everywhere :)

I'm surprised that some PDF renderers have problems with JPEG streams that contain comments and restarts. Actually, glancing at the JPEG spec, I didn't even realise that restarts would be needed, I thought this was just done with comments. Is this really "bending" the spec, or is GhostScript buggy, or is the problem that it's assuming that restarts don't occur inside comments without escaping?

<comment><USER6> I did submit it, but I don't think anyone saw it: [https://news.ycombinator.com/item?id=13729920](https://news.ycombinator.com/item?id=13729920). I was trying to finish it in time to catch daytime folks but work got in the way :)

Comments are limited to 65536 bytes, and the JPEG spec doesn't offer any way to break an image stream up except for progressive mode or restart intervals (otherwise, the image stream must be a single intact entity). The trouble is that it's probably not _technically_ legal to stick comments before restart interval markers (putting comments _after_ the markers just breaks all the readers since presumably they are expecting image data). So, GhostScript's JPEG reader gets confused when it fails to see a restart marker immediately following an image data chunk, and aborts decoding.

</comment></comment></comment><comment><USER8> Wow, it works! I thought this was supposed to require Google-level resources and months on processing time. Did the initial collision somehow enable more similar ones?

<comment><USER9> yea, this.. wow.

Is this the first hash function which went from "secure" to collision-as-a- service in a matter of days? Was sha1 particularly weak, or the published research particularly strong? or maybe something else?

<comment><USER10> As other people explained, and just to summarize, this service is a way of reusing Google's specific collision (as a prefix to new collisions), and isn't a way of making arbitrary colliding files or file formats. You can't, for example, use this to make something other than PDFs that collide; for that, you'll have to redo a computation on the scale of Google's!

<comment><USER11> I thought I heard that some file formats have "headers" that go at the end of the file. I think a demo of this was a file that was both a zip and a PNG or something. If I remembered right, you should be able to make a similar hack here.

<comment><USER10> If the _only_ headers are at the end, then yes, that's a really neat idea. I'm not sure of the constraints for zip files. Maybe it would be interesting to brainstorm with some people with a lot of file format knowledge to find additional such formats. But if the formats have any constraints at all on the first few hundred bytes, those generally won't be satisfied by the prefix here.

</comment></comment></comment></comment></comment><comment><USER12> I just constructed a little POC for bittorrent: [https://github.com/michaf/shattered-torrent- poc](https://github.com/michaf/shattered-torrent-poc)

Installer.py and Installer-evil.py are both valid seed data for Installer.torrent ...

<comment><USER13> Nice, you're able to line it up so the colliding data starts right at the beginning of one of the torrent blocks.

</comment></comment><comment><USER14> According to [the Shattered paper]([http://shattered.io/static/shattered.pdf](http://shattered.io/static/shattered.pdf)), the reason why the proof-of-concepts are PDFs is because we are looking at a

> identical-prefix collision attack, where a given prefix P is extended with > two distinct near-collision block pairs such that they collide for any > suffix S

They have already precomputed the prefix (the PDF header) and the blocks (which I'm guessing is the part that tells the PDF reader to show one image or the other), and all you have to do is to populate the rest of the suffix with identical data (both images)

<comment><USER15> Yep. With any hash function that takes input as blocks, if you were to ever get two messages to generate the same _EDIT_ internal state, you could add the same arbitrary data to both and get those new messages to have the same hash.

</comment></comment><comment><USER16> Is this length extending [1] the already existing Google attack?

[1] [https://en.wikipedia.org/wiki/Length_extension_attack](https://en.wikipedia.org/wiki/Length_extension_attack)

Edit: yes, looks like it is.

As sp332 and JoachimSchipper mentioned, the novelty here is that it contains specially crafted code in order to conditionally display either picture based on previous data (the diff). I can't grok PDF so I still can't find the condition though. Can PDFs reference byte offsets? This is really clever.

Edit #2: I misunderstood the original Google attack. This is just an extension of it.

<comment><USER8> It seems so. I can add the same arbitrary data at the end of two pdfs generated by this tool, and they are still a collision. I didn't know SHA-1 is so susceptible to length extension. Is there no internal state in the algorithm that would be different even if the hash output is identical?

<comment><USER15> If you were to somehow get two messages with the same SHA-3 hash, you could keep on appending the same data to both and they would keep the same SHA-3. But SHA-3 is explicitly not vulnerable to length extension attacks.

<comment><USER16> No they wouldn't, since its internal state is different than the output.

Same goes for SHA-224 and SHA-384.

<comment><USER15> Damn, right, you have to get them with the same _internal state_.

</comment></comment></comment></comment></comment><comment><USER17> I don't understand the whole collision thing. I mean a sha1 is 160bits so if you are hashing information longer then that collision is a fact, being able to forge a piece of information with constraints is the challenge and even that with enough power you end up being able to try all the combinations. What I understand from that collision reported is that they use PDF format which can have as many data inserted to it as comment /junk as you want so all you need is enough processing power to find the right padding/junk to insert to get the collision. Am I missing something here ?

</comment><comment><USER18> I would imagine that a lot of old data is secured by SHA1, which may be available for attack.

Does anyone have any idea about a broad risk-assessment of systems worldwide that might be vulnerable as SHA1 becomes easier and easier to beat?

<comment><USER15> If by "secured by SHA1" you mean "someone generated a hash using SHA-1 and we use the validity of that hash to guarantee we have the right document," that's still okay. We're a long way from being able to make documents with a given SHA-1.

(Edit: Any _newly_ signed documents, or documents signed recently, are not safe, because an nasty person could have made two, one to get signed by the system, another to do evil with.)

SHA-1 is officially deprecated for certificates, because of the example that OP shows. You can create two certificates, have the decent one get signed by a CA, and then use the evil one to intercept traffic.

<comment><USER18> Thanks for the info. Good point, I suppose anyone relying on SHA1 in 2017 has had ample warning about its weaknesses.

It seems that there is also a very strong incentive for anyone receiving anything whose authenticity is verified by SHA1 to request an improved hashing algorithm.

</comment></comment></comment><comment><USER19> Practical question: does this generate a "harmful" (harmful to a repo system like SVN) PDF if the flaw in the hashing is enough to crash/corrupt the system?

<comment><USER19> To answer my own question looks like Gmail has flagged PDFs generated with _this_ specific hash as harmful.

</comment></comment><comment><USER20> Doesn't work for me. One of PDFs always says "Insufficient data for an image" (sometimes for the same image that worked before).

<comment><USER21> I found that you have to reload the page if there's an error or it'll stick. In my case it was too big of a file.

</comment></comment><comment><USER22> Damn that didn't take long to go from $100K to carry out this attack to a single day to offer a website for SHA1 collision as a service...

<comment><USER23> Just wait until you get the invoice

<comment><USER24> Is the invoice in a PDF? What's the SHA-1 hash of it?

</comment></comment></comment><comment><USER25> What's the smallest file you can make collide? Could you make two files collide that are actually smaller than their hashes?

</comment><comment><USER26> Question: What does git use to hash blobs? Is it SHA-1?

Would that be a problem? Ramifications are unclear to me.

<comment><USER26> Ah, note that apparently it is SHA-1, and this question was common enough that Linus himself has addressed it:

[https://news.ycombinator.com/item?id=13733481](https://news.ycombinator.com/item?id=13733481)

</comment></comment><comment><USER27> Just tried it and it really got me the same sha1... damn...

</comment><comment><USER28> I think the coolest proof was that the Bitcoin SHA1 collision bounty was claimed:

[https://bitcointalk.org/index.php?topic=293382.0](https://bitcointalk.org/index.php?topic=293382.0)

The OP_2DUP OP_EQUAL OP_NOT OP_VERIFY OP_SHA1 OP_SWAP OP_SHA1 OP_EQUAL script was making sure that only a person who finds 2 SHA1 colliders and publishes it can get the 2.5 BTC bounty.

<comment><USER29> HN discussion: [https://news.ycombinator.com/item?id=13714987](https://news.ycombinator.com/item?id=13714987)

</comment></comment><comment><USER21> I just tested Backblaze and found that its deduplication is broken by this. If you let it backup the two files generated by this tool, then restore them, it gives you back two identical files instead of two different ones.

<comment><USER30> I have never been particularly comfortable with the idea that you can simply assume that if the hashes are the same, then the data must be the same.

The fact that collisions happen so rarely (and sometimes only after the a hash function has become compromised) makes this even more dangerous.

It's like a couple of decades of strong hash functions has made people forget what hashing actually is.

</comment></comment><comment><USER31> I wanted to try this tool, but the upload requirements were so stringent (must be jpeg, must be same aspect ratio, must be less than 64KB) that I gave up. Would be nice if sample jpegs were provided on the page.

<comment><USER32> ... are you joking?

<comment><USER33> (not OP) I understand why the requirements are strict so I can see why s/he was being downvoted, but I do agree that samples would be nice. Like, do I have to go search for some images with these requirements just to see if it really works (since it was supposed to take $100k)? By now someone mentioned it works in the comments though, so I guess I'll trust them.

<comment><USER34> Is it really that hard to take a screenshot of a couple windows on your desktop and run:

$ convert screenshot.png -resize '500x500!' bar.jpg

or

$ sips -s format jpeg -Z 500x500 screenshot.png --out bar.jpg

Generating some workable source images is trivial. You're not interested in making a 'perfect hack pdf' for some nefarious purpose, just seeing if the service does what it says it does.

<comment><USER31> > Generating some workable source images is trivial

Everything is trivial if you know how to do it off the top of your head. Generating white noise samples is trivial. UV-mapping a texture to a mesh is trivial. Soldering resisters to a PCB is trivial. Generating an ARM toolchain with Yocto Project is trivial.

Is it a crime that I didn't feel like researching a bunch of CLI tools I've never heard of to try to use an app I was only mildly curious about?

<comment><USER35> This overstates the difficulty of creating a JPEG. If you have an image editor on hand that isn't MS Paint (or you're using macOS, which has Preview), you merely need to export a JPEG and choose a low quality setting.

For a Windows user with no decent image editor or viewer installed, though, it could certainly be a hassle.

<comment><USER31> None of the things I listed are difficult. They just require the right tool and know how. For someone like me with Xubuntu, I had neither the tools nor know how for creating small jpegs. I didn't fell like wasting 30 minutes researching and I didn't feel like walking over to a different computer that has MSPaint.</comment></comment></comment></comment></comment></comment></comment>


Logswan – Fast Web log analyzer using probabilistic data structures - <AUTHOR> https://github.com/fcambus/logswan <comment><USER2> If you're interested in the probabilistic approach, this is how it works: [https://en.wikipedia.org/wiki/HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog)

"The basis of the HyperLogLog algorithm is the observation that the cardinality of a multiset of uniformly-distributed random numbers can be estimated by calculating the maximum number of leading zeros in the binary representation of each number in the set. If the maximum number of leading zeros observed is n, an estimate for the number of distinct elements in the set is 2^n."

</comment><comment><USER3> If anyone involved in the project is reading this the DNS entry for "www.logswan.org", available as a link on the github page, does not exist.

<comment><USER4> Thanks for reporting this.

Indeed, there was no site configured on logswan.org when this was posted to HN. I made the required changes but due to the nature of DNS, it'll still return NXDOMAIN for some users until caches are cleared.</comment></comment>


Apps for staying connected while quarantined - <AUTHOR> https://www.apartmenttherapy.com/apps-staying-connected-social-distancing-36736720 <comment><USER2> Also:

Pidgin

Xabber

Konversation

OpenMeetings

Asterisk

Thunderbird</comment>


Pure-Ruby solution for a highly scalable Redis-backed social activity feeds - <AUTHOR> http://kig.re/2017/02/19/feeding-frenzy-with-simple-feed-activity-feed-ruby-gem.html <comment><AUTHOR> Github URL is here: [https://github.com/kigster/simple- feed](https://github.com/kigster/simple-feed) and the ruby gem is here: [https://rubygems.org/gems/simple-feed](https://rubygems.org/gems/simple-feed)</comment>


A Guide to Aussie Slang - <AUTHOR> https://nomadsworld.com/aussie-slang/ <comment><USER2> Have to laugh at the list.

I would dispute 2 and 21.

38 and 55 are impolite.

12 is actually a tine can (about 2 litres in size) with a wire handle tyo boil water - for tea or coffeee (in a pinch)

36 and 51 not used much.

Never heard of 41, 49, 62, 68, 78 and 98.

A couple of them like nuddy and u-ie are pronounced differently from written. nuddy has two pronunciations that I know of - the first is like nuhddy and the second is like nouddy (as in could). u-ie is pronounced like you-ee.

The funny thing about the Australian language is that we can still be speaking slang when we are speaking formally.

Have fun</comment>


New lawsuit says Google refused to hire white and Asian men - <AUTHOR> https://www.theverge.com/2018/3/2/17070624/google-youtube-wilberg-recruiter-hiring-reverse-discrimination-lawsuit <comment><USER2> Its being posted so much because people want to talk about it without it being flagged or duped.

Plus its google and hiring related.

<comment><USER3> To be fair though, you're posting from an account that was created to post on this story a few days ago. The third comment on this is from an account created 24 minutes ago, for this thread.

Combined the story has more than 500 points from time on the fp, but that's not enough? I'm trying to imagine the reaction if the "What Happens To Us..." story were similarly treated?

<comment><USER2> Yea because its related to hiring and discrimination.

<comment><USER4> It's a super hot topic that gets discussed a ton, probably more than any other in the last year. There are plenty of other threads about this theme and will be plenty more. Obviously it isn't being suppressed.

What does get penalized on HN are (a) dupes and (b) copycat or follow-up posts trying to hog in on their slice of the click-pie. That's a site quality issue that has nothing to with political sides.

</comment></comment></comment></comment><comment><USER3> Is this the fifth or sixth time this story has hit the front page? Seriously, I'm losing track. I want to think that the fetishistic reposting and popularity of this story doesn't say terrible things about some people here, but each time it makes the front page that's harder to believe.

76 points [https://news.ycombinator.com/item?id=16498586](https://news.ycombinator.com/item?id=16498586)

351 points [https://news.ycombinator.com/item?id=16497551](https://news.ycombinator.com/item?id=16497551)

17 points [https://news.ycombinator.com/item?id=16513739](https://news.ycombinator.com/item?id=16513739)

23 points [https://news.ycombinator.com/item?id=16504820](https://news.ycombinator.com/item?id=16504820)

58 points [https://news.ycombinator.com/item?id=16500834](https://news.ycombinator.com/item?id=16500834)

Sixth time it seems.

<comment><USER5> Bashing Google over this on HN (and elsewhere) seems close to (persistent) astroturfing to me. It would be interesting to actually do the work and investigate where the various mentions on HN (and the original articles) come from.

But then again, this stuff sells in the news, so why suspect ulterior motives ;)

<comment><USER4> It breaks the site guidelines to insinuate that behavior you disagree with must be astroturfing. Please see [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html).

I realize it's a very persuasive-feeling impression. It's increasingly the first thing people bring up to account for disagreement, but that is a huge problem. Overwhelmingly the real reason for disagreement is that people just disagree, and we get nowhere by pretending otherwise.</comment></comment></comment>


The Science Behind Ear Boners - <AUTHOR> http://www.nature.com/neuro/journal/v14/n2/full/nn.2726.html

<comment><USER2> Title should match that of article: _Anatomically distinct dopamine release during anticipation and experience of peak emotion to music_

<comment><AUTHOR> No.</comment></comment>


Ask HN: what does Obama's to-do list look like? - <AUTHOR>

Our to-do list-ey web app FolderBoy (www.folderboy.com) needs a sample to-do list.  We're going to pre-populate our list with a fake Obama to-do list.
It's easy to find out what Obama's done, but since I don't live in America, I don't know what his upcoming to-do list would look like.
E.g.
Work tasks
<comment>----
Withdraw from Iraq
Stabilize Afghanistan
Confirm Elena Kagan (done)
Reform health (done)
???? </comment><comment><USER2>
      Install Basketball court at White House
      Burger with Russian President (done)
      Sign stuff

:P

<comment><AUTHOR> Thanks! This is exactly what I'm after. I assume I can use those things in the demo account? :)

<comment><USER2> Go for it!

I think you're better off using more like my examples than making political statements, like the Iraq withdrawal etc.

I'm sure you can find more humorous items to add to the list. Just Google _Obama meets with.._ and find your next target

</comment></comment></comment><comment><USER3> well, most things are debatable as we don't all agree on what things like "reform health" mean.

There is a site that tracks his campaign promises and their defintions of the promises. <http://www.politifact.com/truth-o-meter/promises/>

Seems like a good place to start, but of course, it's debatable

</comment><comment><AUTHOR> clickable: <http://www.folderboy.com></comment>


Ask HN: What are the benefits of having a project webpage vs. GitHub README? - <AUTHOR> Aside from Google Analytics are there any other benefits to having a project webpage for an open source project?
Are there any guides or statistics on the impact of having a webpage vs just a github repo?
I'm artistically challenged, so I prefer to just have a README.md, but wondering if I'm missing out by not doing so? I'm currently at ~450 stars with just readme and an asciicast gif: https://github.com/aelsabbahy/goss
However, I noticed other open source projects in this same category have webpages. <comment><USER2> If you ever want/have to move off GitHub, it's easier if people link to a domain you own instead of GitHub. Or if you want to establish a site later, e.g. with more detailed information.

<comment><AUTHOR> So right now I have a simple 302 redirect from a domain I own. I guess my question is more about the benefits of having a "site with more detailed information" vs readme + docs in markdown format in the repo itself?

<comment><USER2> Readmes and most docs on GitHub are fairly linear, a normal site with navigation menu is IMHO easier to navigate. I want to be able to search docs once they are longer than a single page.

What I mean with linear:

With the repo you linked, I have to scroll past the file list, then the readme-header, then your twitter links, then the gif to even get to the introduction which tells me what I'm looking at. Next is installation, which has to be in the readme, but I don't care right now because I still want to see more details. With e.g. a navigation bar on the side I could have simply clicked on "Introduction" and then "quickstart" to get what I want.

And all GitHub repos look the same, so I won't recognize the site by it's design if I find it again a while later. Nothing memorable there.

That doesn't mean that a site should be a high priority for you, but these are some reasons why you might want one.

<comment><AUTHOR> Makes sense, any examples of good/simple sites that come to mind?

</comment></comment></comment></comment><comment><USER3> I think that a project webpage gives a more attractive impression to other people.

<comment><AUTHOR> Does it though? I feel I usually prefer to go to a github page of a project and will only resort to their website if the documentation only lives there. Maybe I'm in the minority :(</comment></comment>


Ask HN: How Is Azure SQL Data Warehouse? - <AUTHOR> We're looking into Redshift and Snowflake for a project since some of our sister companies use it and they seem more established but we're also on the Microsoft stack and use Azure for miscellaneous small projects. Our DW will be roughly 2-4TB initially.
FiveTran had a fantastic benchmark on GitHub (https://news.ycombinator.com/item?id=17952205) but I want to hear direct user experiences.
And specifically, if you are using Gen2 (https://www.zdnet.com/article/azure-sql-data-warehouse-gen-2-microsofts-shot-across-amazons-bow/)
1. What did you like about it over XYZ DW? What was lacking compared to XYZ DW?
2. How were development tools and third party tool support?
3. Satisfied with performance? <comment><USER2> We did a migration from Vertica to ADW a couple of years ago.

ADW was slower and about 2x more expensive than Vertica on 8-year-old hardware, and it hit its node limit at around 60. Loading was also a PITA; there was no json support.</comment>