How My Popular Site was Banned by Google - <AUTHOR> http://kbrower.posterous.com/banned-from-adwords-and-google-search-in-less <comment><USER2> You have an autogenerated web site that consists of practically nothing other than affiliate links to Amazon. You can make an infinite number of autogenerated pages on your site, e.g.

<http://www.filleritem.com/index.html?q=hacker+news>

<http://www.filleritem.com/index.html?q=31.69>

<http://www.filleritem.com/index.html?q=teen+sex>

and each autogenerated page consists of literally hundreds of affiliate links stuffed with keywords for unrelated products.

When Google's webspam team takes action on websites in our websearch index, we can pass that information on to the ads group so they can check for violations. But it's a one-way street: we can send the ads team signals or information about spammers or other violations of our quality guidelines, but the ads team doesn't send information over to the quality/webspam team.

<comment><AUTHOR> Hi Matt,

Thanks for responding.

The point of the site is to find items of a particular price that qualify for free shipping on Amazon.com. If you want I will give you access to my google analytics to show you that this is a site people want and use.

I guess I should not let people link to the search results page?

<http://www.filleritem.com/index.html?q=hacker+news> and [http://www.filleritem.com/index.html?q=anything+that+is+not+...](http://www.filleritem.com/index.html?q=anything+that+is+not+a+number) returns the same thing as if you did a search for $0. I will fix the bug so it returns an error. These are pages that no one has ever linked to as far as I know.

<comment><USER2> My advice would be to use robots.txt to block out the autogenerated pages; when users search for e.g. a long-tail phrase and then land on a page that's nothing but affiliate links with a lot of keywords, they tend to complain to us.

Users would be happier if they landed on the root page of your site or the root page of <http://www.superfillers.com/> or <http://www.filleritemfinder.com/> than if they landed on a deep page full of links and unrelated products.

<comment><USER3> <http://www.filleritemfinder.com/> is the first hit for "filler item" on google right now.

A search for hacker news results in a page full of affiliate links, just as the example you gave above. Only difference is that they didn't re-write the URL.

filleritemfinder.com has no robots.txt that I was able to pull up.

So, filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the number one result.

Further, shouldn't this kind of advice be given to people who appeal being excluded from the index? Or should we all post to Hacker News when it happens to us so that you can come explain directly?

I think %50 of the problem is the arbitrary picking of sites to block (and it's not working, btw[1]) and %50 of it is that google seems uninterested in explaining or advising people when it happens to them.

[1] Been buying gear for a project lately, and so doing a lot of google searches in the form of product-model-number review or product-name review. Overwhelmed with spam sites, and mindless human generated spam sites like dpreview.com, etc.

<comment><USER2> When I do the search [filler item], one of the top results I see is [http://www.makeuseof.com/tag/top-5-amazon-filler-item- finder...](http://www.makeuseof.com/tag/top-5-amazon-filler-item-finders- qualify-free-shipping/) which shows five different services to fill this information need, and that also has pictures. I do think that's a pretty helpful result.

I mentioned filleritemfinder.com as a random example (there are many of these services), but filleritemfinder.com appears to use AJAX to keep results on the same page rather than making a new url.

"filleritem.com, a google customer, was blocked, but filleritemfinder.com, doing the same thing, is the number one result."

The filleritemfinder.com site is not doing the same thing, because it's not generating fresh urls for every possible search. But you're not really suggesting that we should treat advertising customers somehow differently in our search results, are you? The webspam team takes action regardless of whether someone is an advertising customer or not.

"shouldn't this kind of advice be given to people who appeal being excluded from the index?"

This advice is available to everyone in our quality guidelines. It sounds like the site owner reached out to the AdWords team, which gave him clear guidance that the site violated the ads policy on bridge pages. It sounds like the site owner also filed a reconsideration request, and we replied to let the site owner know that the reconsideration request was rejected because it was still in violation of our policies. It doesn't look like the site owner stopped by our webmaster support forum, at least that I could see in a quick look. At that point, the site owner did a blog post and submitted the post to Hacker News, where I was happy to reply.

<comment><USER4> Would it be likely that if the site was to use another monetization plan that it would be less likely to be penalized e.g. advertising instead of affiliate links?

<comment><USER2> I recently talked for about a minute about the topic of "too much advertising" that sometimes drowns out the content of a page. It was in this question and answer session that we streamed live on YouTube: <http://www.youtube.com/watch?v=R7Yv6DzHBvE#t=19m25s> . The short answer is that we have been looking more at this. We care when a page creates a low- quality or frustrating user experience, regardless of whether it's because of affiliate links, advertising, or something else.

<comment><USER4> Thanks Matt, took a look at the video, that answers my original question.

Also is there a preferred monetization model, e.g. do Google think advertisements are more or less harmful to the user experience than affiliate links, sponsored posts, etc?

Obviously across different models you can't just track space taken up, so is there some kind of metric that tracks the rate of content diluting via monetization?

<comment><USER5> I know you were looking for an answer from Matt but I thought I would offer up my opinion here (as you might have noticed, I love talking about this stuff).

Our models currently suggest that the presence of contextual advertising is a significant predictive factor of webspam.

We use 10-fold bagging and classification trees, so it's not all that easy to generalize. But I pulled one model out at random for fun.

The top predictive factor in this particular model is the probability outcome of the bigrams (word pairs) extracted the visible text on the page. Here are a few significant bigrams:

relocation _companies products_ providing products _pure quality_ book recruitment _website tickets_ our these _traffic representing_ clients today _play tours_ high registry _repair rent_ properties wedding _portal printing_ canvas pr _human privacy_ protection providing _efficient way_ trade printing _stationery prices_ everything website*daily

Next, this model looks for tokens extracted from the URL and particular meta tags from the page. Similar to above, but I believe unigrams only. A few examples follow. Please keep in mind that none of these phrases are used individually... they are each weighted and combined with all other known factors on the page:

offer review book more Management into Web Library blog Joomla forums

The model then looks at the outdegree of the page (number of unique domains pointed to).

From there, it breaks down into TLD (.biz, .ru, .gov, etc)

The file gets pretty hard to decipher at this point (it's a huge XML file) but contextual advertising is used as a predictive variable throughout.

Just from eyeballing it, it appears to be more or less as significant as the precision and recall rate of high value commercial terms, average word length (western languages only), and visible text length.

Based on what I'm looking at right now, my answer would be that sponsored posts are going to be far more harmful to the user experience than advertising.

Can't answer the rest of your question which I assume relates to the number of ad blocks or amount of space taken up by ads... we don't measure it.

Edit: Just realized that Google will probably delist this page within 24 hours. Should've used a gif for those bigrams. Oh well ;-)

<comment><USER4> Thanks for your response, is the data you are viewing publicly available?

<comment><USER5> No...

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> From google's bridge page definition:

Not acceptable: "Websites that feature links to other websites while providing minimal or no added functionality or unique content for the user Added functionality includes, but isn't limited to, searching, sorting, comparing, ranking, and filtering"

The page in question provides the added functionality listed (searching/filtering at least)

<comment><USER7> The filleritem.com website is a very simple tool to search for items of a certain price on Amazon. The purpose is to help select an item to fill the gap in the shopping cart to qualify for free shipping. There is no reason for Google to ban this site. What we MAY have here is an abuse of power by Google.

</comment></comment><comment><USER8> I can't judge if you broke the rules or not but not showing up on Google's SERPs is terrifying and a stark reminder on how dependent we've become on Google traffic.

<comment><AUTHOR> I included an image of my adwords account. The ad had been running for years(albeit infrequently due to very low budget), and I had not touched it since creating it.

I checked after this all happened, and there were several warnings from adwords, but nothing from webmaster tools. I simply ignored the adwords warnings as I was not actively using the account and figured they were just disabling ads. If I had know these were the consequences, I would have just deleted my adwords account.

<comment><USER9> Unfortunately, deleting an AdWords account won't remove the violation; it may actually make it worse. Google will look back several years and disable ads that haven't run for a very long time. Sometimes they even ban accounts that are inactive or deleted!

It is very important to remember: when dealing with Google AdWords, the definition of a violation changes with time. If you violated the current policies in the past, you may still be banned. You take a great risk running there and any and all warnings you get should be dealt with immediately.

That said, seeing an exclusion from organic based on an AdWords suspension is extremely alarming. That may be exactly what the DoJ is looking for in terms of violations by Google.

<comment><USER10> _That said, seeing an exclusion from organic based on an AdWords suspension is extremely alarming. That may be exactly what the DoJ is looking for in terms of violations by Google._

According to Matt Cutts, it was the exact opposite: he got the organic suspension _then_ the AdWords suspension.

<comment><USER9> I saw, much less alarming now.

</comment></comment></comment></comment></comment><comment><USER11> Google should really get some healthy competition.

filleritemfinder.com is reacheable from google, and as far as I can tell, has the same functionality, with better design. Maybe some google guru could comment if a redesign would save you.

</comment><comment><USER12> Another great example of Google randomly removing sites based on flimsy or nor evidence. The fact they don't give you a chance to plead their case is pretty scary.

My hunch is one of your competitors got the site taken down. I've had a few clients who had their competitors file complaints with Google, or inform them they were using black hat SEO to get on the first page of the SERPS. In one case they were successful, but I had the site back up in less than 24 hour,s so it wasn't a huge deal.

It sounds like it might be an ongoing issue and are not going to put your site back in the SERPS for a while. I feel for you brother.

<comment><USER13> But often there is a reason why a site vanishes from the Google index. We would have to exclude all possible violations of the Google guidelines, before we could safely claim that this is a great example for random removal by Google.

Edit: In a comment above Matt Cutts from Google gives an explanation.

</comment></comment><comment><USER14> Google's definition of a bridge page: [http://adwords.google.com/support/aw/bin/answer.py?hl=en&...](http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=190435)

</comment><comment><USER15> It's a website filled with autogenerated content and links, how are you surprised that you got sandboxed?

</comment><comment><USER16> To Matt Cutts and the search quality team: Why does sites like this rank for every possible term with autogenerated results (such as filleritem.com)?

Nettkatalogen.no in Google.no ranks for about every possible term.

[http://www.google.no/search?pws=0&q=h%C3%A5ndverker+oslo](http://www.google.no/search?pws=0&q=h%C3%A5ndverker+oslo) [http://www.google.no/search?pws=0&q=snekker+trondheim](http://www.google.no/search?pws=0&q=snekker+trondheim) [http://www.google.no/search?pws=0&q=r%C3%B8rlegger+troms...](http://www.google.no/search?pws=0&q=r%C3%B8rlegger+troms%C3%B8)

(They use the "powered by Google logo". They use very aggressive phone sales tactics and scam people. They buy links from newspapers.)

Is www.nettkatalogen.no violating Google Quality Guidelines? Or is this okay?

</comment><comment><USER17> Dropping out of the organic results sounds like a result of the Panda update ([http://searchengineland.com/official-google- panda-2-3-update...](http://searchengineland.com/official-google- panda-2-3-update-is-live-87230))

My best guess is that you'll need to include some additional 'unique content' on the site to get it back in the listings. At the moment the Google crawler will see not a whole lot of text, quite a few Javascript buttons and some links, and will probably flag it as low quality.

It may be a bit of a hassle, but may be worth adding a kind of 'interesting/unique filler items' blog to the site to increase the content to links/js ratio.

</comment><comment><USER18> <http://www.filleritem.com/index.html?q=teen+sex> This should have no results. Other queries should have relevant results, but they don't. How can anyone complain over his site being axed?

The guy needs to code a better site and I sure as hell would never use that so he can get affiliate money.

Sorry, but your fortunate this site has survived for so long.

<comment><USER19> If you look around, almost everything you click on the web is set up to make someone money. It's none of Google's business how a website is making their money as long as it's legal and follows FTC policies. Browsers make money every time you use that little search box in the top right corner of Firefox. People have a right to be compensated for their work. If the site provides a good service, I'll use it. If you don't want people to make money off your clicks. It would be best to close your browser and never open it again.

My site got hit by Panda (no manual penalty). After relaunching my site (which I had luckily been working on for over a year), my bounce rate is 30% and my direct link rate is 50%. Still, Google has me by the throat. My traffic is almost dead. Panda doesn't care if you're solving a problem for users and giving them the eye candy they want. It apparently just wants your site to be a "unique, well-written" news site. People come to my site to browse vintage items and find stuff that they didn't know existed in the niche. They usually visit during work hours and late in the evening. Reading ease for these items is on a college level (Which Panda doesn't like). They don't want to read a 1000 word article, they usually just want to hear it, see it, maybe ask question or two about the item and see who may be selling one. I offer that.

Panda's obvious Bayesian nature doesn't understand nuance. It only knows napalm.

Niche markets are the ones getting the crap end of the stick. And it's funny, niche markets used to be what made the web so great! Not anymore. Now it's 100% Googlized Walmart-ization.

And yeah, I'll be using www.filleritem.com when shopping Amazon. I didn't know it existed until I read this post. I don't mind, transparently, giving someone a few pennies for a good service. You do it all the time.

</comment></comment><comment><USER20> Yea that sucks, it's happened to me MANY times. The fact of the matter is, it's Googles world and we're just living in it...

<comment><USER13> I don't think that it is as simple as that. There is some form of ecosystem on the web: there are the content creators and website providers, there are the searchers, and there are the search engine providers. Each of these has its own interests and guidelines.

Searchers have quality guidelines too, though they are not explicitly published by them on the web. If a search engine delivers low-quality results, the searchers move on to the next search engine. A search engine without searchers is hardly profitable.

Most of the quality guidelines of Google are just common sense, following them will in many cases increase the usability and findability of a site.

If this has happend to you many times, you are likely doing something wrong. I have never had one of my sites banned from Google.

<comment><USER20> > If this has happend to you many times, you are likely doing something wrong. > I have never had one of my sites banned from Google.

Right. So. YOU'VE never personally had a site banned, therefore others who have must be doing it wrong.

You know what, I've never been hit by a car while crossing the street, therefore others who have must be crossing the street wrong.

I've never been in a plane crash, therefore others who have must be doing it wrong.

</comment></comment></comment><comment><USER21> lol, poor Matt, always under fire from everybody who hasn't got the rankings they want. It's simple really, Google try to make it fair for everybody (and granted it doesn't always work perfectly, which is why the continue to make changes, it's evolution) and if some sites give themselves an unfair advantage and are caught, then they're punished. How else could a search engine work? If it were any different then al of our results would be greed pages with no useful info and everybody would stop Googling stuff.

As somebody once quoted, "You can please some of the people, some of the time, but you can't please all of the people, all of the time!"

<comment><USER5> This is a fair comment but I believe the issues stem more from the process Google follows.

It is obvious that Google cannot communicate exact reasons why a site was penalized as that would help spammers. However, there is nothing that prevents them from adding a step to warn the offending website and give them a heads up before the ban/penalty takes place, along with an explanation of the policy that is/was being violated.

Most of these heads up would go ignored, some would not and yes, it would incur a support cost. However, the number of websites which are significantly penalized isn't onerous... I believe fewer than 1,000 each year?

When a company has become the defacto gateway to the internet, I believe they have a responsibility to webmasters. Google has lost a lot of goodwill over the years because of these seemingly arbitrary penalties... Instituting such a practice would be a worthwhile investment.

<comment><USER2> option1138, I'm afraid you need to recalibrate your expectations of spam on the web. Blekko made a site called Spam Clock that estimates 1 million spam pages are created _every hour_ : <http://www.spamclock.com/> .

There's 200+ million websites out there. 1,000 spam sites would be a spam rate of 5.0 × 10^-6. If you remember the days of Altavista before Google, the actual rate of spam on the web is much higher. Here's one stat: I once heard a search engine rep (not from Google) say that they had to crawl 20 billion pages to find 1 billion non-spam pages.

So yes, we do tackle more than 1,000 websites a year. There's a ton of spam on the web, and Google has to operate on the scale of the web (e.g. in 40 different languages) to tackle all that spam.

<comment><USER5> Matt,

You are of course correct. The fault is mine for miscommunicating... I find myself becoming less self-editorial these days when I write on the web and tend to think everyone is on the same page as I am.

I was actually referring to an informal study I did earlier this year. I measured sites which were receiving an average of 50,000 or more visitors from Google US search (organic) per month over a six month period. Then I compared those with a similar set from a subsequent six month period to see which had significantly dropped off in traffic and rankings. The purpose of this was to estimate the number of _significant_ sites which were penalized over that period of time. The final estimate came to about 700 sites/year which were penalized. There are lots of uncontrolled variables here of course... but I was looking for an "order of magnitude" answer simply for curiosity's sake.

The 1 million spam pages created per day were of course excluded from consideration as they never received much traffic from Google in the first place.

So just to clarify my earlier response, I am advocating for a policy that would apply to websites exceeding a certain threshold of organic traffic for a significant period of time.

</comment></comment></comment></comment><comment><USER22> For better or worse Google is the gateway to the internet. If your site isn't on Google it is virtually invisible to most of the world.

Since Google has such an enormous position of power, they are really going to have to do better with customer service (and by customer service, I mean providing a way to talk to a human on their end).

I understand it's expensive, but if it doesn't get any better, eventually some government somewhere (that has jurisdiction) is going to regulate them.

</comment><comment><USER23> Just out of curiosity, if only 30% of your results are from organic search, why did your hits drop by more than 30% after you stopped getting organic search hits?

<comment><AUTHOR> that is the google analytics for just organic traffic from google. the site gets ~1000 visitors a day.

</comment></comment><comment> general-marty University should offer courses on Google policy. A thorough understanding of such policies will be the primary determining factor for your success on the web.

<comment><USER24> I disagree with you there general-marty, I think the primary factor determining your success on the web is a decent business plan and marketing strategy. Without that you rely on free traffic from Google. Any company that puts all its eggs in one basket (e.g. Google Organic) is skating on thin ice.

</comment></comment><comment><USER25> wow, good karma +50 to Matt on this one

</comment><comment><USER26> Thanks matt

</comment><comment><USER27> They usually change policy, or enforce existing ones before an algo update that doesn't prove any connection in you case and it's not like they are dependent on your ad expenditure.

</comment><comment><USER28> Your site looks totally spammy-doorway.

</comment><comment><USER29> Amazon probably complained to Google and Google despite already having the rules on the books decided to enforce it on this one site, while also keeping many other low traffic clone sites offering the same service alone. Call me paranoid, or cynical but I've seen much worse from Google on the AdWords side of their offerings.

<comment><USER2> Nope, that didn't happen.

<comment><USER30> This has been an amazing thread but there really is a pattern that is hard to follow for webmasters. In the case of the original site couldn't Google have just de-indexed the "extra" pages but left the the index page alone since it is indeed useful?

Also, how did his site get canned but others like sportslinkup.com, which is an ebay affiliate spam site cloaked as a link directory, have over 7 million indexed pages and 8 million indexed images (all hosted by ebay, not the sports site) and it gets over half a million Google visitors per month (according to compete.com)?!? I think the sports site is even scraping Google for keywords, it's full of examples of what not to do, but it's been sailing along for years with Google approval, or at least no automated detection.

The fine line between sites getting canned and sites getting MASSIVE traffic for essentially the same thing is very confusing.</comment></comment></comment>


Apple's 2014: By the (ridiculously large) numbers - <AUTHOR> http://mashable.com/2014/10/20/apples-2014-by-the-numbers/

<comment><USER2> I wonder how the new iMac 5k Retina is going to influence sales for their iMac line numbers. They increased but I don't think it's just because of this release?</comment>


A Vote for Discrimination - <AUTHOR> https://www.wsj.com/articles/a-vote-for-discrimination-11593127619 <comment><AUTHOR> I am disappointed to see California's legislators vote in favor of removing the following text from the state constitution:

> "The state shall not discriminate against, or grant preferential treatment > to, any individual or group, on the basis of race, sex, color, ethnicity or > national origin in the operation of public employment, public education, or > public contracting."

Equality of opportunity and equality before the law are crucial for a functioning society that is built on laws. This proposed change enables discrimination by the state, in the form of affirmative action for college admissions and potentially other situations as well (like government contracting). I find this to be explicitly racist, but additionally it may even be counterproductive to the stated goals. Following passage of Proposition 209 (which banned discrimination), enrollment of underrepresented minorities in at the state's university system actually increased significantly (see [https://www.ocregister.com/2020/06/23/california- legislature...](https://www.ocregister.com/2020/06/23/california-legislature- moves-to-allow-discrimination-in-higher-education-janet-nguyen/)).</comment>


Name that Blue - <AUTHOR> http://namethatblue.com/

<comment><USER2> I really wish they showed stats. I'd like to see which is the most recognized blue (I immediately recognized Facebook).

</comment><comment><USER3> I have never heard of 80% of the names in there. How can I name that blue?</comment>


You'll access Google's Stadia cloud gaming service - <AUTHOR> https://techcrunch.com/2019/03/19/heres-how-youll-access-googles-stadia-cloud-gaming-service/ <comment><USER2> Actual title is "Here's how you'll access Google's Stadia cloud gaming service". Why was it changed?</comment>


Aaron Swartz v. United States - <AUTHOR> http://blogs.law.harvard.edu/sj/2011/07/24/aaron-swartz-v-united-states/ <comment><USER2> This whole thing weirds me out. Here we are on Hacker News, and yet there is a super-large contingent of people going "Well, he DID technically commit a victimless crime...".

So yes - he did do something that was against some terms of service. But jail time? For breaking a clickthrough "license" and computer hacking at MIT in the service of public knowledge? This sucks unutterably.

MIT: Playful physical hacks okay, but don't try to mass harvest the knowledge of the world or the DOJ will come down on you like a sledgehammer.

JSTOR: All the world's knowledge, as long as you don't try to access all of it.

DOJ: We'll break you just because we can (or for other reasons that we are not stating).

Hacker News: Well, they do have a point - he did access semi-public data in a non-approved way, and he had to plug into the network in a strange way to do it.

<comment><USER3> And doesn't the (alleged) fact remain that he continuously circumvented virtual and physical security systems in order to mirror a reported $1.5 million worth of assets in to his personal possession?

When you boil it down it doesn't exactly sound like he should be receiving 10 hours of commuter service because he ran wget a few times. MIT and DoJ are just supposed to -assume- that he has nothing but perfectly angelic intentions with $1.5 millions dollars of an asset that he secretly obtained? I don't make that logical leap with you guys. The facts will come out. He will present his case with his defense team, and hopefully justice will be done.

<comment><USER4> > MIT and DoJ are just supposed to -assume- that he has nothing but perfectly > angelic intentions with $1.5 millions dollars of an asset that he secretly > obtained?

Yes. That's known as "innocent until proven guilty".

<comment><USER3> Let me rephrase that: They are supposed to -ignore- an action they are aware of that they believe constitutes criminal conduct? Prosecutors, DoJ don't drop cases they think they can bring to trial when, in their perception, illegal activity is taking place just on a hunch. Right?

<comment><USER4> No, but they're meant to have a good faith belief that they can actually make the case.

[http://www.litigationandtrial.com/2011/07/articles/series/sp...](http://www.litigationandtrial.com/2011/07/articles/series/special- comment/aaron-swartz-computer-fraud-indictment/) examines in some detail why they probably don't.

</comment></comment></comment></comment></comment><comment><USER5> I don't understand why people are bringing issues of the availability of scientific journalism into this. I agree that the prices and paywalls involves are ridiculous and a bad idea for a variety of reasons, but that seems orthogonal to the issue here.

I don't think anyone is denying that Swartz committed a (possible series of) minor crimes. Claiming that "information should be free" doesn't stop them from being crimes. What's shocking is the response from the government, which appears to be using this incident for its own purposes rather than to preserve the rule of law. No one seems to really be asking _why_ this is happening, and I think that is by far the more interesting question. Is the justice department trying to expand its reach, as the article suggests? Is this a deterrent for future "hackers"? Has one of Swartz's numerous hornet nest- kickings pissed off someone high up, who wants him removed from the playing field?

<comment><USER6> If there's evidence yet to come that Swartz intended to anonymously push the corpus he took from JSTOR up to BitTorrent, the crime is no longer minor.

I think I share Swartz's politics, but I'd like to believe that law enforcement would at least take a plan like that seriously.

On the other hand, it's equally possible that there is no evidence Swartz was going to publish what he took from JSTOR, and that instead the DoJ is upset about the PACER incident, which it couldn't prosecute, and jumped on this case, which it clearly can prosecute.

Swartz is lucky to have such well connected and influential friends.

<comment><USER7> As a tangent, isn't it scary that you can effectively piss off entities such as the DoJ and they can hold a grudge against you?

Or from the article:

> Attorney Jerry Cohen, a Boston IP lawyer, suggests this aggressive use of > criminal charges rather than civil charges is part of a trend in government > prosecution of such cases, like taking "a sledgehammer to drive a thumb > tack… It's intended to terrorize the person who's indicted and others who > might be thinking of the same thing."

Is it even legally okay to publicly make an example of someone like that?

<comment><USER6> I guess. What do you expect? They're people just like everyone else. This isn't one of those "1,001 crimes you can commit in the morning while making orange juice" things from Reason magazine. Aaron went way out of his way to expose himself to this situation. It's possible that where you see an Internet folk hero, the DoJ sees someone bent on forcibly publishing every closed database pertaining to the public interest, laws be damned.†

Let me add something else that I think some other HN people may be familiar with:

My Confirmation sponsor is/was a criminal court judge (friend of the family). Very nice guy. Spectacularly nice guy. I babysat his kids once. I've seen him maybe twice in the past ten years, but I'd be surprised if he didn't remember the names of my kids. When I was younger, we saw him all the time. Every weekend. My dad played in the church folk group with him.

I can't remember the particulars, it may have been 4th of July firecrackers (illegal in Chicago) or it may have been shaving cream on Halloween, but either way he caught me and some friends doing something technically illegal. It was NOT. OK. I remember the legality of the incident being taken VERY. SERIOUSLY. Entirely different demeanor. I've seen the same thing with friends' cop dads.

I think people on HN don't fully appreciate the extent to which prosecutors and judges take the law seriously. They've dedicated their life to it. Our country is ruled by laws.

There are definitely times when the law is wrong, or when its diligent enforcement doesn't ultimately serve the social good. This may very well be one of them. But I think it's a very bad idea to build a worldview around the notion that the criminal justice system is going to casually look past the law. The law is a big deal.

† _I don't know Aaron Swartz personally and am not asserting any of this to be true._

†† _To your later edit: there's no statute against "public example making". If you don't want to be an example, avoid felonies. That's going to get me downvoted, so can we assume good faith and accept this new emoticon I just made up: :# > as shorthand for "I don't really think the person we're talking about is culpable for a felony based on the information we currently have"? :#>! :#>! :#>!_

<comment><USER8> _I think people on HN don't fully appreciate the extent to which prosecutors and judges take the law seriously._

On the other hand, a relative is a DA. She has explained how around the holidays, she collaborates with the prosecutor to ram cases through the system or slap people on the wrist and send them home. This way all the noble guardians of the law can take an extra few weeks off between Thanksgiving and Christmas.

<comment><USER6> I said they were people, and people who had dedicated their lives to the idea of the law. I did not say they were superhuman. They can be wrong and have faulty judgement. In fact, I feel like I went out of my way not to ascribe moral judgement on their worldview; I simply wanted to point out that their worldview exists and shouldn't be ignored if you plan on operating at the frontier of the law.

It's good of you to point out the limitations of that worldview. I don't mean to criticize you. I'm just saying, be careful if you think that a charge of "hypocrisy" is going to help here. The real world is not an Internet message board argument. Lots of prosecutors, all of them riven with human frailties, nonetheless believe passionately in the law. You're not going to talk them out of it.

I am for the most part happy about this. Unlike a lot of HN people, I think that the law by and large serves the common good, and protects the weak far more than the powerful.

(My uncle was/is? an ADA. I didn't hear any stories like this, but didn't ask. Totally believe it though.)

<comment><USER8> My point is not a charge of hypocrisy, I just think the language you are using imparts a phony gravitas. I'm a systems engineer but I would LOL if I read that someone wrote I "dedicated my life to the spirit of the command line." Likewise, many lawyers are just in it because it's something they are good at and it pays the bills. Not every lawyer is Harvey Silverglate.

<comment><USER6> I'm really not trying to impute gravitas. I am being completely serious. These people we're talking about at the DoJ are _different people_ when matters of law come up. They are not kidding around about it.

That doesn't make them granite monuments to justice. But it might mitigate the concern that they're being petty. There is a reasonable narrative here in which Swartz is purposefully causing harm to the social good. I don't really agree with that narrative (based on what we know now), but I can see it.

If it helps, think about pro-life people (I'm pro-choice) and their attitude towards abortion. They are not kidding around about that issue. Do they have a lot of gravitas to you? Maybe not! But they believe human beings are being killed. So I can see where they're coming from, even though I disagree with them.

<comment><USER8> Yes, in this particular case you are right. This case is interesting because Aaron is doing this as an "activist" and as such it becomes political-legal theatre. He is friends with a bunch of lawyers and legal researchers who are working at the intersection of IP rights and freedom of information rights.

If he was just some nobody kid in Schaumburg stealing PDFs from Motorola, he'd either be in jail already or slapped on the wrist and sent home. Nobody would have heard about it, and there'd be no internet dialectic about it.

<comment><USER6> We don't really know if this was an "activist" crime.

It's very possible that he just intended to analyze the documents and publish aggregated results. If that's the case, I think this prosecution is a mistake; an extreme overreaction. Scholarly analysis of JSTOR documents is not a criminal intent. His actions may leave him culpable to a variety of minor crimes, but his impact on the social good is minimal at worst.

On the other hand, it's possible (though less likely, I think) that the intent here was to mirror JSTOR onto BitTorrent. I might sympathize with this goal, but I don't think its prosecution as wire fraud is an overreaction. I'll root for Aaron at trial, though.

<comment><USER9> The problem with that analysis is that there's nothing criminal (or even illegal) about sharing public domain works over BitTorrent. So whether he copied the documents for personal use or distribution, it was still the initial copying that was at fault.

It's a little like someone broke into a library every night for a few months and scanned a bunch of public domain books.

The only actual crimes are breaking and entering, and using the scanning equipment without authorization. What he intended to do with the scans is irrelevant.

<comment><USER6> JSTOR's particular collection of documents is protected by copyright and has been valued, by virtually every one of the most learned and respected educational institutions in the country, at millions of dollars annually.

Let's say I come into possession of a trove of public domain documents from the 1800s, and I take the time to scan every one of them painstakingly and from those scans to assemble an academically useful database, for which I charge $100/person/year for access. Your contention is that because the underlying documents are public domain, you are entitled to unfettered access to my work product, despite the fact that I took steps to gate access to my work product.

The law isn't going to see it that way, I don't think. But maybe you're right. I'm not a lawyer.

Like everyone else on HN, I wish all these documents were freely available too.

<comment><USER10> > JSTOR's particular collection of documents is protected by copyright

We've discussed this on HN before. Reproductions of public domain works are not protected by copyright.

Edit: Of course a significant portion of JSTOR's archive does consist of copyrighted works.

<comment><USER6> I thought collection copyright applied, but am happy to be wrong about that.

<comment><USER10> There is a "collective works" copyright, so I guess -- I am also not a lawyer -- that if one made a copy of some database, it may be infringing on the creator's rights even if the contents of the database consisted solely of public domain works.

So I think you are correct.

<comment><USER9> [http://fairuse.stanford.edu/Copyright_and_Fair_Use_Overview/...](http://fairuse.stanford.edu/Copyright_and_Fair_Use_Overview/chapter8/8-a.html)

>You are free to copy and use individual images but copying and distributing the complete collection may infringe what is known as the "collective works" copyright. Collections of public domain material will be protected if the person who created it has used creativity in the choices and organization of the public domain material. This usually involves some unique selection process, for example, a poetry scholar compiling a book -- The Greatest Poems of e.e. cummings.

I don't think JSTOR is using what could be described as a unique selection process.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Empires in decline get progressively more brutal. Sounds like Aaron Swartz became an embarrassment so is being side-tracked from the good work he does.

I am not 100% sure of this, but I think this is probably true: similar to the case of Eliot Spitzer who as governor of NY was investigating Wall Street. So, I think that Wall Street had their lackey the US Government (via the FBI) dig up something on Spitzer to bring him down. I believe that this situation is called a plutarchy.

<comment><USER6> This is tinfoil-hat stuff. Governors who frequent prostitutes can't expect to stay in office, full stop. The likelihood that Swartz does a single overnight in custody over this is practically nil. The brutality we are arguing about here is the possibility of him ending up with a felony on his record.

<comment><USER12> To be honest, Swartz is independently wealthy so he won't have to worry about the felony impacting his ability to work for a living.

In his post-liquidity-event career as an activist, a felony conviction will serve nicely as a bona-fide.

So the only real downside for Swartz in this is that he spends time in jail, which as you say is quite unlikely.

<comment><USER6> One wouldn't want to downplay the badness of having a felony record. For instance, Swartz may imagine one day running for office, or being a key person in a political campaign. Or needing to work directly with institutional investors.

<comment><USER13> Makes getting any sort of security clearance a no-go. No buying weapons, etc.

<comment><USER14> Or, you know, voting.

</comment></comment></comment></comment></comment></comment><comment><USER15> To paraphrase Feynman, this might have some relevance to the situation:

"...he also worked with Shireen Barday at Stanford Law School to assess "problems with remunerated research" in law review articles (i.e., articles funded by corporations, sometimes to help them in ongoing legal battles), by downloading and analyzing over 400,000 law review articles to determine the source of their funding. The results were published in the Stanford Law Review."

<comment><USER6> I speculate that Pacer is more relevant: during a trial run of free public access to Pacer, Swartz is alleged to have mirrored almost 20% of the database; as (it is alleged) a result, the public trial of Pacer was shut down and an investigation launched into the security of the Pacer system.

One possible narrative inside the DoJ: we can't launch public trials of open access to databases, because this Aaron Swartz guy has decided that his Python code will have the final say in any policy decisions we make. But that's not for him to decide! And here he is again, taking the same approach, this time to a commercial database that produces 7 figure annual recurring revenues.

I don't agree with this mindset (:#>! :#>!) but I'd understand it.

<comment><USER16> Fwiw: JSTOR is non-profit, not commercial.

</comment></comment></comment><comment><USER17> > ... and Aaron faces a possible fine and up to _35_ years in prison, with > trial set for September.

<http://www.bbc.co.uk/news/world-europe-14270655>

> Under Norwegian law, Mr Breivik faces a maximum of _21_ years in jail if > convicted...

WTF.

<comment><USER18> Two different law systems, two different legal philosophies. In the Nordic model, prison is largely rehabilitative. I don't think they even _give_ life sentences. In America, prison is decidedly punitive, with little thought to rehabilitation, and the death penalty is commonplace.

<comment><USER19> "the death penalty is commonplace."

This is why you're being downvoted so heavily, in case you were wondering. Such flamebait is not appropriate.

<comment><USER18> >Such flamebait is not appropriate.

In this particular case, the death penalty is _relatively_ commonplace. As in, using OECD as a benchmark, the US executes more people than the average OECD member. I wanted to illustrate the contrast between the American and Norwegian legal systems, not make a statement about the use or validity of the death penalty.

[http://en.wikipedia.org/wiki/Death_penalty#Global_distributi...](http://en.wikipedia.org/wiki/Death_penalty#Global_distribution)

<comment><USER19> Then it would be appropriate to say "the United States executes prisoners while Norway doesn't." That makes the point fine.

Car crashes are commonplace here. Drunk driving accidents are. Executions occur less often than once a week in the US, mostly in a few select states, and the US has a population of over _300 million people_. We (I'm from the US) even have a disproportionately large prison population and the death penalty is still extremely, extremely rare.

<comment><USER20> That there is death penalty at all is already pretty barbaric and outdated.

</comment></comment></comment></comment></comment></comment><comment><USER21> shouldn't that be _United States v. Aaron Swartz_ because the US is bringing charges against him? _Aaron Swartz v. United States_ makes it sound like he is suing the government, but there's nothing on that page that says he is.

<comment><USER22> Yes it should, I read the title and thought there might be an interesting new twist when there was not.

</comment></comment><comment><USER23> Why aren't Google and Bing going to prison every time one of their bots crashes some web server?

(Perhaps because "systematic downloading" isn't actually a crime?)

<comment><USER3> When Google and Bing start sending 'human bots' out to Universities to facilitate the downloading of data, I'm pretty sure a lot of people will be going to jail. Just saying.

<comment><USER6> Didn't Google recently get in a bunch of trouble recently for capturing raw data from public wifi networks?

<comment><USER3> To me, there is no comparison. Inadvertantly capturing a few random packets from open, public Wifi signals while driving down the road? They changed wifi channels about 5 times a second. All investigations that I have read concluded that no meaningful data was retrieved.

And to compare that to specifically circumventing physical and virtual restrictions after warning the user against a set of actions? Where the target data was not worthless, as in Google's case, but worth over $1 million dollars?

I don't see the comparison.

<comment><USER24> It wasn't inadvertent. It was by design. Lets get that straight. If, for example, you're running tcpdump(1) with a packet capture size of 1500 (I don't know if G was doing that, but I'm giving an example), then you can't claim that you captured the packets' contents inadvertently.

Many years ago, when writing web crawlers was the cool thing to do, I wrote one (like an idiot, I wanted to see how deep the web was). Unfortunately, I didn't have a good synchronization scheme, so it ended up beating on a poor website for too long. The operator of that site sent a stern email to our DNS contacts, complaining. And we shut the crawler down. Today, I fear I'd be indicted for "wire fraud"!

</comment></comment></comment></comment></comment><comment><USER25> So he's suing the United States now, eh? Because usually the first person named is the complainant. Getting the simplest and most basic details wrong like this is a reliable indicator that everything which comes afterwards is going to be similarly ill-founded. This is a prime example of the yawning gulf between blogging and proper journalism.

Kindly do not misread that as support for the legal _status quo_ , JSTOR, or anything else.

</comment><comment><USER26> Regardless of who was hosting them before and how much their hosting costs were that were used to justify the paywall, Google Scholar should be brought in to host them all going forward. They're out in the open, might as well put them to good use.

<comment><USER9> The NSF should host them free of charge. We shouldn't expect a private money- making corporation to host them as a public service.

</comment></comment><comment><USER27> Well, Mr.Swartz should probably thank Anonymous and Lulsec for provoking the US Government. They're fed up and have determined whoever they get their hands on (minor hacking or not) they're going to drag you into federal court and make an example out of them.

This reminds of back in the 90's when there wasn't any laws in place to address hacking. But man, the Feds did not like kids making them look like fools. Once they got the laws of the books, it was open season on hackers.

I'm pretty sure the next few years are going to see a major crackdown on hacking again. Just like the recent arrests of supposed Lulsec and Anon members.

This guy will be lucky to get out of federal prison in 15 years.

</comment><comment><USER28> To me this looks like a career move by the prosecutor who wants to get some convicted hackers on his CV, which will look good when it comes to promotion time as these crimes get more and more relevant. The US justice system seems to have degenerated so that prosecutors do anything to get some people convicted, as long as they are a) popular or b) it helps them in some way, regardless of the quality of thee evidence or the merit of prosecution to the general public.

</comment><comment><USER29> I'm super curious to know if there is someone else out there who believes that there is a connection between Aaron's political activism and his indictment. Not that his political views would represent the major cause of his indictment, but still. Anyway, just ranting...

</comment><comment><USER30> We must evolve & develop economic & political systems that eliminate the government's authority to railroad a person like this. It's disturbing and a blatant flaw in a free nation.

</comment><comment><USER31> I'm just curious why someone who did this is still a fellow at a center for _ethics_. It seems like the whole situation, even if it's been cleared up with MIT and JSTOR, is completely unethical, regardless of the legal case.

Is the position tenured?

<comment><USER32> Could you comment more expansively on why you find this situation unethical? If so, can refrain from leaning on the legality of the situation. I haven't heard a good explanation that didn't use the legal ramifications as a central support.

</comment></comment><comment><USER33> Nice spin: he's been charged for "excessive downloading".

Way to ignore the physical break-in to install a computer directly on a network switch. I, for one, hope that he gets a criminal record at the very least (plea deal). This really puts the crime in a different league.

Unless I have my facts wrong, in which I'm willing to be corrected.

<comment><USER34> Well, a physical break-in would be a matter for the local police, maybe state. Federal prosecutors don't get involved with those cases unless it's the Pentagon or something.

This is only at the federal level because the "unauthorized access" crossed state lines. (Aaron was at MIT when he was knocking over JSTOR servers, probably in NYC.) So the prosecutor is only going to worry about that part.

<comment><USER6> I don't get why I'm expected to care about whether local or federal employees take an interest in the event. The state criminal court system isn't warm and fuzzy either.

<comment><USER34> He said "way to ignore the physical break-in" and I'm pointing out that federal prosecutors (almost) always ignore physical break-ins. It's not a federal crime.

<comment><USER6> I take your point, but I'd like to point out that it's a side track. It gives the impression that you don't want to talk about whether a crime happened, and instead focus on the purported injustice of federal involvement. Well, what's unjust about that? That the DoJ and federal court system is more severe than Massachusetts?

<comment><USER35> It's not a side track to point out that he should be charged with one set of crimes by the state government, but that he is instead being charged with a different set of crimes (Wire Fraud, Computer Fraud, Unlawfully Obtaining Information from a Protected Computer, Recklessly Damaging a Protected Computer) by the federal government.

<comment><USER6> You think MA doesn't have computer crime statutes? The subtext to this thread seems to be "if they charged him locally they'd only be able to get him on trespassing". Uh, no.

<comment><USER36> There is a difference with charging him locally though (in my mind). The attorneys that would prosecute Aaron in MA were hired by a left government accountable to a fairly liberal population. Those prosecuting Schwartz federally were hired by a government that leans slightly to the right. I honestly think that makes a difference in the outcome of this particular case, given the nature of the crimes. Also when we consider Aaron's personal history, I think it's safe to say he has fairly influential friends in MA and possibly some enemies on the federal level.

All of this is pure conjecture and speculation of course. But I would be lying if I said I wouldn't be more comfortable about the situation if Aaron was being tried by the MA government.

<comment><USER6> I see what you're coming from but I don't really buy it. I think the difference between federal and state is strictness and deliberation, not right vs. left.

The state criminal justice systems are deluged by drug violence and domestic violence and are largely reactive (I write while missing the court date for my mugger's prosecution, due to a sick daughter).

The federal system is more deliberate (word chosen carefully) and somewhat less reactive. It has more discretion about what cases to chase down. It's probably more inclined to keep its teeth sunk into anything it starts too.

The same political environment that appointed this prosecutor also appointed the prosecutor that took down Scooter Libby.

<comment><USER36> Re: left vs. right and deliberation: that seems reasonable. Also sorry about the mugging. :( Still though, that doesn't address the advantage Aaron has in MA in terms of friends and enemies. And as far as public support is concerned, you can be sure that a good number of the people that support Aaron have ties to MA or Cambridge. When we talking about a couple thousand people, that goes a lot farther on the local level than on the federal level I think. A speech by a group of MIT and Harvard professors for example might make a difference in MA but surly doesn't have that big of an impact nationally. Perhaps I am seeing differences where there are none though. ^^

<comment><USER6> The mugging was almost fun. Makes for a great story.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER37> Correction: Aaron Swartz v Some Powerful Politicians or People Who Are Afraid Of What Playing With This Data May Reveal.</comment>


Shrink Your Team to Increase Effectiveness - <AUTHOR> http://robertgreiner.com/2012/11/shrink-your-team-to-increase-effectiveness/

<comment><USER2> But never compromise on productivity</comment>


Ask HN: Good Social Sentiment Analysis Tools? - <AUTHOR>

I've been looking for a social sentiment analysis tool (One that scrapes data from social sites: Twitter, Facebook, Pinterest, etc and lets you run analysis on said data) I.e. Heartbeat, Social Mention, Viral Heat.
This is one of those go mad by the amount of choices research topics...The main thing making it difficult though is that it's hard to find any real reviews or information outside of the different platforms' websites.
So does anyone have any experiences with these platforms and could give a recommendation?
I'm aiming for a nice balance between price and power, so nothing absurdly expensive and if it's cheap it has to do its job well.
Thanks! <comment><USER2> Have you looked at Sentiment140? <http://www.sentiment140.com/> It's a nice tool for sentiment analysis on Twitter with a free REST API.

<comment><AUTHOR> I have, it looked like a good tool for Twitter, but I'm looking for a tool that covers more platforms.

I should also state: our main platform is Facebook, so the tool needs to include support for fb.</comment></comment>


Facebook's killer app is here. No, it's Not places. - <AUTHOR> http://blog.eyesandfeet.com/2010/08/facebooks-killer-app-is-here-no-its-not.html <comment><USER2> Personally, I think Facebook's killer app is that it fixed email.

For a huge population, email never worked very well, mostly since they had to find out someone else's email to talk to them. Facebook fixes this completely for any personal email use - all your real-life friends are friends on Facebook.

I'm guessing a huge percentage of personal mail has moved from email to Facebook messages - does anyone have any data about this by any chance?

<comment><USER3> Hmm, facebook mail is absolutely horrid. It's a combination of spam messages and just contact that I will never read. I have 1000+ unread messages.

I actually read a fb message in my normal email box and if it's something actually valuable, ill then respond on FB.

<comment><USER4> Facebook didn't fix email with its Facebook 'mail', it did it with wall posts, comment conversations, and events.

<comment><USER5> This.

My experience with non-geeky friends is that they check facebook for messages and interact with friend/family through comments and wall-posts. Some of them get email notifications for these things but they use them as a prompt to go to facebook.com to view and respond. Email is the medium but not the message.

</comment></comment></comment></comment><comment><USER6> This is just the latest revision of AOL Keywords. How did that work out for them?

<comment><USER7> 20 years after they were a startup, they made $400M in profit on revenue of $3.2B last year. I'm not saying those are sexy numnbers, but they're not exactly shabby, either.

<comment><USER6> They don't control the web though. So it's significant, but not as dire as the article implies it will be.

</comment></comment></comment><comment><USER8> I personally don't see the appeal of Facebook pages. The only function I can see of "liking" any of these pages is to give Facebook more information so they can try to sell you more stuff.

<comment><USER9> The appeal to businesses is clear.

The appeal to users could be social ("What brands do I associate myself with?") or promotion-related ("Become our 1,000,000th fan and get this free thing!", "Facebook-exclusive discount!"). Also I don't think Facebook lets you put in bio information (favorite movies, etc.) anymore without it linking to a page.

<comment><USER10> Facebook's increasing focus on business customers will be at the detriment of regular users, who will be demoted to the rank of mere 'consumers'. It's a subtle - but important - distinction.

Facebook's decision-making will increasingly be driven by thinking like "how can we monetize all this wealth of consumer data" or "how can we introduce tiered fanpage packages to business customers as a revenue stream" rather than focusing on what makes a great user experience.

Currently, knowing that my friend has achieved another 'badge' in Mafia Wars adds zero value to my life. It only gets worse as Facebook focuses more and more on business brands. What do I care if my friend 'likes' Apple or Nike? How does that improve my relationship with that person? People I am _really_ friends with in real life don't care what brands I like, or what isotonic sports drink I drink. The like me because of _me_. Much as businesses would like to think that people define their lives by the products they buy (this is like the opening scenes in 'Fight Club' where Edward Norton's character tries to pick out stuff from an Ikea catalogue that defines himself) that's not a basis for a relationship. And Facebook used to be all about relationships. Now I look at my Live Stream...and it's got all this random flotsam floating downstream. I care about none of it.

The reason I love HN, incidentally, is because it's the polar opposite of MySpace and what Facebook is gradually becoming. Real people, that I share a lot in common with, expressing their real opinions, no auto-generated crap, and zero bling.

You see comments like "What could cause FB to die?" here on HN. They're so big at this stage, with the power of network effects and lock-in, that external competition is no threat to them realistically. The only way they will die is if they continue exactly the way they are now, making people's experience ever-more spam choked, till people realize "hey this experience is actually quite shit, even if I do have 500 online friends" and start looking for alternatives. We're not quite there yet though.

<comment><AUTHOR> I didn't fully get your point earlier, but it's clearer now. You raise valid points. By the way, two points that may interest you: (1) I personally filter my facebook stream to weed out all the "spammy auto-updates" you've mentioned - so it feels a lot more 'human' to me. I get exactly what you mean (mafia wars is a pet peeve) (2) The whole thing about the Dunbar number-<http://en.wikipedia.org/wiki/Dunbar%27s_number>

<comment><USER10> _(1) I personally filter my facebook stream to weed out all the "spammy auto- updates..."_

Forcing users to 'prune' their status updates is in fact a symptom of something broken in the user experience. I check into Facebook every now and again, and I shouldn't to have to act like a gardener who has ignored his back yard for a month. A lot of status updates are incredibly frivolous too - Facebook's culture is not suited to my taste I guess. Virtually all of my (Facebook) friends are not hacker types and will post "I'm bored" or "feeling tired" or something equally awe-inspiring. Granted, this is something that Facebook would have never solved as they are trying to appeal to the mainstream, not early adopters. I love the fact that by logging onto HN I often learn something new, and quickly learn to a) write well and b) only talk about stuff you are knowledgeable about. Anything else and you will get caught out pretty quickly.

 _(2) The whole thing about the Dunbar number_

This had occurred to me also when writing the original comment, but I didn't mention it as I don't see a decline as inevitable as an online community grows in size - I believe HN is proof of that. By deliberately appealing only to a narrow segment of the internet population through a strict, human-enforced culture (see HN guidelines), and allowing users to only maintain very loose ties if any with each other (contact info in your HN profile for example), it's possible to have meaningful, no-spam interaction on an online community. Pg mentioned recently that HN now gets something like 60k uniques per day. I've been on HN for quite a while and I think the quality of article and comments has been consistent over time. I think the Paul Buchheit philosophy of building something that a small number of people _love_ , and that most people would hate (HN), rather than something that tries to appeal to everyone (Facebook) comes into play here. See my below comment on the 'tribes' concept for a possible solution to this problem in a social networking context. I believe it would work as it models how people organise themselves into groups in real life.

</comment></comment></comment></comment></comment><comment> loup-vaillant Scary. That's centralization to a degree I hardly imagine. I hope this will fail.

<comment><USER9> I don't think this is really centralization. Just another avenue for advertising. For example, even if (less mainstream) bands generally use a Myspace URL as their web presence, Myspace doesn't necessarily control the music industry.

I find the Facebook-controlled universal login much more unsettling, personally.

</comment></comment><comment><USER11> MySpace band pages were the rage a few years ago. Times change. The MySpace pages at least provided value for both sides--you could hear the music and the bands could let you know about shows. Most business pages I've seen on Facebook are one sided in that the fan gets nothing out of it other than letting all their friends know what corporation they are a "fan" of.

<comment><USER12> It really depends on the business. There are a couple of local businesses here that I follow: a liquor store, a crepe cart, a coffee shop, etc. They all regularly post with specials, events, new dishes/type of coffee/beer, etc. YMMV, but knowing that I can pick up grilled fig, honey and goat cheese crepes on the way to work tomorrow is pretty cool in my book.

Though I imagine most pages are not quite so well utilized. These are all small, local businesses.

</comment></comment><comment><USER13> I've been noticing this trend as well. I commented to a friend recently, "Remember when there was more than one web page?"

<comment><USER14> Yea me too. As someone who doesn't have a Facebook account I notice it a lot.

I have actually started removing sites from my favourites list that are peppered with Facebook widgets. I just don't visit them anymore. There is still plenty of choice out there but stuff is slowly morphing into Facebook.

<comment><USER15> I've seen plenty of commercials with: facebook.com/myproduct as their URL. Crazy shit.

</comment></comment></comment><comment><USER16> When combined with Places, Pages becomes a significantly more compelling platform. For business owners it becomes a more vibrant and useful set of data. For consumers, it becomes more real time and as a result more social. For developers, it becomes a more interesting data set to build upon. This excerpt from GigaOM ([http://gigaom.com/2010/08/20/facebook-places-the-real- target...](http://gigaom.com/2010/08/20/facebook-places-the-real-target-is- yelp/)) was telling:

 _We're hearing that Facebook is pouring resources into pitching the Places feature as a tool for local businesses in dozens of markets, by approaching individual store-owners and business people and selling them on the idea of setting up a Places page for their location. According to Facebook's description of the new feature, anyone can create a page for a place, but businesses can claim their page — by responding to a phone call from a Facebook representative, or by uploading some kind of official documentation that proves they have the right to that location._

</comment><comment><USER17> All true only if organizational default url becomes their fb fan page and that is not likely to happen

</comment><comment><USER18> Facebook, the most visited site in the US, has only just now found it's killer app?

<comment><USER19> Did you read the article? The whole point is that they had already found it (or so claims the author). It's the author that just now realised.

</comment></comment><comment><USER20> Old news. Though if fan pages really took off with places like suggested then time is really running out for Google Me. Unless there's some super cool feature that really blows Facebook out of the water, but it is hard to imagine how anything could be.

</comment><comment><USER21> FACT CHECK: 500 million users is over 25% of the world's online population.

Actually, the last time I checked <http://www.census.gov/main/www/popclock.html> shows the world population at 6.8billion. So 500 million would only be 7% of the worlds population. Impressive, but nowhere near 25%.

<comment><USER22> Not everybody in the world is online, and thus not part of the worlds _online_ population.</comment></comment>


Blokable uses Mender for OTA updates for scalable housing creation – webinar - <AUTHOR> https://mender.io/blog/sign-up-for-a-webinar-with-blokable <comment><USER2> Seems to be modeled after this work between Google and Mender [https://cloud.google.com/blog/products/iot-devices/mender- an...](https://cloud.google.com/blog/products/iot-devices/mender-and-cloud- iot-facilitate-robust-device-update-management)

</comment><comment><USER3> If I am going to have a wired house I would definitely want the software to be up to date.

<comment> Lex-2008 Yep! When someone hacks into your computer - it's one thing; but if somebody "hacks" into your home - it's a completely different issue!</comment></comment>


Managers: Don't translate your stress - <AUTHOR> https://medium.com/p/managers-dont-translate-the-stress-842b6afeb6f4 <comment><USER2> As a manager this has been my biggest challenge.

I never telegraph my stress to my team members but used to often do it around my peers (managers on other teams).

Probably because I thought they could empathize so they'd be a sympathetic ear. A few complaints to the CEO about me later and now I've learned to do what the author describe. Even if the person is not someone who reports to you; just go get a coffee, take a walk, etc.

I am lucky in one regard though, Our CEO flat out told me if I need to stress out around someone do it around him. Which seems healthy and more productive then bottling it up.

</comment><comment><USER3> Another important thing is to accept that anger is normal human reaction/emotion and not to feel guilty over being angry. Some responses to anger are bad (when your emotions blow up and you start to insult people just to get back to them or vent), but anger itself is not something shameful or bad.

</comment><comment><USER4> OMG. Hunger. #1 cause of me being ill-tempered. Never be hungry at the office.

(Get fat instead?)

<comment><USER5> Ditto. When I started eating better (i.e. less) hunger immediately started affecting my mood.

Now I handle this with \- Cashews in my desk \- Frequent water and coffee breaks \- Minimum 15 minute walk outside during lunch

It's worked out pretty well for me so far.

</comment></comment><comment><USER6> This is such a great point. Anyone have tips on tactics for catching yourself externalizing your stress?

<comment><USER7> Maybe ask others for your signals.

For me, I tend to talk faster, raise my voice a bit, frown more, and generally behave more combatively.

So basic things like slowing down, relaxing my expressions and smiling more (even and especially when you're on the phone!!!) go a long way to not only changing how I present myself, but also changing how I _feel_.

</comment></comment><comment><USER8> "one of the greatest individual contributors of all time".... oh really! cool.</comment>


401(k)s are a sham - <AUTHOR> http://www.salon.com/2013/08/06/big_finance_lied_401ks_will_not_save_aging_americans_partner/

<comment><USER2> The problems with retirement savings going forward are many and varied.

First, it is increasingly difficult to achieve good average returns in the financial markets as a small-time retail investor. There are myriad reasons for why this is but the fact remains and is unlikely to change soon. That so much capital is chasing increasingly meager investments is one of the reasons we have cycles of overheated markets and crashes.

Second, Americans are _terrible_ at saving money. An unbelievable amount of income for the average American household is wasted on frivolous consumption and collecting status symbols. When I was genuinely poor I always managed to save some money (indeed, it is what put me in the habit when I was not poor) but I know countless people who make six-figure incomes who have no savings and lack even the most rudimentary discipline to do so. In most cases, socking away 20% of their take home pay would not even require a material lifestyle change, just making more prudent financial decisions. This falls squarely on individual Americans; few have legitimate reasons why they cannot save at least some money.

Lastly, the government really can't solve this problem for us by doing the saving for us. As poor as the return in the markets is for many people, the returns on capital handed to the government are consistently even poorer.

For most people, the second point is the most important. Live below your means and save the balance. 401(k) is really beside the point. I don't even put my savings into retirement accounts, in part because the investment options suck in those accounts. Simply not pissing money away on frivolous and imprudent expenses is sufficient to start building significant savings for most people.

</comment><comment><USER3> This article is so slanted I am amazed that its not hawking some kind of retirement product. You'll note that the article cites research by the "National Pension Rights Council", an organization basically only quoted by this article, and an organization likely only interested in preserving its outdated way of dealing with retirement funding (IE: 100% employer funded).

While figuring out what to do with your IRA or 401k may seem daunting, its not, and its certainly better than a pension...

</comment><comment><USER4> The only actual point this article makes is that its hard to save up enough money to live off of for 20-30 years. You can't trash 401(k)s without providing an alternative that is better. It's hard to save for retirement. That's a fact of life. 401(k)s are one of the easiest ways to do it though.

<comment><USER5> The best thing about 401(k) is the employer-matching. Am I missing something? It is just an investment vehicle, right?

<comment><USER4> Plus you don't pay federal income tax on the money when putting it in and then don't pay capital gains taxes on your gains year after year. They neglect mentioning that not paying capital gains on your 7% gains saves you more or less 1.4% of your money with is more than the 1% fee they complain about.</comment></comment></comment>


Why is DropBox Thriving?  - <AUTHOR> http://www.markevanstech.com/2009/09/28/why-do-some-online-services-thrive/ <comment><USER2> It's thriving because it _just works_. Works when you're offline. Works if you try to save conflicting copies of a file. Works if you accidentally delete a file. Works if you need to retrieve an old version of a file. Works if your internet connection is spotty.

Rather than punting on the hard issues and creating another WebDAV-like service, they actually created something that _just works_.

<comment><USER3> It more than just works. It works awesomely across multiple platforms.

It's simple (automatic) to use but not simplistic. Its power ramps up when you need it (file recovery) and fades away when you don't.

It doesn't get in the way (with advertising, installing crapware, charging micropayments, proprietary restrictions/formats, etc.)

<comment><USER4> It might work "awesomely" one some platforms, but the Nautilus integration is pretty buggy - it was a nightmare to try to get it working on my Ubunutu 09.04. I ended up having to just use the command line utility a user wrote.

<comment><USER5> Odd. I had no trouble on Ubuntu 9.04.

</comment></comment></comment></comment><comment><USER6> It must be noted that Dropbox solves a problem that Microsoft has been trying to solve for decades. Ever since the MS DOS LAN Manager, MSFT has been working on a file synchronization and sharing solution (and they are only beginning to tackle the synchronization part).

SMB, Microsoft Groove, SharePoint, SkyDrive (aka LiveDrive), LiveMesh, Windows LiveSync (aka Folder Share), SyncToy, and even Briefcase, are different attempts at solving the same problem.

Still, my Mac and my Windows 7 PC sitting on the same desk won't talk to each other and Dropbox is the only thing that works.

</comment><comment> brown9-2 Article kind of seems like linkbait - the author seems to ask the question "What makes an online service successful?" but of course doesn't provide any specific answers or really anything specific about Dropbox at all.

Surprise surprise, the same things that make an offline service successful are what make an online service successful too.

<comment><USER7> The title given on HN is different from the original which is "Why Do Some Online Services Thrive?".

</comment></comment><comment><USER8> Here's why the founder (and pg) thought it could thrive:

<http://news.ycombinator.com/item?id=801503>

</comment><comment><USER9> Well, one thing is that it stands out in my mind as best of class. So if someone asked me about file syncing, they come to mind. Whenever people want to share files, I tell them about dropbox.

<comment><USER10> Agreed about the "it just comes to mind" aspect of the site.

I think it really helps that they picked such a great name for the service. It's easy to remember and tell others, and the name describes exactly what it does. I'm sure this has been a factor in the speed at which they're signing on new users.

Dropbox is one of the best examples of great product naming that I can think of.

</comment></comment><comment><USER11> Just want to note they give users the incentive of extra space when a friend joins and installs the application. They also have a nice tracking tool to 'spy' on which people showed interest and how far in the registration-process they got.

</comment><comment><USER12> How many of the 2 million users are paying customers? Is dropbox losing money for now because of the 2GB it gives out for free? I believe dropbox will make money but right now it seems that it is still burning money.

<comment><USER13> 2gb of storage costs $.24 a month on S3, and you figure they only average about half full. Call it an even $.50 with data transfer costs, and you've got just one paying customer (at $9.99/month) subsidizing ~15 free users. A 1-in-15 conversion rate seems like a modest goal. They may not be losing any money at all.

<comment><USER14> I use dropbox and am happy with it. I save files to it every day. I share files using it about once a fortnight and I grab files via the web app about once in 2-3 months.

I'm at 23.1% of 2.5 GB.

I don't think I will upgrade soon unless I start to use two machines regularly.

I would guess less per average free user (maybe as low as $2 per year). I would guess that 1/15 conversion is also a high estimate. But I might convert in a few years time. I imagine that it'll take another year or two before they really know what conversion rates look like.

</comment></comment></comment><comment><USER15> <http://www.youtube.com/watch?v=-oRy-vM1OQs>

Couldn't have said it better myself, including the "actually no maybe not" part when going into my pictures folder ;)

Even with 25GB versus 2GB, Dropbox is more popular.

</comment><comment><USER16> Dropbox also has a bit of secret sauce. They have some technology that can pick up which bits changed in a file and only sync those (which makes their service way faster than others)

<comment><USER13> librsync (<http://sourceforge.net/projects/librsync/>) does this. It's LGPL'd so I wouldn't be surprised if they were using it. Certainly a lot simpler than reinventing the wheel.

</comment></comment><comment><USER17> I think it is thriving because is simple and transparent. Users dont have to learn how to use a new application if they don't need the advanced features. They just drop files into folders like they always have.

It was perfect solution for integrating version control for my diverse team of designers(macs), developers(linux) and everyone else(windows).

</comment><comment><USER18> They made something you want to tell your friends about. I noticed people started using it (independently from me) at my engineering college last year and by now shared Dropbox folders are widely used on group projects.

</comment><comment><USER19> Given their quick growth I'm curious to know what kind of marketing they did initially or what their main strategy was for getting the word out.

<comment><USER20> Love 'em or hate 'em, I initially heard about Dropbox on Techcrunch.com. I immediately signed up for the free 2Gb account (as I usually sign up for almost anything free).

I think it was a couple of weeks later before I ever got around to installing it on a second PC and actually using the sync functionality across boxes, and probably a day or so after that their usability and overall perfection convinced me to buy.</comment></comment>


Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution - <AUTHOR> http://vllab1.ucmerced.edu/~wlai24/LapSRN/ <comment><USER2> That animated gif is ridiculously misleading. To someone who doesn't know what's going on, it would look like this can hallucinate Emma Watson's face out of a 5x5 px image.

Instead, this is essentially a fancy 2x upsampling, and the gif shows every other frame as a super-resolution result. In fact, if you look at details like her eye, it's not even doing that great of a result (unsure if this is supposed to be impressive given the current state-of-the-art).

<comment><USER3> It is impressive given current state-of-art. Your summary doesn't seem fair. Take a look at the first set (Set5) [1], and set upsampling scale to 4x and flip between Bicubic and LapSRN. What seems most exciting to me, is that LapSRN is more resource efficient than bicubic.

[1] [http://vllab1.ucmerced.edu/~wlai24/LapSRN/results/Set5/](http://vllab1.ucmerced.edu/~wlai24/LapSRN/results/Set5/)

<comment><USER4> Whether or not their sampling is impressive given the current state of the art (though in that same set the other, non-bicubic methods seem to do pretty well compared to LapSRN), that gif definitely seems misleading. It's not obvious _at all_ what is happening here, and it doesn't really even show off their method very well (as, say, separate gifs for each of the downsampling rates and/or clearly labeled frames). The way it is now, it really seems like each progressive enhancement is another stage in the image reconstruction.

<comment><USER3> The banner gif is confusing, reminds me of the Super Troopers Enhance scene. Not fitting for an academic publication.

</comment></comment></comment></comment><comment><USER5> When I was still in computer vision, "super-resolution" meant taking multiple images and combining them to really be able to see what was in the scene.

Now it often means: take a single image and have the computer guess/make up image content. Not the same at all, IMO. There are worse examples than this, though.

<comment><USER6> I don't know when you were in computer vision, but when I was in computer vision a bit more than a decade ago it meant both.

<comment><USER7> I would also call it a perverted use of the term super-resolution. NNs perform image manipulations beyond resolution enhancement. They add features based on trained sources.

<comment><USER6> And? Do a search on scholar for "Example-based Super Resolution" or "Markov Networks for Superresolution". The term has been used this way in literature for maybe 20 years now.

</comment></comment></comment></comment><comment><USER8> The results look more blurry than necessary, and in some instances they look too sharp.

I think deep learning is a more promising approach, as for superresolution you really need to "invent" missing pieces of the image, and this can be highly context dependent.

</comment><comment><AUTHOR> Reminds me of Bourne-style "enhance" feature. Too bad the GIF on the main page appears to be marketing (seems impossible from a single frame like that) and takes away from the achievements of the paper.

<comment><USER9> Yes, that image is deliberately misleading to the lay person... it is showing how it up-samples from various inputs, each having greater resolution. It is impressive, but it seems to imply that it is doing something magical to produce a full resolution picture from a few mere pixels. What is actually doing is impressive enough, but it seems like an odd choice of promoting the work given the obvious visual implication which can't be an accident.

<comment><USER10> Something like every odd frame is input, every even frame is upsampled output?

</comment></comment></comment><comment><USER11> That gif is not even close to how it actually works, that's such a dirty thing to do, that's no accident...

</comment><comment><USER12> Am I missing something, or where are the originals?

(I'm assuming "super-resolution" in this context is like a function B = super_resolution(A). I can see lots of what I think are Bs, but where are the As? Aren't they super-relevant _?)

(_) No pun intended.

<comment><USER3> The example sets are clickable. That's where the As are.

<comment><USER12> As far as I can see there's only other kinds of Bs there (B = cubic(A), B = some_other_super_resolution_menthod(A), etc, etc). Can't find the real As... :P

<comment><USER13> The left-most item in the examples is labeled "ground truth". That is your "A" unless we misunderstand you.

<comment><USER4> I think that the parent means that the "Ground Truth" is the high resolution original, and they never show a "low resolution" original.

Presumably the process goes:

High res. (A') --(Downsample)--> Low res. (A) -- (Upscale) --> Reconstructed (B)

The sets show A' and B, but not A (unless I'm way off on what's happening in this process).

<comment><USER12> Yes, that's what I meant!

If the Ground Truth where the A then I'd implement my very own "optimal super- resolution" like this:

def optimal_super_resolution(A): return A

It would be a huge breakthrough. :P

</comment></comment></comment></comment></comment></comment><comment><USER14> There are signs in the data that the downsampling technique that is being used is not gamma-correct. That would somewhat undermine the results (and also the NNs, if they were trained on similarly broken inputs). Can one of the authors clarify that gamma-correct downsampling/blurring/convolution was used?

</comment><comment><USER15> With this idea of NNS to predict how to upscale images, I'm kind of wondering if LenPEG isn't becoming more relevant than ever:

[http://www.dangermouse.net/esoteric/lenpeg.html](http://www.dangermouse.net/esoteric/lenpeg.html)

</comment><comment><USER16> Please be sure to leave time to start loading the various filtered versions, if you hover too fast the alternate image won't load, silently, and you'll be left with the last successfully loaded one.

<comment><USER14> Almost every site like this gets this wrong - if the switching only happens once the image has finished loading, you need to otherwise hide the now-wrong image. Having to second-guess whether an image has loaded is asinine.

<comment><USER16> some have a slightly different UX making it more obvious, IIRC fabrice bellard site for BPG format uses mouse down events to show the filtered version, it's way random than hovering on a link so you intuitively wait more on the first held down.

</comment></comment></comment><comment><USER17> Could this be applied to cosmological images from large telescopes.

<comment> semi-extrinsic In the trivial sense, yes of course it could be applied. The real question is "would super-resolution be useful for images from large telescopes?", and I believe the answer is "mostly no".

I guess (one of) the exception(s) would be if you have many smaller telescopes that could scan the sky much faster than the few big ones, the small ones could use super-resolution techniques to look for objects that the big ones might find interesting. But I think "astronomer time" rather than "telescope time" is usually the limiting factor.

Mind you, telescopes already do a host of physical tricks to improve resolution, like sensor cooling, stacking images, adaptive optics with laser guide stars, advanced noise filtering etc. Whether ML super-resolution stuff would actually be useful on top of all those tricks is a question for the astronomers in the crowd.

</comment></comment><comment><USER18> Before the inevitable "enhance" comments come in, please note that these nets are making up the information to insert based on information from the training set.

</comment><comment><USER19> I wonder what this can do for astronomy.

<comment><USER20> How do you mean? AFAIK since those networks "halucinate" the additional data based on similar but actually unrelated data, its not of much use in science where accurate data is important.</comment></comment>


Show HN – Tastebuds: Tinder for Concerts - <AUTHOR> http://tastebuds.fm/a/music

<comment><USER2> earbuds would have been a better name for this imo.</comment>


Show HN: FuriganaTextView – UITextView that can display Furiganas - <AUTHOR> https://github.com/lingochamp/FuriganaTextView

<comment><USER2> Related: Firefox 38 (recently released) is the first widespread browser to support full Ruby annotations in HTML+CSS: [https://hacks.mozilla.org/2015/03/ruby-support-in-firefox- de...](https://hacks.mozilla.org/2015/03/ruby-support-in-firefox-developer- edition-38/)

<comment><AUTHOR> Wow, that's interesting. Thanks for the feedback.</comment></comment>