"The DNS Camel", or, the rise in DNS complexity - <AUTHOR> https://blog.powerdns.com/2018/03/22/the-dns-camel-or-the-rise-in-dns-complexit/ <comment><USER2> What happened to "rough consensus and running code" thing? Is running code not required these days?</comment>


Facebook users drop in the US and Canada - <AUTHOR> http://www.insidefacebook.com/2011/06/12/facebook-sees-big-traffic-drops-in-us-and-canada-as-it-nears-700-million-users-worldwide/ <comment><USER2> Suppose you're in a room with 5 close friends having a conversation. You're comfortable, animated, energetic, outgoing. In the midst of this engrossing conversation 400 other acquaintances silently enter the room. Suddenly you notice them and realize they can all hear every word you're saying.

Most people find this scenario a bit socially uncomfortable and clam right up.

Facebook's growth has created this scenario for a lot of people. The basic currency of Facebook (voyeurism) relies upon a loose security model that prevents a person from walling off his/her 5-10 closest friends and using FB for the bulk of their online socializing. Eavesdropping is essential to the business model. The problem is, thusfar most FB users are not really aware just how intricately engineered this voyeurism is and how publicly their lives are being broadcast.

<comment><USER3> You can wall off groups of people within Facebook, but the effort required to do this is beyond the motivation level that I'd expect most Facebook users (myself included) to have. Like privacy, making Facebook more tolerable to use is an opt-in thing, requiring proactive measures by the user.

<comment><USER4> If walling off your group is equivalently difficult to creating a yahoo/google group, then yeah.

Facebook's entire premise is that people LIKE being heard/seen and LIKE seeing/hearing others, in their social graph.

What is really needed is multiple distinct social graphs per user with distinct walls between them, done elegantly... this isn't an easy problem to solve.

There was a really nice presentation from a Googler (who subsequently moved to Facebook) which described this problem eloquently... but I can't seem to find the link.

<comment><USER2> Interesting. I'd love to see the presentation if you (or anyone else) end(s) up finding the link.

It's interesting to consider the threshold for the sort of social discomfort I described above. In my case it hit around 2/3 of my current friend count.

<comment><USER5> Here's the previous HN discussion for the (likely) article in question: <http://news.ycombinator.com/item?id=1867807>

</comment></comment></comment></comment></comment><comment><USER6> I don't think any single service will kill Facebook. Hell, I don't think you can do anything but diminish them at this point.

My bet is that the micro-niche makes a comeback and is what catches on going forward. We dont' care about high school friends we never talked to, or family members that blab about their dinner. What we care about are the things that interest us...HN is a great example.

As more people move to avoid the noise, they'll find solace in mini- communities, and I think that's the way forward. Not just one service, but hundreds.

<comment><USER7> Unless of course Facebook figures out a useful and effective way to provide the same focused view of a niche community within the larger Facebook expanse. At some point they may come up with a way to easily and intuitively allow users to expose different facets of their interests and persona to different groups; if they figure this out the niche sites (even HN) are toast.

<comment><USER8> _Unless of course Facebook figures out a useful and effective way to provide the same focused view of a niche community within the larger Facebook expanse._

Facebook private groups

<https://www.facebook.com/groups>

are working quite well among my circle of friends to provide a focused view of one or another niche community that forms a natural subset of my friends. There is a particular Facebook secret group that I visit absolutely every time I'm on Facebook, which is several times a day. That's a practical place for me to ask focused informational questions, and also a place where I can share a lot of camaraderie with dear friends who have had similar--and yet very diverse--life experiences. Because of the stronger filtering abilities of Facebook, I can indeed see how it could supplant many niche sites. (Thus far, HN gets the second share of my attention after Facebook, among all websites, and it will be interesting to see what the consensus here is about the overall community experience in the next year or so.)

<comment><USER9> One problem with Facebook Groups I find is that you can't have wall posts from your groups appearing in your primary timeline/newsfeed. Because of this, you have to make a concious effort to visit the group to catch up.

Since none of the groups I joined have the same level of activity I'm used to on specialized forums for those subjects, I never bother to check up on them - which of course lends itself to a vicious circle of inactivity.

<comment><USER10> Posts from groups that I'm a part of do show up in my main feed. Maybe you need to tweak some settings here or there?

<comment><USER9> Do posts made by others to the group wall show up?

</comment></comment></comment></comment></comment></comment><comment><USER11> I have long believed that Facebook/MySpace is like a TV show. Once the "content" becomes less interesting, people will slowly tune out. I am personally seeing fewer and less interesting updates from my friends.

In fact, the general theory of mine is that any business whose key metric is "time spent per user", is basically a time sink AND in the media business and needs to keep up with new ways to entertain people. Those businesses have shorter shelf lives.

On the other hand, Google/Microsoft are in the productivity/time saving business and they have more durable advantages since they don't have to keep coming up ways to make people stick around. People stick around and use their service, because people save time.

Watch out for Twitter to do to Facebook what, Facebook did to Myspace and what Myspace did to Friendster.

Twitter on the other hand may not exactly be like a TV show as much as TV channel with each mega Twitter star (e.g. Charlie Sheen) a TV show within that channel.

<comment><USER12> Same here. I'm getting more and more bored with it. What was most interesting about Facebook for me was learning about certain people. This guy I work with likes cheesy movies, goes out to Irish bars with his wife, and likes to tweet quotes from whatever movie he's watching when he's drunk. Learning that was interesting. Seeing the same pattern every day is horribly boring. Same thing with most of my high-volume friends. A close high school friend's wife, whom I haven't had a chance to get to know in person, loves mariachi music, can't write a coherent sentence to save her life, goes to la pulga (the flea market) every weekend, enjoys drama with her coworkers, and is constantly sharing coupon deals. Fascinating to know, boring as shit to see every day.

And of course the highest volume people are the most boring. All my friends seem to have about the same volume of inanity, funny stuff, somewhat interesting comments, worthwhile content, and vital life updates to share. They just vary in what their sharing threshold is. If I could tune out everything below "worthwhile content" then Facebook would be great. I'd also only need to look at it once a day for two minutes to keep up with everybody. Alas, there is no "setLogLevel" method on my Facebook feed.

<comment><USER13> I am not on Facebook, but the last few games I tried on iOS have a "post achievement on Facebook/Twitter" _every single level_. I can't help thinking that if anybody is actually pushing those buttons it must drive their "friends" absolutely bonkers.

On some level I can understand the concept of "keep your friends up-to-date with your life". But not using these definitions of "friends" and "life".

<comment><USER14> You can block specific applications from ever being shown to you. If it wasn't for that feature Zynga games would have made me quit facebook a few times already.

Game updates are annoying, but they aren't much better than anything else anybody posts, ever.

</comment></comment></comment></comment><comment><USER15> We're two of those deserters here. It seemed too much like tv, in reverse. Everybody transmits and nobody watches.

<comment><USER16> This is a problem inherent in any form of social media. Unfortunately, it's not easily fixed. You need to find a way to strike a balance between rewarding content creation and punishing spam, and the line in thin.

<comment><USER17> people bring this stuff on themselves by treating fb as a massive 'who I know or am related to' index instead of following who they are interested in. I suppose fb brought some of this on themselves when they went down the twitter- emulation path a few years ago.

<comment><USER18> It's interesting that you mention the "who am I related to" aspect, because Geni.com is exactly that. After altercations with unhinged second cousins- once-removed and unpleasant uncles, I set privacy to maximum and abandoned Geni (Geni's habit of spamming people who you added also led to a few unpleasant encounters at family reunions). The family tree idea was good, but the social aspect turned out to be unpleasant.

Some of the same relatives are friends on Facebook, and it is much more pleasant experience, because they are nominal "friends", but are in a high privacy group, so we don't have much opportunity to interact.

</comment></comment></comment></comment><comment><USER19> In related news for the first time ever, LinkedIn trumped Facebook in a survey about which social networks people care about.

[http://www.bnet.com/blog/business-research/linkedin- trumps-f...](http://www.bnet.com/blog/business-research/linkedin-trumps- facebook-in-popularity-contest/1660)

Apparently helping get you sucked in to Farmville is not as valuable to people as helping you get a job and stay connected with what is happening with old co-workers. Go figure.

<comment><USER20> I'm not sure you could call LinkedIn a social network. It can be nice to find out about an old friend, but most of it is pure business. So I'm not surprised about the survey.

<comment><USER21> The best thing about LinkedIn for me is their recommendations system. Whenever I'm looking at people's profiles on LinkedIn, I'm looking at their work history and their recommendations, the other stuff being basically useless. And mostly I value recommendations coming from people I know.

So it's a social network and I think it is more pure and valuable than the Facebook graph since on LinkedIn you only want a thousand connections if you're an HR person, not because you need friends for Zynga games, so most people on LinkedIn have between 10-200 connections representing old colleagues.

<comment><USER22> The recomendations aren't going anywhere these days. I distrust people with a huge number of recommendations and when you read through, you understand it's a mutual favors system. Not that mutual favors are bad, just that most of it so shallow.

</comment></comment></comment></comment><comment><USER23> Twitter for me. One of the things I wanted out of Facebook is to talk to people I knew regarding various topics. I found out that most people just play Farmville, post happy birthday notes and pictures of their kids. While catching up with old flames was fun, the interest is gone.

On Twitter, I can search the entire world for people who have the same interests as me, read their thoughts and correspond with them.

So basically I left Facebook because I really wanted to connect with people on an intellectual level. Turns out, I don't know anyone who I can correspond with in that manner.

By left, I mean I deleted the account, whatever that means in FB land.

</comment><comment><USER24> Does anyone have any good guesses as to what the "next" Facebook will be? I mean, in my experience, it basically went Friendster -> MySpace -> Facebook. While Facebook is certainly in the drivers seat to a much greater extent than MySpace was five or six years ago, history indicates they'll eventually be knocked off their throne. However, I haven't seen or heard of any serious newcomers in that space. I've personally seen enthusiasm for Facebook diminishing over the past year or two, but will Facebook be end up being the ultimate "social network"? Is that basically settled?

<comment><USER25> My hope is that people have simply tired of the concept. The privacy implications are staggering.

<comment><USER26> I don't see anything wrong with a general social network that supports sharing information with people you know. Regardless of how far people have stretched and abused the concept of "people you know" (and "information"), the infrastructure seems useful. So I don't want to see social networking go away; I want to see it entirely decentralized, such that each person's social network goes through that person's personal site.

<comment><USER27> By their very nature, centralized social networking sites like Facebook force you to share information not only with the people you know, but also with the owners of site.

Some people don't mind this, but I for one don't want Facebook, MySpace, or any other social networking site knowing who my friend are or what I'm sharing with them. It's simply none of their business.

The only hope I see for a truly privacy-respecting social network is a distributed, peer-to-peer network. That way you'd be communicating only with those you know, at least in principle.

But even there there are major problems, such as the need for information to travel over untrusted hops to get to its destination. Encryption can help with some of it, but would still be subject to traffic analysis. Maybe a social network could be created over TOR. I don't know.

<comment><USER28> Encrypted, peer-to-peer social networks are prohibitively difficult. More likely, we'll see node-to-node social networks first.

</comment></comment></comment></comment></comment><comment><USER29> I don't think this is just Facebook, perhaps the so-called social media jumped the shark. I think it jumped the shark a year ago for both Twitter and Facebook (CNN, Ashton Kutcher contest) but the numbers are starting to catch up now. Also don't discount the Linkedin IPO where some would argue the social media was exposed.

</comment><comment><USER30> It could be that the churn just exceeded new users for the first time because most possible users in USA have been reached already. Some decline is inevitable if everyone is already using your product.

There's always churn. Even as FB was growing, a part of those new users were always churning out. Each user makes their own decisions about staying in the system, they are not a sentient whole except as far as media or changes to the system might affect larger groups at once.

I imagine your tendency to leave the system to be some sort of a graph. Maybe you are 30% likely to leave and never ever come back after your first day on Facebook. After using it for a week, maybe that gets reduced to 20%. After a month, maybe you are even more unlikely to leave. If they had for example 1 million people joining per day, then 300k would be quitting per day. If the next day all of USA had been tapped out and only 200k people joined, that would look like a sudden -100k turn to decline, even though the trend was same as always.

What happens now will depend on what that churn graph looks like. Maybe it's sticky enough that they'll be able to keep a big part of the users they got and over time make it even stickier to keep even more. There will be improvements to get those users to be more active on the site. Even if there was no more user growth in the USA, there could be growth in the amount of time and money the existing userbase spends on the site.

<comment><USER31> That this is true is a bit of a tautology. Assuming the graph is monotonically decreasing sounds unlikely to me, though; I think there's likely to be multiple peaks - like the proverbial "7-year itch" for marriages.

Another graph that would be relevant would be the likelihood of people who have been exposed to facebook through their friends, media saturation and advertising, yet still haven't joined, to be tempted to join later. In other words - facebook has already saturated the internet world's radar, and people who haven't joined already are completely aware of its existance, function, and benefits, and still not joining. Those will be very expensive users to get, so IMO retention should be their major focus. I stayed through the "6-month updates-full-of-farmville problem", because they fixed it, they barely kept me through the "1-and-1/2-year privacy panic" after backing off on some portion of their stridency (and profitability per user), and they lost me completely, along with a number of other people I know, after the "2-year bored-to-shit-with-people-I-went-to-HS-with-telling-me-where-they-went-to- lunch-and-what-they-think-of-Obama lull" which I'm not sure they can do anything about.

If a large cohort of similarly aged users hit some peak that facebook is unable or unwilling to iron out at the same time, and there's no supply of cheap "Wow, there's a place on the internet where I can keep track of all of my college buddies!" users to replace them, there could be a precipitous fall.

</comment></comment><comment><USER32> I see their numbers. How can I trust them? Mostly I see a lot of stuff trying to sell to you their service but I don't see how they actually collect their data.

<comment><USER33> As someone else mentioned, they use Facebook's own advertising tool to estimate the numbers. And their could be many things wrong with it. But if you put that number in the context of other metrics, such as Google trends ([http://www.google.com/trends?q=facebook&ctab=0&geo=u...](http://www.google.com/trends?q=facebook&ctab=0&geo=us&date=all&sort=0)), then the numbers make sense. Now the question is, are they gonna have slow sustained growth from now on like Skype ([http://www.google.com/trends?q=skype&ctab=0&geo=us&#...](http://www.google.com/trends?q=skype&ctab=0&geo=us&geor=all&date=all&sort=0)) or fall off a cliff like MySpace ([http://www.google.com/trends?q=myspace&ctab=0&geo=us...](http://www.google.com/trends?q=myspace&ctab=0&geo=us&geor=all&date=all&sort=0))? Remains to be seen.

<comment><USER34> that Google Trends result is filtered to show the USA only, yet the 'top languages' are, in order, Turkish, Spanish, Italian, French and then English in fifth.

makes me very skeptical about their numbers

</comment></comment></comment><comment><USER35> I felt this was bound to happen. Too many "friends", too much spam, and it became too "mainstream", which means the early adopters would want to move on to something else.

If there actually was "something else" right now, a paradigm shifting social network, the decay of Facebook would've been even faster, but unfortunately there isn't much to replace it right now, but I think there will be by the end of the year.

</comment><comment><USER36> I'm still not sure why FB has not figured out an easy way to segment and categorize friends. FB puts me in the same room with my mom, coworkers, school buddies, and nephews. Awkward.

<comment><USER18> It allows you to group/categorize your friends, and set privacy for each group for most actions. If you wanted to hide everything from your mom, you could. If you need to share an update with your nephews, but not your school friends that's easy.

Perhaps having separate Walls for different groups is the next step.

<comment><USER37> That's true for a certain category of content - wall posts, photo albums, notes - although they don't make it especially easy. For comments, you not only can't control the privacy setting (it's controlled by whoever posted the original content), you can't really even tell what it is. For other things - likes, your friend list - you can't control it at all.

</comment></comment></comment><comment><USER38> As any other social site ever created it seems to be following the same path. Each month what I'm seeing on my landing page is getting less relevant and more annoying (even without growing my friend base too much and having around 20 friends, I cannot imagine what happens to those over 100). Most of that spam is generated by few active friends, while the rest does pretty much nothing. Most of communication that I see taking place is very very meaningless, somebody posts a video: "lol", "haha", "nice one", that's pretty much it.

For me it's a socially filtered spam [funny stuff?] site.

If some site can encourage people to have meaningful communication again it will take on Facebook, no doubt. Get 3 of my important friends on the site and I'm sold. Then Facebook just becomes a big contact list. And at that point I'm not sure where it goes. It might be a contact-list site for a long long time.

Facebook might not die as a website, but competitors will definitely takeaway a lot of value from it. And so far I don't see it becoming much more than a contact-list.

And keep in mind that the less engaged the users are the less ad revenue there will be. I have no doubt that as we're speaking right now users are less and less engaged with the site. Growth is still fast so the revenue is rising, but once that stops it will crumble.

The will have to get more aggressive with ads or people will simply stop to notice them at all. Or perhaps start selling more of the user data. At this point there's not much you can do.

Also the growth they see outside of the western world might be very much junk. I'm in Poland, Facebook craze is already old news here yet I get english ads? It's great that AT&T has high speed internet for 20$, but I'm sorry, I cannot purchase that here.

Everybody knows ad revenue is tricky. Facebook is no different.

Nobody is going bankrupt, I'm not saying that. Facebook is here and it will stay for few more years for sure, but to compare it to google is crazy. Google can be conquered too I'm sure, but Facebook definitely makes for a far more fragile target.

Personally I'd say it will simply decay and we might not see another social site being that big in a long time to come. Facebook has half of the world on there, cool, but guess what, I am not connecting with those people and never will.

Facebook managed to get a lot of people registered in one place, but has it managed to truly connect a lot of people? Not at all.

</comment><comment><USER39> Anyone know their methodology? I clicked around a bit and couldn't find any information. Without knowing more my prior would attribute it to a bug.

<comment><USER40> The methodology is basically using the Facebook Ad Tool. The tool let's you target a set of users for your ads and tells you the size of that population.

The article clearly discloses the inherent issues at play.

> Going forward, we'll be watching closely to see what longer-term trends > emerge. Bugs in the Facebook advertising tool that we draw this information > from, seasonal changes like college graduations, and other short-term > factors, can influence numbers month to month and obscure what's really > happening.

<comment><USER41> Other factors like Facebook finding and removing spam accounts, users enabling privacy features, etc. seem like a much more likely cause.

I could see people getting bored / busy and not bothering to use it any more, but it's unbelievable that 1.5 million would delete their accounts in one month.

</comment></comment></comment><comment><USER42> I deleted facebook about six months ago because it was a huge stressor that added very little value to my life. I have too many subsets of friends, which makes it impossible to maintain one personality without offending someone. This just isn't how human's interact and I think it will eventually kill off facebook, unless they find some way to mitigate it.

For reference: 1) Highschool friends and family who are incredibly religious/right wing/etc 2) Friends from skiing, many of whom make a living on the mountain, who are about as vile as one can imagine 3) Friends from college or work, which consists mostly of PharmD's, MD's, and nurses. 4) Other random friends from startup stuff, partys, etc

<comment><USER43> I think humans inherently want to nurture each of their relationships, whether close or not, individually. Along with tailoring our communication to each person, we exclude large parts of our lives and minds from people that we want to be friends with. Doing that naturally and through different contexts is one thing, but having to electronically do that for each relationship is just impossible.

</comment></comment><comment><USER44> 134 points, 76 comments, and maybe 3 of them critically examining the methodology :/

</comment><comment><USER45> To quote the Disney Hipster meme, "it's too mainstream". Perhaps I have an obscure point of view, but Facebook lost it's luster for a lot of people when the older crowd started to really embrace it. It's not as cool as it once was. This is a big problem for a lot of companies, not just social media.

<comment><USER46> This. Was pretty cool at first connecting with classmates and writing messages about papers, projects, and planning stuff for the weekend. Now it's some gossipy aunt you haven't talked to in months and your seven year old niece sending a friend request.

</comment></comment><comment><USER47> I don't have figures to back it up, but Facebook in Belgium also feels like it's declining. Not necessary new users, that may be still even rising. But mainly active users becoming inactive. Im daring to bet that's declining big in Belgium.

</comment><comment><USER48> "Most prominently, the United States lost nearly 6 million users..."

You know who else lost nearly 6 million users?

Yep. AOL.

</comment><comment><USER49> If true, this is pretty interesting in the context of the privacy and other mishaps from Facebook over the last few months.

I wonder what the breakdown would be for the reasons for quitting.

</comment><comment><USER50> China is really a stumbling block. I've actually been thinking about the best way to penetrate the Chinese market... I'm thinking the strategy is to start young, and keep good relations with their government. I guess this is tricky, especially if the founders can't speak Chinese, and have no family ties.

<comment><USER51> Well Mark Zuckerberg is into Asian girls so I suppose that's his in.

</comment></comment><comment><USER52> They don't say why or where those users are going. If the users aren't going anywhere else, they just decided that Facebook weren't for them. This might be alright for Facebook because I guess they want committed users that log on the site several times per day, not users that log once in a while.

<comment><USER3> Where else would they go, at this point? I don't know of any competitors that are live and able to target Facebook refugees. Diaspora is still in Alpha, for example, and I suspect that the buzz on that one has been fading for some time.

<comment><USER24> Yeah, I'm not sure there's really such a thing as a "Facebook refugee" yet. That is, someone who leaves Facebook and is looking for something better. My guess is that those leaving Facebook are those saying "This whole thing is dumb", and are not looking to continue to engage in online social networking... at least in the way it's been presented so far.

<comment><USER53> Anecdotal: yes, that's exactly the thinking of my friends and I who have given up Facebook. It's turned us off the very concept.

</comment></comment></comment></comment><comment><USER54> It's trendy to hate Facebook these days but I think it's more than that.

People want to stay connected especially people 20 to 30 years-old and how do they do that now? Smartphones, iPhones, iPads and other mobile devices.

Facebook hasn't lost the battle it's just become obsolete.

</comment><comment><USER55> I am personally seeing the growth from Mexico. In the past month, a large chunk of my cousins have added me. But like others have said, most of their updates are from the Spanish version of Farmville, etc.

</comment><comment><USER56> And what do you know, The Social Network came out on cable around the same time. Coincidence?

</comment><comment><USER57> I blame teenybopper exams.</comment>


Ask HN: Recurring payments for static pages - <AUTHOR> Hi, I'm in the process of building a static page where we want to make it possible for users to give a monthly gift. If it is possible I would like to avoid setting up a back end so I'm looking for a SaaS that I can embed on my static page to start charging recurring payments. Do you  know of any such solution?<p>Thanks! <comment><USER2> You have some services made to support creators, like "Buy me a coffee" or Patreon

</comment><comment><USER3> PayPal?</comment>


France orders Microsoft to stop tracking Windows 10 users - <AUTHOR> http://www.theverge.com/2016/7/21/12246266/france-microsoft-privacy-windows-10-cnil <comment><USER2> Is there any accurate analysis of exactly _what_ Microsoft collects in Windows 10? I understand you cannot totally disable the telemetry (with the exception of an Enterprise version) but when put down to the "basic" level is there any capture of what is being sent? Everything I have seen is bullshit anti- Microsoft fairy tale stuff.

I understand text and voice data will be captured and sent if you use Cortana but that is pretty obvious, the same is true of Google, Bing, Siri, etc. [0]

What I want to know is when I put things at the lowest setting possible what do MS get and how often?

Edit: [0] I mean captured and sent for processing. I expect (perhaps wrongly) for it to be deleted from Microsoft's servers as soon as my request has been answered. Unlike Google which stores _everything_ you say to Google Now for example.

<comment><USER3> As far as I understand it, there is no way to know what Microsoft sends, unless you work at Microsoft.

As such, the "bullshit anti-Microsoft fairy tale stuff" is just as valid as whatever you believe that they send. Especially also under the connotation, that Microsoft does actually reserve the right to send anything they want in their Privacy Statement.

And we shouldn't treat privacy as "innocent until proven guilty". If they cannot provide a lower boundary where you have a guarantee that they respect it, then we should assume the worst.

<comment><USER4> But why just target Microsoft? What about Google? Apple?

Big data is huge and there is no end in sight as far as growth goes! Telemetry is a big success. And we're talking about regular usage now. Hospitals use new state of the art EMR software with all kinds of telemetry built it as well.

<comment><USER5> Is this a calculated derailment technique? Why MS? Because MS is the subject being discussed in this post, is why. Wanna talk Apple? Open up an Apple privacy related article's comments and you'll read the same concerns accompanied by more red herrings asking 'why just Apple?'. Repeat for Goggle, if you need further clarification.

Why not Verizon? Why not NSA? Why not the traffic strips counting cars? why not the ticket guy @ the cinema?

<comment><USER4> Let's rephrase that, why isn't the French government going after Google, Apple, and others for telemetry?

<comment><USER5> They have. Repeatedly. And for whatever reasons most are unaware of, they appear to be getting away with their actions more or less. Why? I don't know, ask a lawmaker. Or better yet, ask a lobbyist. Even better, ask the DHS PAC:

Edit: US example, I have no idea how France government works. [http://www.boilingfrogspost.com/2011/11/30/bfp-report- meet-t...](http://www.boilingfrogspost.com/2011/11/30/bfp-report-meet-the- department-of-homeland- security%E2%80%99s-%E2%80%98distinguished%E2%80%99-privacy-advisory-committee- members/)

</comment></comment></comment></comment></comment></comment><comment><USER6> I would like laws that would force companies to disclose the bad side of what they are doing.

When they ask if you want to share what you type or say with them, in order to improve the experience and for you to get more relevant suggestions or more accurate spell checking or whatever, they only focus on the positives.

But that's not enough. I want them to say that your delicate and private conversations might leak and be used for nefarious purposes by disgruntled employees, state agencies, hackers or future owners of that data, because that's the truth.

Much like how cigarettes packs have graphical warnings on them. I'd like that very much, because as an ex-smoker I can tell you that those work. But of course, it would hurt their business to admit it, so they'll never do it willfully.

<comment><USER7> Or like the financial industry, which products are deemed too complex to the average client. They must disclose and inisist on the risks of a product.

Windows pretty much checks the box in term of complexity to the average user.

<comment><USER8> _> They must disclose and inisist on the risks of a product._

By giving me a 50+ pages brochure. Just like EULA's hide the nasty stuff among huge amounts of text.

Sure it's better than nothing, but still far from perfect IMO.

<comment><USER7> No no. If you trade something on margin or borrow money, in many countries like in the UK, all the marketing material has to clearly state the risks. All mortgage ads have a big banner "if you do not make your payments your house may be repossessed". A derivative contract with a mid-size company is now deemed missold if the marketing material doesn't highlights adverse or worst case scenario.

These are not font 6 footnotes in the terms and conditions.

<comment><USER9> Yes, and what great nanny-stateism. The only people whom that big warning helps are people who don't know what a mortgage actually is, yet are planning to take one out anyway. I'm sure such people exist in very small numbers but this is how you get "Warning: may be hot" on coffee cups. Where do you draw the line? Surely someone who walks into a bank and says "I'd like a mortgage" should be expected to know what repossession is?

<comment><USER10> And yet in the US look at any prescription drug ad, where there's this ugly extended component of the ad that details dozens of possible issues, which to your logic would be the job of the physician (and I agree), not the advertising.

</comment></comment></comment></comment></comment></comment><comment><USER11> People are giving special attention to the things that Microsoft is doing, but history shows that they will eventually accept it and live with it.

Facebook has been doing this for a long time, to extremely high degrees of invasiveness. Google as well, and pretty much every single web startup in existence. Collecting data is how you compete in modern business.

If you think this Microsoft stuff is a big deal you should have another look at the entire foundation of modern tech.

<comment><USER12> You are right, but I bag to differ. Giving out your personal emails or your private life to advertisers is one thing. But an operating system is the lowest dominator and the PC platform is where most data is stored and real work in being done. So Microsoft's 180 degree turn by the CEO from being a trustworthy company to a turning their customers to products is unheard of and very very concerning - especially given they still have a quasi PC monopoly (ca 90% market share). There is a long legacy history , some run up to 30 years old software on Windows and now are concerned with little option to move forward.

<comment><USER11> Is that much different from Android? Google is known for being incredibly pervasive, and also invasive.

I'd argue that my phone is almost a lower common denominator than my computer. My phone knows my geographic location at virtually all times, has logged into most of my web accounts, handles all of my social contacts (definitely more than Facebook. But even people who primarily use Facebook likely use it mostly from their phone and not their computer).

If we want to care about this stuff, things need to change in very dramatic ways.

<comment><USER5> "If we want to care about this stuff, things need to change in very dramatic ways."

Absolutely & unequivocally. Hard part is, the technological means is baked into every modern SoC & NIC, huge amounts of money are being made from the 'big Data' industry created & the psychopaths are driving the bus. We survived just fine back in the Stone Age(pre-smartphone) and we _can_ hold out for change($=vote). Principals often require certain sacrifices.

</comment></comment></comment></comment><comment><USER13> Are there other countries as active as France is concerning the enforcement of user privacy laws and data protection?

<comment><USER3> Switzerland is usually pretty good at it, and Germany tends to also care, because of their Stasi-history, but the recent trend of "Terrorists killed a handful of people, let's throw away the fundamental rights of our population." hasn't left those two unscathed either...

<comment><USER14> >Terrorists killed a handful of people

234 people dying from terror attacks in France in the last 18 months is not a handful. Don't just dismiss their deaths so heartlessly as a "handful".

Shame on you.

<comment><USER7> Realistically it's an average air crash. At the scale of a population like France it is almost meaningless. Drowning kills several orders of magnitude more. I don't see calls for changing the constitution to reduce the number of drownings.

Or suicidal airline pilots...

<comment> angry-hacker But it's one airplane crash too many. I don't understand your point. We shouldn't care?

<comment><USER15> Not to answer for the parent but IMHO it's not so much we shouldn't care. It's we should be very careful about what rights and freedoms we give up and how much power we give to a state to scrutinize our daily lives when the very thing we are trying to prevent has not had a major impact on our daily lives.

Yes, it had a major impact on some people and their families and their pain is not to be dismissed lightly but on the other hand we should not give up our way of life to prevent these things from happening, especially when evidence supports that the new laws that are written after these events seem to do very little to actually further protect us.

While a little sensational I think the graphic in this article concerning the TSA in the US does a pretty decent job of summing up exactly what we've gotten from "enhanced" security in the US.

[https://www.techdirt.com/articles/20120405/04390118385/tsa-s...](https://www.techdirt.com/articles/20120405/04390118385/tsa- security-theater-described-one-simple-infographic.shtml)

</comment></comment></comment></comment></comment></comment><comment><USER16> Announcement at CNIL: [https://www.cnil.fr/en/windows-10-cnil-publicly-serves- forma...](https://www.cnil.fr/en/windows-10-cnil-publicly-serves-formal- notice-microsoft-corporation-comply-french-data-protection)

</comment><comment><USER17> I switched to Ubuntu because of Windows 10. Using Linux as a primary operating system works great as a developer.

There is a saying which says if a product are free "you are the product". Microsoft made the upgrade to Windows 10 free a guess so that they can mine data about you and your habits. That data is valuable for marketing purposes.

Wireshark traffic dumps show a lot of data going to Microsoft telemetery.

I choose to say no to that data collection, instead wanting to keep a bit of privacy.

Have some security wiz MITM the Microsoft telemetry server with their own cert to inspect the data collection traffic?

<comment><USER18> I recommend disabling some of the Amazon integration in Unity if you're concerned about your privacy. This shouldn't be too difficult, even if you're new to Linux. I 'd also suggest getting Unity-tweak-tool and playing around with some of the settings in there.

EDIT: Just realized that since 16.04, Ubuntu no longer has online search results enabled by default in the Dash. Still though, play around with some of the settings!

<comment><USER19> One doesn't really need Unity, does one? Openbox is awesome. It doesn't hog system resources either.

<comment><USER3> Yeah, but Openbox isn't suitable for new users, at least not on its own. As part of something like LXDE or LXQt, it's definitely workable, but without a deeper understanding of Openbox, those are pretty much as good as any other Desktop Environment (except maybe that LXDE/LXQt are also among the lightest DEs around).

I would rather recommend either Ubuntu GNOME, if you want a rather unconvential, but highly integrated, highly user-friendly interface, or Linux Mint Cinnamon, if you want a cleaned up Windows-like interface, or Kubuntu, if you want a Windows-like interface which just smothers you in options, tweaks and customizability.

</comment></comment></comment></comment><comment><USER20> > the four-character PIN system used to access Microsoft services is insecure, > because there is no limit on the number of attempts a user can make.

I just tried logging in with my pin.

After a handful of tries I was given a string to enter before I could try again. I did that. After another try I got told to restart the device before I could try again.

So it doesn't look like 10 tries and locked out forever, but rather increasing penalties for incorrect attempts. Which is fine.

oh and my pin is 6 characters long.

If they don't have this right why should we believe them about any of their other claims?

</comment><comment><USER21> _> Microsoft: users are in control with the ability to determine what information is collected

> Microsoft: so enterprise customers will be able to completely turn off > telemetry if they choose[1]_

Which is it, Microsoft?

[1]: [http://www.techrepublic.com/article/windows-10-now-lets- you-...](http://www.techrepublic.com/article/windows-10-now-lets-you-turn-off- tracking-but-only-if-youre-a-business/)

<comment><USER22> "Determine" also means "find out"; not necessarily "decide".

<comment><USER21> > find out

What is the wire format of the telemetry data? How do I access the UI that tells me _exactly_ what telemetry has been sent to Microsoft?

<comment><USER22> Don't ask me, I'm not MS. Just pointing out that there's enough weasel words in there that the text can be twisted to be sort-of-not-entirely-false.

<comment><USER21> I can agree on that point. The point of my comment wasn't necessarily the truthfulness of either quote, merely that they really need to follow through on what they are saying. Their hands and mouth seem to be in a state of disconnection.

</comment></comment></comment></comment></comment><comment><USER23> This is a step in the right direction, but no fine they can levy will be sufficiently punitive.

Companies like this will continue on and consider things like this simply the cost of doing business.

Kind of like banks. They don't give a fuck.

<comment><USER24> The EU commission levied a $2bn (yes, billion) fine for Microsoft antitrust problems, i.e. the stupid removal of the choice popup for the browser in XP SP3 and not giving the API documentation to some editor. Those two things were very stupid misgivings by Microsoft: They were almost going out unscattered, and they played with the fire towards the end of the trial.

Since then, I like the EU commission. As long as they can bend the master plans of dominating US companies.

However, I don't see how the forced Windows upgrade didn't lead to a requirement to reimburse every user of their stripped Windows 8 license.

</comment></comment><comment><USER19> With LinkedIn [0], Microsoft has much more in its arsenal.

That said, who cares. I've hardly seen anyone use uBlock Origin, Ghostery or Privacy Badger. OTOH, people love tools [1] that read your email and notify about due bills and the like.

[0] [https://twitter.com/darylginn/status/590664399041519617](https://twitter.com/darylginn/status/590664399041519617)

[1] Google Now

</comment><comment><USER25> I have LittleSnitch on my Mac and observe the requests that my Windows VM makes. I believe you can use an equivalent tool on Windows, such as GlassWire, or also the very useful tool O&O ShutUp10 with which you can disable telemetry settings.

<comment><USER3> Wireshark is most commonly used for packet sniffing.

[https://www.wireshark.org/](https://www.wireshark.org/)

</comment></comment><comment><USER26> Good luck with that.

</comment><comment><USER27> Windows 10 petite edition - coming soon.

<comment><USER28> Or perhaps the opposite; sell only the Enterprise edition in France, with its ability to disable telemetry... at a price.

<comment><USER24> ...and lose market share. Microsoft really needs to keep its product proeminent, especially today, especially in home products. So the EU commission actually has leverage to forbid certain behaviours.

Microsoft, then Android, please ;)

<comment><USER29> Oh please, what are people going to do, switch to Macs? Or Linux? People have shown over and over that they're willing to accept any treatment Microsoft sees fit to give them, and that they will not abandon the Windows platform no matter what. MS can do whatever they want and customers will just take it.

<comment><USER24> The upper comment says "What happens if Microsoft only sells the Enterprise edition in Europe?", it will increase the price. In the absurd event when Microsoft doesn't sell the OEM anymore, people would have to pay $250 for a Windows license, which will make a lot of people switch. Yes, to Linux for the best, and to Mac for the new buyers.

<comment><USER29> Paying $250 for a Windows license is still less money than they're going to pay for a new Mac. Macs are very expensive.

And even so, lots of people have software that isn't compatible with Macs (or Linux for that matter).

I'm honestly curious what would happen if MS decided one day to jack up their Windows license costs to, say, $1000 per copy. Or what about $500? Would their revenue go up or down? Obviously, some people would switch to something else, a bunch more would just stick with what they have, but people do buy new computers now and then, and businesses are always refreshing. Combine this with "updates" to existing Windows versions to make them slower and slower and slower (like iOS does) to force people to get new versions of Windows, and it seems to me MS would probably make a lot more money by gouging customers as much as they can.

<comment><USER27> If MSFT charges 1000$ for Windows it wouldn't matter much (other than negative press) most windows licenses are sold through the non-retail programs such as:

Enterprise/Volume Licensing, SMB Licenses, Educational Licences (Students and Teachers), Developer Licensing (MSDN), Microsoft @ Home (buying considerably discounted licenses through your employers) and OEM licenses for retailers and system builders...

Since Windows 7 the vast majority of the windows licenses that are not VL are cooked into the machine (stored in the BIOS) and are effectively non- transferable (OEM lic's were never transferable, but now there is no sticker with a key anymore).

If microsoft decided to charge 1000% for the retail the majority of the licensing programs would be immune, OEM's/System Builders would love it (can rack up the prices since Windows is so expensive now), and pretty much none of the normal users would care/be affected by it directly.

<comment><USER29> Ok, maybe I'm not being clear, I'm not talking about MS just raising prices on certain licenses, but really across the board. What if they also jacked up their license costs for the ones "cooked into the machine", and also their Enterprise/Volume licensing?

Heck, I think they have even more room to screw their customers with the Enterprise licensing, not only for Windows but all their other enterprise products too. What are big businesses going to do, suddenly switch to Apple? Obviously, they couldn't do this overnight, but they could certainly jack up their enterprise license costs to 5x when they come up for renewal. They'd probably want to avoid doing this too much for some things where there's actually competition (like with SQL Server: businesses might switch to Oracle), but with Windows, MSDN, etc., they could. Where are customers going to go?

<comment><USER24> It's a company with millions of dollars on marketing. You can bet they've studied their price point and it is exactly at the right ratio between sale quantity and sale amount. If Linux became unsafe/untrusted (e.g. unapproved by S&P500 audit companies for example, or if black hats activities were discovered at the kernel level of Linux), however, they would become a monopoly on PCs, and they indeed could triple the price.

<comment><USER29> I'm sorry, I have to disagree with this. I don't see how the perception of Linux's security would have anything to do with Microsoft's pricing for Windows. As much as I wish the case were different, Linux is simply almost never considered as a viable alternative to Windows (on the desktop), except in a few rare cases which are so rare, they become big news items (e.g. city of Munich government). Both companies and individuals are too invested in Windows and its software ecosystem to even contemplate a change, except maybe to Macs. I do see more people switching to those, both individuals and companies actually (I almost interviewed at a small company that was all-Mac). So if MS quintupled their prices for Windows Home/Pro, I could definitely see that spurring a defection to Macs, though I still doubt it'd be that big because of the installed base of Windows software that doesn't have a Mac version. Also, Macs just don't seem to be set up for enterprise use the way Windows is, and while Linux can be, it takes more IT competence than I've ever seen at a large company (which these days seem to be outsourcing to external vendors anyway).

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Ah, CNIL.

What counts as "excessive"? Apparently whatever someone at CNIL thinks is excessive. I can imagine that Microsoft learning what apps you download is inevitable given their reputation based malware detection scheme: no way for that to easily work except by IE checking in with Microsoft to find out if a program is known malicious or not. And figuring out if a program is actually interacted with or not seems like a pretty good signal to determine if a new, unknown program is a silent botnet or not.

"4-PIN limit is insecure, because there's no limit on the number of accesses" is exactly the kind of bureaucratic central-planning nonsense that France has so many problems with. You do not need absolute counted limits on a password/PIN system to make it secure. You just need to take other steps to make brute forcing infeasible, like throttling the rate of attempts. Why is CNIL attempting to micro-manage the code for the Windows authentication systems, something they are clearly not qualified to do? The details of Microsoft's security system is their concern alone: if users dislike the way Microsoft do it, then they have other alternatives they can easily switch to.

I suspect Microsoft may do what other big companies do and simply ignore CNIL completely. They can only hand out relatively small fines and it's easy for big companies to just pay them off to make them go away. Their rulings have a long history of being completely unreasonable so it's usually the easiest path.

<comment><USER30> > What counts as "excessive"? Apparently whatever someone at CNIL thinks is > excessive

"Loi n° 78-17 du 6 janvier 1978 relative à l'informatique, aux fichiers et aux libertés" is quite specific about collection & processing of personal data. A good example of what falls foul of this legislation: logging everything for unspecified purposes to cross-tabulate it with other unspecified records in case it might be useful in some way (which might not be in the user's direct interest) within an undetermined timeframe, without letting the user know about it precisely nor letting him opt out.

CNIL is annoying and their enforcement is spotty for lack of budget (so they have to focus on landmark cases) - but their actions are well grounded in legislation and actually protective of people.

> I suspect Microsoft may do what other big companies do and simply ignore > CNIL completely

Please do that - I'm off to fetch some popcorn !

<comment><USER9> The use is always specified: usually something like "running existing services and supporting the development of future services".

Oh, that's not good enough? Well now you are back to what I said: it's simply central planning nonsense where a regulator makes up rules on the fly.

I have seen _no_ evidence that CNIL or indeed other bodies like them protects people from anything. Please show me one, completely unambiguous case of someone who was clearly suffering whose suffering was rectified by CNIL forcing some change to a privacy policy somewhere. And I mean really has a problem, not some emotional airy-fairy feeling that they'd prefer things to be different, I mean concrete, quantifiable issues: like monetary loss.

There is no need for popcorn. I think the biggest fine CNIL can usually hand out is like 300,000 EUR or something. Just pay it Microsoft and get on with things.</comment></comment></comment>


Cult of the Founder - <AUTHOR> https://pando.com/2017/02/28/its-more-fate-just-uber-cult-founder-risk-and-lot-vcs-are-thrilled/45ad4fbcc1abf06f490952e362091eb1c5533100/ <comment><AUTHOR> _" The cult of the founder came from a good place-- many VCs did overreach in the late 1990s and treated founders like shit. Mark Zuckerberg has proven that a 21-year-old kid doesn't need a grown up CEO to run his company. But that doesn't mean that founders are gods who should never be challenged."_

Great observation and even-handed. Founders are inexperienced and need a lot of guidance to over-come their deficiencies. This comes with time and prudent advice. That is if founders are capable of taking it.</comment>


Ask HN: API to get flight prices and info? - <AUTHOR>

I wonder where do startups like Hipmunk (specially when they first started) or Trabber get their data? I asked the latter and they replied they couldn't tell. Seems like too many variables and changing too quickly (prices) as to be scrapped. Do they scrape when you perform a search? Is there any API that gives you that info for a lot of airlines? If so, is it free or you have to pay for it? Or do they negotiate with each airline independently? <comment><AUTHOR> Found this at StackOverflow: [http://stackoverflow.com/questions/4011123/is- there-an-api-f...](http://stackoverflow.com/questions/4011123/is-there-an-api- for-booking-flights-and-or-cruises)

Does anyone have experience with any of them?

</comment><comment><USER2> I remember talking with a flight scanner startup at Dublin Web Summit - I'm pretty sure they said they scraped at the beginning...</comment>


New Canon 5d MkII dSLR can shoot stunning 1080p video - <AUTHOR> http://blog.vincentlaforet.com/2008/09/22/without-further-ado-reverie/

<comment><USER2> As a 5d mk1 owner I've started making my lunch every day versus eating out, just so I can buy this the day it comes out, without too much guilt. It has a ton of cool features, and video didn't really rate too highly with me.

</comment><comment><USER3> It should be noted that well over $10k in additional lenses were used to shoot that video.

<comment><USER4> Note a typical film maker probably only needs 1 or 2 lenses. They probably went overboard and used 8 or 10 different lenses -- but you can get 1 good lens for about $1000.

<comment><USER2> Canon's 50mm/1.4 is under $500, and is an excellent choice for a beginner.

<comment><USER4> Good call. Heck, the 50mm f/1.8 at $80 runs circles around any video recording lens. =)

No autofocus is rough though.</comment></comment></comment></comment>


Ask HN: Environmental Cost of Semiconductors? - <AUTHOR> Is there a recent review of the environmental cost of semiconductor electronics covering the full life-cycle from resource acquisition (including mining footprints) through manufacture (including release of greenhouse gases) through disposal or recycling? <comment><USER2> You are asking for an LCA of electronic components. The best ones are proprietary, such as the one we put together at Apple ten years ago, which is probably still updated and unrivaled.

The good news is that electronic components often obey commodity pricing. And for commodities there is a very good approximation for LCA: their market price.

An LCA is a sum environmental impacts, weighted by importance. Market price represents some sort of consensus weighting. You can argue for other weightings, but such arguments are at least as subjective as the market weights and seldom give a significantly different result.

Interestingly, even non-commodities reveal their environmental impact through their price. The difference is, their prices include all downstream environmental impacts, whereas traditional LCA and commodity prices only reflect impacts within some "system boundary".

<comment><USER3> LCA = Life cycle... assessment?

<comment><USER2> Usually (sometimes Analysis)

</comment></comment></comment><comment><USER4> Toxic materials (photoresist and associated solvents, acids for cleans, dopant chemicals, among others) and lots of electricity get used in the semiconductor manufacturing process. And that's just at the wafer manufacturing site. It doesn't include materials for packaging or PCBs. Nor does it include the chemical plants that supply materials, or the environmental impact of producing the manufacturing equipment (which will only be used for a decade, 15 years at the outside), and its consumables. If you're concerned with the environmental footprint of your personal computing, you should probably use older equipment for longer. I think the balance is different for datacenters given their high utilization.

</comment><comment><USER5> This would be a fascinating read if one exists, however, if I had to guess with no actual basis for it, what you describe is too broad and would need to be broken down. A few simplifications:

* The environmental impact of mining, refining and shipping of the ultra pure silicon needed for semiconductor development.

* The energy required to research/development then produce next generation semiconducting and at what yield or adoption rate would it make sense to push Moore's Law.

* Percentage of electronics that end up in landfill verse what is recycled.

I am having trouble trying to think of other clear examples beyond the fab due to too many variables. How do distinguish processors and GPUs in someone's gaming rig using power from a local coal plant to a cloud farm that is using wind and solar to power it? To throw another wrench into this there is also you[https://en.wikipedia.org/wiki/Jevons_paradox](https://en.wikipedia.org/wiki/Jevons_paradox).

</comment><comment><USER6> Call it anecdata, but a few of the things I noticed after a couple decades in the industry:

1\. Water

Water is extensively used and it has to be as pure as you can imagine. This requires a regular investment in filters and energy. I wouldn't be surprised to see hundreds of water wash steps for a wafer of chips, plus water wash steps for the equipment.

2\. Acids and bases

You can dump these in the third world, sure, but neutralizing them tends to be favored, and also very expensive. Sulfuric or Hydrofluoric acid may require almost zero energy input, but the caustic soda, soda ash, or baking soda you need to add to passivate their waste requires a lot.

3\. Silicon itself

This one's a bit of a surprise. A gram of hyperpure silicon doesn't really take that much energy, or even generate all that much waste. There's the reduction cost where you burn out the oxide with coal and wood chips, plus the follow-up stages of purification and crystallization. You lose half again by wafering. But even if it's 10 grams waste per gram of chip, you're not doing all that poorly. An idling car engine makes 10 grams of waste in no time. Plus the waste is almost entirely mg-Si, CO, and CO2; very manageable. China has a bit of a silicon-tetrachloride problem from some of its purification steps, but that stuff is avoidable.

4\. Photoresists

These are the absolute worst. Carcinogenic and complicated to manufacture.

5\. Specialty gases

What's a little diborane, phosphine, or arsine among friends? These players are bad news, but you use such trivially small quantities of them that their impact is pretty light. You still have to scrub their waste gases out before you release them, and that's a bit of a pain.

6\. Bulk gases

Hyperpure nitrogen, oxygen, and argon get used in large amounts as well but they're almost as cheap as air in bulk. Release at will.

7\. Air handling

Cleanrooms require more airflow than some bitcoin farms, and specialty filters have to be changed out occasionally. It's just electricity though.

8\. Packaging

All that copper, tin, silver, and gold, plus whatever weird plastics they're using these days may end up being far worse for the environment than the chips they encase.

9\. Actual use

Unless they're generating power (PV) they're going to suck down power, whether in your cable box set to off or the switchgear. This is by far the biggest energy consumption associated with semiconductors.

10\. Vacuum!

Along with to a lesser extent compressed air, vacuum greases and machinery can get up there in terms of cost. Roughing pumps will spew oil, and turbo pump and cryopump units cost a lot to build, run, and maintain.

11\. Organic solvents

Cheap to make, expensive to dispose, doubly so when you mix them with some of the other expensive-to-dispose stuff.

<comment><USER4> Great answer, and nice to see another person from semiconductor manufacturing on this site! Are you still in the industry?

<comment><USER6> Trying to be, but can't get people excited about making silicon wafers with higher purity, zero waste products and at 10% the cost.

So in the meantime I write apps.

<comment><USER4> > higher purity, zero waste products and at 10% the cost

That's disappointing! For an industry that's at the heart of so much modern technological advancement, it's surprisingly conservative. And that goes double for choosing suppliers, unfortunately. Myself, I got worn down by the hours and left the industry.

</comment></comment></comment></comment><comment><USER7> An independent review? I haven't seen one. There was a pretty egregious report alleged to be from the Silicon Valley Toxics Coalition[1] which appears to have been taken down. (It was conflating ground water contamination from fabs in the 60's with modern fabs, for example)

If you were looking for a project it would probably be a good one, I for one would be interested in reading something that was backed up by solid research and objectivity.

[1] [http://svtc.org/resources/reports/](http://svtc.org/resources/reports/)

</comment><comment><USER8> A nice map view of how polluted Silicon Valley is [https://whatsdown.terradex.com/#10/37.3489/-121.9228/](https://whatsdown.terradex.com/#10/37.3489/-121.9228/)

</comment><comment><USER9> Follow the law. We need to have Federal laws that require Electronics Companies or Importers to recycle their equipment. Those Federal laws do not exist today.

In the USA, corporations fight right to repair laws. That's the current state of the affairs. It has to transition from that into a more thoughtful usage driven buying/selling process.

There is nothing wrong in buying newer, better electronics - but junking the old ones as e-waste is absolutely wrong.

</comment><comment><USER10> I'm not sure if there is or not, but I would hope that any review would also include the reduction of power consumption from using semiconductors instead other non solid state devices. Eg Vacuum tubes vs transistors. Also the increased efficiency of switching power supplies over linear.

</comment><comment><USER11> yes, there are .. contact BAN-Waste in Seattle to start if you are in a hurry, and given 48 hours or so, more doc references here in this thread TBD. A keyword search with "Industrial Ecology" from the early 2000s might get things rolling, too.</comment>


Pure, functional JavaScript - <AUTHOR> http://cjohansen.no/talks/2012/sdc-functional/#1

<comment><USER2> I enjoyed these slides quite a bit, are there more resources around for learning to write this kind of JavaScript?</comment>


When is the next Y Combinator Program? - <AUTHOR>

Hey guys,
I would like to know when YC is gonna open their shop for another set of startups ?? <comment><USER2> Usually the application cycles open in September and February with deadlines in late October and late March, and the program starts at the beginning of June & January. There's some variation each year; if you Google "YCombinator application deadlines" you can see how they've shifted in the past.</comment>


Coinbase is erratically overcharging some users and emptying their bank accounts - <AUTHOR> https://www.theverge.com/2018/2/15/17017374/coinbase-cryptocurrency-bitcoin-ether-unauthorized-charges <comment><USER2> Flash message currently on Coinbase Support website[1]:

> Partial System Outage: We're investigating an issue where some customers > recently were charged incorrectly for purchases of digital currency with > credit and debit cards. This is related to the recent MCC code change by the > card networks and card issuers charging additional fees. We have identified > a solution and any future purchases will not be affected by this issue. We > will ensure any customer affected by this issue is fully refunded. We expect > this to happen for customers automatically through their bank. If you > believe, you were affected by this issue. Please contact support at > support.coinbase.com.

[1] [https://support.coinbase.com/](https://support.coinbase.com/)

</comment><comment><USER3> Earlier discussion here: [https://news.ycombinator.com/item?id=16386774](https://news.ycombinator.com/item?id=16386774)</comment>


Ask PG: One submission per day? - <AUTHOR>

Re: http://news.ycombinator.com/item?id=903841 (and similar discussions about signal-to-noise)
What if HN limited submissions to one per day?
Wouldn't this encourage more quality submissions and discourage the "hurl everything at the wall and see what sticks" approach?
One question would be a case where someone has a few good stories to submit. In that case (if they're really that good) I imagine one of two things happening:
- others submit them and have the opportunity to participate on the site (wouldn't broader participation be a good thing?) - nobody submits them and they can be added the next day
The basic idea is that you have one shot per day, so make it a good one: helpful, interesting, meaningful, etc. <comment><USER2> The real issue though is that nickb stopped posting. That guy was pumping out more "quality content" than the next 10 submitters combined.

I have no idea how to fix it, but that's your problem; even with a lot of noise, nickb's posts kept the front page clean and interesting. (It still is, just not to the same level as before)

Perhaps some sort of submission allotment per karma? 1 post default, extra posts for every 1000 over karma? You could also factor in how the posts/person performs...i.e. if your last post doesn't earn to enough points, it counts for 2 submissions, etc.

Conversely, you could make the first post use all your available post points, and then as it performs and gets voted up, you earn them back. That way you would post your better stuff first and leave the poorer stuff to the end, but you still get a chance to submit it.

</comment><comment><USER3> I understand the point here, but I think the quality would go down. I actively want to share great articles on here. If I were limited to only 1 thing a day, there are a lot of things that just would never end up here. You may be able to say: "oh, but someone else will find it." For really obscure articles, this probably isn't true.

Apply this to those under a certain Karma threshold and you _might_ be on to something.

</comment><comment><USER4> Bad idea. Some days I see several things I think are worth posting (most apparently don't agree, only some get much karma), most days I don't see anything that hasn't already been posted. Maybe have an initial karma hit when you post something, say "pay" 5 points to post, if many people like it, you'll get it back.</comment>


Ask HN: Open source OCR library? - <AUTHOR> Does anyone have a recommendation for a good character recognition library? I'm trying to pull text out of images. I tried ocrad.js but haven't had much success with it. <comment><USER2> As others pointed out, Tesseract with OpenCV (for identifying and cropping the text region) is quite effective. On top of that, Tesseract is fully trainable with custom fonts.

In our use case, we've mostly had to deal with handwritten text and that's where none of them really did well. Your next best bet would be to use HoG(Histogram of oriented gradients) along with SVMs. OpenCV has really good implementations of both.

Even then, we've had to write extra heuristics to disambiguate between 2 and z and s and 5 etc. That was too much work and a lot of if-else. We're currently putting in our efforts on CNNs(Convolutional Neural Networks). As a start, you can look at Torch or Caffe.

<comment><USER3> In my experience, CNNs offer the best performance. It is also easy to treat as a black box with many tunable parameters.

But the existing frameworks are mostly _bad_ as an engineering product. Caffe, for example, calls `exit` every time an error occurs, including recoverable ones like files which do not exist.

<comment><USER2> Yes, even we found Caffe to be overwhelming. This library seems promising considering there are no dependencies and is quite portable to Android or iOS. [https://github.com/nyanp/tiny-cnn](https://github.com/nyanp/tiny-cnn)

</comment></comment></comment><comment><USER4> Does it have to be open-source? If free, but not trainable and restricted to Windows apps/phone is good enough, then I recommend the Microsoft OCR library. It gives you very, VERY good results out of the box. An excellent piece of work from Microsoft Research. To test it, see for example [https://ocr.a9t9.com/](https://ocr.a9t9.com/) which uses Microsoft OCR inside.

And for comparison, an OCR application with Tesseract inside: It has a dramatically lower text recognition rate: [http://blog.a9t9.com/p/free-ocr- windows.html](http://blog.a9t9.com/p/free-ocr-windows.html)

(Disclaimer: both links are my little open-source side projects)

<comment><USER5> Are you talking about this library from Microsoft: [https://www.nuget.org/packages/Microsoft.Windows.Ocr/](https://www.nuget.org/packages/Microsoft.Windows.Ocr/) ?

<comment><USER4> Yes

</comment></comment></comment><comment><USER6> I tried using Tesseract and could not get it to work reliably. I tried a bunch of different pre-processing techniques and it turned into a very frustrating experience.

When I compared Tesseract to Abbyy, the difference was night and day. Abbyy straight out of the box got me 80%-90% accuracy of my text. Tesseract got around 75% at best with several layers deep of image pre-processing.

I know you said open source, and just wanted to say, I went down that path too and discovered in my case, proprietary software really was worth the price.

<comment><USER7> Finereader engine Abbyy for Linux looks fine, but I can't find price data, is it <10k/year? One time license?

<comment><USER8> Abbyy, and most good commercial OCR, charge a runtime royalty based on the number of cores you are deploying to. Depends on volume.

<comment><USER7> Can you point me to some prices, per core / core-year?

<comment><USER9> This is from years ago, but it was $4,900 for an SDK license for development and testing, and then the runtime licenses that you actually deploy are negotiable depending on pages/month, cores, languages, and other options. I think if you're just buying 1 runtime license they try to make it so it is around $5K so the total deal size is $10K. But they are negotiable. Maybe mention if you're a startup.

They also sold Recognition Server which has fewer options to integrate with programmatically (it was only 'hot folders' at the time), and I think only runs on Windows, but costs less.

And their mobile OCR which is lightweight, designed to run on smartphones, they worked out deals where they get a percentage of revenue.

</comment></comment></comment></comment></comment><comment><USER10> I was actually looking in to Tesseract yesterday, so this post coincides nicely.

I have a hobby project where I scrape instagram photos, and I actually only want to end up with photos with actual people in them. There are a lot of images being posted with motivational texts etc that I want to automatically filter out.

So far I've already built a binary that scans images and spits out the dominant color percentage, if 1 color is over a certain percentage (so black background white text for example), I can be pretty sure it's not something I want to keep.

I've also tried OpenCV with facial recognition but I had a lot of false positives with faces being recognized in text and random looking objects, and I've tried out 4 of the haarcascades, all with different, but not 'perfect' end results.

OCR was my next step to check out, maybe I can combine all the steps to get something nice. I was getting weird texts back from images with no text, so the pre-processing hints in this thread are gold and I can't wait to check those out.

This thread is giving me so much ideas and actual names of algorithms to check out, I love it. But I would really appreciate it if anyone else has more thoughts about how to filter out images that do not contain people :-)

<comment><USER11> My solution with regards to bad facial detection in OpenCV is to do the following:

1\. Use an LBP cascade on the picture. This is lower quality, higher false positives. Uses integer math so this is fast. Its named lbpcascade_frontalface.xml

2\. Capture the regions of interest that the LBP cascade identifies with a face and throw into a vector<Mat>. This means you can capture (potentially) arbitrary amount of faces. Of course, with OpenCV you are limited to a minimum of 28px by 28px minimum face.

3\. Run the haar cascade for eye detection on the ROI's you saved in the vector. Ones that return eyes show a good match. Haar cascades are slower(because they use floats), but the reduction in pixels means its relatively fast. Its named haarcascade_eye_tree_eyeglasses.xml

I can maintain 20fps with this setup at 800x600 on a slow computer.

</comment></comment><comment><USER12> Tesseract is ok, but I gather that a lot of the good work in the last few years on it has remained closed source within Google.

If you want to do text extraction, look at things like Stroke Width Transform to extract regions of text before passing them to Tesseract.

<comment><USER13> Is it possible to extract text from pre-formatted documents? Let's say I have a document issued by the government, and I am only interested in the fields that have been filled. Could I process such a document fully automated using Tesseract? Maybe some image pre-processing would be needed?

<comment><USER14> If you'd be interested in using something like this as a paid API/service and have decent volume, feel free to contact me. I'm currently in pre-beta for API rollout of such a thing.

In summary, you need templates that map the field positions -> meaningful keys so that you can get back useful data as json/csv/xml. I have some tools that are still being polished that automate much of the template creation and do a lot of the pre-proc for you.

email is my username (at) gmail

</comment></comment></comment><comment> j-pb Not really. Everything out there is ancient and uses techniques from the 80s/90s.

It's pretty sad considering that OCR is basically a solved problem. We have neural nets that are capable of extracting entities in images and bigdata systems that can play jeopardy. But no one has used that tech for OCR and put it out there.

<comment><USER15> It's because the kind of massive training sets that are required for accurate speach/text recognition are difficult and time-consuming to amass. It's just too difficult for hobbiests at the moment.

It's kind of the same reason there is no good open source solid modelling software (like Solidworks or NX). The problem has been "solved" for years, but actually doing it is too mammoth a task for open-source software.

<comment><USER16> [OT] What about FreeCAD? It looks like good open source solid modelling software: [http://freecadweb.org/](http://freecadweb.org/)

<comment><USER11> FreeCAD is starting to get good, but the problem is it lacks core functionality that Solid works already has built in. Admittedly, the underlying python calls are exposed, and you can use them... if you know how to make the appropriate calls.

When they get good GUI hooked up to the calls, it will be in parity with the pay solutions.

</comment></comment></comment></comment><comment><USER17> [http://www.danvk.org/2015/01/11/training-an-ocropus-ocr- mode...](http://www.danvk.org/2015/01/11/training-an-ocropus-ocr-model.html)

<comment><USER18> Yep. Back when I was at NYPL Labs[0], we developed a bit of an obsession with OCR. It turns out that so much of it depends on the context of the documents, original scan quality (unpaper and deskewing be damned), typography, how much handwriting you're willing to suffer (can someone please pick up the work of applying Convnets to large handwriting corpora that aren't just MNIST?).

A lot of this comes down to how much time you're willing to put into it to get it up and running and if you're willing to put in any effort to additionally train/refine a model.

(ps all the elements I'm about to mention here have been mentioned throughout this thread. I'm just providing a bit more context and bringing what I know together).

_________________

Gotta have something now: Tesseract

Not saying it's "bad" but it ain't as great as you want. Bindings for every language, lots of configs, documented well enough that it's 80% of the suggestions here. For what it is, it's a damn miracle. Most languages and alphabets and lots of kinds of fonts likely start giving you results somewhere between 80% and the Uncanny Valley immediately. You also get a pretty good Layout Analysis engine [1] for if you're working with complete pages of text. The existing models it ships with are robust, but if you want to get better outputs, retraining it is a real pain (you have to segment on a character-by- character basis). You're better off trying to apply a general preprocessing to clean the image or...

__________________

Gotta have a proof of concept this afternoon: Send just text to Tesseract

By now you've realized that asking an OCR engine to OCR a tree (or a picture of a tree alongside some text) somehow always comes back looking like a cat just napped on your keyboard. As alluded to in a few other places here, Just Give It Text! Tesseract (and most traditional OCR) was developed in a text document centric world where you could safely assume it's just a page of words, and sometimes you might have to deal with columns (that's where the Layout Analysis comes in). Probably not your real world.

Depending on your type of documents you might be able to develop a few hueristics for identifying text regions in the picture, then sending only those sections over to Tesseract. This'll dramaticlly help Tesseract out, but it can increase some of the complexity you'll have to juggle. You might be able to come up with some hueristics specific to your documents (which can be very good, especially if it lets you infer more information about those regions that you might want later).

You can also use something like Stroke-Width Transform (all the links I would use were graciously linked in this earlier comment [2]) which was discovered by Microsoft trying to spot text in the wild for their Street View efforts. ccv has a very nice SWT implementation [3], and their http server for the whole library [4] that with a bit of makefile finagleing can have a very nice SWT preprocessor -> Tesseract api over HTTP up in an hour or so.

Also it looks like the OpenOCR[5] project is now using SWT -> Tesseract with a nice HTTP API written in Go, conveniently packaged up for Docker and very well documented[6].

_____________________

Take me down the rabbit hole, but maybe get results in 2 days:

The roots of tesseract were planted by HP in 1985, so there had to be a better way at some point. OCRopus[7] was supposed to be the great state-of-the-art hope that would save us all. The approach was incredible and the results being published were great. But the documentation came in the form of a mind map[8]. Recently the project was picked up again by the original developer and rechristened OCRopy[9] and has garnered a pretty active and growing community in the past few months.

You'll have to do a bit more work here than just tesseract, but the LSTM neural network[10] approach completely blows away Tesseract's results with just a little training.

To get started, Dan Vanderkam's tutorial is excellent to start working with the out-of-the-box model[11] immediately.

But results get INCREDIBLE when you take some time to train your own model[12]. I provided Dan with the source images for the project[13], from The New York Public Library's historical collections, and the source text was in a font style the default model had never encountered before. But in an hour or two of transcribing the training data (training data in OCRopus is awesome because you just feed in a line at a time; no need to align and segment each letter like in Tesseract), he was getting an under 1% error rate!

Layout analysis isn't OCRopy's strongest suit, so you might get even better results if you pre-segment with something like SWT, but again, not completely necessary.

____________

That's it. Way too much for a Newsy Combinator comment, but a pretty decent tour of the big stuff in open ocr world. There's some magic state of the art going on inside Google, Microsoft, and Abbyy, but hopefully you'll be able to teach your computer to appreciate whatever it is you want it to read through these now.

[0]: [http://nypl.org/labs](http://nypl.org/labs)

[1]: [https://en.wikipedia.org/wiki/Document_layout_analysis](https://en.wikipedia.org/wiki/Document_layout_analysis)

[2]: [https://news.ycombinator.com/item?id=9776034](https://news.ycombinator.com/item?id=9776034)

[3]: [http://libccv.org/doc/doc-swt/](http://libccv.org/doc/doc-swt/) and [http://libccv.org/lib/ccv-swt/](http://libccv.org/lib/ccv-swt/)

[4]: [http://libccv.org/doc/doc-http/](http://libccv.org/doc/doc-http/) and [http://libccv.org/lib/ccv-serve/](http://libccv.org/lib/ccv-serve/)

[5]: [https://github.com/tleyden/open-ocr](https://github.com/tleyden/open- ocr)

[6]: [https://github.com/tleyden/open-ocr/wiki/Stroke-Width- Transf...](https://github.com/tleyden/open-ocr/wiki/Stroke-Width-Transform)

[7]: [https://en.wikipedia.org/wiki/OCRopus](https://en.wikipedia.org/wiki/OCRopus)

[8]: [https://www.mindmeister.com/192257150/ocropus-overview- publi...](https://www.mindmeister.com/192257150/ocropus-overview-published)

[9]: [https://github.com/tmbdev/ocropy](https://github.com/tmbdev/ocropy) | all day all week

[10]: [https://en.wikipedia.org/wiki/Long_short_term_memory](https://en.wikipedia.org/wiki/Long_short_term_memory)

[11]: [http://www.danvk.org/2015/01/09/extracting-text-from-an- imag...](http://www.danvk.org/2015/01/09/extracting-text-from-an-image-using- ocropus.html) | though you've got to download the model separately because it's too big for github

[12]: [http://www.danvk.org/2015/01/11/training-an-ocropus-ocr- mode...](http://www.danvk.org/2015/01/11/training-an-ocropus-ocr-model.html) [13]: [https://www.oldnyc.org/](https://www.oldnyc.org/)

<comment><USER19> Great list of links. You have done the research! Personally, I have used ocropy to great effect. Right now, I am moving beyond forms at work and researching cartographic map vectorization using deep learning techniques. There is little science papers on the subject however. Have you done any research on that by any chance?

<comment><USER18> Indeed! Map vectorization is one the things NYPL Labs is actively working on: [https://github.com/nypl/map-vectorizer](https://github.com/nypl/map- vectorizer)

It's designed to work primarily with old fire insurance atlases (e.g. Sanborns) and is a bunch of hand-tuned heuristics. But it powers Building Inspector ([http://buildinginspector.nypl.org](http://buildinginspector.nypl.org)), which is where all the data is validated consensus crowdsourcing, providing ground truth for the data coming out.

Unfortunately (or perhaps fortunately if you're trying to get a Computer Science PhD) I haven't come across anyone applying deep learning to map vectorization either. Frankly, it could open up a whole new field with respect to historical mapping (among other things).

Would love to talk more about it and see how deep learning could be applied here. We always structured the outputs of Building Inspector so they'd be useful training sets for unsupervised or reinforcement learning so it could hopefully apply here as well.

</comment></comment></comment></comment><comment><USER20> The accuracy of GOCR is very high and it doesn't require any learning.

[http://manpages.ubuntu.com/manpages/dapper/man1/gocr.1.html](http://manpages.ubuntu.com/manpages/dapper/man1/gocr.1.html)

<comment><USER21> GOCR is ok, certainly much better than GNU Ocrad, but I've had much better results with Tesseract.

<comment><USER22> GOCR is simple enough to install from your distro's repository and wrap into a script without fuss though. Tesseract is wildly more complicated to get started with.

If you're doing real time street sign recognition or involved with a book scanning and archival startup, investigate Tesseract for sure. But even then you'll probably want to prototype with gocr first.

</comment></comment></comment><comment><USER23> (Disclaimer: I work for creatale GmbH)

As you mentioned ocrad.js I assume you search for something in js/nodejs. Many others already recommended tesseract and OpenCV. We have built a library around tesseract for character recognition and OpenCV (and other libraries) for preprocessing, all for node.js/io.js: [https://github.com/creatale/node- dv](https://github.com/creatale/node-dv)

If you have to recognize forms or other structured images, we also created a higher-level library: [https://github.com/creatale/node- fv](https://github.com/creatale/node-fv)

<comment><USER24> I can attest to this. I used Node-DV in a production environment and it's pretty damn amazing with all its built-in functionality.

There are options to adjust the image in various ways and once Tesseract runs, it's easy to get the result in various formats.

</comment></comment><comment><USER25> I've used tesseract to great affect. I don't know how your images are but if only part of the image has text in it, you should only send that part to the OCR engine. If you send the entire image and only a portion of it has text in it, chances of the OCR extracting text are slim. There are pre-processing techniques [1] you can use to crop out the part of the image that has text

[1]: [https://en.wikipedia.org/?title=Hough_transform](https://en.wikipedia.org/?title=Hough_transform)

<comment><USER26> Could you explain how to use a Hough transform to find areas that have text? I only recently managed to wrap my head around how it works for line detection and other shapes with the generalized form, but how would one recognize text?

</comment></comment><comment><USER27> [https://code.google.com/p/tesseract- ocr/](https://code.google.com/p/tesseract-ocr/) is pretty good

<comment><USER28> Tesseract does no layout analysis.

So if the source image contains text columns or pull quotes or similar, the output text will just be each row of text, from the far left to the far right.

<comment><USER29> Could you add another layer on top (eg. image processing) to detect boundaries of blocks of content, and send each block to Tesseract in sequence?

<comment><USER11> Yeah, you could do that. And I wouldn't think it would be that hard either.

I'd make a cascade that detects all letters and numbers from major font sets. That shouldn't be too terribly difficult.

Now, use the cascade to scan the document. Now, convert the document to a list of all detected characters (we don't actually care what the chars are).

Once you have this, do best fit bounding boxes around the data. You'll have to figure out what distance you want to exclude from the bounding boxes.

Now what you should end up with are a few boxes indicating the regions of data on the document. Now, crop each of these regions of interest and feed them into Tesseract.

</comment></comment></comment></comment><comment><USER30> Been looking at Number Plate Recognition and [https://github.com/openalpr/openalpr](https://github.com/openalpr/openalpr) has been my go to solution at the moment. It uses Tesseract and OpenCV.

</comment><comment><USER31> Judging by the comments there is much space for improvement on open source OCR libraries, maybe someone with experience in the field should start a kickstarter for it, there must some here in HN itself; maybe something with CLI and GUI and layout recognition.

</comment><comment><USER32> I used Tesseract and OpenCV to process Words With Friends tiles.

<comment><USER33> Haha, this is awesome. Have you written about your program anywhere? Would love to hear more about it.

<comment><USER32> It's pretty much as other people talked about, I used OpenCV to try to detect the board, and then isolate each tile, and then used Tesseract to try to figure out the letter. The biggest problem I had was whenever the red score bubble landed on a tile, it would fall on top of the latter and thus lose information so I ended up having to create training data and associating that to get the letter. I was able to get to about 98% accuracy and it was relatively robust until they created a new app that I haven't had time to adjust for.

</comment></comment></comment><comment><USER34> I've tried using Tesseract before, the biggest of the open source libraries. It goes from "okay" to "terrible" depending on the application.

Our particular application was OCRing brick and mortar store receipts directly from emulated printer feeds (imagine printing straight to PDF). We found that Tesseract had too many built-in goodies for image scanning, like warped characters, lighting and shadow defects, and photographic artifacts. When applied directly to presumably 1 to 1 character recognition, it failed miserably.

We found that building our own software to recognize the characters on a 1 to 1 basis produced much better results. See: [http://stackoverflow.com/questions/9413216/simple-digit- reco...](http://stackoverflow.com/questions/9413216/simple-digit-recognition- ocr-in-opencv-python)

</comment><comment><USER35> A Guide on OCR with tesseract 3.03:

[http://www.joyofdata.de/blog/a-guide-on-ocr-with- tesseract-3...](http://www.joyofdata.de/blog/a-guide-on-ocr-with-tesseract-3-) 03/

</comment><comment><USER36> It depends on what you're trying to do. For myself, I wanted to OCR scanned documents and I've been moderately successful using ScanTailor to process the images and then Tesseract to OCR the result. Certainly, it's far from perfect and the documentation on Tesseract and its options is spotty, but I've been moderately happy. As a note, I've had better luck with the current trunk in the git repo of Tesseract than the point releases. Some of the older versions of Tesseract modified the images when processing to a PDF and this was unacceptable to me.

</comment><comment><USER37> I used the go wrappers for tesseract[0] in my full text search indexer for disks[1] and it worked great. No issues it handled pretty much anything I threw at it.

With the caveat that none of the stuff was handwritten.

[0] [http://godoc.org/gopkg.in/GeertJohan/go.tesseract.v1](http://godoc.org/gopkg.in/GeertJohan/go.tesseract.v1) [1] [https://bitbucket.org/zaphar/goin/](https://bitbucket.org/zaphar/goin/)

</comment><comment><USER38> you may try baidu OCR API [https://github.com/JeremyWei/baidu- ocr](https://github.com/JeremyWei/baidu-ocr)

<comment><USER39> Is that trainable for new alphabets? API use policy?

</comment></comment><comment><USER40> Tried tesseract, it worked very poorly. I don't think there is anything that compares well to Abbyy.

Unfortunately the native Linux version is a bit pricey: [http://www.ocr4linux.com/en:pricing](http://www.ocr4linux.com/en:pricing)

Otherwise I would use the command line version to help me index all my data.

</comment><comment><USER41> An important aspect is stripping text from image, before OCR'ing: [http://sourceforge.net/projects/tirg/](http://sourceforge.net/projects/tirg/)

Try TIRG.

Examples here: [http://funkybee.narod.ru/](http://funkybee.narod.ru/)

</comment><comment><USER42> I'm working on these problems for applications in the financial industry right now. We are building some interesting technologies and have quite a bit of funding from top VCs. We are looking to hire people who know this problem cold. Let me know if you're curious. nate@qualia-app com

</comment><comment><USER43> [http://projectnaptha.com/](http://projectnaptha.com/)

[https://github.com/naptha](https://github.com/naptha)

Apologies, I am not sure if its open source.

<comment><USER44> It seems this is just built on top of Tesseract as well.

</comment></comment><comment><USER45> We've had great success with CCV's SWT to indentify text zones and Tesseract to extract it.

[1] [http://libccv.org/doc/doc-swt/](http://libccv.org/doc/doc-swt/)

</comment><comment><USER46> Tesseract [https://github.com/tesseract- ocr/tesseract](https://github.com/tesseract-ocr/tesseract)

</comment><comment><USER47> A coworker had great success using Ocropus (which uses neural nets under the covers) extracting test from the backs of many photos for Old New York.

</comment><comment><USER39> Tesseract with ccv for SWT works but is a bit of a slow dog. Anyone used deep learning (caffe maybe?) for OCR?

<comment><USER39> Thought I'd reply on my own comment.

Caffe has been used for handwriting, seems like OCR of typefaces would work just the same with a typeface dataset.

[https://github.com/BVLC/caffe/tree/dev/examples/mnist](https://github.com/BVLC/caffe/tree/dev/examples/mnist)

Here is the only other OCR example with caffe i could find:

[https://github.com/pannous/caffe-ocr](https://github.com/pannous/caffe-ocr)

</comment></comment><comment><USER48> Any references on OCR to extract MICR (font E13B) for desktop as well as Android?

<comment><USER49> Dunno about desktop, but as far as Android is concerned, tess-two is a fork of Tesseract tools for Android: [https://github.com/rmtheis/tess- two](https://github.com/rmtheis/tess-two)

This dude did a pretty nice writeup on getting it going: [http://gaut.am/making-an-ocr-android-app-using- tesseract/](http://gaut.am/making-an-ocr-android-app-using-tesseract/)

</comment></comment><comment><USER50> apache tika + ocr is an excellent combination - [https://wiki.apache.org/tika/TikaOCR](https://wiki.apache.org/tika/TikaOCR)

</comment><comment><USER51> Few years ago I used Cuneiform, but looks like it's dead now.</comment>