Jack Ma's theory of how America went wrong over the past 30 years - <AUTHOR> http://www.businessinsider.com/alibaba-jack-ma-davos-america-history-globalization-2017-1 <comment><USER2> The Wall Street comments are a bit vague/nebulous, but the amount we've spent on unnecessary war-mongering even after Vietnam is a relatively obvious problem that apparently is no longer obvious, now that we lost our "anti-war" party (the formerly anti-war Democrats).

Our bureaucratic programs at home are enough to dig us into a deep financial hole (war-mongering wasn't the source of Greece's problems), but with the added trillions wasted on building old weapons to perpetually blow up and rebuild bridges (not to mention the magnitude of the atrocities committed by the US), we are truly buried financially.

He probably doesn't even know about the CIA's less publicized warfare in Central/South America and the financial/other consequences of the drug war on our own citizens and especially on our neighbors.

<comment><USER3> ..but but without frequent wars how are we going to run a welfare state for big corporations that produces artifacts with no natural demand. How are we supposed to channel tax payer's money to them so that they can share some of them back by employing people

<comment><USER4> Ethanol credits, duh.

</comment></comment></comment><comment><USER5> He forgot celebrity worshipfulness and the consumer culture.

The machine that produces celebrity/consumerism today is a hyper efficient over optimized machine.

He is basically saying people didn't get what they need.

And that's because the celeb/consumerism machine's only job is to keep giving people what they want.

What they need has nothing to do with it. Whether its Obama or Trump, Zuckerberg or Musk, subprime mortgages or iphones it's all about what people want and not need.

<comment><USER6> The advertising industry (which includes general media) have a lot to answer for there. They've been "programming" 2 to 3 generations now to just buy stuff and pay attention to people making the most noise for attention.

Not having had a TV in the house for over 8 years now, I was reintroduced to the world of free to air TV during our Christmas holidays when we went away for a month and stayed with friends who were avid TV watchers.

Been that long since I had seen a TV advertisement, and I was astounded at the how banal and low brow most advertising is these days. I seem to remember a time when ads were clever and creative, but the ones I kept seeing with monotonous regularity were nothing more than quick sound bites ordering people to go buy junk or watch more crap. Everything was really disjointed and jarring to me - I actually came across a couple of shows that I wanted to watch, but they never seemed to get started - every few minutes there was an interruption telling me what else I "couldn't miss" watching, which are just more shows that will in turn have interruptions to watch other shows. Oddly, my friends who owned the TV didn't seem to see a problem.

Really makes me wonder if TV audiences these days are just conditioned to have such a short attention span and are used to being forced fed suggestions like this. Or perhaps they are so hardened to it that advertising companies have to just blast short messages to them with all the subtlety of a drone strike?

Time for humanity to take control back over their time and attention, otherwise it will just be "All glory to the Hypnotoad"...

<comment><USER5> I am with you. The attention economy has all kinds of unintended side effects that don't seem to have any great solutions.

And its not just advertising most entertainment/propaganda/marketing is designed to just get an instant reaction. No contemplation necessary. Kind of like porn.

And if we have to be honest hacker news is very similar. Nobody needs this much news.

<comment><USER6> You hit the nail on the head. The ads I saw didn't seem to elicit any thought or feedback or discussion. It was more like "Here - go do/watch this. I'll be back shortly to tell you again soon!". Almost like training a dog. Rote repetition and ever increasing sense of urgency and the fear of missing out.

A clear case of over promising and under delivering - and yet my friends seemed to be content with lapping that up rather than questioning it or making moves to reject it. It really is like those dystopian stories where humans are enslaved by the messages emanating from their TVs. Honestly, I felt like taking a shower after each TV watching session...

I've been home a month now, and not missing having a TV one iota!

</comment></comment></comment></comment><comment><USER7> This all may be true, but I think the more fundamental problem is the ignorance of the average American about how their government works. A former Supreme Court justice lays it out in this prescient video from 2012: [https://www.youtube.com/watch?v=rWcVtWennr0](https://www.youtube.com/watch?v=rWcVtWennr0). People are frustrated with their government but don't understand how to fix it.

<comment><USER8> The argument he's putting forth is so incredibly vapid and simple-minded that it's a stunning indictment of the lack of vision of a supreme court judge into the American zeitgeist.

Americans understand the basic concept of the vote. A freaking six year old understands it. There are leaders, and they sycophantically grovel for the favor of the public, which the public gives freely in exchange for the politician promising to give them their hearts' desire. And then the public may or may not get what they want. If the politician cannot appropriately explain the reasons for this, they are dumped and the next sycophant is wrung through.

That's about all they know, and all they need to know, because the people should not have to lobby their government in a democracy, the democratic leaders should already know what the people want and achieve it on their behalf, and in general do good for the people. But that is not what happens in America.

Here, our political leaders don't serve the people. And the people know that. But the people can't do anything about it. And the reason they can't do anything about it is the people voted their way into this mess. Trying to vote their way out will just result in exactly the same thing.

What the former Justice is saying is, the biggest problem in politics and public life is people don't understand how the legislative and judicial and executive processes work. But people do not need to know how this works, they merely need to voice their opinions and vote. And half of the people do vote. The other half don't vote either because they can't, or because they know what the result of voting is.

The American democracy in the 20th and 21st century is an artisanal machine which produces a specific output. Votes and labor go in, and money comes out. But the machine is designed to mostly spit out money which can only be used by corporations and rich people; only a small percentage of that comes out in a form usable by the people. The people know their votes are mostly meaningless, but it is their only tangible method left to express their discontent. So they support the system that produces an awful result because it's the only way they can express that the machine produces an awful result.

I think the people see this contradiction, and I think this is why so many do not vote: it's like complaining that your shoe hurts and then putting it on again to see how it will fit.

<comment><USER9> I'm not an American and I don't really understand why you say this. Only a small percentage that the machine produces comes out in a form usable by non- rich people? Then what's, for example, Obamacare, which seems to be a system that finally resembles how health insurance works in many European countries? And isn't that same Obamacare under fire by lots of people who think that the machine _shouldn 't_ support the people and that everybody needs to support themselves instead of taking a piece of tax money?

I'm more confused than before.

<comment><USER10> No, Obamacare only _pretends_ to resemble European-style healthcare. In fact, it merely forces everyone to sign up with _private_ health insurance companies. Which still take their _huge_ profits.

<comment><USER4> Yes, and private health insurance companies have profits so huge that they're pulling out of the insurance markets left and right and saying "sorry, we're just not interested in this business anymore!" Arizona's had it worst – many places there are on their last insurer – but that's just a taste of the phenomenon.

The premium hikes are paying for the treatment of the "uninsurable" and chronically ill. Even the profitable for-profit insurers are only running margins of 3%, maybe 4% in a good year. You could turn them all nonprofit and it wouldn't really help premiums. (And people tried! Look at the health-care co-op collapses in Colorado, Iowa, Kentucky, Louisiana, New York, Nevada and Tennessee, and Oregon, as well!)

<comment><USER10> OK, maybe Obamacare is melting down in some states. But national healthcare works in Europe, Canada, etc. So what's so special about the US?

<comment><USER4> If we want to compare apples to apples, national healthcare doesn't work in Europe. It works in _a variety of individual countries within Europe_ , some of which are imperiled. (And Canada has a population a tenth the size of the US.) Some of those countries have culturally homogenous well-educated populations smaller than the United States illegal immigrant population, so there's a demographic difference. That said...

Euro health care, writ large, works in the sense that "this is a substantial tax on our national incomes and possibly contributes to an economy which continues to suffer low growth rates and youth unemployment rates regularly in excess of 25% – and given the Euro crisis on top of that there may be some ugly cuts in the future, especially in places that aren't Germany." (Canada's health care at least has less of that problem.)

These national health care systems generally deal with cost control for the chronically ill and the elderly by rationing care through waiting lists for various operations... or, in the case of the Netherlands, by building a culture which pushes the elderly into assisted suicide.

(This is as opposed to the US who has historically dealt with them by making them pay for their own care, but has recently switched to dealing with them by hiking everyone's health care premiums to the point of unaffordability.)

Don't think I'm pitching solutions here!! I've just got buckets of cynicism for _everyone_!

<comment><USER11> > Some of those countries have culturally homogenous well-educated populations > smaller than the United States illegal immigrant population, so there's a > demographic difference.

Dogwhistle for "can't have nice things because blacks". Notwithstanding the relative homogeneity of their populations (the US is well down the list BTW), what difference should it make?

<comment><USER4> > Dogwhistle for "can't have nice things because blacks"

Eh. To constrain it deliberately to white people, think "established prosperous modern family, maybe WASPs or atheists, living in the Washington, D.C. suburbs with two stable incomes, commute on the Metro" vs, oh, "army veteran in a community in rural Appalachia which is reeling from the end of coal mining and wreaked by both prescription painkiller abuse and obesity, heavy smoker... local no-name church is the main community center ... considering a job driving a truck."

"Inner city African-American community" or not is but a small portion of this class of differences. Copenhagen vs Aeroskobing is a smaller gap than New York City vs Beards Fork, WV.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> This "brutal theory" is the banal obvious stuff you can find on Twitter for free in 5 minutes.

<comment><USER13> Mods should remove the word brutal from the title.

</comment></comment><comment><USER14> Jack Ma is very protective like China, if Americans / rest of the world was like that, chances were unlikely that China could grew so much.

Also, by any measures, China is cheating right now and trying to minimize the capital bubble risk created by the government ( loans that are not payed back with worthless warranties eg. Failed buildings, subsidized shipping, manipulated / protected stocks,... ). Let's see what the future brings.

PS. If downvoted, please explain why and if you have ever travelled to China :)

</comment><comment><USER15> It's always about the choice between short-term vs long-term benefits, or between us and them, or rich and poor. When people don't act as if other people (or even themselves, in the future) are connected to them, they make bad choices in the long term. On personal terms it's a battle between temptation and moral action.

It's basically a problem of strategy in games (here, the game being well- being). There are two main strategies: optimize for low level features (immediate benefit) or high-level features (what you should do in order to succeed). Companies battle the same choices, often very rich and successful ones optimizing for short term instead of long term, as if they expect to close shop in a few years.

The main purpose of politics should be to balance the focus between short and long term strategies, but short term relies on corruption and long term on consciousness, so it's an uphill battle. It's also called principle of "delayed gratification" \- few can handle the rigors of it.

With the new Trump administration we should prepare for a lot of short-term based choices that ignore the long-term consequences.

</comment><comment><USER16> Jack Ma: a guy whose success can only be attributed to the existence of China's Internet firewall. I try to avoid reading about his "insights".

<comment><USER17> He is not a neutral source of information, but his views might be interesting to read because of his unique bias. He has a perspective on the world which is very different from most other sources I read.

</comment></comment><comment><USER18> The idea that the US's economic problems are due to war expenditures is just plain silly.

The economic golden age for the US was during the cold war, when the country spent 6-10% of its gdp on the military. Then the cold war ended, and military expenditures have been below 4% most of the time, but the US economy has done worse. Oh, and China has boomed at the same time it has been engaging in a huge, expensive military modernization program.

I am not saying high military expenditures produce economic prosperity, just that they don't prevent it (unless they are much higher, as the Soviet Union's were).

</comment><comment><USER19> So... hindsight is 50/50\. Anyone can look back and say what someone shouldnt have done and be "correct" but other action would have resulted in a different unknown outcome.

It is cool that theres a dialogue about this at such a forum and spearheaded by a respected individual, but its nowhere near new information. Without an alternative its also just non-constructive criticism.

<comment><USER20> On the other hand, what he's describing is still the status quo.

</comment></comment><comment><USER21> Ma makes a good point, but he's implicitly saying the China's authoritarian regime wouldn't have lost the money as "free" market actors did in the US.

Sure, fine, but most Chinese would trade that money for American political freedoms and economic opportunities in a heartbeat.

<comment><USER22> I don't know if "free market actors" is exactly the right way to describe the two things he talks about. War is certainly not a function of the market, Wall Street is kind of a grey area that uses the language of free markets to describe something that is an odd state-empowered cartel. Clearly the bailout was not a "free market" event.

So...I don't think he is implicitly or explicitly saying anything about China's authoritarian regime. It wasn't free markets that lost the money he's speaking of. And, there wasn't a trade of that money for political freedoms and economic opportunities. The American people just got a bad deal out of their government and their financiers. They didn't buy freedom with that money, they just got fleeced.

</comment></comment><comment><USER23> I think it might be much more than $2 trillion the US spent on wars. Remember Sept 10th 2001? Donald Rumsfeld said on this day, they can't find out how they spent $2.3 trillion.

</comment><comment><USER24> What rubbish!

This is the thinking of someone that lives in a country with a philosophy that puts country before individuals.

By almost every and any measure, we are all better off than 30, 40 or 50 years ago - Chinese, Americans, Indians, Europeans, almost everyone. There IS a subset within those places that has benefited less, or even gone backwards, no doubt.

But even for those people, what is the economic value of the wonders of the modern, internet and mobile phone age? When for the cost of an internet connection, people have access to millions of free flash games? Is that a better life than 1970s Harlem? I'm not sure either way, but it is a lot closer run thing that life is worse for the "losers" of globalization than people realize, and that is the comparison for the small minority that have missed out in the modern world, not the mean or median experience.

The only way people are worse off is if they compare the best of 1960s living - e.g. have only the husband work, own a 4 bedroom house in the burbs, make a decent salary without a degree - and ignore the things missing that are present in the modern lifestyle. Because in the 1960/70/80s west, people ate only basic food (nothing "foreign" like Thai or Indian), very few had cable TV, and went out at most once a week to the movies. They never bought an espresso, never owned a modern electronic device like a computer, never got to use the internet to settle an argument or play a computer game, unless they were extremely well off and got to play Pong circa say 1980.

If that sounds like a life people prefer - a big house, no lifestyle, no electronics, basic cable if lucky and only white people food - then I reckon most people could live like that even today if they so chose. But man, that isn't a life I'd be happy with. Not even slightly.

If that is "losing" \- better lifestyle, better food, better (and cheaper) entertainment and slightly smaller living conditions and more people working, man, "winning" would have been insanely good.

<comment><USER25> We eat less fresh food, we are less healthy, we are less happy, we are less able to go out in the woods or to the beach, because the woods have been logged or otherwise privatised in such a way that they are no longer accessible . The products we own and buy are of lower quality, they break more quickly. But that lower quality has not caused their price to drop tremendously. There are fewer Americans with the capital to sustain themselves, aka fewer small business owners who are able to make a living without having to find employment from some larger firm. And that directly results in reduced freedom for everyone, because there is less choice both in the employment sector and in the sector of choosing who you buy from.

And the only thing you have to offer is shitty cat videos with 30 second advertizements before 2 minutes of user created content.

Oh, and your stupid GDP. You know what the GDP measures? It is value transfer economics. It measures how many times money passed back and forth between two parties. If there weren't taxes you and I could spend all day passing $20 bills back and forth and our collective GDP would be higher than that of Switzerland. Completely meaningless number.

<comment><USER26> People today eat less fresh food and are less healthy largely by choice.

People like to sit around, watch TV, play video games, and eat junk food. No one has to force us to do those things.

<comment><USER25> I dissagree. I know of a number of famillies who used to have a small business, grow their own veggitables, and sell some vegitables on the side. However, the price of food, and the simple services of that these people provided has dropped, due to mass production, and those famillies were forced by lack of money, to change their lifestyles. They had to either sell land or take up a full time job elsewhere which meant that they no longer had time to live the way they used to.

<comment><USER26> No doubt the economy has been hard for small farmers, but that's not why people aren't eating fresh food or exercising enough.

As you said, the price of food has dropped, and exercise is free.

<comment><USER25> The price of food has dropped, but the price of fresh food has risen.

If you live in LA, unless you have the time to go out to one of the farmers markets, its not even possible to buy fresh food. The "fresh" food in the stores isn't fresh. It is usually weeks or even months old. Sometimes it has been on a ship across the ocean stored in a special atmosphere, maybe it "looks fresh" but it isn't. I grew up in Seattle, things are a bit better there, but you still have to drive out of your way to find fresh food. The concept of fresh and the concept of supermarket are just incompatible.

<comment><USER26> But as you said, fresh food is available in farmer's markets. And if the price of fresh food has risen, it should be possible to make a living growing and selling it.

Except that the majority of people don't seek out fresh food.

<comment><USER25> It might be possible to make a living growing and selling it, but it is actually quite unrealistic that a sizable portion of the population could choose to start doing so. %100 of the land in the US is owned now. Its not like the fronteir days when anyone who wanted to start a farm could just go out west. If 1 million americans wanted to start a fresh foods farm within driving distance of a city, that wouldn't be possible. The price of land would skyrocket. And that would only be one third of one percent of all Americans. Not even a significant shift. What if 20 or 30 million Americans wanted to make that transition. Is that economically possible in todays world? What if the government were to give anyone who wanted to start a familly farm a million dollars to buy land. Would it be possible then? I don't think so. I think that it isn't about choice. I think that it is litterally impossible for our society to change without the fundamental ideas regarding property rights or urbanization changing.

<comment><USER26> You just said you know families who owned land who couldn't make a living growing and selling it anymore.

So it's not lack of land that stopped them.

<comment><USER25> "So it's not lack of land that stopped them." You're missing a sense of timescale here. For a long time the number of familly farms has been decreasing all across the country. Recently, it has become possible to make a living selling fresh food if you live within driving distance of a city where you can sell the food for a premium. The high cost of a fresh head of letuce in LA in 2016-17 doesn't help a small time farmer in rural Colorado in the 90s.

<comment><USER26> Still leaves the question of why people in the 90s weren't buying fresh food from the small farmers.

<comment><USER25> That's true. It is a good question. And its the question we should be asking. We should be trying to debug this problem, rather than pretending that it is a non-problem.

I personally don't see my own anti-capitalistic views as being a matter of politics or political direction. I see the problems posed as being similar to aging or the need to find better forms of transportation. No one argues that aging is good. Everyone agrees that the outcome of aging is bad, people get slower, less inteligent, suffer and die. That is bad... I see the current outcome of American globalist capitalism to be similar. Obviously, the outcome sucks. Most people are unhealthy, unhappy, there are advertizements everywhere, we are destroying the earth, and living in smelly toxic concrete filled cities. I'm not a communist, or a socialist, or a progressive, I'm an engineer who is baffled by a huge problem that I don't know how to solve. I am totally baffled by the huge number of people who seem to question whether these problems are real. Its like if this thread [https://news.ycombinator.com/item?id=13419352](https://news.ycombinator.com/item?id=13419352) was full of people writing that "well you can point to some problems with aging, but really, we're all better off getting old and dying". My response would be similar, it would be like WTF? I don't have a solution to American corporate globalism and the problems it causes, I don't have a solution to aging either, but I want a solution to both.

<comment><USER26> I'm not pretending it's a non-problem.

It's a much much bigger problem: human nature leads us to make bad choices, willingly.

But short of a miracle pill or a totalitarian regime that forces us all to live healthy lives, I don't see a solution.

<comment><USER25> You are also not the OP who I origionally responded to. thisnotmyacc wrote "By almost every and any measure, we are all better off than 30, 40 or 50 years ago - Chinese, Americans, Indians, Europeans, almost everyone." And that is pretending we have a non-problem and that our society is functional.

Edit:

I think that it is worse than just a problem with human nature. As I wrote, previously, I think it is economically non-possible for Americans to choose to go back to growing local food. %100 land is owned, and most owners are not selling, the price of land is high, and most Americans cannot afford to buy that land. Perhaps charging large farms land rent (aka, only charging land rent to large corporations) would help [https://en.wikipedia.org/wiki/Georgism](https://en.wikipedia.org/wiki/Georgism) but I'm not sure if I believe that democracy is capable of implementing any sort of reform non-corruptly.

<comment><USER26> I think we're talking about different things. You're talking about being self- sufficient small farmers; I'm just talking about eating healthy and exercising.

People could certainly choose to do that if they wanted to.

<comment><USER25> I don't think that they could. Because in order to be able to buy fresh food, you need a suply of fresh food, and where is that fresh food going to come from, if not from small time farmers? And if it is impossible for a large number of people to become small time farmers, then if more people choose to eat fresh, the price of fresh food will only rise to the point where they no longer have the choice to do so.

<comment><USER26> If enough people wanted fresh food, more would be grown.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Tell HN: Check out my little app - <AUTHOR> http://chatifier.heroku.com/ <comment><USER2> click here

This is working, but I suppose it shouldn't

</comment><comment><USER2> It was hard for me to notice the open chat button.

<comment><USER3> Same - and it only seems to work on certain sites. I can't get it to open up on HN.

<comment><USER2> It appears to run only in websites with jquery

<comment><AUTHOR> The bookmarklet loads jQuery, so the websites don't need to have it on.

</comment></comment></comment></comment><comment><USER4> I'm afraid I didn't read the front page carefully enough, then got very confused after creating a chat room and clicking the Chatify button.

Eventually I noticed the bar at the bottom of the screen, expanded it and saw the chat, but it was only after returning to the home page and reading the description again that I realised why it was working the way it did.

Works nicely with a lovely clean look.

<comment><USER5> I agree in the aspect that the creator should create an arrow or something after clicking "Chatify" button which points down to the chat and says something like "Pull Up".

Also it would be nice for instead of "Guest182" I could see "You" by default. (I understand I _could_ edit it; however, I'm guessing everyone would then see 'You'.) Also maybe an option to view time/date stamps- and not to view them. Possibly a feature to print chats? Also when you open it you don't have to click in the chat box before typing. (You could just start typing and then hit enter.)

Also you may want to consider a membership service to keep chatrooms longer than a week even if people don't use them. Just a thought.

Well that's my two cents as of now. However, for the review: The App seems nice. Elegant in design, but doesn't seem to work. So overall- it'd be nice in my opinion- if it worked.

<comment><AUTHOR> I will definitely take those into consideration. I'll adding message explaining how to use the chat window. I'll also extend the duration of the chatroom to a month.

<comment><USER5> You should also keep the deletion after a week if it isn't used in the first week. (That way if people make a bunch they don't steal resources from you.)

</comment></comment></comment></comment><comment><USER6> Good idea, but it would be better if we could just create chatrooms on they fly for every conceivable web page we visit.

Say I am on google.com and I click on the chat bookmarklet (or jetpack, or extension), it should be able to create a public chat for that url (or any) and start chatting right away.

<comment><USER6> If drop.io has an API you could try this:

\- shorten any url with goo.gl and get only the uri.path

\- create the chatroom in your bookmarklet as:

      http://drop.io/chat_{shorten}/remote_chat_bar.js

\- profit?

</comment></comment><comment><USER6> Bookmarklet explained:

      if not jquery: load jquery
      body prepend(drop_io_script_tag)

</comment><comment><USER7> I like it. Its a good idea.

I like to know about the technical details of it as well.

<comment><AUTHOR> It's using standard XMPP, but, really, I just built a wrapper around drop.io's real-time API.

<comment><USER8> I find this more interesting than the idea itself :)

So everytime someone 'creates' a chatroom it actually creates a drop.io and uses the chatroom associated with that drop.io?

FYI: I have asked the above to jmonegro in the chat: "yes, it's built around their streaming api. but the drops are password protected by a hashed function, so they aren't publicly available"

</comment></comment></comment><comment><USER9> HN chat room: <http://chatifier.heroku.com/hackernews>

I think there should be an option to chat without installing the bookmarklet.

<comment><USER10> Just click on the bookmarklet.

</comment></comment><comment><USER11> Very good idea and app, but I keep getting internal server error every other click.

You might want to check your server settings and logs for 503's...

<comment><AUTHOR> That usually means that the room does not exist. I haven't quite worked on the error pages.

</comment></comment><comment><AUTHOR> By the way, what I built was a simple wrapper for the chat and real-time services offered by drop.io

</comment><comment><USER12> Add padding to the bottom of the page so that it is not obscured by the floating chat portion.

</comment><comment><USER13> Reminds me of _why's hoodwink.d.

</comment><comment><AUTHOR> I fixed many bugs - it should work on HN and most other sites as well.

</comment><comment><USER2> I think that it only works in websites with jquery?

<comment><AUTHOR> The bookmarklet loads jQuery from Google.

<comment><USER14> I get this behavior too (tested on about 20 varied sites)

</comment></comment></comment><comment><USER9> Is there some way to password protect chat rooms?

</comment><comment><USER15> Bookmarklet doesn't work for me in Chrome.

<comment><AUTHOR> I _built_ it on chrome. You might be using an old version, I pushed a new version that should fix many of the bugs.

<comment><USER15> Oh, drop.io is blocked by my work ISP. Sorry for the false report.</comment></comment></comment>


RFC4180 – Common Format and MIME Type for CSV Files (2005) - <AUTHOR> https://tools.ietf.org/html/rfc4180 <comment><USER2> I don't see the semicolon separated CSV format going away. CSV is and will be a mess.

The RFC should perhaps capture the current state of affairs, something like new Web standards

<comment><USER3> If someone comes up with a clever alternative it'll be gone really quickly. Look at how quickly json caught on.

CSV is easy to write and easy to parse if you only care about 90% of the use cases. If you need to do the other 10% it's much harder and you get a lot of interoperability problems. And that 10% is only becoming more important due to internationalization.

</comment></comment><comment><AUTHOR> submitter here.

I've submitted the link to the formal specification of the comma-separated value (CSV) file format as I've noticed that someone else had posted a link to a CSV parser.

[https://news.ycombinator.com/item?id=12770373](https://news.ycombinator.com/item?id=12770373)

A CSV parser is trivial to implement, and the complexity of the endeavour is so low that it doesn't even justify as a tutorial on how to implement a proper parser.

<comment><USER4> It's worth pointing out that nobody actually uses the RFC 4180 version of CSV. And so ...

> A CSV parser is trivial to implement

A parser for any one dialect of CSV is trivial to implement. A parser which can robustly parse many different dialects of CSV is quite a challenge. Have a read of Python's:

[https://github.com/python/cpython/blob/master/Lib/csv.py](https://github.com/python/cpython/blob/master/Lib/csv.py)

EDIT: It's probably not fair to say that nobody actually uses the RFC 4180 version of CSV. Some programs use it explicitly, and many more conform to it by chance because it codified the most common practices around CSV. For example, it appears that Excel conforms to RFC 4180:

[http://superuser.com/questions/302334/true-difference- betwee...](http://superuser.com/questions/302334/true-difference-between- excel-csv-and-standard-csv)

<comment><AUTHOR> > A parser for any one dialect of CSV is trivial to implement.

That's what the formal definition is for. If a document doesn't match the CSV description then it isn't a CSV document. It's something else.

> A parser which can robustly parse many different dialects of CSV is quite a > challenge.

A hand-written parsers that's less than 1k loc is not a challenge. JSON is also a trivial format to parse, and a JSON lexer alone exceeds 500loc.

<comment><USER4> > That's what the formal definition is for. If a document doesn't match the > CSV description then it isn't a CSV document. It's something else.

Right, and if there was a formal definition of CSV, that's the situation we'd be in. RFC 4180 isn't one. It's a description of _one dialect_ of CSV, which was written decades after CSV came into use. There are dialects of CSV out there which don't conform to RFC 4180, and which are perfectly valid in their own right.

> JSON is also a trivial format to parse, and a JSON lexer alone exceeds > 500loc

FWIW, i wrote a JSON tokenizer in 235 lines, and it really should be shorter:

[https://bitbucket.org/twic/jsonomic/src/e0cae9587b20def003b9...](https://bitbucket.org/twic/jsonomic/src/e0cae9587b20def003b90d7b2f3a166213a692a3/src/main/java/li/earth/urchin/twic/json/parser/Tokeniser.java?at=default&fileviewer=file- view-default)

<comment><AUTHOR> > Right, and if there was a formal definition of CSV, that's the situation > we'd be in. RFC 4180 isn't one.

You're mixing things up. You're confusing "formal definition", which rfc4180 is, with the concept of an official standard, which no RFC is.

Yet, that doesn't stop the world from working based on RFCs such as this one.

> It's a description of one dialect of CSV

There you go.

> FWIW, i wrote a JSON tokenizer in 235 lines, and it really should be > shorter:

I've browsed through your code and I have to say you did a good job. I see your coding style is very frugal with the linebreaks and in breaking up statements, and in a manner that doesn't affect readability that much. OTOH, your absence of comments should be taken care of.

However, I should point out a couple of design flaws in your lexer.

First, your lexer handles any invalid token by throwing exceptions. Not only is that bad form and a misuse of what exceptions actually are (a document with a parsing issue isn't an exceptional event), but any parser that uses your lexer is unable to catch any syntactical errors, provide informative error messages, and most importantly be able to recover from parsing issues.

One standard technique is to support frequent lexial/syntactical errors in the parser and use them to handle parser errors.

The other design flaw is that you failed to include length limits on your strings. That's a potential source of problems, including security issues.

Another issue is that your lexer fails to parse JSON numbers, which is the most complex part of the lexer. Your lexer simply dumps any character matched by isNumber to a string, and then try to convert it to a number by calling Java's Double.parseDouble(). As you may know, JSON's definition of a number doesn't match JAVA's specification. If your lexer supported JSON's number format then it would easily double the number of states it already has.

So, eventough your lexer is a couple of hunded loc shorted than my estimate, it is in fact missing basic and fundamental chunks required by a lexer for JSON. Once you add those in, you could recount your loc and see how it steers away from the 500 estimate I pointed out.

<comment><USER4> Good point about the numbers! Double::parseDouble accepts a superset of JSON numbers, so the risk here is that something which is not a valid JSON number will be accepted. That could be tackled by building up the string, checking it against a regular expression for JSON numbers, and then parsing it. It wouldn't add any states to the parser, and would take two lines of code.

I would suggest that in a JSON parser, attempting to recover from parse errors, or supporting frequent errors, would be a mistake. JSON, unlike CSV, has a rigorous definition, and we wouldn't be doing the world any favours by tolerating deviations from it.

I'm not sure what the security issues with long strings would be. Text gets accumulated in a StringBuilder, which will handle strings up to 2 GB and then throw an exception. I don't think there's any way to get into an incorrect state.

Your other points relate to design decisions, and ones where i'm quite happy with the options i chose.

</comment></comment></comment></comment></comment></comment><comment><USER5> Do CSV files on really mainly use CRLF line endings?

<comment><USER6> That's a good question. I know email is supposed to use only CRLF line endings, but we use LF across a number of vendors' email API endpoints and have never had any issues.</comment></comment>


Reading 52 Books a Year Will Not Change Your Life - <AUTHOR> https://medium.com/@gregswierad/reading-52-books-a-year-will-not-change-your-life-f8147618da9b <comment><USER2> Jimmy Gatz (aka Jay Gatsby) was a fan of self help books, but that didn't make much of a difference to Daisy's relationship with Tom Buchanan.

Some books are read for charity / And some for fancy dress / But when they're read for pleasure / They're the books that I like best

</comment><comment><USER3> The vast majority of self help books are useless junk, the only person they help is the writer with their bank balance.</comment>


Let's stop shaking people down for their email addresses - <AUTHOR> https://m.signalvnoise.com/lets-stop-shaking-people-down-for-their-email-addresses/ <comment><USER2> Endless pop-up demands have become so irritating that I refuse to interact with them at all anymore. I won't even click on them to make them go away; I just go straight into Reader View. If that doesn't work, I assume the article probably wasn't worth reading and just close the tab.

</comment><comment><USER3> > If you have something to say, say it. If you have something to share, share > it. Don't invent things to say or to share just such that you can package up > that pink slime as a golden nugget of truth to trade for someone's contact > information.

Well said!

</comment><comment><USER4> Another article -- similar in nature -- from this source, posted here on HN as well:

"Marking the end of pixel trackers in Basecamp emails"

\- [https://m.signalvnoise.com/marking-the-end-of-pixel- trackers...](https://m.signalvnoise.com/marking-the-end-of-pixel-trackers-in- basecamp-emails/)

\- [https://news.ycombinator.com/item?id=20834729](https://news.ycombinator.com/item?id=20834729)</comment>


Spaced repetition for study and learning - <AUTHOR> http://www.gwern.net/Spaced%20repetition?2 <comment><USER2> I've been using SRS (anki, ankiweb, ankidroid) for almost four years now. I'm up to 4000 Japanese sentence cards and my amazement is constant. When I started I thought I was fooling myself, there was no way I'd be able to read real quantities of Japanese. Instead I can now read the manga for the anime I watched as a kid and enjoy the stories I thought had ended.

I've gotten into a rhythm, four new sentences per day which at my memorization rate works out to ~60 cards to review a day. Thus 20-30 minutes per day of review. It has become such a part of my day that skipping a day makes the day feel like a massive failure.

Its interesting how 4 cards can translate into 60 even with high memorization rates: for every card you add you'll see it again in 2 days, ~4 days, ~8 days, ~2 weeks, ~1 month, ~2 months, ~4 months, ~8 months, ~1 year... A review session is thus made up of multiple standing waves. The effect is noticeable you'll see a workload increase as the next standing wave of review starts. Right now some reviewed cards are getting scheduled next in ~8 years!

<comment><USER3> I have been using SRS to learn Mandarin Chinese for the last 2 years now and I can also verify it is extremely effective when used correctly, and when understanding its' advantages and limitations.

I think sentences are also a much better way to learn than learning single vocabulary words, because they show the usage of the word in context. This is very important for speaking naturally and idiomatically.

Once you get beyond a beginner level, choosing a direction of study can be a little difficult. My technique is to find sentences or phrases from conversations I have with people, books/articles I read, and from movies/television shows - and then input it into the SRS. Preferably, each sentence has 1 new novel phrase or vocabulary word. Although this can be time consuming it gives a huge benefit. Let me give an example:

      Question side: "問題是。。。你不覺得這樣,很。。。虛偽嗎?"

      Answer side: 虛偽 (xūwěi - hypocritical, artificial)

So, in this case the second last two characters made up the word that I was unfamiliar with, so I put that on the answer side. The huge benefit is that I found this sentence in a book I am reading - so every time it comes up in the SRS, I not only recall the sentence, I recall the _context of the story_ (in this case a dialogue between two characters), and this has a double reinforcing effect. In the same way, that if a TV or movie character says a sentence over and over again in a video, it will cause you to remember the scene of the sentence, and at which part of the story the movie was in at the time.

The contrast of this method would be to find "pre-made" decks of cards for sentences in the language you are learning, but I find that it is harder to remember the items when the sentences are not as meaningful to you.

A key point for using SRS effectively is to use to review material _already_ learned, not to learn _new_ never before seen material. The article itself even mentions this.

(Also, Mnemosyne is my SRS of choice [http://mnemosyne- proj.org/](http://mnemosyne-proj.org/), but Anki is a good choice too.)

<comment><USER4> I'd like to learn Mandarin.

Can you give details on how you're doing this?

Like what you use for initially learning the material and what you use for SRS?

<comment><USER3> Definitely - contact me at the email in my profile and I can tell you more about my specific approach for Mandarin. (Don't want to sidetrack the thread too much.)

<comment><USER5> Yes, please just say it here and share it with the greater community. :)

</comment></comment></comment></comment></comment><comment><USER6> I think spaced repetition software, which typically involves flashcards, is great for memorising a large quantity of facts. E.g. for learning foreign language vocabulary, anatomy, etc... For this kind of learning it can really save you time, especially when you have 100s or 1000s of items to remember.

Using flashcards to learn mathematics, computer programming, physics, or anything requiring more creative or analytical thought is not so useful. For these subjects the number of raw facts required is usually smaller, and the thought processes, concepts and problem solving ability are much more important.

However, spaced repetition could be beneficial to these more creative and analytical subjects if it was applied more dynamically. In this case the units to learn could be the rules, laws, concepts which can manifest themselves in different ways _within_ different problems, instead of being static flashcards. Does anyone know of software which works in this way?

My interest in this stems from [http://readlang.com](http://readlang.com), a site I'm working on at the moment for learning foreign languages. I use SRS to keep track of words, but in future it could be nice to think about applying SRS to grammatical concepts too.

<comment><USER7> _Using flashcards to learn mathematics, computer programming, physics, or anything requiring more creative or analytical thought is not so useful. For these subjects the number of raw facts required is usually smaller, and the thought processes, concepts and problem solving ability are much more important._

My programming flash card usually consists of evaluating pieces of code like:

Question -->

Evaluate the following:

puts "Hello world"

Answer --> Hello world

I cannot memorize the solution as if I memorize the answer to multiplication problems, which is just straight rote memorization. I memorize the process for code evaluation instead. In any case, I do not think anki and similar spaced repetition flash card software are in any way a replacement for actually just writing code and solving lot of different problems. They are always complementary to learning the materials in other ways.

<comment><USER6> The danger is that it's very possible to memorize 100s of answers like this without having any idea of how the code is working.

Imagine if instead of static flashcards there the SRS worked on items like:

\- puts

\- the '+' operator

\- variables

\- functions

Then an algorithm would generate problems incorporating those concepts. The exact same problem would never appear twice so you can't learn by rote and it ensures you actually learn the concepts. Your progress on each concept would be tracked by the SRS algorithm to help schedule your learning.

I think this would be a huge improvement on static flashcards.

<comment><AUTHOR> I agree, for a lot of algorithmic bits of knowledge, it's better to study a dynamic set of questions revolving around the algorithm, and that's almost the same example I give in [http://www.gwern.net/Spaced%20repetition#prospects- extended-...](http://www.gwern.net/Spaced%20repetition#prospects-extended- flashcards)

But so far no one has stepped up and done the programming & card writing to demonstrate it, aside from perhaps Khan Academy (they generate questions, AFAIK, and are supposed to have incorporated spaced repetition at some point).

<comment><USER8> I am a co-founder of Lymboo Math ([http://lymboo.com](http://lymboo.com)) We designed Lymboo Math to be a comprehensive, structured, and rigorous math enrichment program. While most programs focus only on short-term mastery of math concepts, Lymboo Math's emphasis is on both short-term proficiency and long-term retention. We do this by incorporating spiral reviews at periodic intervals. Our spiral reviews are 'spaced' both in time and in content, i.e., students are assessed on multiple topics in the same test. The mixed spiral reviews improve students' context switching skills and at the same time enhance retention of learned concepts.

You can learn more about our philosophy here: [http://lymboo.com/about](http://lymboo.com/about) and [http://lymboo.com/tour](http://lymboo.com/tour)

Thanks. Bala

</comment></comment></comment></comment></comment><comment><USER9> Spaced repetition is great for material which is divided into very small chunks and where you have a ton of time for comparatively little information (Vocabulary, especially in languages you are not learning for a degree).

Problems arise in practical situations when you want to pass an exam. Or a couple of them in a certain period of time.

Proponents of SR will say that it takes at least 6 times the amount of time/effort over learning "short term". So if you are learning for an exam, you better have plenty of time to spare. And you better start early, like 6 months before the exam. All that makes it pretty useless in a lot of situations I've been in.

Short term learning isn't as bad as it looks. If you really "cram" for an exam, and you think you forgot all of it a couple of days or weeks later, that's not what happened. Instead, your brain is now primed for the material, and when you come into contact with it later, a lot of information is still there and re-learning will be easier.

<comment><USER7> _Proponents of SR will say that it takes at least 6 times the amount of time /effort over learning "short term". So if you are learning for an exam, you better have plenty of time to spare. And you better start early, like 6 months before the exam. All that makes it pretty useless in a lot of situations I've been in._

This is a bit of a strawman. If you need to pass an exam quickly, you should and will "cram". Afterward, you can resume your spaced repetition learning.

<comment><USER9> For me, exam-focused learning has been the norm, not the exception. I don't hear very often that people use these SR systems continuously for years, and if so, it's usually in language learning

</comment></comment></comment><comment><USER10> Gwern, how long does it take to put together a comprehensive document like this? Also, what motivates you the most in compiling the information? Is it a rabbit hole of fascinating discovery which you joyfully explore until your knowledge is satiated or does something else compel you?

<comment><AUTHOR> > Gwern, how long does it take to put together a comprehensive document like > this?

This particular page is one of the most citation-stuffed pages on my site, and I read everything I link if the fulltext is available (which is currently almost all of them after my last batch of requests)... I know I spent at least 40 hours compiling the core of the literature review for the LW contest, and I feel like that was less than half the time I spent writing the rest of it, updating it with new papers, that sort of thing, so I would guess this took somewhere upwards of 80 hours.

</comment></comment><comment><USER11> I always like sticking to the very best, so SuperMemo was the choice for me. I remember spending a couple of weeks just reading the whole website ([http://www.supermemo.com/](http://www.supermemo.com/)) because it contained so much interesting information on learning and memorization. It also took a while to understand how to use the program properly because it probably has one of the worst UIs one can imagine ;-).

Anyways, I ended up dumping SuperMemo because once my item database grew I became too lazy to spend 30 minutes (or more!) each day going through this drill. I might get back to it at some point, though.

Edit: By the way, SuperMemo offers a few more interesting tools.

One is "Incremental Reading", where you can import an article or text file into SM and SM will let you incrementally process it. For example, it might pop up as part of your repetition routine, and you can read a few paragraphs (you read as much as you want), and pick out the important parts of what you've read (parts you want to remember). Once you're done reading, you continue and the whole text will come up again and again, until you finish reading it and then you can dismiss it. The parts you cut out from the text become separate learning items and they too will come up in your daily routines, which let you further work on them until you ultimately reduce them into a Question & Answer form.

Another tool is sleeping monitoring. SM will let you track your sleep data, and it will tell you how your sleep affects your memory performance.

<comment><USER12>
        > Anyways, I ended up dumping SuperMemo because once my item database
        > grew I became too lazy to spend 30 minutes (or more!) each
        > day going through this drill.

Yeah, same happened to me. But I didn't catch on so soon, and I found myself spending 2 hours/day on it before I started to scratch my head.

When I started, it wasn't obvious, but clearly there is an upper limit on the rate of new entries that the system can support. Plus, you have to factor in time spent doing data entry. I think these factors combine to make for a life- time limit on the number of entries. Suppose you use SuperMemo for 20 years (7300 days) at 2 hours per day (14600 hours) and 100 new entries per day. That's only 1.4 million unique entries.. (this analysis is severely lacking and deserves more thought).

Insert here some humbling story about the benefits of improving meta-aspects of how and what you pick to spend time on. Obviously, inserting everything you ever come across into SuperMemo is a bad idea.

Anyway, then I realized this was all stupid because I remembered everything anyway. So now I'm off SuperMemo. Hooray memory!

<comment><AUTHOR> > I think these factors combine to make for a life-time limit on the number of > entries. Suppose you use SuperMemo for 20 years (7300 days) at 2 hours per > day (14600 hours) and 100 new entries per day. That's only 1.4 million > unique entries.. (this analysis is severely lacking and deserves more > thought).

That's fairly similar to Wozniak's estimates: [http://www.supermemo.com/articles/theory.htm](http://www.supermemo.com/articles/theory.htm)

In the FAQ he estimates that probably 200k is the max for most people.

> Anyway, then I realized this was all stupid because I remembered everything > anyway. So now I'm off SuperMemo. Hooray memory!

Fortunate for you, but the rest of us must make do with our fallible memories. :)

</comment></comment></comment><comment><USER13> Spaced repetition is not new, nor fancy, and has direct biological evidence. It works. That said having a good tool and motivation are required. I can recommend ankiweb.net / anki desktop for the first, but can't help with the second. The motivation to follow through on doing cards is what I have trouble with.

<comment><USER7> I managed to hacked myself to build a habit of 30 minutes a day of spaced repetition, building off my momentum to consistently do khanacademy an hour a day(A project which I finished a while ago) in conjunction with my time tracking tool. The key seems to be a bunch of small repeatable goal that allow for small wins. It works for me, though I don't know if it will work for anyone else. Plus, it's only anecdotal, so take it with a grain of salt.

(I wish there is a accessible book that rigorously about which habit building strategies work according to scientific research)

<comment><USER14> The Power of Habit and [http://tinyhabits.com/](http://tinyhabits.com/) are both great.

<comment><USER15> That sounds like such a scam, the wording is the same as every "With these easy steps, you'll feel better and be more fit!" program I've encountered.

Not saying it is a scam, but I'm weary of clicking through.

<comment><USER16> Wow it sure does. Only slightly more plausible than "one weird trick".

<comment><USER17> Well, I took one for the team and registered. The world hasn't ended yet. I'll let you guys know. :)

</comment></comment></comment></comment></comment></comment><comment><USER18> I largely credit spaced repetition with my improvement as a chess player from a rating of 1800 to 2000, something pretty unusual for a middle-aged adult. I wrote about about my experience at [http://dfan.org/blog/2013/07/07/mnemosyne- part-3/](http://dfan.org/blog/2013/07/07/mnemosyne-part-3/).

</comment><comment><USER19> This is one of the greatest pieces I've ever read.

The anecdote where Hamming went to Bode's office to talk about Turkey was like a lightning ! Is it possible to see so many great names in one paragraph !

I've always seen some analogies in things with half-life and the charge and discharge of capacitors.

I'll give an example: I've always thought about learning as a capacitor charging through a resistance, and thought about the RC constant. In my mind, it takes relatively a short period of time to get to 63.2%, but it takes a very long time to get from 63.2% (it will never reach that, since it's an exponential).

What this tells me about learning ? Pick a field and then decide how much do you want to master it. Then know that it'll take you longer to get from 63.2% to 100% (which you will never achieve). An example would be the fact that English is my fifth language, which I only started speaking in 2009. Do I want to be an English major and write in a Kipling way, or do I want to have a command good enough to get my message to you (and possibly learn other languages) ?

It depends, but I find it a fascinating subject.

The other funny thing is that I used a mix of cramming and spaced repetition to study: I used to skip college for a year, and then get a month and go hard- core. Five courses. I'd draw a pentagon with the five courses and study two courses a day. 1,2 - 3,4 - 5,1 - 2,3 - 4,5 - 1,2 ...

You'll notice that I go back to each module each other day. The results I got from that are just astounding, as I was capable to memorize entire pages, chapters and I would see them. I would know exactly where I've written my notes on which pages. I aced the final exam of physics and had the best grade with questions that were never asked before (it was theory, and no 'values' or 'calculations' were there). I was writing things I've never written before, but I knew my maths checked out because I just knew. These weren't exercises or something.. I've been accused of cheating many times. A part of me found it insulting, the other part found it flattering as you only accuse someone of cheating if what he did was an accomplishment.

</comment><comment><USER20> I recently read this study of Halo players from Microsoft, and the authors found that spacing out your play made your gain rating faster per game played, BUT if you space it out too much, then you'll simply be outmatched by players who play a lot more. In other words, if you want to improve at Halo efficiently, space out your play more, but if you want to improve in sheer absolute skill, you will want to simply play more games.

[http://jeffhuang.com/Final_HaloLearning_CHI13.pdf](http://jeffhuang.com/Final_HaloLearning_CHI13.pdf)

</comment><comment><USER21> I've been considering making spaced repetition software to help reteach and remind me of technical concepts I've learned over college that I don't recall too well. Stuff like fluid mechanics, microelectronics, thermodynamics, control systems, complex analysis, computer architecture, etc. Basically I have access to a lot of relevant course notes I took over the years (and digitized), but I don't remember it, so I would use SRS on this material to relearn it. I was thinking 20 mins a day would be a good amount of time for this.

But wait, let's take a step back. Does it _really matter_ if I can recall all this engineering material? Supposedly my brain "knows" enough of it such that if there were a big project in a field requiring that knowledge, then I could relearn it all, because I've been trained to think in a way pertinent to that material. Is my time better spent just learning new topics and building things, which is what I currently do? Why dwell on having perfect recall? It only seems to benefit test takers.

I would love to hear counter-arguments for why SRS for engineering materials would be a good idea for those no longer in school, maybe then I'll be motivated to build some gamified spaced repetition software. It'll need to be fun and reward-based, imo.

<comment><USER22> I've been thinking about this for a while--I created a spaced repetition system to digitize and learn material in college (alas, I don't use it anymore). I have a philosophical approach to the question here (also cited in Gwern's article): [http://blog.learnstream.org/2012/08/spaced-repetition-in- nat...](http://blog.learnstream.org/2012/08/spaced-repetition-in-natural-and- artificial-learning/)

Your distinction between learning facts and learning ways of thinking is interesting. Most of the spaced repetition research and design is around fact recall--what's called "paired associates" because you recall some response paired with some stimulus. I don't have any evidence for it, but I'd say that the schema can be lost just as well as facts can be.

One thing I will say is that spaced repetition is generally known for the repetition--that you retain something over time, and the spacing is more thought of as convenience. But the truth is that spacing is also part of better learning--when recall is difficult it's learned better (called "desirable difficulties"). I thought of that because Robert Bjork does that research with skills as well as facts. (See [http://www.wired.com/geekdad/2012/01/everything-about- learni...](http://www.wired.com/geekdad/2012/01/everything-about-learning/) for an overview.)

<comment><USER21> Yes it's not obvious to me how to impart knowledge in a SRS format that's more than fact recall. I was thinking of starting with thermodynamics because it's one of the more straight-forward equation courses - but then how do you properly do problem solving? I suppose if you walk through a problem broken up into enough chunks phrased as questions, and somehow give users a toolbox from which to choose equations from, it might work.

<comment><USER22> Not sure how deep down the rabbit hole you want to go, but here's what I know:

The field of intelligent tutoring systems is essentially how to break up a problem and estimate your knowledge on each piece (called a "knowledge component"), and then how to instruct based on the resulting student model. I got started by reading about Andes physics tutor: [http://oli.cmu.edu/wp- oli/wp-content/uploads/2012/05/VanLehn...](http://oli.cmu.edu/wp-oli/wp- content/uploads/2012/05/VanLehn_2005_Andes_Physics_Tutoring_System.pdf) but there are many examples.

Most of the research doesn't actually incorporate spaced repetition. One exception is Philip Pavlik's work: [http://optimallearning.org/](http://optimallearning.org/)

It's still a lot of effort to break things down into knowledge components and figure out the sequencing and instruction. That's why companies like Carnegie Learning and Knewton exist, and even those are only targeting elementary or intro level material for now.

</comment></comment></comment></comment><comment><USER23> I have been experimenting with spaced repetition algorithms for around two years now, and I've found that for me, an a simple exponent with base of 2.71828 (e) to be optimal for learning short question/answer pairs.

This means, every time I successfully recall the answer to a question, I increase the interval of when I will be challenged with it again by about 2.71828 times since I had seen it previously.

This is with a recall failure rate of about 5%, which I'm trying to maintain. (If it's nearing 0, then you're probably wasting time being challenged too frequently.)

This is on a data set used daily for 30 minutes, reviewing about 50 older questions and encountering around 3 new ones per day.

</comment><comment><USER24> Comprehensive, as usual. An obligatory nitpick:

> _At least with physical fitness there isn't a precisely dismaying number > indicating how far behind you are!_

Many sports are precisely measurable and in such sports, the measurements determine who has won, to various resolutions. Shortest running time over a given distance, heaviest weight snatched, furthest distance jumped and so on.

Athletes in such sports can very accurately track their performance over time. As of yesterday I demonstrated that I can clean & jerk 25kg less than my lifetime record, for example.

<comment><AUTHOR> I think the better analogy would be if you had some sort of workout schedule and you had missed n workouts. Just bad performance doesn't indicate much: if I score a 2 on all my flashcards today, that's bad, but I'm not 'behind' since the cards' next review will just be moved up as necessary to compensate.

</comment></comment><comment><USER25> For me it was absolutely surprising that out of the 200 Android apps, only 4 used spaced repetition but beeing an Android developer myself, I took the challenge to make it better: [https://play.google.com/store/apps/details?id=de.leowandersl...](https://play.google.com/store/apps/details?id=de.leowandersleb.fluxcards)

FluxCards provides spaced repetition learning with easy to edit and create own cards. You can sneak the answer line by line, you can finger-paint and the stats are fast and meaningful.

It is free to use with 0.5, 1, 2, 4, 8, 16 days intervals. Beyond that you need the premium version but your cards are yours, so if you consider it not worth the fee, you can export your cards to SD and use them in Anki.

And yes, spaced repetition rocks. I can only encourage any flash cards programmer to implement it, as all these apps that implement flash cards but no spaced repetition learning are just not worth your time.

</comment><comment><USER26> I use SRS (Mnemosyne) for memorising traditional Irish tunes. It feels like each tune is a bit too much of a large chunk to remember at one time for best use of an SRS (you're supposed to chop the information up into the smallest element possible), but the benefits of the system are amazing. Before I started using SRS I'd just practise the tunes I liked and already knew!, with the new tunes, and anything in the middle got forgotten about. Now mnemosyne forces me to practise tunes that I don't know, and the number of tunes I'm fluent in has increased dramatically.

I'm also building a language learning startup that uses SRS. If you're interested in trying the system out in beta, or being notified when it's ready for general consumption, email me at firstname.lastname@example.com and let me know which language you want to learn.

</comment><comment><USER27> at kenHub[1], we built an online trainer for anatomy. It uses a game-like sequence of exercises designed to teach and test you at the same time about anatomy structures and their terminology. We've implemented a very basic spaced-repetition algorithm, but there's always more to improve.

One interesting thing we've found early on, was that when we encouraged people to actually "take a break", not to repeat too much, but instead let your brain process and come back to it later, people didn't like it. They wanted the ability to keep training and training. This resonates well with what the article describe about students' cramming tendency. So some time you have to manipulate even a solid algorithm, to fit with reality.

[1][https://www.kenhub.com](https://www.kenhub.com)

</comment><comment><USER28> I combined spaced repetition with machine learning to attempt to predict memory recall for my thesis. Turned out fairly effective, correctly predicting recall up to 70% of the time.

Spaced repetition is pretty amazing. Adding records feels like you're entering them into your brain, provided you use it every day. I managed to go from knowing almost nothing to scoring 99% in a Japanese kanji exam once, entirely thanks to Anki.

My thesis is available here: [https://github.com/jordwest/Thesis- Report/raw/master/report....](https://github.com/jordwest/Thesis- Report/raw/master/report.pdf)

The online SRS I built is also available: [https://github.com/jordwest/Membit](https://github.com/jordwest/Membit)

</comment><comment><USER29> SRS software (I mostly used Anki) was invaluable to me when I was learning Japanese, both for vocabulary and the kanji.

I always wanted to make my own SRS, one that had ease of use , stability and data integrity at its core rather than cutting edge features. This year I finally set aside time to make it: feel free to try it out and let me know what you think. It is very simple (there is only one deck), but multiple decks will be the next feature to be added.

[http://benkyo.co](http://benkyo.co)

On top of that platform is an implementation of a rather interesting way to learn Japanese reading which it would also be great to have feedback on:

[http://benkyo.co/iikanji/](http://benkyo.co/iikanji/)

<comment><USER30> iikanji looks good, but have you seen wanikani? The concept seems similar. [http://www.wanikani.com/](http://www.wanikani.com/) Some feedback: I know how to read all of the stuff I tried (the first 10 kanji), mostly thanks to wanikani. But I haven't studied much gramar, so I'm not always sure what the sentences mean. You should probably add translations for these in the back of the cards.

</comment></comment><comment><USER31> Does anyone know of any command line spaced repetition tools? if not could be a fun side project.

<comment><USER32> i am looking for that as well.. if you use emacs there are org-drill.el and flashcard.el:

[http://sachachua.com/blog/2013/02/rediscovering-emacs- featur...](http://sachachua.com/blog/2013/02/rediscovering-emacs-features-or- what-to-do-after-you-get-carried-away-installing-packages/)

</comment></comment><comment><USER33> It's great for learning facts and exploring a subject. Spaced repetition is fascinating. Especially in the realm of language learning. Vocabulary acquisition can be optimised by deciding the right order of words to maximise your ability to read a foreign language text. gwern has a link at the bottom of the article on modelling vocabulary lists as a travelling salesman problem. Feeding this into SRS Flashcard program is powerful. I guess one of the big problems ( if speed is your goal) is the creation of the cards themselves. The time to create the card is much greater than the time to learn the card and can be time a sink if you are trying to learn your first 3000 words of language.

</comment><comment><USER34> I've been using Anki for learning Japanese. I've particularly appreciated the decks that include audio of complete sentences and have found that it has greatly improved my ability to understand spoken language.

One thing that I've wondered is whether it is a good idea to try to think up mnemonics for tricky words when using spaced repetition. It certainly seems effective in helping recall but I worry that I might train my brain to always rely on going via an indirect path to get to the information and that this will slow down my ability to understand real language.

Has anyone seen any research along these lines?

<comment><USER35> From my personal experience, using such a method doesn't cause problems. I studied the kana using Heisig's Remembering the Kana, where it connects each character and pronunciation to a mnemonic story. I'm long past the point where I actually need the stories to read kana, the sounds are simply tied to the characters now, with the stories available for recall if i have a brain flub and can't remember one for a bit.

</comment></comment><comment><USER36> I was working on mobile app that helped with spaced repetition as a side project. I built a pretty good looking user interface but ran out of time to work on it between school and my company. If anybody is excited about spaced repetition and wants to join me on my side project, I'd love to try to make a little bit of extra time to get it live. I _really_ wish that the application already existed - my midterms would be going much, much better :)

Email me at firstname.lastname@example.com

<comment><USER25> [https://play.google.com/store/apps/details?id=de.leowandersl...](https://play.google.com/store/apps/details?id=de.leowandersleb.fluxcards) would need a developer for a browser version. I have big plans and if you convince me, I might pay you for that or we agree on a share. It already has 10k users and a 4.5* rating.

</comment></comment><comment><USER37> I used anki throughout high school for classes and SAT prep - pretty much the main reason I ended up at an ivy league school (now a junior). It was especially helpful for classes like AP Bio / AP US History where there was a ton of pretty obscure stuff to memorize.

It's not so useful in college now that I'm majoring in applied math, but occasionally I'll use it for a humanities class. (throwaway b/c I always use throwaways)

</comment><comment><USER38> For those using org-drill in Emacs, I have a write-up of my own card/directory layout and custom elisp to provide a navigation prompt to select which "category" you want to train: [http://ixmat.us/articles/2012-12-01_usable-org- drill.html](http://ixmat.us/articles/2012-12-01_usable-org-drill.html)

</comment><comment><USER39> I don't have anything constructive to say besides that I'm always impressed by your work, Gwern.

Thanks for all the references as always.

</comment><comment><USER40> I use Anki every day. I think part of the project is in learning how to use it best. I have little interest in learning word-of-the-day or country names. I am more interested in learning new skills, and I shared the concern that it wouldn't work so well for that.

I think it's doing all right though. Here are some of the decks I'm most proud of:

\- Using lilypond snippets and a midi-to-wav generator, I wrote a perl script that generates a deck for every (non-accidental) note on a guitar fretboard. It shows the fret position with a question mark, and then when you identify the note, it plays the pitch and shows the note on a treble clef. I'll probably expand it to include accidentals soon.

\- Also using lilypond, I wrote a jazz theory deck for major chords. Questions like, "What chord quality and inversion is this?" where it plays a chord, and answer would be m7b5 (half-dim) 2nd inversion. My ear training has improved a ton. And, what is the key scale for bII7/iii in B major? (Plays the chord audio). Answer: B melodic minor or E lydian/mixolydian, playing and showing the scale on treble clef. And an effect like gwern described has happened. I haven't actually memorized the answer to every card in the deck (since the perl script generated about 2500 of them). Instead, I've gotten faster of doing the actual music theory processing in my head to calculate/recognize the right answer almost instantaneously. End result is that it has improved my speed in reading lead sheets. (It has not, however, improved my melodic improvisation abilities. :-) )

The rest of my decks are basically based off of Coursera courses. Unfortunately, I haven't finished any of them yet. I routinely run into the problem where partway through the course, the amount of time I need to spend keeping up with the new cards limits my time available to watch new videos and do new assignments, so I don't finish the course. I am now taking the Scala course again, and I'm hopeful that since I'm fully reviewed up through week 4, I might be able to power through and finish it with a deck that fully reflects the course.

This also means that I am improperly creating a separate deck for each subject matter, when Anki doesn't let you scramble them. I should be putting them all in one deck, but I kinda don't want to before a deck is "complete". I'm also a couple of chapters into "Learn You A Haskell", with the rest of the book on pause.

Finally, I can echo that practicing helps a lot. For instance with the Haskell deck, I got up to fully reviewed, and each card was still kind of hard. Then I spent a couple of hours playing with the first Project Euler problem. After that, the entire deck was much easier and faster. So, retaining familiarity still is somewhat different than learning the concepts. If I have a fully reviewed Anki deck from Odersky's Scala course (on coursera), it won't mean I know Scala. But if I go start a Scala project, the Anki knowledge will help me a ton, and then afterwards, the experience from hacking on the project will make the Anki cards much easier, maybe even to the point that I could delete a bunch of them.

BTW, that's feature I wish I had from the deck - some kind of advice for when it might be appropriate to delete a card. I worry that I'm doing the equivalent of writing pointless unit tests sometimes.

<comment><USER41> How much of a music theory background did you have before you started making those cards? What do you think is the minimum someone needs to know to be able to benefit from making such a deck?

<comment><USER40> Well, my college degree was in music and I always had a knack for theory, testing out of college courses etc. But you don't have to have university level music theory knowledge to study jazz theory. I'd say the prerequisites for the more complex jazz deck of mine are knowledge of scales (major, melodic minor, major bebop, minor bebop), chord qualities (M7, 7, m7, m7b5), chord inversions, and treble clef notation.

For the people reading this that do jazz, it's based off of Randy Halberstadt's concepts of planets and harmony. Scale choices differ from the classic guideline of being simply chord based. For instance, dmi7 does not necessarily imply a d dorian scale. If it's in the key of C at that point, then you'd instead play C major bebop.

</comment></comment></comment><comment><USER42> Derek Sivers wrote a blog post on this back in january[0]

[0] [http://sivers.org/srs](http://sivers.org/srs)

</comment><comment><USER43> Rethinking the exam structure in US schools to better accommodate and motivate spaced learning would be a significant step toward improving education quality.

'cperciva noted this effect from his time at Oxford...

"When I tell North Americans about my time at Oxford University, one aspect of its undergraduate program inevitably surprises them: Final examinations. Rather than writing examinations at the end of each term, students in most subjects write a single set of examinations at the end of their final year, on which their entire degree performance is measured. (In recent years, some subjects have switched to annual examinations.) As terrifying as this might sound, the system has some merits: Whereas many students at institutions with termly examinations study the day before an exam and forget the material the day after the exam, attempting to 'cram' three years worth of material at once is largely an exercise in futility; as such, Oxford's examinations often provide a better measure of how much material a student has learned and retained. Given some of the conversations I've had with — and code I've seen from — people who have received computer science degrees from major universities and are actively working as software developers, I think that last part is very important: There's a lot of people out there who have forgotten some very basic — and very important — material" ([http://www.daemonology.net/blog/2012-10-08-software- developm...](http://www.daemonology.net/blog/2012-10-08-software-development- final-exam.html)).

You can obviously practice spaced repetition regardless of your school's exam structure, and Cal Newport specifies a method for doing this in "The Art of Stealth Studying: How To Earn a 4.0 With Only 1.0 Hours of Work" ([http://calnewport.com/blog/2007/10/03/the-art-of-stealth- stu...](http://calnewport.com/blog/2007/10/03/the-art-of-stealth-studying-how- to-earn-a-40-with-only-10-hours-of-work/)).

However, most students are unaware of this method nor have the disciple to follow it on their own so to realize its benefits on a mass scale, we would need to revamp the education system around these principles.

And as 'qwern noted, sleep is a key factor in spaced repetition learning. Research shows sleep is when the brain prunes itself by separating signal from noise...

"Sleep researchers at the University of Wisconsin-Madison School of Medicine and Public Health believe it is more evidence for their theory of 'synaptic homeostasis.' This is the idea that synapses grow stronger when we're awake as we learn and adapt to an ever-changing the environment, that sleep refreshes the brain by bringing synapses back to a lower level of strength. This is important because larger synapses consume a lot of energy, occupy more space and require more supplies, including the proteins examined in this study."

"Sleep — by allowing synaptic downscaling — saves energy, space and material, and clears away unnecessary 'noise' from the previous day, the researchers believe. The fresh brain is then ready to learn again in the morning" ([http://www.sciencedaily.com/releases/2009/04/090402143455.ht...](http://www.sciencedaily.com/releases/2009/04/090402143455.htm)).

</comment><comment><USER44> Spaced repetition, also known as rote memorization, is a terrible way to "learn" anything. The trouble is, you aren't learning at all, you're hacking your brain to force information into it that has no relevance and really doesn't want to be there. It's not a useful or pleasurable activity. It definitely won't make you smarter. Even Mr. Supermemo himself admitted that after FIVE YEARS of constant daily marathon memorization, 60% of it is gone within the first year of stopping ([http://supermemo.com/articles/kowal.htm](http://supermemo.com/articles/kowal.htm)). The brain does not want to work this way.

I'll propose an alternative to rote memorization for learning: critical thinking.

<comment><USER40> That's funny because you can write a deck to help you learn concepts of critical thinking. Necessity, Sufficiency, Contributory Causes; Propositional Logic, Predicate/First-Order, Modal... the deck won't TEACH you critical thinking, but it will make it much easier for you to learn it.

<comment><USER45> Learning the names of things is not learning those things. This is particularly true when you're learning ways of thinking.

What is important is not that you can identify and name methods of reasoning. It is important that you can apply the appropriate reasoning method to problems in your life.</comment></comment></comment>


Orchestra.io bought by EngineYard - <AUTHOR> http://blog.orchestra.io/post/orchestra-engine-yard

<comment><USER2> We have this already: <http://news.ycombinator.com/item?id=2916374></comment>


US capital punishment in crisis as Pfizer blocks drugs used in lethal injections - <AUTHOR> http://www.telegraph.co.uk/news/2016/05/14/pfizer-blocking-use-of-its-drugs-for-lethal-injections/ <comment><USER2> Just switch back to firing squads and hangings, crises over.</comment>


Arduino + Chromebook - <AUTHOR> https://www.sparkfun.com/news/1803

<comment><AUTHOR> I find this pretty exciting because of how much it lowers the barrier to entry - you can now go from not even owning a computer to programming hardware for less than $200.</comment>


Distribution vs. Innovation - <AUTHOR> https://a16z.com/2015/11/05/distribution-v-innovation/ <comment><USER2> I wonder if this is what Paul Buchheit meant in this tweet:

[https://twitter.com/paultoo/status/652547232504217601](https://twitter.com/paultoo/status/652547232504217601)</comment>