What's Really 'Immoral' About Student Loans - <AUTHOR> http://online.wsj.com/article_email/SB10001424127887324688404578541372861440606-lMyQjAxMTAzMDIwNzEyNDcyWj.html?mod=wsj_valettop_email <comment><USER2> It is criminal that serious student-loan reform isn't on the table. As a recent graduate making monthly payments towards a 5-figure principal, I say fuck capping the interest rates. Please cap tuition and fees.

Don't give me that BS about how universities need the money.

During undergrad, I squatted in a 600 sq. ft. apartment shared with 3 other people. I seriously doubt my health is unaffected by the mold, mildew, vermin, etc. I had to live with every day. Across the street was the president's mansion(!). He had a team of gardeners, fountains, a gilded mailbox, and some pretty nice carS in the driveway. His army of administrators never worked a minute after 5:00, as evidenced by the steady stream of Bimmers, Benzes, and Lexuses that poured out of the administrative building's parking lot every day at 5:01 pm. Oh BTW, their building was the only one on campus with glass doors, bronze handles, oak furniture, marble floors, etc.

Trim the fat, and make the university answer to a more sane market force.

<comment><USER3> how about making academic debt dischargeable without question during bankruptcy - so that the school is 100% on the hook for the balance of the funds if it educates the student poorly and can't get them a job, instead of keeping them in (albeit voluntary) virtual slavery to service their debt?

Yes, interest rates would go up, to cover the risk of defaulters, but then only serious students would go to college.

<comment><USER4> Schools would immediately close departments that tend to graduate low-paid folks -- humanities, acting, psychology, history, anthropology, all gone. Not sure if it would be a problem for society because those who actually produce art tend to learn to do it before college and just spend their time in college refining, which they could as easily do in clubs and societies without the BS that goes along with college.

<comment><USER5> Based on that criteria, wouldn't universities also have to close math, biology, physics, foreign languages, environmental science, psychology, architecture, civil engineering, etc?

<comment><USER3> I'm sure there will be some rich people around who could pay for tuition in general up front, the departments that generate 'students that can get modest jobs' might become smaller. Of course, then the supply of educated mathematicians, biologists, physicists, would get smaller, too, and their pay will go up, and everything will reach a saner equilibrium.

</comment></comment></comment></comment></comment><comment><USER6> For an opposite view point, see the Salon article _Tuition is too damn high_. It argues for the vast majority of people tuition has increased because government has drastically cut funds to public education. Not because of Pell Grants or "easy money". The author has some interesting data, as well.

Here's a glimpse:

"The first step in grappling with the rise in the cost of higher education requires understanding where students go to school. There are three main categories — public schools (which include both four-year public universities and two-year community colleges), private nonprofits (the Ivys, most liberal arts colleges, etc.), and the for-profits (Kaplan, University of Phoenix, Corinthian Colleges, aka "career schools"). Here's the key statistic: Fully 70 percent of the 19 million undergraduates and 3 million graduate students enrolled in post-secondary education in 2010 attended schools considered to be in the public sector — by which it is meant that some portion of their funding comes directly from government."

"The problem: The word "public" doesn't mean as much as it used to. Direct state support for public colleges has cratered over the past 10 years, and really fell off the cliff after the financial crisis. Yes, tuitions have risen, but not by as much as state and local appropriations for higher education have fallen. Just between 2008 and 2009, for example, average tuition revenue at public research institutions increased by $369 per student, but the loss in state and local appropriations per student was $751. Similarly, at public community colleges, tuition revenue rose by $113 per student, while appropriations fell by $488. Since the recession of 2001, tuition hikes, as exorbitant as they have been, still haven't kept pace with the fall in government support."

[http://www.salon.com/2012/05/11/tuition_is_too_damn_high/](http://www.salon.com/2012/05/11/tuition_is_too_damn_high/)

<comment><USER7> The article you link touches on it, but before you even consider where students go to school you must ask _why_ they do. And that is a very vacuous answer most of the time. I got my CS degree only because my peers said it would be significantly easier to be consistently employed with a BS even when it cost me 20k to get.

In reality, young adults don't go to college to learn. They did that for ~15 years (even worse, they did for 15 years by compulsion without much say in the matter) and the vast majority want to do something else. Most of them are there for _credentials_ , not knowledge, and that is the problem.

The most important problem here is the necessity to disconnect education from schooling. The intent of the latter is to cause the former, but the former is not dependent on being confined to the latter. You can apprentice, you can self-teach, you can dive in as a high risk newbie in an industry. None of those involve going to prohibitively expensive summer camps turned annual to get lectured at.

The Internet enables a lot here - where previously it was very hard to find like minded individuals interested in participatory group learning about a subject, now it is stupidly easy, a search away. Learning resources are also rampant online in what used to be confined to a tome from the local library or a prohibitively expensive book purchase.

So the third viewpoint is that the problem has nothing to do with how the government behaves here because it is symptomatic of a baby-boomer irrational quest to get pieces of paper saying "bachelors" on it for everyone currently under the age of 30. The solution is to get a significant chunk of the people currently attending college and accruing these insane debts into more productive and better directed paths to careers in the things that interest them without the cultural baggage touting a phd in some field writing on a chalkboard as the golden goose of learning.

The real, true, bottom of the barrel reason _I_ think this entire debacle even exists or came about in the first place, and I think it is at the root of most societal shifts for 30 years, is that the ultra-concentration of capital and wealth into so few people has dramatically slowed down the productivity engine of the first world. There is little motivation when you already pull the strings of international business to invest in risky youth labor, and when you control a huge chunk of the economy your lack of risk taking on new talent means huge droves of the population are never given a chance to succeed. It is safer and (here is the key) more profitable to play games with rigged fiat money and banks or stocks of fortune 500ers than to actually create goods and services by growing productivity and creating value.

<comment><USER8> Universities are older than the printing press, older than telegraphs, older than fast & cheap mass transport, older than radio and television.

To be quite honest I don't see the internet usurping them either.

<comment><USER9> Are you being facetious? All of those things have been dramatically changed by the internet. Or are you saying because they are older, universities are impervious to change?

<comment><USER10> He's saying that universities have not been significantly changed by the numerous communication technologies that came after them, and implying that the Internet does not have a property distinct from those previous technologies.

<comment><USER8> Right. And I thought I said it pretty clearly:

> _To be quite honest I don 't see the internet usurping them either._

I mean, OK. The internet is a _big damn deal_. But it's not a quantum leap as the printing press, steam or telegraphs were. It has antecedents in all those three technological revolutions.

The printing press was the first time knowledge could be cheaply conveyed to masses of people without needing someone to manually sit there with the learner and communicate with voice. It totally upended the concept of how individuals could receive their education. It didn't replace the university.

Steam was the first time that production and transportation could happen faster than muscle power or wind could move objects. It didn't cause a meaningful change to the university model.

Telegraphs did to thought and knowledge what steam had wrought in the physical world. One could be anywhere on earth and communicate to anywhere else on Earth within the hour. Minutes in some cases. You could use it to form communities of like-minded learners, and people did exactly that. It didn't unseat the universities.

For some reason, the concept of a vertically-integrated, physically co-located community of scholars and students seems _really, really_ hard to shift. Technologies come along and modify this or that part of it, but students plucked from Bologna in 1088, from Oxford in 1167, Harvard 1636, Sydney 1850 and so on and so forth would find that the modern experience is broadly the same beneath all the pizazz.

Students relocate to be near a community of scholars. They pay for access. The scholars read aloud. The students take notes. The scholars test the students and write letters of introduction that verify that the student knows the subject.

What has changed throughout that long string of technological revolutions -- technological revolutions that destroyed the universality of the Catholic Church in Europe, revolutions that destroyed feudalism and absolute monarchy, revolutions that totally upended the whole model of the entire world -- is almost nothing. Hardly any damn thing.

Given that the internet is new only in _degree_ and not in _kind_ , forgive me for being skeptical that it will do away with such resilient institutions.

</comment></comment></comment></comment></comment></comment><comment><USER11> I have wondered for a long time now where all the college tuition money goes.

Consider that: (1) the cost of tuition has grown FAR faster than inflation, and is now sky high, (2) A huge percentage of students need and get student loans, (3) student loans are one of the few types of liabilities that bankruptcy can't wipe out.

So "everyone" (speaking loosely) goes to college, and "everyone" gets a loan. The college gets its money up front, and bears no risk of having that money taken away. This represents a huge flow of money into universities. Where does it go from there?

How does college tuition revenue yearly compare to Apple's revenue from iphones yearly? In the case of Apple, I can see where the money goes ... Apple employs a huge number of people working on expensive ongoing operations.

What are colleges doing with all this money? Surely all of it is not being soaked up by overpaid administrators?? A money stream that large ought to have a big, obvious wake behind it, but I don't know where that money goes. Can anyone enlighten me?

<comment><USER12> Where does it go? Well, some universities apparently are tempted to put it in a nice slush fund. See the University of Wisconsin as one known example. [http://pjmedia.com/instapundit/167425/](http://pjmedia.com/instapundit/167425/)

<comment><USER12> More reports on one university so far discovered to have a slush fund, the University of Wisconsin:

UW Slush Fund Exposes Hypocrisy of Liberal Group [http://mediatrackers.org/wisconsin/2013/04/23/uw-slush- fund-...](http://mediatrackers.org/wisconsin/2013/04/23/uw-slush-fund-exposes- hypocrisy-of-liberal-group)

UW slush fund insults citizens [http://www.beloitdailyneaws.com/opinion/editorial-uw- slush-f...](http://www.beloitdailyneaws.com/opinion/editorial-uw-slush-fund- insults-citizens/article_5909b342-acf8-11e2-a407-0019bb2963f4.html)

Outrage grows as University of Wisconsin System admits it 'did not draw attention' to cash [http://www.jsonline.com/news/education/outrage-grows-as- uw-a...](http://www.jsonline.com/news/education/outrage-grows-as-uw-admits-it- did-not-draw-attention-to-cash-r59lh2e-204159081.html)

I wonder how many more state-run institutions of higher education will have similar scandals in the next few years.

</comment></comment></comment><comment><USER13> _Colleges have responded to the availability of easy federal money by doing what subsidized industries generally do: Raising prices to capture the subsidy._

Exactly.

This is an American problem though, the Europeans figured out many years ago that making tuition free for the students is a win-win for society. Everyone wants to go to college, so just make everyone pay for it, progressively, over time.

<comment><USER14> I was under the assumption that college is free for students in many European countries, assuming the student passed certain tests. If those tests aren't passed, no college. Is that not correct?

<comment><USER15> That is correct.

For example, if you want to study engineering you have to have certain math skills. Otherwise university won't let you in, no matter how much you pay.

Oh, _free_ college no. But tuition is something like $700 annual in germany/france. There is no housing or anything.

<comment><USER3> this is all and good for something like engineering where it is possible and maybe desirable to set some reasonable, objective standards. What about something like political sciences? Almost all of the upper-level public servants in a country like France come from the same school (ecole sciences- po)... Doesn't this become dangerous? Who gets to be the gatekeeper?

<comment><USER16> Well the entrance tests for Faculty of Philosophy that my close friend attended have been in the form of interview and they mostly asked about what authors did you read, what were they trying to say and such. It's actually not that complicated for people invested in some area (of science, politics, culture) to spot good candidates when you give them a chance.

Sadly, the quality of education in Czech Republic have degraded significantly in capitalism. We have gotten rid of Marxism-Leninism but have also let significantly more people in and set the incentives to produce a lot of graduates with much less knowledge, with rest of the bulk being thrown out after 2 years so that the university gets maximum state support.

We are probably going to end up with student loans and lower government participation, which will ruin our education system completely.

<comment><USER3> This sounds like a mess. So If I've read the 'wrong authors' (say, bastiat for a right-wing critique of the government and bourdieu for a left-wing critique), I'm basically hosed in the interview. Can't have any radical changers in the public service, now can we...

"the quality of education in Czech Republic have degraded significantly in capitalism... so that the university gets maximum state support."

I wouldn't think 'university gets maximum state support' to be a shining example of (free-market) capitalism. If anything, it sounds exactly like what the original article complains, if only a bit more direct and less sneaky than channeling the mechanism of subsidy through a convoluted, corrupt, and rent- seeking banking system.

<comment><USER17> I don't understand: there's nothing stopping you from reading what you want.

<comment><USER3> so, I'll read "Harry Potter" and "see spot run", and be entitled to a position at the school? There have to be _standards_.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER18> _> A 2010 study by the Goldwater Institute identified "administrative bloat" as a leading reason for higher costs. The study found that many American universities now have more salaried administrators than teaching faculty. _

It's not just the number of administrators but also the pay. They tend to make many times that of actual professors. And there's no question which camp brings more value to students.

</comment><comment><USER19> Any form of government subsidized education is immoral. The universe doesn't owe you an education, which, like all values, is inherently expensive, and must be produced. Neither does society; and neither do I.

This may seem cruel, but it's actually kind. Were it not for government control of education, education would have already undergone the same kind of massive transition that has happened to every free-market industry: much better, much cheaper, much more widely available.

If you want to see what happens to an economic sector as the government becomes more invasive, look at the finance sector, which is the most controlled sector in the US, or look at the "federal telecom bureaus" (AT&T, Time Warner and Verizon).

<comment><USER20> I really, really wish all objectivists had been born (actually) underprivileged.

<comment><USER21> It is better to take cognizance of how things really are rather than how you really, really wish them to be.

Where you start in life doesn't matter, but how big a jump you make in improving your lot does, and that's something that's entirely up to you. Please lay off the Rawlsian stuff - it doesn't square with the things I've observed in my own life. Nor does it square with the experiences of many of my friends that started life out poor, and chose to make something of themselves.

At this point, some might say, "you didn't earn your brain/parent's money/etc." I say that's nonsense, because the entire idea of "earn" arises in order to distinguish real people who _choose_ to act toward a goal from those that choose not to. To do that, we have to look at adults - lots of them, ranging from bums to billionaires, across time and professions, on and on. Then at young adults, then at small children (to fully see the contrast with adults). Choice is the crucial thing here: nobody could arrive at the idea of "earning" or "deserts" or "justice" by studying newborns or fetuses.

Relying on "earn" (which depends on the idea of choice and free will, as I've indicated above) in a statement intended to undermine the idea of choice and free will is bogus.

EDIT: I am making some serious points here. Disagree all you want in comments, but downvoting me reflexively does not refute my argument.

<comment><USER10> > It is better to take cognizance of how things really are rather than how you > really, really wish them to be.

> it doesn't square with the things I've observed in my own life.

> To do that, we have to look at adults - lots of them, ranging from bums to > billionaires, across time and professions, on and on. Then at young adults, > then at small children (to fully see the contrast with adults).

I like that you've indicated your own confirmation bias right there.

<comment><USER19> He's actually doing it the right way: looking at lots of empirical evidence and forming the right abstractions from it. That's how science is done, and it's how philosophy _should_ be done.

You are free to present contradictory evidence and conclusions if you think he's made a mistake.

He's not talking about evidence limited to his personal, one-on-one interactions. He's talking about all the evidence available to us, about people in society.

<comment><USER10> It is better to take cognizance of how things really are rather than how you really, really wish them to be.

Where you start in life matters. It squares with the things I've observed in my own life; hardworking people who spend every waking hour working for the betterment of their friends and family are nevertheless constantly battered by mischance and bad fortune. I've learned to differentiate the good luck that I possess in my own life from the bad luck that others have. Despite my continued work in supporting them with financial aid, useful contacts, and marketable skills, they are still forced by circumstance to end up destitute.

At this point, some might say, "you didn't earn your brain/parent's money/etc." I say that's nonsense, because the entire idea of "earn" arises in order to distinguish real people who choose to act toward a goal from those that choose not to. And of course, that is the point. None of these things were earned. This is a basic framing problem[1], and the entire basis of chaos theory[2]. Initial conditions matter. A proper scientific experiment includes a control group measured against an experimental group, and both of these groups are impossible to meaningfully distinguish at the beginning of the observation. If this is not the case, say because one group is more intelligent by some accepted measure of intelligent, then the measurement of a variable, say the application of choice to arrive at earnings, becomes utterly nonsensical. The causative agent could be either the initial difference or the experimental variable.

Using simplistic philosophical bases for one's beliefs is tantamount to unexamined religious dogma and in no way demonstrates any kind of empirical basis. Furthermore, claiming to have surveyed "bums to billionaires" without proof is merely, in the sophistic style of javert, a logical fallacy.

But it's cute you two believe you have any grasp of epistemology, rhetoric, or logic.

[1] [http://en.wikipedia.org/wiki/Framing_(social_sciences)#Exper...](http://en.wikipedia.org/wiki/Framing_\(social_sciences\)#Experimental_demonstration)

[2] [http://en.wikipedia.org/wiki/Chaos_theory](http://en.wikipedia.org/wiki/Chaos_theory)

<comment><USER21> _Using simplistic philosophical bases for one 's beliefs_

Whoa there - why do you call what I've written "simplistic"?

 _But it 's cute you two believe you have any grasp of epistemology, rhetoric, or logic._

 _And of course, that is the point. None of these things were earned._

That wasn't my point. Let me put it another way: it is not valid to apply the concept of "earn" as you do, in a situation where no choice exists. You can't say that a person has _earned_ his brain, nor can you say that he _hasn 't_. I've already explained why. This is not some minor epistemological issue that I'm going to simply let pass.

 _But it 's cute you two believe you have any grasp of epistemology, rhetoric, or logic._

Please stop with the insults.

<comment><USER10> > Whoa there - why do you call what I've written "simplistic"?

Because it does not appreciate complexities. This is the danger of confirmation bias; it reduces a complex reality down to a simple explanation by dismissing contrary evidence.

> You can't say that a person has earned his brain, nor can you say that he > hasn't. I've already explained why.

And you're wrong. Your central claim is this: "A person has earned any given X if and only if this person has made a choice." We can say "A person has earned any given X" is P and "A person has made a choice" is Q. Thus, the claim is a simple P <-> Q, which is shorthand for P -> Q AND Q -> P.

Your next claim, however, is "If a person has not made a choice, then this person has neither earned nor not-earned the given X." This _looks_ like ~Q -> ~P, which is logically valid, but it's not. The claim depends on a statement which is _completely new_. You've defined a third category of existence.

> Please stop with the insults.

Earn my respect. That's a choice on your part, right?

<comment><USER19> > Earn my respect.

This is supposed to be a community where we all respect each other. There is no excuse to be disrespectful to other people here, especially over disagreements about politics and philosophy.

If you want to be part of a community where you are free to be disrespectful to people until they "earn" your respect, you should go somewhere else.

I'm not even commenting on how you made fun of me and maxharris, which is kind of irrelevant because it doesn't support your argument intellectually. I'm only talking about your explicit assertion that you don't have to be respectful as part of the bargain of being in this community.

<comment><USER10> Such an authoritarian. If you want me to be respectful, then make a choice and earn that respect from me. What is this, a pg-subsidized respect space? That's immoral.

<comment><USER19> > What is this, a pg-subsidized respect space?

Actually, yes, yes it is.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER3> I'm surprised the absolute nondischargeability of student loans is not brought up in the article. This is, ultimately immoral, because that is exactly (fractional, voluntary, nonchattel) slavery.

The US, to Obama and congress's credit (those who know me know I'm not a huge fan) engaged in some credit reform in the wake of the 2008-2009 economic crisis. Credit cards firms were required to allow their customers to restructure their loans and became obligated to outline the loan payments afterward, and rotating credit became dischargeable in bankruptcy proceedings. One wonders why college credit hasn't undergone such reforms. Does academia have that powerful of a lobby?

</comment><comment><USER22> Good grief, I always forget how misleading the WSJ is.

"Gov'ment causes those poor helpless colleges to overcharge."

Bullshit.

The growth in the cost of college is tied to big fat administrator salaries. It is control fraud pure and simple. To paraphrase, the best way to rob a college is to run one.

<comment><USER23> So you're saying if we didn't have government guarantees on student loans or any other form of government subsidy, banks would just give 20somethings who want art degrees $200,000 completely unsecured?

<comment><USER24> > banks would just give 20somethings who want art degrees $200,000 completely > unsecured?

Of course they wouldn't. That's the idea though. Back-pressure. The idea is that it would then become unrealistic for institutions to try and charge 200k for an arts degree which no one can afford, even with loans. This should lead to lower tuition.

Bear in mind that that what any institution wants to charge is the absolute maximum the student can borrow, plus any government subsidies or fees they can extract.

<comment><USER25> The demand for college education is still elastic. College loans merely allow colleges to extract values greater than the student's ability to pay.

<comment><USER3> and since college loans aren't dischargeable, you get to keep the student in hock for the rest of their life!

</comment></comment></comment></comment></comment><comment><USER26> So.. We make it so that only people likely to pay the loan back get student loans. Then, we watch as the actuaries figure out that students frompoor families are the ones who disproportionately pay their loans (since they don't have the same familial support during and after college.)

So, poor students are disproportionately denied loans, and we continue to see generational wealth inequality expand.

There are no great solutions, and there are tradeoffs everywhere. However, leaving it to the market seems immoral too.

<comment><USER3> why is 'leaving it to the market' immoral? If you think the poor should be granted preferential financial treatment, then you should reach into your own pocket and help pay for them. Or help scholarship funds raise money. Or at least pressure your alma mater to set aside some funds for the economically underprivileged. If you don't do any of the above, then you don't really believe in it, do you?

It's also kind of insulting for you to assume 'the poor are the ones who won't be able to pay back'. In my experience (and it's possible that I had a unique college experience) it was the students who came from tougher backgrounds that buckled down, didn't get stupid degrees, or even if they did, managed to pull decent, well-paying jobs out of college, because they used their education, and it was the upper class students that were loafers or chose their degrees in a silly fashion, so in a way, the good education of the less privileged was subsidized by the poor choices of the rich.

<comment><USER3> should clarify, my 'unique college experience' was that I went to a fairly expensive private research university in the US; I strongly suspect that the waste of time in college cuts across socioeconomic lines at many big box state schools.

</comment></comment></comment><comment><USER27> You should only be able to use the "principal" pun once per article.

</comment><comment><USER22> I just realized my earlier comment on the WSJ being misleading did not really clear.

He has the facts right but, to get published in the WSJ, he can't point at the real issue. It is same problems we have the with banks. These institutions have tons of hidden subsidies. They always will. Subsidized industries behave differently from non-subsidized ones in general but the WSJ approved solution is to pretend that it is possible to remove the subsidy.

For public goods (utilities, parts of finance, education), you have to have government intervention and deal with the side-effects. That means regulation and rules about what you do with the money.

I don't think the author would disagree when I say that the administrator are stealing from the taxpayers. But what is 'bullshit' is saying that we just have to accept it. When people are stealing from you, you deal with it head on. If they were employees, you would fire them.

I don't have a great answer for colleges but you need to at least let the administrators know that they are acting immorally. It is not the loan process that is immoral -- it is the people who are running the colleges.

</comment><comment><USER28> College is an interesting situation. It might be argued that collectively we all have an interest in seeing a more educated society, regardless of the earning power. It might also be argued that collectively we need to focus more on not "going to college" but instead focusing on things like "learning a sustainable trade".

And, for an individual looking to maximize earnings & employment, a bachelor's in STEM/business is still a really good buy, supposing you didn't go to $$$ SLAC/Ivy League. Whereas subjects that are less fiscally shiny lead their students into a dark hole of debt.

One solution is to simply collectivize the cost: everyone gets free college. That's really expensive and without good cost controls, well, is susceptible to being taken advantage of.

Another solution is to go at colleges with the dieting plan. That's the current one. IMHO, that's exactly the wrong way to go about it.

Another solution is to aggressively force state schools to cut overhead; i.e. cut the administrator staff. Well, asking people to fire themselves is sort of utopian, doncha think? :)

\---

I don't really buy any of the solutions that I currently know about in the US. My thought is that there are far too many colleges in the US. Too many states trying to stretch too few dollars over too many campuses. I instead think that federal funding should go to a select few - perhaps three or four - in the US. These colleges would be fully free and fully funded by the feds. Professors would be encouraged to congregate there and focus on having _huge_ departments where all sorts of research could go on. The overhead per college is so high; it's considerably more scalable to focus on having a few large campuses than the small ones everywhere. Each campus requires a mini admin to be set up. Worse, the larger a department is, the more interesting collisions can happen: small departments work against this by not having that interesting person to run into(they are in the other state).

Anyway, that's my undercooked idea to help college education.

Education in the US is, I think, a wreck, and it has a variety of causes. Among them, the historic anti-intellectualism of Americans, the loss of historic mission, the rank foolishness of levelling egalitarians, the shrinking dollars for defense research, etc. More causes could be added.

<comment><USER29> About collectivizing costs, you speak as if it has never been tried and you discount the idea based on your own assumptions that it would be too expensive? As someone who has gone to university both at a free*(you actually get money to study) (~top) Scandinavian university as well as a (~top 10) US university, there is not much difference in the actual education from what I could tell. Where there's a real big difference though is research. US universities consistently outspends and outproduces European universities by a large factor. My theory is that the undergrads are paying for the professor's research and grad students, but I'm not sure. The money has to come from somewhere. Either way, neither I nor my parents would ever have been able afford US tuition + living costs. I took a loan for all the living costs that were not covered by subsidies, a loan with an interest of about 1.9%, so I can't even complain about that. My point is that in the US, I would probably not have been able to afford a decent college, and I think that's a broken system. Research and fancy campus gyms should not be funded by stepping on poor undergrads.

By the way, it also seems that student loans for US students mainly consists of tuition payments. There is rarely any mention of livings costs which I find curious. Do people just take it for granted that the parents will pay that part? Myself, after 5 years, I was roughly $50,000 in debt, all of it living expenses (note, that's only $10,000 a year in a country with high cost of living). My family didn't (and didn't have to) pay anything to put me through college. From what I can tell, this is the norm here, students take loans for their living expenses and parents are not expected to support them.

<comment><USER28> Traditionally[1], I believe US universities have been funded by the government to a large degree. The defunding of the universities I believe corresponds to the ceasing of the Cold War and the dropping of heavy DoD/DoE funding.

My understanding of collectivizing costs is that it's proved extremely expensive for countries such as Germany and the UK, which are moving towards a paying model (last I heard). There's also a pyschological effect when you're paying for your own way vs. someone else paying for it. I'll let someone else more learned in physchology & motivation research comment on the details, but the change in mindset does exist. My gut feeling is that its entirely reasonable for society to generally pay the bulk of the cost of college in exchange for getting the benefit of an educated society.

If you examine the historical cost of education in the US, the tuition began its upwards run around 1980 and has not ceased. So did healthcare. I don't know if there's a connection; and, if so, why. I do know that educational costs have gone wildly up beyond inflation.

To your aside; my student loans were designed to cover the cost of housing & life in general.

I'm very sorry that you did have to go into such steep debt for college. I don't think it's right that higher education costs so very much either. I do want it reformed, but I don't want it done in ways that simply funnel money into someone's pockets without lots of people getting a quality benefit.

[1] between 1945 and ~2000

<comment><USER29> You don't need to feel sorry for me, that loan is a government loan that every student is eligible for. The interest is 1.9%, so the government is actually losing money on it. I pay roughly $900 a year in interest, which is very manageable with a near 6-figure salary that this education gave me.

I think it's a pretty good system, it's good both for the students and their parents. Students don't have to rely on their parents (if they are rich enough to support their children anyway) and parents don't have to save up for years for tuition and living expenses. It still hurts though, it's not like it's free to get an education even without tuition and with subsidies and favorable loans. You also have to think about opportunity cost, you could be making money for 5 years, but instead you're in school. So I don't think you need the additional expenses to feel motivated.

</comment></comment></comment></comment><comment><USER30> Why not just cap the annual payout of federal loans to a reasonable level? This would surely put downward pressure on tuition.</comment>


Ask HN: What made SQL databases so popular? - <AUTHOR>

I propose a persistent storage solution for ANY idea to a traditional technical community.  Immediately, a SQL relational database is proposed as the solution.  I suggest other solutions.  I'm sad now because everyone is telling me that I MUST implement it in SQL otherwise I'm wasting my time.  According to them, once SQL stops handling the solution, then can I look at the other "eccentric" solutions.
I'm trying to figure out how did SQL get to be the de facto solution to every single persistent data problem out there?  Data can be stored in so many ways but a SQL relational database is pushed by so many people that their voice can be overwhelming at times.  Why is this?
Added Note: I feel sad for the SQL language since it is one of the few ways many programmers learn about declarative programming.  With that being said, this SQL language is different from the more commonly used form of "SQL" which is the nickname people call the well known SQL "relational" databases <comment><USER2> Quite honestly i often do not understand the extreme focus HN puts on NoSQL databases, as if it would be the new solution that fits every problem. The point is that SQL solves a lot of problems rather well and every NoSQL database has its own drawbacks. In the end you have a lot of NoSQL databases that solve some specific problems particularly well but have major drawbacks in other situations. SQL databases tend to fit a lot of solutions quite well, they are tested, stable, do transactions, can even scale and (for me) most important: SQL is a standard, the query language is well understood and documented, it is relatively easy to switch between SQL databases. But it's for practical purposes impossible to switch between a given set of NoSQL databases, let alone you need to learn a lot of new things or need another set of developers. Recent example in my project: I had a problem that did fit particularly well into Redis (small, self contained writes) but i also needed good querying capabilites. So after fiddling around with Redis and trying to make it work i had to abandon Redis. Next choice would have been: MongoDB or MySQL. The project already uses a MySQL server and i am easily able to replace MySQL with SQLite, Postgres or Oracle by using an ORM (SQLAlchemy). So the choice was easy. MongoDB would have been the worse choice but a possibility.

I'd wish someone would come up with a common query language that could be reused on different NoSQL databases, that would make NoSQL much more an option. UnSQL[1] looks abandoned :(

So after all: Every problem has its solutions, every program a good fit of a database. Either NoSQL or SQL, don't just point to some NoSQL database because it is the current buzzword, but ONLY because it can solve a problem better then a given SQL database. Then it doesn't matter if you call it SQL or NoSQL. Your NoSQL solution simply needs to redeem the benefits of SQL (standardized query language, transactions) with other benefits.

[1] <http://www.unqlspec.org/display/UnQL/Home>

<comment><USER3> I think the same applies to quite many new innovations on our field. They look interesting, because they make things simple. Things that used to be very complicated.

The problem is that in some cases there are reasons why the things are complicated. Quite often it is easy to dismiss these issues during the initial crush with the new technology. "Sure I can live without transactions, why would I need to enforce schema for my objects, who needs type information in language"

Once the initial crush is over, people start slowly adding features to make the shiny-new technology look more like the old stuff. Enforce JSON schema for Mongo, add lightweight type information for Javascript functions etc. Usually the thinking goes we are just adding these few good bits of structure from the Old Technology, surely we are not going to replicate the bloat and complexity.

</comment></comment><comment><USER4> "Democracy is the worst form of government except all those other forms that have been tried from time to time."

Relational DBs & SQL aren't perfect, but they've proven better than 30 years of attempts to replace them. I think the burden is (rightly) on you to explain why your alternative is better. There are applications where other approaches are appropriate (e.g. web search); if you can't explain why something is better solved using a non-relational system, then perhaps it isn't.

</comment><comment><USER5> Short answer: SQL databases were easier to use for developers and non- developers than other client server relational database systems in the late 80s and early 90s (dbase derivatives, btrieve and ctree were common then). Tools like Btrieve and ctree were largely libraries, and really didn't have the ad-hoc query capability that SQL has. Dbase suffered from incompatible implementations... so an application written on one Dbase clone wouldn't work exactly right in another. Old school client-server databases like Btrieve and ctree were extremely tightly coupled, and there was little separation of concerns between storage and logic. Lots of messy, messy code... and writing the storage and retrieval parts of your application were non-trivial. Since end users had no way to do their own queries (outside Dbase), most reporting was written in C, pascal or even BASIC. There were a few reporting tools like Crystal Reports that tried to solve the reporting problem, but it was often just as easy to roll the report in C...

When the internet took off, you had MS-SQL server on Windows and several open source SQL servers on Linux (MySQL, mSQL and Postgres) that freed web developers from having to write lots of custom data storage/retrieval code. A singe SELECT or UPDATE command did what 150-200 lines of C could do. It also meant that web apps could go to market without advanced reporting capabilities or CRUD for detailed configurations because you could always just roll an SQL query to deal with that later.

SO, here is what you are up against in selling noSQL vs SQL:

* Old developers who lived through the 80s and 90s database hell and see SQL as the thing that freed us.

* An extremely mature toolchain vs. new applications without the toys needed by non-developers (reporting, ad-hoc query for end users, integration with spreadsheets, etc...)

* Familiarity with the query language. Lots of devs know SQL

* Risk of something new.

* Fear of having your job turn into writing reports constantly because no one else in the company can.

SO if you want to be persuasive in selling a noSQL solution:

* Define the problem clearly and be prepared to show why Mongo, Couch or whatever is the right solution.

* Avoid arguing that SQL isn't relational. You might be right, but Oracle, IBM, Microsoft, Ashton-Tate, Sybase, Progress and everyone else with an SQL product has outspent you on marketing.

* Be ready to deal with questions about reports, ad-hoc queries, stored procedures and security. Each of these issues is solved and documented in mature SQL based systems.

* Giving Sally in accounting access to a unix command line is not going to be as well recieved as giving her access to Excel, Access, MS SQL Management Studio or Navicat.

</comment><comment><USER6> In no particular order:

1\. Transactions are not implemented in most of the "eccentric" solutions

2\. SQL solutions are more battle tested.

3\. The relational model fits most solutions quite well

Where the relational model doesn't fit or transactions are not required, then it may be worth looking at other solutions.

Polyglot persistence is also another moving part too, it needs time, expertise and infrastructure to deal with, so even if a non relational model makes sense and there is currently no need for transactions. Who knows when transactions would be a good idea?

<comment><USER7> If it doesn't have transactions it's not a database.

</comment></comment><comment><USER8> SQL is popular because for many classes of problems it _is_ awesome and uses a very powerful abstraction (tables) and query language. Most implementations are also very robust and able to provide very interesting guarantees (like ACID).

The only problem of SQL was that the computer industry was trying to solve everything with SQL databases, that is not a good idea.

It's good that there are other databases now, but I don't think SQL is going to face away. For instance many NoSQL databases like MongoDB, RethinkDB, may well be SQL databases with different implementation tradeoffs but just picked a different query language because... well because... I don't know why honestly. Not that's a problem but the model is very very similar to objects with fields (tables) I can query.

</comment><comment><USER9> It's the de facto solution because for most use cases it is a valid solution and for many it is the best solution.

If it wasn't developers would leave it in the past like any other obsolete technology. _Especially_ today technology competes on merit before marketing or anything else.

</comment><comment><USER10> My group developed the latest, fastest NoSQL system on the scene (<http://hyperdex.org>), one that provides ACID transactions as well as a ton of other exciting features like scalability, consistency and fault-tolerance.

Yet when someone asks this question, I usually respond with "try a traditional RDBMS first."

The simple reason for this is that there is additional, implicit information in the question: the fact that someone is asking it indicates that they are not up-to-date on one of the most exciting revolutions taking place in data management. Such users will require extensive hand-holding, and there are a lot of SQL-related resources out there that can help them with this.

But there is a twist: what people are doing is kind of like how some religions have to turn you away before you become a true convert. Will an RDBMS user be completely happy? Perhaps, but if the application is at all demanding, they will wake up in the middle of the night because of a wedged DB, running out of internal table space. Or have to frantically hire DBAs to "optimize" their queries. Or hand money hand over fist to Oracle. Or discover that an AWK script is faster than their multi-million dollar RDBMS installation.

So it's not like people are turning you away from NoSQL altogether -- you will end up with a NoSQL system eventually, they're just making sure that you do this for the right reasons.

</comment><comment><USER11> I recommend this book: <http://amzn.to/13NgjU9> (affiliate link), which I bought for inspiration on how to build my own database from scratch and finally put down, after reading several chapters, with a reasoned appreciation for the way things are.

To simplify, the SQL data model exquisitely balances:

* expressivity - queries involving both GROUP BYs and subqueries, which I've needed more than once, are challenging at best to translate into Mongo's query model, which is one of the most expressive outside SQL

* speed - as long as your query can run on a single machine, SQL query planners are the best, period. Other models tend to be more horizontally scalable, but this was not a priority for most of database history, nor for most real-world situations.

* compactness - the on-disk overhead of a SQL database is fairly limited, about 40-50% in practice. Obviously, a flat file has even less overhead (<10%, generally), but some non-SQL systems consume storage willy-nilly (it's not uncommon to see XML overheads surpass 500%, and Mongo overhead hovers around 90-100%).

* robustness - in the SQL standard, "undefined behavior" is kept to an absolute minimum. In particular, transactions are invaluable anti-Heisenbugs, but cascading rules, sanity constraints, and fixed schemas also reduce production gotchas (at the cost of development agility, of course).

<comment><USER12> If i think about what scares me most about using a noSQL database, it's the loss of expressivity.

I believe noSQL databases can be fast, compact, and well-behaved. But SQL is a phenomenal query language; it anticipates Alan Kay's (alleged) maxim that "simple things should be simple, complex things should be possible". Not only does it make complex things possible, but modern query planners make a decent fist of making them fast, too. NoSQL databases either don't support rich queries at all (eg Riak, which is in all other respects rather wonderful), or have substantially less power (eg Mongo). Even databases which are queried with map-reduce operations using functions written in real programming languages (eg CouchDB) are lacking compared to SQL, due to the lack of anything like joins, subqueries and so on.

(The only kinds of noSQL databases that have something approaching SQL's power are graph databases (eg neo4j, whose Cypher query language is really rather nice), some object databases (eg ObjectDB, which supports JPQL), and some of the pre-SQL options like Pick databases. However, these are rarely the ones which get the attention, and it's notable that their data models are actually not far removed from the relational model.)

The reason this flexibility matters is because it supports change in the future. I might be storing and querying data for some purpose now, but it's possible that a year down the line, i will want to do something entirely different with it - usually yawn-inducing things like reporting and auditing, sometimes just implementing new features that look at the data in a different way. The expressiveness of SQL lets me be confident that i will be able to do that with my relational database. With a noSQL solution, i run a substantial risk of not being able to do it, and so being forced to export my data into another store to do it, with all the headaches that that entails.

</comment></comment><comment><USER13> You may find [http://en.wikipedia.org/wiki/Database_management_system#Hist...](http://en.wikipedia.org/wiki/Database_management_system#History) informative

</comment><comment><USER14> Here are some good reasons for it: [http://stackoverflow.com/questions/3756100/how-did-sql- becom...](http://stackoverflow.com/questions/3756100/how-did-sql-become-the- dominant-database-language)

In particular, you can read this book: [http://www.nap.edu/openbook.php?record_id=6323&page=159](http://www.nap.edu/openbook.php?record_id=6323&page=159)

</comment><comment><USER15> Relational databases provide a _framework_ for structuring data and retrieval. And it is an approach informed by decades of practical experience and, well, plain mathematics.

Would you try to program a web application without a framework? Most of us do, at least once or twice in our career, not thinking about the next guy who needs to maintain our POS spaghetti. Perhaps if you're very bright or experienced you could do a good job structuring a web app from scratch, but in effect you would writing your own framework. Still it makes sense to depend on and learn from the great web frameworks -- rails, django, etc. -- so you can think about getting your real work done.

Same goes for sql. It is structured to keep us from being an idiot.

Stand on the shoulders of giants.

<comment><USER16> Your analogy with a web framework is a good one, but it extends beyond that into growth of application complexity.

True, my giant sprawling complex web application appears that it should have been done in a complex framework, little does anyone know it started out 5 years ago as a single perl CGI formatting the output of "df" on a webpage...

A similar thing can happen with non-relational databases... are you willing to bet your entire dev budget and/or the company on NEVER EVER needing features in the future? Sometimes thats OK, usually not.

</comment></comment><comment><USER17> You seem to be asking this question in a vacuum, which makes it impossible to address the costs and benefits of relational vs. alternative approaches with any context. What types of problem are you trying to solve? What alternatives to an RDBMS are you proposing to solve them with?

Having said that, I write transactional business applications (often dismissively referred to as "CRUD Apps), and in that context an RDBMS is a sensible default, but not for every part - so on Google App Engine for example I'm using CloudSQL for the business data, but Google's data store for the session data - because that has very different characteristics.

[1] I'm reading "SQL" as shorthand for relational, although the two are hardly synonymous.

</comment><comment><USER18> Relational databases have a sound mathematical model. You may not think of it this way, but a table can be seen as a _predicate_ in logical terms. You can frame most logical queries as a chaining of predicates and functions, and you can create arbitrary (finite) predicates using tables. That's powerful.

NoSQL came into style with the web, in which you have large data sets but don't need most of the relational features. You might have a trillion Wuphfs, but you'll never need ad-hoc queries because you're doing mostly CRUD on the Wuphfs. Sharded SQL is a painful way to deal with that problem space. NoSQL systems are often better adapted to it. NoSQL is almost always _less powerful_ abstractly but easier to scale.

The major problem that I've seen with NoSQL systems is that they tend not to handle the edge cases very well. Remember the old joke about the man who goes to the doctor and says, "It hurts when I do this", so the doctor says "Don't do that". Sometimes, you have to "do that". Many NoSQL products are also quite low-level, requiring that more traditional database functionality be written in client code. That gets nasty-- slow, flimsy, hard to coordinate. You don't want, for example, clients to be responsible for generating sequential unique IDs because the clients all have to agree on what the rules are, in addition to generating a lot of communication overhead each time a new block of IDs is requested.

Furthermore, if you're using NoSQL to build something that will be marketed as a database product (e.g. within your company) and are exposed to requirement accretion, you'll find that the requirements are often inspired by expectations derived from relational models: ACID transactions being the big one.

I am not saying that SQL is the best for all problems, but it is a damn good technology. Personally, I think that a lot of SQL hate comes from the limitations of MySQL. If that's the source of your headaches, look into Postgres. Postgres is surprising in how modern and powerful it is.

<comment><USER10> >NoSQL is almost always less powerful abstractly but easier to scale.

This is no longer true. We now have second-generation NoSQL data stores that provide strong consistency, fault-tolerance and ACID transactions, based on the sharded NoSQL architecture that scales easily.

<comment><USER19> Just out of interest, how are your opinions on the superiority of NoSQL systems received by

<http://www.cs.cornell.edu/johannes/>

and

<http://www.cs.cornell.edu/annual_report/99-00/Demers.htm>

?

<comment><USER10> All I can say is "some of my best friends have worked on RDBMSs." I don't hold it against them. ;-)

More seriously: Al, Johannes, and many other notable researchers in the DB space have complex, nuanced opinions on NoSQL systems. And these opinions are being revised daily as new information comes in; we've seen some researchers go from decrying the whole NoSQL movement to becoming some of its fiercest proponents, seemingly overnight. I would do them disservice if I were to try to present them here.

So I'd encourage you to ask them directly. Or perhaps we could have a panel at a conference. That'd be fun if done right.

</comment></comment></comment></comment><comment><USER20> Nothing else offered Codd's relational model of:

0\. simple user commands

1\. data independence (from hardware and software implementation)

2\. automatic navigation (instead of single record access).

The rest was competitive history with the giant of the time, IBM, who created and used SQL in their relational System R product. SQL variants rapidly became commercial successes, with the earliest external adoption and most notable being that by Oracle.

In time, SQL became a standard and, as a relatively simple declarative language (the HTML of data if you will), it is easy to learn and widespread.

</comment><comment><USER21> This is just a personal opinion. But as apps scale you run into concurrency problems. You want to maintain a reliable consistent state but due to latency and resource limits that is difficult to do within your own code. So the responsibility is moved onto another system designed for that purpose. NoSQL simply solves the same problem in a slightly different way. The consistent state is allowed to be abstract and doesn't need to be transactional.

The real dichotomy is between handling state yourself in code and letting outside software do it for you.

</comment><comment><USER22> Whoah whoah whoah. SQL IS NOT RELATIONAL. Let's put that to rest right now. You can have relational databases if you like, but put SQL in front of it and now you don't have a relational database system anymore. you have an SQL database. This is because SQL actually violates a few of CODD's axioms for what constitutes a "relational" database. The tragedy of NoSQL is really the setting up of this dichotomy between "SQL/Relational" and "NoSQL/NotRelational" which ignores the possibility that "SQL CAN'T BE RELATIONAL" and "Relational with no SQL"

<comment><USER12> I don't believe this is correct. SQL certainly includes some things which don't exist in the pure relational model (LIMIT, stored procedures, array types), but i am not aware of anything in SQL which prevents you using a pure relational model. SQL is not a pure implementation of the relational model, but it's close enough that is's reasonable to talk about SQL being relational.

<comment><USER22> 1\. The existence of a null value (3-value-logic) 2\. permitting duplicate rows 3\. permitting duplicate columns. 4\. "outer" joins.

and.. other things, that you can enforce in your client application, but then that rather defeats the whole point of having a "relational database management system". you might as well just have a dumb key-value store, if you have to enforce so much of what a "relational database" is in your app.

note that not even the SQL standards use the word "relational". anywhere. Years ago I couldn't believe this, but go ahead and look. It's not there.

</comment></comment></comment><comment><USER23> The majority of HN won't like me for saying it .. but the major factor in NoSQLs "popularity" is not needing to learn SQL. Which is a crying shame, as it's one of the corner stones of computer science.

</comment><comment><USER24> Relational DBs work for processing of business transactions -- when you need to get the amount of money and all the other details exactly right, when you need to retrieve the right record according to a variety of criteria which may come from other types of data, and when you need to be sure the info is inserted in exactly the right format.

For this, there are DB transactions, SQL SELECT with joins, and schemas.

Much of this is possible in other technologies, but RDB does it well.

</comment><comment><USER25> Simple. Same reason people code in C or Python or Bash. And not C#, Ruby, ZSH, or much worse: their own implementation or whatever: It standard. Everyone knows how it works. It's proven.

So unless it's really not good enough, diversity for the sake of diversity is the opposite of being efficient.

Hence, you're asking the wrong question. In fact, you shouldn't ask a question. Show or prove your solution brings a REAL benefit. If you can't, you're indeed wasting your time.

<comment><USER26> Perhaps it's standard, but implementations vary so wildly that I doubt this is of any importance. MySQL vs PostgreSQL vs Oracle, it's just madness.

</comment></comment><comment><USER27> Tools. All data processing tools right from analytics to integration need to work with the datastore. RDBMS has accumulated these tools for so many years. It is not a level playing field for NoSQL to beat that ecosystem.

</comment><comment><USER28> Direct answer, SQL is the only thing that learnt at schools.

</comment><comment><USER29> \- Codd's relational model

\- ACID

\- SQL

\- Chen's ERM

</comment><comment><USER30> What exactly are you trying to store? Depending on what you are persisting, SQL may make perfect sense.

</comment><comment><USER16> Amazing no one's quoted the inner platform effect.

<http://en.wikipedia.org/wiki/Inner-platform_effect>

Hmm, SQL query too slow takes 10 seconds for one query. I know, I can use a non-SQL solution. Each query will take 100 milliseconds. Whoops turns out the users need all the relational "stuff" for my reporting and also in my CRUD to keep it consistent enough. I mean, everybody knows databases are consistent and codd normal form and relational, and nosql is a database, therefore when the users spec a database they'll be fine either way? Well no big deal I'll just write my own JOIN. After all, I'm as good of a programmer as the team at Oracle who's been working on JOINs since before I was born, so I'm sure a couple cans of red bull and I'll have something better. Whoops, each "nosql" query does in fact take 100 milliseconds but now I need to run 1e6 of them and shove them into my homemade JOIN routine, so I've now "improved" my total overall system time from 10 seconds to 10 kiloseconds. Whoops.

I've never been able to implement a nosql solution at work because I always get stuck in inner-platform tar pit, or massive consistency issues making the CRUD a nightmare, etc. Usually if you complain about that in the nosql community you get pretty intense complaints that your business is doin' it wrong, lol at a paying gig thats not going to work very well... I'd like to try something new but I haven't had the business case for it yet, or rephrased a deep enough analysis has so far always excluded nosql solutions and that Might indicate a positive analysis isn't deep enough. Hear hooves, think horses not zebras. I've bought the "seven databases in seven weeks" book and its an excellent introduction unfortunately its something like a guided introduction into smooshing up against the business requirements limits of each DB as much as a technical intro.

Another point is the original request was "persistent storage" not why does nosql not work most of the time. Back in the olden days we called that a "filesystem" but lots of $5/mo webhosters don't allow that type of thing, but they do give you mysql... Also its hard for the PHP coder types to not shoot themselves in their foot with filesystem access, for example, allow the user of the order.php form to store their picture in username.gif what could go wrong other than lack of input sanitizing and some goofball is going to register the username order and upload a "picture" with the filename order.php that actually does something rather naughty, etc. Even if it works, next week, the boss asks to store two image files now, one full size and one icon size now. Hmm the filesystem soln doesn't scale... Rather than all that input sanitation, wouldn't it be simpler to just dump it into a blob column in a mysql table?

</comment><comment><USER19> "I'm sad now because"

Why would you be _sad_? SQL databases are just a tool, and happen to be the _right_ tool. Why does that make you _sad_?

It's like saying "I wanted to eat soup with chopsticks and everyone told me to use a spoon instead. That made me _sad_."

</comment><comment><USER31> Woaw. HN, the place created by pg, who advocated trying to "beat the average" and have "frighteningly ambitious startup ideas", etc.

Yet HN has become the place for the status-quo. Where people are going to vote everything encouraging the status-quo (like articles about the benefits of C# and SQL) as if it was the holy gospel...

 _"Immediately, a SQL relational database is proposed as the solution."_

For a start you're probably not trying to solve a problem involving big-data ; )

Ask the big data players who are using _fast_ and _gigantic_ key/value stores how did SQL work for them...

For smaller needs, then SQL offers, basically: ACID. Most companies do not care about what SQL can really do: they'll wrap everything into ORMs and perform business logic outside of SQL and in crazy models that aren't relational at all and then store back everything into SQL.

What most business care about is ACID, not being truly relational (which SQL DBs ain't anyway), not set theory.

The most saddening thing about traditional SQL DBs is that there's no basis for queries: the same query shall return different results depending on when the query is made (because most SQL DBs are "update in place" DBs / value- oriented DBs).

In the Real-World [TM] this is _very_ problematic because when you can't reproduce a problem you have to try to reproduce the _state_ the DB was in at a particular time and query from there. It is a gigantic time waster.

In my view for really big data needs SQL simply doesn't cut it while for smaller needs I still want ACID but I want something CRA ("create read append- only"), not CRUD.

I have nothing against using something like Datomic (which is CRA) backed with a SQL datastore.

But the "traditional" SQL DB that the herd --bent on never creating anything disruptive-- is using? Not for me.

CRUD is dead baby. CRUD is dead.

There are, today, people who've read and understood pg's writings who are using Lisp dialects (like Clojure) and CRA DBs (like Datomic, backed or not by SQL) to gain a competitive edge.

 _This_ kind of stuff is the reason why I come to HN. Not to read why the status-quo is acceptable (even if it's all most of the HN crowd knows).</comment>


Charon's Surprising Youthful and Varied Terrain - <AUTHOR> https://www.nasa.gov/image-feature/charon-s-surprising-youthful-and-varied-terrain

<comment><USER2> [https://news.ycombinator.com/item?id=9893412](https://news.ycombinator.com/item?id=9893412)

<comment><AUTHOR> It is a different article tho.

<comment><USER2> But clearly the same story, and already on the front page.

<comment><AUTHOR> Wasn't when I submitted it. Sorry about that.

I insist about the story beign different. This story is about Charon, and the very first picture of its surface.</comment></comment></comment></comment>


2 Days Trying To Create a Multiplayer HTML5 Game - <AUTHOR> http://marcgg.com/2013/01/14/html5-nodejs-websockets-games-learning/

<comment><USER2> Very interesting to read! I guess I'll have to try my hand at multiplayer HTML5 game as well some time, because even reading you, I believe you, but at the same time it's hard to believe that it's hard like that :)

Care to elaborate a bit on the part where you say communicating the entire board state was too much data? Looking at your game, I don't see how the board state could be much more than 100 bytes or so? Was it the JSON that bloated it up?

If I were you, I'd do it like this: split the state into a static and dynamic part. Player's names won't change during the game. Or if they might, maybe even better to split the state into a high and low latency part. If you see a Player's name update a second too late, it's no big deal, so communicate that via a different, slower, channel. All the other data, the board state, player positions, scores, you don't need JSON if you're transmitting the same fields every time. So make sure everything's a ranged integer and squeeze them together either using a base64 library, or maybe just simply Number.toString and ParseInt with radix 36 (max natively supported radix encoding in JS, and still 5.17 bits/byte), or maybe you hack Unicode/UTF8 to translate integer code-points into bytes and vice versa (you should probably go for the latter, because it'll be _awesome_ ).

Oh and then, a tip, when you wrote:

> I added some rules and gameplay elements to make it feel more like something > one would like to play, including: Feedbacks when you get hit

I was reminded of this really cool presentation about game-design and "juice": <http://www.youtube.com/watch?v=Fy0aCDmgnxg>

As you can see, all those tricks are "merely cosmetic", but they truly do make the game a lot more fun to play. The second most important thing, almost all of them are simply transformations to the "view" of the game, so they can all be done client-side, cost nothing extra network-wise (apart from loading initial extra gfx). But the first most important thing is that all these extra tweening functions and particle animations help cover up any accidental lag or latency! If you got a bare bones squares-on-a-grid game, any lag and stutter will be super obvious, because that's the only thing to see. But if everything is bouncy and wobbly, and there's explosions and smoke flying all over the screen, those things can keep moving and animating because they don't depend on server information, while the game engine waits for the next data packet, and the player won't suspect a thing. It similar to "suspension of disbelief", in a way.

Please do post follow-ups if you continue developing this game!

<comment><AUTHOR> The board was not very big, but since every movement would broadcast the entire board to all players, it would add up. I didn't bench exactly how bad it was, but as I was adding data to it I could feel the game slowing down. You have to keep in mind that at that point every movement would make the server send the whole board, and in that game you'd move constantly.

Thanks for the tips. I don't think I'll continue developing this game, but if I get some time I was thinking about doing something cross devices (iPhone / Desktop). I don't know how yet, we'll see.

</comment></comment><comment><USER3> I've actually been working on and off on a multiplayer HTML5 game for a while now.

<https://github.com/stevenleeg/thegrid2_client> \- for anyone who's interested. It's a pretty simple brand of RTS, so a lot of the realtime code could be used for anyone needing a reference for creating their own.

</comment><comment><USER4> That's cool, keep up the effort.. I've been working on multiplayer game for the last 6 months. I use HTML5/JS, WebSockets, WebGL and node.js (I rewrote my server 6 times while flicking between node and Java, but eventually stuck with node.js as it was frustrating swapping between languages when developing the client and server at the same time)

My game has real time twitch-based combat, meaning you use the controls to directly move your player around and shoot at 30fps+. This was a real pain to build with TCP as I have to have a lot of client side prediction and lag correction to handle packet resends. I can't wait until UDP is more widely available with HTML5's DataChannel: <http://www.thrupoint.com/2012/11/webrtc- data-channel/>

I've gone with my own VPS (around £5 a month) but will probably move over to AWS if I get further with the game.

</comment><comment><USER5> FTA "My first version was hosted on Heroku and lagged like crazy."

Hmmm even on heroku you should have pretty damn good responsiveness as I __assume __it should be using what is known as "COMET"<http://en.wikipedia.org/wiki/Comet_(programming)>

The polling is just to reestablish the kept alive connection. Which while still less than ideal... I wouldn't think would be terrible.

Not saying the author is wrong but being that it seems like his first foray into socket.io it's a possibility.

<comment><USER6> WebSocket has much less latency and overhead than long-polling Ajax. Heroku just doesn't support it yet. The solution for now is to run your own WebSocket server on AWS or subscribe to a service like Pusher.

<comment><AUTHOR> That's how I understood it

</comment></comment></comment><comment><USER7> Interesting and timely. I've been thinking about getting into some simple game development as a side project. Is there a demo online somewhere?

<comment><AUTHOR> There is a demo, but it's a bit buggy and wouldn't survive the Hacker New effect ^^.

The code however is on github: <https://github.com/marcgg/websockets- experiments> It's VERY poorly written as I kept throwing code away and trying out stuff so don't look at it too closely.</comment></comment>


Most of the Mind Can't Tell Fact from Fiction - <AUTHOR> http://nautil.us//blog/most-of-the-mind-cant-tell-fact-from-fiction <comment><USER2> Previous discussion here (7 days ago): [https://news.ycombinator.com/item?id=20976567](https://news.ycombinator.com/item?id=20976567)</comment>


Ask HN: How and where do you keep your notes/references? - <AUTHOR> I've been trying to consolidate my notes, references, long term lists. I tried using software like OneNote, Evernote, iOS Notes app, etc but it seems to get very disorganized very quickly.
I see some GitHub links posted here that shows people using their Readme.md as a reference file for various things.
Then it occurred to me that I can start organizing my stuff on a local static html page.
But I am also wondering how everyone else here stores and keeps track of their "knowledge"? <comment><USER2> Have you tried searching here on HN for previous discussions of exactly this question? It comes up nearly every month (at a guess) although it feels even more frequent.

In particular, just search for OneNote and Evernote in both stories and comments and you'll find dozens.

If those discussions don't give you what you need, perhaps you could be more specific.

</comment><comment><USER3> [https://notion.so](https://notion.so) does everything I need it to

</comment><comment><USER4> isn't it more a problem of organization and modifying your habits and not one of tooling ?

</comment><comment><USER5> [https://tiddlywiki.com](https://tiddlywiki.com)</comment>


WikiLeaks is not a country. MasterCard is not a country. - <AUTHOR> http://www.cnn.com/2010/TECH/innovation/12/09/wikileaks.cyber.attacks/index.html

<comment><USER2> Some useful sanity. And a couple of good quotes:

"War isn't just nameless attacks between parties that are not nation-states to begin with," said Mikko Hypponen, chief research officer at F-Secure. "WikiLeaks is not a country. MasterCard is not a country."

And Bruce Schneier: "War. W-A-R. It's a big word," Schneier said. "How could this be a cyber war? It's certainly a cyber attack, right? It's certainly politically motivated. But this stuff has been going on for a couple of decades now. Do you mean there have been thousands of wars that haven't been noticed?"</comment>


Ask HN: How are IP addresses geocoded to latitude and longitude? - <AUTHOR>

There are a bunch of API services out there that are able to determine the latitude and longitude (and other data, e.g. state, city, etc.) for a given IP address. How are they able to do this? How does one know what part of the world an IP address is associated with? <comment><USER2> Good-ish article in this Stackoverflow article[0]. The short version is: Any website that has your location (e.g. city, address, etc) can sell that data on and do.

In the US in particular there are nearly no data protection laws (except healthcare records). So companies "share" data with impunity.

[0] [https://stackoverflow.com/questions/1996106/how-does-ip- geol...](https://stackoverflow.com/questions/1996106/how-does-ip-geolocating- work)

</comment><comment><USER3> First of all you can do a whois on an IP address to get where the company that registered is located. If you traceroute the address you can sometimes get hints on where it is. And then some big sites may use geo locotation services on their websites that query the browser/APP GPS where the user is located and then it may be concluded that the whole /24 net is within that area and then the site will sell this information to geo location services.

</comment><comment><USER4> You can download a DB with this info here: [http://dev.maxmind.com/geoip/geoip2/geoip2-csv- databases/](http://dev.maxmind.com/geoip/geoip2/geoip2-csv-databases/)

I integrated that into my own webserver so I get fun logs; there's a tingle when you see people reading your own posts after all.

You can also buy a bigger, better DB from the same people.

<comment><USER5> This is a useful piece of information but I think the OP is asking how it is technically achieved. i.e. how was the database created in the first place.

<comment><AUTHOR> Exactly. I am curious from a technical standpoint how a company/service is able to figure out the coordinates associated with a given IP address.

<comment><USER6> Usually they use a data provider/library like maxmind or a few others.

If the question is how someone like maxmind or others assemble that data it is usually done through a mix of ARIN, data aggregators and ISP data. ARIN will give which ISP and which country it is assigned to. ISP's generally have blocks regulated to specific areas of the country, e.g. mine is "RR-SOUTHEAST- BLK2", as I am on brighthouse, and that block has 67.8.x.x in it. And data aggregators purchase data from ecommerce and other sources to gather location data on individuals and their IP's. This is then aggregated, split and sold in many ways.

There are also some companies/startups working on GEO location to be able to target a person on any device based upon knowing the IP's they have used and some algorithms around the user-agent of the browsers etc. This means they can target you with marketing based upon who you are and your purchase history.

*EDIT: As for lat/long, ZIP codes in the US have a centroid lat/long as assigned by Tiger data (and others). So they will use that as a starting point, and if an aggregator has purchased your data from someone they may be able to get the lat/long even closer through the matching algorithms.</comment></comment></comment></comment>


Ask HN: Where to apply for SF/SV startup jobs? - <AUTHOR> What's the best way to find job openings nowadays without dealing with recruiters? <comment><USER2> At the top of Hacker News is a jobs tab, for YC funded companies. One the first business day of the month, Hacker News has a "Who is hiring" thread. Just search for the city names. Indeed.com, Simplyhired.com, Monster.com, LinkedIn.com are job boards that can be searched by location, try 25 miles around Palo Alto. Make a sign about 2 feet square with "Hire Me" and a less than 10 word resume. During lunch, set it on an outside table at a restaurant on University Ave, or in the Mission District. Subscribe to the [https://www.startupdigest.com/](https://www.startupdigest.com/) to get weekly email on startup focused events. Go hang out.</comment>


India to build cyber attack proof operating system - <AUTHOR> http://www.newdelhinews.net/story/695571

<comment><USER2> They're going to want to read Fred Cohen's "Computer Viruses - Theory and Experiments" (<http://all.net/books/virus/index.html>) and maybe some of his other papers, first. After understanding that paper, they might want to lower the bar a bit.

</comment><comment><USER3> Security through obscurity will not solve their (or anyone's) problems.

Also, someone claiming a system to be "cyber attack proof" is clearly misguided.

</comment><comment><USER4> Isn't that what the NSA tried with SELinux?

SELinux is a good thing, but it isn't attack-proof, and I doubt anything can be.</comment>


Ask HN: Any interest in a Salt Lake City or Provo, UT Hackers/Founders meetup? - <AUTHOR> I'm thinking about spearheading a Salt Lake or Provo Hackers/Founders meetup (depending on feedback from both areas) and wanted to gauge interest here first. If you are interested in coming to or helping to organize such a thing, shoot me an email (in my profile). <comment><USER2> Definitely interested! You might even be able to coordinate it with some local startup programs.

BoomStartup (<http://www.BoomStartup.com>) - I'm actually in this program right now.

The Foundry (<http://www.business.utah.edu/node/1203>) - I know some people in this if you're interested.

Launch Up (<http://www.launchup.org>) - They meet tomorrow in South Jordan

Web Startup Group (<http://webstartupgroup.com/>) - A BYU startup community

<comment><USER3> I have some connections to the folks that run BoomStartup, Launch Up, and the Web Startup Group and would be happy to help if needed.

<comment><USER4> Nate, you know everyone.

</comment></comment></comment><comment><USER5> I've enjoyed the hackers and founders meetup in mountain view. I really do enjoy how unstructured it is compared to Launch Up and the Web Startup Group.

Another good group is the CTO Breakfast hosted by Phil Windley at the Novell Cafeteria <http://www.windley.com/cto_forum>

<comment><USER6> What exactly takes place at these meetups?

<comment><AUTHOR> I've been to one in SF and one in Mountain View. In SF, people basically showed up, the organizer had pizza (which sponsors paid for - in that case it was WePay and MixPanel), and people just talked and hung out. Very informal. Very low-key.

In Mountain View it was the same story, except for the first 30 minutes or so there was a talk on the evolution of db technology and pros/cons of different platforms.

<comment><USER6> Thanks. I'm asking because I thought about doing an event in the Dallas area. Problem is, I have nothing to talk about :)

</comment></comment></comment></comment><comment><USER7> I don't know if I'd be interested or not. Maybe. It'd be cool to meet people but I'm not sure if it'd go well or be worthwhile. I'd probably be willing to try one out if it's not too far away (I'm in Sandy).

</comment><comment><USER8> As would I. I'm working on starting a BYU Hackers Club (in the Engineering department.). (I need a better name though... I don't think it will make it past administration.)

</comment><comment><USER9> This is awesome to see. I'm an SF'er now, but I had previously launched some startups in Utah. I hope you guys have a killer time.

</comment><comment><USER10> Thanks for the links! I will try to attend tomorrow. Semi-techy (non hacker) working on a few idea's :)

</comment><comment><USER11> I am totally down for something like this!

</comment><comment><USER12> I would be interested as well!

</comment><comment><USER13> Count me in.

</comment><comment><USER14> Yes.

</comment><comment><USER4> I am.

</comment><comment><USER15> yes.</comment>


Ask HN: Data scientists, how do you organize and store your data files? - <AUTHOR> <comment><AUTHOR> One problem I'm stumbling into is that multiple projects and experiments require the same source data, which may be several gigabytes in size. While it would be nice to have each project folder be self-contained, this isn't practical due to limits on hard disk size. Unfortunately my work place doesn't have S3, etc., and probably even if it did, keeping multiple copies would be costly.

I imagine this is a common problem. I'd love a solution that allows me to maintain metadata/comments about the dataset: who created it, when it was created, and what it contains (at a high level). It'd also contain a link to the full dataset which I could copy into Jupyter notebooks. Maybe there's a better solution that eludes me right now. A related issue is version control of large datasets, which I'm aware is a difficult problem getting worked on. But for now, maintaining a library of datasets is the task at hand.

<comment><USER2> I'm one of the creators of Pachyderm (github.com/pachyderm/pachyderm) that might be able to help.

We do require an object store of some kind, but if you dont have s3 you can always use Minio in front of whatever storage you do have.

Pachyderm will let you mount that data locally in a jupyter notebook so you dont need to constantly copy it around. As many people as you want can do this and there will only ever be one true copy of the data in the centralized system. That central copy also includes version control for your data set so you can make changes to it and get deduplication of the files while still maintaining all your data lineage info.

<comment><AUTHOR> Neat! I'll have to check this out.</comment></comment></comment>


Rate my Startup - CollegeSolved - <AUTHOR>

Hi everybody!
Launched the full version in Sept after some initial beta testing.
http://www.collegesolved.com/
Three separate channels:
1) College search is hard: so we created the Kayak of college search.  The search tool is an easy way for prospective college students to narrow down their list and research colleges.  Once they have a manageable list, they can create an account to manage the application process.  It's a crowded space and we've separated ourselves by building a better search and including content not available anywhere else, including graduate salary data and a personality wheel for individual colleges based on surveys completed by college students.  Users can get a sense for what students on campus are like, whether they trend Social, Scholarly or Indie.  High school users love this feature in testing.  You can also tap your LinkedIn (parents) and Facebook (HS students) profiles to see if you know any current students or on thousands of college campuses.
2) Getting the real scoop is harder:  so we built a searchable network of 1,110+ college students on over 260 campuses across the country.  Prospectives can search by college, interests (newspaper, ROTC, athletics, Greek life) and connect with a college student to get the real scoop.  After filling out some profile info, the first call/email is free.  We've built a private, secure backend scheduling, call and email system to facilitate connections.
3) Getting in is the hardest: so we took a totally fractured industry, private college counseling, and aggregated the best in the business.  We partnered with IECA, the Independent Educational Consultants Association, to offer up the best consultants, as the market is ripe with marketing-heavy, results-light advisors.  So, if I'm a student in Portland, Oregon who wants to go to Harvard I can search for a former Harvard admissions officer to help with my application process.  I don't have to use the counselor down the street that my friend recommended.
Any thoughts about UX, level of usefulness, and ways to reach high school students would be appreciated.  Less interested in feedback on revenue model at this point, looking for thoughts on UX and distribution. Thanks! <comment><USER2> Clickable link: <http://www.collegesolved.com/>

First of all, I hope this takes off! You guys have done a lot of work to assemble all of the students and college data.

Some suggestions:

1) The UI is really cluttered, because I think you are trying to do everything! Have you considered going through and doing a UI/UX revamp? For instance, some of your buttons in the College Search use serif fonts, and some use sans serif. I'd suggest using more whitespace, and using more of a grid- based layout to line up columns.

2) Is your target audience mostly on Facebook? You should make parts of this a Facebook app and/or otherwise add ways to let high school students share this on Facebook.

<comment><AUTHOR> First of all, thank you for your thoughts!

1) Clarity of message is certainly something we're trying to focus on. We're always looking at ways to revamp.

2) Target audience is absolutely on Facebook. There will be an app for sure! Viral loop, right...

</comment></comment><comment><USER3> Very interesting project!

One suggestion: I am not that keen on the stock photography. Would be great if you could use "real" photos for example for the admission expert

More of a question: How did you decide to partner with the IECA? It seems to me that the college consulting market is very messed up looking from the outside. Limiting your choice to one provider might seem a bit odd. If you would stay more independent you could probably expand into evaluating the success rates of different consultants through your users.

<comment><AUTHOR> Noted on the stock photos. Not all of our counselors need to be IECA members. It's a great way for us to confirm quality, because they are the gold standard. There's 1,000 IECA members (not all are in our network) so there's not a supply/demand issue at this point. Thank you

</comment></comment><comment><USER4> Already went through undergrad, but I was curious where it would have told me to go. It was telling me Princeton or Yale. That wouldn't have ended well.

What I found interesting was that my alma mater, Purdue, wasn't listed in my searches. I spent forever messing around with the search parameters until I figured out that it was blocking me on SAT scores. I'm guessing that you're matching against average scores of admitted students?

</comment><comment><USER5> Have you thought much about the SEO? It seems like that's a big distribution channel for you competitors (Collegeboard, Cappex, CollegeProwler).

<comment><AUTHOR> We get some decent organic pickup from Google searches but are looking for some SEO tactics and pagerank boost - we're a 4/10 now and the competition seems to be a 6/10 - any recommendations would be much appreciated!

</comment></comment><comment><USER5> Might be good to resubmit this another day. I think everyone who could be, is at Startup School (or the after-parties, now).</comment>


Facebook Common Stock Value Update - <AUTHOR> http://vcexperts.com/vce/news/buzz/archive_view.asp?id=1006

<comment><USER2> Its easy to justify the fundamentals on why Facebook is worth $50bb or more. Zynga revenues alone ($800mm) with 30 of that going to Facebook ($240mm). Multiply that by 5 (assume Zynga has modest growth) and give it a 30x multiple and you start to hone in on $50bb. But the main reason Facebook is worth more than $50bb, I wrote about here:

[http://www.jamesaltucher.com/2011/01/why-facebook-is- worth-5...](http://www.jamesaltucher.com/2011/01/why-facebook-is-worth-50bb- part-ii/)</comment>