The destruction of graduate education in the United States - <AUTHOR> https://www.scottaaronson.com/blog/?p=3542 <comment><USER2> I found this article to be quite hysterical (and I follow and enjoy Aaronson's blog). It fails to present (or even consider) that there is a reasonable justification for these tax code changes, and instead leaps to "Trump and the Republicans must be evil and are out to crush all liberal edifices".

An article recently posted here on HN gives a more nuanced explanation for what's going on: [https://news.ycombinator.com/item?id=15807842](https://news.ycombinator.com/item?id=15807842)

TL;DR, the tuition fee deferral scheme that universities currently use is a grant-money laundering scheme. It's used to take federal grant money that comes with strings attached (e.g. can only be spent on certain things like paying salaries for grad students, or research costs) and launder it into money that can be spent on whatever the university wants (e.g. building new campuses or paying for nice desks for administrators). It's probably not unreasonable for the federal government to try to close this loophole.

The issue is that students are caught in the crossfire here, and I'm not sure that the incentives are there for universities to protect them fully.

But to be clear, the universities could just stop billing the government false grad student salaries, and then grad students' take home pay and tax exposure would remain unchanged, and there would be no crisis. The problem is that then universities would have to find a new money laundering scheme, or would have to make structural changes to their budgets that they would prefer not to make.

<comment><USER3> It's sad that there seems to be little internal critique coming from within universities of the complete takeover of schools by administrators. When it does happen it's almost entirely the result of some external rumblings or a direct affront like this being twisted.

I guess it's because these same administrators ultimately control the narratives (and budgets, jobs, etc) coming from the schools, creating a self- protecting cycle which only generates more administrative power.

So when attempts to curtail their power happen they can twist it to make it seem like it's an attack on teachers, students, and research - instead of the separate group of people who consume the greatest amount of resources within the school, who don't generate the positive research and economic utility for which universities are typically known.

Which is a shame, because there are probably no better people to stand up for this than the professors and researchers. Plenty of research has shown these people and their work aren't responsible for the massive increase in tuition and costs associated with schools. Yet the problem is perpetually the amount of revenue and income the school can make, not how it's being used.

<comment><USER4> _> It's sad that there seems to be little internal critique coming from within universities of the complete takeover of schools by administrators_

This is just patently false.

The criticism of admin bloat by faculty and students precedes the right-wing press criticism by _decades_ , is _far_ more harsh, and is still going strong.

The whole damn criticism, as it exists in the popular press today, can be traced quite directly back to academia [https://en.wikipedia.org/wiki/Benjamin_Ginsberg_(political_s...](https://en.wikipedia.org/wiki/Benjamin_Ginsberg_\(political_scientist\))

Or a more recent example, [https://home.isi.org/sites/default/files/MA59.3_Seery_Sympos...](https://home.isi.org/sites/default/files/MA59.3_Seery_SymposiumEssay.pdf)

And from the student perspective, the recent votes at places like Chicago to unionize (and, more importantly, the justification provided by the student groups that organized those votes)

Far from complicit, faculty and students are _leading the charge_ , and have been since the 1980s.

BTW, if you wanted to curtail administrative bloat/power in universities, there are ways to use the tax code and science funding policy to do so. and lots of serious policy proposals by the people listed above would very likely have great effect.

This policy targets students instead.

<comment><USER5> > And from the student perspective, the recent votes at places like Chicago to > unionize (and, more importantly, the justification provided by the student > groups that organized those votes)

I'm at a school that underwent a grad student unionisation drive, and pretty much everything that the union proposed would have amounted to further empowering the administration at the expense of the academics (e.g. introducing new mediator officials with extensive compulsive power over advisors if their students should feel wronged in one way or another, hiring more staff to enforce diversity-friendly admissions and hiring outcomes, ...). At least the politicised end of the student body seems to consider the administration, rather than the academics, to be their natural ally.

<comment><USER4> This is debatable and its own rabbit hole that is beat to death on other recent stories. But anyways, I understand the point, and this is why I stated:

 _> and, more importantly, the justification provided by the student groups that organized those votes_

<comment><USER5> I'm sorry that I can't provide more details or evidence (lest I leak bits of identifying information), but as far as I can tell, our union has not made any statement to the effect of considering excessive administrative power or even bloat an issue.

</comment></comment></comment></comment></comment></comment><comment><USER6> The systematic erosion of public education in the United States doesn't serve anyone in the United States' long term interests, except maybe the religious right (who don't matter that much in the federal government). It just doesn't serve monied interests. It doesn't help big oil, it doesn't help big pharma, it doesn't help the legal or medical professions, it doesn't help finance, or insurance, or banking. So I'm a little puzzled. Who does it benefit?

I share the author's bewilderment in the face of the realization that this is, in fact, terrible for universities, and (seemingly, given what we know) not just weird news spin-doctoring or scare mongering:

 _" Given the existential risk to American higher education, why didn't I blog about this earlier? ... It's simply that I didn't believe it—even given all the other stuff that could "never happen in the US," until it happened this past year... Surely even the House Republicans would realize they'd screwed up this time, and would take out this crazy provision before the full bill was voted on? Or surely there's some workaround that makes the whole thing less awful than it sounds? There has to be … right?"_

I'm also confused. I decided that I didn't have the right kind of information to understand what was happening in the Federal government a long time ago, and I still think that, but I find this whole situation very puzzling, because it doesn't make much sense given how I think the world works (which is, of course, a deeply flawed and nearly useless model of the real world).

<comment><USER7> I think if you view this through the lens of identity politics it makes a bit more sense why you're seeing such an attack on education and universities.

When climate denial and the rejection of basic science is one of the pillars of the party then this is a logical extension.

<comment><USER8> This is the point of the tax bill: to punish blue states and the educated.

The removal of state income tax deductions; the cutting of mortgage interest deductions; and the removal of the graduate school tax break all point the same direction.

As for Scott's suggestion that maybe universities can not charge, I can't possibly see that flying with the IRS. If the students still get that valuable education, taking something that used to cost $30k or more, declaring it "free" while still delivering the thing, and telling the irs that therefore, tada! no taxes are due! is... very unlikely to work.

What we have is the removal of a specific tax break aimed at graduate school education in order to punish the educated.

<comment><USER9> None of these changes punishes anyone. Each of these changes removes some form of favoritism.

Right now, people in states with high tax rates, people with mortgages, and graduate students pay lower federal tax rates than other people who don't share those favored statuses. One could more credibly argue that the current system punishes people who don't receive these tax breaks.

(Graduate students are favored if you count free tuition as compensation in the same way benefits like cars or apartments given to employees by other employers count as taxable compensation.)

<comment><USER10> You can argue that this isn't punishment. But I don't think you can argue that this isn't targeted or a clear expression of priorities. Churches aren't being taxed here. They could have lost their tax exempt status. The Increase to the Child tax credit that would have helped low income families isn't in the bill. There is no reduction in military spending to offset the cost. And it sure looks like the states most severely affected by removing the state tax deduction vote democratic. Especially since these states actually subsidize most of the other states through their much larger tax base and its contributions to federal tax that is redistributed to the rest of the country.

Let's remember that grad students are now being taxed for money they never receive. It's not like money is changing hands for tuition. If they were a business and the university paid them $100k they would be able to expense the tuition.

<comment><USER9> A grad student working a different job has to pay taxes on the income used to pay tuition, as does a student who borrows the tuition and repays it while working a different job in the future.(1) This is a very specific favoritism - not even favoring all grad students, but only grad students working for universities. And ultimately, favoring the universities by lowering their grad student labor costs.

1: Excluding some limited credits and deductions of at most a few thousand dollars each. Will graduate assistants be eligible for these?

<comment><USER10> What PhD program requires the payment of tuition?

<comment><USER9> What PhD program allows students who aren't working for the university as a teaching or research assistant and doesn't charge them tuition?

<comment><USER10> How is that relevant. Grad students work because they are doing research relevant to their degree and are teaching so they can learn to be professors. It's all part of the program. The get paid for their work and they pay tax on it. The government pays their tuition in the form of a grant or the university endowments pays the tuition because the university needs to cover operational costs. Tuition has nothing to do with a grad students income. The only reason they are invoiced for it is that represents their portion of the university operating budget.

<comment><USER9> Tuition is part of the compensation graduate assistants receive for working for the university.

There's no reason universities should receive special treatment here. They should pay their employees enough to live on, after taxes, like all other employers, and the fringe benefit of free tuition should be taxed like other fringe benefits.

<comment><USER10> In what way is it compensation? So we should calculate sales tax based on sticker price rather than actual paid price?

Grad students receive a salary for their work and a discount from the sticker price of the tuition. They are separate transactions.

<comment><USER9> The transactions are tied: grad students have other (often much better) options for employment; they choose to work for a grad school because they value the free tuition.

<comment><USER11> No, they choose to work in PhD programs because they value _doing research_.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> When some of endowments for many universities are in the millions and Harvard has an endowment of 37.5 billion, I find it hard to have sympathy towards this.

The colleges and universities have become corporations that run on student loans and graduate student slave labor. This has to change, and it will be a painful process no matter what happens.

[https://www.usnews.com/education/best-colleges/the-short- lis...](https://www.usnews.com/education/best-colleges/the-short-list- college/articles/2017-09-28/10-universities-with-the-biggest-endowments)

<comment><USER13> Is massively raising taxes on graduate students actually going to help matters?

<comment><USER14> Couldn't the universities just appropriately price the graduate school tuition instead of pretending it's worth tens of thousands of dollars a year and then giving it away for free?

<comment><USER15> They don't give it away for free. It's typically paid by someone.

For example, my graduate students are on NSF grants that I applied for. The grants pay the graduate students a stipend and health insurance costs, they pay my university the full cost of in-state tuition and fees, and they pay overhead on those costs. These are all budgeted for.

<comment><USER16> Then I think it's fair to ask if it's really OK to subsidize that compensation delivered in terms of valuable education.

You can argue that graduate students and universities are just another special interest lobbying the government for a tax exemption that benefits them. There is nothing special about graduate education when you're conferring something that's inherently valuable (namely MS and PhD degrees) to a lucky few.

You could further argue that it's unfair to ask blue-collar workers in fly- over states to pay higher taxes because of the impact of such a subsidy.

There is a lot to like in the tax bill: interest on second homes isn't going to be subsidized nor are new mortgage interest on homes over $1M. Frankly, I think it should have been applied to ALL homes over $1M since many economists have argued vociferously that such an exemption is a huge subsidy to the high- cost states.

I'd also like to see a revisitation of the taxation of carried-interest for hedge funds. There is no reason to consider performance bonuses as a return on capital if they don't have any basis in the investment/capital at risk.

But arguing that graduate students (LUCKY graduate students) are worthy of some kind of exemption is just lobbying. You could easily make the same argument for active-duty armed service members. Frankly, I'd support a tax reduction on those people. They've suffered from our national follies to a disproportionate degree.

<comment><USER4> _> LUCKY graduate students_

The overall tenor of your post is a bit disingenuous. The data tells us that getting a PhD does not substantially improve your material existence, or at least that the picture is murky.

The data also tells us that the science that comes out of the research performed during those PhDs has an enormous positive ROI for society.

So in other words, phd students are under-paid take a lifelong financial hit to provide large but diffuse benefits to society, benefits that most students won't be able to capture because, well, that's the nature of basic science..

 _> You can argue that graduate students and universities are just another special interest lobbying the government for a tax exemption that benefits them_

Since everything's a special interest, we might as well throw the money at oil executives?

No, of course not. Some of those special interests really are worth investing in. Roads, schools, and labs provide enormous and measurable, if diffuse, value.

 _> You could further argue that it's unfair to ask blue-collar workers in fly-over states to pay higher taxes because of the impact of such a subsidy._

This bill will most harm the public and low-endowment universities in those flyover states. The schools those workers' children will attend will be less well-staffed and there will be fewer opportunities for funded graduate programs.

 _> You could easily make the same argument for active-duty armed service members. Frankly, I'd support a tax reduction on those people_

Good news: they already receive very large tax exemptions. As they should; the armed services are also an important special interest...

<comment><USER16> If you don't think earning a PhD substantially improves your material existence, I'd beg to differ. There are plenty of folks who don't have PhD's who would be better positioned in the job market if they had the talent and opportunity to earn one. For every student lucky enough to earn a spot doing research there are plenty that don't have the opportunity and have to go out into the real world and earn a living.

I meet plenty of PhD's in my regular practice. Not every person earns a PhD and stays in academia. PLENTY earn that degree and use it as a stepping stone to a high-paying job in the private sector.

As for oil executives: they're not going to be the biggest winners here. Shareholders likely will be. That repatriated capital is going to lead to a huge share buyback and dividends. Wanna know who's going to get a large chunk of that? PENSION FUNDS for state employees. If you think CALPERS doesn't stand to benefit from this surging stock market you're not paying attention.

I wonder if you realize how much you sound like a lobbyist? Without a tax subsidy, there is a calamity. Every time someone warns that a tax increase will affect investment or job creation or cause a shift in the deployment of capital, people in favor of said increase downplay those fears.

All the folks who said that a $15 minimum wage would kill job creation....were those folks wrong?

And here we are, hearing that the removal of a special tax treatment will massively impact graduate education. Well, maybe. Maybe not. I guess we will have to see.

So the cost to a student to receive a PhD goes up a bit. Big deal. Is that gonna kill research entirely? There is already a glut of PhDs here in the USA. Maybe a few less isn't such a bad thing.

<comment><USER4> _> There are plenty of folks who don't have PhD's who would be better positioned in the job market if they had the talent and opportunity to earn one._

Cool anecdotes and all, but there's data on this question and you're wrong. PhDs do not increase lifetime earnings.

This isn't a case of dueling anecdotes. It's a normal fact we can verify with data.

 _> As for oil executives: they're not going to be the biggest winners here. Shareholders likely will be._

And I suppose oil executives store their fortunes in bold bars...?

 _> PENSION FUNDS for state employees_

This is such a BS excuse for favoring capital.

It is, in fact, possible to write tax code that differentiates between the two.

The current tax bill does not, in fact, actually do so.

And worse, it even rolls back benefits that _did_ differentiate between "filthy rich" and "working class saving for retirement"!

 _> I wonder if you realize how much you sound like a lobbyist?_

Says the one conflating upper class investment income with pension funds while defending a policy that slashes retirement benefits and dismissing _hard facts_ with personal anecdotes about how well-off people making 30k/yr are...

<comment><USER16> AHEM.

[https://www.wes.org/wp- content/uploads/2017/05/blog_20140722...](https://www.wes.org/wp- content/uploads/2017/05/blog_20140722-02-Work-Life-Earnings.jpg)

<comment><USER17> AHEM. As above, it takes work to be as wrong as you are, so this seems like either an amazing amount of ignorance, or deliberate deception.

1\. What it shows: Students good enough to get accepted into PhD programs are good enough to get paid more in general. Good we're talking about this -- the market clearly values these people!

2\. What it misses: ... But what if a PhD-capable person instead just went directly into industry? What is the opportunity cost of taking a pause of 4-7 years to do R&D for the public?

3\. What economists found: By looking at students graduating from a good school and therefore PhD-capable, and then comparing the result of them doing 4-7 years of PhD vs not.. the students who took time to contribute to science lost out on a ton of money. They're quite literally donating their time to society.

Phrasing it as _subsidizing_ these people to do R&D is misleading. The reality is the government is underpaying for the public service of growing the economy, especially relative to market rates for people equally qualified to make a bunch of $$$ working on social media ads and fancy juice machines.

But that breaks the narrative. The actual studies show the salary sweet spot is somewhere between a BS and MS. That means federal institutions must _lure_ PhDs, not drive them away.

<comment><USER18> > _AHEM. As above, it takes work to be as wrong as you are, so this seems like > either an amazing amount of ignorance, or deliberate deception_

This sort of uncharitable swiping breaks the HN guidelines. So do insinuations of astroturfing, which you did elsewhere in this thread. I realize it can be hard to restrain oneself when someone else is wrong on the internet, but please make the effort and edit such acid out of your comments if you want to keep posting here. You can always set 'delay' in your profile to give yourself time to edit before others see your comment; that's what I do.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> An uneducated society is easy to manipulate — easy to convince that climate change isn't real, that evolution is a conspiracy against religion, that creating a social safety net and investing in public education, health care and modern infrastructure is a bad thing.

I hate to turn social issues into black and white, partisan debates. But anti- intellectualism and the denigration of education is a uniquely right-wing phenomenon in the U.S. It's used often by the Republican party here to criticize "elites" and is employed by a long list of other power-hungry demagogues throughout world history [1].

Isaac Asimov wrote a column in January 1980 in Newsweek that describes this phenomenon in the U.S.:

> There is a cult of ignorance in the United States, and there always has > been. The strain of anti-intellectualism has been a constant thread winding > its way through our political and cultural life, nurtured by the false > notion that democracy means that "my ignorance is just as good as your > knowledge."

I encourage everyone to read the full column [2].

[1]: [https://en.wikipedia.org/wiki/Anti- intellectualism](https://en.wikipedia.org/wiki/Anti-intellectualism) [2]: [https://aphelis.net/wp- content/uploads/2012/04/ASIMOV_1980_C...](https://aphelis.net/wp- content/uploads/2012/04/ASIMOV_1980_Cult_of_Ignorance.pdf)

<comment><USER20> > An uneducated society is easy to manipulate

This is nonsense. There's no Illuminati out to make the population dumber. There _is_ a bunch of identity politics ignoramouses out the gut colleges that "corrupt our values" and it's an easy thing to point to as a win when your base (who keep you in power) are mouth breathing retards who see higher education (and intelligence in general) as nuisances.

Make no mistake: their goal is more money for companies, and this is merely a way to keep their base happy so they can keep their shareholders happy.

</comment></comment><comment><USER21> The version of this bill which passed the Senate last night did not end the tuition tax credit [0]. Additionally, tax cuts do sometimes expire after a few years. For example, this info from Intuit indicates that the tuition tax credit in question expired at the end of 2016 [1].

Editing this upon a close rereading of [1]. I'm not sure if that refers to the same tuition tax credit. Any armchair tax lawyers care to weigh in?

But, I have called my representatives and asked them to make sure that the credit survives reconciliation of the bill between the House and Senate, because I don't think that graduate student pay should be used as a bargaining chip.

[0]: [https://www.washingtonpost.com/news/wonk/wp/2017/11/30/what-...](https://www.washingtonpost.com/news/wonk/wp/2017/11/30/what- is-in-the-senates-massive-tax-bill-and-what-could-change/)

[1]: [https://turbotax.intuit.com/tax-tips/college-and- education/d...](https://turbotax.intuit.com/tax-tips/college-and- education/deduction-for-higher-education/L0krerdUK)

<comment><USER22> The House version of the bill eliminates the student loan deduction for graduate students. This will be resolved in conference so it is still very much on the table.

<comment><USER21> Yes. My intent was to foster not complacency but hope - everyone reading this, please get on the phone with your representatives as soon as their offices open and tell them to keep the tuition credits while the bill is in conference!

[https://whoismyrepresentative.com](https://whoismyrepresentative.com)

</comment></comment></comment><comment><USER23>
        > As you've probably heard, one of the ways Republicans intend to pay for their tax
        >  giveaway, is to change the tax code so that graduate students will now need to
        > pay taxes on "tuition"—a large sum of money (as much as $50,000/year) that
        > PhD students never actually see, that can easily exceed the stipends they do
        > see, and that's basically just an accounting trick that serves the internal needs
        > of universities and granting agencies.

If I'm working for a company, the company decides "it is required for you to have X. We will provide it to you as part of your compensation." That X is part of my compensation for working there. I don't see why it shouldn't be taxed. "It will cost me more money" is not an excuse to not be taxed. "The students never get the money" is invalid because colleges have deemed their "X" to be worth 50k.

Let me phrase this situation a different way. Assume I start working for Bloomberg's HF trading group. They want me to live closer to my office and buy my a penthouse apartment half a block down. They pay for it (made the contract, own the building, pay the staff, etc) but it's in my name. It's my apartment. Is that part of my income? Shouldn't I be taxed for this apartment? I never actually see the money! I never actually get to spend the money! It's just something that has to happen for me to be on Bloomberg's HF trading team!

I think that should be taxable. What do you think?

<comment><USER24> It's not really like the apartment example. When I was in grad school the tuition number was kind of an imaginary number. It was 60k or something like that, no idea. I never signed anything with that, was never billed for that, nothing. The only way I could know what that number was somewhere on my online account, where I saw:

>> Charge: +$60,000 from Graduate School Division

>> Payment: -$60,000 from Graduate Department of XYZ

And that was not the value of the education either. Even in quarters where you are not physically in the school, with no classes, library access, etc. you would still be "charged" that magic number. It's just a way to allocate expenses between each department and the graduate school.

In most cases they will just change the accounting to something even weirder, and the government will get nothing. Some schools, with arcane rules from donors (such as that their money can only be used for education and not infrastructure) will have problems. But for the most part it's just a stupid thing to do from the govt side, that will just complicate things even more.

<comment><USER23>
        > And that was not the value of the education either

Why not?

        > Even in quarters where you are not physically in the school, with no
        > classes, library access, etc. you would still be "charged" that magic
        > number. It's just a way to allocate expenses between each department
        > and the graduate school.

This isn't just for allocating expenses. This is what a grad student is assessed to cost an institution. Did you ever contact the PI of your grant while you were away? Did you attach the universities name to your work? Did you access journals with subscriptions paid for by your institution? Did you use your student email?

<comment><USER24> I did a finance PhD. We don't have PIs. Email cost is trivial, etc.

Put it another way: I don't care if my employer just spent a bunch of money on a better building or more comfy office chairs. I don't care if the IT department charges my unit $XYZ for expenses (call it "tuition" if you want).

That's something that happens on the back end of my employer, and is not an income for me. That's the same thing for grad school. I got paid 20k to work on research, teach, grade, and help in maintaining the school's name so they can get MBA students (who ultimately pay the bills in our case). Why should I care if one unit at my school charged another unit? Heck, I paid taxes and social security on those 20k, got health insurance, had a W2, etc.

For all practical purposes, PhD students are employees. Make them pay taxes on their income, and not on the incidental expenses of their employers.

</comment></comment></comment></comment><comment><USER25> I think that the grad student tax would _increase_ the number of grad students.

If the tax increase goes through, universities will probably respond my significantly increasing "indirects" and decreasing tuition. They could twiddle the numbers so that the university still makes the same average off of every grant, and students take homes don't change at all.

But there would be in important difference: from a PI's perspective, grad students would be relatively cheaper. Paradoxically, taxing tuition may have the net effect of creating more grad student positions (while decreasing the spend on capital investments).

</comment><comment><USER9> Why wouldn't the obvious solution work? Raise graduate student stipends to cover the tax.

<comment><USER26> The money doesn't exist to do that. Graduate student stipends are paid out of government grants, and the amount of money issued by the government as grants is going down.

<comment><USER9> They could make it grant-neutral by lowering the tuition rates at the same time. The universities would receive less money but it need not be "the destruction of graduate education".

<comment><USER26> The university is using that money for _something_ already, and now it isn't. I guess this depends on what that something is. If they're just stuffing it under their mattresses or using it to pay some associate under provost for squirrel education a $150k salary, or putting marble plating in the athlete dorms, then this probably isn't a big loss. If it's going to something like students with disabilities or need based scholarships...

<comment><USER9> It's only a bit of an exaggeration to say that a lot of money is used to "pay some associate under provost for squirrel education a $150k salary, or put marble plating in the athlete dorms". Surely the universities could find some expenses to cut that wouldn't hurt too much.

<comment><USER26> Maybe! You'd have to look at the numbers. I remember the last time someone showed me numbers, actually most of the money went into core operations cost to offset money that the state used to contribute.

Though I think in this particular case, for my particular institution, the University could totally mitigate all financial cost to graduate students by cutting the subsidy the University gives to the athletics programs (about 15% of the athletics budget is paid for by the students in the form of fees the students are assessed on top of their tuition).

</comment></comment></comment></comment></comment></comment><comment><USER27> Republicans are anti-science, anti-immigration to the extent they want to curb university research and stop the flow of world's smartest immigrants to USA.

That's the only rational explanation of what's going on.

If that wasn't the intent, the people who created and voted for this bill are the biggest idiots.

</comment><comment><USER28> cached: [https://webcache.googleusercontent.com/search?q=cache:YefHUe...](https://webcache.googleusercontent.com/search?q=cache:YefHUeCtjZ0J:https://www.scottaaronson.com/blog/%3Fp%3D3542+&cd=1&hl=en&ct=clnk&gl=us)</comment>


Building an Application upon Riak - Part 1 - <AUTHOR> http://www.productionscale.com/home/2011/11/20/building-an-application-upon-riak-part-1.html

<comment><USER2> Thanks for sharing. We have been using Riak for the last year and have had lots of fun learning a few lessons the hard way. Now we have learnt it though, everything seems to be fine - apart from when nodes stop responding when they start merging Bitcask data.

I think one of the biggest issues we had was that we saw it as a kind of drop in replacement for MySQL. We originally used the Ripple library (protip: don't) which talked to Riak over the HTTP interface (it now supports Protobuffs, but still don't) and plonked loads of data in, then wrote some lovely MapReduces to get it out. It worked fine, except it was slow. Dog slow.

The issue is that to do a MapReduce you need to get a list of keys to process. However in Riak listing the keys of a bucket is slow as it has to walk the whole keyspace (even other buckets) to find the matching keys. I'm guessing this is why Riak Search is slow for you as well. We didn't try it, but if you kept track of the keys and fed these to a MapReduce I suspect it would be a lot faster.

Now we are using it as in your epiphany as a persistent distributed version of memcached. We use predictable keys for everything, so just need to do gets and puts. We also found that trying to deal with siblings is a pain, so we turned them off (allow_mult=false, last_write_wins=true). For any important documents that need to be updated, where we can't afford to lose data due to race conditions, we just write a new version with a different predictable key (kind of like a linked list). It works lovely now :)

As stated in the article, enterprise support is good, however we have had most of our questions answered rather quickly in IRC or the mailing list. There is usually always somebody around from Basho to help you out. The Riak Recaps are also a great way to keep up to date with what is going on in the community.

</comment><comment><USER3> Riak is interesting, but its performance characteristics are unfortunately interesting in a negative sense.

First, single-node performance is awful. You would think that even on a single node, simple key/value lookup would be able to compete against something like Postgres; after all, the read pipeline should be much simpler. But it doesn't come close, in my testing. Not just key/value lookups; it's also dog slow at chasing links, and scales very badly with the number of links.

Now, Riak is intended to be deployed on many nodes, but in my testing with three nodes I did not see a significant increase in performance. The overhead seemed to be constant. In a master-less, distributed database, some compromise on performance is acceptable, but Riak looks much worse than would be acceptable.

Riak is focused on random access. Sequential access similar to that offered by Cassandra and HBase just won't scale beyond a few hundred objects. This problem is made worse by the fact that a Riak "bucket" isn't really a bucket, but a namespace; all data in a Riak database resides in the same storage, and bucket operations simply filter that data based on the bucket name. This means that any sequential access is off the table.

The Riak people suggest using links in order to emulate sequential access patterns and entity relationships. For example, if you want to blog post with bunch of comments, you would store the post in one key, and then comments in other keys, and then use child/parent links to glue them together. Unfortunately, this seems to scale very badly with the number of objects linked.

Not just read performance, either; it seems links are stored in a single chunk whenever an object stored, so when adding a single child, you have to write all children back to the object! Obviously that does not scale to thousands or even millions of objects. For a database that has no other means to deal with relationships, the weakness of the link feature is serious.

After all, you would face the same problem if you stored the list of objects as a key value (you have to rewrite the entire object every time, since Riak does not have partial updates), and since it's not feasible to scan keys based on prefix, for example, you end up with no alternative. I suppose you can use Riak Search, but last I checked it was built on top of Solr, which is not fast.

There are various minor annoyances, too. For example, unlike Cassandra, there is no operation to delete the entire contents of a bucket -- something which you want to do before/after unit tests --, so you have to sequentially enumerate the bucket's keys and delete each one, something which is just excruciatingly slow.

Here is a test case I wrote in Ruby to test performance: <http://dl.dropbox.com/u/12091499/Share/riaktest.tar.gz>. It contains a script to test similar operations/queries against Postgres. The included Riak test results are for a three-node dedicated test cluster (plenty of CPU, RAM, I/O). I am open to the possibility that the config needs tuning, of course.

</comment><comment><USER4> _I/O - It's not easy to run Riak on AWS. It loves I/O. To be fair, they say this loud and clear so that's my problem. We originally tried fancy EBS setup to speed it up and make it persistant. In the end we ditched all that and went ephemeral. It was dramatically more stable for us overall._

If you're running any type of a database or service on EC2 where latency is an issue, you should be using ephemeral/instance storage. Not EBS.

<comment><USER5> If you're going to run Riak on AWS, why not just use AWS S3 service, which is the Dynamo that Riak is based on?

<comment><USER6> Because s3 has latencies an order of magnitude higher than a local riak. This is especially important if you have many unparallizable requests.

</comment></comment></comment><comment><USER7> Unlike a lot of other NoSQL solutions, there's remarkably few "real-world" stories about Riak, so I love hearing about it when I can. It's also interesting to hear about some of the "real-world" problems that they've encountered. I've been looking at Riak for a while now for a project of mine, so these kinds of articles are very welcome!

</comment><comment><USER8> I have used riak in a production app. Just recently switched to redis.

Riak was great for the most part. There were three nodes running on three different machines. Setting up a new node is a breeze. Eventually there become a data inconsitancy between the nodes that couldn't get resolved, and would happen intermittently.

When this happened, any sort of map reduce operation would fall. Individual keys could be requested, but only from a node that had the item.

In addition, removing a node in this inconsitant state was a no go. The other nodes would stay in the inconsitant state for good.

Interestingly enough, this only happened on certain buckets. There was a logger bucket that had keys updating and being created all the time that never had an issue.

Perhaps it was a bad configuration on our end...

<comment><USER9> I assume the cluster is no longer running? I'm curious whether your inconsistent looking bucket had allow_mult set to true?

<comment><USER8> No, allow_mult was set to false. Similarly to the author, it was running on AWS, so that may be a factor.

</comment></comment></comment><comment><USER10> I really enjoyed this review, it came off very honest and you didn't try to be very pro or con, just what you ran into over the past year.

You mention no reasonable Scala client exists, why not create and release your own? I know Riak talks about offering fulltext search, but one thing I've learned over the years is using a specifically designed search engine (such as lucene/solr/indextank) is typically far more performant/scalable plus easier to implement then a generic datastore. I've never used riak for search so I have a few questions. Did you ever attempt to use a different datastore for search? If you did, at first did riak perform just as well, if not why not?

<comment><USER11> Riak Search is built on top of Solr, if i'm not mistaken.

<comment><USER12> You are indeed mistaken: [https://help.basho.com/entries/477225-what-is-the- relationsh...](https://help.basho.com/entries/477225-what-is-the-relationship- between-riak-search-and-lucene-or-solr)

</comment></comment></comment><comment><USER13> Great read. One thing that would have been useful would be to know what sort of size dataset they were working with? If you're not working with millions of documents, is the search and link walking still viable?</comment>


Non coding job types for digital nomads - <AUTHOR> https://www.remotetribe.life/remote-jobs/best-remote-non-coding-jobs-for-digital-nomads-2020/ <comment><AUTHOR> There are plenty of non technical IT related jobs you can do, without writing a single line of code. Yes, roles like this exist and we've curated a list for you</comment>


Accused Russian reporter Ivan Golunov to go free after outcry - <AUTHOR> https://www.bbc.com/news/world-europe-48600233 <comment><USER2> Shady indeed.</comment>


Is There a Limit to AI Diagnosis Accuracy? - <AUTHOR> https://www.computer.org/web/computingnow/insights/content?g=53319&type=article&urlTitle=is-there-a-limit-to-ai-diagnosis-accuracy- <comment><USER2> It seems to me that there is a limit to diagnosis accuracy period.</comment>


Ask HN: Good interview questions for average programmers - <AUTHOR>

I have a slightly strange problem. I need to hire average programmers. Reasons are many - but primarily budget and the type of tasks they are going to be assigned to - not cutting edge work.<p>I have been struggling to find good questions to ask. Fizzbuzz is too easy - and my other questions I try seem to be too hard. So what are your favorite interview questions to ask an average programmer? <comment><USER2> Implement a bubble sort, then a binary tree.

Ask how they'd approach the travelling salesman problem.

Given an SQL query, a request, a set of objects and required output, ask them to optimise the whole process from start to finish. The smarter ones will ask about caching, look at optimising the query, look at the way the output is generated etc. The not so good ones will focus on the .NET/PHP/<language>

<comment><AUTHOR> The SQL query one, definitely sounds great.

</comment></comment><comment><USER3> Do you really think FizzBuzz is too easy?

Hmm.

OK, here's one. Tell them to find out what a Bloom Filter is, and then implement it. Ask what decisions they had to make on the way, and why they made the choices they did.

<comment><AUTHOR> Hmm. Bloom Filter? Hmm, that's a bit of a tough nut to crack. Don't know if it would work out in a face to face interview over the desk.

<comment><USER3> You're asking about an average coder - they don't need to re-invent it, they need to be able to look it up, understand the description, follow instructions and implement something fairly straight forward.

If you're hiring an average programmer then they need to be able to look up things, understand them, and implement what they've learned. Seeing what they do and how they do it is what's important.

</comment></comment></comment><comment><USER4> Ask them to show you something they have coded and ask questions about their implementation. If you can't do it yourself, invite one of your senior devs to join the interview.

<comment><AUTHOR> That's been my fallback plan, but the problem is they don't always have anything with them to show me.</comment></comment>


PHP: the quiet powerhouse - <AUTHOR> http://blog.appfog.com/php-the-quiet-powerhouse/ <comment><USER2> If you are just starting out as a web developer and aren't planning on changing careers anytime soon, the #1 language you should learn is JavaScript. It, or languages that compile to it, will still be in fashion in 5 or even 10 years. That's because the web is shifting to thick clients, and JavaScript is the only choice in browsers. JavaScript is also finding its way into server- side code (Node.js).

"I think PHP is the perfect language for beginning web development for two reasons: Direct HTML embedding."

Server-side view rendering is on its way out. So this is not a real advantage.

"I find it more than a little strange that PHP is simultaneously (a) out of fashion, and yet (b) absolutely everywhere."

This is simply how technologies rise and fall. By the time a technology is ubiquitous, the original problems it solved have changed and new solutions are being created. Just because you have a bunch of legacy code running lots of web apps doesn't mean it's the future--it means it's the past.

Also, why pick a language that pays poorly? Ruby is by no means inferior to PHP, and you will get paid more per hour to program in Ruby.

<comment><USER3> > Server-side view rendering is on its way out. So this is not a real > advantage.

No it isn't, client side view rendering is just experiencing a faddish phase; it'll fade. It'll never overtake server side rendering on the web.

<comment><USER4> The seductive advantage of server side rendering is centralized control of what all clients see. Whenever you push too much business logic down to the clients you have to contend with how to ensure all of those clients are using the same logic at all times.

<comment><USER5> This is just a result of loose architecture. If your models are cleanly abstracted using REST principles on the server-side, then server templating becomes much more appealing.

I say this because I am doing it -- building apps with REST interfaces on the server side. It's not a common pattern yet but I think this is where things will actually go (as opposed to the client), because it's actually easier to use, understand and iterate on.

If you can write Javascript templates that interface with your back-end using REST calls like "GET /accounts", you can do the exact same thing in a server side template with better performance and dramatically simpler security.

</comment></comment></comment></comment><comment><USER6> I don't think MediaWiki or Wordpress are good examples of how PHP scales. Those are the two examples of extremely bloated pieces of software that require a lot of hackage to get to scale. I've done deployments of those on mid sized VPS'es and it's always a pain getting it responsive while still able to hold a good number of concurrent connections.

The only way anyone gets Wordpress to scale is to put caching in front of it. But now the heavy lifting is being done by memcache or varnish, not by anything engineered into Wordpress or PHP.

<comment><USER7> Any sort of scaling is a hack (not always bad). Good luck if you expect to scale to Wikipedia or Wordpress(.com) levels without using external tools such as Memcache and Varnish.

</comment></comment><comment><USER8> Here's an idea.. If PHP is so bad.. Don't use it. Simple as that. Let the people who want to use it... There's absolutely very minimal things wrong with the language. Everyone's quick to compare it with Python or Rails or something else saying "PHP does this wrong that Python does right" well, if PHP did everything the same way as Python, Rails, or something else, when what makes it different? Who wants to use a language that doesn't have some severe difference?

It's annoying seeing people waste their breath on arguing why PHP is so bad... Especially when many people/companies have proven PHP is good and can be scaled and used on high trafficked sites, yet 90% of the arguments are how it doesn't scale.

Just because you have used Wordpress or built a quit todo app in PHP that took you 5 minutes and that's the extent of your usage with the language, doesn't make OR break the language.

Quit complaining and let PHP users use PHP, and you continue using whatever the language you use is.

Signed, A Python, PHP, and Node.js developer.

<comment><USER9> Some of the problems with PHP are things that are problematic or irritating to development (e.g. its inconsistent standard library) but other things are _actively dangerous_ to security and correctness. For example, in PHP, '9223372036854775807' == '9223372036854775808' returns true.[1] Now imagine those two strings are hashed passwords.

The overall problem is not with specific flaws of PHP; the problem is that flaws in a language lead to errors in programs written with that language, and when that languages is used to write some kind of web infrastructure, then use of that language by people less than experts (i.e. almost everyone) is harmful to the whole internet. PHP can be used to write useful, scalable, secure web apps in the hands of someone who knows what they're doing, but in the hands of someone who doesn't _quite_ know what they're doing, or who does but is tired enough to miss individual details of their code, it is much more dangerous than other comparable languages.

That is why I discourage the use of PHP if possible.

[1]: <https://bugs.php.net/bug.php?id=54547> (Both strings appear numeric, so they are casted to numbers before the comparison, but because they are larger than maxint, they are rounded down. The solution, if you really must use PHP, is to use === instead. The fact that a solution exists does not mitigate my concern.)

<comment><USER10> php > var_dump('9223372036854775807' == '9223372036854775808'); bool(true)

This made me cry.

<comment><USER8> Because in a real application people are checking to see if those lazily match every 3 to 4 lines... Half of this stuff that makes the haters "cry" is unrealistically used things.

</comment></comment></comment></comment><comment><USER11> I enjoyed PHP. I enjoyed Ruby. If I was to do a complex project that would require a lot of integration with external services, I would want a good OOP language like python or ruby.

There are something PHP does nicely than Ruby does. Visa versa. However in all the PHP apps I have made / used, their structures started out with good intentions and then descended into pure chaos. There isn't a unified package manager for Python and PHP that is widely accepted. Ruby has rubygems and bundler.

Maybe next time when I start building another PHP app with namespacing, I'll behave better and make my designs more decoupled so that components can be removed and added, provided their interfaces match.

<comment><USER12> I would say that pip is regarded as the standard, unified package manager for Python.

<http://www.pip-installer.org/en/latest/index.html>

</comment></comment><comment><USER13> [http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad- de...](http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/)

9600+ generally concise words on many, but not all, of the issues with PHP. As a summary, PHP isn't "out of fashion" as much as people are tired of continuously being misled and abused by their language of choice.

<comment><USER14> On that note, a physical metaphor for PHP which that post inspired:

[http://www.flickr.com/photos/raindrift/7095238893/in/set-721...](http://www.flickr.com/photos/raindrift/7095238893/in/set-72157629492908038/)

</comment></comment><comment><USER15> "Dozens of new language-specific communities have now entered the fray and brought along their legions of devotees with them, and many of those devotees see PHP as being passé. I even had the following happen to me, and this is absolutely a true story and can be verified by our VP of Operations, Richard Kotulski: I was in an elevator with O'Reilly's very good "PHP Cookbook" in hand and an older gentleman scoffed at me and said, "PHP? What is this, 1920?" This really, actually happened. I doubt it would've have happened if I had had a Python or Ruby book in my hand."

Which... is even odder because some evangelists like to point out that python and ruby are _older_ than PHP, as if that means much.

While I generally agree with the points, many PHP strengths are also weaknesses. For example - "Direct HTML embedding." It's nice and easy, but because it's so easy, it leads to unescaped output, and xss exploits and such.

PHP does let you as productive as you want to be _on the web_ , either with raw PHP or with one of the myriad frameworks. The author's point that no other language gives you that is, I think, another one of PHP's strengths (but yes, is essentially a weakness in the wrong hands as well).

EDIT - have been using PHP since early 1996 - almost since its inception, and have watched the criticism (and occasional _hate_ ) ebb and flow over the years. That seems to be the only constant to PHP. :)

<comment><USER16> With regards to "Direct HTML embedding."...

PHP is a Web scripting language (which I consider to be very different from a general-purpose programming language or even a scripting language), designed and developed to generate markup.

I have seen a lot of "templating" libraries (Smarty among them) and all they do is mimic the basic syntax of PHP using something else that is not PHP.

I know the theory ("designers don't know how or want to program"), but learning PHP conditional statements or loops is as difficult or simple as any "templating" library that asks you to learn its exotic syntax.

Now, MVC is good and one should avoid blending presentation code with models and controllers, but most MVC frameworks fail when they require you to instantiate a gazillion extra objects, just to render a Web page.

<comment><USER15> agreed on templating systems...

</comment></comment></comment><comment><USER17> There's no denying PHP's popular, but that doesn't mean it's the best tool for the job.

Most of the arguments I see against it though are really problems with the programmers using it. "Look at this horrible code I have to put up with! The comments are inane and useless, there's no structure at all, and I'm stuck in a chain of nested if statements 10 layers deep and I have no idea what's going on. PHP sucks!"

As Jeff Atwood said, "sufficiently incompetent coders can write FORTRAN in any language."

"But the converse is also true: sufficiently talented coders can write great applications in terrible languages, too."

PHP wasn't exactly designed… it was formed, like a ball of mud picking up various debris as it grew. It shows, too, especially if you've had the pleasure of working with PHP 4. (I can only imagine what earlier versions were like.) I can't for the life of me think of a language with more brain-dead "reference" semantics. It's just bizarre. Their date functions miraculously manage to be _worse_ than their C counterparts. I could go on and on.

But in the end, using PHP doesn't doom your project to failure any more than using the current flavor of the week guarantees your success. Jeff Atwood, again: "The only conclusion I can draw is that building a compelling application is far more important than choice of language."

So why are we still comparing programming languages instead of writing code?

</comment><comment><USER18> I think people are comparing apples to oranges. Ruby on Rails is as some have stated, a framework, much like Zend or cake or CodeIgniter.

I think the author is spot on in this post. PHP is not without its warts, but as a programming language for web development, it is scalabe, reliable and robust.

Can you perhaps do things faster or more elegantly in other languages? Sure, but writing good clean code is really in the hands of the developer. I can do everything in python because its written in python doesn't mean that that code is going to be designed well or scale well.

<comment><USER19> "Scalable" and "reliable", perhaps. "Robust" actually means something, though, and PHP is not it. Its wretched approach to error handling prevents that.

</comment></comment><comment><USER20> PHP's problem is it's soul. I will make an analogy about this. My analogy will be local, but i will explain it so you can understand.

PHP is my university, Gazi University. It first established in 1926 as a Teacher Training Institute. Nobody knew that there will be a medical school or faculty of fine arts or even a faculty of dentistry. 1926 was the third of our newly established Republic and we needed teachers. So, Teacher Training Institute was born.

Our main campus is very small. Law School or dentistry school is not in the campus. Because, as i said, nobody knew we need them in 1926.

Ruby, C# and many other languages are METU: Middle East Technical University. METU was designed as a university. So, they got a gigantic land in Ankara (capital city of Turkey). They build all their building in there.

Back to the PHP: PHP was born to solve one problem. Just one problem. When they got another problem, they solved it and added the function or implementation in the language. PHP is not designed at desk, it was not "feature-complete" at version 1.0. It's still getting more mature as version numbers increase. For example we get "traits" in 5.4.

PHP is also Windows too (both Microsoft and PHP in a HN comment :) The most un-loved things in this community). Remember the video which a guy installs Windows 1.0 and successfully upgrade it to every version of windows with all his data and programs up to Windows 7? even doom from 1993 works! (<http://www.youtube.com/watch?v=vPnehDhGa14>) PHP is like Windows. I know the "parameter order inconsistency" problem too. But you can't solve it after this point of time. You just can't. Like windows, every program should work after the upgrade. There are millions of millions PHP sites. after a "apt-get update;apt-get upgrade" they should work.

By the way, sorry for my bad english.

</comment><comment><USER21> Back when all you had for choices were Perl, Java, and C++ it was easy to see why PHP got big. However there are issues with starting out on PHP now:

1) It's really only meant for web development.

2) There are other languages now that are simple to learn, flexible (more than just for web apps), and powerful (Python, Ruby, and now even Javascript)

Unless PHP has another major evolution, like its move to OO; (or unless I just missed something else big) I just don't see the point of using it unless you or your company been using it for years already. This article doesn't have any real compelling arguments.

Maybe all of this will change once Facebook releases Hiphop VM AND people actually start using it. To this day, I'm not sure why no one started using Hiphop enmass and why people love using the inefficient eval method. O yeah there's also an issue with PHP and logic dealing with concurrency, in that there isn't a good way to distribute work within an app. Then again I could be completely wrong.

<comment><USER15> I've been working with PHP since 1996 and _rarely_ see eval being used. I've worked with hundreds of companies training PHP developers, seen inside (and written) some very large codebases handling millions of dollars of transactions, and don't recall seeing eval in any of them.

"it was easy to see why PHP got big"

It's easy to see why it's staying big:

1\. cheap shared hosting to get started 2\. most hosts offer one-click installation of any sort of application people need on the web (blogs, shopping carts, calendars, social networking, file hosting, etc) 3\. 15+ years of knowledge about what works and what doesn't when attempting to scale PHP to large needs. 4\. ubiquity which keeps a large pool of reasonably priced talent available, which perpetuates the cycle.

When there's a competitive language that can be placed in a directory as FOO.abc, I can hit it as /FOO.abc and have it execute, with html content type sent by default, it'll have a running jump at dethroning PHP.

The barrier to entry is the lowest out there, but the heights it can scale to are facebook-level and beyond.

<comment><USER21> > I've been working with PHP since 1996 and rarely see eval being used.

For some reason most of the major frameworks use it, hence their incompatibility with hiphop; then again I could be wrong.

> 1\. cheap shared hosting to get started

One can argue that all of the major languages have access to this. PHP doesn't have an edge in this area.

> 2\. most hosts offer one-click installation of any sort of application > people need on the web (blogs, shopping carts, calendars, social networking, > file hosting, etc)

With the rise of the cloud hosts and automated config, as well as more automated tools available (e.g. Chef, Puppet), this is a temporary advantage. Deploying on Linode and Heroku and (insert cloud host company) are getting easier and easier.

> 3\. 15+ years of knowledge about what works and what doesn't when attempting > to scale PHP to large needs.

No argument here. My points only apply for people starting now, and not for PHP vets.

> 4\. ubiquity which keeps a large pool of reasonably priced talent available, > which perpetuates the cycle.

No argument here either. My points were being made for individual programmers who are just starting out, and not for companies.

Even from the point of a company though, PHP's current lack of flexibility lowers the chance of innovation imo. The other dynamic languages are just as easy to learn and they just offer more than PHP does.

<comment><USER15> >No argument here. My points only apply for people starting now, and not for PHP vets.

I wasn't meaning for people - I meant 'the industry' as a whole (or a large segment) - standing on the shoulders of giants and all that.

>> 1\. cheap shared hosting to get started

>One can argue that all of the major languages have access to this. PHP doesn't have an edge in this area.

Really? I think there's far more options for people with 'PHP' in the hosting arena. It's not that you can't use other languages/platforms, but ... competition for hosting basic PHP stuff is massive - not so much for Ruby/Rails, Python, etc. along with all the extra stuff you'd need (community libraries and such) - much more is baked in to PHP hosting than you get with the majority of other platforms. Java in particular is expensive to host anything useful because it's RAM hungry.

re: eval - do you have pointers to examples? I just grepped Zend Framework 1.10, and there was _1_ eval, in Serializer/Adapter/PhpCode.php, with a warning that it shouldn't be used.

<comment><USER21> > I wasn't meaning for people - I meant 'the industry' as a whole (or a large > segment) - standing on the shoulders of giants and all that.

Ruby and Python aren't very new. There are giants there as well. If you're going to make that argument, then Java has a bigger advantage.

> I think there's far more options for people with 'PHP' in the hosting arena.

As I've already said, I don't disagree. I just said that this advantage is going away. Personally I'd also rather go for host that's mid-ranged as opposed to hosts that are in the race to the bottom. I don't think paying $10 more a month is exorbitant especially if it buys you a lot of language power and it takes over more sysadmin duties from you.

> re: eval - do you have pointers to examples?

Every major PHP framework uses it. How do I know? To my knowledge you can't use any of them with Facebook's hiphop out of the box. I suspect this is one of the main reason's no one uses hiphop even though it gives you a good deal of performance gains for PHP.

<comment><USER15> "Every major PHP framework uses it. How do I know? To my knowledge you can't use any of them with Facebook's hiphop out of the box. I suspect this is one of the main reason's no one uses hiphop even though it gives you a good deal of performance gains for PHP."

Zend Framework 1.10 - a fairly high-profile framework - has _1_ instance of eval in a component I've never known anyone to use. I suspect hiphop has a few other issues with it, but... even more to the point, I don't think most people _think_ they need hiphop. It's an unknown - it's a compilation step, and adds a couple extra steps of complexity. People can get a lot of performance gains by using Zend Server caching or APC with far less unknowns vs digging in to hiphop.

Additionally, hiphop has had some issues with postgres/pdo, closures, imap and more PHP stuff over the last year or so. It's a nice thing from facebook, but it was written for their own problems, not as a general use tool for everyone. If it can evolve in that direction, the adoption may go up. Until then there's far more well-tested ways to improve your performance of PHP.

</comment></comment></comment></comment></comment></comment><comment><USER22> The issue with PHP isn't that you can't make it do a whole bunch of things that are wonderful...

It's why PHP would be the best tool for the job once you have a specific objective in mind.

When you have a specific problem to solve other than 'supported by millions of cheap webhosts' PHP is rarely the best choice.

It's like a swiss army knife, it's highly useful in many situations and a good tool to have in your pocket, but if you know you're going to need to unscrew something you should probably bring a screw driver.

<comment><USER23> Well... of course you should always determine Use Case first and then choose technology based on that.

</comment></comment><comment><USER24> _[off-topic]_ If PHP and Javascript are the only programming languages you know, you _really_ should experiment with at least one or two others. It comes easier than expected. For example, check out <http://www.diveintopython.net/toc/index.html>. You can't know what you're missing out on until you try something new.

<comment><USER25> But if you already know PHP and Javascript, why not try branching out a bit instead of learning Python, which is very similar overall? How about a lispy language like Racket, or a statically typed functional language like OCaml, or a 'big' mainstream applications language like Java?

<comment><USER24> That's the point. Python _is_ similar to php/js (which is why it's easy to pick up), but in contrast to them it is extremely well designed (imho). You suddenly understand how many of the quirks you have to go through every day coding in php/js are in fact artifacts of these two inconsistent poorly designed (imho) languages rather than programming as a whole.

</comment></comment></comment><comment><USER26> This does not justify chosing PHP for a new project.

There are hundreds of examples that show how bad PHP really is. Why make the choice of using a broken language when there are so many better alternatives?

I strongly believe that using a complex, non consistent and hard to debug language actually slows down development and is even dangerous.

Being "absolutly everywhere" is not an argument. COBOL, PASCAL and FORTRAN used to be everywhere too.

<comment><AUTHOR> No, "absolutely everywhere" is not an argument by itself. But sometimes ubiquity is earned.

If Facebook powered its front end using FORTRAN, I would call it a quiet powerhouse, too!

<comment><USER26> You can always build things with at scale, even with flawed languages (look at javascript).

The thing is, why choose a flawded language when there are others less flawed (nothing is perfect) for new code?

</comment></comment></comment><comment><USER27> I agree that PHP is good for starting learning web development and for small sample apps, but from experience I really think that large-scale PHP applications are not easy to manage. The codebase becomes disorganized very quickly, and tracking down a piece of code's "hierarchy" (what it inherits from, what methods it has available, etc.) can be a whole new kind of hell. IMO, of course.

<comment><USER28> All large apps are a pain regardless of the framework. It's called coupling.

Building your application as a series of small applications that work with each other is a solution that has worked for me.

</comment></comment><comment><USER29> I have a few nits with this post, but am mostly irked by the comparisons between PHP (a language) and Rails (a framework)

<comment><AUTHOR> You're quite right that the analogy isn't perfect. The point I was trying to make, though, is precisely that PHP development can proceed without any recourse whatsoever to a framework. Jumping straight into web dev without an accompanying framework like Rails or Sinatra is a complete non-starter if you're a Rubyist.

So you're quite right, but I hope you would agree that the Rails/PHP comparison, in spite of its imperfections, is compatible with the point I was trying to make.

<comment><USER29> Yes, I agree. You make the point a lot better here than in your linked post - i.e. with PHP, you can 'just start'. With Ruby you need to choose a framework, understand enough about how it works, and where things go before you can even start to do "hello world"

Although that is not necessarily a bad thing.

</comment></comment></comment><comment><USER30> It's interesting seeing so many people defending PHP by suggesting that because Facebook use it, it's good enough for anyone. From what I understand, Facebook continue to use PHP because it's heavily embedded in all of their systems, and they continue to develop with it despite other languages being more attractive? ( see [http://www.quora.com/Why-hasn-t-Facebook-migrated- away-from-...](http://www.quora.com/Why-hasn-t-Facebook-migrated-away-from- PHP) )

</comment><comment><USER31> > I think PHP is the perfect language for beginning web development for two > reasons: "Direct HTML embedding" and "Straightforward file organization"

After thinking about it a bit, it seems that your reasons for suggesting PHP for beginners are about the same as the reasons people gave three decades ago when suggesting BASIC as a language for beginners, namely the norm of having GOTO and global variables everywhere.

Just as GOTO and global variables make it easy for a beginner to easily pick up programming, embedding raw code in your pages and having only one file organization makes it a lot easier to pick up web programming. And yet at the same time, all four are eventually shown to be "considered harmful" - the only ones who advance are those who are willing to abandon the first things they learn while the rest are doomed to mediocrity.

Now don't get me wrong, I don't think PHP or even BASIC is inherently bad (well, they _are_ , but at least they're not MUMPS or Brainfuck). I just don't think teaching bad practices to newbie developers is a good way to promote a language.

</comment><comment><USER32> "It's been very interesting beginning my trajectory in web development in 2012... [PHP] might be the best language to use in gaining an understanding of the fundamentals of web development... No web development language outside of PHP grants you the kind of flexibility to both (a) produce genuine functionality from scratch, outside of an MVC framework, and also to (b) work within the bounds of MVC in deeply sophisticated ways if you wish."

I believe you have overstated your case, perhaps due to inexperience with other languages. In particular, claiming that any modern scripting language is inflexible sounds a bit crazy.

I have attempted explaining how to get a WAMP stack running, with EasyPHP, to an absolutely new-to-web-dev indiviual who was curious about creating a chat room. Let me just say that the situation is ten times better now than when I was the same age and was poring, lost, over Bruce Eckel's _Thinking In Java_ trying to do the same thing, which must have been sometime around 1999/2000.

These efforts to teach a newcomer reminded me that there's a lot I take for granted. There _is_ a step where you are saying "okay, we've got to see where Apache is; what port is it listening on, how can we get you to phpMyAdmin; here's how you set up rows and columns in your tables and think about a database; no the reason it's not working is because you saved as '.html' and not '.php'; I know you've seen URLs that don't end in '.php' but that requires invoking the Rewrite Engine and teaching you about .htaccess files; oh the header modifications can't be the first thing you emit; here's the docs, you basically have to search them because there is no organizing principle and no way to find out what you can do..."

I agree, Python+Django will make you learn a bunch of weird "modeling" stuff first. To me, setting up the data model at the beginning of an app is like inputting a character name at the beginning of an RPG: as some sources testify [1], you can easily waste several hours on purely aesthetic concerns before you even really start to play.

Nonetheless, Python+Django comes with a lightweight dev server and an interactive console and a routing table, as well as a template language. The knowledge of the database is internalized to the language, so that I don't have to teach someone how to use phpMyAdmin but rather when to use a ForeignKey. If we talk purely about _learning to develop on the Web_ then PHP has much, much more useless overhead.

Sure, eventually we will have to get a production-ready web server communicating with the application, but that can be abstracted until later with Django: it occurs as an up-front teaching cost with PHP.

I say this not even as a Python aficionado per se. If anything I like Node, and Node is nowhere near either of them in offering pleasant functionality.

[1] <http://www.cad-comic.com/cad/20030630>

<comment><AUTHOR> I certainly didn't mean to say that languages like Python and Ruby are inflexible!

What I meant to say is that you can't use either of them in direct conjunction with HTML the way that you can use PHP. This was what I meant when I said "genuine functionality from scratch." Would I prefer Python+Django/Flask for lots of use cases. Absolutely.

As for the "useless overhead," there are things like the command line (which Python and Ruby web dev absolutely demand) standing in the way of some of the other languages. Personally, I strongly prefer the command line for almost everything, but for beginners this is another barrier.

My argument was not for the superiority of PHP over other languages, but rather against its casual dismissal from consideration.

<comment><USER26> What advantages does this "direct conjunction with HTML" have over templating solutions? Why do most PHP frameworks implement templates then?

Also, if command line is a barrier for an aspiring (web) programmer, he/she should start looking for a new career path.

<comment><AUTHOR> 1\. I wouldn't want to do anything at a meaningful level of complexity without templates. No question about it. And I think that having so many templating frameworks is wonderful (ERB, Jade, haml, etc.). I was only arguing that PHP lets you dive in without NECESSARILY needing to jump through a templating hoop.

2\. Is it unthinkable that a person could start learning web dev and pick up the command line along the way? I started out learning PHP/MySQL on my old Windows machine, terrified of the CLI, and now I do almost everything on it (and have converted my old PC to Linux). If the command line were a barrier at all points in the learning trajectory, then you might have a point. But I also don't think that command line mastery should be held up as a sine qua non of learning web dev.

<comment><USER26> 1\. Ok so when is it appropriate/useful to use the power of embedded php (in html) instead of templates? Just for learning the first couple of hello worlds? In my experience it becomes a mess to manage when you start having more than one commiter and more than a couple of lines of code.

2\. You don't have to be a command line expert to start using it. All the books at <http://learncodethehardway.org/> show that command line can be integrated in the language learning process seamlessly. No reason to be terrified of it, on the contrary, new coders should embrace it as soon as possible.

<comment><USER33> @1. When the project is only a couple of pages.

</comment></comment></comment></comment></comment></comment><comment><USER4> I've lived Ruby land and Python world for quite some time and I must say it's good to be back in home in PHP where getting things done doesn't involve clever semantics and tying yourself up in layers of abstraction. Oh yeah, and it's nice to read code that has comments explaining what the hell is going on too (seems Rubyists and Pythonistas don't believe in comments, you know, because that'd make the code too obvious)

<comment><USER28> +1 for pragmatism.

Despite the stigma attached to PHP, it's my go to language for web stuff because it gets shit done very very quickly.

You can't beat drop file, edit, refresh and entire deployments which can be done with cp/scp and no process restarts.

<comment><USER26> "gets shit done very very quickly"

In what way does PHP help to get things up and running faster than other languages?

<comment><USER34> Have you used PHP? Spend 5 minutes with it and you'll understand; a very expressive language, simple to learn, massive library of functions, massive number of examples to copy, no framework to learn, no syntactical overhead like in Java, practically everything works with, or has bindings for PHP.

I prefer to work in python, but I don't much care anymore, each language has it's strengths and weakness' but PHP most definitely is good for getting things done "very very quickly"

<comment><USER26> Yes you can get up and running quickly with php, but my comment was mostly about the fact that it is not much slower in other languages (most of the time not slower at all).

And yes I did a lot of PHP but that was 5-6 years ago. I know things have changed a lot since then but the language still has a lot of the same flaws.

<comment><USER28> It is when you consider the deployment models, the development server setup etc.

The language has as many flaws as it's always had but it still works.

</comment></comment></comment></comment></comment></comment><comment><USER35> > _But I also think that it might be the best language to use in gaining an > understanding of the fundamentals of web development_

OP admittedly has less than one year of web development experience. Perhaps it would be wise for OP to gain a more thorough understanding of web programming before coming to conclusions like this.

<comment><AUTHOR> I use words like "might" very intentionally! And yes, right you are that my conclusions about things might change radically. I'm already appalled at some of the things that I thought even weeks ago...

</comment></comment><comment><USER36> I will simply point out that one can do a good deal with a little code using WSGI. As for templating, Python has for some time had very handing formatting.

</comment><comment><USER37> PHP is the broken english of the Internet. Easy to learn, practical, inelligant, ad-hoc.

</comment><comment> guard-of-terra You're killing kittens by using PHP.

PHP could be a wonderful thing if not for its original sins. It is easy and fast, but it is also bad. It is "good enough", and for many years it prevented just "good" from arising and gaining adoption.

I would pick "good" over "good enough" even if it costs me a bit because I think this is fundamentally right.

<comment><AUTHOR> But how often does trying-to-be-better-and-failing come along and torpedo good-enough-for-the-job?

<comment> guard-of-terra Would you listen to "good enough" music or watch "good enough" movies? When I ask this, I think, many do. But I don't.

Do you?

<comment><AUTHOR> No. Sometimes I spend a lot of time and effort seeking out niche genres and digging through blogs and news sites to find "good" music. But sometimes I don't want to put in all that effort, and then putting on "Kind of Blue" or the first Weezer album is "good enough." There is room for both approaches to music listening.

</comment></comment></comment></comment><comment><USER38> Bleh. Would you call McDonald's a culinary powerhouse?

<comment><AUTHOR> Culinary powerhouse? No. But it is undeniably a powerhouse in delivering food consistently and reliably.</comment></comment>


The Acid Aesthetic: A Brief History of Psychedelic Design - <AUTHOR> http://www.printmag.com/design-inspiration/acid-aesthetic-history-of-psychedelic-design/ <comment><USER2> There was a renaissance of grassroots psychedelia in the overlapping hacker and rave culture of the 1990s. I miss that stuff.

Today's culture is disturbingly vapid. There is little in the way of either genuinely creative counterculture or mainstream high culture. The pop culture is five chord pop that all sounds the same and endless remakes of the same film, and geek and fringe culture seems full of low effort meme trash and mean spirited nihilism. It's not at all surprising that fascism has risen up in those festering swamps.

It reminds me of the eerie artistic anticlimax that served as a prelude to collapse in one of Lovecraft's stories about the history of a vanished elder race. "The Cursed City" I think. Like the old ones our art and culture reached a peak from the 60s to the 90s and then gave way to blobs scrawled in MS Paint.

<comment><USER3> "The pop culture is five chord pop that all sounds the same and endless remakes of the same film"

Hahaha. I remember lamenting about the same thing back in the early 90's.

Most of everything is always dull and vapid and so are most people who at best are imitators. When you are there anyway. Looking back it always seems the past was so groundbreaking and exciting. In my late teens/early 20's I wished I could have lived through the 60's (this was in the late 80's- early 90's). Because that was such a truly great and groundbreaking time according to the books. Now I realize it was probably a general stifled societies with a bunch of copycat posers growing their hair out and saying silly things they thought were cool and taking LSD. I laugh my ass off whenever I see 90's idolizing millennials.

Old man rant aside, there has been a pretty big cultural movement in the last 6 or 8 years. I guess a big part of it is the internet. In a way I'm glad I'm not 23 again, what a confusing age, but in some ways I'm a bit jealous. Ah well, I got the 90's. There were some cool times.

<comment><USER4> In the early 1990s, mainstream pop music was arguably a broader scene than it is now, inasmuch as hip-hop and R&B were both undergoing considerable evolution and present in several distinct forms in the charts, rock was still a strong force, and country music was briefly expanding to new markets. Compare that to today when rock is dead and hip-hop and R&B have increasingly merged into a single electronica style.

That said, beneath the surface of that pop mainstream is an incredible diversity of music available to any listener who cares to seek it out, and we are living in a music golden age even if we might be living in a pop culture depression.

<comment><USER3> I don't even know what pop music/culture is anymore. Justin Beiber I guess? We'll we've always had crappy commercial imitations with little artistic value. The 80's were full of it so I listed to "real" music from the 70's (not realizing the 70's were likely full of the same thing but only the quality stuck around).

This ignorance of current culture might because I'm older and don't have a TV. Or it might be that the internet has allowed me to isolate myself in my own little bubble of excellence. And there is some great music around now. The quantity of good stuff is mind boggling. You had like 10-15 choices when I was young. And the mechanisms for discovery. It truly is a dream time for people who love art and music.

</comment></comment></comment></comment><comment><USER5> Very brief, indeed. It left me wanting more.

In particular, I would have mentioned the explosion of independent printing shops in the 60's. There's a great list of counter-culture and political print shops here: [http://www.docspopuli.org/articles/BayAreaRadicalShops.html](http://www.docspopuli.org/articles/BayAreaRadicalShops.html)

I would also love to see more on the role of experimental, underground comics in the psychedelic movement. But then comics are not quite as beloved by design historians and critics the way posters are.

<comment><USER6> If you're interested in the counter-culture of the 1960s the book _BAMN (By Any Means Necessary): Outlaw Manifestos and Ephemera, 1965-70_ is interesting.

[https://www.amazon.co.uk/BAMN-Any-Means-Necessary- Manifestos...](https://www.amazon.co.uk/BAMN-Any-Means-Necessary- Manifestos/dp/0140032673/)

[https://www.goodreads.com/book/show/987215.BAMN](https://www.goodreads.com/book/show/987215.BAMN)

[https://books.google.co.uk/books/about/BAMN_By_Any_Means_Nec...](https://books.google.co.uk/books/about/BAMN_By_Any_Means_Necessary.html?id=dHK6AAAAIAAJ&redir_esc=y)

</comment></comment><comment> castle-bravo > Nazi scientists were among the first to explore LSD's psychopharmaceutical > potential

I'm gonna need a source for this. My understanding is that Swiss chemist Albert Hoffmann discovered the psychedelic effect of LSD in April 1943 [1], and I think it very unlikely that the Nazis could have incorporated it into their purported mind-control experiments before the end of the war.

[1] [https://en.wikipedia.org/wiki/History_of_LSD#.22Bicycle_Day....](https://en.wikipedia.org/wiki/History_of_LSD#.22Bicycle_Day.22)

<comment><USER7> According to Operation Paperclip [1] part of the motivation for the US investigating weaponized LSD was the fear the Nazis were investigating it. But I'm not sure Nazi use was documented to the extent it was in the US, with the CIA's MK-ULTRA program experimenting on unknowing citizens by dosing them with LSD [2].

[1] [https://www.amazon.com/Operation-Paperclip-Intelligence- Prog...](https://www.amazon.com/Operation-Paperclip-Intelligence-Program- Scientists/dp/031622104X) [2] [https://en.wikipedia.org/wiki/Project_MKUltra#Drugs](https://en.wikipedia.org/wiki/Project_MKUltra#Drugs)

<comment> castle-bravo If I understand correctly, saying that Nazi scientists were experimenting with LSD is technically true, but very misleading. The "scientists" (read: professional torturers) were /former/ Nazis working for the CIA /after/ the end of the war. This source [1] states that a paper-clipped Nazi scientist brought LSD to the attention of the CIA in 1948 (though its credibility is suspect because it suggests that a CIA operative died die to LSD poisoning, a highly improbable occurrence).

[1] [http://www.thedailybeast.com/articles/2014/02/11/what- cold-w...](http://www.thedailybeast.com/articles/2014/02/11/what-cold-war-cia- interrogators-learned-from-the-nazis)</comment></comment></comment>


Show HN: Mmap.it – personal search without context switching - <AUTHOR> https://mmap.it <comment><USER2> I love how focused it is! Would this work on linux in case I build it from source? Just tried it out on my mac and seems to be a little buggy at the moment. The editor doesn't seem to reliably respond to keystrokes and the search window went blank a couple of times. I guess it needs some polish, but I like the UI a lot.

<comment><AUTHOR> Thanks that really means a lot, the feedback is very much appreciated!

Building from source should work on linux. I'm still ironing out a lot of the kinks, the search window going blank is at the top of my list and I am also working on editor reliability.

Hope you can get the full value out of this once I've made it more reliable!

</comment></comment><comment><USER3> This is neat! Can you tell me how you got around to building this and what it's most useful for? I'm not a programmer but I've been thinking a lot about migrating out of Evernote to Roam or Notion and this made me think of that.

<comment><AUTHOR> Thanks! Glad you think so. I've found it's most useful for workflow documentation and code snippets.

The main reason I built mmap is I started a new job a few months ago which required learning a ton of new workflows that leveraged a bunch of unfamiliar technologies. Documentation for these was scattered over github, confluence, google groups, etc. which made it both time consuming and difficult to find what I was looking for.

I wanted to be able to save documentation and code snippets in my own words so I could quickly find it without switching windows. It's lead to a noticeable jump in my productivity so far.

</comment></comment><comment><USER4> As this runs on macos, any plans to make it universal binary for both macos and iOS/iPadOS?

<comment><AUTHOR> We are planning on releasing for both windows and linux soon. Due to the app being focused on accessibility for developer workflows, mobile applications are a little further out on the product roadmap but will be coming eventually!

</comment></comment><comment><USER5> Any plans for dark mode? It's a dealbreaker not having it on Mac post-Mojave IMO.

<comment><AUTHOR> Yes 100%, you're not the first person to ask. It might be sacrilegious to admit but I never use dark mode so it slipped to back of mind.</comment></comment>