Every attempt to manage academia makes it worse - <AUTHOR> https://svpow.com/2017/03/17/every-attempt-to-manage-academia-makes-it-worse/ <comment><USER2> _If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability._

Absolutely, could not agree more. This is the only way to beat gamification of metrics in my opinion.

There is a worrying trend of treating staff as equally capable but untrusted cogs in a wheel, and metricising performance. I fundamentally believe that no amount of rules or metrics can turn a bad academic into a good one, an unsafe doctor into a safe one or a crooked politician into an honest public servant. We have to calm down with the metrics, accreditations and regulations, and accept that some people are better than others at certain things, hire the right people and trust them to do a good job.

<comment><USER3> > There is a worrying trend of treating staff as equally capable but untrusted > cogs in a wheel, and metricising performance.

That's a great way of summing up what I've heard called "managerialism".

There's a theory behind, e.g., the MBA that a competent manager can manage anything. That what one manages makes no difference as long as you're a real manager. One of the corollaries of that theory is that all one has to do is find the right short-term metrics and make them go up and to the right.

I don't think this theory makes much sense when you examine it. But it's excellent at justifying a caste system, one where all workers are controlled by wise managers, and where there's a stable hierarchy of executives above that. I note that the hierarchy established is basically equivalent to those of feudalism. It also has the handy property of justifying high compensation for managers and executives; since workers are cogs and higher-ups do the important thinking, all gains should flow upward.

There are other ways of organizing things, of course. But this has become such the default that people have a hard time even imagining alternatives.

<comment><USER4> I tend to find people with your view have never been an executive managing the managers, or even a manager. When you are (like myself) then you see that good managers / leadership (including tech team leads, etc.) really do make a massive difference in performance with the same people.

Things like metrics, values, and re-orgs might seem like Dilbert-esque corporate speak (a view I used to share), but they're incredibly important once you're trying to get a bunch of people to do something with excellence and speed.

<comment><USER5> I would be more inclined to believe this if the pro-manager view didn't come from the managers the vast majority of the time. It seems too convenient and self-service. Of _course_ managers want to think they make a massive difference.

Certainly some aspects can: keeping politics away from your group so that they can work, focusing work tasks by eliminating extraneous ones, advocating for people in the company. However, once you start getting into things like values and metrics it starts becoming a lot more grey. On the one hand metrics are a great idea, a tiny step towards making management more science driven. But on the other, the metrics so often used are such a poor and overly simple representation of a very complex system that it's often hard to see how they provide an accurate view of any given work place.

Goals and values are often completely lame. We value hard work! Your goal is to do your job! Somehow managers must know that they are ridiculed for this. After all entire movies are made doing so. Yet they still seem to want to believe this rhetoric works.

<comment><USER6> I believe "The Zen of Python" is one of the reasons for Python's success. I also believe that most "goals and values" that CEOs obsess over are bogus, but there's a kernel of truth that setting the philosophy for the company has a subtle, but strong effect.

Your skepticism based on who's speaking is warranted. After all, you shouldn't ask your barber if you need a haircut. Yet, maybe your hair stylist has the most expertise. Should you trust your dentist on how often you should get your teeth cleaned?

</comment></comment></comment></comment></comment><comment><USER7> _"If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it._

\-- Attributed to J.J. Thomson (although I've not been able to turn up a definitive citation -- anyone know where it comes from?)

Does make me think some people have been pondering this for a long while though.

<comment><USER8> There's a long tradition of important advances in knowledge being made by people with an undemanding job [1] [2]. The 19th century university became a hotbed of research because the teaching workload was relatively light and they didn't _require_ that you also did research.

[1] [https://en.wikipedia.org/wiki/Parson- naturalist](https://en.wikipedia.org/wiki/Parson-naturalist)

[2] [https://en.wikipedia.org/wiki/Baruch_Spinoza#Lens- grinding_a...](https://en.wikipedia.org/wiki/Baruch_Spinoza#Lens- grinding_and_optics)

<comment><USER9> Siddhartha Gautama (the Buddha) was (became; established a whole tradition of being) a hermit/beggar/monk; Socrates was a stone mason (occasionally); Diogenes the Cynic lived in a barrel and begged for food; Jesus was a carpenter and then a wandering preacher; Spinoza (as you note) was a lens grinder; Kant was a tutor; Einstein was a patent clerk. And, as you note, especially in the 19th and 18th centuries, many proto-scientists were people with either an established income or independent wealth, who had the freedom to pursue whatever interested them.

We ought to be finding ways to broaden the number of people who can pursue their interests, while not forfeiting security, dignity, and social respect. This is one of the arguments for universal basic income, that it could decouple (some) people's desire to pursue knowledge from the need to earn a living, and maybe even especially from the need to produce regular, measurable results. Right not we're channeling most of our people who are interested in and capable of developing new knowledge into jobs which demand "results" more than they demand that truth be followed wherever it leads, and however long it takes. This will lead to a great deal of dubious Normal Science [0], and very little revolutionary thought [1].

[0] [https://en.wikipedia.org/wiki/Normal_science](https://en.wikipedia.org/wiki/Normal_science) [1] [https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...](https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions)

<comment><USER8> Note that having a job seems necessary though. Maybe it's a case of the golden mean? On one side you have people whose job absorbs all leisure. On the other you have some who draw sufficient income without having a job at all. These are Marx's infamous "coupon clippers", and historically they're not as prominent in intellectual history as people with undemanding jobs (personally I'm not aware of many philosophers / scientists who were rentiers). If leisure was the only variable, we'd expect people with guaranteed incomes (annuities, for example, have been sold since the Middle Ages) to have been the most prolific thinkers of history.

<comment><USER10> i think you have to go back to the 18th century for that. i'm thinking britain where science was something acceptable that gentlemen could do in their spare time (with smart+poor doing a lot of heavy lifting) and in particular Antoine Lavoisier in France who literally bought the rights to be a tax farmer so he could do his research. he lost his head in the revolution so there's that. YMMV.

<comment><USER11> Kelvin, De Broglie, and Brahe come to mind as well.

<comment><USER8> Kelvin taught at Glasgow, Brahe was a cathedral canon, and De Broglie did his best work for his PhD thesis.

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> That list is correct, but it's only half of the problem. The other other half of the problem is supply.

In fields like mathematics, physics, computer science, grad students have excellent non-academic options. There is an oversupply of PhD students with respect to the number of available positions, but the excess PhD students have fantastic career avenues.

In fields like the humanities, and, to a lesser extent, the molecular biology/ecology, there is a huge excess of PhD students, and there is a very limited number of career options outside of research (for bio, there is pharma, for everything else, eh). This is compounded by the fact that for professors, postdocs are an absolute steal ([http://www.sciencemag.org/careers/2017/01/price-doing- postdo...](http://www.sciencemag.org/careers/2017/01/price-doing-postdoc)). Professors get to pay 50-55k (and that's on the high very high end) a year for a PhD trained researcher who will often work themselves to the bone in the hope of getting the publications that will allow them to become a professor. Thus - the very people who have the power to change the system instead greatly benefit from the status quo.

The real reform that has to take place is that PhD programs need to become way more selective. Not every PhD should have a guaranteed spot as a professor (that would be insane) but when less than 8% of postdocs become professors, the system is in crisis.

<comment><USER13> > Not every PhD should have a guaranteed spot as a professor (that would be > insane) but when less than 8% of postdocs become professors, the system is > in crisis.

There are lots of industries like this, e.g., corporate law, sports, acting, music. Are they all "in crisis"?

[http://slatestarcodex.com/2015/07/28/non-dual- awareness/](http://slatestarcodex.com/2015/07/28/non-dual-awareness/)

[EDIT] Others: military officers, investment banking, organized crime, tech start-up, trade guilds, etc.

<comment><USER14> With the exception of performances (sports, acting, music) you can make a decent living despite not being at the next level. It is generally not so with academia.

And even with the stuff like acting and music, the advancemnet of society doesn't depend on the efforts of _professional_ athletes and performers. So we tend to be a lot more tolerant of people falling out of that pipeline and winding up in more prosaic positions and honing their craft in community theater or open mic nights or whatever while making ends meet some other way. With research it's not the sort of thing you can do on the side after a day spent waiting tables.

<comment><USER15> > With research it's not the sort of thing you can do on the side after a day > spent waiting tables.

Depends on your level of engagement. Can you explore your own hypotheses and put together your own experiments? Probably not.

Can you engage in citizen science projects like BOINC or Zooniverse (essentially performing drudge work that would otherwise be thrown at some hapless research assistant at the bottom of the totem pole)? Of course!

<comment><USER16> > Can you explore your own hypotheses and put together your own experiments? > Probably not.

Can you expand on this? I don't see why you couldn't, outside of the amount of money that may be needed for certain research.

<comment><USER15> My argument for why you couldn't is more one of pragmatism than anything else. After 8 hours slogging away in a cubicle, you're unlikely to have the energy to work with the laser-like precision necessary to tackle a scientific problem. Perhaps it is possible in some exceptional cases, but for the vast majority of people I don't see it happening.

Plus, science is as much about your interactions with your peers as it is your work. Individuals that aren't highly embedded in a scientific clique are liable to miss important information that would have an impact on your research. You can't develop the same level of social capital as you'd need doing science part-time as a hobby.

</comment></comment></comment></comment></comment></comment><comment><USER17> I'm sure this resonates with university lecturers all over the Western world. The late writer and university academic Mark Fisher has some interesting points about this effect in his 2009 book "Capitalist Realism: Is there no alternative?"[0]. He saw it as an inevitable byproduct of the infiltration of neo-liberal ideology into the academic sphere. Setting targets and measuring performance indicators are justified with efficiency arguments, but ultimately cause an increase in bureaucracy and a decline in mental health.

[0] [https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_n...](https://en.wikipedia.org/wiki/Capitalist_Realism:_Is_there_no_alternative%3F)

<comment><USER18> Setting targets won't solve anything since this is a structural problem. You cannot leave those structures in place and expect a different outcome. That is just treating the symptoms.

In my view the solution is simple: instead of letting bureaucrats and administrators decide, let parents and students choose the classes and teachers they want. Those teachers that offer classes and a teaching style that students actually want will do fine, those that don't will have to find other jobs. Problem solved without artificial metrics that never work anyway.

Besides, it's the students money. Why shouldn't they be able to choose what they want to learn and who they want to learn from? I suppose you could make the authoritarian argument that the experts know better than you and should be allowed to force you take the classes they think best. But that kind of thinking is the root cause of the current problem.

<comment><USER19> > I suppose you could make the authoritarian argument that the experts know > better than you and should be allowed to force you take the classes they > think best.

I'm not sure that's especially authoritarian rather than just practical. I have a CS PhD focused on optimization and machine learning. I trust a student to mostly decide whether they feel like computer science is a good field for them to go into, but once they've made the choice to work on machine learning, they have very few useful opinions to offer. I don't care if the student thinks statistics or linear algebra are useful, boring, whatever. Part of what that money of theirs is buying is for someone like me to make the plan that gets them from where they are now to where they want to be.

<comment><USER18> I don't disagree with your point. I was mostly thinking of general education and the high school curricula.

On the other hand, I'm not a big fan of the PhD style program. It seems old- fashioned and possibly obsolete to me. I would prefer to see it replaced by something like, in the case you mention, an advanced certification in machine learning. I don't know the field well enough to say what such a certification might entail but maybe testing and some kind of project?

<comment><USER19> I think what you're describing is a need that's filled pretty well today by a master's degree.

I don't think the PhD is so much archaic as just overused. It is intended to signify that a person has demonstrated that they can contribute significant new knowledge to a field of study, and incidentally, that they can teach (graduate) university courses. Society probably needs far less of both those traits than it can supply people who want to do them.

</comment></comment></comment></comment></comment><comment><USER20> My wife works as a teacher at one of the largest schoolboards in Québec, and she can tell you that the solution "hire good people, pay them the going rate and tell them to do the job to the best of their ability" doesn't work in a unionized system.

Complacency a systemic problem. When your union protects your job, you don't need to actually put effort into doing the job. Of course, that doesn't mean ALL teachers are lazy; but it does mean that a teacher who works hard and gets better results than average is not rewarded for it. It also means that bad teachers cannot be replaced by good ones.

Of course, preventing public school teachers from unionizing will simply allow the government to exploit them. Perhaps the solution would be to replace unions with a professional order, similar to how engineers and lawyers are regulated.

<comment><USER13> > Of course, preventing public school teachers from unionizing will simply > allow the government to exploit them.

What are the examples in modern liberal democracies where governments exploited non-unionized workers?

The arguments for unions assume an adversarial relationship between employer and employees. But they don't go through for governments.

[http://www.economist.com/blogs/democracyinamerica/2011/02/bu...](http://www.economist.com/blogs/democracyinamerica/2011/02/budgets_and_bargaining_power)

<comment><USER21> Probably more abstract than you were hoping, but look at the controversy surrounding public education in the US right now. It's always been a politically 'hot' area and each new administration wants to de-regulate or re- regulate the sector in some new and ever more confusing way. If it wasn't for the unions, I'm sure some administration would come up with the idea of firing the bottom 10% of teachers based on standardized test scores or something similarly misguided.

<comment><USER13> My criticism of that example isn't that it's abstract, it's that it's purely hypothetical. I'm claiming there's no evidence to back up the assertion that there would be bad effects without public sector unions, so yet another assertion of this does not really change anything.

<comment><USER21> Fair. I would assert that one would only have to look as far as the parent article to get an indication of where things would go however. As others have said, adding metrics that become targets is bad, and in academia this usually turns into using publishing to determine who gets tenure. In this sense, professors are essentially non-union teachers and their promotions are based on random metrics.

I should also mention that we've already seen some of these effects in public schools with the "teaching to the tests" debate in which students are often subjected to a large number of standardized tests that are used to make funding decisions.

I think the problem is that in the end, any debate over schools at the federal level will always turn into something reminiscent of the gun regulation debate where neither side's argument is founded in reality and as such, the teachers will be the ones to suffer.

</comment></comment></comment></comment></comment><comment><USER22> I'm currently a lecturer at a small midwestern liberal arts university. I don't know what kind of job I will have next year because the Dean decided that there will no longer be one year full time contracts and they won't add any more three year contracts. Theoretically they want to have more tenure track job but in practice our department isn't getting any more.

So at worst I will be part time next year which means I won't have insurance. Best case one of the job applications I have pans out and I get a tenure track job.

The reason for the change is, of course, money. It is cheaper to hire 5 part time people to teach the load of 3 full time people because the part timers get less pay and no benefits. Of course finding 2 extra people who are qualified doesn't seem to be something they are worried about...

Strangely, they haven't suggested switching to part-time administrators...

<comment><USER23> of course that would be outrageous! we should be giving them millions in bonus!

</comment></comment><comment><USER24> I think this is somewhat well understood in things like Sports Psychology. You don't tell a receiver to catch X number of throws per game, you just encourage them perform. Likewise, training that more accurately simulates performance is more effective than working toward some metric.

I learned it from Pat McNamara who has a special way with words and delivery: "When we are consumed by the outcome it will sabotage our ability to perform." \- [http://soldiersystems.net/2012/09/15/gunfighter-moment- pat-m...](http://soldiersystems.net/2012/09/15/gunfighter-moment-pat- mcnamara/)

<comment><USER25> This works because both sides have a well-understood model of what 'performing' looks like. And if you don't, you'll be shown the door very quickly.

<comment><USER26> I think this is key. In the sports arena, the metrics are VERY clear: if you don't win games, and your stats are poor, you're fired.

I understand where academics are coming from, but at the same time it seems like many people want to be isolated from any objective assessment of their performance at all. FWIW, I see this a LOT among software developers as well.

</comment></comment></comment><comment><USER27> >"could it possibly be … that the best way to get good research and publications out of scholars is to hire good people, pay them the going rate and tell them to do the job to the best of their ability?"

Sounds very reasonable. Does anybody have good examples of this approach? I have the impression that this is similar to the environment of Bell Labs during the "golden years".

<comment><USER28> You're going to need some sort of controls, if only to kick out the folks who retire in place.

<comment><USER29> Retiring in place is actually quite boring, and the feeling of uselessness that results is depressing. People won't actually do that unless the alternative is worse. If you hire people who are passionate about whatever it they were hired to work on, they might not do work that is profitable, but they'll do work that is interesting and has some benefit.

<comment><USER30> I believe you undevalue the passion some people have for not working.

<comment><USER29> Those people will still have a passion for something, or they're going to be profoundly unhappy. Consumption (whether it be TV, video games, drugs, etc) can only provide some distraction from the pain, it doesn't make it go away. In fact, the longer you try to deal with the pain by distraction, the worse it gets.

</comment></comment></comment></comment></comment><comment><USER31> This isn't an academic issue, the same is true of almost every white collar profession.

To apply standard numerical management techniques you either need to know exactly what a person needs to do as part of the process, or be able to evaluate the output, ideally both, and generally neither are true.

</comment><comment><USER32> Reminds me of James C. Scott's "Seeing like a State", which chronicles how various attempts to make complex, organically developed institutions organized and intelligible has historically made things much, much worse, from planned cities to collective farms. If it ain't broke...

</comment><comment><USER33> While I think there is a lot of truth to this, I don't see how it can effectively be applied to our existing educational institutions. They've sort of become a self-serving bureaucracy at this point, so even without misguided policy pushing them in the wrong direction, I don't think doing the best work they can is actually their primary focus.

<comment><USER34> Naive response: Throw out the bureaucrats. Leave those with subject knowledge. Tell them to do their work to the best of their ability? Stop measuring things. Talk to them once in a while.

<comment><USER35> _Throw out the bureaucrats. Leave those with subject knowledge._

My wife works in academia and all the 'managers' in her research department are experienced researchers that often split their time between management and research. And it just does not work according to her. They don't understand the job of a manager, deem it less important than their research duties and basically see dealing with 'minor' personnel issues as a waste of their time. To run any organization of a non-trivial size requires some structure and bureaucracy. And you need people to understand, set up and own those structures.

<comment><USER36> Really though, _what_ is there in an university that justifies a dedicated _team_ of managers? One person in power (say, the dean) is enough to make sure grants are coming on time, and negotiate those beforehand. They need one accountant to make sure salaries, taxes etc. are in the legal bounds and people receive what they were promised. Honest question: what else is needed?

Most of the managerial work I've witnessed in an university is easily doable by several slightly complex Excel spreadsheets. It just requires people to be diligent and not overlook the unquestionable need for accountability and visibility.

As an additional question -- what exactly is there to "run" in an university?

<comment><USER35> _Really though, what is there in an university that justifies a dedicated team of managers?_

The same thing there is in any company with several thousand employees (let's not even start to consider the administrative hassle of dealing with several thousand students). Probably a lot more actually, since very few companies are doing research in oriental languages, nano technology, abstract algebra and contemporary music at the same time.

<comment><USER36> You might be right, but my girlfriend currently is a "leader" of her group in the university -- she's responsible to bring the student's "grade books" (sorry, my English fails me here) to the proper administration, she creates polls for the freely choosable disciplines, makes calendar events for the occasional meets etc.

Takes her 2-3 hours a week and is FAR more efficient than the awfully rigid systems I come from, 15 years ago.

Self-organization is a powerful phenomena. It can't solve everything for sure, but a good balance between it plus a smaller bureaucracy has in my eyes shown a strong promise for the future.

</comment></comment></comment></comment></comment></comment><comment><USER37> In the most non-trolling way that I can manage, why can this not be generalized to the economy?

Why is academia a special case scenario where bureaucracy and regulations create unintended consequences and cause people to expend effort to get around them, but we can't say the same thing about the economy as a whole?

Why do we believe that people can't even manage PhD's at a university, but we can take the same people, put them in government, and expect them to, for example, manage the monetary supply or see bubbles in advance and create proper policy to avoid them?

<comment><USER38> I think one of the hard things about academia (by ways of "pure research"), is measuring impacts. In some positions there are metrics which are more representative of real progress than others. My belief is that measuring an academic's "impact" is a very fuzzy thing in the long term, especially when you bring in people who don't specialize in the same field. That's a reason this can't be generalized fully, though you can probably find corollaries.

But, your last sentence to me, seems like a good one. Presumably, that would be an excellent topic to pursue research in, considering we clearly don't have a working theory on it (but that's not my specialty so who knows, maybe there's a bunch of smart people being ignored about it). Thus, it's hard to measure progress in, and we institute perverse incentives per the article.

</comment></comment><comment><USER39> So many early-career researchers (myself included) bemoan this system and it pushes many of the away from academia and towards fields where you aren't constantly playing tricks (fake co-authorship, salami publishing, over-hyping conclusions) to get ahead in a game where everyone's striving for a tiny number of badly paid jobs.

</comment><comment><USER40> > as soon as you try to measure how well people are doing, they will switch to > optimising for whatever you're measuring, rather than putting their best > efforts into actually doing good work.

The Heisenberg Uncertainty Principle has found a new refuge. While atoms are evading it ([1]), people implement it.

[1] [https://www.scientificamerican.com/article/atomic-spins- evad...](https://www.scientificamerican.com/article/atomic-spins-evade- heisenberg-uncertainty-principle/)

</comment><comment><USER41> If the blog post seems anecdotal, there's a great, underused book, 'Measuring and Managing Performance in Organizations' by Robert Austin that covers his CMU PhD work, which came to the same conclusion as the quote attributed to Harford: "If a job is complex, multifaceted and involves subtle trade-offs, the best approach is to hire good people, pay them the going rate and tell them to do the job to the best of their ability." It doesn't hurt to make sure they don't have to worry about where to get, e.g., paper clips, lunch, or health care, so some level of administration is helpful.

</comment><comment><USER42> My take on the desire to regulate/manage is that some people would rather have an economy of perverse incentives than tolerate the inevitable abuses caused by lack of accountability. Management in the programming world feels the same way. At first, to many of us, this seems foolish and counter-productive, but then think: how does a university policy maker or a manager explain the rotten eggs (low output academics, unproductive developers) to their "boss," which might be the tax paying public or a budget strapped firm? How does a competitive university distinguish itself in a communicable/verifiable way? Could I be fired, or in the private sector sued for letting smart people fuck up unsupervised? It is difficult to trust others when the worst case outcome exposes us to catastrophic consequences.

I personally agree with the author, but I think there's serious work that could be done to mitigate or remove the incentives _to_ manage, and I don't think those are as thin as "tell people to stop trying to control what they can't"

</comment><comment><USER43> I have a different take its the lack of pay for professors and the lack of respect for public education (On average a college professor makes less then a public school teacher, in my discipline I would have averaged about $20k less IF I got tenure)

As a former academic professional (Librarian at a college) The high cost of college is based on the demands of students. 30 years ago there was no internet, no smoothy bar equipped fitness rooms, no pretty dorms and everything upscale. No pay increase for staff and I actually went through 3 years with no cost of living increases while the cost for college went up 15%.

Then the idea that Public Education in the US is a failure (Totally false narrative except in the cities and other low income communities). So people believe that private investment and legislation will fix things. This just has people coming in making and taking the money from teachers and the community and taking them else where.

<comment><USER44> When I ask the admins at my school about this, they laugh. If you look at the pie chart of where the money goes, it basically goes to salary. So the rise in cost _has_ to be from a rise in salaries paid.

They say the driver in the US is health care, but there are also more people on the payroll.

<comment><USER43> LOL it is also the Admins that got the biggest increase in that pie. I lost all faith in any college admins and is 100% why I left.

There is no evidence that the payroll of colleges have increased and evidence points to lower salaries in since 1970. Administration numbers have increased 4 fold. Less then half of professors are full time now.

> [https://www.nytimes.com/2015/04/05/opinion/sunday/the- > real-r...](https://www.nytimes.com/2015/04/05/opinion/sunday/the-real- > reason-college-tuition-costs-so-much.html)

[https://www.nytimes.com/2015/04/05/opinion/sunday/the- real-r...](https://www.nytimes.com/2015/04/05/opinion/sunday/the-real-reason- college-tuition-costs-so-much.html)

</comment></comment></comment><comment><USER45> Back in 2010 there was an attempt to put Deliverology in place at California State University. It basically means that academia will be measured by _targets_. A horrible idea that made the California Faculty Association meet and decide on almost mutiny. They even invited John Seddon who wrote several books on the subject of how Deliverylogy twisted and perverted systems in the UK. Fortunately for us, the video of his presentation is available on YouTube at [https://youtu.be/8EGMlCau6iU](https://youtu.be/8EGMlCau6iU) for part 1, and has about 5-6 parts.

If you liked that, then I would highly recommend watching John Seddon's presentation about how settings targets and implementing broken IT systems to enforce those targets are a horrible idea. [https://youtu.be/hbNsQFd8DQw](https://youtu.be/hbNsQFd8DQw)

<comment><USER45> The full explanation of CFA Deliverology problem in 2010 is located on their page on the subject: [http://www.calfac.org/resource/deliverology-etc- resources](http://www.calfac.org/resource/deliverology-etc-resources)

</comment></comment><comment><USER46> I think we need to rethink what we consider academically important which should be taught at universities and then what should be taught as self studies.

I can't help but thinking that one of the real problems is that a lot of academia isn't that valuable anymore as a field of study.

Don't get me wrong I consider ex. philosophy as one of the most important fields but I also feel like it doesn't belong as an indiependent discipline anymore. Maybe I am wrong but after postmodernism I really don't see what new revelations are going to be of fundamental value to society.

I see plenty of need for applying philosophy in various fields as interpretation of those fields such as neuroscience, quantum mechanics, technology etc. but as a stand alone field I think it gave us what we needed. And even if I am wrong you don't need an academic faculty to support it anymore.

I feel the same with something like psychology which to me is even more problematic as it it's based on a premise that you can learn about the human mind by studying humans. Again it should be tied to something like Neuroscience instead.

That way the fields are put to good use rather than being self servant.

I am aware this might not be a popular opinion and it's not something that I am 100% certain of myself but it seems to be that a lot of academia could be learned without large institutions to support and and thus with the need to measure it the only way large institutions normally can.

Thoughts?

<comment><USER47> It's really easy to make punchy-sounding arguments about why a certain field isn't as important to teach, so we should be rather reluctant to go along with them.

For example your argument about psychology could equally applied to almost any field and physics. "I feel the same way about (Computer Science|Biology|Chemistry) which to me is even more problematic as it is based on the premise that we can learn about (computers|life|chemicals) by studying (code|proteins|chemicals). Again it should be tied to something like quantum physics instead." In fact this argument makes no sense to me, why wouldn't studying human behaviour be the best way to learn about human behaviour?

Same for the argument about philosophy, it's too easy to assume something valuable a field produces is too amazing to ever be topped. "After [the light bulb] I really don't see what new revelations [about electricity] are going to be of fundamental value to society."

<comment><USER46> I know which is why I said it with some reluctance. But your rubutals aren't good examples.

Physics, Computer Science and Chemistry still have many still unanswered questions and provide clear benefits and ways to uncover those.

In psychology we have a much better way into the human mind through neuroscience. Asking humans about their own behavior is filled with the very bias that we try to battle through the scientific method.

Comparing philosophy with the lightbulb doesn't make much sense as the value of the lightbulb doesn't need to be peer reviewed to show it's value.

The problem with many of these old fields is that they have become self- referential. Their contribution to society the last many decades aren't warrenting the popularity of them.

We aren't studying alchemy anymore either because we have found more and better ways to study chemistry all I am saying is that perhaps it's time to park some of these fields in the alchemy department.

Edit: Ironic that someone disagreeing with me on this very important discussion didn't find much more argument than a downvote :) Oh well.

<comment><USER47> Why do you think neuroscience is a better way to learn about human behaviour (which is what psychology deals with)? Neuroscience is incredibly difficult and the brain is super complex. We have learned far far less about human behaviour through neuroscience than through psychology studies.

Learning about behaviour through neuroscience is working at the wrong level of abstraction, we're not capable of comprehending large systems at low levels.

You cite bias in psych studies, but neuroscience has lots of issues too. See fmri study that can find results on a dead salmon. Also inability of neuroscience tools to figure out what the net of a tiny microprocessor does, let alone anything larger.

I wasn't the one who down voted you.

<comment><USER46> What I said was that psychology should be used as an extension of interpreting neuroscientific findings.

We haven't learned anything about the brain through psychology we have learned about how humans interpret the brain but we don't have a lot of raw data. That's what neuroscience gives us and so it allow us to focus on interpreting the data rather than making a lot of flawed experiments which are often anecdotal more than anything else.

Again I am pretty sure psychology will be seen as alchemy in 50 years if not before.

And yes I know you didn't downvote :)

<comment><USER47> Just because psychology doesn't tell us about how the brain functions doesn't mean it isn't useful. Learning about how humans behave in practice is arguably more useful.

And yes tons of psych studies are flawed but the reason we have the replication crisis is that we can actually run sufficiently large well- controlled studies that we have some confidence in the results, and when we run those we do learn important things like "that previous study was totally wrong". A lot of things like our knowledge of heuristics and biases (not the scientific ones mentioned earlier but cognitive ones) and our solid psychometric measures have come from psych studies.

I do agree with you that we basically shouldn't believe the majority of psych studies, _especially_ if they are in the news. However, the crappy studies help guide where we bother to do the large studies, and provide fodder for the meta-analyses. That's what we learn from.

<comment><USER46> I never said it wasnt useful what i said is that its not the best method anymore and thus shouldn't be a separate faculty but instead be tied to neuroscience exactly to better utilize the raw data which neuroscience gives us.

Not really sure why this is such a controversial suggestion.

<comment><USER48> It's seen as an overly medical response ("Take your meds, you'll be fine") to a thing that has a mix of biological, psychological, and social causes. Treatments should include the range of bio-psycho-social options, even for the more biological illnesses.

<comment><USER46> We are talking research here there is nothing in my model that hinders anything all it does is realign psychology with were it belongs in a scientific context, otherwise we should still have alchemy too.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> This article, and the follow-up on it (link is in the article) is about something very important, and should be more widely known about and discussed by anyone with an interest in science, research, and funding for it all.

It points (yet again - as we've all seen the various p-hacking commentary) to the very real problems in scientific research that seemingly threaten to undermine it - especially from the public's point of view.

It has been said that "religion poisons everything" \- there is probably a corollary of "politics poisons everything" as well; both of these are really just social control and exhibition proxies. Ultimately, human nature rears its head, and the want for more money, power, and prestige (at all levels) leads to these results.

...and society becomes poorer for it.

The questions and the proposed ideas aiming at solutions to the problems discussed sound plausible (or workable) on the surface, but I tend to wonder if any and all solutions will just be able to be gamed anyhow?

At any rate, I'm glad that this was posted, though I despair at it leading to any solution, as it seems the problems lie within our psychology and society, and are thus nearly intractable.

</comment><comment><USER49> I had a similar problem with hiring product designers. How do you get & measure great design? What I ultimately decided was, the only way to get great design, is by hiring great designers, and only other great designers recognize who's very promising vs. who's not, with some degree of success.

Research is the same. However, due to the inherent entanglement of historical biases with merit, it became very fraught in the U.S. to rely on expert judgment to pick new professors, every school clamoring to find some objective metrics to avoid charges of discrimination.

This is coupled with the fact that most departments are filled with mediocre people who are not particularly qualified to pick promising young scientists to begin with.

So ironically, discrimination is still there & at times worse, but now there is metricized cover for this bias. "We found that tall white dude X has more publications than Y".

Meanwhile, academia right now is almost irrevocably filled with pedantic scientists, drip-feeding papers with maximal verbiage at a steady pace.

</comment><comment><USER45> "Tell me how you measure me and I will tell you how I will behave. If you measure me in an illogical way… do not complain about illogical behavior…" - Dr. Eliyahu M. Goldratt

</comment><comment><USER50> The system is managed perfectly to minimise agendas being undermined. Those in power have worked their entire lives to gain it and will have no issue papering over any accusation that veracity is not their sole priority, let alone a complete farce. "In many scientific fields, results are often difficult to reproduce accurately, being obscured by noise, artefacts, and other extraneous data. That means that even if a scientist does falsify data, they can expect to get away with it – or at least claim innocence if their results conflict with others in the same field. There are no "scientific police" who are trained to fight scientific crimes; all investigations are made by experts in science but amateurs in dealing with criminals. It is relatively easy to cheat although difficult to know exactly how many scientists fabricate data" [https://en.wikipedia.org/wiki/Scientific_misconduct](https://en.wikipedia.org/wiki/Scientific_misconduct)

</comment><comment><USER51> I think one way to 'treat' this would be to acknowledge that certain things just cannot be optimized easily with outside incentives, and to simply accept that there will be some _inevitable_ waste when you let 'good people do their job as best as they can'.

It's ironic that we fret so much about professors that 'retire on the job', but at the same time think it is perfectly fine to steal time from the motivated researchers, by making them playing these silly games too.

Who really thinks the next breakthrough will come from a person that was forced into writing another paper, just to pass some arbitrary performance threshold?

If you consider research as a 'stochastic race', i.e. many people working on a problem with their own intuitions and approaches, it makes no sense to slow down those that are most likely to come up with a good solution (those that are self-motivated). IMHO, it does not really matter if those that are unlikely to contribute much (retired on the job) get a free pass. After all, even to receive tenure takes a lot of effort (and thus, ambition) nowadays, so at least in fields with a competitive job market (i.e. good salaries outside academia) it makes no sense to strive for a tenured position and then not do what you love (i.e., research).

BTW Daniel Lemire's blog has many interesting posts on the more absurd aspects of academia, including research grants (e.g. [http://lemire.me/blog/2009/09/15/the-hard-truth-about- resear...](http://lemire.me/blog/2009/09/15/the-hard-truth-about-research- grants/)). IIRC one of his suggestions was so fund people (for longer terms), not 'projects'. This may not always be possible (e.g., if you need expensive equipment), but in many fields it would make much more sense.

EDIT: language

</comment><comment><USER52> I would suggest to teach and emphasize the role of science and technology for our future. Social esteem for the role of the scientist and technologist is important to nurture scientific talent. Reading Paul Drucker book about how America respect for the technological man and British respect for science man explains why America took the lead in technology. For young people motivation to solve some very important problem by means of science and technology and giving them freedom, good money and respect is the right combination to make the any field advance.

</comment><comment><USER52> Since here many things are about start-ups, what about a start-up for developing a better academia, a way for great people to expand the frontiers of knowledge?

Tim harford Ted talk about error and evolution is about how the only way to manage complex system is about trial and error, perhaps he would subscribe that trial and error is the only way to search for a better way of managing academia.

[http://www.ted.com/talks/tim_harford/transcript](http://www.ted.com/talks/tim_harford/transcript)

</comment><comment><USER52> There a a lot of comments here and the take-away for me is that managing academia is a very difficult task and nobody knows how to do it. Some people recognize that we could make more harm than good by adopting short-sight metrics, I consider fortunate that people recognize that sometimes we don't know all the answers and that the best we can do is to share information about this complex task.

</comment><comment><USER53> _Management and other myths_

 _1\. Complex systems tend to produce complex responses (not solutions) to problems._

 _2\. Great advances are not produced by systems designed to produce great advances._

[https://en.wikipedia.org/wiki/Systemantics](https://en.wikipedia.org/wiki/Systemantics)

<comment><USER54> Also, thank you so _very_ much for _Systemantics_. I've run across it before, but very much needed to run across it here and now.

</comment></comment><comment><AUTHOR> Please look at the comments section of that post too.

</comment><comment><USER52> I think than more than never working in a team is important. But a team must be prepared to accept A people what is not a small feat.

</comment><comment><USER55> I'll just note that this entry was posted 8 times by different accounts. Someone wants you to adopt an opinion.

</comment><comment><USER52> Could we use deep learning to select who is going to do a good job in academia?

<comment><USER54> DL is a great way to reimplement extant bias whilst obscuring the reimplementation.

</comment></comment><comment><USER56> >as soon as you try to measure how well people are doing, they will switch to optimising for whatever you're measuring, rather than putting their best efforts into actually doing good work

Similar to [https://en.wikipedia.org/wiki/Hawthorne_effect](https://en.wikipedia.org/wiki/Hawthorne_effect) ?

<comment><USER57> The Hawthorne effect describes how any change in environment can temporarily lead to positive effects. The sentence you quote is more akin to Goodhart's law ([https://en.wikipedia.org/wiki/Goodhart%27s_law](https://en.wikipedia.org/wiki/Goodhart%27s_law)).

</comment></comment><comment><USER58> You cant put a metric on the future.. In the literal sense..you cant measure whatyou yet dont know..or how valueable it will be for humanity.

</comment><comment><USER59> It would be great to see Nassim Nicholas Taleb's response to this. He's been working on a draft called _Skin In The Game_.

Overall, I think that the GP is right but seeing that requires us to consider that nebulous ideas like character and cohesive culture need to be at the forefront more than market reasoning.

</comment><comment><USER60> All employers want their employees to "do the job to the best of their ability". But the freedom given to you to decide how your success is measured is proportional to your experience and past performance. There are those in both academia and private industry that get a lot of freedom to do their work/research.

But there is a hierarchy to this just like everything in nature. The exceptionally strong and bright animals have more freedom in why they choose to do and how they do it. And that system has worked to get us all this far.

The system this article is pushing against isn't fun for many but inevitable. The further you are from the top the more subject your performance is to poor evaluation schemes conceived by those far from the top too.

<comment><USER36> Scientists aren't employees. They are given the tools, real estate and funding to do their thing, usually through research grants or private funding.

The fact that nowadays they are treated as employees is exactly the problem.

...Oh, and by the way, nature has no hierarchy.

<comment><USER60> Scientists are employees in many contexts (almost all contexts). We may just have to agree to disagree on that one. Be they in service for an NGO, school, or oligarch.

But to say nature has no hierarchy? I suppose "hierarchy" can mean many things. But in my case I mean fitness and selection for it. The most fit are at the top. Nature tends to select for those better suited to their environment. And those best suited thrive and push the success of us all forward. It is how you got here and have what you have. Intellectual, emotional, and physical traits are all subject to this selection. The thriving academics with the most freedom are closer to the top. The best employees (however you want to define employee) tend to be given the most freedom and trust.

<comment><USER36> If you look at any nature's habitat of N kinds of animals, even there what you see can't be called a hierarchy. It's a semi-stable state on the verge of chaos which can be very easily broken by introducing a species of animals new to the area.

So no, it's not hierarchy. Lions can kill almost anything if they feel like it, but hyenas kill lions on a regular basis. Does that make them the top of the food chain in the savannah? Definitely not. It only makes a pack of hyenas stronger than a single lion or lioness.

Rock, paper, scissors. No hierarchy. It's the same in the human systems. Hierarchy is an artificial system enforced by humans and has almost zero relation to anything natural.

<comment><USER60> Not all lions kill as well as other lions though. Take a snapshot of all the lions in the world alive right now and rank them by their ability to kill and the number of offspring they have. Some are better at these two things (they killed more and attracted more mates). This is quantifiable and trivial to fit into a hierarchical scheme of some kind.

It isn't hard to extract similar hierarchies from other species...including humans.

<comment><USER36> If we're talking about a single species, sure.

</comment></comment></comment></comment></comment></comment><comment><USER61> This is a bunch of cool sounding scientific nihilism, but it's false. Nclb was a great attempt to manage academia and it worked.

[http://www.nber.org/papers/w15531](http://www.nber.org/papers/w15531) [http://www.nber.org/papers/w20511](http://www.nber.org/papers/w20511) [http://www.nber.org/papers/w16745](http://www.nber.org/papers/w16745)

<comment><USER62> The authors themselves admit that NCLB was a mixed bag: "Specifically, we find that NCLB generated large and broad gains in the math achievement of 4th and (to a somewhat lesser extent) 8th graders. However, our results suggest that NCLB had no impact on reading achievement for 4th or 8th graders."

You'd think that you can employ drill practice to improve results in mathematics, but that approach won't work in English. American math instruction feels quite alien for foreigners, the emphasis is on the application of algorithms, not on understanding why and how they work. Equipped with plenty drill practice, an Amarican child can fill in the correct bubble on the test, no matter if or not it understands mathematics. Any understanding of mathematics gained in that system is purely incidental.

That's all a hypothesis and would require further testing.

<comment><USER61> That's net positive - an increase in some subjects and neutral in others.</comment></comment></comment>


Ask HN: What does one need to do to get a software job out of University - <AUTHOR> It looks as if no one will hire a dev out of university except big companies. What do you do when you've tried all your social connections, contacted your previous internships, applied to all local jobs, applied to several remote jobs and jobs in other cities.
Options: hire someone to go over my resume and cover letter to help improve it, hire a company to match me with an employer, increase my applications to jobs in a city with more jobs, obtain online certificates to bolster my resume.
I'm intentionally leaving this question broad as to help future readers but I will include some vague description of my current situation.
- I have internships at 3 decent companies and 1 pseudo internship at a very notable company.
- I've been applying for 3.5 months. ~50 CL, ~50 without CL. Anythin from startups to big companies intentionally targetting jobs requiring ~1 year experience.
- I've had my resume reviewed by a couple professional friends.
- I've had 3 interviews and ~7 coding challenges. All interviews were at great companies. <comment><USER2> Job search doctors will tell you that hiring is an elimination process; if you are getting interviews then your resume is "good enough" and it isn't the problem, it is your ability to pass the interview.

Most people make mistakes in the interviewing process that cause them to interview below their level. Don't be that guy.

One class of mistakes is actions that offend some people: the range of these are large and if you want the job you won't be leaving posts on social media complaining that other people are too sensitive. For instance I interviewed for a job at a company that was making small satellites in Southern NH and got turned down because I touched the knob of an oscilloscope without permission and the guy who ran that particular lab thought that meant I couldn't be trusted and I was passed over, even though everybody else thought I was great.

When you become aware of this you can certainly stress it because any person you see could blackball you but you have to eliminate any sign of negativity, hostility, etc. If you radiate "I've been hurt" it is all over.

One way or another you have to face it and you will.

You can certainly try a temp agency like Kelly if the wolf is at your door, but you can get paid better anyplace else.

Don't think about remote if you don't have experience. If you want to move somewhere I say anywhere but the bay area. New York City is great in 2018, but there are tech scenes in cities like Boulder, CO; Henderson, NV; Boston , MA as well as general areas such as Southern California, New Jersey, Delaware, Maryland, Salt Lake City, and Research Triangle Park.

If you can get a security clearance (clean rap sheet) you will find work that is often interesting and innovative for defense contractors who are geographically distributed and might even move you around from one site to another in the flyover states.

I'd suggest figuring out a way to live light and move to a low-cost tech hub and get into that scene and be ready to move again in 6 mo to 1 yr.

I like the SkillIQ tests at Pluralsite for a quick assessment of "how well you know a technology". I have taken many of them and they reflect my own assessment of things I am good at and not good at.

<comment><AUTHOR> You make a solid point about what you radiate in an interview. I can tend be passive which might come across as aloof or uninterested. I could probably be more active.

I would love to live in Boston but I'm Canadian. Do you think I would have a chance of getting a visa and job in Boston? I've definity heard that if you want to progress in the software business you need to move companies often, unfortunately.

I'm intrigued by the tests you mentioned. Would definitely be helpful for finding weak points.

<comment><USER2> Many firms that have software operations in the US have operations in Canada as well, usually in the same time zone.

For instance Pac Northwest firms such as Amazon and Microsoft hire in B.C., Calgary has a scene in systems programming, embedded systems that I know of and probably lots that I don't. Toronto has everything from Geoff Hinton to internet porn. Montreal has game super-studio Ubisoft as well as ASP.NET and mainframe programmers, great nightlife, a goth loli clothing store, ...

So far as personality I would be worried more about a person who was too reckless than I would somebody careful.

If you want to appear interested in people it helps to ask questions. Sitting on either side of the interview table I would see it as an opportunity to get information I couldn't get any other way about the state of the industry so I am always interested. People mostly like talking about themselves and if they are talking you don't need to think of something to say or open your mouth and risk scoring an own goal.

To get in the mood you can pretend that you already have the job and you are coiled like a spring to start and you are going to ask all the questions you need to start such as "what kind of computer do i get?", "what is the schedule?", "where will I be working?", as well as getting an understanding of the job enough to get ready to do it.

If you find that hard to do consider acting lessons at a Meisner school and they will teach you to be coiled like a spring in the second term!

</comment></comment></comment><comment><USER3> I graduated a couple of years ago. I sent over 200 application (Linkedin, custom CV/cover or generic CV and email ...) , all I can tell you is not to give up, there is a lot of luck in the process but eventually you'll find your breakthrough.

</comment><comment><USER4> I heard the other day about a partner at a major law firm who despite his first class degree applied to 500 positions before he got his first job.

As rubbish as it is, send more letters!

<comment><AUTHOR> I was wandering that myself. My numbers seem low. I wrote version control for my CLs to improve the speed but I need the discipline to sit and create them.

<comment><USER4> You do have to play the numbers game. If you had sent me a dev CV at most points in the last 5 years it would go in the bin, but suddenly one week I need a dev...and you would get a shot.

Where in the world are you btw?

<comment><AUTHOR> Eastern Canada. I'm willing to relocate to anywhere west of Toronto or go all the way to BC.

</comment></comment></comment></comment><comment><USER5> This question is better asked on /r/cscareerquestions

</comment><comment><USER6> Did you ask for feedback from the internships?

Usually intern conversion is the best way to hire new grads

<comment><AUTHOR> All the companies gave a general overview of my ability at the end of the intership. The first company I worked at was a start up and they folded up. The second company was pretty great, code/tech was ok, told me they would love to hire me and the team lead said that I was probably a better dev then some of the current members on the team but they didn't reply when I messaged them. The 3rd company I refuse to go back and work for: would never progress my career, underpaid devs, Nightmare code, etc.

is their any other options beside previous internships?

<comment><USER6> I assume you've applied to all the big companies. Try applying again 6 months after the rejection.

Have you tried AngelList? It's where startups post their jobs.</comment></comment></comment>


Designer to make logo? - <AUTHOR>

can anyone recommend a good designer to make a logo?  I'm thinking a cartoon animal... also, any ideas about what to expect to pay? <comment><USER2> Friends have had good luck with <http://99designs.com>. You pick the investment you want to make up front, and pick the best design. You also get to give feedback during the design process.

<comment><AUTHOR> this site looks really sweet. thanks for the tip.

</comment></comment><comment><USER3> <http://99designs.com/> is a good suggestion. I also recommend <http://plasmadesign.co.uk/> \-- they really understand design, are amazing to work, and produce beautiful work.

</comment><comment><USER4> From experience $250-$700 should get you a nice logo with 2-3 sets of changes.</comment>


How to write a search engine - <AUTHOR> http://www.acmqueue.com/modules.php?name=Content&pa=showpage&pid=143 <comment><USER2> Unfortunately, the author largely glosses over what I consider the most difficult part of search--extracting semantic information from unstructured source materials (also refered to as "magicks"). However, the discussion of bandwidth and server allocation - of starting with what you can afford, and spending your effort/money on what matters (i.e. your algorithms) resonates with a lot of the ideas behind YC. When it comes right down to it, anyone who can sweet-talk a VC can get fat pipes and big boxes--the (human) brains behind the (algorithmic) brains are what differentiate Google from Lycos.

<comment><USER3> '... the author largely glosses over what I consider the most difficult part of search--extracting semantic information from unstructured source materials ...'

This is a hard set of problems. Now the question is how do you think they go about it? My guess is they work from whole to part on a document they find determined by extension/mime type. Then analyse the document in detail for commonly found information (based on statistical info).

I remember reading about how google analysed web authoring statistics ~ http://code.google.com/webstats For instance 'a' links within html pages which reveal links to other documents can be parsed to extract lots of useful information as found here ~ http://code.google.com/webstats/2005-12/element-a.html

But unstructured data is a different beast. How for instance does google work out that 'cm' (what I mean by centimeter) is both 'Columbus McKinnon' and a unit conversion as highlighted in a returned search? [0]

One way could be using techniques similiar to the 'Normalised Google Distance' algorythm or NGD. Developed by 'Rudi Cilibrasi' and 'Paul Vitany' (National Institute for Maths & Computer Science, Amsterdam). They build a database model of how close words relate to each other & use this to compute word combinations. The closer the word combinations, the closer the association. So my example of 'cm' would have close association with 'Columbus McKinnon' and 'Centimeter' etc as returned by the google search ~ http://www.google.com/search?q=cm

You can read more about this 'Google's search for meaning' [1] and the abstract , 'Automatic Meaning Discovery Using Google' [2].

Reference

[0] Slashdot, ' Deriving Semantic Meaning From Google Results' [Accessed Wednesday, 7 March, 2007]

http://science.slashdot.org/article.pl?sid=05/01/29/1815242

[1] New Scientist, 'Google's search for meaning' [Accessed Wednesday, 7 March, 2007]

http://www.newscientist.com/article.ns?id=dn6924

[2] arxiv.org,Computer Science, abstract 'Automatic Meaning Discovery Using Google' [Accessed Wednesday, 7 March, 2007]

http://www.arxiv.org/abs/%20cs.CL/0412098

</comment></comment><comment><AUTHOR> Some of this is a little dated - for example there's not much point in managing a crawler yourself when you can just pay alexa to do it for you.

I still found a few useful tips though.

<comment><USER4> The article is meant to be taken as a general guide. The author does suggest that we use a pre-programmed crawler rather than code one ourselves. Refer to this passage:

"Crawler. If you don't use an open source crawler, my advice is a super-simple multistep crawler. This is very important advice that will cut months off your development time, so if you ignore everything else, don't ignore this."

</comment></comment><comment><USER5> Thanks to Doug Cutting you don't have to write a search engine. You just use Nutch and Lucence and Hadoop.</comment>