The independent researcher - <AUTHOR> https://nadiaeghbal.com/independent-research <comment><USER3> I've done an enormous amount of independent (and fairly successful) computer science research over the last two decades. Computer science is amenable to it if you are willing to make a large investment of time. The challenge is that making this research useful to _other_ people requires an even bigger investment of time, which creates a somewhat adverse incentive.

At least for me, there is one giant gap I've ever never been able to cross. I am willing to spend an inordinate amount of time solving interesting theory problems and doing novel algorithm design work _because it is useful to some cutting edge software problem I am working on_. However, I have almost zero incentive to spend the significant amount of time to publish any of my research, whereas academics do. My time is enormously oversubscribed as it is. My benefit is that I can reduce it to practice and build software that does things no one else can do (and I enjoy the research generally). I know a few other people in the same boat; great researchers, no time or incentive to publish.

I've never been able to reconcile this. There is a similar analogue in open sourcing software. Open sourcing your software often _increases_ the amount of time you need to invest in it in practice, which disincentives people with limited time to open source code (and after they already invested their extra time creating the code in the first place) that they otherwise would be amenable to open sourcing.

<comment><USER4> I think an important thing to take into account is that a very important part of what makes a good scientist is not just how good they are at "doing science", but how good they are at _communicating it_. A good scientist is a good communicator. Even if that means finding people who you can work with who are good communicators, it's part of the job. Coding up a really clever solution and just kind of hoping people will come across it and appreciate is not really doing science. (Not saying that's what you are doing, I'm just going on a tangent here.) The point is, ever since the enlightenment, a strictly critical part of the job of being a scientist has been the ability to elaborate your ideas to others, and to argue for its merit.

So yeah, doing the work is the fun and motivating part, but a big part of the reason that the academic community values well-written scientific papers is because all that work can go to waste if it is not described (and defended!) in a way that is clear and convincing (to the relevant audience of course).

Grad students may complain about having to learn to write effective papers, but it's literally just as important as the technical work they are doing --- maybe moreso, since in the future they may very likely change topics, and the writing and communication skills is the part that will carry forward throughout their career.

<comment><USER5> I'm reminded of the saying that "Publishing research without data is simply advertising, not science". Not even publishing the research means it doesn't have a chance at advancing the state of the art. I don't understand how such research can be successful at anything other than selling a few extra copies of a software product likely to disappear within a handful of years. Unless it's implemented as the absolute killer feature of a very successful product it's very unlikely anyone will ever know there's anything novel in there.

<comment><USER3> The research does get disseminated, just slowly. The transmission vector is contact with other people with firsthand knowledge of the research. It becomes an oral tradition, which makes it difficult to break into and is inefficient. I've come across many non-publishing research communities in various parts of computer science that continue to develop and share the state-of-the-art amongst themselves. The research isn't being lost, it is just compartmentalized.

For example, there is an elegant cache replacement algorithm I learned from a database kernel engineer a decade ago, one of those algorithms that is so obvious in hindsight that you can't believe it never occurred to you. It solves a key problem with the algorithms everyone knows in literature. I've used it ever since and taught a bunch of _other_ database kernel engineers the same algorithm. I have no idea where the person I learned it from got it, but the literature still treats it like an unsolved problem. Many engineers are using it in more and more systems, and there are a few different variants in the wild, but no one has bothered to write it down AFAICT.

The salient point is that academia is not the only community that develops and defines the state-of-the-art in computer science, but at least they publish. Without an incentive for non-publishing research communities to publish, it becomes difficult for an outside academic to break into some fields because there isn't an obvious entry point.

<comment> d-- Is there a chance you can point us to a code snippet of said cache algorithm, or elaborate a bit?

If it's simple and elegant in hindsight this insight might be interesting in other CS applications. Not cross sharing and reinventing wheels is a somewhat sad implication from the compartmentalization you mention.

</comment></comment></comment></comment></comment><comment><USER6> The true problem is time, at least in fields like CS where expensive instruments aren't really needed to do research (not as much as other fields).

You need the freedom of time and being able to fully dedicate yourself towards a specific study if you want to make any real contribution.

And I'd say that academia is struggling at doing science nowadays also because you're stuck doing all kinds of other things with your time rather than actual research.

It's hard to tell what the right formula is for being a productive scientist and surely everybody's free at doing it.

I have a feeling that something like Universal Basic Income would give this freedom of time to crowds that can lead towards more people doing science and hence more discoveries.

<comment><USER7> > I have a feeling that something like Universal Basic Income would give this > freedom of time to crowds that can lead towards more people doing science > and hence more discoveries.

Agreed. When I was in graduate school I remember thinking that if we had UBI, everyone would become a scientist. I'm sure this was biased by my own interests, but the work often was truly thrilling. Sadly, it is very difficult to find opportunities in academia or industry to pursue work that one finds meaningful without major strings attached.

Most of my graduate school colleagues have left research. I'm a research scientist in industry, but it is increasingly obvious to me that this is relatively rare, particularly given the intellectual freedom and institutional support I am provided. And certainly much of my time is focused on work that has benefit to the corporation, versus open ended projects.

<comment><USER8> _if we had UBI, everyone would become a scientist_

Very few people want to be scientists. To see why, just replace "scientist" with "nerd", because for most people those are synonyms.

Moreover, anyone who wants to do science can become a grad student. That's at least 5 years of "basic income", and access to mentors/tools/resources. All you have to do is to find a lab that does what you're interested in.

<comment><USER9> I never heard of a grad student program that pays you to do _your_ research project. The ones I've seen, always offer a choice between paying you to work on someone else's research project, or you get to work on your project but you don't get paid; you do get access to facilities, but you have to pay an exorbitant fee for this. Am I missing something?

<comment><USER8> That's why I said you have to find a lab that does what you're interested in. I have done just that, and for the last 5 years as a phd student I've been paid mostly to do what I would do as a hobby otherwise (read/implement deep learning papers), and try to advance state of the art.

<comment><USER10> from my own experience, this is the exception, though. even in CS, if you pick some of the more theoretical problems or a not-so-hot area, you might not get funding - (almost) no matter how good you are. and that's CS. move to other subjects like sociology, linguistics, history, and you will find that barely anyone gets funded.

<comment><USER8> I haven't heard about anyone in any PhD program in US who's not funded one way or another. Sure, you might need to TA a class, but it's usually not that hard, and leaves plenty of time to do research.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I have mixed feelings about this.

It is true that modern funding of scientific research is flawed. Certainly in the US and probably everywhere. Almost any working scientist has plenty of stories about the silliness of bureaucratic requirements for governments, universities, or nonprofits. Almost any working scientist can name some people in their field who they think are severely overrated or even unethical (this seems to be more of an issue in lab sciences, somehow). Almost any working scientist has peer-review horror stories and admits that the process is noisy at best.

Still: there are real benefits to being a non-independent researcher. At this point it's hard to get to the forefront of research — in pretty much any field — without extensive contact with the people who are there already, most of whom are not independent researchers. Failure to do so doesn't just mean you want produce hip research that caters to the specific elite tastes of the establishment. It also drastically increases the likelihood that you're wrong. Good research is hard; it's even harder in isolation. Ther are hundreds and hundreds of papers about your favorite math/physics/computer science (it does seem easier to be an independent naturalist or something) conjectures floating around out there. Many of them sucked away years of their authors' lives, and somewhere around 0% of then are correct.

I suppose my two cents for anybody seriously interested in independent research is to make it as easy as possible for credentialed researchers to interact with you. Learn what you can, read accepted literature, acquire some credentials yourself if possible, learn the language on Stackexchange or Mathoverflow. Definitely do not open your interactions with grandiose claims. Most researchers actually like their subjects; if you have something useful to tell them, they want to know. You just gotta make it as easy as possible for them to see you're not a crank.

<comment><USER12> _I suppose my two cents for anybody seriously interested in independent research is to make it as easy as possible for credentialed researchers to interact with you. Learn what you can, read accepted literature, acquire some credentials yourself if possible, learn the language on Stackexchange or Mathoverflow. Definitely do not open your interactions with grandiose claims. Most researchers actually like their subjects; if you have something useful to tell them, they want to know. You just gotta make it as easy as possible for them to see you're not a crank._

That jibes with my experience for sure. The sample size is small, but I've found "traditional" academics to be quite receptive to engaging with me when approached the right way.

For example, I cold-emailed one of the two authors of a book I was reading, and asked for some datasets they had used in their research. The professor in question quickly replied with the data, a pre-print of a forthcoming paper, and an invite to keep him informed on how things were going.

I later emailed him with a technical question about something from the book, and he replied that he didn't know the answer offhand, as the research in question was done 20 some odd years ago, so he cc'd the other co-author and solicited his response. It took a few weeks (turns out he was on vacation) but he also responded, helped address my question, and asked some questions about what I was working on.

One of the two professors in question said during some of our email exchanges that they believe there's a real chance to come up with something useful going down the path I'm going down, which is encouraging.

Anyway, I suspect part of the key to engaging with academics is:

1\. (as you said) don't open with some grandiose claim.

Rather, ask a question, preferably a "good" question (that is, something that shows you have done your homework, something that is specific enough _to_ answer, etc.). Not "Can you tell me how to get started in X" where you could have just googled those exact same words and gotten an answer.

2\. Do your homework and make it clear that you've your homework. If you start off with "I was reading your paper on X, and on page 3, in the equation for flibblegazz, I don't quite understand the notation you use..." or something, it shows that you're not just pulling crap out of the air. And if you're emailing the person who wrote the paper, you're appealing to their ego a bit in that you're telling them that you care enough about them, and their paper, to dig in. I suspect that unless someone is a "celebrity researcher" to the point that they get more requests than they can handle, they'll probably be inclined to try and help.

3\. Do your homework in the sense of reading the basic texts in your area of interest, read papers in the area, learn the relevant math (mathoverflow and various sub-reddits are good places to ask math questions, BTW), etc.

4\. It's related to a previous point, but showing interest in the work someone else has done is usually a good icebreaker. Asking for a preprint of a paper, or asking for a dataset, saying you want to try and replicate their work, things like that.

Do those things, go slowly, and just plain "be nice", and I think you can build a relationship over time, with the folks in "proper" academia. And that could be mutually beneficial. For example, you may eventually want to publish a paper in a journal, and your academic friends can probably help you there. You may even wind up collaborating to a level that justifies a co-authorship scenario, who knows?

<comment><USER13> Academic here:

Totally agreed with every point. These are great tips even for someone within academia interested in collaborating with someone else. It is a combination of shared interest, impedance-matching, and mutual respect.

</comment></comment></comment><comment><USER14> I am pursuing a self-sponsored PhD at one of India's leading institute of science and technology. I have about 15 years of work experience, including building a product which has seen a good amount of commercial success.

Validation of idea's was one of the reasons I decided to enrol in the part- time doctoral program. Also, vetting of technologies I planned to use in my work was another reason to become a part of a central govt. funded institute. It is easier to collaborate with faculty members when you are part of academia than otherwise.

Having said that, I am now of the opinion that independent research is the most optimum way to pursue one's own ideas. If you are working in the space of citizen impact or developmental projects, engaging with journalism editors has a far more positive impact than engaging with academic faculty. The science editors are able to articulate and help you present your work better than most other people.

Another group of people, I feel, might be helpful are "true incubating mentors". If you are looking at taking your work to the next level, I feel an incubator/mentor will provide you with the right direction (I've myself not experienced this but have seen people benefit from this).

</comment><comment><USER15> This is a great article but if you need a taste:

 _" My first go at independent research wasn't necessarily easy. I didn't know independent research was a viable path, and especially not in Silicon Valley, land of builders and shippers. I didn't know whether I was doing something useful with my time, or whether I was just a crazy person.

A few years later, I've realized that the answer to that question doesn't really matter. Life is short. Do whatever you can't stop thinking about. Documenting your findings in public (regardless of outcomes!) is a worthy contribution to society, full stop. If you're doing something new, and you care about understanding the problem, people will pay attention. What's more, they'll take your ideas and make them better than you'd ever imagined. And that style of research - living in service to the public - starts to look very different from the bloated, ivory tower models we've been accustomed to."_

<comment><USER16> > whether I was doing something useful with my time, or whether I was just a > crazy person.

This is grad school in a nutshell, but in the classical route, you have a lot of people to talk with who can tell you which bucket you're in.

</comment></comment><comment><USER7> > This is the dark side of independent research. Without external validation - > "I teach at Stanford" or "I got a grant from NASA", it's hard to convince > people that you're any good at what you do.

This is one of the reasons double blind peer review can be appealing. It avoids credentialism, where the institutional reputation overrides honest evaluations of the paper quality. Even in my own reviews, it's hard not to be biased when I know the author is a luminary of the field.

<comment><USER17> > This is one of the reasons double blind peer review can be appealing. It > avoids credentialism, where the institutional reputation overrides honest > evaluations of the paper quality.

This is one of the things I really like about the top machine learning conferences at the moment. The reviews are open to all and blind to credentials. Combine that with the fact that even vast compute resources can be obtained fairly cheaply (at least for a short amount of time), and CS research is one of the more "democratic" fields that someone can be a part of nowadays.

This is in contrast to the research I performed during grad school, which required access to a billion dollar neutron source, and reviewers generally knew which institutions the papers were coming from...

<comment><USER18> Eh, more often than not, it's pretty clear who at least one of the authors of the paper is by just looking at what they cite. Double blind review is not a foolproof system of avoiding creds and appeals to authority.

<comment><USER19> From an article [0] published in last month's CACM, describing attempts by three 2016 conferences to anonymize papers:

 _We find that anonymization is imperfect but fairly effective: 70%–86% of the reviews were submitted with no author guesses, and 74%–90% of reviews were submitted with no correct guesses._

[0] [https://cacm.acm.org/magazines/2018/6/228027-effectiveness-o...](https://cacm.acm.org/magazines/2018/6/228027-effectiveness- of-anonymization-in-double-blind-review/fulltext)

</comment></comment></comment></comment><comment><USER20> I just figured out being an independent researcher is the best for me over time. It gives me the freedom to look at a problem space without noise. I consider even a partner or a group to be a noise. Also, it is true as said in this article that you do not need others approval if you can fund yourself and find a interesting problem to work on. I do like to go beyond just research for the sake of it and engineer / productise the ideas.

</comment><comment><USER21> I became an "independent researcher" after I left my physics research job and moved to the country. With my trusty iMac and Internet access, I write and run computer codes using Xcode, do graphics and some analysis with Mathematica, read and get journal articles remotely (electronically) from a friendly university library, and write up the research using Lyx. I post my papers on arXiv (at no cost); there they are freely available, searchable, citable, and forever archived.

<comment><USER22> Can I ask for your Arxiv? I am near the end of my physics PhD and also near the end of my patience with the bureaucracy of academia. Just wondering what sort of research is possible independently

<comment><USER23> [https://arxiv.org/search/?searchtype=author&query=Van+Siclen](https://arxiv.org/search/?searchtype=author&query=Van+Siclen)

</comment></comment></comment><comment><USER24> I'm pursuing this route right now. Sure it has some downsides but like it's insane how productive you can get when you actually devote 12 hrs a day to your research. A PhD student will have to go to meeting, teach classes, spend time commuting etc. I get 12 hrs a day of pure research.

That being said I'm far from wealthy.

</comment><comment><USER25> Has anyone here published in a journal independently of a large company or a university/research institution? I cannot think of any papers I have read that are attributed as such. I have been doing a bunch of independent research in some areas in signal processing that I have only recently started thinking about writing up, and I was wondering how realistic that is. It has been a long time since I published anything in a journal.

Also since I started working on this, I realize just how nice it was in university to be able to click a link to an IEEE paper and skim it. I feel lucky to work so close to a large university library, but I see why people are so drawn to SciHub.

<comment><USER26> Here's a good example of influential paper published outside of academia by a company which was really small at the time: [http://www.onboard.jetbrains.com/is1/articles/04/10/lop/mps....](http://www.onboard.jetbrains.com/is1/articles/04/10/lop/mps.pdf)

It has a plenty of citations: [https://scholar.google.com/scholar?cites=1113313642009476352...](https://scholar.google.com/scholar?cites=11133136420094763520&as_sdt=40000005&sciodt=0,22&hl=en)

It all resulted in an innovative product: [https://www.jetbrains.com/mps/](https://www.jetbrains.com/mps/)

<comment><USER27> Doesn't hurt that he has a graduate degree in Mathematics, so he's already credentialed for research.

</comment></comment></comment><comment><USER28> I often recommend to young people just out of college that if they want to do serious intellectual work, their path ought to be:

1\. Get a finance job on Wall Street.

2\. Spend a few years saving up a decent nest egg of a couple million dollars.

3\. Retire to the country side and start doing your intellectual work.

<comment><USER29> This is not as easy as it sounds. If it was, everyone would do it. A finance job on wall street does not guarantee you will become a millionaire in a few years. I worked for a few years in the finance department of a Wall St company and made decent money, even above average for someone out of college with a bachelor's (still not great for NYC), but if you want to make the millions you have to get with a very close-knit crowd of the dealers/brokers/market makers who are not open to letting in new people easily. And given the scandals and fraud which we read about in the news regularly, I can deduce at least one reason why. Just my n=1 but I knew a significant number of people who worked with me, even ones who were closer to the higher ups who still ended up with a nice but not extraordinary salary

</comment></comment><comment><USER27> The discipline of research seems - historically - to be equal parts social maneuvering and scientific rigor.

Just look at the very public spat between Ian Goodfellow and Jurgen Schmidhuber about Generative Networks to get a flavor for that.

Chomsky has perpetually raised the flag when his research won't get published because he's a vocal critic of some institutional priority.

Even back to Einstein, Kant etc... in multiple fields the powers at be thought these guys were cranks until they blew everything open. If you can't get a consortium of your peers in research to even take the time to read your work, then you won't be able to make any changes.

As with all things, Institutional association (as long as it's the right institution) serve as the social proof that you're doing real important work.

It's unfortunate, and I wish it weren't the case, but it seems to be intractable.

</comment><comment><USER23> Here are a couple of well-known independent physics researchers:

Julian Barbour: [https://en.wikipedia.org/wiki/Julian_Barbour](https://en.wikipedia.org/wiki/Julian_Barbour)

Garrett Lisi: [https://en.wikipedia.org/wiki/Antony_Garrett_Lisi](https://en.wikipedia.org/wiki/Antony_Garrett_Lisi)

Lisi runs an informal hostel for physics collaboration at his home in Hawaii:

[http://www.pacificscienceinstitute.org/](http://www.pacificscienceinstitute.org/)

</comment><comment><USER30> Perhaps part of the issue with people keeping silent about their private research is the world today is full of very vocal quacks and cranks. The last thing you want is to be taken for a crackpot.

<comment><USER23> ... the things to avoid:

[http://math.ucr.edu/home/baez/crackpot.html](http://math.ucr.edu/home/baez/crackpot.html)

<comment><USER30> Oldie but goodie.

Jokes aside, lacking significant result, there's no big incentive to publish filler and "Towards.." type papers if you do something as a hobby. If one does feel so inclined, it's definitely inching towards crackpot territory. Very likely countless amateur projects quietly start and die every year without making a peep.

</comment></comment></comment><comment><USER22> I don't see much discussion on how these people are able to collaborate on projects with others? For a lot of research, you need partners to check each other's work, to bounce ideas off, to stop you from making stupid mistakes or help when you get stuck. This is why in academia people work within research groups.

Is is possible to create research groups while working as an independent researcher?

<comment><USER31> I think independent researchers - at least in the CS context - end up organically collaborating online through the content they share in github, blogposts, etc. Maybe there is some space for a more structured community of independent researchers which help each other and collaborate, but then again, isn't structure that indie researchers are trying to avoid?

</comment></comment><comment><USER32> I want to do independent (computational) research. That's pretty much the only thing I want to do. Not employment in industry, not teaching, not funding agency research (where the agency wants you to do only a restricted form of research; okay maybe I could do that but then funding is hard to come by). But independent research. And I don't care about money (as long as I can live on it, and have access to computational resources).

But I don't know how I can make time for it. I worked for a while, saved up some money, then did my own research using that money for a few months. But working to raise money is very distracting.

<comment><USER33> Part time job? Work 4 hours a day or 3-4 day a week. Programmers salaries are such that it is possible. The key is to watch time at work religiously and ensure you actually work agreed upon time and not more. It works, but you have to be consistent.

I have seen people doing this. Not with research, but with hobbies, teaching, religion or family. If it is possible with these, then it will work with research too.

<comment><USER32> > work religiously ...

that's my problem with part time work. If I have to spend even two or three hours on something not related to the research I really want to do, the rest of the day is ruined.

I'm the kind of person, who does well by dedicating solid chunks of time day in and day out on one topic, with the only distractions being eating, sleeping, and going to the restroom.

However if that part time work involves me showing up and just being there, or following mindless instructions (like burger flipping, or moving containers from one place to another) then as long as I'm not physically tired, and I can go back and do research for the rest of that day. But I doubt part time menial work would raise enough money to pay the bills.

<comment><USER33> Then work 3-4 days and dedicate 1-2 to research and you still have weekend.

I did not meant burger flipping. Burger flipping does not pay enough. I meant programming or other well paid job where you have good negotiating position.

By "work religiously" was padt of semtemce that meant to watch time. Don't stay longer "just this one time".

</comment></comment></comment></comment><comment><USER34> The article doesn't mention one very important expense for nearly every research field: getting access to the research other people have published. Some fields (e.g. computational linguistics) have long had a policy of letting authors keep copyright, so they can post their own papers online, and the collections are also free; but the norm is that nearly all publication goes through publishers that paywall. So just to see what everybody else has been doing, you need to have institutional support.

Only slightly less important is access to conferences, which are not always super cheap: travel + lodging + registration is often in the $1000 range even for the relatively accessible conferences.

<comment><USER35> Fortunately: [https://sci-hub.tw/](https://sci-hub.tw/)

Yes, technically it's copyright violation, but if you ask any of the actual authors, or any human being with any _legitimate_ interest in the papers therein, they will encourage you to go for it.

<comment><USER36> Also: [https://arxiv.org/](https://arxiv.org/)

</comment></comment></comment><comment><USER37> I do Computing Education research, so being able to access learners is actually a necessary part. I guess it'd be possible to do this externally, but it's so hard to roll out new ed tech without quick access to an audience. But more importantly, I feel like I'd be very misguided if I didn't interact with students and understand actual classroom needs.

</comment><comment><USER38> I left academia (in the humanities) to pursue work in tech. Most of this rings true. My experience is that a.) a lot of exciting work is invisible to academia and b.) to wing it without an institution is certainly a question of money but primarily of time and focus (disciplines and research projects will obviously vary on this front).

</comment><comment><USER39> What would a platform for independent researchers (or potential collaborators!) look like?

<comment><USER40> A simple forum running something like phpBB would probably be the minimum viable product. Make many different categories and see what people want to discuss and what other features they want.

I'm a PhD student who's interested in going independent eventually, and I've discussed this with several people. My impression is that most people would be interested in discussing funding and historical examples of independent researchers (particularly how they funded themselves). Discussion of how to improve research and/or academia also seems popular. So perhaps the forum part could focus on topics like those initially.

Later, however, as the platform ages, more important new features of the platform would emerge.

<comment><USER39> A forum would definitely be core functionality. I've also been curious about the merits of anonymous publishing?

<comment><USER40> To me, publishing seems outside of the minimum viable product. I would like to see many changes to publishing, but I think those would be best handled later when independent research is better established.

Publishing is a chicken or egg type problem. At the moment few will use your publishing venue unless it has a certain reputation. But you can't develop a reputation unless your publishing venue is used often.

A publishing venue could be differentiated with unique features, e.g., open access, anonymous publishing option, more rigorous peer review, or more favorable copyright terms (I'm in favor of authors keeping copyright and just licensing the paper), for example. I'm not certain if these features would be enough to overcome the reputation disadvantage.

Perhaps independent researchers should forget about people who care so much about reputation (who probably wouldn't go independent anyway) and instead focus on making a better product.

<comment><USER39> I think this is actually a really great point about making a better product. One could venture to ask: is the reputation system within the scientific institution intimately linked the notion of natural truth? If so, does focusing on approaching science from the lens of product development signify a shift in the notion of what science is about?

I'm a PhD student in math currently and I often wonder what the difference really is between the creation and discovery of mathematical ideas. It seems the difference is really a matter of perspective: are we trying to discover something true or create something useful? Often these two perspectives intersect.

<comment><USER40> I don't think academics are incentived to publish truth in general. In my field (fluid dynamics), researchers seem to focus on producing as many papers as possible that sound good but may have problems beneath the surface. This may be different in math, because the standards are much higher.

The distinction between whether science or math creates or discovers truths is philosophical and not something I'm that interested in.

My goal is to publish papers as likely to be true or useful as possible. So thinking about this in terms of "product development" is really just about increasing the chance of discovering the truth. (Or developing a useful model, which may not be strictly true but is a good approximation of the truth. As an engineer, what I do is more likely to be in the latter category.)

</comment></comment></comment></comment></comment></comment><comment><USER41> Great article--couldn't agree more.

Also don't forget Michael Faraday, a bookbinder's apprentice who managed to figure out much of electromagnetism, among other things.

</comment><comment><USER42> Does Y Combinator fund (independent or not) research projects?

<comment><USER43> You mean [https://ycr.org/](https://ycr.org/) ?

<comment><USER42> Kind of -- if I am a researcher and I see a business application, but for which I would need to go independent but due to the uncertainty of viability continue my research and still be involved in my peer group in academics, is it possible to receive funding and return to my lab?

</comment></comment></comment><comment><USER44> The ancient Greeks had a society where they could afford to have people who did nothing other than open ended thinking. Sometimes I use some maths that was worked out then and I am appreciative of standing on the shoulders of such ancestral giants. A true measure of a society includes whether such a society affords people who are given the time and space just to think.

I find it amazing that in today's world there are employers willing to pay for likewise. It is quite a privilege to be in such a job where you are given some type of giant Lego set and there are no set goals as to what you come up with. In such jobs you really can be 'of independent means' and not subject to the rules that govern how a lot of people work, to value every moment and not be clock-watching until 5.30 p.m.

I follow quite a few people that work in tech and they don't seem to be restricted to boring rules but able to pursue their passion. There are no tax scams for companies to hire such people or other fiddles.

The Greeks may have had slaves and afforded their thinkers their own special way, today we have companies that can afford thinkers in their own special way. It is not just at Google where moonshots happen and neither is it just the Google-me-toos that do this (Facebook). Even small companies have people that are hired just to come up with stuff.

In previous times, before the 'modern' company, patronage was different and there were ways and means. Obviously class hierarchy helped but that was not all of it. Ultimately Darwin went off sailing with the Royal Navy and most of his work built on the work others had done regarding livestock breeding - <USER2> modern horse/cow/pig was 'invented' then. He may have inherited wealth but his work was very much about what mattered to society at the time.

How much really comes out of academia? Do 99% of thesis efforts end up collecting dust to never be read? For the author to get those letters after their name and expect the world to roll out the red carpet for them? This happens, further education is a business. There are plenty in further education who think about themselves and their status anxiety rather than society on some bigger/deeper level.

It has always been a two way street, no matter what the era is you also need people who can do the 1% inspiration/imagination and do the 99% perspiration. Luckily there are people, maybe few and far between, that can excel at that. Ultimately these people really are 'independent researchers' and they will do whatever it is whether they have a patron or an inheritance.

Then there are the wannabees that will insist on getting funding before they lift up a pen. Sometimes this is an excuse, how many investigations have not happened because there is no funding yet in reality all that is needed is for someone to put time into quietly studying the facts?

A comical (well not really) instance of this was after 9/11 when the people doing the 'investigation' said they were 'setup to fail' because they did not have the funding to do a full investigation. These people being those pork- barrel fed politicians who can't wipe their own arses unless they get funding for it.

Once I met a lovely lady on a train who worked for some think tank. Fascinated I asked about what it was really like working for this highly respected group. Apart from the guy who founded it and spent all his time telling people how to live on TV there was no 'research' going on. They just fund raised and paid for their salaries in some pyramid scheme of research. It was most disappointing to find out about.

Tinkering in your shed may not be 'independent research' there needs to be some wider society connection even if research is done in near isolation and with nobody knowing about it yet. Sometimes you need a whole new generation to come along to appreciate the new thinking. Darwin had this problem as did that Galileo chap.

</comment><comment><USER45> Amusing that this type of person goes unnoticed when many of us tell people what we do. I fall into this category as I was describing in another thread. What she misses is many of us aren't rich or the work sustainable: many people in lower to middle classes sacrificing money or status to do deep research and development in a field that interests them and/or they find necessary. They might also not like the priorities of commercial or government funding groups.

For instance, I thought figuring out how to make computers that don't fail or get hacked was a thing we desperately needed. I believed both livelihoods and lives were at stake. That we had access to them was a social good that neither the markets nor FOSS were really serving. It was also an interesting, deep, rabbit hole of a problem crossing many sub-fields of IT, economics, and psychology. That she misses people without money doing it altruistically surprises me more given she wrote the report on FOSS developers working with little to no money or contributions on critical stuff that mattered to them. Same kind of thing I think with different work output.

Still a good write-up that will draw attention to the concept. We might get more people doing it or publishing what they're doing. I think most of us don't publish enough. We should accept some of the troubles of that since the ideas get out there more. I also like this quote focusing on the obsessive nature of deep, independent research:

"I understand, then, why researchers flock to the safety of institutions. Imagine studying something that nobody else is studying, for reasons you can't really articulate, without knowing what the outcome of your work will be. For the truly obsessed person, the need for validation isn't about ego; it's about sanity. You want to know there's some meaning behind the dizzying mental labyrinth that you simultaneously can't escape and also never want to leave."

Edit: I also noted it gets even lower-end in response to a comment elsewhere saying you just need money to live and basic, computing equipment. That's true but maybe not what people picture.

"Even lower barrier than that: many bright folks that were hackers or makers that I met in rural areas were on food stamps or living with someone unemployed without a car. They could usually get a Wifi-enabled phone or old laptop that they could use at nearby McDonalds or something. Many use data plans, too, just cuz cell service is a necessity to them. At one point, I went lower having no PC, phone, or car. Designed on paper or dirt depending on where we were at.

The reading and practicing like you said is what gave us skill. I think peer review and support is just as important, though. I had lots of it once I got online. There's quite a few people out there probably stuck on some subjects, reinventing wheels, or chasing dead ends just because they can't talk to experienced people."

</comment><comment><USER46> With low federal funding rates and every CS Department having doubled in size it is barely possible to do research IN academia. You spend nearly most of your time writing grants that don't get funded. The little federal funding that is available ends up mostly at elite/ivy schools who dominate NSF panels and conference program committees :-( kudos if you can do independent research outside of it though</comment>


Why did Steve Jobs choose not to effectively treat his cancer? - <AUTHOR> http://www.quora.com/Steve-Jobs/Why-did-Steve-Jobs-choose-not-to-effectively-treat-his-cancer

<comment><USER2> While the answer to the question is quite good, the rant at the end really considers any kind of skepticism towards medicine "mumbo-jumbo". The key point is the accuracy of the medical prediction from the measurements. If you indeed have a tumor of the kind Jobs had, then it is certain that you will have problems. However, if you have mild hypertension or high cholesterol, you might have a problem, or then you might not. Treatment itself can cause problems of similar magnitude, and therefore being skeptical can be well founded.

</comment><comment><USER3> > dihydrogen oxide

Water's chemical composition name might sound scary while being harmless, unfortunately, the drugs and procedures for cancer treatment are far from harmless, so let's do away with the 'western medicine is always right for everyone' attitude. This is in general most doctor's point of view, and I can understand it, as it is their area of study and practice.

There is another side to this that is mostly unexplored in America. The idea of a dignified, medically unassisted walk in to death. Some people may be ready to die and that is ok. Not everyone wants to be torn apart with knives and chemicals, for a 65% chance of living 10 more years.

The more information that comes out about how medical costs are dolled out at a rate of 80% in the last years of life causing severe unhappiness, depression, and ultimate body failure should shock us all in to taking a look at how we fear and treat the potential for death. Dying in America is torturous. Maybe it's time we stop fearing it and start trying to understand it in a deeper sense.

Much of my option is informed from this Fresh air interview with Dr. Sherwin Nuland on dealing with cancer in America: [http://www.npr.org/2014/03/07/287272150/fresh-air- remembers-...](http://www.npr.org/2014/03/07/287272150/fresh-air-remembers- surgeon-and-how-we-die-author-sherwin-nuland)</comment>


AMD and ARM's new CPU/GPU virtual ISA - <AUTHOR> http://semiaccurate.com/2011/06/22/amd-and-arm-join-forces-at-last/ <comment><USER2> This is makes sense, esp with regards to the scalar core that is being included in the CU. Most likely that scalar core is ARM derived.

I made noises that AMD needed to remove 32 bit and SSE and just focus on OpenCL. What is being speculated actually reaches much farther than that and is a better design. Thanks for the clue stick.

Also more corroborating evidence, Anton Chernoff one of the main creators of FX!32 (<http://en.wikipedia.org/wiki/FX!32>) works for AMD Boston. FX!32 was/is one of the most badass dynamic recompilation engines I have ever witnessed. It could load X86 executables and effectively JIT them to Alpha. This is the kinda of technology needed to support a virtual ISA.

Anton Chernoff <http://www.linkedin.com/pub/anton-chernoff/1/59/6b2>

[http://www.usenix.org/publications/library/proceedings/useni...](http://www.usenix.org/publications/library/proceedings/usenix- nt97/full_papers/chernoff/chernoff.pdf)

</comment><comment><USER3> Holy crap, memory barriers! It's a modern architecture that's finally caught up to where Symbolics was at in the '80s. Seriously though, this is good news for concurrent garbage collected languages.

<comment><USER4> Barriers are slow, and the more cores are hitting the barrier the slower it will be. The best news on that front was from the new GPU architecture parts, where they admitted that they're still going to expose the relaxed consistency memory model.

<comment><USER3> "Barriers are slow" in what way? Azul was hitting 10-20ms pauses while successfully garbage collecting 500GB+ heaps using hardware memory barriers on their Vega architecture, which had up to 768 CPUs.

Cliff Click, Azul's principal JVM architect, seemed somewhat ambivalent about how much hardware transactional memory actually helped them hit those targets (i.e. it's not a silver bullet), but he chided Sun for not including it on their Niagra chips when they were providing both the language runtime and the hardware platform.

I'll say it again: Getting them on a commodity architecture is a great thing for concurrent garbage collected languages.

</comment></comment></comment><comment><USER5> this is awesome. i had no idea anything like this was coming. is there anything out else out there that confirms this? and will it extend upwards to include bulldozer too?

<comment><AUTHOR> You can find the whole main presentation here: [http://developer.amd.com/documentation/presentations/assets/...](http://developer.amd.com/documentation/presentations/assets/Phil%20Rogers%20Keynote%20Final%206.14.2011.pdf) I'm presuming stuff coming from amd.com is confirmation. It should certainly include future products, what I would worry about is to what extent existing graphics cards will be able to take advantage of this.

Generally, I'd say this looks a lot like Clang but with better ways to express memory parralelism at least. No way to know at this point to what extent this will catch on, but it looks interesting.

<comment><USER6> I think it's bigger than that - the idea that a GPU can execute syscalls is pretty crazy if you really buy into it (i.e. the entire _OS_ is running using this heterogenous environment)

<comment><USER4> This is possible now [1]. FSA will (hopefully) make it significantly more efficient.

[1] [http://www.idav.ucdavis.edu/publications/print_pub?pub_id=10...](http://www.idav.ucdavis.edu/publications/print_pub?pub_id=1039)

</comment></comment></comment></comment><comment><USER7> The part where they mention graphics becoming pre-emptable kinda solves Microsoft's concerns about WebGL DOS attacks.

</comment><comment><USER8> Wouldn't a virtual ISA be against the interests of hardware designers who benefit from ISA lockin?

<comment><USER9> Intel and NVidia want large piece of small pie

AMD and ARM want small piece of large pie

<comment><USER4> Even NVIDIA isn't crazy enough to try to get everyone to use the same ISA again; they're trying to get you to buy into their compiler chain, not into the architecture of a particular generation of GPU.

<comment><USER9> Is PTX really ISA? It's also virtual, but very limited ... Unlike PTX, FSAIL will work on both CPUs and GPUs and even will allow system calls from discrete GPUs.

<comment><USER4> PTX seems quite reasonable to me, but I don't think it's really intended to be a programmer-facing abstraction. NVIDIA wants you to target CUDA, and they'll (try to) ensure that CUDA has sufficient performance advantages to make it worth your while to do so.

</comment></comment></comment></comment></comment><comment><USER10> I hope I'm not the only one who finds this article poorly written; I got a better understanding from the list of bullet points on the slide than I did trying to follow the rambling paragraphs below them.

</comment><comment><USER11> I wonder if Apple will use this with Grand Central?

<comment><USER12> It wouldn't be hard, except for Apple being in bed with Intel.

As they exist today, GCD/libdispatch blocks are very similar to OpenCL native kernels, but with more automatic scheduling and implicit data transfers (which are almost no-ops when the kernel is running on the host device).

On an AMD Fusion system with a shared memory controller, you don't have the huge latency of transferring your kernel arguments to the GPU (and with libdispatch blocks, you are generally transferring less data), so a block could be executed on the GPU with hardly any more overhead than on a different CPU core.

The only problem would be that you would need universal binaries of your blocks to also target the GPU (or LLVM IR to be finalized at runtime). As Apple has complete control over their very modern and flexible LLVM-based toolchain, implementing this would be very straightforward, even without the fruits of an AMD/ARM collaboration. (Though it sounds like ARM may help AMD be able to provide an array of scalar processors that would have good performance characteristics for typical dispatch_async uses.)

</comment></comment><comment><USER13> some damn convincing speculation, i must say.

</comment><comment><USER14> Does this portability compromise performance at all?

<comment><USER15> It compromises performance in a similar way to using any byte code vs final machine code. It remains to see how much of a VM support will be required to make this work, as that's typically the large overhead cost of using non- machine code.

If anything the same exact FSAIL will run faster and faster on the same exact hardware as newer JITs come out.</comment></comment>


Eta – A powerful language for building scalable systems on the JVM - <AUTHOR> http://eta-lang.org/ <comment><USER2> FWIW the demo snippet is a toy and not the real quicksort, which is in-place, rather than copying the list / creating garbage at every step.

Ironically it says "A powerful language for building scalable systems on the JVM" but this literally doesn't scale as quicksort does.

[http://stackoverflow.com/questions/7717691/why-is-the- minima...](http://stackoverflow.com/questions/7717691/why-is-the-minimalist- example-haskell-quicksort-not-a-true-quicksort)

[https://wiki.haskell.org/Introduction#Quicksort_in_Haskell](https://wiki.haskell.org/Introduction#Quicksort_in_Haskell)

This is related to the "Genuine Sieve of Eratosthenes", which is also an imperative algorithm and NOT a functional one:

[https://www.cs.hmc.edu/~oneill/papers/Sieve- JFP.pdf](https://www.cs.hmc.edu/~oneill/papers/Sieve-JFP.pdf)

This brings to mind the slur "Haskell (Lisp) programmers know the value of everything but not the cost".

<comment><USER3> I saw that quicksort code when I looked at Haskell back in 2002. Why is that still used as an example?

I have seen in-place quicksort in Haskell and it is not pretty.

<comment><USER4> The example is simple and expressive, it's relatively easy to understand even if you don't known the language. It gives you the idea of the expressive power.

It's not performant, though, and should not be considered a real quicksort implementation. So it's not showing you the cost of this particular abstraction.

<comment><USER5> "Relatively easy to understand even if you don't know the language" <\- We have empirically verified this by showing code samples of quicksort side-by- side in Java, Scala, Clojure, and Eta to ~100 programmers who had no experience with functional programming at an exhibition. Eta won, followed closely by Scala (people just love their curly braces!). One person liked Clojure because of "it looked like English" and no one liked Java. For this reason, we have posted it on the landing page. We have made no claims that it's the most performant nor that it's the fully correct quicksort (accounting for uniques) - the whole point was to highlight the expressiveness.

<comment><USER6> The fact these people were picked at a conference is already very strong confirmation bias.

Pick 100 random (really random) programmers and I'm betting a lot more of them will feel more comfortable with the imperative version over all the others.

<comment><USER4> Pick 100 random programmers in 1970 and most of them would be more comfortable with GOTO-based programming, for variety of reasons, including performance.

"Whenever you find yourself on the side of the majority, it is time to pause and reflect. (Mark Twain, ~150 years ago)"

<comment><USER7> > "Whenever you find yourself on the side of the majority, it is time to pause > and reflect. (Mark Twain, ~150 years ago)"

That seems to be exactly what hota_mazi is saying.

<comment><USER4> Not that I'm objecting! I'm commenting. Sometimes stating the obvious helps trigger that "pause and reflect" thing.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> For people new to the party, this looks great. For those who had been following the GHCVM project, it's a horrible disappointment.

Basically, this project used to be a fork of the Haskell compiler to support the JVM. Now it's a fork of the Haskell compiler that may or may not be compatible for your use case depending on their opinion of GHC's features.

<comment><USER9> I don't think it is necessarily a bad thing if they remain Haskell 2010 compliant and use pragmas for their own extensions. I certainly would like to see features like extensible records and variants; and this is unlikely to happen anytime soon in GHC, as the focus seems to be dependent types.

<comment><USER5> This is exactly the direction we're going. I'm a big fan of extensible records/variants. We'll be supporting the GHC 8 extensions that are orthogonal to the type system refactoring like Strict, StrictData, etc. pretty much everything other than TypeInType.

<comment><USER9> Exciting stuff! The JVM really needs a modern statically typed functional language.

</comment></comment></comment></comment><comment><USER10> The website is exactly what a programming language website should be. Syntax and features front and center.

</comment><comment><USER11> i feel like each "new" general-purpose language project is just nipping at the boot-heels of the much bigger problem of constructing large-scale systems that can be fully understood by a few (2-3) people.

VPRI's research has shown that one important method for constructing large- scale systems that can be understood by small teams is to have a pipeline of problem-specific languages that express major portions of the system. they were able to reduce LOC for a typical OS with networking and graphics by 3 or 4 orders of magnitude.

general-purpose languages can't compete with DSLs in terms of expression, and yet we keep inventing them. i think our lack of imagination is starting to show. compilation and language design will need to become much more common- place if we expect to continue scaling up.

a tower of babel in computing is _healthy_ , no matter how much employers want us to be easily-replaceable cogs in an IT machine.

<comment><USER4> (1) This language has been invented long ago; its name is Haskell. Eta is a JVM backend / target for it.

(2) A good general-purpose language is good at creating pseudo-DSLs, also known as abstractions or APIs. This can include customized syntax, but usually it's not a brilliant idea. DSLs using the common syntax are quite prevalent in Lisp / Scheme / Clojure, or in Ruby. Haskell is reasonably good at creating DSLs, much better than e.g. Java.

</comment></comment><comment><USER12> Awesome,

Haskell <> JVM with Eta[1]

Scala <> Native with Scala Native[2]

One language to rule them all. Personally am rooting for both becoming a success, though Eta may have the easier path given that it can piggy back on the JVM, while Scala Native will have to, for example, come up with a plausible GC solution (i.e. matching JVM's world class GC is a tall task to say the least) along with porting myriad Java/Scala libs to native.

[1] [https://github.com/typelead/eta](https://github.com/typelead/eta)

[2] [https://github.com/scala-native/scala-native](https://github.com/scala- native/scala-native)

<comment><USER13> I want purity without laziness - I think that's the ideal combination. So I'm most excited for Idris (which has both JVM and native backends).

</comment></comment><comment><AUTHOR> This is the relevant github repository: [https://github.com/typelead/eta](https://github.com/typelead/eta)

And this is the blog post from the co-founder of Typelead (the company behind Eta) discussing about it: [https://blog.typelead.com/https-blog-typelead-com- introducin...](https://blog.typelead.com/https-blog-typelead-com-introducing- typelead-14c4a1bf2df#.7zt3riywq)

</comment><comment><USER14> Why do we need this? We've already had Frege ([https://github.com/Frege/frege](https://github.com/Frege/frege)) filling this role for roughly three years now. Seems to me the DRY principal doesn't just apply to personal projects but to public works as well.

<comment><AUTHOR> This might be helpful:

* [https://github.com/typelead/eta/blob/master/docs/source/faq....](https://github.com/typelead/eta/blob/master/docs/source/faq.rst#how-is-eta-different-from-frege)

* [https://github.com/typelead/eta/issues/3](https://github.com/typelead/eta/issues/3)

<comment><USER15> tl;dr Freda uses a modified version of GHC and doesn't have a lot of commonly used extensions.

<comment><USER16> I'd hope Eta "just" focuses on compiling GHC's core (not Core) intermediate- representation STG (or C--) into Java bytecode: then, no need to forever catch up re-implementing the ever-evolving language extensions, _plus_ this just gets you all the desugaring / Core-to-Core optimization/simplification roundtrips, custom rewrite-rules, type checking / verification etc pp from GHC. In fact no need for any great checks and verifications, "just" (probably highly intricate in the end) a series of code format transformations.. in theory ;) GHC's various intermediate representations are pretty brilliant and thus I think should be leveraged as much as possible when it comes to new compilation targets and transpilation. Fun fact, seems as of 2002 there was for a while an STG-to-JVM backend.

Now there's issues with this approach too! You'd end up with a massively convoluted set of mappings of Haskell's factory defaults to the JVM's. We're talking the equivalents of `Prelude` module, `base` package, the RTS (garbage collector and much more.. ouch!), and for each and every individual instance deciding where to draw the line between Java's built-ins' semantics and GHC's would be a massive challenge.

What's with GHC's existing LLVM backend, does LLVM not have their own Java byte-code backend(s)? (OTOH, additional major dependencies always kinda bite..)

<comment><USER5> Eta in fact only deals with STG code -> Java bytecode transformation. I agree that the intermediate transformations are brilliant, so I am very careful about playing around with the frontend bit of the GHC codebase. Eta currently implements almost all of GHC's primitive operations faithfully. I have taken extreme care in preserving semantics. In some obscure cases though, I just gave up since there are no platform-independent ways of implementing certain things (like vectorised instructions).

<comment><USER16> Now that's awesome to hear! Can't wait to give this a go for making an Android app on the next occasion that pops up

<comment><USER5> GHC 8 will take some time. The current position is to prioritize those new extensions which start creeping up in Hackage libraries. There were significant changes to the codebase in 8 and I want to wait until it's stabilized. Eventually, yes.

<comment><USER16> I thought most extensions "just provide sugar" and are gone by the STG stage.. guess not =)

<comment><USER5> No, you're right. I just don't want to swap out the frontends right now (again, stability). I'll definitely be cherry picking bug fixes and non- pervasive changes once I get a solid test/benchmark suite setup.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER17> Looks like they don't acknowledge it on the landing page, but this is probably an implementation of Haskell, or of a dialect of Haskell.

<comment><USER17> Oh, while the landing page doesn't mention Haskell, the documentation does (it says they don't use the name Haskell in an attempt to avoid the "26 years of psychology and perception that was built around the language").

<comment><USER18> That seems like wishful thinking. Anyone who knows anything about Haskell recognizes it on-sight. Anyone who doesn't will probably find it fairly radically different from what they're used to in a programming language, and not realize that it comes from a deep tradition with lots of community support.

<comment><USER19> I can sympathize with the reasoning behind the rebranding. "A new language on the JVM (which happens to be Haskell)" has a slightly different spin than "Haskell on the JVM". The latter sounds like it is targeted at Haskellers looking for new runtime choices, whereas the former is quite suggestive of moving Haskell a considerable step closer to the comfort zone of a certain group of non-Haskellers.

Unfortunately, the "(that happens to be Haskell)" part is completely left out on the entry page. This gives a terrible first impression, because it seems like if someone would be trying to bootleg those 26 years that went into Haskell and sell them of as their own achievement. The documentation does a reasonable effort to set this straight, but the aftertaste from the first impression remains.

</comment></comment></comment></comment><comment><USER20> I'm interested in what the trade off between "lazy by default" and "Eager by default, with some great lazy data structures you can use" is?

Often programming language support for streams will feel a little second class, and I'd love to have something as first class seeming as Haskells lazy list - but that's a minority of the time for me.

<comment><USER5> Given the recent interest in reactive programming, people will probably like the built-in support for streams via lazy lists in Eta.

</comment></comment><comment><USER21> There is a bug in the code, unless the input is expected to be a set instead of a list.

I believe the right implementation should be something like:

      main :: IO ()
      main = print $ quicksort [1, 123, 42, 90, 1, 23, 24, 12, 3]
      quicksort [] = []
      quicksort (x:xs) = quicksort left ++ [x] ++ quicksort right
          where left  = [ y | y <- xs, y < x ]
                right = [ y | y <- xs, y >= x ]

<comment><USER5> Thanks for the fix! It's up on the site now.

<comment><USER22> Have you considered replacing the example entirely? A toy program that looks good but is terribly nonperformant is exactly what I'd put up if I wanted to make a Haskell-style language look bad.

<comment><USER5> Performance of a tiny code sample is not why people select one language over another. If that was the case, no one would be using anything besides assembly or C. Even in other languages, you'll use an abstraction heavy implementation for the benefit of long-term maintenance and code evolution and then performance optimise the bits that can pack a punch. Performance optimisation in any language looks ugly be it C or Haskell.

<comment><USER23> > Performance of a tiny code sample is not why people select one language over > another.

It's not how people _should_ choose a language, but I'd wager it plays a non- trivial part even if only from laziness alone.

We have to remember we aren't marketing to perfect* developers but more to the average if an aim is mass adoption.

*Perfect as in follows general programming best practices. Following best practices is not always best. Not trying to make a value judgment here.

</comment></comment></comment></comment></comment><comment><USER24> There's a bug in the quicksort implementation example (it assumes all values in the list are unique).

<comment><USER25> Also in the STM example, they only ever read sender's balance and never the receiver's balance.

<comment><USER5> Great catch! I've pushed the update. Should be fixed soon.

</comment></comment></comment><comment><USER26> Looks similar to Haskell.

Edit: Darn, I'm stupid. It's a port of Haskell to JVM.

</comment><comment><USER27> That's great news !

I'm quite disapointed by the decision regarding GHC8, though. Overloaded record labels adresses a problem which the community was quite annoyed with, for instance.

<comment><USER5> Don't worry. Solving the records problem is a high priority item for Eta. We might solve it in a different way than GHC 8, so that's where the 'dialect' part comes in.

</comment></comment><comment><USER28> I don't really mind if Eta ends diverging from Haskell in the particulars but remains Haskell-like à la Purescript.

I'm curious about how the jvm will behave in regards to laziness and typeclasses (MTL-like libraries, in particular).

<comment><USER5> The benchmarks I've run are extremely promising, and Eta is sometimes competitive to GHC after warmup. GHC's aggressive optimisations with -O2 + Oracle JVM's JIT seem to be a very powerful combination. I have avoided publishing the results because I still need to spend more time looking at the numbers and understand why was so fast in certain cases (check up on any mistakes I've made). I'd rather take the time to report proper benchmarks rather than hype unverified results.

</comment></comment><comment><USER29> Does it come with a debugger of any kind?

I think this is the biggest gap in the Haskell ecosystem right now. Every time I try to write something in Haskell, I remember I cannot "talk to my program" easily and just give up.

<comment><USER5> You can use JVM debugging tools and with a guide on how to interpret the results for Eta (or a tool that does the mapping to Eta source code for you), debugging can be pretty awesome. The implementation was specifically designed to re-use the JVM Thread stack as much as possible so that you can get stack traces and figure out the source of the error. You'll get stack traces similar to other JVM languages like Scala and Clojure. See [http://eta- lang.org/docs/html/eta-user-guide.html#debugging-...](http://eta- lang.org/docs/html/eta-user-guide.html#debugging-stack-traces) for an example. You can use tools like slf4j-ext as java agents and get nice traces - it's how I debug the Eta runtime.

</comment></comment><comment><USER30> How is this "scalable"? What does that mean?

</comment><comment><USER17> The quicksort example seems oddly named, I would have called it "unique" or "removeDuplicates".

If you change one of the inequalities, say ">" to ">=", then it does sort (without removing duplicates). It still wouldn't feel fair to call it "quicksort" because it isn't in place.

</comment><comment><USER31> Awesome! built in Bangalore - [http://typelead.com/contact-us/index.html#our- office](http://typelead.com/contact-us/index.html#our-office)

The stock image on the about-us page threw me off. Great going guys!

We have already been sold on Kotlin... but Eta looks interesting.

</comment><comment><USER32> Hmmm...could get me back into Haskell (cough--mean Eta). I always hated the Haskell environment. A nice to work with port to JVM could inject some life back into that interest. There was Frege, but it just seemed fringe at the time...we'll seem if this has any steam behind it.

</comment><comment><USER33> Has a gitter and slack but no IRC?

</comment><comment><USER34> This makes me wonder again: is there any such language but for people wanting an _eager_ (not lazy) Haskell?

(and I don't care for it to be JVM. but I do care for it to be be Haskell... OCaml is not Haskell and has too many weird dark corners imho)

<comment><USER35> Idris

</comment></comment><comment><USER36> When you write about this new language, you'll be on the no-fly list soon :)

<comment><USER37> i guess i haven't been watching the news much... why?

<comment><USER38> [https://en.wikipedia.org/wiki/ETA_(separatist_group)](https://en.wikipedia.org/wiki/ETA_\(separatist_group\))

</comment></comment></comment><comment><USER39> If they had any marketing sense, they would start by telling me why it's better than Clojure. I'm sorry to offend the Scala guys, but if you want to be the best JVM language, you need to improve on Clojure.

<comment><USER40> I hope what you're really saying is that they need to do a better job of marketing to folks not already familiar with Haskell or similar ML-derived functional programming languages.

And that's probably true.

As for specifically competing with Clojure... frankly, who cares? The two are so different as to render such a comparison fairly useless: Lisp-y s-expressions versus ML-style syntax. Strict Hindley-Milner-style type system vs Lisp-style dynamic typing. Lazy evaluation versus strict. Pure versus impure. The list goes on and on.

Clojure is an interesting language, no doubt. But your response would seem to imply it's the end-all and be-all of programming languages, and given how subjective such an assessment would be, it strikes me as a fairly vacuous claim...

<comment><USER39> We are talking about the JVM. And as such, the existing competitors are Java, Scala, and Clojure. Scala had promise, but many significant users discovered warts on large projects. And on top of that, Lisp is not only sexy (rationally or irrationally), but Clojure has a lot of intellectual momentum. Further, we have Clojurescript for nodejs. It is the one to beat.

So if someone is going to sell me on a better JVM language, they need to at least give me a pro/con list vs Clojure. While HN might disagree, the market (people who work in companies that live for revenue, not VC) will agree in my interest for that comparison.

<comment><USER41> I looked at Clojure but the lack of static types and thus losing the benefit of refactoring always put me away. Out of curiosity - how do you safely refactor Clojure code ? Safe Refactoring offered by type safe languages is such a major aid to redesigning code that it has become difficult for me to work in dynamic languages for any large code bases.

<comment><USER42> I've been a full-time Clojure dev for about 4-5 years now. And the number of times you have to refactor (in the traditional sense) Clojure code is pretty rare. And the reason is that Clojure leverages it's use of immutable data structures, namely hash maps.

So let's say I have a function that needs :name and :addr and it adds a new field calls :name+addr. Now in most (all?) statically typed languages I would have to say that this function takes a "name and addr" and returns "name, addr, and name+addr". So I have a type conversion, right? If the time comes that I need to modify these types, I have to do some sort of refactor.

In Clojure we'd just take a hash map and add a new k/v pair to the hash map we get. Any hash map will work, as long as it has the proper entries, and we'll just add a new entry to that. So if the time comes that I want to call this function with a "company" instead of a "person", it just works, as long as I have the proper keys. And to help check this sort of stuff we do have spec ([https://clojure.org/about/spec](https://clojure.org/about/spec)).

TL;DR -- my belief having programed in both static and dynamic languages is that deep refactoring is primarily driven by the inflexibility of static types.

</comment></comment></comment></comment></comment><comment><USER43> One of these must need some division...

        feet2Meters :: Feet -> Meters
        feet2Meters (Feet ft) = Meters $ ft * 0.3048

        meters2Feet :: Feet -> Meters
        meters2Feet (Meters m) = Feet $ m * 0.3048

<comment><USER44> No, in Haskell, if it compiles, it works.

</comment></comment><comment><USER45> Seems 99% written in haskell/eta itself, with a little c.

So, the compiler won't run on the JVM... if it did, there could be haskell dev on-the-device for Android...

<comment><USER5> There are plans to make the compiler self-hoisting once it's more stable. The little bit of C used is just to turn on an option in the GHC runtime for profiling purposes. That would be replaced with some Java code to turn on an option in the Eta runtime if it were self-hoisting.

</comment></comment><comment><USER46> I'd like to know if the most scalable technique for reasoning about program data works: structural induction!

Is Eta strict or lazy?

<comment><USER5> Lazy. I think we have enough strict JVM languages.

<comment><USER46> What a pity. There are several strict JVM languages, but there aren't enough of them with compound values, i.e., unencumbered by pervasive use of object identities when manipulating compound data.

<comment><USER5> Laziness by default does not mean you can't have strict primitive types, which Eta does. Moreover, if you ever want to avoid object references in performance-sensitive contexts, you can allocate off-heap memory and work directly with that via the Ptr mechanism (which is backed by DirectByteBuffers). GHC 8 recently got compound values with an extension called UnboxedSums. It would be tricky to implement on the JVM though in Eta, but not impossible.

<comment><USER46> In a lazy language, a strictness annotation doesn't change the type of the annotated thing - bottom still inhabits that type.

On the other hand, in a strict language with a laziness monad, the difference between, say, `foo -> bar` and `foo lazy -> bar` (ML syntax) is as clear as daylight. The types tell you what's going on.

<comment><USER47> OTOH, most of the things I actually use laziness for in Haskell do not fall into the `foo lazy -> bar` category, because they involve things like floating out IO actions or renaming things, and in fact, changing the type makes them _more_ cumbersome e.g.

        if x then error "bad" else thing

to

        let z = error "bad" in if x then z else thing

Which isn't a valid transformation in any strict language. This general idea is pervasive in the code I write, where the act of binding something is immaterial to its evaluation. I think we use this style a lot more than we give ourselves credit for in Haskell. You can of course wrap this in a thunk, and some of the usage style can be approximated by a monadic type. But this is all just really cumbersome and annoying to do pervasively. It's the best benefit I get from laziness, to structure code this way. You also end up duplicating a lot of strict-vs-lazy code either way you pick, since "crossing the streams" is generally either forbidden by the type system (in your example) or you need the different implicit characteristics (like in Haskell). It's not really clear to me this is a win overall.

I'm not opposed to strict languages, but IMO, I think if you want a strict language, you're better off just forgoing the whole thing, and using lambdas (thunks) where needed for small delays, and a good macro system to define control structures for everything else rather than trying to shoehorn laziness into your types or whatever. Random thunks you occasionally need aren't really the benefit. Being able to decouple definition from evaluation is.

<comment><USER46> In any language, not just a lazy one, "let x = v in t" is beta-equivalent to "t[x:=v]", whenever "v" is a beta-equivalent to a value in the language. Of course, in a call-by-need (or pure call-by-name) language, every term is a value. In a call-by-value language, some terms are not values (and this is a feature).

</comment></comment></comment></comment></comment></comment></comment><comment><USER48> This looks really cool. How are the language services coming along? Can we expect something on par with F#'s?

</comment><comment><USER49> I really don't understand the need to squeeze as much functionality as possible out of as few lines as possible, that the code makes no sense no read. How am I supposed to parse the cryptic beginning example which shows the 2048 game being played? It looks like a ton of stuff has just been hidden way, which is fine for the example but likely all falls apart when you try to add something new to the game.

<comment><USER50> I don't think the code in the 2048 example is complete, it is just some GUI boilerplate. I would love to see the rest of this.

<comment><USER5> You can, see [https://github.com/rahulmutt/eta-2048](https://github.com/rahulmutt/eta-2048). The whole point of that example was to demonstrate that you can build nontrivial things with Eta right now. Just have to deal with Java FFI boilerplate which requires some patience without any IDE support.

</comment></comment></comment><comment><USER51> I'd personally just support Frege more instead of reinventing the wheel. [https://github.com/Frege/frege](https://github.com/Frege/frege)

<comment><USER4> These are different beasts. Frege does not do the GHC extensions. Eta seems to reuse much of the GHC's frontend transformations, while Frege is built from the ground up.

</comment></comment><comment><USER52> Does anybody knows about syntax highlighting support?

<comment><USER5> Throughout eta-lang.org we use a custom syntax highlighter we developed for CodeMirror (used in the playground) and Pygments for the static code examples throughout the docs. We will translate these to other platforms like Emacs, Vim, Eclipse, IntelliJ, etc. once IDE support is better.

</comment></comment><comment><USER53> What an unfortunate name.

<comment><USER16> Your nickname just made my, er, moment..

<comment><USER54> Wouldn't notice if you didn't say it :) Tx

</comment></comment></comment><comment><USER55> nice

</comment><comment><USER55> wow

</comment><comment><USER56> Yay, Haskell on the JVM.

Again.

</comment><comment><USER57> Java's hunger for memory, combined with Haskell's unpredictability wrt memory allocations due to laziness. What could possibly go wrong.

</comment><comment><USER58> Yet another JVM language. YAJVML

</comment><comment><USER59> Its amazing that if you put down all the languages that consider themselves powerful and scalable you somehow reach what religion is...everybody thinks their god is the only one, and if one indeed exists all the rest are wrong by default!

Technology is fair and there is a reason why haskell is not used apart from niche or academic projects. If you want to use jvm to deliver functionality for your users just a "boring" technology. No user, never, cared about the programming paradigm.

<comment><USER60> > haskell is not used apart from niche or academic projects

While haskell doesn't have as wide adoption as the top languages, your comment is not true. Even facebook uses haskell for fighting spam.

> If you want to use jvm to deliver functionality for your users just a > "boring" technology

You're entitled to your preference, but I assume you'd say the same about about Clojure and Scala then, since, after all, everyone should just be using Java?</comment></comment>


A ruby conference you don't want to miss. Paris, France, June 2018 - <AUTHOR> https://2018.rubyparis.org/ <comment><USER2> As an occasional attendant of the meetup, and a member of the related Slack, I'm pretty sure this will be an awesome event!

</comment><comment><USER3> Definitely I will be there!

</comment><comment><USER4> Seems awesome, definitely will be there !</comment>


The woman behind Ikea's living wage calculator - <AUTHOR> http://www.marketplace.org/topics/business/woman-behind-living-wage-calculator <comment><USER2> [http://livingwage.mit.edu/](http://livingwage.mit.edu/)

I found reading the project more interesting than the article.

</comment><comment><USER3> _Olson says it would be too complicated to pay people different wages based on household size_

... also, it would be wildly unfair, and quite likely illegal.

<comment><USER4> Certainly illegal. Marital status is a protected class in the United States.

<comment><USER3> In Canada we have a "disadvantaged group" rule, which basically says that discrimination is fine as long as you're discriminating in the right direction. I wasn't sure if the US had anything similar.

<comment><USER5> Interesting; how does "disadvantaged group" status get determined? Is there a list; or would it be left to a judge's discretion?

EDIT: Is this the law in question?

[http://en.wikipedia.org/wiki/Employment_equity_(Canada)](http://en.wikipedia.org/wiki/Employment_equity_\(Canada\))

<comment><USER3> Which groups qualify as "disadvantaged" is generally up to judges to decide, with the caveat that since Canada has a common law system (portions of Quebec law notwithstanding) these questions have pretty much all been resolved already.

I was thinking mainly of the various human rights acts, but the principle applies across a wide variety of contexts. The federal government is constitutionally barred from creating laws which discriminate against aboriginals, for example, but is allowed to create laws which discriminate in favour of aboriginals since they are considered to be a disadvantaged group.

</comment></comment></comment></comment></comment><comment><USER6> >So what would be enough to live on? That would of course depend on where someone lived, and how much that place costs. And so Glasmeier rounded up some of her best graduate students to create, basically, a giant spreadsheet. They loaded it up with the best regional data available, from government and industry surveys, on costs for housing, food, child care, medical expenses, and transportation.

I don't get it: she "avoids tourist traps" (a few paragraphs up) to find the places people really shop. But when estimating cost of living, she uses detached government aggregates rather than finding how people actually make their wages work, meaning she would necessarily miss eg how grandma provides most of the daycare services.

I would think that the primary question in building out a living wage figure would be to ask how people are currently pulling it off on low wages, rather than extrapolate one's own life through some categorized figures.

<comment><USER7> Most likely the professor is pulling together whatever information she can with the resources she has. Can you imagine how big of a ground effort it would take to collect the information you described?

<comment><USER6> Absolutely! That's why I wasn't criticizing the missing country-wide tabulation of such data, but trying to reconcile its absence with the praise of the researcher at the beginning, and how she collects another kind of data on the ground.

Of course, that could just be stuff getting scrambled by the PR machine.

My main criticism was this: if you're going to calculate a living wage, wouldn't the number one question be "How are people currently living on their wages?" Even knowing how _one_ family pulls it off would be tremendously informative, and from that point you could figure the cost of the missing amenities. In contrast, the researcher's approach is to assume they buy everything from some model budget (probably based indirectly on her team's own lives), but less of it.

</comment></comment></comment><comment><USER8> The wonderful thing about the free market is that you don't _have_ to use magical algorithms to determine the "correct" price of things, you just set the price you're willing to buy/sell at, see how many takers there are, and then adjust the price upwards/downwards.

If you need to hire twenty people, and you find twenty people willing to do the job for eight bucks an hour, and you can't find twenty people willing to do it for seven bucks fifty an hour, then eight bucks an hour is the correct price. It's not that tricky.

</comment><comment><USER9> IKEA Richmond BC. Google that + strike, they have been striking for over 1 year over labor dispute.

They demolished the existing one and opened a bigger one right next door. It was barely open before strike shut it down.

The changes in this article do not help Canadian IKEA workers get a fair wage

<comment><USER10> Given its about Ikea in the United States, why would it help Canadian workers? Plus doesn't Canada have a much better minimum wage in the first place?

</comment></comment><comment><USER11> Kind of depressing that the largest charity in the world pays just enough to live.

[http://www.economist.com/node/6919139](http://www.economist.com/node/6919139)

<comment><USER12> What? A charity's purpose is to help those most in need. Every dollar a charity gives to someone beyond what they _need_ is a dollar they can't give to someone whose needs still aren't met.

<comment><USER13> I think you missed reading the linked to article. According to the article, IKEA is actually legally incorporated as a charity, though it's only "charitable" purpose is "innovation in the field of architectural and interior design," and the charitable foundation appears to only be a shell built to avoid taxes and funnel money to the founder (in order to maintain the facade, it donates minute amounts to the Lund Institute of Technology).

So IKEA, despite being from all outward appearances a privately held for- profit enterprise, it technically categorized as a charity, and despite all of this pays many of its employees below a living wage, though has now promised to start paying them a living wage, at least according to this calculator.

<comment><USER12> Oh, I read the article. I object specifically to the idea of my parent, that -being a charity- implies they should be giving their employees more. As if charities were in the business of being generous.

</comment></comment></comment></comment><comment><USER14> 10.76 x 40 x 52 = $22,380.80

Pretty appalling.

<comment><USER15> I have a wife and a son and we did fine on my salary of $21,000 last year.

There are a lot of parts of the country where that's a perfectly viable wage.

<comment><USER16> That seems nuts to me. I significantly more than that, and it still feels like we're scraping by. Our location doesn't have a particularly high cost of living, either.

<comment><USER15> one man's "barely scraping by" is another man's "I've got everything I need".

I've got a house, reliable transportation, health care, food, a great school for my kid, and lots of family nearby. My cost of living would go way up if I ate out more, bought all new furniture, or spent big money on activities for my son, and then it would seem like I needed a lot more money.

<comment><USER17> I think what he was saying though, which I'd agree with, is that in most places if we assume a minimum livable wage to be ~10$ (which is a generous statement, I think), one barely comes out at ~20k (napkin math) yearly. Yes, you may be able to cover the necessities, but you mentioned healthcare, if you had an unlucky year where you were responsible for the maximum of your copay, that would probably put you into savings? Or if there was an "act of god" that wasn't covered under, say, car insurance (to give an example that nailed me recently, a hailstorm), there's another few k.

As some sort of point to all this, the discussion on livable wage needs to encompass the fact that there are some big holes in our financial system that people can still fall through that will hamstring you for years to come, even if the base wage is technically livable.

</comment></comment></comment></comment></comment><comment><USER18> Hasn't IKEA heard of the Sharing Economy? Living wage is socialism. We're hackers, we should be figuring out ways to externalize the real costs of doing business onto a contract labor force with flexible hours and no guarantee of basic income. Keep people hungry, that's how you squeeze the most of them.

</comment><comment><USER19> Federal minimum wage is below the survival line (without children), as showed by Amy Glasmeier. So low, that corporations have to find what is the real lower bound for the survival of their workforce. (Not even reproduction of the workforce, just its survival) Isn't that absurd?

Let rise the federal minimum wage now.

<comment><USER15> There are lots of people who work but don't need (and aren't worth) survival wages -- high school kids who still live with their parents, for example. Should they be unemployed and therefore unable to gain real-world working experience because they're not worth survival wages?

There are lots of others who need, but aren't worth, survival wages -- like my autistic friend. He partly supports himself working, with the rest coming from government programs. Should he be unemployed and therefore be 100% government- supported instead of 25% government-supported?

It's good that minimum wage and living wage are not the same thing. Let's stop confusing the two.

<comment> FLUX-YOU >There are lots of others who need, but aren't worth, survival wages

That's got to be such a small (and unfortunate) pool of people that I'd happily send some taxpayer money to meet the article's living wage number. We can probably pay for it by not buying another single jet next year or something.

And honestly I'm not trustworthy of corporations' judgments of low end labor worth anyway. For one thing, you can work at extremely busy stores and get paid the same as workers at the very dead stores in the next county over because they use blanket values for these positions.

<comment><USER15> > _" I'd happily send some taxpayer money"_

We already do that. Remember how some people were criticizing Wal-Mart because some of its workers were also receiving government benefits?

When you set the "minimum" to be 100% of a living wage, then people have to make the jump from pure-government-funded to pure-self-funded without any intermediate stepping stones. They have to show up and be ready to produce at a level that completely supports themselves on day one. Whereas a minimum that's set at 50% (or 25% or 75%) of living wage means someone can show up and be semi-productive and earn _some_ of their own way, and then either step up to 100%, or stay at the level they're at as long as they need to based on their own abilities.

</comment></comment></comment></comment><comment><USER20> It's sad that such a worthwhile tool like this uses data that is four years old. I realise the publish cycle of government agencies is slow and as such is a limiting factor, but in this day and age with everything being fast moving, it can be limiting.</comment>


Ask HN: Custom share button vs official share button? - <AUTHOR>

Hello there,
I was wondering, which option do you think is better:
-A custom image for a share button for Facebook and Twitter.
-The official share and tweet buttons provided by the developers sections of Facebook and Twitter.
Which one, is more attractive or more trustworthy?
Let me know what you think, Thanks, <comment><USER2> Personally, I have a higher propensity to click on official buttons than their third-party counterparts.</comment>