Is Arduino no longer open-source? - <AUTHOR> http://lists.oshwa.org/pipermail/discuss/2017-June/002127.html <comment><USER2> Having worked in the Open Hardware community for a while, I think the most important part is:

PDFs are not considered design files [https://www.oshwa.org/definition/](https://www.oshwa.org/definition/)

If you apply this rule, you can exclude 90% of the "Open Source/Hardware" products on the market.

Also 99% are missing a BOM (Bill Of Materials). That's like shipping an open source software without the config files, forcing the users to figure out the settings by reading the sources. Unless they are buying the pre-compiled binaries from the author.

<comment><USER3> _> PDFs are not considered design files [https://www.oshwa.org/definition/](https://www.oshwa.org/definition/) _

This is a very good point!

BTW, in that sense most Creative Commons work is not "open source", either.

For example, a typical Creative Commons video that is just released as webm or mp4 can only be remixed, but not improved upon - which would require access to the raw video material, raw audio material and the project file of the used video cut program.

Same for CC music distributes just as mp3/flac/opus, you'll need the samples, notes, music cut program file, and so on. One notable exception are tracker files (mod, s3m, it, ...), and in some sense also the MIDI files (as long as you have access to all instruments).

<comment><USER4> The definition from OSHWA is tailored for hardware. OSI has the authorative for software.

But yeah, music and graphics frequently also don't include the actual 'source' \- just a rendered output. This is way better that having a propiatary license on the output, but does not fulfill the full potential.

Some of it has to so with the tools and proceses, which frequently aren't really built to make collaborating on the source materials easy. And often the formats and tools used are proprietary. Freeware VSTs abound for instance...

<comment><USER5> Perhaps a key difference is that OSHWA isn't authoritative.

OSHWA are the FSF of hardware. As far as they're concerned, only their certification constitutes open hardware. Everyone else is wrong. Sound familiar?

As someone else pointed out, a lot of people don't provide BOM files, including people who have OSHWA certified hardware.

</comment></comment></comment></comment><comment><USER6> My colleagues have developed a "spitting noise" ritual after any utterance of the word "Arduino" (<hock> ptui). As embedded developers attempting to teach others, they're not enamored of Arduino's burgeoning, and dominant, market of "people who want to do embedded development but not learn anything about embedded development" sucking up all the oxygen. I know, that's an oversimplification, but not as much of one as I could wish.

I don't want to wish anyone ill, but the emergence and success of competitors that don't go so far out of the way to hide the complexity (and the need for good practice) of working with embedded systems and SBCs, wouldn't be such a bad addition to the ecosystem.

(As one of two developers who actually fool around with Arduinos and Pis a fair bit (although I detest the Arduino dev environment and 'sketches'), I'm tired of all the vandalism and graffiti in my cubicle, TBH).

<comment><USER7> Lets view what your saying through another lens:

My colleagues have developed a "spitting noise" ritual after any utterance of the word "PC" (<hock> ptui). As MAINFRAME developers attempting to teach others, they're not enamored of PC's burgeoning, and dominant, market of "people who want to do COMPUTER development but not learn anything about MAINFRAME development" sucking up all the oxygen. I know, that's an oversimplification, but not as much of one as I could wish.

I don't want to wish anyone ill, but the emergence and success of competitors that don't go so far out of the way to hide the complexity (and the need for good practice) of working with COMPUTERS systems and MAINFRAMES, wouldn't be such a bad addition to the ecosystem.

(As one of two developers who actually fool around with PC's and APPLES a fair bit (although I detest the APPLE dev environment and 'PRODOS'), I'm tired of all the vandalism and graffiti in my cubicle, TBH).

Am I suggesting that this is going to be how things turn out? No probably not. It baffles me that many people who code, have little to no understand of their mainframe roots and commit sins that were solved ages ago. However some of this reaction is to the simplification and replacement of technology with things that are more affordable and more in reach even if they aren't the same.

<comment><USER6> There's another plane on which the analogy fails: the first generation of PC enthusiasts were working with a genuinely new thing. The consumer/hobbyist- grade hardware that the introduction of the 4004/8008/6502 made possible was quite literally impossible only a few years prior. And there was good competition (OK, we Apple ][ folks despised the 'Trash-80', and everyone looked askance when the Commodore 64 with its glitzy marketing toward people who didn't even _own_ soldering irons showed up, but we peripherally recognized each other as factions of the same tribe, and there were options).

Neither of those things is true of the Arduino. I learned how to design simple control systems on Microsequence Controllers ancestral to the AVR more than _thirty years ago_. And the techniques I learned there still apply to the more sophisticated chips available now. Even the hobbyist platform stuff isn't new. I was wire-wrapping DIP 650*'s into perf-board in the early 1990's, and I built an HC11 model rocket launch controller for my kid the better part of two decades ago. And I know more than a dozen people, not all of them professional engineers, many with only two-year college diplomas who were doing sophisticated hobbyist projects well before the turn of the millennium.

The Arduino isn't analogous to the Apple ][. We had to learn how to program in BASIC and needed to know what a memory map was. It's analogous to Windows 95: a well-engineered product that completely dominates the market, establishes that you don't really need to know anything, and stifles innovation.

<comment><USER8> > stifles innovation

I'm really curious about this - what innovation does Arduino stifle?

From my position with some school level dabbling in embedded, it seems like the industry is quite healthy and that there is a lot of innovation happening in terms of hardware outside Arduino, especially on the commercial side of things, where the perception is that Arduino has next to no market share.

</comment></comment></comment></comment><comment><USER9> The answer to the question if these products are open source, the answer is: no.

Here is my blog post from more than 2 years ago regarding the status of the Arduino Yùn:

[http://www.wifi4things.com/arduino-yun-what-is-under-the- hoo...](http://www.wifi4things.com/arduino-yun-what-is-under-the-hood/)

And here is my clarification request on the Arduino forum, which is still unanswered:

[http://forum.arduino.cc/index.php?topic=187766.0](http://forum.arduino.cc/index.php?topic=187766.0)

<comment><USER10> I would say that the Betteridge's law of headlines has been proven once again by this thread, but this is no article and the headline here seems more click- baity than the actual subject line of the thread.

<comment><USER11> The answer to your parent comment's question is "no", but their question was the inverse of the post's title.

So, this actually is an exception to Betteridge's law; the answer to the question is "yes, arduino is no longer open source".

</comment></comment></comment><comment><USER5> Hopefully I can provide a bit of insight on this, currently working on open hardware myself.

OSHW certification is a self-certification process. Individual products are certified, not companies. What Torrone is doing is conflating Arduino the company (or foundation, or companies, or whatever it is this week, guys please stahp) and Arduino the product _series_ , and individual Arduino products. He knows what he's doing, but in this email does it anyway. Perhaps for good cause, but it's what he's doing.

There is no requirement for Arduino to certify all of it's hardware with OSHWA. In fact given the explosion of things-that-are-branded-arduino it would be unsurprising to see hardware that cannot be certified as open due to commercial agreements that override open licences. Whether this affects Arduino or not long term is a different matter but the Uno, Nano and Pro Mini are all still open.

OSHWA is an attempt to define what constitutes Open Source Hardware and to define some (hopefully) sane defaults. It's the hardware equivalent of the FSF, and it's licence is the maker equivalent of the GPL. It is not the only fruit.

My project[1] uses the V-USB library, which comes with an open licence[2]. The project is derived from another project, the digispark[3]. We're operating within the licence constraints, and consider our project to be open hardware (we're posting our board designs when we ship at the end of this month).

To further complicate things, when Arduino uses the term Open Source it rarely clarifies what that means. It may be referring to the software stack. It might be (but probably isn't in this day and age) referring to new hardware.

Either way, while it is right to pressure Arduino to clarify their position, I think it's unfair to mandate that every product they ever produce must comply with OSHWA's definition of Open Source Hardware, or that only OSHWA certified hardware can be considered "open hardware".

[1] - [https://hidiot.com/](https://hidiot.com/)

[2] - [https://www.obdev.at/products/vusb/license.html](https://www.obdev.at/products/vusb/license.html)

[3] - [https://digistump.com/](https://digistump.com/)

<comment><USER12> The OSHWA requirements for compatible open hardware licenses [1] seem to allow for permissive open source licenses, similar to Apache 2.0. Attribution may or may not be required. Derivative works are required to follow the terms of the original license, but there doesn't seem to be a default requirement "share- alike".

[1]: [https://www.oshwa.org/definition/](https://www.oshwa.org/definition/)

</comment></comment><comment><USER13> I think the Arduino civil war is inevitably going to lead to ESP8266/NodeMCU boards taking over this market.

<comment><USER14> What counts as "open source"?

We often run (open-source) Linux on a (closed-source) Intel processor and even a lot of diehard FOSS warriors are okay with that.

Arduino Unos are a bunch of (open-source) wiring and components plugged into a (closed-source) Atmel microcontroller and people seemed to be okay with that.

How does an ESP8266 differ from an ATMega in this regard? If we consider it a basic component just like the ATMega, it's not particularly any different.

<comment><USER15> This question comes up a lot.

It doesn't mean "open source all the way to the materials mined."

It means "open source to what _I_ used to produce this.*

Such that another person could replicate your work.

<comment><USER14> Sure. But just like one could source an ATMega, one could source an ESP8266 and replicate the work. It's an easily-available drop-in component. So I don't actually see anything inherently different about the new crop of Arduinos from what we had before.

</comment></comment></comment></comment><comment><USER16> I started with arduino a few years ago as a hobby and loved it as an easy way to get into the field but lately I have been realizing how many terrible habits I've obtained by learning this way. It took me weeks to retrain my brain to prefer looking at schematics instead of the breadboard views and I'm still trying to learn atmel studio and directly programming chips. It's coming along but slowly.

I still think arduino is a great way for interested people to break into electronics but there is no clear 'line in the sand' when they should stop learning that way

<comment><USER17> Do yourself a favour and just use 'make' and 'g++' directly, ignore the Arduino wrappers, they're just toys that hamper you after a while.

<comment><USER18> How?

<comment><USER19> The wrappers are slow. I have a project where I need to sample data at 50kHz and update at 5kHz, 14-bit output. Can't do it with Arduino's syntactic sugar.

The IDE is terrible for large, multi-file projects. It just doesn't have the level of integration of, e.g., Visual Studio (that Atmel Studio is based on).

It's a fantastic beginner tool and a great source of ultra-cheap hardware, but the IDE runs out of steam pretty quickly.

</comment></comment></comment></comment><comment><USER20> Good questions on AdaFruit's side. I am interested in knowing what is the result of this inquiry.

</comment><comment><USER21> Also relevant: [http://hackaday.com/2015/03/28/arduino-srl-to- distributors-w...](http://hackaday.com/2015/03/28/arduino-srl-to-distributors- were-the-real-arduino/)

<comment><USER22> [http://hackaday.com/2016/10/01/arduino-vs-arduino-arduino- wo...](http://hackaday.com/2016/10/01/arduino-vs-arduino-arduino-won/)

</comment></comment><comment><USER23> Creators do not like to follow rules. Avoidance of a bunch of nitpicks is probably the only way to motivate them.

IMHO, it is not a war, just a childish fight that is mostly positive.

</comment><comment><USER24> 3/5 of the boards mentioned here have not actually been publicly released as far as I can tell (Primo, Star Otto, Star LCD).

</comment><comment><USER25> You cannot remove the open-Source from something that was previously open- source. I'm pretty sure its a one way street.

<comment><USER26> Once you've licensed something as open-source and published, that version with that license will always be there. You could take a future private version of the thing (assuming no code contributions) and license that differently.

<comment><USER27> Well, except that gratuitous licenses are (irrespective of their surface terms) revocable at will. Promissory estoppel may mitigate, in some way, the effect of such revocation against people who took action in reliance on a promise of nob-revocation before or without knowledge of the subsequent revocation, but it is far from clear that the Free version will always be free; license terms don't override the governing law.

(Now if there is a Free version under a contracted-for rather than gratuitous license, that's a bit more secure, though there are ways that could go away, too.)

<comment><USER28> Do you have any court cases to back this assertion? The GPL is the license of millions (maybe billions) of lines of code. We've all bet on them being non- reversible on released code.

Given the way Linux is copyrighted (with _many_ holders), it'd be an absurd situation if any copyright holder could just decide, after the fact, that they don't want their code being distributed under that license anymore. In fact...I think that's been litigated in SCO vs. IBM. So...what are you basing your legal theory on here?

<comment><USER27> > Given the way Linux is copyrighted (with many holders), it'd be an absurd > situation if any copyright holder could just decide, after the fact, that > they don't want their code being distributed under that license anymore.

Even leaving _aside_ general issued on the revocability of licenses that aren't special to copyright law, the US has a special provision making _all_ licenses _and transfers_ of rights by authors under copyright revocable by written notice, during a 5 year window 35 years from when they occurred; see 17 USC Sec. 203.

> So...what are you basing your legal theory on here?

The general American (Anglo-American, I think, as I'm fairly certain the principle is a common law one which is older than the US) of licenses.

<comment><USER28> 17 USC Sec. 203 also covers derivative works and the fact that they remain distributable under the original terms (though future derivative works are up in the air if your interpretation holds water, which I still don't really buy).

Further, revocation of copyright on works with multiple authors must be signed off on by a majority of copyright holders, per the law you've cited. That's literally impossible with something like Linux (but maybe not with something like Arduino, if it only has a tiny number of authors, I don't know).

If it's so simple and obvious, why has it never happened in 30+ years of GPL software, when billions of dollars are at stake?

"The general American (Anglo-American, I think, as I'm fairly certain the principle is a common law one which is older than the US) of licenses."

Many things in "common law" have been replaced by written legislation and case law. Modern copyright bears no resemblance, and only has only tenuous connections, to common law. Copyright is among the most debated and litigated categories of law in the modern world, with legislation, legal precedent, and even international treaties covering it. If your position is that it is as you say because common law is as you say, that just sounds really shaky. Now, I need to ask you to back up the assertion that "common law" is the law in force on copyright in any developed Western nation, because that seems to be the crux of your interpretation of the law.

I don't know, man. I'm not an expert, by any means, but I'm just not following your reasoning here, at all.

My understanding of the GPL is that it is a one way street for released code. New releases can be under a new license if all of the authors agree to it, but once something is out there under the GPL, it is always under the GPL. Nothing you've said makes me think otherwise because the weight of precedent seems to disagree with you.

</comment></comment></comment></comment></comment></comment><comment><USER29> Adafruit is not as cool as one might expect. I was sourcing boards from them and my parcel got missing from courier company. I complained to them. Initially they tried to help but nothing came out of it. Guess what they did next ? My account with them was disabled . It almost knocked my little startup off the ground. I wrote mails but to no avail.</comment>


Dyslexia caused by faulty signal processing in brain - <AUTHOR> http://www.sciencedaily.com/releases/2012/08/120807104938.htm <comment><USER2> "... shown that many difficulties associated with dyslexia can potentially be traced back to a malfunction of the medial geniculate body in the thalamus."

I always wonder if things like Dyslexia are more a symptom from something else, like how ADD can be caused in people who have food sensitivities or allergies but don't realize it - or perhaps a terrible childhood where the person / mind / brain learned distraction behaviours for coping, and therefore until those distraction behaviours are explored that ADD person will never learn how to focus.

And why is it considered faulty? How can one determine the pros and cons of how a mind will develop due to how the brain works? It really bothers me that much of very narrow scientifically-done research doesn't reach back out into a holistic view. I understand that if you keep the narrow context than their statements remain 'true' - however that isn't beneficial on an individual-by- individual basis when our body and minds are so complex.

A quick fix sounds nice, is 'cheap' in cost (if you ignore side-effects and that many times medications are bandaids that tend to fall off or let symptoms [from something underlying] worsen), and is purely driven by profit-motive - at least that's where funding comes from, even if it's in researcher's hearts to trying to find answers to help others. We know things like MSG and artificial red food dies cause ADD in a large percentage of the population, though in North America we still allow it in our food - whereas in much of Europe such things are ban.

Anyway, clearly I am for preventative/proactive health and not putting temporary bandaids on things. Research is showing more and more that bandaids (not literally) end up costing society more than dealing with the base issues.

<comment><USER3> "We know things like MSG and artificial red food dies cause ADD in a large percentage of the population, though in North America we still allow it in our food - whereas in much of Europe such things are ban."

First, MSG has not been banned in Europe, and you can buy this food additive and use it for your own home cooking, I should know because I sometimes use it instead of salt. Second, you'll have a hard time finding an actual study proving the harmful effect of MSG, and links to websites with no credibility like rense.com doesn't count. It's not a coincidence that most links point to stuff like that on the first page of google when you search for MSG and they are popular among conspiracy nuts/fear mongers.

If you listen to the conspiracy theorists, MSG causes diabetes, alzheimer, autism, heavy migraine/headaches, parkinson, seizures, addiction to MSG itself.. stop propagating the bull*.

<comment><USER4> So, I know nothing about MSG, but when I see people say things like "you'll have a hard time finding an actual study proving the harmful effect of X" I tend to go check, as it is do easy to do so: it makes for a good check of academic honesty.

Regardless, this claim is clearly wrong: PubMed has tons of papers about random things being associated or impplicated with MSG, including neurological effects. In particular, here are some papers about excitotoxins (MSG in particular) and fibromyalgia.

<http://www.ncbi.nlm.nih.gov/pubmed/11408989>

<http://www.ncbi.nlm.nih.gov/pubmed/22766026>

<comment><USER3> Unfortunately for you, you haven't proven anything about MSG being the root of any disease whatsoever, as both links only proves what it can do to someone who already has some kind of disease (fibromyalgia in this case). This is not so different from the harmful effect of gluten to someone who has coeliac, or sugar to a diabetic, but no sane person is going to demonize gluten or sugar just because a very small subset of the population are harmed by those substances. Or milk. Or peanuts.

Almost any type of food source could be thought of as harmful based on your own criteria. I can find the kind of study you link to for so many things that you'd develop a fear of eating itself.

The truth remains : there is no _STRONG_ correlation between MSG and the kind of horrors the demonization campaign stirred up by conspiracy theorists pretends it to do to the human body. You type "monosodium glutamate" in google and one of the links on the FIRST page is a link to rense.com, a website maintained by 9/11 deniers and UFO believers, and it's for a good reason if that link is on the first page, because talk about MSG is most popular amongst that group of people.

Anything that is not "natural" (for some highly subjective value of) is prone to be hated and part of a fear mongering campaign from the type of people that are obsessed with fear, fear of technology, of progress, of science, of politics [...]

<comment><USER4> The fact that there are nut jobs that believe something extreme that have high page rank does not mean that a more moderate position is false. Meanwhile, the argument that science cannot "prove" anything undermines the idea of science being valuable at all.

<http://scholar.google.com/scholar?q=MSG+glutamate>

That search result is littered with articles in legitimate sources testing MSG in various contexts, showing random negative effects on hormone regulation and brain function in general. Are these important effects? Maybe not; but you have to look at the effects in question.

What is quite clear, though, is that tons of actual scientists have studied MSG and have turned out actual papers showing actually statistically significant effects, albeit in models that you might disagree with under dosages you might consider absurd. Others, of course, might find them highly reasonable.

You can't, however, just wave your hands and dismiss all of this work: to do so is just ignoring science, and apparently trusting some combination of your gut feelings on the matter and Google's page rank algorithm to establish truth. There is science out there, and that science should be discussed.

<comment><USER3> People who are NOT researchers but spreading the same shit everywhere on the internet are nothing but conspirationists. Including you. Because you contest what is officially recognized as OK, and do so without yourself having the medical authority/background to back up your claims. A random hodge podge of various, completely unrelated studies does not a point make. The experts of the FDA and various health authorities around the world have read the same studies as you do, and allowed MSG to be used in our food, and you have no authority to contest that.

I can bring up the same hodge podge about gluten and ask for a ban.

<http://www.ncbi.nlm.nih.gov/pubmed/16436335>

<http://www.ncbi.nlm.nih.gov/pubmed/18787912>

<http://www.ncbi.nlm.nih.gov/pubmed/16423158>

<http://www.ncbi.nlm.nih.gov/pubmed/1727768>

<http://www.ncbi.nlm.nih.gov/pubmed/19664354>

<http://www.ncbi.nlm.nih.gov/pubmed/20170845>

<http://www.ncbi.nlm.nih.gov/pubmed/15497770>

<http://www.ncbi.nlm.nih.gov/pubmed/16436335>

Here you have equivalent material on gluten, hell you'll even find an association with schizophrenia. Let's fear-monger and stop eating bread and pasta !

With your really low, actually unscientific standard of using science studies, I can make it so that you'll never eat any food again, because I can find the same stuff about most food sources.

Since everything is so unsafe why don't you just kill yourself and be done with the unsafe misery that is life, cowering in your primal fears. You can't eat msg, but you can't eat gluten either, nor table salt, or sugar, or various carbohydrates, or milk.. oh but you can't eat some meat either because there are correlations to cancer. In fact, studies on meat have shown greater negative effects on the body than studies on MSG.

"A 2011 study of 17,000 individuals found that people consuming the most grilled and well-done meat had a 56 and 59% higher rate of cancer" [http://www.nature.com/bjc/journal/v106/n3/full/bjc2011549a.h...](http://www.nature.com/bjc/journal/v106/n3/full/bjc2011549a.html)

Are you going to stop eating meat forever because of the relatively small increase of risk of getting cancer ?

This is why I call you a conspirationist. There are "natural" food sources, like red meat, that have been proven as much more dangerous (and it's still a relatively small danger, statistically speaking) than MSG yet people like you focus on MSG and Aspartame because FUCK ARTIFICIAL PRODUCTS right ? Red meat is natural so it's all good right ?

My point wasn't that I trust pagerank more than science. It's that there is a popular FOCUS on something that ISN'T MORE dangerous than stuff like read meat, and it's for a reason : fear mongering about anything that's artificial, man-made, call it whatever you like. The science speaks for itself because none of the study you will link to will prove MSG as a bigger danger than something like red meat. And yet, you don't see anyone trying to bring up a fear-mongering campaign against meat. The only people who campaign against meat are the psychotic vegetarians from PETA who do so in the name of a ridiculous "ethics", being against the killing of animals.

Seeing educated people like you spreading this bullshit against MSG doesn't help to educate most people about the reality of food. Most of your food sources can be toxic to a % of the population. Big fucking deal.

</comment></comment></comment></comment></comment></comment><comment><USER5> The grammar in the article is terrible. For a minute i wondered if i were looking at some kind of spammy fake article.

But now, i think it more likely it is a poorly written summary of legitamite research.</comment>


How many dev shops do code diagnostics as a service? - <AUTHOR> https://blog.oursky.com/2018/03/19/code-review-better-software-project-estimate/ <comment><AUTHOR> How big is the demand for this and what types of clients would be in that position to know that they should do one but also trust a third-party to give an accurate assessment (without trying to upsell)?</comment>


Python or Ruby for fresh startup - <AUTHOR>

Which one to choose? Both have great MVC frameworks. But I've heard that python has a bad mysql provider. Ruby is difficult to deploy... Is it easier to find rubypeople or pythoners? <comment><USER2> There's nothing wrong with Django's MySQL-driver, and RoR might be the easiest thing in the world to deploy with Heroku. It seems to me you should read up a bit on both, try them both out, and rely a bit less on rumor and hearsay. In reality they are both great choices for many types of web apps, but which one is right for you depends on a lot of different factors. Find out which one you like to work with the most, and you probably have your answer.

</comment><comment><USER3> Finding Ruby and Python people is the least of the considerations. There are tons of people for both, including a huge swath of them who can do both well.

I'd recommend reading the discussion here: [http://stackoverflow.com/questions/91846/rails-or-django- or-...](http://stackoverflow.com/questions/91846/rails-or-django-or-something- else)

Some great back-and-forth that may help illuminate your decision.

(I assume you're talking about Rails vs. Django, but Python has some lighter- weight frameworks that can speed up development. Ruby may too, but I'm not as familiar.)

</comment><comment><USER4> I will suggest you to start with the one you are most comfortable with. Both has strong community. Being a pythoneer, I can assure that python community and IRC is very has lots of people who are really very helpful. I dont know much about ruby.

</comment><comment><USER5> "Ruby is difficult to deploy..."

That's simply not true, and I wonder why you think it is.

Setting up Passenger Phusion is snake simple, and deploying could not get any easier ("cap deploy", as one example).

You can also run your app using JRuby and deploy it to any standard Java app server as a war file. (Using JRuby gives you other benefits as well, such as speed and connection pooling, plus the use of every Java library.)

"Is it easier to find ruby people or pythoners?"

I've had trouble finding Ruby devs. And while I've never tried to find a Python developer I get the impression there are fewer Python devs in my area (Phoenix, AZ, USA) than Rubyists. When I talked to some people about trouble finding devs I was told it's a similar problem looking for Java or C# or PHP people.

#ruby-lang on freenode is a great place to get help. #rails, maybe not so much, though I'm basing that more on hearsay and comments from people I know. OTOH, #ramaze rocks, and would be my first choice for any new Web app. Less ceremony, full-featured, great + friendly community.

Bottom line, though, is that you should spend some time with each language, look at a few Web frameworks, and pick what makes you feel most comfortable.

</comment><comment><USER6> I personally prefer Python and Django. Like hucker said, there is nothing wrong with the Python MySQL drivers (although I prefer PostgreSQL too). That said, it's a personal preference and Ruby on Rails is an outstanding platform. If your sysadmin has trouble deploying any one of them, hire a better one.

</comment><comment><USER7> Both are great options. I am not sure where you heard that python has a bad MySQL provider or that Ruby is difficult to deploy. Even though those are opinions, I would highly disagree with both. I have worked with Rails (ruby), Django (python), Pylons (python), and now Pyramid (python). It really is a matter of opinion. You will find much more information about Rails and it is easier to get started with, but I have found that after a couple hours on a Rails project I am fighting Rails more than it is helping me. So it is a highly opinionated topic, but I personally prefer Python using the Pyramid framework. It does have a higher learning curve, but to me it is worth it.

</comment><comment><USER8> Deployment of Django apps are becoming as easy as deploying Rails apps to Heroku. Two simular services for python in private beta are www.ep.io and www.gondor.io

</comment><comment><USER9> Are you choosing a language or a framework? You should try both and play with both of their frameworks and take the path that feels right and gets things done most quickly initially. It's not one or the other. It's have tools in your belt and know how to use them.

</comment><comment><USER10> I chose Ruby because of Heroku.

</comment><comment><USER11> Rails has more plugins than Django. If you choose Ruby go for Rails. If Python have a look at Pyramid and TurboGears

</comment><comment><USER12> Ruby with Heroku is tough to beat. It has worked very well. I'm using Padrino instead of Rails.

</comment><comment><USER13> Ruby isn't at all difficult to deploy. There are plenty of hosts, and cheap hosts at that.

I tried Django in my early coding days, and I liked it at first. I tried Rails and switched immediately. I think it was the fact the Django used mostly Regular Expressions for routes, and Rails wasn't so over sophisticated. The pythonic semantics Django uses just seemed awkard to me when I was a beginner, and the ruby way Rails used seemed a lot easier to pickup.

If you are looking for community members, both ruby and python have large amounts of users, and active irc, as does both Django and Rails. Rails does seem to be more actively developed(to me) than Django. Django hasn't had a major release since v1.0.0, and Rails is on version 3.0.5.

That's just my 2 cents. Whatever you like better, and whatever fits the job, I would go with that one.</comment>


Visual way for program and monitor microcontrollers - <AUTHOR> http://www.agilart.com/community-and-support/video-tutorials/agilart-getting-started

<comment><AUTHOR> This is a simple, well not so simple, project for making programming microcontrollers easy, without to know anything about C/C++, compilers, etc. The main idea is to use web-based tools to make program, debug them and deploy on your hardware. It is REST-API based so you can make HTML based dashboards and control your programs remotely. I'm just curious what you think?</comment>


The case against native engines - <AUTHOR> https://www.scirra.com/blog/ashley/28/the-case-against-native-engines <comment><AUTHOR> Author/submitter here - just some context for HN, we make consumer software aimed at non-programmers, so the post is written to that audience. I think there's still interesting insight for other developers there though.

<comment><USER2> Thanks, this post is also very useful to me as a hobbist working on a FLOSS game (RPG) construction engine. I hadn't heard of Cordova or of the ability to run Win 10 universal apps on XBox One and web apps on Wii U. Sounds like it might be a good idea to port from native desktop to HTML5/JS using emscripten, and then port that to these non-desktop platforms. Was there much work involved in porting to XBone or Wii U?

In [1] you wrote

      It's not clear whether this UWP app support is actually
      intended for games publishing, whether game UWP apps will
      be accepted for the Xbox, or if you're meant to use some
      other publishing option.

Do you know any more about that now?

[1] [https://www.scirra.com/blog/ashley/26/run-your- construct-2-g...](https://www.scirra.com/blog/ashley/26/run-your- construct-2-games-as-xbox-one-apps)

<comment><AUTHOR> The "it's not clear" quote was just excessive caution when Microsoft announced Xbox One support. It's turned out fine!

</comment></comment></comment><comment><USER3> I've done a bit of game development for the Web. Unless things have changed very recently, a major performance bottleneck is draw calls. It feels a lot more like a mobile platform (< 100 draw calls) than a desktop platform (1000 draw calls). Unfortunately, it's quite an effort to reduce draw calls without making graphics compromises, curious how well Construct 2 does there.

If you have the data, I'm curious to know what proportion of games are fillrate bound vs draw call bound vs cpu bound.

<comment><AUTHOR> In most cases, draw calls aren't a problem for us - the OpenGL API (and WebGL inherits this) is designed to be able to batch a lot of work with few draw calls, and the Construct 2 WebGL renderer uses a batching system accordingly. (Note we have a 2D game engine so it's a bit less demanding.) Of course you can make worst-case scenarios which issue lots of separate draw calls, but most cases work nicely. For example if you create 1000 of the same sprite in Construct 2, it can issue a single drawElements call to render them all in one go.

<comment><USER4> For 2D games where you can batch heavily (Textures in atlas, single material) WebGL is fine. And for games like that it indeed does not make sense to go native if it works fine on all target platforms. Except if you care about power consumption.

For modern 3D game, even in mobile, you do need way more than that and 100 draw calls is very limiting.

</comment></comment></comment><comment><USER5> >asm.js is progressing in to WebAssembly, a more mature technology (still in development) with enthusiasm from all major browser makers.

If it's still in development, then in what way is it "more mature" than asm.js?

<comment><AUTHOR> WebAssembly solves problems like growing the heap, as well as being a proper binary format that is smaller and quicker to parse. The design is definitely ahead of asm.js, which is really smart but still a bit of a backwards- compatibility hack.

</comment></comment><comment><USER6> Has the Web Audio API improved dramatically in just the past two years? From what I had previously read [1], it seemed like a disaster.

[1]: [https://chadaustin.me/2014/09/web-platform-limitations- part-...](https://chadaustin.me/2014/09/web-platform-limitations-part-2-web- audio-api-is-a-mess/)

<comment><USER7> I've been working quite a bit with the Web Audio API recently (building a modular soft-synth), and I can confirm that it is a bit wonky, but ultimately more powerful than one might expect.

There are lots of little oddities that I had to research or discover to get the API to do what I want. For example, dynamic sound generation nodes will not run in certain circumstances where the system believes they are unattached from the audio output. These nodes also have a bad habit of garbage collecting their event listener functions.

Performance is definitely an issue as well, and I still get some audio tearing type pops and clicks despite quite a bit of optimization.

I think some of these issues are due to that fact that the Web Audio API spec is still being worked out ([https://webaudio.github.io/web-audio- api/](https://webaudio.github.io/web-audio-api/))! Some API components are deprecated in the newer versions of the spec, but have not been replaced in current browsers. Which leaves the API in a weird limbo state.

BUT! All that said. I've been pushing the API pretty hard to build this modular synth and it is working admirably. I think a game engine would have an easier time getting it to do what is needed. I'm always amazed at what you can do in the browser these days, Web Audio API included.

</comment></comment><comment><USER8> People have been saying this for a while, but this question is only going to get to continue to lean more and more in the direction of html5 clients in general.

It's the easiest method to develop cross platform applications. Things like Electron make it possible to bundle apps to make them look and act like native apps.

This post is mostly about performance, and secondarly about a team's ability to focus. Which this allows them to do and deliver to many different platforms. With the rise of webassembly this will only get better.

Is native finally something of the past? Or will this end up just being a rehash of the Java JWT?

</comment><comment><USER9> So this probably just shows my ignorance, but I was under the impression that modern browsers isolated running js to running on a single processor? The only part of the article to touch on this was the part talking about the operating system possibly running on another core. It seems like the elephant in the room.

Is it just not true any more that browsers isolate js to a single thread?

</comment><comment><USER10> I love Construct 2 because it is easy to use. I made my first game on it and it was the only one that launched. However, Construct games have notable performance issues and as a programmer, we can make things in other engines just as easily.</comment>


Yahoo seeking up to $500 mln for small business unit - <AUTHOR> http://www.reuters.com/article/smallBusinessNews/idUSTRE58K4UG20090921

<comment><USER2> Just out of curiosity, did you see that Yahoo! Store (previously Viaweb) is part of the small business division?</comment>


China is harvesting organs from detainees, tribunal concludes - <AUTHOR> <comment><USER2> seems like OP forgot link

[https://www.theguardian.com/world/2019/jun/17/china-is- harve...](https://www.theguardian.com/world/2019/jun/17/china-is-harvesting- organs-from-detainees-uk-tribunal-concludes)

<comment><AUTHOR> Oops thank you!</comment></comment>


UK trials new breathing aid developed by Mercedes Formula One - <AUTHOR> https://www.platformexecutive.com/news/blog/uk-trials-new-breathing-aid-developed-by-mercedes-formula-one/ <comment><USER2> There seems to be a huge amount of innovation going on in this right now, which I think is ray of sunshine in this otherwise increasingly bleak picture. Part of me wonders though if this is because everyone is focusing on it, or rather we are so desperate that we try anything that looks like it _might_ work.</comment>


Why do I learn computer science? - <AUTHOR>

So, I am confront in a dilemma, Could I learn computer science? Could some cool guys share me with the experience for that? <comment><USER2> Mind expansion and to learn proper English skills.</comment>


AMD Looks Poised to Gain at Intel's Expense - <AUTHOR> https://www.forbes.com/sites/kenkam/2018/01/04/amd-looks-poised-to-gain-at-intels-expense/ <comment><USER2> This is so weird, AMD is just as vulnerable

<comment><USER3> That's simply not true. AMD i not vulnerable to Meltdown only to Spectre.</comment></comment>


Show HN: Reddit on a map - <AUTHOR> http://www.lokeel.com

<comment><AUTHOR> Hi, I'm sure many of you could probably blow this out of the water, but I'm pretty proud of this little app. This is my first app. Having graduated with a degree in Biology I decided to teach myself ruby on rails to put together a "news map" because I felt this was missing when I was looking for my first place to buy, and I wanted to get a vibe of a local area.

<comment><USER2> Don't short change yourself.

This looks fantastic. Would love to see what it looks like when volume increases.

</comment></comment><comment><USER3> Love the idea! Always wanted to do something with news and the map. Really cool way of visualizing whats going on around the world!</comment>


The New York Taxicab Bubble Couldn't Last Forever - <AUTHOR> https://www.bloomberg.com/view/articles/2018-06-19/uber-taxi-medallions-and-new-york-city-s-cab-bubble <comment><USER2> >a glut of taxi drivers made it impossible to make a living as a taxi driver

Then don't be a taxi driver. Supply will return to a more sustainable level and then suddenly it will be profitable again to be a taxi driver.

Instead now we have an artificially restricted supply of taxi drivers so that the state and its cronies can profit off the sale of taxi medallions.

Please.</comment>


Apache Kafka Goes 1.0 - <AUTHOR> https://www.confluent.io/blog/apache-kafka-goes-1-0/ <comment><USER2> Congrats to the kafka/confluent team.

Side note: [https://pulsar.apache.org](https://pulsar.apache.org) also seems to be gaining traction and has a much better story around performance, pub/sub, multi-tenancy, and cross-dc replication. Will be interesting to see the evolution of both going forward.

<comment><USER3> Interesting. Can someone give a quick/high level list what makes it better than Kafka and in what use cases?

<comment><USER2> I recommend reading the Pulsar architecture page: [https://pulsar.apache.org/docs/latest/getting- started/Concep...](https://pulsar.apache.org/docs/latest/getting- started/ConceptsAndArchitecture/)

Biggest difference is that Pulsar splits compute from storage. Pulsar _brokers_ are stateless services that manage topics and handle clients (publishers and consumers) while reading/writing data from a lower tier of _bookies_ , which is the Apache BookKeeper project for low-latency real-time storage workloads.

Better performance and easier scaling compared to Kafka's partition rebalancing. It also natively supports data center replication and multiple clusters working together globally. Pulsar addressing is property/cluster/namespace/topic so multi-tenant isolation is built-in.

Anywhere you use Kafka can use Pulsar and also consolidate pub/sub only systems that need lower latency than Kafka.

<comment><USER4> I think it is beyond partition rebalancing. There is a fact that people didn't realize of making message broker `stateless`. It is actually much better on reacting to failures or shifting traffic, which is critical when running a messaging bus for online services. because it doesn't have to wait for copying the data of a whole partition when error occurs.

<comment><USER2> Agreed, the architecture page and other blog posts do a more thorough job of explaining the details. Having a stateless broker layer on top of a focused data layer makes all the operations much easier. I expect BookKeeper to further integrate with the various cloud storage APIs so it can also start to become stateless cache.

</comment></comment></comment></comment></comment><comment><USER5> For the folks using kafka or kinesis or other products with streaming event architectures - do you have a replay capability? For example, I am using kinesis and I have a lambda that processes events and writes to a postgres database. so say 3 months from now I want to also create a lambda that firehoses to redshift. How do I get the first 3 months of my data into redshift? Right now I have a lambda that writes all events to S3, so I can replay them for other stream consumers (or even for debugging existing consumers). This seems like a reasonable (if naive) solution, but I don't see a lot of talk out there in the first place about ensuring this capability exists, and definitely haven't seen any more robust or thoughtful options discussed. Are people just not doing this generally?

<comment><USER6> We have something overly dumb and simple. We generate about 400GB of logs a day through kinesis to lambda to s3. And a second lambda that reads of s3 and forwards the stuff to wherever using an s3:objectCreated event ...

If we need to replay something we just move the group of files in and out of the folder on s3 (bucket/logs -> bucket/temp -> bucket/logs) and it triggers a new objectCreated and fires the lambda.

We execute the 2 move operations with the AWS cli tool.

The one other thing we do is we have a config file in bucket/config.json that's says event type X should go to redshift, pg, es, where ever. So we can tweak that before a move to send data to additional data stores.

We follow ELT instead of ETL for streaming into data stores.

<comment><USER7> What are you doing that makes a half-terabyte of logs per day? That seems kinda nuts.

<comment><USER8> Ads ads ads!

That's just one system. Our Nginx access logs alone are about 250GB per day (they use that system).

</comment></comment></comment></comment><comment><USER9> I'm currently comparing using Kinesis vs running a small scale Kafka cluster on AWS. The ecosystem around Kafka is great, especially Kafka connect's stuff like Debezium. But I don't know if it's worth the trouble to deal with the extra operational complexity. Any opinions on administrating Kafka at small scale?

<comment><USER10> Disclaimer: I'm one of the creators of Kafka and founders of Confluent

The best way to get the power, throughput, latency of Kafka without the operations is to use a hosted service. The one created and supported by the Kafka team is Confluent Cloud [https://www.confluent.io/confluent- cloud/](https://www.confluent.io/confluent-cloud/)

<comment><USER11> I love companies developing a product also offering a managed implementation as a service, but I cannot find the pricing.

Any time it seems like I have to establish some sort of relationship and get an enterprise tailored solution, I no longer feel like I'm part of a public and elastic market.

<comment><USER12> I beleive the phrase is "If you have to ask you can't afford it".

<comment><USER13> Hehe, that only applies to luxury goods, which are decooupled from supply/demand rules.

</comment></comment></comment></comment></comment><comment><USER14> For workloads that aren't as important durability wise, has anyone considered nats? [https://nats.io/](https://nats.io/)

<comment><USER15> I use NATS in production. Kafka and NATS are almost complete opposites. One is a durable, partitioned queue; the other is a no-persistence message queue.

<comment><USER16> Aren't they fixing that with NATS Streaming Server?

<comment><USER2> There's nothing to "fix" \- NATS is a distributed and fast pub/sub system. NATS Streaming is actually a client, it uses NATS to communicate and basically just subscribes to some topics, saves messages it receives, and will then replay them back to you if asked.

Imagine how you would build a persistent logging app on top of NATS, and that's what NATS Streaming is.

<comment><USER14> We built a metrics relay using it (nats) and are pushing > 500k metrics per second in bursts just fine. I've been pretty blown away with the perf of it actually. That said, it isn't durable, so normal here be dragons warning apply.

</comment></comment></comment></comment></comment><comment><USER17> I really hope they would stop with that exactly once thing.

It does not mean what they use it for, and it does generate a lot of confusion.

<comment><USER18> How so?

<comment><USER19> "exactly once" is not possible, but you can do "at least once" and make the processing of the message idempotent or "de-duplicated". This is true of many messaging systems.

[https://www.confluent.io/blog/exactly-once-semantics-are- pos...](https://www.confluent.io/blog/exactly-once-semantics-are-possible- heres-how-apache-kafka-does-it/)

> Exactly-once Semantics are Possible: Here's How Kafka Does it

> Now, I know what some of you are thinking. Exactly-once delivery is > impossible, it comes at too high a price to put it to practical use, or that > I'm getting all this entirely wrong! You're not alone in thinking that.

blah blah blah ... of course it's at-least-once, with the "idempotent producer" so only one resulting message is published back to another kafka stream. Big surprise.

Now many people think "kafka has exactly-once delivery, that's what I want, I don't want to have to deal with this at-least-once stuff" when really it's the same thing, and others have been doing idempotent operations of various kinds for years, and the user still has to figure out how to do their desired thing (which might not be sending one more kafka message) in a mostly idempotent way.

<comment><USER20> From the very same blog post:

> Is this Magical Pixie Dust I can sprinkle on my application?

> No, not quite. Exactly-once processing is an end-to-end guarantee and the > application has to be designed to not violate the property as well. If you > are using the consumer API, this means ensuring that you commit changes to > your application state concordant with your offsets as described here.

I think that is a pretty clear statement that end-to-end exactly once semantics doesn't come for free. It states that there needs to be additional application logic to achieve this and also specifies what must be done.

<comment><USER19> Right. But that's at the end of a separate article, while the post which this HN discussion is about throws around the words "exactly once" a lot more casually. The argument is over the use of the words "exactly once". They should just refer to the feature as "transactions" or "idempotent producer".

</comment></comment></comment></comment></comment><comment><USER21> Do people use kafka for situations where data loss is not tolerable(eg: accept credit card receipts).

<comment><USER22> Yes, but the default configuration is not suitable to that situation. You will need to make some adjustments if you cannot tolerate data loss.

<comment><USER21> What are the alternatives people are using instead of kafka in these situations. Low volume but high reliability.

<comment><USER23> For low volume my recommendation would be to just use a traditional relational database configured for high availability.

If you want to use Kafka and need disaster recovery capabilities we typically recommend using Kafka Connect or other similar tools to replicate the data to another cluster or persistent storage system such as S3.

<comment><USER24> +1. For use-cases which impose strict data durability requirements (either for business or regulatory reasons), I think it's unwise to use anything fancy like Kafka unless you've maxed out performance of your SQL database.

For example, for credit card receipts, simply by the nature of the type of transaction, you're unlikely to be processing enough of these to put pressure on a SQL database. One $1/transaction per second means you're grossing north of $30m, which is easy to handle in even an unoptimized schema. Citus reckon you can get thousands to tens of thousands of writes per second[1] on Postgres, which would be grossing tens or hundreds of billions of dollars; this tech stack is suitable even when "low volume" becomes quite significant.

Of course, Kafka is designed for situations where you need to process millions of writes per second, which is into "GDP of the whole world" territory if those writes are credit card receipts, so I'd contend you're unlikely to ever need something like Kafka scale for your credit card payment handling components.

[1]: [https://www.citusdata.com/blog/2017/09/29/what- performance-c...](https://www.citusdata.com/blog/2017/09/29/what-performance- can-you-expect-from-postgres/)

</comment></comment></comment></comment></comment><comment><USER25> ugh... i dispise this crap..

" the "it just works" experience of using Kafka. "

Blog post says how it just works, then shows a crazy diagram stringing together 8 different pieces of technology (which themselves are insanely deep). Is there any easy way to understand WTF this crap does without knowing about RDS, hadoop, monitoring, real time BS, streaming etc? Does it "just work" once you've spent weeks and months understanding wtf it is?

<comment><USER2> Nothing in that diagram has anything to do with Kafka, it's just showing that you can connect different systems together using a single message bus. If you are new to that concept then I recommend reading the seminal blog post about using logs as the data backbone: [https://engineering.linkedin.com/distributed-systems/log- wha...](https://engineering.linkedin.com/distributed-systems/log-what-every- software-engineer-should-know-about-real-time-datas-unifying)

After that, you can choose whichever implementation you want, of which Kafka is just one option. And yes it does require some work to setup and run, as most software does, but it's no more or less than any other distributed system at this point.

</comment></comment><comment><USER26> Kafka has come a long way. Kudos to confluent team.

I came across some projects which claim to be full or partial alternative to Kafka with better scalability and less operational overhead.

Jocko - [https://github.com/travisjeffery/jocko](https://github.com/travisjeffery/jocko)

SMF - [https://senior7515.github.io/smf/](https://senior7515.github.io/smf/)

MapR-ES - [https://mapr.com/blog/kafka-vs-mapr-streams-why- mapr/](https://mapr.com/blog/kafka-vs-mapr-streams-why-mapr/)

Has anyone tried it in their project ?

<comment><USER2> None of those are alternatives. Jocko's creator now works for Confluent and it's not production ready or supported, more of a hobby project to recreate kafka in go without zookeeper.

SMF is a prototype RPC system and libraries based on the C++ Seastar framework, not a messaging system in itself. MapR-ES is part of the MapR hadoop/data platform and also not standalone.

If you want alternatives then look at NATS + NATS Streaming [1] or Apache Pulsar [2].

1\. [http://nats.io](http://nats.io)

2\. [https://pulsar.apache.org/](https://pulsar.apache.org/)

<comment><USER26> Thanks for sharing the Jocko's creators story.

Yes. MapR-ES comes with its own File system and tool chain but its more closer alternative to Kafka than NATS as it follows Kafka 0.9 API and can act as drop in replacement. Although it does not support exactly once semantics yet.

<comment><USER2> Usually alternatives are about providing similar functionality, not the exact interface. If you just want a Kafka API then there are multiple adapters available for most other messaging systems, including pulsar.

MapR is an entire data platform and suite of services, one of which is the ES (event streams) offering. It's not drop-in for Kafka unless you're already running the MapR platform which is much more involved than just running Kafka.

</comment></comment></comment></comment><comment><USER27> So can it be used comfortably now with anything else than Java?

<comment><USER28> I've been using it in Rust, but librdkafka had (has?) a bug that's been fixed in tree that cut the read throughput to about 10MB/s using loopback (edit: cuts throughput to 10MB/s in many settings, which hurts especially in loopback). With that fixed, it seemed pretty pleasant for my use cases (framed bytestreams at line rates, nothing fancy or web-scale).

Edit: crate I was using, and enjoy (which brings in librdkafka): [https://github.com/fede1024/rust-rdkafka](https://github.com/fede1024/rust- rdkafka)

</comment></comment><comment><USER29> Is Kafka good for processing hundreads of GB size messages per second? Can also a Kafka Stream (output) be input to another Kafka stream dynamically?

<comment><USER30> Kafka expects messages to be under 1MiB. If you need larger messages, they recommend you store the payload somewhere else and use Kafka to pass around pointers to the payload. So e.g., pass around an S3 URL.

</comment></comment><comment><USER31> kafka transactional producer not working on windows 7, this bug not yet fixed.[https://issues.apache.org/jira/browse/KAFKA-6052](https://issues.apache.org/jira/browse/KAFKA-6052)</comment>


Is There Anything Good About Men? (2007) - <AUTHOR> http://denisdutton.com/baumeister.htm

<comment><USER2> This article is full of speculation and really casts an uncritical eye towards the thinly spread facts within.

"The huge difference in reproductive success very likely contributed to some personality differences, because different traits pointed the way to success. Women did best by minimizing risks, whereas the successful men were the ones who took chances. Ambition and competitive striving probably mattered more to male success (measured in offspring) than female. Creativity was probably more necessary, to help the individual man stand out in some way." \- pure speculation from Baumeister here and in a lot of this talk

The implication of this talk is really clear: that feminists should shut up about the inequalities between men and women, because men have it bad/worse, too. The thing is, a lot of contemporary feminist thought discusses precisely those issues, and sees the destruction of all baggage surrounding gender as the goal.

However, it seems like a lot of contemporary thinkers in the same vein as Baumeister here have missed that boat, and are still stuck with the view that feminism is locked in the pre-simone de beauvoir days where women had to fight to win the vote and own property. Contemporary feminists generally see the fight for lawful (de jure) equality as a more minor concern when compared with social equality. Baumeister's argument that wealth and success was created in the "men's sphere" and women were left behind, is actually more an affirmation of the contemporary feminist struggle against social inequality than anything else.

Contemporary feminist thought discusses many of the topics in Baumeister's talk. The rate of rural suicide amongst men is a problem in rural Australia, and is often bought about by same-sex attraction, loneliness, and an inability or unwillingness to seek help due to social isolation. Contemporary feminism attempts to break down the stigma related to men seeking out help in this way, of the feeling that crying is for women, of the idea that you have to be with a woman to be a success.</comment>


Three-Domain System - <AUTHOR> https://en.wikipedia.org/wiki/Three-domain_system <comment><USER2> In elementary school I got taught about six-kingdom system, so I am surprised there's been an update in 1990. To which domain do viruses belong? Still an odd exception?

<comment><USER3> I was taught the three domain system, although this was a couple decades after the model. Viruses don't fit in because they're not alive.

<comment><USER4> According to Wikipedia[1]:

 _> Scientific opinions differ on whether viruses are a form of life, or organic structures that interact with living organisms._

And:

 _> Viruses are now recognised as ancient and as having origins that pre-date the divergence of life into the three domains._

[1]: [https://en.wikipedia.org/wiki/Virus](https://en.wikipedia.org/wiki/Virus)

<comment><USER5> I was taught that viruses are weird little machines, similar but unrelated to the things we recognise as alive. Interesting that there's difference of opinion on the matter.

Edit: here's a question for the room; if viruses are unrelated to the rest of "life", but are endogenous to the earth, are they alien?

<comment><USER6> It's an overstatement to say that viruses are "unrelated to the rest of life". They have RNA or even DNA, which almost certainly makes them related.

If we found a living organism which didn't have RNA or DNA, that would be a good candidate for an alien. It would help if we found it on another planet though.

<comment><USER5> Thanks for the clarification, that makes sense.

</comment></comment></comment></comment></comment></comment><comment><USER7> Kudos for prominent feature of the dissenting view.

I thought it was the mainstream. Didn't we establish that eukaryotes are descended from archaea that engulfed rickettsia or cyanobacteria, presumably after inventing nuclei? It seems presumptuous to claim a domain just for that, successful as it may have been.

</comment><comment><USER8> Ignorant to even the basics of biology, I happily traversed the wikipedia rabbit hole and learned a ton thanks to this link.

Thanks!

</comment><comment><USER3> The Wikipedia article is sadly quite light on details for a system that underpins modern taxonomy…

<comment><USER9> There's a more detailed article at:

[https://en.wikipedia.org/wiki/Kingdom_(biology)](https://en.wikipedia.org/wiki/Kingdom_\(biology\))

</comment></comment><comment><USER10> Seems pretty pointless when you take horizontal gene transfer into account.

<comment><USER11> I've always thought that as more of a mutation source rather than a more direct driver for evolution and therefore our artificial classification divides.

Mutations happen constantly: transcription error, radiation, and yes horizontal transfer, but many other factors decide if the mutation sticks or not. But gene transfer doesn't break a hierarchical classification system much more than solar radiation does.

</comment></comment><comment> ur-whale Is this classification arbitrary, or does it try to actually reflect some sort of genetic ancestry tree?

Also: does it have any kind of predictive power or does it just scratch the itch of putting things in little separate boxes for its own sake ?</comment>


Why didn't JavaScript adopt the OO model adopted by C++/Java when it was designed? - <AUTHOR> https://www.quora.com/Why-didnt-JavaScript-adopt-the-object-oriented-model-adopted-by-C++-Java-when-it-was-designed/answer/Brendan-Eich?share=1 <comment><AUTHOR> _" If JS didn't make it into Netscape 2, we'd be speaking VBScript."_

source: [https://twitter.com/BrendanEich/status/793685578491437056](https://twitter.com/BrendanEich/status/793685578491437056)</comment>


An Angled Mind – Why AngularJS is awesome - <AUTHOR> http://thunderboltlabs.com/posts/an-angled-mind

<comment><USER2> I like how he uses transitions in CSS for Angular but does it in code for jQuery to make the jQuery code look much more complicated. Why not `$('some- box').toggleClass('open')`

</comment><comment><AUTHOR> Totally true. I chose the example as I think a lot of people do it that way right now. Sure you can write pretty good code with jQuery. AngularJS just makes it a lot easier.</comment>


Steyer claims he divested from fossil fuels a decade, but it's more complicated - <AUTHOR> https://thehill.com/policy/energy-environment/478320-steyer-claims-he-divested-from-fossil-fuels-a-decade-ago-but-its <comment><USER2> Oh it gets better: Farallon Capital Management (the company he founded) had a major stake in Kinder Morgan which owned the Trans Mountain Pipeline. KMP owned Trans Mountain until 2018 when they sold it to the Canadian government. Why does this matter? Keystone XL competes to move oil and gas across North America and Steyer funded Bill McKibben' 350.org's protests of KXL.</comment>


How I Got a 2x Speedup With One Line of Code - <AUTHOR> http://www.naftaliharris.com/blog/2x-speedup-with-one-line-of-code/

<comment><USER2> I'm sad to see that the one line of code, with its magic number and comparatively inscrutible contents, is entirely undocumented. Please, if doing things like that: a good commit message is necessary but not sufficient. (Yes, `git blame` exists, but you're not so likely to use it as a comment in the code.) A blog post is handy but not sufficient. A good inline comment is invaluable when returning to the code later if the situation changes, as it typically will in time for such precise, empirically derived performance tweaks.

       // This single line boosts performance by a factor of around 2 on GCC.
       // The optimal lookahead distance i+3 was chosen by experimentation.
       // See http://www.naftaliharris.com/blog/2x-speedup-with-one-line-of-code/

<comment><USER3> Referencing a URL, especially a blog, that could die anytime in documentation isn't good.

<comment><USER2> I agree with your general sentiment and was initially inclined to make the comment more verbose and not include the URL; I would certainly not be content with _just_ a URL. Still, in this case I would consider it a fair compromise, primarily because the inclusion of the URL is non-essential but value-adding. That is, it explains more, but the two important points—the purpose of the code and the reason for the number 3—are captured by the lines above, but the blog post adds more information which could potentially be of use later. Furthermore, the author of the code is the author of the blog post, which increases the probability of its survival.

<comment><USER4> I'd it the detailed explanation into the commit message. Or you could add it to README or make a file NOTES and explain it there.

</comment></comment></comment></comment><comment><USER5> You can get another 2x speedup if you choose the pivot better. Right now it seems you're doing median of 3.

When you're doing Order Statistics (ie Selection/Quickselect) with 1M elements you should be very very close to the theoretical optimum (1.5N) for complexity. See my comment here:

[https://news.ycombinator.com/item?id=6629117](https://news.ycombinator.com/item?id=6629117)

If you're interested in choosing the best strategy.

HTH

<comment><AUTHOR> Neat! A quick skim of the referenced article in that comment ([http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6193457](http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6193457)) shows that it solves a problem I'd wondered about for a while now, about how to optimally choose the pivots when computing order stats. (Yeah, I'm just doing median of 3 right now). If you want to sort the whole list, then clearly you want the pivots to be as near to the median as possible. But if you just want an order stat, then that's not necessarily true; if you wanted, say, the 10th percentile, my gut feeling would be that you should aim for your first pivot to be roughly the 20th percentile or so. I'll read that article more closely and see if I can improve even more.

Thanks, that's really helpful!

</comment></comment><comment><USER6> Great article. But it seems somehow unfair to characterize this as a 'one-line change'. He had to run various performance experiments that will be invisible to someone else reading the code, and the optimal prefetch distance might silently shift over time, like say if each object grows larger. It's a little like the Carmack magic number in that respect: [https://en.wikipedia.org/wiki/Fast_inverse_square_root](https://en.wikipedia.org/wiki/Fast_inverse_square_root)

I've had this idea for some time that our _representation_ of programs is fundamentally broken, because it excludes a detailed characterization of the _input_ space. I took a first stab at a solution with [http://akkartik.name/post/tracing-tests](http://akkartik.name/post/tracing- tests), which lets me write all my unit tests at the top level, showing how each piece of functionality augments the space of inputs my program can address. See, for example, [https://github.com/akkartik/wart/blob/3039fe6d03/literate/02...](https://github.com/akkartik/wart/blob/3039fe6d03/literate/023tokenize#L83). I suspect reading other people's 'code' could be an order of magnitude easier and more convenient if 'code' included the right information. I'm still working to flesh this idea out.

<comment><USER7> >>It's a little like the Carmack magic number in that respect: [https://en.wikipedia.org/wiki/Fast_inverse_square_root](https://en.wikipedia.org/wiki/Fast_inverse_square_root)

Thanks that was a great read :D

</comment></comment><comment><USER8> > No way, substantial speedups can really only come from algorithm changes.

I've often found the reverse to be true; that implementation details matter most.

The best fixes tend to be altering my code so that the compiler/cpu can be smarter. In this case, the author hinted the cache. Other times, I've found that looping in a different order helps my cache hit rate. Other times, properly naming variables and not reusing the same name allows the compiler to do smarter things.

Other times, when I've tried a new algorithm, it's slower because of bad constants or I've accidentally messed up the implementation to negate all benefits.

As a complete side note, you need to be careful when using "older" algorithms, ie those published pre-2000. I once implemented, based on a Hoare paper, a doubly linked list for spare matrices, and it was significantly slower then the naive method, simply because CPU cache lines have gotten really good while being able to predict linked list memory locations had not.

<comment><USER9> This is all true, but I never encountered cache optimizations like that to cause more than a 2x difference on average and 5x in very rare extreme cases. This is very far from what you can get from algorithmic Big-O changes, where it is easy to get 1000x speedups.

<comment><USER8> That's true. Having been briefly in academia, I only evaluated algorithms based on Big-O and just a bit of practical experience taught me that there tends to be a sweet spot, where the o(n^2.7) is both much faster then your naive implementation, but simpler to code, better constants, and better cache lines then the o(n^2.7003).

</comment></comment></comment><comment><USER10> I don't think this gain came from pre-fetching anything. The compiler may simply have decided to treat the entire function, or even the entire file, differently because of that one line.

I compiled (x86, GCC 4.8) lazysort.c both with and without the prefetch intrinsic, and there were substantial differences in code surrounding calls in to Python, but no change at all to the guts of the partition or sorting routines.

Regardless of what is responsible, it's not good to leave something like this in your code without really understanding what it did. There may have something else that occurred incidentally from this change, that could have lead you to a better solution.

<comment><USER11> > I don't think this gain came from pre-fetching anything.

It may, and its possible that it doesn't show up on some cpus. see this thread for an example of how difficult it is to find a good generic prefetching size: [https://lkml.org/lkml/2013/10/11/534](https://lkml.org/lkml/2013/10/11/534)

</comment></comment><comment><USER12> There are only two really difficult problems in computer science: naming things, cache invalidation, and off-by-one errors.

<comment><USER13> [http://i.imgur.com/LfqyX3p.gif](http://i.imgur.com/LfqyX3p.gif)

<comment><USER14> I don't get it... that's a cellular automaton joke?

<comment><USER15> Getting the joke requires understanding the game of life and this meme

[http://knowyourmeme.com/memes/boardroom- suggestion](http://knowyourmeme.com/memes/boardroom-suggestion)

</comment></comment></comment></comment><comment><USER16> >> The moral of the story, at least for me, is that you can occasionally get substantial improvements by understanding what is actually happening under the hood of your application, rather than fundamentally changing your application itself.

Also known as the rule of leaky abstractions, as in all abstractions are leaky.

<comment><USER17> Is parametric polymorphism leaky?

<comment><USER16> Method resolution order.

<comment><USER17> I said _parametric_ polymorphism. That doesn't involve any method resolution order.

<comment><USER16> OK - different usage of "abstraction".

If you are talking about abstracting, i.e. taking away, similar bits of code into one place in programming, whether it's via polymorphism or refactoring functions, then no, that is not a "leaky abstraction".

I am talking about abstraction as in creating a simplified / unified interface over some complex thing. Abstracting the idea of something from its concrete realizations into its own concept. OSI networking layers, drivers for hardware, modeling real world systems, etc.

If you want to be pedantic instead of snappy, you can change my statement to "all abstraction layers are leaky".

</comment></comment></comment></comment></comment><comment><USER18> Note that `ob_item[i+3]` could read past the end of the array.

<comment><USER19> This jumped out at me too. GCC's documentation says:

Data prefetch does not generate faults if addr is invalid, but the address expression itself must be valid. For example, a prefetch of p->next does not fault if p->next is not a valid address, but evaluation faults if p is not a valid address.

This seems to make the code as presented incorrect (i.e. it results in undefined behavior).

</comment></comment><comment><USER20> I got a 1000 X speedup by removing a sleep(100000) line once. Not as impressive though.

<comment><USER21> I got a 2gb reduction in memory usage and orders of magnitude speedup by adding an _ in one line once.

<comment><USER22> Care to explain that? I don't get it.

<comment><USER21> It was a long involved bug chase, but some django middleware had an optional cache of some s3 information. The actual variable was self._something, with an @property on self.something so that the getter would backfill the metadata if it didn't exist.

In the code, it was checking if self.something existed, and then doing something else if it didn't. This triggered the lazy getter, and had the effect of pulling in the metadata for a 300k item s3 bucket.

Checking for self._something correctly returned None, it then checked s3 for the item, and uploaded it directly.

</comment></comment></comment></comment><comment><USER23> I would expect that that magic number 3 is related to the size of a cache line relative to the size of an array element and the time spent in each loop. You will want to prefetch the next cache line so that it is just available by the time you want to start processing it (and, ideally, not a moment earlier because that would push data from the cache that still could be used before the requested data is needed)

A robust implementation should figure out the size of a cache line and the load delay, and derive the magic offset from that.

</comment><comment><USER24> Er, are you aware of the C++ standard library's std::nth_element() function? (Even my "pure C" programs tend to be compiled with g++ to gain access to that and std::sort().)

<comment><USER10> Yes, this is exactly what the author is trying to port to Python.

Mandatory reference link: [http://en.cppreference.com/w/cpp/algorithm/nth_element](http://en.cppreference.com/w/cpp/algorithm/nth_element) . Note linear complexity.

There's also std::partial_sort:

[http://en.cppreference.com/w/cpp/algorithm/partial_sort](http://en.cppreference.com/w/cpp/algorithm/partial_sort)

</comment></comment><comment><USER25> _Now, I didn 't just "know" to prefetch the PyObject three indices forward in the array. I arrived at this value through experimentation._

There is a good chance that 3 is specific to the hardware you tested on. Different systems will have different memory latencies (and NUMA systems can make it even more complicated). It isn't likely that 3 will turn into a pathological case on any other hardware, but it may well end up being less optimal than say 2 or 4.

</comment><comment><USER26> This sort of thing is why porting naive x86 code to the GPU programming model (scalar programs aka 'kernels' streamed through on the device using a configuration) often results in amazing 15-20x speedup over 6 core, while the expected speedup is only 5-7x. x86 compilers often don't get the pre-fetching right in loops, only using their heuristics, so you have to talk to them. Then you you begin to doubt the compiler more and more, carving out a few percentage points here and there by adding vector intrinsics, loop unrolling and so on. And before you notice, your codebase is now bigger, more complex and more error prone than if you had ported it to GPU from the beginning.

Please note - of course this insight doesn't apply in your case since your algorithm is meant for general purpose usage, not HPC programming meant for systems where you have the hardware under your control. I'm simply urging people where the latter applies to think about whether their algorithms might be suitable for GPU instead - doing lots of prefetch instructions and loop unrolls is a sign of that.

</comment><comment><USER27> Nice article.

One thing that wasn't clear to me in your explanation is that you're sorting a list of pointers to objects. To sort that list you're comparing two of the objects themselves.

        // follows pointer to access *ob_item[i]
        IF_LESS_THAN(ob_item[i], pivot)

So the point of __builtin_prefetch is to deference this pointer in advance and so avoid the latency of the 12 million reads scattered all over memory. Nice.

Another useful thing to do here is to see if you can find a "streaming operator" to dismiss *ob_item[i] and ob_item[i] from cache after the comparison. They won't be needed again for a while.

Another good article on this optimisation (mentioned by OP)

[http://scripts.mit.edu/~birge/blog/accelerating-code- using-g...](http://scripts.mit.edu/~birge/blog/accelerating-code-using-gccs- prefetch-extension/)

</comment><comment><USER28> This is a great post. I guess the idea here is that you can start bringing future objects to be compared up through the cache hierarchy while you are doing the comparison on the current object. If that's the case, then I think the speedup from prefetching will depend on the speed of the comparison which in turn will depend on the type of objects in the collection.

In your post it says you have a collection of 10MM elements but you didn't say what they were. Are they just ints or something else?

<comment><AUTHOR> Thanks!

Oh, thanks for pointing out the oversight! They were floats generated with random.random() and then shuffled with random.shuffle(.) Shuffling them removes any locality that might have arisen when the floats were being generated, even though the distribution remains uniform on (0, 1). (And indeed, empirically the code runs faster if you don't shuffle them).

Yeah, I'm not sure how well this would work if you were sorting different objects. I would guess that it depends on two things--how large the objects are in memory, and how long it takes to compare them.

<comment><USER29> I was thinking about that when I read the article too. How you've obviously done a lot of testing, but that the run-time's allocator algorithm is going to have an effect on this, and how allocating N floats in a loop will have different performance than if the floats were allocated over time (with other allocations mixed in) or if the objects were larger. Especially if the comparison function requires accessing object members from other memory locations... The magic value 3 might have a lot to do with the number of objects that fit in the cache or a page.

Anyway, it's an interesting article and an interesting library.

<comment><USER30> The prefetch will always take same amount of data no matter what the size of python object in question is. That leads two results. 1) You can miss if your cacheline size is less than python object size. 2) You miss if your python object is not alligned at the cacheline and comparison is with the part that is in the next cacheline side.

If these two condition apply then there is limit of 2 on speed up and increasing python object size reduces the effect of prefetch which still helps.

Another about prefetch distance I'd pick 5 based on his testing data. 1) It has no downside on current tests. 2) Due to prefetch constant size any prefetch that hurts because you pick distance of 5 instead of 3 is unlikely to have any meaningfull effect at any prefetch distance. 3) It allows some improvement in comparison speed. Like if there where JIT on python side to specialize the test. 4) It allows a nice factor of improvement on CORE execution speed relative to memory latency. 5) It allows you to handle situations of page misses better.

<comment><USER29> The prefetch may take the same amount of time, but the number of objects that it pulls into cache depends on their size and allocation pattern. If they are large or were allocated over time (= from different memory blocks) fewer will be pulled in. It seems to me that this would make prefetch less effective. That is, if the thing you are trying to prefetch was pulled in because it was close enough to the last one, prefetch is a no-op.

</comment></comment></comment></comment></comment><comment><USER31> Funny, I just experimented with __builtin_prefetch this week, and got no speedup. Does anyone know whether the kernel list.h always triggers hardware prefetching when going over a list?

BTW, my case wasn't the problem the kernel maintainers encountered with small lists, detailed here:

[http://lwn.net/Articles/444336/](http://lwn.net/Articles/444336/)

</comment><comment><USER32> I'm pretty sure you could have saved yourself a lot of work by just using profile guided optimisation (PGO). It's mature & ready to go in GCC at least.

(Integrating it into you build process is a little bit more challenging, I admit. I've set it up in a large dev environment used by multiple large projects, and it was well, well worth the effort.)

</comment><comment><USER33> its one of those things... micro optimisation becomes the big one once you do enough algorithm optimisation. even the humble memory copy becomes faster from prefetching and clever use of registers... and actually making the algorithm theoretically worse. :P

also, note that it is possible to remove the magic from your magic number with some thought about latency, the size and alignment of your data etc. fetching a cache line takes a fairly predictable amount of cycles.

opposite to the prefetch is the non temporal read/writes where you don't pollute cache to prevent big one off copies from damaging the performance of other code...

</comment><comment><USER34> Interesting post! I wonder if the author experimented with different gcc command-line optimisation options as well? gcc _might_ be able to insert some prefetches by itself with some settings?

<comment><AUTHOR> Thanks! I tried -fprefetch-loop-arrays before figuring out that it only prefetches the array memory, (ie, memory with addresses like &ob_item[i]), rather than the memory referenced by the array memory, (ie, memory with addresses like ob_item[i]). And empirically adding -fprefetch-loop-arrays got me no speedups.

I didn't try any other GCC command line arguments because I honestly didn't know what other relevant options there were.

</comment></comment><comment><USER35> Came expecting [http://thedailywtf.com/Articles/The-Speedup- Loop.aspx](http://thedailywtf.com/Articles/The-Speedup-Loop.aspx), was disappointed...

</comment><comment><USER33> also, read this: [http://www.gamedev.net/page/resources/_/technical/graphics-p...](http://www.gamedev.net/page/resources/_/technical/graphics- programming-and-theory/graphics-programming-black-book-r1698)

</comment><comment><USER36> I get at least that good with

        DEFINT A-Z</comment>


Fake Apple Keynote "Leaked" on Wikipedia - <AUTHOR> http://www.techcrunch.com/2008/01/14/fake-apple-keynote-leaked-on-wikipedia/ <comment><USER2> This is obviously fake, so why are people making such a big deal about it? Waste of time, don't bother.</comment>


Julia 0.5 release - <AUTHOR> https://github.com/JuliaLang/julia/blob/master/NEWS.md#julia-v050-release-notes <comment><USER3> A better link is: [https://github.com/JuliaLang/julia/blob/release-0.5/NEWS.md](https://github.com/JuliaLang/julia/blob/release-0.5/NEWS.md)

Better yet, wait for the official announcement once binaries are compiled and posted for download. I believe there will be a blog post to go along with it, too.

<comment><USER4> Yes, there'll be an official announcement in a little bit. Tagging is just the first step (we haven't even added the release metadata to the tag on GitHub yet).

</comment></comment><comment><USER5> Excellent news! I look forward to reading over the release notes.

Edit: this is a game changer for me:

> Support for multi-threading. Loops with independent iterations can be easily > parallelized with the Threads.@threads macro.

</comment><comment><USER6> This could either be brilliant or a total nightmare:

"Support for arrays with indexing starting at values different from 1. The array types are expected to be defined in packages, but now Julia provides an API for writing generic algorithms for arbitrary indexing schemes (#16260)."

Originally I thought total nightmare but now I'm not sure.

<comment><USER7> Pascal and Ada have had arbitrary indexing for decades. It's been fine.

<comment><USER8> And Fortran, where it works fine. Perl lets you change the array index offset but everyone warns you that your computer will reach out and slap you in the face if you do. As is often the case, the devil is in the details.

<comment><USER7> In perl is it APL style where it changes the index for the entire world (until changed again) or is it Fortran/Pascal style where you are expected to declare the lowest/highest index per array?

<comment><USER9> APL-style with a global index origin ($[ in Perl, à la ⎕IO in most APLs).

</comment></comment></comment></comment></comment><comment><USER10> Is Julia 1.0 still scheduled for approximately Summer/Fall 2017?

I really like/liked Julia, but some things breaking between 0.2 and 0.4 made me use it a bit less.

<comment><USER11> I used Julia a bit in the same time span, and I was frustrated with the breakage, but I also saw a lot of things that clearly needed breaking changes. I'm glad they're happening.

Until now, [a, b] meant "concatenate a and b if they're arrays, make a 2-element array otherwise". This struck me as both annoyingly inconsistent and as a failure to think recursively.

Now it's always a 2-element array, possibly an array of arrays. And that's a huge change in syntax, and it's for the better. I look at many things in the 0.5 release notes and they're fixes to specific pain points I had. This has me paying attention to the language again.

</comment></comment><comment><USER12> Arguably, it is better numpy based on multimethods, but it is far from being polished as Python 3.5+ On the contrary it suffers from the kitchen sink syndrome, with process of continuous adding of stuff instead of continuous clarification and refinement, which characterizes a good Python 3 language.))

But there is a lot of enthusiasm.

<comment><USER13> I think that you are missing the big picture.

To you it is a better numpy/python, to me it is a faster/cleaner/more parallelizable version of R, to some people it is a cleaner/faster version of Matlab.

This could be seen as a "kitchen sink" approach but it also is very useful when you want to get things done. My gripes about Julia are really minor considering how young and ambitious the project is.

<comment><USER12> Yes, you are right. It is a better R. I haven't thought of R because I consider it as being nothing special in terms of design of programming languages.

</comment></comment></comment><comment><USER14> Are there large speed improvements from the previous version?

<comment><USER3> Yes, in many areas:

* Higher order functions now specialize on (and possibly even inline!) passed functions

* Anonymous functions are now fast, too

* Fused broadcasting can avoid intermediate allocations and only make one pass through the array

* User-extensible bounds checks allow custom array types to opt-in to skipping bounds checking, enabling SIMD-ification of some for loops

That said, compilation times may take a bit longer due to the LLVM upgrade… but this resulted in an even stronger push towards better performance in many other areas.

</comment></comment><comment><USER15> A feature request, Anaconda partners with Intel and includes MKL default. Could you do same for Julia and add MKL default at least in version 1.0 so no manual work needed?

<comment><USER16> See [https://github.com/JuliaLang/julia/issues/18374#issuecomment...](https://github.com/JuliaLang/julia/issues/18374#issuecomment-245108175). Open-source Julia builds also need to support non-Intel processors and architectures, and operating systems where MKL isn't available.

If you have a specific operation where MKL is noticeably faster than OpenBLAS, report it. OpenBLAS can always be improved if they have specific workloads to target.

<comment><USER15> Thanks for the comment, I understand why it is not option.By the way I do not understand why I received downvotes.

<comment><USER16> Probably because a hacker news comment isn't the best place for a feature request about using a proprietary library and comparing to a commercial distribution of a different language.

By the way, Julia partners with Intel as well, but on things like adding multithreading support to the language, and developing auto-parallelizing compilers ([https://github.com/IntelLabs/ParallelAccelerator.jl](https://github.com/IntelLabs/ParallelAccelerator.jl)). Improved support for building Julia with MKL and Intel's C and Fortran compilers is something that Intel would probably like to collaborate with us on as well, but not much has happened there officially yet (these are different groups within Intel).

<comment><USER15> Keep up the good work Julia team.

</comment></comment></comment></comment></comment><comment><USER17> I haven't thus far been interested in Julia. Unless you're into high-level math, it didn't seem to provide much value, and it did weird stuff with arrays, and wasn't Lua, which gets a free pass for being amazingly well designed in all other respects (arguably, it was well designed in that one as well, but it makes all the array math a pain).

I don't know, maybe it's great. Maybe I should reconsider. But then again, it's strongly typed, which isn't usually my sort of thing (I'm not working with a team, and my programs haven't devolved into chaos yet, so with no empirical data either way, I'll take my favorite)

<comment><USER18> I'm not sure what you mean by "high-level math". It isn't a symbolic language like Mathematica for mathematics research but something along the lines of R for data analysis. The strong typing isn't just there for discipline, but efficiency in execution.

<comment><USER17> Yeah. But it's definitely optimized for mathematical usage.

<comment><USER19> Since programming is fundamentally mathematics (grin) that means it's a "general purpose language". There's nothing whatsoever preventing it from being a first-class string manipulation language for instance.

Granted, the emphasis has been on applied mathematics. I think in the longer run it will branch in several directions, one being realtime simulation - <USER2> then on to games. It is close to C++ efficiency (plus or minus) already, and it will only get better.

I haven't looked into Julia's GC implementation in detail, but I'm hopeful that with a little effort it's easy to avoid "stop the world" GC collections.

<comment><USER20> That old saw is getting dull. Because something can be described by mathematics, does not make it mathematics. Else the whole universe would be mathematics.

Programming is the art of getting a particular machine to do something you want. You may use math to get there (like in accounting or swimming) but its fundamentally a task that's performed by skilled people.</comment></comment></comment></comment></comment>