Scientists say dolphins should be treated as 'non-human persons'  - <AUTHOR> http://current.com/news/91825903_scientists-say-dolphins-should-be-treated-as-non-human-persons.htm <comment><USER2> Hell, yes they should. We've known they can identify themselves in a mirror for years. They are even known to sexually pleasure each other outside of procreation.

(If I recall the study correctly, the scientists introduced the dolphins to a mirror, and then placed an object on the dolphin's back, out of its sight, with suction cups; the dolphin swam to the mirror and contorted to view the object.)

We should really stop poisoning their habitat. It didn't use to be the case that most fish had such high levels of mercury.

 _Edit_ Wow, this comment just went from +7 to +4 in a minute… I guess some people feel very threatened by the idea that humans are overly dismissive of other species.

<comment><USER3> What about an art test: What other species makes art for the sake of art?

<comment><USER2> Is your goal to create a test that only humans pass because we already know humans are the only worthwhile species? Sort of begging the question, isn't it? (Edit: I don't mean any offense — just suspicious of human heuristics.)

<comment><USER4> No offense, but this comment is awfully condescending considering the interesting question it's in response to.

I think Alan's claim of the motive of the original question is unfounded.

*edit: reworded for clarity

<comment><USER5> I don't see how it's condescending. Either you're seeing something I'm not or you're mistaking a rhetorical device for condescension. The idea of art being some kind of moral threshold does seem very arbitrary.

Edit: Incidentally, I'm not one of the people who downvoted you.

<comment><USER4> I would be totally with you if lowglow's question was:

"what about a long division test?" or something similarly snarky.

but art is an interesting cultural phenomenon that very likely occurs in some form in many animals. Alan's supposition that there was some bad intent in posing the question is absurd here, in my opinion.

<comment><USER5> Art is an interesting cultural phenomenon, but not something that even all humans are very interested in (unless you define it very loosely). I wouldn't be surprised to see animals produce some form of art, but neither would I surprised to see a species as intelligent as humans — or even more intelligent — that didn't give a hoot about art. Let's reframe the validity of this test in terms that we don't have thousands of years of cultural baggage around: If an alien species arrives here and teaches us all sorts of miraculous things, would we feel compelled to conclude that they are not intelligent beings if they aren't very interested in the works of Rothko?

It seems particularly likely to me that an aquatic species would not have art at least until it was very advanced, since they don't have the tools or media for creating it that, say, a caveman did.

<comment><USER4> art is not what you learn in "art class"

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I watched the excellent documentary "The Cove" a few weeks ago, and I highly recommend watching it. Though the film did fill me with dread and outrage (as intended), ironically, the portrayal of the brutal slaughter of dolphins by Japanese fishermen convinced me that dolphins are not nearly as smart as many people would like to convince us of.

Here's why: Imagine there were one particular mountain valley tucked away in some corner of the world, and while human beings could range all over the vastness of this earth unharmed, if they ventured into that valley they would be killed. In that valley there was a race of essentially invincible beings that would herd any wayward human beings into a side canyon and spear them to death.

How would the human race react to that? Let's assume that mounting a military attack and killing the invincible beings weren't possible. Well, first of all, we'd put up signs and fences and maybe even sentries to warn people not to go into that valley. We'd tell our children about the danger, and pass the knowledge of the invincible murderous monsters on, generation to generation. If people did go in there and got killed we'd cover the event in detail on TV, as a warning to others (like Shark Week). Even really stupid people would know not to go into the "valley of the stabby monsters."

Well, if Dolphins are so smart, and they have such good communication and advanced social structure, then why haven't the many dolphins that have personally witnessed the slaughter at "The Cove" and escaped (the movie shows dolphins escaping) managed to warn other dolphins not to go there?

Honestly, if dolphins were anywhere near as smart as humans and were anywhere near as ruthless, they not only would have prevented the slaughter, but would have mounted a commando raid wherein expert jumping dolphins would have trained to jump out of the water, knock the fishermen out of their little boats, then beaten and dragged them down into the cove until they were drowned. And I'm being serious about this. Human beings wouldn't put up with that shit.

Other observed dolphin behavior, such as dolphins killing each other and attacking sharks to defend swimming humans does prove that they have the wherewithal to act in both self-preservation, and in the defense of others, even to violence. But they still get speared in the cove. That tells me they're not that smart.

<comment><USER7> Lack of collective memory does diminish an intelligence rating.

Humans didn't have recorded history for tens of thousands of years (maybe a million depending what science you follow, frontal lobes and all that) and were doomed to repeat the same mistakes (heck we still do the same massive mistakes with lots of recorded history).

<comment><USER8> Dolphins don't have hands to write with. Nor did we invent a communication system by which dolphins and humans can communicate ideas.

<comment><USER7> Humans have had spoken-word history long before written. Native Americans for example. Dolphins may have something similar, perhaps a "show and tell" of sorts, who knows, but I admit it's obviously not as evolved if they can't avoid an annual slaughter.

Another extremely intelligent creature is the octopus, I wouldn't mind them having some protection as well. But they aren't as "sexy" as dolphins so I don't expect that to be forthcoming any century soon.

<comment><USER9> I'd guess communication methods available to dolphins aren't as good as those available to humans. I think all aural communication that is available to dolphins is available to humans, and yet we use mostly speech and gesticulation, both of which seem to be either unavailable or very limited for dolphins.

As for the octopus, by quickly browsing the usual suspects you can find that quite a lot of people find them much more sexy than dolphins, but let's not get to deep into that.

<comment><USER10> While dolphins lack the communication methods available to humans, I believe that they have the same methods available to them that whales possess. Whales have a pretty sophisticated "culture" that is encapsulated in the variying songs of different species and among the various geographic and tribal groupings within a particular species.

<comment><USER9> Yes, I know about that. I just doubt that it's as effective at expressing precise meanings as methods available for humans ("area within 100km radius from 33.593316 135.943022 is to be considered unsafe," and let's not get started on bordering the area with 3 meter high fences topped with barbed wire).

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I've always been a little uneasy with the idea that we should assign rights (or personhood, or whatever) to animals in a graded way in proportion to their intelligence. It implies that humans should be deprived of those same rights when their intelligence doesn't meet some similar threshold.

<comment><USER2> Emotional capacity is another axis that matters a lot.

But we _do_ deprive people of rights if they are really lacking, mentally. We put crazies in madhouses (are bad at doing so accurately). We deny minors most rights of self-determination (more than strictly based on intelligence, but the idea's the same).

Heck, to sign your will, you have to represent that your noggin isn't fried.

 _Edit (response to pyre):_ I never said we treat unintelligent people the same way we do animals. Psychologically we don't even like to disrespect _dead_ people (intelligence: 0). I'm just saying that intelligence does affect the rights we afford other humans.

<comment><USER11> I absolutely agree that emotional processing is an important component of intelligence. But I'm still not convinced that it's a good way of ascribing rights.

Regarding your examples: we certainly do act in the interests of the insane, mentally disabled and children. But that is not because they have fewer rights or have less personhood. We act for them _because and in defense of_ their rights. We do this because they are unable to protect themselves and need guardians to defend them from abuses to their dignity, bodily integrity, etc.

<comment><USER12> Up to a point. You can't vote on behalf of an incompetent relative, for example. Some rights are not delegable.

</comment></comment></comment></comment><comment><USER13> Science is all about how things work, not how we should act.

 _The researchers argue that their work shows it is morally unacceptable..._

See, this is what we get when we confuse science and activism. Should a "researcher" have any more weight with the public about what they consider "morally acceptable"?

I think there is some meat to this story: we need a more nuanced scale of what is morally acceptable or not that includes various forms of intelligence (without the same logic being used to treat disabled people poorly) I can't emphasize enough how important these issues are. Dolphins are obviously highly intelligent creatures. But such a discussion needs to happen from a neutral and reasonable standpoint. Asking folks who are deeply in love with their subject to tell us what's morally acceptable or not is whacked.

Doctors shouldn't treat themselves, lawyers shouldn't have themselves as clients, accountants shouldn't audit their own books, programmers shouldn't peer-review their own code -- and scientists shouldn't advocate on the topics they are deeply attached to. This is just common sense.

EDIT: Just to be clear, there are at least three problems with scientists-as- activist.

1) The public gets to make the decision on these issues. This means that any communication from scientists has to have as its goal informing the public, not persuading it. In an article like this, it is not clear to the public where the discussion of science ends and where the discussion of advocacy begins. This confusion, aside from being unethical in any profession, is a poor use of the public's resources.

2) The end result of such advocacy has a direct impact on the scientist's lives, creating a moral hazard. This is like when your chiropractor tells you that you have a bad back -- and need many more chiropractor visits. Maybe true, maybe not, but when a professional judgment can also be self-serving it creates mixed incentives and doubt, which is bad for any group of professionals

3) Morality is not science: you cannot add one and one and get "moral" or "immoral". When a conclusion like this is put forth by a scientist, it glosses over all sorts of other questions which other professionals are interested in, like when life begins, or what constitutes sentience. To make a statement about morality is -- at the very least -- to jump into areas for which you are not trained. As a professional, once again, it's your job to make it clear which things you speak about are part of your work and part of your opinions. The most important thing a scientist can say is "I don't know"

That's not even getting into the entire issue of just being too close to something to make a good call. Or -- if you think such behavior is okay -- the use of scientists as pawns to have proxy battles over other larger political causes.

These are not new standards. Other professions have been living inside of them for years. Ask your accounting consulting firm whether it's moral for you to invest in Indonesia and you'll get a very careful reply which will boil down to "I am not the person to ask that question of"

<comment><USER14> _Doctors shouldn't treat themselves, lawyers shouldn't have themselves as clients_

Why not? Is there a law against that? I'm genuinely curious.

 _and scientists shouldn't advocate on the topics they are deeply attached to_

That's a non-sequitur analogy.

Besides, who do you propose to advocate a position if not those who have learned the most about it? Shall we forbid Dawkins from advocating evolution? Economists (it's a science!) from advocating economic policy?

Should we forbid programmers from advocating software patent reform because "it's a topic they are deeply attached to"?

<comment><USER15> > Shall we forbid Dawkins from advocating evolution?

Yes, please, I'm begging you. Dawkins has made himself the _WORST_ person to be advocating evolution because he's turned it from a rational scientific argument into the moron-capades by arguing about the existence of god.

To put it simply, he's trying to get his finger out of a Chinese-finger-trap by pulling, which makes him a moron because every 5-year old will tell you to give up and push your fingers together because its... you guessed it... _POINTLESS_ , just like arguing about an unresolvable argument like whether god does or doesn't exist.

Does a tree falling in the forest make a noise? Yes, it's called _science_ lets invite Dawkins to talk about it again.

Why shouldn't a doctor treat themselves? Because it's very hard to do open heart surgery on yourself!

Edit: > Should we forbid programmers from advocating software patent reform because "it's a topic they are deeply attached to"?

I thought the conventional wisdom with copyright reform was that the MPAA, RIAA, etc. should be forbidden from advocating because "it's a topic they are deeply attached to" ... you know all those hundred-dollar bills don't roll themselves up into big fat wads and slip into Congressmen's pockets for nothing.

<comment><USER16> There is a war on evolution by the religious right. How is one to try to take that on without talking about God?

<comment><USER17> Because the existence or nonexistence of god has nothing whatsoever to do with evolution?

<comment><USER18> Depends which god you're talking about. A god that created the earth and everything on it in seven days some number of years ago is in direct conflict with observable evidence. The generic "idea of god", where it may be possible that some being with powers that shouldn't be possible according to our understanding of physics exists somewhere in the universe, does not conflict with evolution. If you don't think evidence is powerful enough to let us infer the existence or non-existence of something, well...

</comment></comment></comment></comment></comment></comment><comment><USER19> Does intelligence somehow determine the significance and importance of pain? Is it less real?

</comment><comment><USER20> This is solely my opinion, and I could be (and would love to be proven) completely wrong, but as I understand it we as a race are the most incredible predators on the planet. There is no animal on this earth that could survive a full onslaught by a group of humans. We win, put simply.

Now for dolphins to turn around and say so long and thanks for all the fish is a great thing. I like dolphins. There are people here who think eugenics is a good thing. What will dolphins do for them?

Sadly, the eugenics side will probably live. As long as what we produce is on the scale of Dr Julian Bashir I only have a minor problem. If we're producing Khan Noonien Singh then I have a major problem.

Going back to the problem, I believe that dolphins are great creatures but not on the same terms as us. I'm sorry if you view it differently. I think Dolphins are great in their own right, therefore don't need our help to become intelligent but should have a supporter definately.

</comment><comment><USER21> If you consider intelligence in terms of IQ tests, they're pretty smart. If you consider it in terms of optimizing power, the ability to shape the future to their goals, a quadriplegic low-IQ human with a eyeblink controlled typing screen obliterates them. I would consider them pre-people at most, ripe for uplift.

<comment><USER22> I wonder how brain-computer interfaces and neuroprosthetics might change this in the upcoming years.

Seaquest was right!

</comment></comment><comment><USER23> If dolphins became legal persons then how would you take one to court? If one ever broke into your house or something.

</comment><comment><USER24> I'm all for the humane treatment of animals. Humans are animals. But doesn't "person" imply "human"? Does a human need to be intelligent to be a person? what about someone with extensive brain damage, wouldn't (s)he still be a person? Why muddle definitions?

<comment><USER25> It's a very interesting philosophical question about what implies a person and if such a thing can be lost, and when personhood is gained. It was once very easy to lose personhood, for example being unable to pay your debts, or losing in combat. It seems that for the present personhood is gained at birth and is never lost.

For better or worse, corporations are also treated as persons.

</comment></comment><comment><USER26> Although science can help us know the facts before we make our decision, how we should treat dolphins is not a scientific question but an ethical and moral one.

<comment><USER27> " how we should treat dolphins is not a scientific question but an ethical and moral one."

How did you decide that science and the scientific process should be excluded from ethics and morality?

<comment><USER28> because science is about objective facts. facts are objective because they can be verified by different minds if they use the same processes.

ethics and morality are EMOTIONAL, vary widely between individuals, and must be arbitrarily decided upon by an entity representing tribal leadership.

</comment></comment></comment><comment><USER29> And give them voting rights.</comment>


Turbocharging Walmart.com – FrontEndPerf [video] - <AUTHOR> https://www.youtube.com/watch?v=vB8JUx9Dp08 <comment><AUTHOR> 7 techniques that can help make your site faster without having to do big rewrites or compromising on existing features.

Slide Deck: [https://speakerdeck.com/vasa/turbocharging-walmart-dot- com-s...](https://speakerdeck.com/vasa/turbocharging-walmart-dot-com-speed- without-compromise)</comment>


Apple's Strategy to Dominate Audio - <AUTHOR> https://medium.com/functionsfund/functions-c292b619a0c3 <comment><USER2> they may do ok in the streaming side of things, they may even beat spotify (however i highly doubt ill ever pay for apple music) the problem is with the hardware, those airpods sound like shit and i kind of doubt that the speaker is really any good either.

sure the older ipods sounded OK but apples current audio hardware is lackluster at best, the headphones have always been dreadful and after they ditched the wolfson dac the devices sounded pretty crappy too.

if they want this to work they should be providing premium grade hardware, especially for the prices they charge, you can buy some really good IEMs for the price of a pair of airpods and theyll sound infinitely better, so why buy apples horrible little buds? maybe they should buy out a decent audio outfit and use their tech and skills to improve the shockingly poor sound their current gear kicks out.</comment>


Twitter: Our investigation is still ongoing but here's what we know so far - <AUTHOR> https://twitter.com/TwitterSupport/status/1283591844962750464 <comment><USER2> This thread and the one about the Vice article at [https://news.ycombinator.com/item?id=23853786](https://news.ycombinator.com/item?id=23853786) are close enough that I guess we should merge them.

The main thread about the hack is [https://news.ycombinator.com/item?id=23851275](https://news.ycombinator.com/item?id=23851275).

<comment><USER3> Neither has any meaningful details on the penetration itself...

> We detected what we believe to be a coordinated social engineering attack by > people who successfully targeted some of our employees with access to > internal systems and tools.

...this could mean anything. Seems like they simply want to portray themselves as the victims, even though it's very possible one of their own employees was involved.

</comment></comment><comment><USER4> Am I the only one confused? The "headline" offers no information, other than there is an ongoing information. I click on the title and I see two tweets. One says people can tweet again, the other is essentially a repeat of the game. And that is it. Am I supposed to make up my own story now? Or read a bunch of random tweets to figure out what these two tweets are about? Like how is this site usable for anything? Reading the comments here, it seems someone hacked some twitter Asia , but that is all I've gleaned so far.

<comment><USER5> they made the completely user-unfriendly decision that when you're signed out at least, linked tweets you land on do not unfold the replies below, but rather some facebook-tier 'you may like' type unrelated garbage. You have to click the tweet again, to go to the same url (!) but with the normal display

</comment></comment><comment><USER6> > Once we became aware of the incident, we immediately locked down the > affected accounts and removed Tweets posted by the attackers.

@elonmusk was posting and deleting the scam tweets for hours. In what world is that "immediate"?

<comment><USER7> In the world where they're trying to mitigate PR outrage and legal fallout.

</comment></comment><comment><USER8> Shameless plug: All the companies(Google, Microsoft...) are telling trust us. But, I believe that we should trust us instead of relying on third parties. They always change when businesses interest changes. This is where web3 is coming to play. Technologies like IFFS, safe network are coming. Looking at the scale issue, I guess this web3 takes at least 5 more years. But, this kind p2p technology is possible with small-scaled mesh. Mesh networks within our devices or families. From the beginning, I hate the idea of storing passwords in the third-party password manager. Later, I fell into the same trap because a managing lot of passwords is difficult. So, I building an open-source p2p password manger. Replicates the passwords within your devices, instead of storing everything at the vendor's cloud. It's half-way for the closed beta release. I would like to hear everyone's feedback on this idea.

Thanks</comment>


If We Believe in Dark Matter, Why Not Extraterrestrial Life? - <AUTHOR> http://m.nautil.us/issue/79/catalysts/if-we-believe-in-dark-matter-why-not-extraterrestrial-life <comment><USER2> We have an accurate model for gravity. It fails in galaxies, so the conclusion is that there is something else and we call it "Dark Matter". An alternative is to change the model of gravity, but the attempts so far have been unsuccessful.

We don't have a good model for life. We have only one example and don't understand all the details. The Drake equation is not a model, it's just a multiplication of unknown coefficients.

I think that bacteria-like/arquea-like life in other planets outside the star system is very very probable, almost inevitable in viable planets. Prokaryote- like life is more difficult. But nobody is sure.

Anyway, the main article is about the claim that the 'Oumuamua asteroid in a solar sail. The solar sails are the main research field of the interviewee and he trid to hijack the events and claim that the asteroid is a solar sail, but nobody believed him.

The asteroid has a tiny deviation in the orbit, that is measurable and nobody has explained yet convincingly. But there is not enough evidence to support not only that there is intelligent life in other star system and also that they send a solar sail here.</comment>


Google Trends: Common Lisp - <AUTHOR> http://www.google.com/trends?q=common+lisp&ctab=0&geo=all&date=all&sort=0

<comment><USER2> People get excited about CL, so they use Google to search for some tutorials. Then they notice all the parentheses and quickly abandon it.

On a more serious note, i'm quite interested in hearing a realistic explanation.

</comment><comment><USER3> Ah, it's Google Trends again, the world's most famous chart with no units.

They should put a link to _How to Lie With Statistics_ on every page.

<comment><USER4> Well at least we can compare ist to other things, like c++: [http://www.google.com/trends?q=lisp%2C+c%2B%2B&ctab=0...](http://www.google.com/trends?q=lisp%2C+c%2B%2B&ctab=0&geo=all&date=all&sort=1) Sadly lisp is in a steady decline now, just like C++. Are the old languages dying?

<comment><USER5> The obligatory quote is "Lisp doesn't look any deader than usual to me" (David Thornley).

</comment></comment></comment><comment><USER6> By way of comparison, it's got similar search volume as Algol: [http://www.google.com/trends?q=common+lisp%2C+algol&ctab...](http://www.google.com/trends?q=common+lisp%2C+algol&ctab=0&geo=all&date=all&sort=1) . It loses to Erlang and OCaml: [http://www.google.com/trends?q=common+lisp%2C+erlang%2C+ocam...](http://www.google.com/trends?q=common+lisp%2C+erlang%2C+ocaml&ctab=0&geo=all&date=all&sort=0) , and Clojure: [http://www.google.com/trends?q=common+lisp%2C+clojure&ct...](http://www.google.com/trends?q=common+lisp%2C+clojure&ctab=0&geo=all&date=all&sort=0) . It's well below Haskell: [http://www.google.com/trends?q=common+lisp%2C+haskell&ct...](http://www.google.com/trends?q=common+lisp%2C+haskell&ctab=0&geo=all&date=all&sort=1) .

</comment><comment><AUTHOR> Interesting peaks.

<comment><USER7> Unfortunately I would guess the peaks are simply due to the incredibly low search volume (I say this as the owner of the site that's the second search result for "common lisp documentation").</comment></comment>


How do I choose ML model? - <AUTHOR> I have data and a have a problem. What are the best practices when deciding on what type of model to use for a specific problem? Flowcharts? Any no-brainers? Trial and error? <comment><USER2> Microsoft provides a few hints how to choose a ML algorithm depending on the prediction problem (regressio, classification, etc).

[http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-4...](http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft- machine-learning-algorithm-cheat-sheet-v6.pdf)

<comment><AUTHOR> Very nice cheat sheet!</comment></comment>


Small Brainf*ck Interpreter in Scala - <AUTHOR> http://dindresto.tumblr.com/post/40418904638/small-bf-interpreter-in-scala

<comment><USER2> As any other Brainfuck interpreter, it shows that Brainfuck is a nice, simple, small language. Things become mildly more interesting when you write a transpiler, lets say to Scala.

See here for a sketch of a "Brainfuck IDE" that compiles to JavaScript (just click "load", then "compile"): <http://rolux.org/brainfuck/>

<comment><AUTHOR> Looks pretty cool, I think I'll try doing something like that!

<comment><USER2> As you can see in the example above, there are a few features that can make a Brainfuck developer's life less miserable, like source formatting, error handling or logging.

</comment></comment></comment><comment><USER3> Sorry, but I must be missing what's so interesting.

Great, you wrote a Brainfuck (why do people insist in censoring themselves?!) in Scala. You didn't even implement dynamic memory resizing.

Again, not sure why this is here.

<comment><USER4> Agreed, more or less. Brainfuck implementations are not generally interesting, except as proofs of Turing-completeness for other esoteric languages. And there are many more esolangs out there with much wilder and more beautiful semantics than Brainfuck. Implementing _those_ can be a really valuable learning experience.

Also, if we're bringing Scala into it, I would be more interested to see a type-level implementation.

<comment><USER2> > And there are many more esolangs out there with much wilder and more > beautiful semantics than Brainfuck. Implementing those can be a really > valuable learning experience.

Like, for example, INTERCAL with threading via `comefrom`: <http://www.cse.unsw.edu.au/~malcolmr/intercal/threaded.html>

<comment><USER4> Lazy K[1] and Funciton[2] are two of my personal favourites.

[1]: <http://homepages.cwi.nl/~tromp/cl/lazy-k.html>

[2]: <http://esolangs.org/wiki/Funciton></comment></comment></comment></comment>


Ask HN: Was Bayes Impact the only nonprofit in the S14 batch? - <AUTHOR>

Lots of great press profiling the nonprofits in the W13 batch, but I haven't seen much on the S14 batch nonprofits outside of Bayes Impact. Were they the only one, or ??? <comment><AUTHOR> For anyone that happens upon this, Bayes Impact was indeed the only nonprofit in the summer 2014 batch of YCombinator.

[https://twitter.com/bayesimpact/status/503996905928417281](https://twitter.com/bayesimpact/status/503996905928417281)</comment>


AddGene: Nonprofit Plasmid Repository - <AUTHOR> https://www.addgene.org/ <comment><AUTHOR> The parallels to Github for code distribution and re-use cannot be overlooked ;)

Addgene also works with another Cambridge nonprofit called Seeding Labs. They take donations in the form of scientific equipment, lab services and expertise and then distribute around the world. Following a "local problems are best solved locally" approach.

The power of a model that is not only open but "shared" seems to replicate itself organically. To scientists, a distributed workflow comes quite naturally!

[https://seedinglabs.org/](https://seedinglabs.org/)

</comment><comment> kris-s I used to work at Addgene, really enjoyed it. The nonprofit "open science" business model is awesome!

<comment><USER2> Addgene is a great service. My lab has ordered a couple vectors from them. I do wish the material transfer agreements were less painful, though.

It's great to see more open science. Now we just need to fix publishing, and we're golden...

<comment> kris-s Oh yeah MTAs are a nightmare.

</comment></comment></comment><comment><USER3> I used to order lots of plasmids from Addgene when I worked in bioscience -- it's an awesome service. The plasmids are hugely cheaper than anything you can order from a for-profit supplier, and it lets scientists easily share and remix each other's work.

</comment><comment><USER4> Addgene is a wonderful and necessary service. Sort of like an open source wetware repo. Great place to work too.</comment>


Building a Text Editor for a Digital-First Newsroom - <AUTHOR> https://open.nytimes.com/building-a-text-editor-for-a-digital-first-newsroom-f1cb8367fc21 <comment><USER3> One of the most truly fascinating things I learned from my father was how early 1980s word processors were designed for the magazine he worked for. The different ways to handle fonts, screen placement, and how to include data from a local database.

It was truly an awakening to the hidden wonders of the world.

</comment><comment><USER4> RE the flat paragraph structure - did they just reimplement Emacs in the browser?

[https://www.gnu.org/software/emacs/manual/html_node/elisp/Te...](https://www.gnu.org/software/emacs/manual/html_node/elisp/Text- Properties.html)

(Also I think - from occasional messing with XML in .docx files - but can't be sure that Word flattens styling like that as well.)

<comment><USER5> Any sufficiently complicated editor contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of emacs?

I prefer vim though..

<comment><USER4> Vim ain't sufficiently complicated (that can arguably be seen as a good thing). That's why it's been mostly reimplemented _in_ Emacs.

<comment><USER5> Vim has vimscript (ugh) with a lua and python gaining popularity so emacs sort of got it right IMO anyways.

<comment><USER6> Neovim's main goal seems to be replacing vimscript with lua in a backwards- compatible way.

</comment></comment></comment></comment></comment><comment><USER7> I'm surprised she didn't mention Marijn in the "thank yous" at the end - without him giving away his time building ProseMirror they would have nothing to build upon.

</comment><comment><USER8> I've invested in Mobiledoc (mostly because it was for two ember apps) and it's mostly been fine, but is there any reason why I shouldn't be moving to ProseMirror? Seems to be a lot more powerful and have things like collaborative editing and track changes as well.

<comment><USER9> Just my experience: I had a rough time with ProseMirror trying to use it before it was released. It didn't use modern JS modules for a long time, the source code was a mess as far as I could tell, and JavaScript variable errors would surface with every version.

Maybe the author has overcome these issues, but I'd be inclined to go with a different editor. Also, ProseMirror's architecture is extremely close to DraftJS, which I've had much fewer issues with.

<comment><USER10> Yeah thanks for dropping some deeply outdated impressions in here. (Also not even fair—the code was never a mess and at the time it wasn't clear how ES modules would be used so using CommonJS was a completely reasonable choice.)

<comment><USER11> Hey there,

Just wanted to offer some feedback. Prosemirror's docs are woefully incomplete. The way controls are added is extremely confusing. Why do we need some node module called "prose-mirror-example-setup" ?

I know I'd personally prefer being able to add all the markup and style for the editor, and then just use prosemirror underneath the hood for state management.

Another thing that I found quite difficult, was programmatically inserting images at the cursor.

Finally, storing state as markdown has turned out to be a nightmare. Not really your fault, but perhaps something you should be aware of.

<comment><USER12> I agree the docs are obtuse, and implementing features is hard, but document state is not saved as markdown. The whole point of PM is to separate content from prsentation.

</comment></comment></comment></comment></comment><comment><USER13> Cool, but it's yet another case of re-engineering the same thing. (edit: its not entirely new).

Splitting editors into an abstracted structure for finer control over data and presentation is something that's been done for a decade now in various text editors. Medium and Basecamp started similar projects and there's Mobiledoc which is trying to standardize the format. It'd be nice if all this effort focused on a single project so we could all just get a decent editor instead of different versions with various states of functionality.

FYI, my running list of editors: [https://gist.github.com/manigandham/65543a0bc2bf7006a487](https://gist.github.com/manigandham/65543a0bc2bf7006a487)

<comment><USER14> It might be helpful to update your list with some feature comparisons. For example:

Quill vs (CKEditor and TinyMCE, Draft, ProseMirror, Trix): [https://quilljs.com/guides/comparison-with-other-rich- text-e...](https://quilljs.com/guides/comparison-with-other-rich-text- editors/)

Slate vs (Draft, ProseMirror, Quill): [https://github.com/ianstormtaylor/slate#why](https://github.com/ianstormtaylor/slate#why)

I personally went with Slate because I needed first-class React support but not as low level as Draft.

<comment><USER15> I really like Slate....but the ability to do collaborative editing is relatively new.

Have you done anything with collaborative editing with Slate? If so, how are you finding it?

<comment><USER14> Collaboration isn't a requirement for me. I would definitely go with quill instead if it were. They put a lot of thought into making the delta format natural for operational transforms: [https://quilljs.com/docs/delta/](https://quilljs.com/docs/delta/)

Your document state is always represented by a list of deltas. Two people trying to change the same document would just have their individual list of ops "rebased" together using OT before being concatenated into the existing list of ops, forming a new document that is still just a list of ops.

<comment><USER16> I second using quill. It's really modular and the code itself is so clean.

</comment></comment></comment></comment></comment><comment><USER17> Anyone know if Oak is (or will be) released as Open Source?

<comment><USER18> ProseMirror, the underlying editor they are using, is: [https://prosemirror.net/](https://prosemirror.net/)

<comment><USER17> Yep. That was clear from the article.

</comment></comment></comment><comment><USER19> I'm choosing an editor for my project right now. Can someone compare ProseMirror to Slate and Quill? What is the difference, which one is better? What do I need to know to choose one?

<comment><USER20> I've used a lot of quill in the past couple of years. It has some tricky bits, but seems similar in a lot of ways to ProseMirror - separate internal representation of data that gets rendered into HTML, delta-based state changes, etc. At this point, I use quill out of momentum - it was the only game in town when I started working with rich text, and now I know how it works. I also like it enough that I've never felt the need to go find something else; it's always done what I want.

I think to fully experience all three of those editors, you should build the following: something that generates a rich text input using the editor, updates a data store (can be just in-memory if you want), and then re-presents the rich text in a read-only format.

With Quill in particular, there's two ways to approach the second part - <USER2> could dump the data into another instance of the quill editor, but one flagged as read-only, and let quill handle rendering out all the elements. I don't do that, myself, as it's a bit heavy and relies on the DOM to be present to create the end result (meaning rendering rich text into emails on the server side can be tricky). Instead I use a library (that I wrote) that converts the quill representation into an HTML string that can be later injected into whatever place it needs to go.

In general, I think that's where the one of the important pain points is for any major modern text editor exist - converting the internal data representation into a canonicalized HTML string. This gets further complicated by your framework - it's easy enough to inject an HTML string, and angular 1 lets you even compile that so you can have custom elements there, but angular 2 has no such facility (compiling; you can still inject). So rendering custom elements into angular 2 is feasible but quite tricky.

You'll need to figure out how to handle custom elements too, like at-mentions and so on; those are really domain-specific. How I handle at-mentions in quill (with a pop-up autocomplete) depends on the structure of the overall system as well as the framework I'm in - angular 1, 2 and react all suggest different approaches for this.

Anyway, this is all a long response that means "it depends". Each editor has its own strengths and weaknesses that are exposed by the specific data and presentation needs of your project and the other frameworks in play. But building an atmention or other custom element and converting from the internal data structure to HTML is a good test of functions for all of those.

</comment></comment><comment><USER21> Is Oak going to be opensourced? Looks very interesting.

</comment><comment><USER22> I've been watching this product for a while: [https://www.nuclino.com/](https://www.nuclino.com/) it seems they too have built something really impressive on top of ProseMirror (it's a lot like Quip/Paper but more interesting). ProseMirror really feels like the only editor on top of which you can build an entire user experience and app.

</comment><comment><USER23> So, we had this product (or the larger one anyway) demoed for my work and our takeaway was that only really large organisations could possibly use it. It needs to many human hands between writing an article and getting it published. We want to move to something more automated, and this system would have been a serious setback.

</comment><comment><USER24> Oak was the codename of Java IIRC... May confuse some old people like me

</comment><comment><USER25> This isn't even a test editor. It's closer to a web publishing tool. The article also didn't talk about the normal text editor challenges like the way the text is stored in memory and modified.

<comment><USER26> Agreed.

I carry the personal view that when someone says "text editor" they are implicitly implying plain text. As such the leading paragraph of the article makes me twinge a little when "text editor" is used rather than "word processor".

The article then continues on to describe the editor which sounds very much like a WYSIWYG editor. Indeed ProseMirror[1] homepage says the following.

        ProseMirror tries to bridge the gap between Markdown text editing and
        classical WYSIWYG editors.

        It does this by implementing a WYSIWYG-style editing interface for
        documents more constrained and structured than plain HTML

So whilst I've made a point of it reading oddly (and I would have thought this would also be the view of most others on HN) I am not trying to disparage the authors use of the term "text editor". But rather I am curious as to why such nomenclature happened to be used here as I seldom see it outside of descriptions of things such as Vim and Emacs.

[1] [http://prosemirror.net/](http://prosemirror.net/)

<comment><USER27> What would you call Adobe Indesign?

<comment><USER28> It's a digital publishing system.

</comment></comment></comment></comment><comment><USER29> _Data Warning: 100 MB._

Viewing the header illustration for more than a few seconds crashes my iPad Safari tab. Scrolling until it is off the viewport prevents this.

EDIT: Dev Tools reports 74.8 MB transferred _with_ an ad blocker. The illustration is a single 69.2 MB gif.

EDIT: 101 MB in total when you scroll to the bottom. Three more gifs: 18.1 MB, 7.1 MB, 4.7 MB.

<comment><USER30> Instapaper Text view reduces it to a slightly more reasonable 41 MB.

[https://www.instapaper.com/text?u=https%3A%2F%2Fopen.nytimes...](https://www.instapaper.com/text?u=https%3A%2F%2Fopen.nytimes.com%2Fbuilding- a-text-editor-for-a-digital-first-newsroom-f1cb8367fc21)

(This may require a free Instapaper account to view.)

<comment><USER31> UBlock brings it down to 204KB because the images are on a CDN. Being on a CDN is probably why the author is oblivious to the file sizes.

</comment></comment></comment><comment><USER32> I'd love to work for this engineering team. Their tech blog and github is full of interesting projects.

<comment><USER33> the BBC sounds like it has a really good tech team too. wonder what it is about newspapers that leads to this sort of tech culture.

<comment><USER34> A desire to stay relevant. Print publishers often get criticized for being behind the curve, but a few like NYT and BBC have invested heavily in the technology of their business. Not sure how much it's paid off, but my impression is they aren't struggling as much as some.

I hope they manage to find the right combination to stay around, I loathe the dystopian future where Buzzfeed and Breitbart are the only news outlets.

<comment><USER35> Buzzfeed isn't comparable to Breitbart at all. Buzzfeed does real journalism these days.

<comment><USER34> Their news coverage is pretty strongly slanted to the left. Just because I tend to agree with them doesn't mean I think they are really that much better than Breitbart in terms of providing objective reporting.

In any event, what I meant was that I don't want a future where we only have sources of "news" that serve to reinforce our pre-existing beliefs, with everybody siloed into their own particular little bubbles.

</comment></comment></comment></comment></comment><comment><USER36> > If you're like most people in America, you use a text editor nearly every > day.

Filter bubble detected.

<comment><USER29> The author does not say "work in a text editor", only "use". Considering she's including text editors on mobile devices, which the majority of America will utilize today, that's not a particularly bold statement.

<comment><USER37> I suppose when I send a text I am using a text editor of sorts

</comment></comment></comment><comment><USER38> I'm speechless.

</comment><comment><USER39> XML

</comment><comment><USER40> "We're planning to begin work soon on a collaborative editing feature that would allow more than one user to edit an article at the same time"

You mean like Google Docs or Quip?

</comment><comment><USER41> _> this new story editor needs to combine the advanced features of Google Docs with the intuitive design focus of Medium_

This part made me sad. Imagine how much more powerful our tools could be if we didn't insist on jamming them all into web browsers!

<comment><USER27> But at least they run everywhere.

<comment><USER42> Where everywhere is some subset of browsers in the wild and often not that consistently anywhere.

I mean the web is awesome but it's far from perfect from a programming point of view but that of course doesn't matter for other reasons.

<comment><USER43> > Where everywhere is some subset of browsers in the wild and often not that > consistently anywhere.

Yes, but it's easier to download a version of the browser that works. The alternative is for the developer to build native apps for each OS they want to support. In my experience, it's easier to build cross-browser than it is to build cross-platform.

</comment></comment></comment></comment><comment><USER44> If you have professional writers why not just teach them markdown and git?

Instead of spending engineering $$$ building, maintaining and operating a complicated editor?

<comment><USER45> Because professional writers are not professional coders. Markdown covers the basic text markups but does not define how to format all these embedded media a modern newspaper has. Just to give an example: Inspect the syntax of wikipedia, which is the MediaWiki wikitext ([https://www.mediawiki.org/wiki/Wikitext](https://www.mediawiki.org/wiki/Wikitext)) language. It is simple and therefore Wikipedia pages are rather simple. I expect the NYtimes to have higher aims in markup.

<comment><USER46>
      >> why not just teach them markdown and git?

I myself am surprised they don't just use markdown. Git is another story.

      > writers are not professional coders.

But I have heard of writers clinging to old versions of WordPerfect, which were not WYSIWYG. They preferred the keyboard shortcuts to clicking around with the mouse.

Furthermore if you're just writing the text of the article, you don't even need markdown, just a textarea. They used to do these on typewriters after all. Newspaper articles traditionally lacked anything but plain text. No bold, italics, or subheadings.

      > Markdown covers the basic text markups
      > but does not define how to format all these embedded media
      > a modern newspaper has

Traditionally the jobs were separate --- and I assume they still are. There are the reporters, who research and write the text of the article. Then there is the layout team who lay it out, decorate it, etc. I can see how markdown would be no good for them. They would want a graphical user interface.

<comment><USER47> > Traditionally the jobs were separate --- and I assume they still are.

Is this really a good assumption? Journalists are making complex documents with central interactive features and lots of data visualization. It seems a lot like classic waterfall thinking to assume that reporters could perfectly envision that and toss it over the wall for a separate team.

<comment><USER46> Well yes, I think so, for these reasons:

1\. Most articles have simple layouts, like this one, [https://www.nytimes.com/2018/04/12/us/politics/trump- trans-p...](https://www.nytimes.com/2018/04/12/us/politics/trump-trans- pacific-partnership.html)

2\. Most reporters were journalism majors, which teaches research, interviewing, and writing. Graphic design is a separate discipline. It is possible for one person to be good at both, but rare.

3\. Companies tend to cut up work, especially big companies, like the New York Times.

4\. Complex articles like this one, [https://www.washingtonpost.com/graphics/2018/entertainment/v...](https://www.washingtonpost.com/graphics/2018/entertainment/video- game-movies/), aren't tossed over the wall. But that doesn't mean that one person did it all either. I'm sure there was back and forth between the writer and designers, and likely an editor oversaw it all.

</comment></comment></comment></comment></comment><comment><USER48> Whew, what a gross way to over-engineer a text editor. If the NYTimes structured themselves like a dev team, they could use common sense tools to get much better results.

Why is the journalist responsible for how it's presented? Wouldn't it make more sense to have some kind of product manager or designer handle this? And why is the journalist responsible for programming the look and feel of the their article? Why not have a front end dev do that?

<comment><USER49> The journalist doesn't only write an article, he also structures and plans how the story is going to be presented because that's also part of the job. Hiring another person in-between the article and the journalist is like hiring an extra person to only hammer a nail when there is a construction worker to do that job already.

They develop the underlying technology to let journalists do what they want without needing to hire more employees.

<comment><USER50> In industries that still make money--even ones with better software-chops than the New York Times--writing/researching an article, and presenting it, are _still_ separate jobs--and for good reason! They require orthogonal skillsets and a decent amount of time and hands-on experience if you want a professional result.

No one is "only" hammering a nail here.</comment></comment></comment>


Intel's doomed McAfee foray - will history repeat itself? - <AUTHOR> http://www.zdnet.com/blog/storage/intels-doomed-mcafee-foray/1049

<comment> xl-brain I only hope there's a small chance that shareholders can reject the acquisition.

<comment><USER2> I don't know the schedule, but Intel will likely be rather committed by the next shareholder's meeting. And this is not _entirely_ and clearly insane enough to e.g. muster the failed effort to keep HP HP a while ago.

Robin Harris is a great analyst with a sense of history and he shows all that in this column. I sure looks like it'd just going to be a repeat of those two last failures, especially since there's not (much of) a hardware component (as far as we know) in this cunning plan.

I suppose it's good for Intel that they're swimming in so much cash the don't need this to make or modify another fab line or three, but still....</comment></comment>


Less Known Bits of the Python Standard Library - <AUTHOR> https://medium.com/@vmsp/less-known-bits-of-the-python-standard-library-46dc88490115 <comment><USER2> Nice list, saving this for later.</comment>


Ask HN: Is Data Science still the sexiest job of the 21st century? - <AUTHOR> I have a friend who has been feeling disillusioned about Data Science - the sol-called sexiest job of the 21st century. So trying to understand a bit more from you all...<p>Do you think data science is still the sexiest job of the 21st century? Why or Why not? <comment><USER3> From what I've seen recently "Data Scientists" are really just analysts now. 4-5 years ago it would have been expected that a data scientist would be the cross between a statistician and a senior developer, now they're just people a masters degree and 2 Machine Learning classes under their belt.

From what I can tell title of the aforementioned has changed to "Data Engineer" or something similar.

> "Do you think data science is still the sexiest job of the 21st century? Why > or Why not?"

Seems like a good job as far as jobs go. You don't have to lift heavy things, not likely to die in the process of doing your job. Usually pays well, and your family will be proud of you when they talk about you at cocktail parties. So yea... its a pretty good job.

</comment><comment><USER4> My rule of thumb is to never take career or personal finance advice from mass media articles.

As someone who has been in various roles in SF tech scene for a few years now, I've never thought of "data scientist" as a "sexy" job at all. You are basically a data analyst spending more than half your time cleaning data and waiting for queries to finish running.

Same goes for product managers...very few, if any, have authority over the actual "product vision" and spend most of their time writing tickets, project managing, in meetings.

</comment><comment><USER5> I thought being a Hollywood star or footballer is the sexiest job - <USER2> science is just trying to fit some models to data and then selling your superiors on results of your analysis.

</comment><comment><USER6> What does sexy mean in this context?

<comment><USER7> Most-hyped became 'hottest' which in turn became 'sexy'

Basically what recruiters want because companies want them (irrespective of need), they have high salary expectations and recruiters work on commission.</comment></comment>


Say "no" to import side‐effects in Python - <AUTHOR> http://chrismorgan.info/blog/say-no-to-import-side-effects-in-python.html <comment><USER3> I do most of my work with Perl, rather than Python, but in Perl there are several kinds of stock import-level side-effects which are actually quite helpful. These are pretty light-weight though. They boil down to:

1\. Global lexical changes to Perl. Sometimes this is the whole point of an import (for example Carp::Always, which typically turns any warning or exception string into a full stack dump). I can't imagine doing something like this in Python. However making soemthing like this work properly without breaking too many things requires a heck of a lot of forethought Yes, even Carp::Always may break something.

2\. Manipulation of the importing module's symbol table. This is important for lexical extensions to Perl that you don't want to be global (for example Moo, Moose, and PGObject::Util::DBMethod). Among other things this allows MOPs to be added with greater sophistication than the language typically allows. I am not a Python guru but I could imagine metaprogramming side effects to be useful in setting up a consistent and powerful environment.

The problem the author describes is something which is different though, which not only is a side effect issue but also a violation of separation of concerns. There are certain problems you do not want to solve at import time, and connecting with/configuring external components is almost always one of them.

Why? Because integration with external components is almost always something you want the fine-tuning and decision-making to reside with the application developer. That's very different than setting up a consistent lexical programming environment for use (which is what the acceptable side effects do).

<comment><AUTHOR> In Python, side-effects are simply not acceptable at all. One should instead put the code with the side-effect in a function and call it.

This is the approach taken by gevent, which allows you to replace the entire I/O stack. But importing it does not effect the change; you must explicitly call code to do that. This is done thus:

        from gevent.monkey import patch_all
        patch_all()

c.f. [http://www.gevent.org/gevent.monkey.html](http://www.gevent.org/gevent.monkey.html)

<comment><USER4>
        from __future__ import print_function

<comment><USER5> As I understand it, "from __future__ import ..." statements are actually a special type of statement that doesn't actually import a module at all - <USER2> just use similar syntax for compatibility reasons. There is an actual __future__ module, also for compatibility reasons, but importing it has no side-effects.

<comment><AUTHOR> Future statements do import something as well, when run, but their primary purpose is changing the behaviour of the compiler _for the current file_. They have no behaviour outside of that file, so that is not a publicly visible side-effect.

</comment></comment></comment></comment></comment><comment><USER6> On one machine I tried, help('modules') actually worked successfully with no substantial delays or apparent side effects.

On another, it apparently tried to set up an MPI cluster:

      *** The MPI_Init() function was called before MPI_INIT was invoked.
      *** This is disallowed by the MPI standard.
      *** Your MPI job will now abort.
      [hostname:14114] Abort before MPI_INIT completed successfully; not
      able to guarantee that all other processes were killed!

On a third, I get the following and then it just hangs:

      Python 2.7.6 (default, Mar 22 2014, 15:40:47)
      [GCC 4.8.2] on linux2
      Type "help", "copyright", "credits" or "license" for more information.
      >>> help('modules')

      Please wait a moment while I gather a list of all available modules...

      /usr/lib/python2.7/dist-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion 'g_type_from_name (name) == 0' failed
        import gobject._gobject
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: cannot register existing type 'GtkWidget'
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: cannot add class private field to invalid type '<invalid>'
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: cannot add private field to invalid (non-instantiatable) type '<invalid>'
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: cannot register existing type 'GtkBuildable'
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: g_type_interface_add_prerequisite: assertion 'G_TYPE_IS_INTERFACE (interface_type)' failed
        g_type = info.get_g_type()
      /usr/lib/python2.7/dist-packages/gi/module.py:171: Warning: g_once_init_leave: assertion 'result != 0' failed
        g_type = info.get_g_type()

</comment><comment><USER7> This in no way invalidates the point of the post, but I can't imagine going back to installing everything globally instead of using virtualenv. If you do that, at least you won't have every package you every looked at in your path.

<comment><USER8> I just can't get to use some modules within virtualenv. They seem to create more problems than using systemwide.

<comment><USER9> Which ones? You can get some help. Learning virtualenv is a smart long-term move

<comment><USER7> If you're on a Mac, good luck getting pip to install mysql-python. I use Macports for that.

<comment><USER10> This isn't a good solution. I pip install mysql-python several times a week on my mac ( I use different venv's for different branches and we have several different services that I work on in a given week, thus I'm installing packages via pip A LOT) and we always install it in a venv. Sure it can be a bit of a pain but it's well worth it IMO.

Our steps to always get it working: make sure mysql_config is in your path env variable, make sure the xcode command line tools are probably installed, and make sure the mysql command line client is setup and working correctly locally. Past that is just works for us on Mac OS X 10.7+. I think 10.6 and earlier also work but I'm not sure.

<comment><USER7> I couldn't ever get it to compile, but I haven't tried in a while. I don't actually need it much these days as I'm using Postgres for most things.

I'm sure there are other packages that are similar: not so easy to get working with pip but not impossible.

</comment></comment></comment></comment></comment></comment><comment><USER11> Of course you should do this in any language. I once used a Ruby library that, when loaded, would try to connect to a database on a remote machine. Programs which required this library would take several seconds to display their --help output.

Because of that and similar incidents, I've learned to import argparse up front but nothing else unless necessary. Once argument parsing is done, then importing other modules begins.

<comment><USER12> > I once used a Ruby library that, when loaded, would try to connect to a > database on a remote machine.

How is that at all acceptable? I can't believe that a library that phones home would gain any sort of popularity.

I can't say I know much of anything about the Ruby community, but if they've conditioned you to jump through hoops like importing modules at specific times to avoid delays, that is a serious problem. Conditional/delayed imports have their place, but they should be relatively rare.

<comment><USER13> install my module! Also.. trust me because it's totally safe.

curl [http://foo.com/tBrwn](http://foo.com/tBrwn) | bash

[http://rvm.io/rvm/install](http://rvm.io/rvm/install)

<comment><USER14> Out of curiosity, what would be better to handle the case of RVM?

It's explicitly user space software, and unprivileged user space software at that, so having it require admin interaction (i.e. touching the system package manager) to install seems like a sledgehammer where a flyswatter would do.

They could use a VCS repo of some kind, but that doesn't handle the various folder and script installs that need to happen - and also doesn't help if there isn't that specific VCS on the system.

Upstream security is less of a concern since that hotlink just points to raw code on Github, and over HTTPS no less.

So what are the negatives here? For software like RVM, this seems like the best, most portable solution that works for the most people.

<comment><USER13> being on github and using https doesn't guarantee anything, obviously.

If you look at the actual script it contains a ton of sudo and also additional curl commands. The chain of security is very lacking.

</comment></comment></comment></comment></comment><comment><USER15> It's also worth stressing that even if import side-effects are always going to be fast and not kill the interpreter, they are still a terrible idea.

Even spawning objects that are referenced in modules can have some rather unpleasant properties. The __del__ method will likely never get reliably called and other behaviours that work great in scripts break in subtle ways, especially with a KeyboardInterrupt. Threading and multiprocessing will leave processes running using 100% cpu. Trying to debug these things gets insane, as you can often only find them as the interpreter is dying.

I think import side-effects can be tempting because Python is often introduced using a scripting-oriented approach. Combined with Python following the principle of least surprise, most people doing this won't even realise that it's wrong. This does seem to be a rather common anti-pattern.

</comment><comment><USER16> Putting side effects in the __init__ code seems to become quite fashionable these days but is a pretty bad idea since it removes the possibility to "just" import the functionality defined by the module without performing any initialization. Personally, I always try to avoid having a system that relies on some global configuration (like e.g. Django, Matplotlib or Flask do). In matplotlib for example this causes a lot of problems, since importing the pylab module will automatically (among other things) load and initialize a backend, which is then set in stone for the rest of the session.

IMHO, the way to go here instead is dependency injection:

Inject the configuration into the module through a function or class method (e.g. Flask.initialize({config state}). Wrapping all module functionality that depends on configuration in a class is a good idea here since it allows you to use multiple configurations in parallel and makes your code more modular.

As an example, in BlitzDB (a document-oriented database for Python, [https://github.com/adewes/blitzdb](https://github.com/adewes/blitzdb)) there is no global configuration at all, so you can initialize and use multiple backends in parallel as you please without worrying about side effects. SQLAlchemy does it in a similar way btw.

<comment><USER17> FYI, it is possible to call Matplotlib in an 'object-oriented' way without global state; though it's a bit more cumbersome than just using the pylab interface. See [http://matplotlib.org/examples/pylab_examples/webapp_demo.ht...](http://matplotlib.org/examples/pylab_examples/webapp_demo.html) for an example.

</comment></comment><comment><USER18> Agreed. Twisted's behaviour of installing a reactor on import has caused me problems which could be worked around by importing conditionally or at a later time, but in cases like this, where one is importing modules dynamically, one doesn't know ahead of time which workarounds one needs.

<comment><USER11> Twisted's problem is the good old singleton (anti-)pattern. There can only be one reactor ever, and most galling of all, it can never be restarted once stopped.

<comment><USER19> It should be said that Twisted has put a lot of effort into making it possible to making it possible to deal with multiple reactors (mostly so they can run their test suite including all their supported reactors), and even to make it possible to unit-test Twisted code without a reactor at all (by having standardized mocks for the various things the reactor does).

Of course, that doesn't help the mountains of code written for Twisted that expect a singleton reactor, so we're stuck with it for the foreseeable future. Perhaps in the Brave New World of Python 3, where Twisted is just an implementation detail of the asyncio module, life will be better.

<comment><USER11> Why can't we at least restart a stopped reactor? That seems possible without breaking compatibility.

</comment></comment></comment></comment><comment><USER20> For those not in the knowing, avoiding import side effect means never having top level code in any python module except class, functions and constant definitions, other imports and the occasional if name = main. And constants must be built-in types.

</comment><comment><USER21> Qt and Gtk seem to be the problem here, and they won't change. They aren't python modules as much as they are application frameworks and assume to be in full control. Gtk calls sys.setdefaultencoding("utf-8") when imported, changing the way your strings behave in the whole process.

<comment><AUTHOR> With regards to Qt, the problem is not Qt but rather that something else is actually trying to _use_ Qt at a time when it shouldn't.

pygtk setting the default encoding to UTF-8 is concerning—if true, that is certainly bad behaviour.

<comment><USER22> I haven't used GTK in a long time, but when I was using it, I was under the impression that pygtk was being deprecated, and everyone should be using GObject-Introspection instead.

<comment><AUTHOR> I dunno—I was making assumptions, not being very familiar with the GTK scene in Python.

<comment><USER21> Not me either anymore. I tried to port to introspection + Gtk3 but ran into problems with widget subclasses and no documentation to resolve it with.

<comment><USER23> I only extensively used it from Perl and Vala, but with GObject-Introspection I tend to more often look at the original library documentation than something language specific. This can be a disadvantage at first, but for me it turned out more convenient, since GIR-inflated bindings can be more complete, as long as the inflation supports the features the API describes. They also tend to be more consistent in their differences to the original.

The problem about documentation seems to be the usual dilemma that as soon as you know enough to implement an API browser reading GIR that outputs the API in your language, you know enough about how the bindings work themselves to just use the original documentation.

<comment><USER21> well it doesn't really explain how to port from pygtk or how python classes interact with g-i.

<comment><USER23> That is true, and unfortunately I have used Python only sparingly until now, and have no experience using PyGtk at all.

I took a quick look at PyGObject[0] (unsure if that is what one would actually use for this), and the most helpful part seems to be [1], giving some hints on extending GObject.Object, which should be translatable to extending other existing GObject type classes.

As for porting, I agree that can be a pain in that case. When bindings move from a manual implementation to inflating it from an external description. You often end up with good tutorials for the first, and good API reference for the second. If I need to understand other kinds of Gtk bindings, I search for examples on custom TreeModels or other potentially messy things like that to get a feel for it.

[0] [https://wiki.gnome.org/action/show/Projects/PyGObject?action...](https://wiki.gnome.org/action/show/Projects/PyGObject?action=show&redirect=PyGObject) [1] [http://python- gtk-3-tutorial.readthedocs.org/en/latest/objec...](http://python- gtk-3-tutorial.readthedocs.org/en/latest/objects.html)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER24> If there's one thing I've learned from working with old PHP codebases, it's that side-effects from include/require are a horrible horrible idea.

I'm OK with Java's static-initializers, but that's because they've got a whole bunch of rules around them preventing common kinds of abuse.

</comment><comment><USER25> I am an author of a library [1] that does this. I understand the what the OP is talking about and agree with it, but like anything for me the rule is "(1) Don't do dangerous behavior X. (2) If you are an expert, do dangerous behavior X sparingly and with caution."

Mind you, my library does not connect to a database, or any such crazy thing. It simply creates a singleton which is then useful throughout your application. There would be very few cases where you would not want that singleton and you would want your own instance. If so, you are free to completely ignore the created singleton and make your own instance. The creation process is also idempotent, except the data you put into that singleton; that data is a special case: it is your explicit responsibility to namespace it, which is indeed the whole point of this library. I feel pretty good about this thing.

[1] [https://github.com/ipartola/groper](https://github.com/ipartola/groper)

<comment><USER26> Thinking about it, I do the same thing with my Ruby library mime-types[1]. The library is only useful if you load some data into its registry, so it does so automatically, unless you specify RUBY_MIME_TYPES_LAZY_LOAD in the environment.

I've just put myself a task to reverse the behaviour so that lazy loading is the default.

[1] [https://github.com/halostatue/mime- types](https://github.com/halostatue/mime-types)

</comment></comment><comment><USER27> I have the unfortunate "luck" to be using a Python library at work that _loves_ to use import side-effects. Importing it like God intended makes it parse command-line arguments and fail if it doesn't like what was passed in. And that's just the start. Nearly every __init__.py has code in it, including class definitions. I have no idea why.

<comment><USER28> It's like programmers who find out about metaprogramming. Usually they grow out of it.

<comment><USER27> I haven't grown out of metaprogramming at all. I just try and use it only when it's the best option. Usually because it means reducing duplication.

I liked metaprogramming to an extent in C++. I _love_ it in D.

<comment><USER20> Replace "best option" with "only practical option" and you're good to go.

<comment><USER3> That may depend on the language you are using.

</comment></comment></comment></comment></comment><comment><USER29> Would anyone like to share their experiences avoiding this sort of problem in the context of web frameworks and building the back end for larger web sites/apps?

As an example for discussion, the first time I wrote a Flask-based back-end, I backed myself into a corner almost immediately in the following way.

Firstly, the WSGI file that the web server uses to start the application followed the suggestion in the Flask docs by doing this:

        # webserverseesthis.wsgi
        from yourapplication import app as application

That's not so bad, but then I started doing application configuration and loading various Flask plug-ins as side effects of that import:

        # yourapplication/__init__.py
        app = Flask("yourapplication")

        # Do some general application configuration.
        app.config.from_pyfile("/path/to/configuration/file")

        # Set up some overarching security things that modify application behaviour.
        from flaskext.securityplugin import SecurityPlugin
        sp = SecurityPlugin(app)

This seemed at the time like the obvious place to put such things, but of course, this is really just a variation on the mistake we're discussing here.

To compound the error, I then used Flask's decorators to wire up routes from various URLs to the relevant parts of my code. Those decorators work on the application object (sticking with ideas common to many Python web frameworks and avoiding getting into anything more Flask-specific like blueprints) so I was effectively creating circular dependencies from almost everything to that top-level package:

        # yourapplication/pages/home.py
        from yourapplication import app

        @app.route('/')
        def home_page():
            # Render home page

and then from the top-level package onto almost everything so all those decorators could take effect:

        # After setting up the application object in yourapplication/__init__.py
        import yourapplication.pages.home

Now, as long as this kind of code only ever runs as a WSGI application behind a web server, you get away with these dependencies up to a point. In practice, your WSGI set-up imports the top-level application package, which in turn sets up the application object everything is going to depend on and only then imports all the supporting modules/packages, and everything "works".

However, as soon as you want to write tests or otherwise reuse any of the code in a different context, the entire system is a big bowl of spaghetti with all the usual problems. The moment you import any part of the system to run a unit test on something in it, you get much of the rest of the system as well, complete with the side effects of any imports therein.

This was of course all horribly naïve on general programming principles, but the nature of these frameworks tends to push in this direction, and even Flask's own documentation features various simple examples that follow a similar approach, so I'll forgive myself for falling into the trap the first time. I've since experimented with various techniques to break the cycles and avoid the side effects on imports, with some success, but frankly I've never found a satisfying, general strategy for organising larger code bases built around a web framework.

How is everyone else doing this?

<comment><USER30> The example on the Flask tutorial with the app at module level is really only viable if you cram everything inside one module, it gets old soon. You should use a factory pattern, like this:

        def app_factory(config):
            app = Flask("yourapplication")
            app.config.from_pyfile(config)
            # ...
            return app

Then whenever you need access to your app object, you use the provided proxy:

        from flask import current_app as app

You can't use it at module level though (because there isn't an application context setup by that time), so this doesn't work:

        @app.route('/')
        def home_page():
            # ...

Instead, hook up views inside your app factory:

        def app_factory(config):
            # ...
            app.route('/')(somemodule.home_page)
            # ...

For the test suite, you can now instantiate apps with a different configuration:

        from flask import current_app as app
        from myfoo import app_factory
        import unittest

        class MyFooTest(unittest.TestCase):
            # ...

        if __name__ == '__main__'
            test_app = app_factory(test_config)
            # The app proxy will point to that inside test cases
            unittest.main()

TL;DR: The factory pattern is your friend. Parametrize all the things. Avoid singletons at module level, this leads to spaghetti. If you need convenience, create proxies.

<comment><USER29> _You should use a factory pattern, like this: [...] You can 't use it at module level though (because there isn't an application context setup by that time), so this doesn't work: [...] Instead, hook up views inside your app factory: [...]_

That's basically what I did on my second iteration. It is an improvement in some respects, particularly breaking the circular dependencies caused by using the decorators on the global application singleton. On the other hand, now you need some variation of God Object that not only imports all your modules that used to have decorators but also knows enough about their internal implementation to set up the routes and things like pre- and post-request logic directly on the application object you get back from the factory.

The next logical step after that then seemed to be having each module/package that contains views or similar logic provide some sort of initialization function that is declared when you import the module and takes an application object as a parameter. Then we can use app.add_url_rule and friends to wire up the various handlers within each package/module but decoupled from any sort of global application object that needs the circular import. This is the tidiest style I've found so far, and all my Flask projects in recent years have used something broadly like it. It does only require one import followed by one initialization call for each package/module, which logically seems to be as good as we can get, given that our starting point is a desire to avoid including any initialization implicitly within the import itself and to avoid depending on global singletons.

Somehow, it still doesn't quite feel right for some reason. I think it's because even with that general design, I've still got a recurring pattern in each of how I create these modules and how I import and then initialize them. My instinct says we ought not to need that extra boilerplate in a highly dynamic language like Python, but I've yet to find any alternative that is neater in general. At least in the most simple cases this only adds a couple of extra lines (converting the decorators to an init function in each package/module, and then calling that function at the top level after importing the package/module), which is clearly better than the earlier, more highly connected designs.

</comment></comment></comment><comment><USER31> Can anyone suggest best practice alternatives to import side-effects?

<comment><USER32> Don't run any functions at the root level of your module.

Instead, if you really need long-lasting objects which get initiated once, then use an object, and put any initialisation stuff in it's `__init__` method. Then the module can be imported whenever, but your initialisation stuff is only called when the user of your library creates a new instance of that class.

For bonus points, make your classes able to be used with the `with ...` syntax, so then lifetime is kept to a minimum, and errors/whatever are dealt with by default.

If you really really need to monkey around and take control of the whole python interpreter (gevent, twisted, and possibly some GUI frameworks come to mind...) then don't do that at import time, do it with a `run_forever()` or `take_control` type function.

But yes, a virtualenv for every project does help a lot with not accumulation cruft.

<comment><USER33> I sometimes make calls to collections.namedtuple and other class building functions at the same time as the rest of my definitions.

<comment><USER12> If you pass verbose=True to namedtuple(), you can see that it's essentially just defining a new class. This is something that people already do at the top level of a module.

</comment></comment></comment></comment><comment><USER34> This is one reason why I prefer Haskell ;)

<comment><USER28> I think most statically typed languages don't care for this kind of shenanigans.

<comment><USER35> In addition to the many other languages listed here, Go allows side effects on import.

<comment><USER36> That's interesting. I can see the motivation... it's helpful to have modules come into the world fully initialized, and initialization often involves side effects.

Is Rust going to allow side effects on initialization?

<comment><USER35> No. Having module import perform side effects delays program startup unnecessarily and makes the semantics of the program depend on the order in which modules got initialized, which is confusing.

</comment></comment></comment></comment></comment><comment><USER37>
        $ echo 3 > /proc/sys/vm/drop_caches
        $ time python2 -c 'help("modules")' >/dev/null
            /usr/lib64/python2.7/site-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion 'g_type_from_name (name) == 0' failed
          import gobject._gobject
        python2 -c 'help("modules")' > /dev/null  1.58s user 0.23s system 69% cpu 2.626 total
        $ time python3 -c 'help("modules")' > /dev/null
        python3 -c 'help("modules")' > /dev/null  2.00s user 0.17s system 74% cpu 2.928 total
        $ time python2 -c 'help("modules")' >/dev/null
        /usr/lib64/python2.7/site-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion 'g_type_from_name (name) == 0' failed
          import gobject._gobject
        python2 -c 'help("modules")' > /dev/null  1.26s user 0.11s system 99% cpu 1.375 total
        $ time python3 -c 'help("modules")' > /dev/null
        python3 -c 'help("modules")' > /dev/null  1.75s user 0.09s system 99% cpu 1.852 total

perhaps your issue is having too many things installed?

<comment><AUTHOR> The laptop I'm working on was decent when new, six years ago, but is now not the fastest thing on the block. It has certainly collected quite a lot of things there (mostly from system packages), but it is probably just one or two things that are spending most of the time (excluding I/O time).

It's interesting; now that I've tried running it a few more times, `help('modules')` on my Python 2.7 is getting down to six or so seconds. (On Python 3 it takes around 0.15 seconds.)</comment></comment>


MWC hangs by a thread after Nokia, DT and other big names back out - <AUTHOR> https://techcrunch.com/2020/02/12/nokia-pulls-out-of-mwc-over-coronavirus-concerns/ <comment><USER2> What are these companies concerned about? There are 2 confirmed cases in Spain. Are they concerned attendees from China will bring the virus with them?

<comment><USER3> > Are they concerned attendees from China will bring the virus with them?

Yes.</comment></comment>


Ask HN: has anyone had success with odesk or elance? - <AUTHOR>

I have a PHP project I want to get built and don't have much money.  I'm thinking of using elance or odesk.  Has anyone had any success using these services?  Any pitfalls to look out for, or tips on how to succeed?  Anyone willing to say how much their project cost them? <comment><USER2> I've used both. ODesk for some HTML/CSS work and Elance for some help uploading content into a wiki and in a second project on Elance writing some Java.

My impressions:

* I had a bad experience on ODesk with HTML/CSS work. The person just wanted an easy buck and did a poor job. It would have been faster for me to do it myself. (This is just one anecdote, not an indictment of ODesk.)

* I had a good experience getting some Java code done on Elance. Why? Because I wrote a very strict interface that had to be extended and produce very precise results to be considered successful.

* I also had a good experience (again on Elance) getting a small wiki updated and cleaned up.

All projects probably cost me $500.

In general, I would avoid these places if you have someone local you can work with. If not, then you have to be __super super clear on what you want __. If there is any ambiguity then prepare for disaster. You'll need to have clear specs, or interfaces, with very clear success criteria for them to work with.

Sometimes I find writing clear specs and managing the outcome takes longer than just hacking it myself, which is why I don't use them much anymore.

<comment><USER3> I've had similar experiences. My first experience on Elance was phenomenal. I found someone in Baltimore who worked wonders with a PPC advertising campaign, turning a money-waster into a profitable venture. I thought it would always be that easy.

The next 2 were not so pretty. The first was a simple content writing task (provider from Texas). The first results were on time and of great quality. Then the delays started, then the excuses (sick pet, sick child, storms and power outages, death in the family). Finally I got no response at all for weeks, and when all the milestone dates had passed I got a refund.

The last one was so-so. The next was a simple design job (provider in Pakistan)...5 HTML templates for 5-page brochure sites. The provider's portfolio looked great, and the bid price was right. The designs were amazing, but the coding was terrible...I had to have it re-done by someone else. And the designs came quickly but the coding took forever. I'm guessing the guy was a designer who usually works with a partner for the HTML/CSS coding, and he tried to cut his partner out of the deal by doing it himself.

EDIT: Forgot to include this: the final solution to my problem was finding my own team in the Philippines. I lucked out when I found an amazing project manager who put a team of 10 developers, designers, content writers, and data entry guys together and he manages everything so I don't have to. Now we do transparent outsourcing for other web design companies.

<comment><AUTHOR> How did you find the phillipines company?

<comment><USER3> I put an ad on philweavers.net, which is a community for Filipino web developers. That's how I found the first guy. He then found the rest if the team. They all report to him, but they all work for me directly.

</comment></comment></comment></comment><comment><USER4> _In general, I would avoid these places if you have someone local you can work with. If not, then you have to be super super clear on what you want. If there is any ambiguity then prepare for disaster. You'll need to have clear specs, or interfaces, with very clear success criteria for them to work with.

Sometimes I find writing clear specs and managing the outcome takes longer than just hacking it myself, which is why I don't use them much anymore._

summed up, if you are not a developer already, your chances of a successful outcome are slim. I try to explain this to people all the time who have no idea of what they are getting themselves into.

It is not a place that you can "offload" your technical work without deep technical oversight. It is a site where an in-house or near-shore team can write a system, develop a plug-gable architecture, define interfaces, and solicit developers to build a bunch of plug-ins that the in-house team would have no hope of completing on time.

If you do not posses that technical talent, then you first need a onshore guy that can help provide technical know-how. He can then help you make decisions as to what will succeed on sites like that, or most likely will link you with a Eastern-European or Indian development house directly.</comment>