How Sift Science Trains Thousands of Models Using Apache Airflow - <AUTHOR> https://engineering.siftscience.com/sift-trains-thousands-models-using-apache-airflow/ <comment><USER2> I'm a big fan of Airflow. Used it to replace our "Python script run on a cron" batch ETL process - you have to conform to Airflow's model of DAGs and operators, but it's really good for monitoring a workflow and restarting it from a failed task etc.</comment>


Is There Anything Good About Men? - <AUTHOR> http://www.psy.fsu.edu/~baumeistertice/goodaboutmen.htm <comment><USER4> Summary: Men are both better and worst than women. They are most at the top and also the most at the bottom. More men in CEO position and also more men in prison.

He tied in the fact that a society needs as many womb as possible and a few penis would do the job. Basically, saying the more people in a society the stronger it is. With this fact men basically are expendable, thus encouraged by society to participate in high risk endeavors. With high risk comes high reward. Hence a few successful men at the top.

Also, he points to the fact men have lots of shallow relationship while women value intimate relationship. But having a many shallow relationship allows men be more innovative since concepts are passed between then quicker compared to women.

Women would form small groups of intimate relationship while men will form large group. With large group emerges synergy and the whole is greater than the sum of it's parts. Hence, men form cultural things like religion, universities, sport teams, etc. What comes with forming and creating them is men dominate them. Also, the system men set up are design that only a few can be rewarded and recognized.

There wasn't no real conspiracy by men to dominate.

<comment><USER5> _There wasn't no real conspiracy by men to dominate._

"There was no conspiracy" theory seems to have become as problematic today as all-embracing-conspiracy-theories.

In any social species, subgroups with common interests has colluded to different degrees. The collusion has been more effective when the group had dominant social power. Many societies have strictures which aim to keep women in-line and faithful to their husbands. It rather clear this is organized in the interests of the husbands (and not in the interests of the handsome vagabonds passing through...).

One can be certain there have been _many_ overt and implicit conspiracies among groups of men as well as among groups of women, now and through-out history. Just not a single one since men and women are each members of other groups as well.

<comment><USER6> Emergent collusion due to groups of people sharing a goal is not equivalent to a conspiracy. There is no such thing as an _implicit_ conspiracy: that's a contradiction in terminis.

Similarly, _a_ group of men conspiring to do something is not equivalent to a conspiracy for male domination. It's usually a conspiracy for _their_ domination; their maleness is incidental.

</comment></comment></comment><comment><USER7> A long, but interesting read. This line stood out as pretty interesting - I'd never given this sort of thing much thought:

"If a group loses half its men, the next generation can still be full-sized. But if it loses half its women, the size of the next generation will be severely curtailed. Hence most cultures keep their women out of harm's way while using men for risky jobs."

Overall, it's a pretty objective (and in-depth) view of the gender debate.

<comment><USER8> That reasoning only works if all members are still willing to mate. What if the remaining species are heart broken/distraught/ect and unwilling to mate? Do we force them to mate? How do our values and ideals change when this occurs? Do they change at all?

When we imagine this apocalyptic doom of population, and mankind, we imagine a perfect world in some much that we think everyone will breed with everyone. Some men my have a problem(ED), some women may not be able to bare children(thus the creation of fertility gods), and some-be it male or female- may not want to "do it" period(homosexuality). We may also still hold old world values/moral beliefs, such as monogamy, dear.

The quote posted by Obiterdictum[1] from Orwell, one of my favorite writers, is very intriguing to me. The part that stands out the most is:

 _''But this is a fallacy based on the notion that human beings can be bred like animals.''_ -One the topic of utilitarianistic gender values

Animals in captivity are usually force breed to maintain a linage or specific trait. Animals can build emotional bonds, however, in the wild we see very few mammal species that follow the same guidelines we find social acceptable[2]. For us to believe that every male will mate with every female we must also believe that every female will mate with every male, and vica versa.

Part of monogamy is breed into our subconscious, part is based on proximity, part is due to social context, and part is, well, emotional[3]. There are countless stories of widows never marring after their husbands are gone[see Penelope in The Odyssey]. We, many of us, enjoy monogamous relationships. Something about them brings us a joy that is very hard to put a value on. We can be tricked into them, stockholm syndrome, we can even be forced into them, rape, but we can rarely artificially create them, Truman show.

We must also asses the value of monogamous relation ships versus open relation ships(I am sorry I do not know the proper word for it); thinking that everyone will value them the same. I do not have enough data to pass judgment, although, the animal kingdom does seem to be doing well for itself ;).

Then we have to break down the proper breeding matrices. No one has said anything about age distribution in population. So are we assuming all the males/females too young/old to breed died OR are we not including them in the first count? To inspire discussion I will say we never included them in the first count...but then the situation doesn't really work as we have a new generation coming of age at some point; a boost if you will. Hypothetically, using the example given with a population size 150(aftermath) we are left with 100 women and 50 men. Breeding can be done in at least 1 day; a week at the utmost. Now those women are taken pretty much out of commission at the ~7,8,9 month. That leaves 100 women more vulnerable to starvation, illness, miscarriage, death giving birth. We also do not have the number of still births. What is the status of food? How did this population destruction occur? Too many variables.

For every perfect scenario there exists an imperfect one just as likely to occur as the latter. No person, be it male or female, is more valuable because they were born with a functioning set of genitalia. Value is always in the eye of the beholder[see currency, trade, marketing].

[1] <http://news.ycombinator.com/item?id=1635248>

[2][http://www.trinity.edu/rnadeau/fys/barash%20on%20monogamy.ht...](http://www.trinity.edu/rnadeau/fys/barash%20on%20monogamy.htm)

[3] <http://www.csulb.edu/~kmacd/346monogamyAnimals.html>

[4]<http://www.mythweb.com/odyssey/book01.html#notes5back>

I had to edit and remove some of my comment as it became more of a rant than an acceptable comment. The tl;dr version is, we are all different and thus behave differently. Some may be more willing to breed for the sake of it while others not so much, however, we can all be conditioned so that we are prepared for such a situation. I am not condoning this. I feel that place male and female on a scale is boarding the lines of eugenics- to which I am more opposed than I am for. Some men are strong, while others are weak. Some women are stronger than a weak man. Some men are cowards, others are not. Some women are brave, others are not. Measuring strength is easy, measuring other- not so physical traits- is harder.

<comment><USER9> > We may also still hold old world values/moral beliefs, such as monogamy, > dear.

I think monogamy spread because St. Augustine made it part of the Roman Catholic dogma that became the foundation for all Christian dogma, Christianity took over Europe, and then Europe colonized pretty much the rest of the world. I tend to think if things had gone differently, we'd be looking at a polygamous world.

(Edit: Did I just argue sexuality is a social construct? Oy vey...)

<comment><USER10>
      >I think monogamy spread because St. Augustine made it
      > part of the Roman Catholic dogma that became the
      > foundation for all Christian dogma

Hum, monogamy was the norm in the Roman and greek world anyway. And wasn't monogamy the common jewish habit, too? It predates christianity by centuries, anyway.

<comment><USER9> > Hum, monogamy was the norm in the Roman and greek world anyway. And wasn't > monogamy the common jewish habit, too? It predates christianity by > centuries, anyway.

Sort of - I'm under the impression that Greek and Roman monogamy was more "soft monogamy" so to speak. You had your wife, but then you were free to do whatever your like beyond that as long as you didn't break your duty to the family, yes?

Interesting, most of the patriarchs in the Bible were polygamous, it was a famous rabbi's teachings (forget which one) that mandated monogamy as the way. For instance, here's Wikipedia on Abraham:

"Jews, Christians, and Muslims consider him father of the people of Israel. For Jews and Christians this is through his son Isaac,[2] by his wife Sarah; for Muslims, he is a prophet of Islam and the ancestor of Muhammad through his other son Ishmael, born to him by Sarah's handmaiden, Hagar."

King Solomon was known for his many wives. I think hard monogamy - <USER2> prostitution, no bisexuality, no mistresses, concubines, or sexual relations with slaves - was largely established as the norm by Christianity. I know the Romans and Greeks definitely had other lovers and orgies and things like that, whereas once the Roman Catholic Church was established by Constantine this sort of thing was outlawed pretty quickly.

<comment><USER11> Just a minor point: Constantine didn't "establish" the Catholic Church in the sense of making it the official religion. Rather, he issued a decree which made it legal to practice Christianity.

<http://en.wikipedia.org/wiki/Edict_of_Milan>

<comment><USER9> This is correct, my mistake. Constantine de-criminalized Christianity, Theodosius made it the official religion of Rome at the Council of Nicea:

[http://en.wikipedia.org/wiki/Theodosius_I#Nicene_Christianit...](http://en.wikipedia.org/wiki/Theodosius_I#Nicene_Christianity_becomes_the_state_religion)

</comment></comment></comment></comment></comment></comment></comment><comment><USER12> Roy F. Baumeister recently expanded his original essay into a book:

 _Is There Anything Good About Men? How Cultures Flourish by Exploiting Men_

[http://www.amazon.com/There-Anything-Good-About- Men/dp/01953...](http://www.amazon.com/There-Anything-Good-About- Men/dp/019537410X/)

</comment><comment><USER13> As a (happy) father of three I feel the right to share a dirty little secret: taking care of young kids is boring as hell.

But somebody has to do it; in mammals it's only natural that it's the female, since she has to be around anyway in order to breast feed (which is not the case with birds for example: both parents can find and bring back food to the young).

So maybe the adventurous sailors explore the world simply _because they can_ ; not "in order to come back rich, and mate" but because, having mated, they can leave their family behind and go do something else.

<comment><USER14> "taking care of young kids is boring as hell."

There were articles about how couples with children are unhappier over here recently:

<http://news.ycombinator.com/item?id=1494011>

Yet I believe I'd be unhappy in my old age if I don't have children (not to mention several good points in that discussion).

I keep going back to the "experiencing self vs remembering self" talk many people here directed me to:

[http://sheshtawy.wordpress.com/2010/03/01/experience-self- vs...](http://sheshtawy.wordpress.com/2010/03/01/experience-self-vs- remembering-self-experience-vs-memory/)

The experiencing self is unhappier, but the remembering self is definitely happier :)

<comment><USER13> Like I said, I'm a very happy father (my kids are 5, 2 and 1). I love them (they helped me understand what "love" actually means). They are the joy of my life.

But spending too much time with them is not as fulfilling as actually doing something -- or even reading a good book.

There is no fulfillment in wiping a child's ass, or even feeding them. You do it because you have to, and I actually don't mind doing it. But as useful as it is for them, it's pointless for me.

That is not a problem as long as I have other things to do / to care about; it would be if they were the beginning and end of my day.

</comment></comment></comment><comment><USER15> Evolutionary psychology without lots of backing data means very little. The hypotheses are very fun to think of, but there's no guarantee that you've thought of everything that could have happened a thousand millennia ago(and you probably haven't). Baumeister brought up some very interesting points (that I'm sure riled up a lot of feminist feathers), but there are also holes in his argument, such as the two below.

First, there is no mechanism of action that backs up his claim that society uses men as expendable. It's easy to see that it would be adaptive (for the society itself) to use men this way, but there's no guarantee this will happen without a mechanism of action. In fact, society seems to have developed mechanisms (such as monogamous marriage) that act against allowing men to be disposable sources of semen.

Second, even accepting everything else, his essential claim that a woman can't increase the number of her descendants by acquiring wealth and prestige is clearly false. All she has to do is find a way to transfer the wealth/prestige to her son(s), who in turn can use it in the ways Baumeister indicated.

</comment><comment><USER16> "Men go to extremes more than women, and this fits in well with culture using them to try out lots of different things, rewarding the winners and crushing the losers."

Is there anything more fitting to the audience of this site? Does this fit in with the sex distribution here? (surely)

</comment><comment><USER17> > Today's human population is descended from twice as many women as men

Anyone know where that conclusion originates? It's kind of hard to google - <USER3> keep finding myself back at Roy Baumeister.

</comment><comment><USER18> Hey, I've devised a simple rule for determining whether articles are completely vacuous! Just divide the number of times the author uses weasel words like "most" by the number of references cited:

(38 most) / (0 references) = ∞ vacuousness!

<comment><USER19> He does refer to other "research", but since it is a transcript of a talk they aren't really explicit citations.

That aside, I observed a ton of hand waving in this talk, and I'm willing to bet it isn't backed up by the appropraite amount of rigor in his "research". In particular, I am concerned about how hard he tried to poke holes in his own theory, given the risks that might pose to his forthcoming (at the time of the talk) book.

Since we just had Cargo Cult Science posted the other day, this is a great time to think about this. I sometimes think that we (hacker news readers) might actually be more suseptible to accepting theories that haven't rigorously examined because our need to understand things is unusally great. So when we come across a theory that is internally consistent, not obviously wrong, and just sort of seems to make sense, we tend to reject any attempts to discredit the theory, lest we risk going from understanding to not understanding.

In other words, I think we sometimes value the feeling of understanding something over any utility that would come from actually understanding it.

</comment></comment><comment><USER20> I think a key takeaway from this is that while there may be fewer motivated/ambitious women in business or skilled work, the women who ARE motivated/ambitious are equally as capable as motivated/ambitious men and should be given the same mentoring and opportunities (probably their biggest disadvantage right now).

</comment><comment><USER21> This speech was posted to MetaFilter a few years ago and generated a much deeper discussoin than it seems to here. <http://www.metafilter.com/64034/The- Waw-effect> Definitely worth reading some less glowing discussion about the talk.

<comment><USER22> I mostly saw moaning about how evolutionary psychology is bullshit, and not much serious discussion about the merits of his claims.

I do agree though that it starts out good, and becomes less and less substantiated, but on the whole it is interesting.

<comment><USER23> Precisely my thought. People take a specific literal quote and use it as an attempt to refute his main argument - but you have to refute the statistics instead.

One argument was that the adventurous sailor would be subsumed by the cobbler or tavernkeeper in the mating game. True, but he wouldn't be subsumed by the beggar, now would he?

Another argument is to find a counterexample. "There were female scientists. Good female scientists. Your claim is now disproven!". Except for the fact that the claim is statistical and you just picked an outlier. There are a couple of main claims, which are quite objective:

Are males more extremely distributed than females? Is the minimum wage pertubing the mean wage (yes, it is - but significantly?) Is the absolute grade scale pertubing the results (yes, but by how much?) If all three are confirmed, you could say the societys obsession with the mean is what causes all the trouble.

I think your point about less and less substantiality is warranted. It is the first couple of paragraphs which has the most meat and is the most objective. It would be bad to dismiss them, if the latter parts are weaker. Yet, the weaker parts are the ones people will attack in an argument - they are the low-hanging-fruit. Especially if they have an agenda to be fulfilled in the gender wars. Though the more objective parts are the parts you should seek to refute.

I think the future will be even more interesting than now. In the most developed parts of the world, women surpasses men in higher education. Note that this fits the presumption: males have a more extreme ability distribution. This _will_ have a great impact. There will be less well- educated men for the well-educated women. And this is a first in the history (of Europe). It will be interesting to see how that one plays out.

<comment><USER13> > One argument was that the adventurous sailor would be subsumed by the > cobbler or tavernkeeper in the mating game. True, but he wouldn't be > subsumed by the beggar, now would he?

Well, if the beggar is Diogenes...

The main counter argument is that most world conquerors are not especially fertile; Napoleon had maybe six kids (max.); Alexander had two and was maybe gay; Hitler had zero. How would the "conqueror" trait have so much influence on humanity if conquerors did not father many children (and there were not enough successful conquerors to start with).

I try to develop a more substantiated analysis of this article here: <http://blog.medusis.eu/of-parrots-and-men>

<comment><USER24> Because 3 men from modern history are utterly insignificant to human evolution.

The argument is Evolutionary Psychology, which is a rather soft science. So is economics, but that doesn't mean it isn't useful.

However, in order to make it useful you have to think statistically about the behavior of humans in LONG LONG period of pre-history.

<comment><USER13> From the article: " _perhaps nature [designed men] to strive, mostly unsuccessfully, for greatness_ ".

Leaving aside the fact that nature does not design anything, if evolution is to select a trait that is "mostly unsuccessful", then this trait is to be incredibly successful when it actually succeeds.

And in fact, Genghis Khan is the exception; other well-known world conquerors are not exceptionally prolific, casting doubt on the possibility that "unknown world conquerors" (if such people ever existed?) would have been prolific enough and numerous enough to have an impact on the human species.

I'm well aware of Evolutionary Psychology, its premises and its shortcomings; it isn't "soft science" but rather "non-science" since it can't be refuted. Most of its claims may be right, or wrong, but that doesn't matter because _we have no way to tell_.

</comment></comment></comment></comment></comment></comment><comment><USER25> These are always interesting articles, but this one definitely slips into what I feel is a dramatic overstatement about the differences between men and women.

In particular, I've never agreed that men shoot for the moon while women are happy to live more modestly in a low-risk way. Have the people who say this ever been to Hollywood? The place is full of young women who risk everything to achieve that incredibly high status position of movie star.

Or, like the Dixie Chicks wrote in "Long Time Gone"

Me, I went to Nashville Tryin' to be the big deal Playin' down on Broadway Gettin' there the hard way Living' form a tip jar Sleepin' in my car Hockin' my guitar Yeah I'm gonna be a star....

Now, one place where I agree with the author of this article is that there there are most likely meaningful innate differences between women that may have resulted from evolutionary pressures, and that we should not be suppressed by a culture of political correctness from honestly discussing them, even when some of these differences favor men.

But the conclusions still seem very spotty to me, not at all consistent with my observations. The big about diapers? The men I know are deeply involved with their children, extremely paternal, and change a hell of a lot of diapers. Women may not start tech companies as often as men, but they are definitely willing to take great risks to seek status.

I don't mind the question, and I'm not at all offended by the answer, but I am not at all convinced. There's a lot more to this story.

</comment><comment><USER26> Does someone want to donate that guy a CSS file? Talk about making good content unreadable. Just a simple margin:20px would help!

<comment><USER27> Readability bookmarklet is your friend

<http://lab.arc90.com/experiments/readability/>

</comment></comment><comment><USER28> And men and women will always have different and distinct _priorities_ in life.

</comment><comment><USER29> Some general introduction for those, who really interested in such topics. [http://oyc.yale.edu/ecology-and-evolutionary- biology/princip...](http://oyc.yale.edu/ecology-and-evolutionary- biology/principles-of-evolution-ecology-and-behavior/)

btw, you may skip hardcore genetics in lectures from 2 till 8. ^_^

<comment><USER30> Two books I often recommend:

Geoffrey Miller, _The Mating Mind_

Sarah Blaffer Hrdy, _The Woman That Never Evolved_

They're probably the best, most thoroughly researched, and least political of the evolutionary bio books I've found that aren't primarily about hardcore genetics and heavily math-based game theory.

</comment></comment><comment><USER31> Absolutely fascinating. I guess mostly it comes down to XY and XX chromosomes. XY leaves more room for random chance. XX would be more moderate because it would presumably have more genetics to balance out its extremes.

<comment><USER32> No, it has nothing whatsoever to do with that.

<comment><USER31> Why don't you elaborate then instead of just giving some smug answer? Hacker News would be a lot better without pointless contrarians like you.

<comment><USER33> Go read a little about genetics. Your comment is too silly to even critique rationally.

<comment><USER31> Yeah, thanks troll. Translation: don't start conversations unless you're an expert...

</comment></comment></comment></comment></comment><comment><USER34> disclaimer; I have not read the link.

I'm bothered by this title. Had a water cooler chat yesterday and was talking about question structure. This came up, yes/no; "do you still kick your dog?". Say, no, and it implies that you used too.

The title seems to imply that we need to explain what's good/bad about men when the entire question is dumb.

</comment><comment><USER35> The start of the article totally lost me. Everyone thinks women are better? Really? I thought it was the general consensus that men are better at stuff that matters like math, programming, logic, leading countries and resolving disputes with reason, and women are better at soft stuff like art, sex, fashion and emotions.

<comment><USER36> Where I live, at least amongst the people I know, this is not the "general consensus". In fact, one of the most brilliant prime ministers of this country (United Kingdom) was a woman.

My point is though that you have to be careful when discussing such delicate matters. While an argument could be made that men tend to perform better at abstract reasoning and women tend to perform better at other mental tasks, small differences in wording can make a difference between offensive and objective.</comment></comment>


Make money with these crypto-based alternatives to Reddit - <AUTHOR> https://thedailyshitter.com/make-money-crypto-reddit-alternatives/ <comment><USER2> Great site name!</comment>


Ask HN: Does all this negative press affect Facebook at all? - <AUTHOR> I've heard about the 'end of Facebook' since the days of Diaspora. Their userbase and revenues have kept growing and their portfolio has continued to expand to include 4 apps with over a billion users.
Will 'breaking up' Facebook really have any impact on the industry? I was too young in the days where Microsoft went through a similar investigation.
I understand the backlash and don't use any Facebook products outside of Instagram.
I know many here and on reddit hate Facebook, so do many in the media and in politics. Yet, this is still a small vocal minority it seems.
Most people I know have mixed feelings about social media but do not dislike them enough to delete their accounts. <comment><USER2> Progressives such as Elizabeth Warren want Facebook to censor political ads by candidates that contain falsehoods. They believe that falsehoods are predominantly from the right. Zuckerberg has resisted such censorship, and I agree with him. I think conservatives may be warming up to Facebook.</comment>


Ask HN: Where are the articles or discussions about net neutrality on HN? - <AUTHOR> <comment><USER2> There have been many articles on net neutrality in the last couple of days, some with significant discussion:

[https://hn.algolia.com/?query=neutrality&sort=byDate&dateRan...](https://hn.algolia.com/?query=neutrality&sort=byDate&dateRange=all&type=story&storyText=false&prefix&page=0)

<comment><USER3> I was going to say there's at least two a day, but that's severely understating it.

</comment></comment><comment><USER4> For those of you who aren't concerned about Save Net Neutrality, This is what paying for internet in Portugal looks like, and they don't have it. We need to stop the government from killing net neutrality.

</comment><comment><USER5> [http://www.letmegooglethat.com/?q=Net+neutrality+site%3Anews...](http://www.letmegooglethat.com/?q=Net+neutrality+site%3Anews.ycombinator.com)</comment>


People Working in DevOps Should Read the Toyota Way - <AUTHOR> https://zwischenzugs.com/2019/11/06/why-everyone-working-in-devops-should-read-the-toyota-way/ <comment><USER2> Toyota, as a manufacturing organization, makes some of the best cars in the world.

Toyota, as a software writing organization, fucks it up so bad that people die. There was a single procedure in their ETCS system that was 1300 lines of C with no unit test plan and global variables out the fuckin wazoo. Whole ETCS had 10,000 globals.

[https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_...](https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf)

<comment><USER3> Jesus, reading that was completely horrific. It's total software gore and terrible practices, in critical car systems..

Accountability and proper practices just go out the window when dealing with invisible code, I guess. It's 2019 and we're still dealing with things like storing passwords in plaintext or overusing global variables. Sometimes I wonder what's the point of even being a programmer?

<comment><USER4> personally it helps with imposter syndrome, its like thinking youre a terrible writer and then reading some godawful bestseller like 50shades of gray and realizing "hey, i guess i cant really be that bad"

<comment><USER5> Yes, once you realize the level of imposterism and mimcry going on in software you realize 'hey I can do this too' it's empowering in a twisted way -- I suggest doing it from a position of first principles and reinvention rather than feeding the frenzy

</comment></comment></comment></comment><comment><USER6> I thought the Toyota Way/DevOps analogs had been done to death to a degree that they became practically a cliche. (Though admittedly I was attending a lot of DevOps events at one point.)

In general, this is a worthwhile read though. The lessons _can_ be overapplied in part because a lot of lean manufacturing approaches do have a lot of focus on inventory which mostly doesn't apply in a software development context. However, as the author notes, there are also philosophical aspects to the the Toyota Way which are relevant.

<comment><USER7> > inventory which mostly doesn't apply in a software development context

I heard a guy (a Factorio youtuber who was manager in his dayjob, haha) say that it did apply indeed, if you see inventory as "what has been produced but not shipped, stuff for which the value has not been delivered for", in software context that being unreleased features. It made a lot of sense, and ties all nicely into the notions of short iterations and release early, release often. That was interesting.

<comment><USER6> I don't disagree. Hence my "mostly" qualifier. One can certainly imagine processes that lead to a lot of unintegrated components, code that should be "done" but won't pass tests, etc.

It's still rather different from manufacturing inventory management but it's not totally unrelated either.

</comment></comment></comment><comment><USER8> I stopped believing in TPS when I saw how companies did terrible things with workspace/office organization and supply management under 5S methodology.

Basically, workers got uncomfortable, de-personalized workspace; no spare parts and materials and less inventory.

Maybe it is aestherically pleasing for the management, maybe its easier to audit, but no worker like to work in such system and many of the dogmas actively hurt productivity.

<comment><USER9> ...curiously, this has not had the effect of workers voting to unionize at any of Toyota's plants in the US, where they have that option.

Of course, as the managers of Suzuki put it when they were building out the CAMI site in Ontario...you don't adopt another company's production system exactly, any more than you would wear a suit tailored to someone else.

<comment><USER10> Doesn't stop programmers from arguing for a process because Google does it.

<comment><USER11> Especially when they are completely ignoring the fact that what works for Google usually works for Google because they have achieved a certain scale, not that Google achieved that scale by adopting these methods.

<comment><USER12> To be honest there are a lot of processes that don't work for google either but with their treasure chest of money and profits from an easily defended monopoly flowing in, it's really hard to argue that.

I can still remember all of the people who defended the brainteaser questions microsoft and google used to toss at interviewees. If I was right that they were bullshit why wasn't _I_ worth a billion dollars?

<comment><USER5> I must be in the twilight zone -- could this thread be the watershed moment where sanity prevails in software?

</comment></comment></comment></comment></comment></comment><comment><USER13> I would also say that anyone espousing The Phoenix Project should also read The Goal, on which it is based.

<comment><USER14> I really enjoyed the Phoenix project but I have to admit the practices they followed in the last 1/4 of the book were barely described and simply worked almost like magic.

<comment><USER13> I think they ran out of time or ideas by that point, and intended to follow-up with another book: The DevOps Handbook, which took them a long time to finally get around to writing.

</comment></comment></comment><comment> mr-ron I thought the Google SRE book was more actionable with a better set of principles to follow.

<comment><USER14> How much do they overlap conceptually?

</comment></comment><comment><USER14> I never found likers writing to be compelling. I'm sure the foundational ideas are great but I found other books about Kaizen were more interesting</comment>


Overview of Python dependency management tools - <AUTHOR> https://modelpredict.com/python-dependency-management-tools <comment><USER2> People always get up in arms about this, but as someone who has used Python as her daily driver for years it's really... never been this serious of an issue for me?

I have used virtualenv/venv and pip to install dependencies for years and years, since I was a teen hacking around with Python. Packaging files with setup.py doesn't really seem that hard. I've published a few packages on pypi for my own personal use and it's not been too frustrating.

A lot of the issues people have with Python packaging seem like they can get replaced with a couple shell aliases. Dependency hell with too many dependencies becomes unruly in any package manager I've tried.

Is the "silent majority" just productive with the status quo and getting work done with Python behind the scenes? Why is my experience apparently so atypical?

<comment><USER3> Let me start by saying: I love python, and I love developing in it. It's the "a pleasure to have in class" of languages: phenomenal library support, not too painful to develop in, nice and lightweight so it's easy to throw together test scripts in the shell (contrast that with Java!), easy to specify simple dependencies + install them. (contrast that with C!).

That said... if you work on software that is distributed to less-technical users and have any number of dependencies, python package management is a nightmare. Specifying dependencies is just a minefield of bad results.

\- If you specify a version that's too unbounded, users will often find themselves unable to install previous versions of your software with a simple `pip install foo==version`, because some dependency has revved in some incompatible way, or even worse specified a different dependency version that conflicts with another dependency. pip does a breadth-first search on dependencies and will happily resolve totally incompatible dependencies when a valid satisfying dependency exists.[1]

\- If you specify a version with strict version bounds to avoid that problem, users will whine about not getting the newest version/conflicting packages that they also want to install. Obviously you just ignore them or explain it, but it's much more of a time sink than anyone wants.

\- In theory you can use virtualenvs to solve that problem, but explaining how those work to a frustrated Windows user who just spent hours struggling to get Python installed and into their `PATH` is no fun for anyone. Python's made great strides here with their Windows installers, but it's frankly still amateur hour over there.

\- Binary packages are hell. Wheels were supposed to make Conda obsolete but as a packager, it's no fun at all to have to build binary wheels for every Python version/OS/bitness combination. `manylinux` and the decline of 32-bit OSes has helped here, but it's still super painful. Having a hard time tracking down a Windows machine in your CI env that supports Python 3.9? Too bad, no wheels for them. When a user installs with the wrong version, Python spits out a big ugly error message about compilers because it found the sdist instead of a wheel. It's super easy as a maintainer to just make a mistake and not get a wheel uploaded and cut out some part of your user base from getting a valid update, and screw over everyone downstream.

\- Heaven help you if you have to link with any C libraries you don't have control over and have shitty stability policies (looking at you, OpenSSL[2]). Users will experience your package breaking because of simple OS updates. Catalina made this about a million times worse on macos.

\- Python has two setup libraries (`distutils` and `setuptools`) and on a project of any real complexity you'll find yourself importing both of them in your setup.py file. I guess I should be grateful it's just the two of them.

\- Optional dependencies are very poorly implemented. It still isn't possible to say "users can opt-in to just a specific dependency, but by default get all options". This is such an obvious feature, instead you're supposed to write a post-install hook or something into distutils.

\- Sometimes it feels like nobody in the python packaging ecosystem has ever written a project using PEP420 namespaces. It's been, what, 8 years now? and we're just starting to get real support. Ridiculous.

I could go on about this for days. Nothing makes me feel more like finding a new job in a language with a functioning dependency manager than finding out that someone updated a dependency's dependency's dependency and therefore I have to spend half my day tracking down obscure OS-specific build issues to add version bounds instead of adding actual features or fixing real bugs. I have to put tons of dependencies' dependencies into my package's setup.py, not because I care about the version, but because otherwise pip will just fuck it up every time for some percentage of my users.

[1] I am told that this is "in progress", and if you look at pip's codebase the current code is indeed in a folder marked "legacy".

[2] I 100% understand the OpenSSL team's opinion on this and as an open source maintainer I even support it to some degree, but man oh man is it a frustrating situation to be in from a user perspective. Similarly, as someone who cares about security, I understand Apple's perspective on the versioned dylib matter, but that doesn't make it suck any less to develop against.

<comment><USER4> > struggling to get Python installed and into their `PATH` ... it's frankly > still amateur hour over there

But that has been solved on Windows for quite a while hasn't it?

Python installs the "py" launcher on the path, which allows you to run whichever version you want of those you have installed. Just type "py" instead of "python". Or "py -3.5-32" to specifically run 32-bit Python 3.5, or "py -0" to list the available versions.

<comment><USER3> It's gotten a lot better, but we still hit tons of issues with users who don't know what Python version they installed their application in. Oh and of course our "binaries" in Scripts/bin don't seem to show up in the PATH by default. So I get to tell people "py -3.8-64 -m foo" on windows, "foo" everywhere else.

This gets much much worse when a new version of Python comes out and we don't support it yet (because of the build system issues I mentioned). I spent several weeks teaching people how to uninstall 3.8 and install 3.7 before we finally got a functioning package out for 3.8.

<comment><USER5> I like Mozilla build system on Windows, you click "start-shell.bat" and it runs console. Python, mercurial, rust - just works, never checked PATH.

[https://firefox-source- docs.mozilla.org/setup/windows_build....](https://firefox-source- docs.mozilla.org/setup/windows_build.html)

</comment></comment></comment></comment></comment><comment><USER6> pip-tools is almost never mentioned because it's boring but great. I always default to it.

[https://github.com/jazzband/pip-tools](https://github.com/jazzband/pip-tools)

<comment><USER7> Yes !! I just create a Makefile target and pip-tools is all I need. I create a requirements.in and that is all. So far never feel that has to be more complicated than that. And when I want to upgrade a package I update the requirements.in if I need to and run `make -B` for this:

      default: requirements-develop.txt
        pip install -r requirements-develop.txt

      requirements.txt:
        pip-compile -v requirements.in

      requirements-develop.txt: requirements.txt
        pip-compile -v requirements-develop.in

I so nice to just write `make` than doing all the Poetry, Pipenv stuff, that honestly I feel is not adding nothing really really useful to the workflow.

<comment><AUTHOR> Does it pin versions of 2nd degree dependencies too? Like pip freeze would do? Also, when you remove a package, does it know to clear packages that were its deps and are not needed anymore?

<comment><USER8> pip-tools can do both of those, yes.

For the second, pip-compile computes the new requirements.txt (which is effectively the lockfile) from scratch, and pip-sync (not shown in that Makefile fragment) removes packages that are no longer listed there.

<comment><AUTHOR> Thanks a lot for the reply. I'll include it some time soon in the article update

</comment></comment></comment></comment></comment><comment><USER9> Every attempt to solve this problem in Python seems to eventually end up in a pretty terrible place. Pipenv got off to a great start but got slower and slower to the point that it was more painful to use than not. Poetry (which is still my preferred option) started off with something seemingly beautifully thought through, and very fast too. But after only a few version updates, it seems to be hitting the same problems Pipenv did. On one project I was working on recently I managed to screw up the Poetry.lock file, so I ran `poetry lock` and it took 18 minutes. I still have high hopes for poetry, but I spend way more time trying to work around its shortcomings now (v1.0.5) than I did when it was at version 0.10.0 two years ago.

<comment><USER10> I get that resolving dependencies is a SAT problem and inherently intensive; however, I don't understand why it's so much slower in Python. Is it just that all of these resolvers are implemented in Python (and Python is really that much slower than other languages?), or does Python require you to download an entire package just to determine its dependencies? In the latter case, that seems pretty dumb, right? Like as bad as exposing the entire interpreter as the extension interface, rendering optimizations and competing interpreters virtually impossible.

<comment><USER11> > does Python require you to download an entire package just to determine its > dependencies?

yes - the standard way of defining dependencies in Python is in setup.py, which has to be invoked as a Python script in order to work. this script may also need to read files from the rest of the project, so you do indeed need to download the whole package to determine its dependencies.

even if the Python community were to agree on a new configuration format tomorrow, there would still be a ton of packages out there that wouldn't migrate for years.

<comment><USER12> It seems that this information should cacheable after an invocation of setup.py, at least for an installation without any extras. And even with extras requested, perhaps.

Or is there any even greater hidden challenge from using setup.py?

<comment><USER13> setup.py can check the OS and pick necessary requirements. so it can have different dependencies in different OSes.

I've used it like this - [https://github.com/JaDogg/pydoro/blob/b1b3de38ac15b9254ef1be...](https://github.com/JaDogg/pydoro/blob/b1b3de38ac15b9254ef1bef636b6150e446272da/setup.py#L26)

</comment></comment></comment></comment></comment><comment><USER14> The missing ingredient to really, REALLY solve these problems once and for all is an authoritative decision to switch package formats and run the whole dependency resolution stack by the core python language contributor team.

I get backwards compatibility and open-source governance and bla-bla, but the reality is that this cannot be done by a third-party library author and needs to become part of the core stack, including proper support rather than just shipping a tool which covers 90% of cases. It's crazy that apart from venv and pip, nothing else comes with python and you're left on your own.

npm + the registry is part of node

apt-get + registry is part of a normal linux distro

budnler comes with ruby

This is a solved problem elsewhere. What we lack is a fully-supported, agreed upon, working DEFAULT choice, so people don't have to make their own choices. I don't know if not having that DEFAULT is a function of how the python community thinks or its diversity, but it's painful to watch. I've almost given up myself and seen many newcomers give up because of a trivial problem like this.

<comment><USER15> While npm comes with node, and bundler comes with Ruby, the governance of these projects/tools are separate from the language.

<comment><USER14> Yet someone has made the decision to do bundle them with the language and provide a default. I'm not suggesting we have common governance, but these decisions need to be made.

</comment></comment></comment><comment><USER16> Dependency management can be pretty overwhelming for a lot of people entering Python. This is especially true in the data science realm, where many don't have a SWE background. Even after you have selected a tool, it can be easy to use it in a poor way. I have recently written a short article on how I use conda in a disciplined way to manage dependencies safely: [https://haveagreatdata.com/posts/data-science-python- depende...](https://haveagreatdata.com/posts/data-science-python-dependency- management/)

<comment><USER17> Python dependency management of packages using C or C++ behind the scenes is really problematic and sometimes, the installation may fail. In this case, a solution is to use Conda or mini conda which provide many pre-compiled packages and also Clang C++ compiler.

An alternative way to allow people without software engineering background to play with Python data science and machine learning tool may be providing pre built Docker images with everything pre-installed which may save one from configuration trouble.

Docker is also useful for learning about new programming languages without installing anything. With just one command $ docker "run --rm -it julia- image", one can get a Docker image containing a GOLang compiler; a Julia language installation; a Rust development environment and everything else. Docker is really a wonderful tool.

<comment><USER18> Docker is definitely an interesting tool for that, but my biggest problems is that I have to teach them Docker, which is a totally new layer of abstraction they haven't seen before.

How do you approach this? How technical are people you prepare Docker images for?

<comment><USER17> You don't need to teach docker. All you need is providing a docker image with everything pre-installed such as Julia, R language, Python, numpy, pandas, Tensorflow and maybe Vscode. And also any Linux distribution, then one can just type "$ docker --rm -it -v $PWD:/cwd -w /cwd my-image ipython" For better convenience, it is better creating a command line wrapper or shell script that saves one from typing that such as $ ./run-my-image ipython. I don't prepare anyone, but I guess that if I knew anything about docker and was given a docker image with everything ready and pre-configured and also a shell script or command line encapsulating all docker command line switches, I would find it more convenient than installing everything myself or fighting some dependency conflict or dependency hell. So, docker can be used as a portable environment development. VScode, aka visual studio code, also supports remote development within docker containers with extensions installed per container. I am a mechanical engineer by training, but I found docker pretty convenient for getting Julia, Octave, R language, Python, Jupyter Notebook server without installing anything or fighting with package manager of my Linux distribution when attempting to install a different version of R, Julia or Python. This approach makes easier for getting bleeding edge development tools without breaking anything that is already installed. I even created a command line wrapper tool for using docker in this way that simplifies all those case: $ mytool bash jupyter-image; $ mytool daemon jupyter-notebook ...

</comment></comment></comment></comment><comment><USER19> > Pipenv or poetry?

If you used pipenv for a complex project with huge dependency tree, or used it for a long time, you definitely run into a blocker issue with it. That is the worst package manager of all, and probably the reason why Python has such a bad reputation in this area. It's because it's fundamentals are terrible.

Just go with Poetry. It's very stable, easy to use, has a superior dependency resolver and way faster than Pipenv.

<comment><USER20> Am I misunderstanding Poetry? Because it seems to me more suited as something for packaging your python code up ready to be pushed to Pypi? As in, starting up a project creates an init.py, and a python file referencing distutils: neither of which I need or want to do if I'm writing an app to go into a docker container.

<comment><USER19> The first use case is actually handling project dependencies. If I remember correctly, it couldn't build packages at the first time, so the "build" subcommand was introduced only later. It's the same type of package manager (with lock file) as other languages already had like Cargo, Bundler or NPM.

</comment></comment></comment><comment><USER21> If you're interested in the technical issues behind Python packaging, a recent Podcast.__init__ episode features three people working on improving Pip's dependency resolution algorithm. My use cases are simple enough that I've gotten by for years just using pip and venv with requirements.txt files, but it was still fascinating to listen to how package management is approached in more complex situations.

Dependency Management in Pip's Resolver: [https://www.pythonpodcast.com/pip- resolver-dependency-manage...](https://www.pythonpodcast.com/pip-resolver- dependency-management-episode-264/)

</comment><comment><USER22> I've spent far too many hours fighting with these tools in two completely different scenarios

* Developing and deploying production Python solutions

* Helping beginners run their first script

While it's great for beginners to use the same tools that are used in industry, I strongly believe that the problem nearly all of these tools face is that they can't decide whether they want to _manage_ complexity or _hide_ complexity.

You can't do both.

Some of them do a fairly good job at managing complexity. None of them do a good job of hiding it. The dream of getting Python to "just work" on any OS is close to impossible (online tools like repl.it are the closest I've found but introduce their own limtiations). I recently saw a place force their beginner students onto Conda in Docker because getting people started with Conda was too hard. If you're battling with the complexity of your current layer of abstraction, sometimes it's better to start removing abstraction rather than adding more.

That said, I'm also a happy user of `pip` and `virtualenv` and while I'm sure that many people can use the others for more specific needs, I think defaulting to them because they aim to be "simpler" is nearly always a mistake. I still teach beginners to install packages system wide without touching venv at first - it's enough to get you through your first 2-3 years of programming usually.

<comment><USER23> This is a good point about complexity. I started with pip + virtualenv, and I'd recommend pip + venv to anyone learning Python. venv is in the standard library, so there's official documentation for it.

I picked up Pipenv when a point-point release of a dependency broke a production deployment. Pipenv's dependency locking meant that I wouldn't get surprised like that again.

Part of why this topic comes up so much is the desire to run with a language before learning to walk with it, perhaps. I'm a big fan of Poetry, but I like it because I know what it gives me compared to vanilla pip and a setup.py file.

Installing dependencies at the OS level will get you far as a beginner. And when the time comes that you need a virtual environment, you'll probably know.

</comment></comment><comment><USER24> I've been working in python roles for some years now and I never understood why the python dependency tooling is so poor.

Pip feels like an outdated package manager, lacking essential functionality that package managers of other languages have implemented for years. For example, credential redacting in pip was only introduced in 2019, 8 years after its initial release!

Not to mention the global-first nature of pip (package is installed globally unless the user explicitly requests for a local installation). You can still install packages locally, but this only shows that pip was not built with environment reproducibility in mind. As a consequence, the need for additional environment tooling (like venv) arose, which increased the complexity of the local python setup.

Tools wrapped around pip are also under par. I cannot see why Pipenv is that resource intensive, leading to long and noisy builds (my machine gets close to exploding on a pipenv lock), with very fragile lock files. Debugging an unsuccessful locking in the CI of an enterprise project is a mystery that could take an entire week to solve. Its javascript counter-part (npm) does the exact same thing, faster and with less CPU usage.

Trusting the OS community, I understand that there would be very good reasons for Pipenv to perform like this, but as the consumer of a package managing tool all I see is the same generation of file hashes I see on npm, but with npm doing it way more efficiently. I really see value in the principles that Pipenv is promoting, but to me the developer experience of using it is suboptimal.

</comment><comment><USER25> Serious question: What is the difference between virtual environments and just having several Python installs like:

      /home/foo/a/usr/bin/python3
      /home/foo/b/usr/bin/python2

Python is so fast to compile and install that I just install as many throwaway Pythons as needed.

I do not recall any isolation issues between those installs, unlike with conda or venv, which are both subtly broken on occasion.

But I dislike opaque automation in general.

<comment><USER26> That basically _is_ what a venv is, an entirely separate Python install. Some files are linked rather than being copied, but it looks the same. venv gets you a couple extra conveniences, like the activation script.

I wouldn't call venv "opaque automation," there's not much magic going on there.

</comment></comment><comment> globular-toast The trouble is these tools all do different things and aren't really comparable. I wouldn't even include Docker in this kind of thing as it doesn't really do anything on its own.

For me, there are two main choices today:

* An ensemble of single-purpose tools: pip, venv, pip-tools, setuptools, twine, tox,

* An all-in-one tool, for example Poetry, Pipenv or Anaconda (or Miniconda).

I prefer the former approach, but if I had to choose an all-in-one tool it would be poetry.

<comment><USER18> I agree with you that Docker should not be there, but the reality is that people us it to replace some other tools (like venv).

I wonder why you prefer the former approach.

<comment><USER27> On the contrary, thanks for having included Docker in that list. It's the obvious answer to so many problems (developing, running and deploying apps, replicating deterministic Python environments, not installing linux dependencies required by Python packages directly on your machine, and so on).

BTW, to comment one of the point you made in the article, it's not that hard to run CUDA inside a container. It's less straightforward but quite well documented. You basically need nvidia-docker [1] on the host and start your containers with the runtime 'nvidia'. docker-compose still doesn't support it officially but there are workarounds. [2] I'm running it on ~50 instances in production and automated all the setup with ansible successfully.

[1] [https://github.com/NVIDIA/nvidia- docker](https://github.com/NVIDIA/nvidia-docker)

[2] [https://github.com/docker/compose/issues/6691](https://github.com/docker/compose/issues/6691)

<comment> globular-toast Why thank him for including Docker if it's the "obvious answer"? You're already using it, that's great.

Docker doesn't do anything Python specific on its own. It can be part of a pipeline but only with support from the Python specific tools which is what should be discussed in this kind of article.

</comment></comment></comment></comment><comment><USER23> I think this is a good basic overview of the dependency management landscape. I have a few things to add.

One is that because Python has been around for so long, it's easy to find outdated or conflicting advice about how to manage Python packages.

I think it's important to stress that pyenv isn't strictly a dependency manager, too, and depending on your OS, isn't necessary. (Supported Python versions are in the AUR[0].)

A lot of pain from Python 2 -> 3 is that many operating systems were so slow to switch their default Python version to 3. Unless something has changed in the last month or so, Mac OS _still_ uses Python 2 as the default.

It's a shame to see Python take a beating for OS-level decisions.

[0] [https://aur.archlinux.org/](https://aur.archlinux.org/)

</comment><comment><USER28> > [Pipenv] loads packages from PyPI so it does not suffer from the same > problem as Conda does.

False. Conda manages packages installed from PyPI. This is discussed under the Conda section, so I'm surprised the quoted line wound up in the article.

<comment><USER18> Hey xapata, thanks for pointing this out.

Any chance you could give me some reference so I can fix it in the original article?

<comment><USER28> [https://docs.conda.io/projects/conda/en/latest/user- guide/ta...](https://docs.conda.io/projects/conda/en/latest/user- guide/tasks/manage-environments.html#pip-in-env)

Basically, use Conda to manage environments, use Pip to install packages. If you're using Conda to install anything, do that first.

</comment></comment></comment><comment><USER29> pipenv is terrible. poetry ain't there yet. seems author forgot to mention the problems with pipenv and poetry. virtualenv + pip will take you far. then to reproduce pipe to requirements.txt. poetry etc are still using pip under the hood

</comment><comment><USER30> I've alwyas used conda since I use the scipy stack. Can anyone clue me in if I can instead use pipenv and it will download all the requisite binaries etc. ?

</comment><comment><USER31> Could someone summarize the issues with Pipenv (and by Extension Poetry). Been using them happily for the last few years, didn't know people disliked them.

With Pipenv, last year ownership switched from the Request's lib owner to the Pypa, so more or less an officially blessed solution.

The only downside on this thread that I could understand so far is that it might be slow to install dependencies on larger projects, can't think of anything else.

<comment><USER32> I've been using pipenv happily for a few years now, but on projects that don't have a huge number of dependencies (Django, DRF, MySQL/Postgres, AWS, Kubernetes, a few other random libraries), and haven't seen too much slowness. I suppose data-science projects with large dependencies that pull in many other dependencies might have more issues.

I've been the person to document setting up development environments for others in macOS (and Homebrew) with a view to deploying in Linux, and pipenv (and pyenv, and Docker/docker-compose for setting up software context/datasets) definitely overall minimized the complexity for those configuring their dev environments.

(EDIT: documenting dev enviroments)

</comment></comment><comment><USER33> After hitting some weird PyInstaller bugs, I gave up and started compiling Python myself. One interpreter for every project. Shell scripts to set the paths. All libraries go directly in site-packages, not some other layer. A little more complicated at the outset, but this approach has yet to let me down. And compared to the nightmares I was trying to fix, building Python is dead easy.

<comment><USER34> I can get PyInstaller working with venv, but not conda.

</comment></comment><comment><USER35> Reading this makes me grateful that I can get away with just _apt-get_. I wonder how prevalent this is, since not every project needs specific or latest versions of the runtime and libraries, only a minimum. Some are just plumbing tools that stick to the stable core, and the Python 3 ecosystem has been mature for enough years that older distro packages are still useful and capable.

</comment><comment><USER36> Personally I just try to avoid Python development because I hate feeling like I'm dealing with what should be a solved problem. Recently I had to work with an outdated Python Tensorflow framework and the only way we could get it to work correctly across different dev and deployment machines was with a fat Docker image that took hours of head scratching to build. It was miserable.

</comment><comment><USER37> Using conda for environment and dependency management works really well.

<comment><USER38> Conda is a great tool.

But it forks the ecosystem, twice:

First, Conda packages have to be maintained separately from PyPI packages.

Second, the "default" repo is maintained by Anaconda, but the community maintained Conda Forge repo is also separate, and officially the packages in one are not compatible with the packages in the other. (In practice they usually play nice).

Having three incompatible package repos is not ideal.

</comment></comment><comment><USER39> pip-tools should really be included here. That's the single-purpose tool that handles environment reproducibility, if you're going with the single-purpose- tool route of pyenv + pip + venv instead of the all-in-one route of poetry/pipenv.

</comment><comment><USER40> Tbh, this is one of the reasons why I moved away from Python to Ruby for my side projects.

<comment><USER28> Are bundler, rvm, and rbenv not as confusing?

<comment><USER36> Definitely not. In Ruby I can immediately understand how to run a correct copy of any project because they all use bundler. Furthermore rbenv makes switching between specific Ruby versions for those apps trivial.

</comment></comment></comment><comment><USER41> You know how pathlib does for paths and files? Python needs something like that for distribution/versions/import hacking, eh?

Glyph (of Twisted fame, whence pathlib IIRC) pointed this out ages ago: model your domain [in Python] with objects.

</comment><comment><USER42> Oof, that footnote:

> It's 2020, but reliably compiling software from source, on different > computer setups, is still an unsolved problem. There are no good ways to > manage different versions of compilers, different versions of libraries > needed to compile the main program etc.

I wonder how much stuff like this has to do with python's popularity. When I have opaque issues like "libaslkdjfasf.so is angry with you and/or out to lunch and/or not doing expected things," it's the most frustrating part of programming. I'd pay devops people infinite money to not have to deal with installation/setup issues anymore.

<comment><USER18> I think this is not a problem specific to python packages, but a general problem of how we compile C/C++ software. There is no concept of packages and compiling one thing often requires installing a -dev package of some other library.

The issue is that lack of packaging C/C++ world spreads to all other communities that depend on them.

</comment></comment><comment><USER43> Everytime I read an article about all these tools I really can't help but think what would happened if Linus would have taken over the desktop. All the tools really largely seem to try to poorly replicate Linux package management and the fact that because of this devs now don't care anymore about api stability and not always building against the latest and greatest.

I admit a pyenv is nice for testing against different python versions if necessary. But on my Linux systems generally fine with just installing system packages and doing pip install --user for the odd package that is not in the repositories

<comment><USER18> I think that works when you use Python cli tools, but not when you're working on 5 different projects, each running different python version.

<comment><USER44> Outside of Python 2/3 differences, are Python interpreters not backwards compatible?

In other words, while obviously a program written for 3.3 won't work in 2.7, but will a program written for 3.3 fail to run in 3.8?

If it runs fine, why the need for multiple interpreters? I'd think you'd get by just fine by having the latest 2.x and 3.x installed.

<comment><USER45> Because underscore-functions aren't truly private, I have once seen an upgrade from 2.7.8 to 2.7.13 fail. A commonly-used package was importing one from a core python module.

</comment></comment></comment></comment><comment><USER7> I just use Pyenv and pip-tools. Create a requirements.in and Makefile targets to build the requirements.txt based on it. So far I haven't find an sceneario where that combination is detrimental.

</comment><comment><USER46> This is a good overview of something that took me an annoyingly long time to learn. My personal preference is to keep things simple with pyenv, venv, and pip.

Tangentially related is the tool tox [1], which is often used to run a test suite inside of virtual environments created by venv, on multiple versions of Python managed by pyenv.

Now if only setuptools could work well without hackery...

[1]: [https://tox.readthedocs.io/en/latest/](https://tox.readthedocs.io/en/latest/)

<comment><USER47> Wouldn't you still need something like pip-tools to lock down subdependencies and handle conflicts?

<comment><USER48> Plain old pip and venv can do that. just "pip freeze >requirements.txt" and elsewhere "pip install -r requirements.txt", inside venvs.

<comment><USER47> I think that will end up installing the subdependency version of whatever is last in the requirements.txt. You need a dependency resolver to deal with problems with conflicting versions.

More details here: [https://medium.com/knerd/the-nine-circles-of-python- dependen...](https://medium.com/knerd/the-nine-circles-of-python-dependency- hell-481d53e3e025)

<comment><USER48> pip handles the simple cases: if you install a new pkgA that depends on 'pkgB<3', it installs the latest appropriate version of that, e.g. 'pkgB==2.5.6'. This works even if you already installed 'pkgB==3.0.2', it will uninstall that first. The problem is if some 'pkgC' depends on 'pkgB>=3'. You probably want for pip (or similar) to figure out that an older version of 'pkgC' is compatible with 'pkgB>2'.

But I actually don't want it to be too smart. Better to keep your dependencies minimal and explicit, and manually specify older 'pkgC' if you need to. I have a few non-trivial services in production, the most complex one with 16 total dependencies + sub-dependencies. That is quite manageable.

So, I strongly recommend manually curating the most appropriate versions of the few tastefully chosen dependencies you really need. Then, pip+venv can easily reproduce that exact set of dependencies anytime. I also do something very similar to this with C applications, and Go. Sub-dependencies should be a big factor in how you choose your direct dependencies.

<comment><USER47> The problem with doing things this way is that you're not going to know there's a problem until there's an issue in your tests (hopefully) or production. You'll eventually install something new, it'll update some subdependencies to a version that another library doesn't support, and then things get broken. Pip-tools is easy to use and it tells you there's a problem before it's too late.

</comment></comment></comment></comment></comment></comment><comment><USER49> It's worth noting that on Linux it's slightly different because most of the popular libraries can be installed with the system package manager (no problem of dependency management, updates, ...), I rely on alternative solutions only when I want to use a version of a library different from the one shipped with the package manager (which is not that frequent with fast paced distros like Fedora) or when the library is not packaged.

<comment><USER50> No. Don't mess with os packages and your dev setup. On a very small scale I can adapt to use the os package version. But when you start to work on 4, 5, 15 projects, each of them that need to work with some specific version of some package, you need to detach from the os

<comment><USER49> And my point is, use the system packages whenever you can, they are there for a reason, while developing atbswp[0], I faced a situation with wxPython, where the only package available on Linux was the one provided by the package manager, it's a known situation[1], the workaround I used was, instead of "detaching from the os" to work with the OS, more specifically, install the package with the OS, then copy the wx folder from the system's site-package to the venv's site-package[2].

0: [https://github.com/rmpr/atbswp](https://github.com/rmpr/atbswp) 1: [https://wxpython.org/pages/downloads/](https://wxpython.org/pages/downloads/) 2: [https://github.com/RMPR/atbswp/blob/master/Makefile](https://github.com/RMPR/atbswp/blob/master/Makefile)

</comment></comment></comment><comment><USER51> For some of the problems that Node.js and JS at large have with a centralized package manager, I for one am very happy that it's not in the python situation. 100% of the packages I've tried to install in the last 3+ years are simply `npm install PKG`.

<comment><USER28> Do popular Node packages rely on C and Fortran?

<comment><USER52> I don't recall any popular Node package relying on Fortran, but there are two popular packages that rely on C: fsevents and node-sass

It works on macOS and Linux without any issues. Windows usually requires some extra steps to setup node-gyp

</comment></comment></comment><comment><USER53> Python dependency management is much like Python 2 to 3, a mess. It's shocking to think pip and pipenv are so widely used and still such terrible tools.

</comment><comment><USER54> Great article Mario! Was a pleasant surprise to open HN and find this

<comment><USER18> Thank you :)

</comment></comment><comment><USER55> pipenv also loads any .env file it finds in the directory, so it is a little more convenient to use than poetry, so I didn't make the switch.

</comment><comment><USER56> I like this overview. However, it points to a fundamental problem with Python environment, going much against its own credo:

"There should be one - and preferably only one - obvious way to do it." \- The Zen of Python; see also [https://xkcd.com/1987/](https://xkcd.com/1987/).

When it comes to the package, environment and dependency management, I think that ironically JavaScript environment is light years head, vide: [https://p.migdal.pl/2020/03/02/types-tests- typescript.html](https://p.migdal.pl/2020/03/02/types-tests-typescript.html)

</comment><comment><USER57> Anyone installs Conda on my shit I hit the roof... I sympathize with the individual but cannot tolerate the act.

<comment><USER58> Can you elaborate?

<comment><USER59> Perhaps I can guess some causes of the irritation. Let me start by saying that _on Windows_ conda is probably an improvement.

On Linux, however, I do not see much benefit, unless you frequently install large binary C library based packages.

To me it feels cleaner to compile these packages from source. You are sure to have no glibc mismatches etc.

Conda, despite its advertising, _does_ have library issues. C Libraries are shared between environments, compiling inside an environment can lead to surprising results when stale libraries are in the miniconda path.

All in all, it feels like a second OS shoehorned into the user's home directory. Compared to apt-get it is really slow and bloated.

It feels too intrusive on a Unix system.

Also, I'm not sure if the repositories are secured in any meaningful way.

<comment><USER57> This is a perfect summary which exactly captures my frustrations.

I should have said that on Windows Conda is a much more understandable choice.

</comment></comment></comment></comment><comment><USER60> anyone with insight into the Debian/Ubuntu packaging care to comment ?

</comment><comment><USER18> It's 2020, but the python community still has not converged to a small set of sane solutions.

It seems to me that Ruby, PHP, JS, and Rust communities have solved the problem.

<comment><USER61> I'm only familiar with Python, Javascript and Rust. It seems to me that Rust is the only one that has "solved" this problem.

I dont think there are any real Python devs who thinks dependency management is solved. However, why would you claim Javascript has a good solution? The inconsistencies between node and web dev is odd at best. Babel compilation is annoying and slow. Are we even standardized on webpack yet?

Can anyone say with a straight face that getting a new JS dev caught up on what all these different parts to compile a JS program is a solved problem?

I dont fault either Python or the JS ecosystems. As pioneers in dependency management, there was a lot of trial and error. New languages like Rust benefited from it and that's ok.

<comment><USER62> One thing node.js got right is the module resolution logic.

Node doesn't even need something like venv since module lookup is always local.

Also no problem with dependency hell. Each dependency can have its own private dependencies, even different versions of dependencies shared by sibling modules. Tools like yarn/npm can remove duplicates across a project.

<comment><USER63> It's a lot more of a pressing concern when you have an average of 1,200 dependencies per project.

(I'm actually not sure if it is the average but my anecdotal experience is that it's an order of magnitude higher than python, and 1200 wouldn't be unusual).

<comment><USER64> That's probably quite accurate for front-end projects which pull in a ton of packages for 1. A compatibility layer wih older browsers. 2. Build tooling.

Node projects tend to have a lot smaller dependency graphs.</comment></comment></comment></comment></comment>


Payback time: First patent troll ordered to pay "extraordinary case" fees - <AUTHOR> http://arstechnica.com/tech-policy/2014/06/its-payback-time-as-findthebest-wrests-legal-fees-from-patent-troll/

<comment><USER2> I knew nothing about FindTheBest before reading this but it gives me a warm fuzzy feeling towards them. Newegg also recently made the news for beating a patent troll. People who stand up to this make the world better for all of us.</comment>


Lenovo Unveils ThinkStation P410/P510: Intel Xeon E5 v4 with Up to 8 Cores - <AUTHOR> http://www.anandtech.com/show/10427/lenovo-unveils-thinkstation-p410p510-intel-xeon-e5-v4-with-up-to-8-cores-nvidia-quadro-m6000 <comment><USER2> Does it come pre-trojanned for your convenience, like their laptops?</comment>


Godot Engine Opinions by Tynan Sylvester (author of Rimworld) - <AUTHOR> https://old.reddit.com/r/godot/comments/8mhzfo/tynansylvester_of_rimworld_fame_is_evaluating/ <comment><USER2> I see lots of positive feedback for Godot from developers, but there aren't many high profile games made with it. I looked through the showcase, and the only ones I recongised were "Deponia" (the iOS port) and "Dog Mendonça & Pizzaboy".

So it's hard to judge, I worry that it could be one of those engines that looks good initially, but has limitations that are only apparent once you're already waist-deep into a large project.</comment>


The iPad's Dirty Secret - <AUTHOR> http://codesketch.com/2010/05/the-ipads-dirty-secret/

<comment><USER3> I remember that the iPod's dirty secret was that you had to pay big money for a battery replacement since it wasn't user serviceable. Other dirty secrets for popular consumer products include things like child labour being used to build sneakers or unsafe factory/dormitory combinations that kill hundreds of people when the factories catch fire.

I get the sense that the phrase "dirty secret" is being watered down in the same sense that the word "evil" is being watered down to simply mean "something I disagree with."

<comment><AUTHOR> It was meant as a rebuttal to Steve Jobs proclaiming the iPad offers "the best browsing experience you've ever had" which is simply not true.

While it's true that other "dirty secrets" (some of which you listed) are more important, it doesn't change the fact that is one of Apple's dirty secrets.

<comment><USER4> To me, "Dirty little secret" implies a factual piece of information that Apple is knowingly hiding from the public.

"I don't like the iPad browser because it doesn't have tabs." is NOT a "dirty little secret".

<comment><USER5> Yeah, but 'only 256 mb RAM' might be for a lot of people that have got used to more.

<comment><USER5> What? It was news to me from reading the article. I had assumed it would have about the 1gb which is a common spec on $300 netbooks now. I think people are overreacting a bit to the the title...it's just one of those headline cliches, like 'anatomy of a flame war'.

</comment></comment></comment></comment></comment><comment><USER6> On a side note: The author makes the assertion that anyone can be tossed an iPad and figure out the interface on their own... I challenge that assertion. I've taken a few relatives through the process of getting iPhones up and running, and I would claim that the interface is more difficult than the standard desktop metaphor.

At some point an article floated through here suggesting that the iPhone interface isn't simple. Rather, the author argued, it's the fact that it's very _fun_ to use that contributes to its success.

I can't find the article right now, but I do agree with the author. The iPhone/iPad lacks affordances and all sorts of cues about what the hell you're supposed to DO with the thing. Again, my experience is that folks find it incredibly unintuitive (although they enjoyed it immensely when they 'got it').

I recognize my sample size is extremely small. However these constant assertions that Apple has managed to create an instantly usable and discoverable user experience are based on even more tenuous science. Does anyone have any hard data or usability testing they'd like to share? :)

<comment><USER7> Donald Norman (founder of Human Interface Group at Apple) once said [paraphrasing] 'give people something beautiful, and they'll assume it is simple'.

I can't find the original quote, but he said it again on Canada's BNN a few weeks ago.

<comment><USER8> It's not quite what he's saying. At least not if you read his book, _Emotional Design_. There have been research studies that show that beautiful things are perceived to work better -- after real usage, too, not just "Look at this picture. How do you think it will work?"

Aesthetics are so very important!

</comment></comment></comment><comment><USER9> Instapaper solves your reading later problem nicely. I'd give it a try - <USER2> simple bookmarklet and you can read whenever, wherever you want.

<comment><USER10> Instapaper is nice and usable, but for this it is also a kludge.

It forces the user to pick ahead of time which things they will and won't want to read later, instead of the simpler possibility. That being, letting the user just stop at any point in the interaction, confident in the assumption that the machine will take care of whatever annoying details are necessary for the page to still be there when they come back to where they left it.

Having to pick ahead of time isn't all that onerous, so the work-around is a pretty good one. It also has the benefit that you can pull it up on another machine, so that is nice. However, that too could be done more simply by having all of the user's browsers sync the their histories.

<comment><USER11> Well, in fairness, picking out which pages he wanted to read was one of the scenarios in the article, so even though your point is valid, Instapaper would solve that particular part of the problem.

</comment></comment></comment><comment><USER12> Is this because the iPhone OS doesn't use paging? It seems like this could be handled from the software side of things: the HD is solid state, and though sacrificing a second of time to switch to a web page that's been cached in the HD isn't _Apple-gorgeous_ it's certainly faster and nicer to see than a blank white page and a twirling loading symbol.

<comment><USER13> It's just the problem with the interface design paradigm that says a user should be able to do anything immediately without instruction. If you want to cache, you need to have an understanding of how the Internet works, so it's "too hard" for "most people" to understand or care about.

<comment><USER12> You're missing my point: the OS should be able to cache the (up to _n_ ) pages that haven't been explicitly closed by the user in the HD, and when there's a page fault in main memory when a user tries to access the ones the 256MB can hold, the OS should go to the disk and grab that page, and swap out another page that's currently in RAM with the requested page.

Bam, no re-downloading the page. Should be over in under a second (depending on the memory used in the iPad.)

<comment><USER14> If they made an assumption on the usage pattern of the drive they could buy cheaper parts that were not expected to be used as often. If there is no wear leveling built into the drive then it could reduce the lifespan of the drive. I am all for swap but I think they chose to omit the swap space for some other reason that I cannot think of right now.

<comment><USER11> The entire device doesn't have to use swap for Safari to save the state of a particular "tab" to disk.

</comment></comment></comment></comment></comment><comment><USER15> I think most people would like to see page/tab management and caching improved in mobile Safari. It's hardly a "dirty secret". Depite that, I still love browsing the web on the iPad. It's like you're flying over your web pages.

</comment><comment><USER16> This isn't the most well thought out post. I can admit that having only 9 open tabs is annoying and it uses an arbitrary method of closing previously opened tabs (or lets call them new windows, because they're not exactly tabs). But I guess this is something we'll just have to live with.

Instapaper is an app that's about $5. It solves all my offline reading needs for me. I've never looked back since.

</comment><comment><AUTHOR> The lack of tabs didn't surprise me, but the way the iPad OS poorly handled memory management did.

<comment><USER17> That's not poorly handled memory management, it's good handling. Poorly handled would be to give you an out of memory error, or make the other pages disappear completely as if you'd never opened them at all.

<comment><AUTHOR> I would have to disagree. My Droid also has 256 mb of memory, and I can have more pages open at once without it having to flush pages. This is still true with Pandora running in the background.

Overall I'm a fan of the iPad, it just falls short in this area.

<comment><USER18> What is the screen resolution of the droid? What is the screen resolution of the ipad? If there is a big difference then it may be because the software keeps bitmaps of rendered screens laying around for a quick switch.

<comment><USER11> The same thing is pretty much true on the iPhone as well. In theory, it keeps pages in memory, but in my experience it reloads the page way more often than I would prefer. It is definitely on my wish list for future Safari updates, once they have consolidated the iPhone OS to have the same version on both devices.

</comment></comment></comment></comment></comment><comment><USER19> I actually love this limitation. My ADD has plummeted and I don't feel like I'm so much less effective.

<comment><USER20> Yeah, there may be ways to overcome RAM constraints but iPhone OS represents opinionated computing that apparently values single-tasking on the part of humans.

Of course, it will value human single-tasking until it no longer does, at which point Apple will go to great lengths to show how many more tabs iPhone OS can keep open than other mobile OSes.

</comment></comment><comment><USER21> Perhaps the thinking behind non-caching is that people will forget to clear their cache, so why not do it automatically? It might be best to leave that as a default for people who don't care but offer a cache preference in Settings for people who know what that is, no?

</comment><comment><USER22> Breaking old habits is hard but sometimes worth it. Is opening a bunch of background tabs a really efficient way of doing things? On a PC it works if you have a high enough resolution to display all the open tabs and enough RAM/CPU to quickly switch between them. On mobile devices it pays to focus on more efficient ways of doing things because you're dealing with limited resolution, potentially slower 3G networks, limited CPU/RAM/battery, etc. That being said there _are_ browsers on the App Store that support tabs.

<comment><USER23> I would argue that tabbed or multi-windowed browsing is most important on limited resource and low bandwidth devices. The main advantage of opening a tab/window in the background is not having to wait for it to load; the longer the wait, the more that functionality matters. I used the ability to open links in a new window extensively as soon as browsers offere that functionality, which was probably on a 133MHz machine with less than 256MB of RAM on a 14.4kbps (or 9600 baud...not sure) connection if I remember my computing timelines correctly.

</comment></comment><comment><USER24> I'm really hoping there is a fix for this in 4.0. After all, 4.0 does allow apps to 'state save' - I can't imagine a cached webpage would use much more ram than a cached app.

The Safari refresh thing has always bugged me on my iPhone, but I think it's slightly more acceptable since I have an always-on connection and just have to wait 10 seconds for a page to reload.

Perhaps Apple's way of getting me to cough up the extra $130 for the 3G...

</comment><comment><USER25> I don't see how this UI decision — a screen of pages vs. a row of tabs – could be related to a hardware limitation. All other things remaining the same, why couldn't the switcher look like tabs instead of pages?

Page caching is the real issue, but I think this could also be addressed by software if Apple decided to.

Of all the things that could be considered deal breakers for the iPad, this is among the least of them.

</comment><comment><USER26> Rentzsch (developer of Click To Flash) posted his thoughts on this a bit ago. [http://rentzsch.tumblr.com/post/382362022/persistent- mobiles...](http://rentzsch.tumblr.com/post/382362022/persistent-mobilesafari- pages)

I agree it is very frustrating though. Hopefully it will be addressed in the next update.

</comment><comment><USER27> FWIW, there is a tabbed browser for the iPad. It's called Atomic. Seems to have good reviews but not tried it yet.

</comment><comment><USER28> That the iPad got the same amount of RAM as the existing iPhone defintely seems weak.

</comment><comment><USER29> That's exactly why I waited to get the 3G version. A mobile device that rely exclusively on WIFI doesn't make sense to me.

Also, as somebody else already pointed out, Instapaper solve this problem in a very elegant way.

</comment><comment><USER30> I don't own an iPad so I can't test but Opera Mini for iPhone does keep the other tabs as I left them, no need for refreshing or to show some blank pages. It really improved my browsing experience on the iPhone.

</comment><comment><USER31> A lot of things about the iPad could use some tweaking and fixing up. That said, it's still better than everything else out there for "simple and ease of use" devices for the technically inept.

</comment><comment><USER32> I disagree on the need for tabs, but that is a nice looking webpage.

</comment><comment><USER33> There's an app for that.

</comment><comment><USER34> The worst part is the constant refresh on Back and Forward buttons.

</comment><comment><USER35> Apple should come up with a gesture to switch between tabs.

IMO, one of the big achievements of Fingerworks keyboard was it's vast array of gestures. Apple drastically watered down the gesture support in it's various products. Perhaps to make it simple for grandma?

<comment><USER36> the problem here is once you put two fingers on the surface it starts expanding/contracting the page

</comment></comment><comment><USER37> This is FUD. The multitasking to be introduced in the next iteration of the ipad OS will almost certainly allow for the presence of tabs. I agree that right now it's somewhat annoying to have tabs lose state once and a while when I'm browsing, but since I only use my ipad at home where I have a persistent and fast wifi connection it's not an issue to wait a second for the page to reload.

The author gushes about how awesome it is to browse the web on the ipad and then pigeonholes it because he couldn't recall a tab while he was away from an Internet connection, a closet case if there ever was one. It seems self defeating to focus on such a minor annoyance. This thing is an amazing piece of hardware, and no it's not perfect.

<comment><USER38> The iPad OS is perfectly capable of multitasking since version 1.0, this is just not exposed to the user, so there are no current technical limitations related to multitasking in order to implement tabs in Safari.</comment></comment>


Ask HN: Keep Python or switch to Rust/OCaml/Haskell? - <AUTHOR> Using Python makes hiring easy, but it's hard to verify and bugs are frequently missed when code is pushed to production. Safe languages with strong typing like Rust, OCaml, Haskell catch more bugs before deployment and are easier to verify, but are hard to learn and hard to hire for.<p>Does any one have experience of introducing these in mission critical environments? Was the shift worth the effort or is sticking with the status quo a better policy? <comment><USER2> Before going down this path, I suggest you pick up and read The Mythical Man- Month. The experience the author has closely matches my own experience.

What you're describing has very little to do with python and a ton to do with your software developers and your environment. If you have bugs that are not caught before you reach production, how is your testing? Do you have a good level of unit test converge which can easily catch type safety issues? Do you have integration test and regression tests?

As the mythical man-month states there is no silver bullet. No change in language will solve any significant portion of issues without further changes to your process. Nor will it increase productivity in any significant way.

It also mentioned the theory of irreducible numbers of bugs. Changing languages will change where you see bugs but not the number or severity of the bugs.

Of course you also have the second system effect which can be quite interesting. It basically states you'll have many of the similar issues from your first system but also a bunch of new ones from your second.

So what do you do? All is not lost. I would focus on making small but meaningful changes within your current team and environment. You'll find the most long term success there.

Going with a new language is new and exciting but it's also extremely risky.

</comment><comment><USER3> Are you using the optional types in Python? Plenty of people (i.e. most) are using non-functional languages in production. There are probably many best practices you could adopt to cut down your bug count without throwing away your code base.

</comment><comment> a-saleh It depends.

If I would have at least one other developer excited about more advanced static types, I would try to sell to the project lead something along the lines "Hey, me and $OTHER_DEV would like to take $SMALL_PART_OF_THE_APPLICATION and try to rewrite it in $LANGUAGE_OF_YOUR_CHOICE ... we think the type-system should catch quite a lot of those bugs we have been dealing in prod!" If the lead will take you up on that, you might have a nice way to introduce the language and figure out if it really is that much better.

If that is not an option (i.e. no easily identifiable sub-component/service) you still might be able to add types through mypy, and maybe even investigate something like mokeytype

If there is really strong pushback to do static-type checking, you can still make mission critical systems in untyped language. Erlang does this, and you can take the lessons learned from i.e. erlang and apply it in your python code-base. Based on my experience, limitting local state, and use more functional approach does wonders to testability and stability.

If the team is not a fan of functional programming, Maybe even throw in some advanced testing techniques, like fuzzing endpoints, doing property-based tedts or if it is applicable something like chaos-monkey :)

If there is no time for that, then focus on the process. Good review culture, good test-writing culture, have CI and integration tests ...

To be honest, I would try all of these, starting from the last suggestion, making my way to the first :D

[1] [https://github.com/Instagram/MonkeyType](https://github.com/Instagram/MonkeyType)

</comment><comment><USER4> Naturally there are tons of aspects to consider for a change like this; the biggest probably being your team's existing experience, preferences, and the kind of software you're building.

A ton of python programmers love go (aka golang) and at least where I work thats what out python codebases are turning into. Those are mostly command line tools but go should work well for services.

If you're building service I'm going to throw out an alternative that you might not want to hear: Java or Kotlin. I personally prefer Java but coming from python it is going to feel repetitive and verbose. They're a kind of sweetspot between straight-forward, battle-tested, and popular.

Compared to Rust, OCaml, Haskell either Java or Kotlin will probably: \- Have a larger talent pool to hire from similarly to python. Java programmers won't have a ton of trouble getting on board to Kotlin. \- Have a large ecosystem of library support similar to python. Kotlin has access to java's substantial library ecosystem. \- Have fewer new programming language concepts to learn. (I think java just has primitives and generics?) Kotlin has more going on but you won't need it all at once. \- Produce a codebase more architecturally similar to your python code base. I imagine it will be fairly more straight forward to migrate.

Nothing against Rust, OCaml, Haskell, I've dabbled in them and they're all compelling. But as a frequent python programmer myself I'd say Java and Kotlin make more sense given your concerns about hiring and migrating python.

Disclaimer - I'm a big fan of JVM land. I suppose most of what I said could apply to C# and F# too but ... ehhh.

Hope this helps! Let us know what you do!

</comment><comment><USER5> Define "mission critical". Are people going to die if bugs get shipped? If so, be aware that static type checking will not get you there alone, but it should be considered mandatory.

Beyond that: If your team is most familiar with Python, it is probably best to stick with Python. [http://mypy-lang.org/](http://mypy-lang.org/) might be the best option: you get static type checking for new code, you can add static type checking to your existing codebase, and you get to use something you're already familiar with.

I recommend reading [http://boringtechnology.club/](http://boringtechnology.club/) \- especially the "let's switch" part.

</comment><comment><USER6> I used elixir at work for a few projects, and we will use it in more projects. We rewrote a rails app using the Phoenix framework and now we are using elixir in services.

From my anecdotal experience functional programming leads to code that is easier to understand and test. You can make quality code in OOP, but I think functional programming makes it easier.

We also used Elm for a few prototypes and we're immensely pleased. We do not use it in production because static typing + purity is not something you can do easily find devs for.

Also, Erlang's OTP and BEAM are a pleasure to use.

</comment><comment><USER7> [https://www.rust-lang.org/friends.html](https://www.rust- lang.org/friends.html) is a list of companies using Rust in production environments, and [https://www.rust- lang.org/whitepapers.html](https://www.rust-lang.org/whitepapers.html) is a list of whitepapers we're building up. Both may be of interest!</comment>


Why people don't want tech giants like Amazon moving into their neighborhood - <AUTHOR> https://www.washingtonpost.com/world/2019/02/15/why-more-people-dont-want-tech-giants-like-amazon-moving-into-their-neighborhood/ <comment><USER2> This article mentions that Bezos owns the Washington Post but fails to point out that the proposed NY headquarters was to be built with a $3 billion subsidy from public money.

Could it be that was the real reason for the public's outrage?</comment>


Best practices for Zoning on Brocade switches - <AUTHOR> https://community.emc.com/thread/186205

<comment><USER2> Cool</comment>


Softbank-Backed Brain Corp Expands Its Robot "Brain" to Autonomous Delivery - <AUTHOR> https://www.forbes.com/sites/amyfeldman/2019/04/02/softbank-backed-brain-corp-launches-delivery-robot-software-in-expansion-beyond-floor-cleaning/#53ec8d66163c <comment><USER2> This is great. Mankind can use their time on real stuff with bots helping do such tasks.

<comment><AUTHOR> Exactly! More quality time for people. And with an aging population and shortage of service workers, robots will help improve our lives.</comment></comment>