New bill would let companies force workers to get genetic tests - <AUTHOR> https://arstechnica.com/science/2017/03/companies-want-employees-genetic-info-new-bill-lets-them-take-it-by-force/ <comment><USER2> The bill does no such thing:

[https://www.congress.gov/115/bills/hr1313/BILLS-115hr1313ih....](https://www.congress.gov/115/bills/hr1313/BILLS-115hr1313ih.pdf)

Relevant quotes:

"...workplace wellness programs and programs of health promotion or disease prevention offered by an employer _that provide for more favorable treatment_ of individuals with adverse health factors as described..."

"...the collection of information about the _manifested disease or disorder_ of a family member..."

But _nothing_ about forcing genetic testing on employees.

</comment><comment><AUTHOR> And at the end of the article:

> On Wednesday, the House Education and the Workforce Committee approved HR > 1313 along party lines, with 22 Republicans supporting and 17 Democrats > opposing the bill.

</comment><comment><USER3> tl;dr -- Employers can charge you thousands of dollars per year not to give your (non-anonymous) genome to third parties who can turn around and sell it.</comment>


World's Most Cramped Airline Seat to Launch Next Week - <AUTHOR> http://www.wired.com/gadgetlab/2010/09/worlds-most-cramped-airline-seat-to-launch-next-week/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28Top+Stories+2%29%29 <comment><USER3> I can't imagine the government authorities would allow this for at least two reasons:

1\. safety. What if the jet needs to be evacuated because of an emergency and you're in a window seat and there is a very portly person in the middle seat who happens to be unconscious? Your normal exit path would be blocked and you can't easily climb over the seat in front of you.

2\. children and infants: How would a five year old sit in one of these contraptions? How would you take a baby on board?

<comment><USER4> More to the point:

Current airliner passenger loads are determined by a mandatory requirement to be able to evacuate the passengers from the hull within a set period.

Cram in extra seats, and you're up against the evacuation limit, which is a hard limit dictated by the number of bodies and the number of doors in the fuselage.

To work around it, you'd have to cut holes in the fuselage for additional emergency exits and then recertify the aircraft, which means an airline wanting to use these seats will need to buy new airliners certified to carry more passengers. But this higher density seating only makes sense for short haul ultra-low-cost tickets, which are less profitable, ergo ...

(And then there's the issue of weight. Passengers weigh, say, 75Kg on average, including clothing. They fly with luggage, another 5-25Kg. A Boeing 737-800 currently has a limit of 189 passengers and a payload of roughly 40,000kg, of which up to 26,000kg is fuel: put more than 189 passengers (18,900Kg by my BOTE estimate above) and you're going to be squeezing right up against Maximum Take-Off Weight unless you reduce the maximum fuel load. Even if there's room in the hull, you're going to have to redesign a lot more than the seats and the emergency exits ...)

<comment><USER5> An alternative option would be to cram the economy class into a smaller space, so they can have more space for premium economy, business and first class seats. That would be useful for routes between London and New York where there's a big demand for business class seats.

<comment><USER6> No matter what your age, financial status, or physical stature/condition, I really find it hard to imagine anyone riding a saddle with a seatbelt on an already-torturous-in-economy transcontinental flight.

</comment></comment></comment></comment><comment><USER7> > Is it one step closer to just drugging us and piling us onto shelves like > suitcases, or a legitimate next-step for cheap air-travel?

I think I'd rather be knocked out.

<comment><USER8> Actually that would really not be too bad . . . unless you need to get off the plane early for any reason.

</comment></comment><comment><USER9> If the tickets are cheap enough, people will use these seats. The question is, how much cheaper does this make tickets?

Edit: Made some very rough estimates Flight from Chicago to New York with United is $178, (found on Expedia). There are 112 economy seats = $19,936. Flights aren't going to be full to capacity but let's just assume for now. Assume that United replace it's entire economy section of the flight with these seats.The economy section is 617" long. 617/23 = 27 rows of seats * 6 seats/row - 2 emergency exits = 160 seats.

$19,936/160 = $124.60

So if you replaced the entire economy cabin with these seats, a flight from NY to Chicago would go from $178 to $124.60. That's a pretty significant price cut.

<comment><USER10> Your math assumes that there is no per traveler cost to the airline, which seems unlikely. Extra people increase the weight (lowering fuel efficiency), slow down boarding times, drink extra soda, etc.

</comment></comment><comment><USER11> This is beyond ridiculous.

I'm also six foot three but I also happen to weigh in around 130kg and I would genuinely prefer to stand than use these seats.

Those afflicted with arthritis, parents with infants and almost anyone with a physical disablility will struggle massively. I appreciate it's a concept at the moment but Ryanair will bend over backwards to get something like this approved. They are already floating the idea of standing room only.

<comment><USER12> Of course people with disabilities or special needs need to buy the ticket that works for them. My son sometimes has to buy a 1st class ticket for himself and his cello - its pointless to whine about how economy class should be bigger to accomodate his needs.

<comment><USER11> Carrying a large musical instrument and being afflicted with a physical disability are poles appart.

Agreed, airlines shouldn't increase space to accomodate his cello but I see no valid argument justifying making an MS sufferer pay more because they would struggle to support themselves in these seats.

<comment><USER12> How about a paraplegic?

Come on, it sucks to be sick, but every airline seat is not going to be quipped with life support systems.

<comment><USER11> Out of context again Joe.

I'm not advocating that airlines should incorporate ridiculous measures to accomodate all passengers, I am simply stating that these new seats will prevent a significant amount of people who _can_ currently travel in economy from doing so in future.

<comment><USER13> The article says that this will be a new class, below economy, called "cabin class." If that's the case, everyone who used to travel in economy could continue to do so.

</comment></comment></comment></comment></comment></comment><comment><USER14> If they're looking to reduce space, they should have us lie down horizontally. I don't think I would mind less space if I could lie down the whole time.

</comment><comment><USER15> Sweet, get ready to sell chafing relief products in the terminal.

</comment><comment><USER16> I'd rather fly standing up!

<comment><USER17> Ryanair and its plans for £5 standing only tickets:

[http://www.telegraph.co.uk/travel/travelnews/7864921/Ryanair...](http://www.telegraph.co.uk/travel/travelnews/7864921/Ryanair- to-sell-5-tickets-for-standing-room-only-flights.html)

<comment><USER18> Does anyone remember when Ryanair were an airline, not a punchline?

I flew with them once, because they went from an airport much closer to me than the big carriers like BA. I'm not a fan of flying generally, but this was such an unpleasant experience that if I ever make the same trip again, I will surely fly from a different airport with a different carrier even if it means a significantly longer journey from home to airport.

<comment><USER17> I've flown with Ryanair a few times between the UK (Prestwick) and Sweden - <USER2> thought it was perfectly OK.

However, their website is an abomination.

</comment></comment></comment></comment><comment><USER19> There's a website for that :

<http://seatguru.com>

After being through the Lima to Amsterdam wringer with half of the front seats floor occupied by the entertainment box (literally) I appreciate a website that gives me detailed seat layouts before buying a ticket.

</comment><comment><USER20> Yet another reason why the airline industry needs to be re-regulated to stop the ongoing race to the bottom in customer service.

</comment><comment><USER21> I think the airlines might have just found my point where comfort becomes more point than price

</comment><comment><USER22> Well at least she seems to be enjoying the turblence!

</comment><comment><USER12> I wonder about thrombosis...

<comment><USER23> For short haul flights deep vein thrombosis shouldn't be an issue, and the awkward position would probably better for circulation. Passenger evacuation and customer rejection are bigger obstacles to come.

</comment></comment><comment><USER24> I've never said this of a fellow startup, but I hope in earnest that they fail, not out of contempt, but because I don't want ever to have to sit in a seat like that.

Note: "below economy" class is a terrible idea for us, because it means that those seats will cost current coach prices, and what we call coach will become business class. (This has happened before; business now is what coach was in 1990.)</comment>


The Libertarian Futurism of Peter Thiel (2011) - <AUTHOR> http://www.newyorker.com/magazine/2011/11/28/no-death-no-taxes <comment><USER2> It seems like Silicon Valley is full of people who love espousing their noble ideas but got rich via (as Thiel admits) at best socially neutral things like Facebook and Paypal.

As for the anti-aging research: Fund the NIH. Basic science like that needed in aging research isn't going to come on a startup timescale. Thiel makes a big deal about encouraging people to leave universities, but the smartest PhD students are generally not the ones leaving academia.

<comment><USER3> The nih's run of good Basic science is over and may never come back. Government has this pesky thing called accountability that it needs to satisfy. There is no way to hold a scientist accountable when the fruits of 'basic science' are not provable for decades or centuries. It's just too easy for a snake oil salesman to extract funding from clueless program managers (the smartest PhD students are not the ones becoming program managers).

As for smart PhDs leaving academia... They are. Getting a faculty position is basically a crap shoot that requires politicking, landing a lucky project, getting a high impact factor publication, or all three. The labor market was so saturated by decades of encouraging more people to be PhDs (it kind of became a dumping ground for students who didn't want to med school or look for a job) that the distinction between a good scientist and a bad failed to be the standard academic metrics. Good scientists who diligently pursued difficult research were more likely than not rewarded with little because such projects might not be doable in a five year PhD or a two year postdoc. The other route out is fraud.

Edit: oh, I see, you're in the MSTP. Please consider Upton Sinclair's quote: "it is difficult to get a man to understand something when his salary depends on his not understanding it"

<comment><USER2> My main point was that product-driven start-ups are not going to be able to do the basic science research needed to get revolutionary advances in things like aging research. You're complaining about getting only 7 years of protected research time with no real expectation of a viable product: How long will these types of places have?

If you're going to call me out on my vested interested in NIH funding, I think you should acknowledge you have a pretty similar (opposite) conflict of interest :)

Certainly the NIH basic science funding situation right now is absolutely dismal. However, it's ludicrous to suggest that the reason is because of some lack of "accountability." It's also not exclusively "clueless program managers" who are choosing how funds are disbursed: There are panels full of (decidedly) non-clueless professors who do that.

The real reason is because there are more PhDs than there used to be, more professors than there used to be, more people applying for grants than there used to be, and NIH funding has become nominally stagnant (declining in real dollars) after a period of massive expansion.

I'm not arguing that getting a faculty position is definitely something of a crapshoot, especially if you aren't a dual-degree candidate and don't have a clinical aspect to your work that you can fall back on.

<comment><USER3> What's my conflict of interest? I'm not paid by my nonprofit, and it doesn't do basic research.

I think you missed my point. If the smartest are leaving academia, then the professors that are reviewing grants and papers ARE clueless. How did Arsenic Life get through review with glowing reviewer comments? How is it that Reza Ghadiri got $6M to pursue DNA nanopore research sequencing with zero previous experience in the field? I could go on and on.

As for alternatives for basic science, the idea that science must be funded by either the NIH or startups (or even industry writ large) is a false dichotomy: consider Peter Mitchell.

[http://en.wikipedia.org/wiki/Peter_D._Mitchell](http://en.wikipedia.org/wiki/Peter_D._Mitchell)

It's also worth noting that one of the most major medical advances of the 20th century, the eradication of polio, was done similarly without the NIH.

<comment><USER2> I know I probably invited inflammatory responses by suggesting that the best PhD students are able to stay in academia, which I admit is an overgeneralization.

From what I see (and this is getting more personal than I'd like to from a response to my lighthearted comment) you left academia to work at a pharmaceutical (non-profit) company you started. I may have gotten that wrong. If so I apologize.

I do think it's a bit beside the point to be arguing about this instead of the ideas, but so be it.

Peter Mitchell was exceptional, as was every other Nobel Prize Winner, the vast majority (the rest?) who were academics. If you're arguing that moving to a model where individual scientists soliciting charity for their research is better, I would have to disagree. You think the general public is better able to assess research value than the NIH?

<comment><USER3> It is not just an overgeneralization, it is basically a dangerous defense of the status quo, that doesn't match up with reality. The problem is largely that science has become a kleptocratic system where there's even a term 'grantsmanship' that everyone knows involves a level of dishonesty and overselling. If you'd like for your scientific leaders to be polluted, at a fundamental level, with this toxic attitude, then perhaps you deserve the scientific system we have now.

On a personal level, I find it to also be a very distressing appeal to authority - "well these people are professors because clearly they deserve to be"... I mean, I have worked under/with professors who definitely deserve it (Ham Smith, Clyde Hutchison, Dan Gibson, Vanessa Hayes, Jeff Kelly, my boss at the University of Maryland - who does incidentally now work for the NIH, but is planning on quitting) but there is a very, very long list of professors who do not... I'm not really going to name too many names for obvious reasons.

<comment><USER4> The fact that _[t]here is no way to hold a scientist accountable when the fruits of 'basic science' are not provable for decades or centuries_ is an inherent problem with science. I can't think of an alternative system which would resolve this. If there was no uncertainty with regards to the profits resulting from research, then it wouldn't be science anymore - it would simply be consulting. And I honestly can't imagine how you can prevent _dishonesty and overselling_ when laymen are involved in funding decisions through donations.

Edit: I've just read the comment by eli_gottlieb further below and would argue that's a far better improvement for science rather than abandoning the NIH.

<comment><USER3> Exactly, you have hit on my point. You can't prevent dishonesty and overselling period.

When a person gives out of their own pocket to an outlandish and unlikely project, that is on them. When the government gives out to an outlandish and unlikely project, the donors _had no choice_.

In that way, government is dangerous, and therefore must be held to a higher standard of accountability. A charity or private nonprofit, if it screws up, it will find itself in a more difficult position with regard to seeking further resources for its work.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> Don't get me wrong, I really like Peter Thiel, but it's hard to consider him 'Libertarian', whilst selling advanced snooping software to the U.S. government.

<comment><USER6> Why? Libertarians think gov't is needed for certain institutions like an army, or a justice system.

<comment><USER7> Logically consistent libertarianism requires anarchy. More here if you like to know why:

[http://www.walterblock.com/wp- content/uploads/publications/b...](http://www.walterblock.com/wp- content/uploads/publications/block_radical-libertarianism-rp.pdf)

<comment><USER8> I get the funniest looks when someone says that they are a Libertarian. My response is oh your an anarchist, but at what level? EDIT: The libertarian looks at me funny.

Seriously, libertarians have no idea of the history of the political movement they are involved in.

<comment><USER9> Why are libertarians the only group held to a pure ideological standard? No one blinks when they meet a conservative or liberal with a few positions that go the other way.

For example, I frequently hear liberals say that if libertarianism were pushed to its logical extreme, you'd end up with Somalia. For some reason, liberals are insulated from the question on what happens when leftist ideology it pushed to its extreme.

<comment><USER10> "Why are libertarians the only group held to a pure ideological standard?"

Because they keep espousing positions that have proven to be terrible ideas. They forget about history, and why things like government regulations came about.

<comment><USER3> I'm not sure libertarians are not opposed to regulation, they just require regulations to have a rational basis, and be restricted in scope. For example, not allowing murder is certainly a regulation, and I'm pretty sure libertarians generally are ok with that regulation.

Not all but some libertarians are arguably more familiar with history, because they understand the degree to which regulation historically has been twisted (generally: to screw over the poor), wheras their detractors have been sold the line that exists to defend the status quo on any given regulation.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I have to say, the writing style of this article is an immense time-waster. Even skimming it doesn't get to the point quickly because the history of Thiel (large in size but mostly unimportant) is inter-mixed with the ideas he's espousing (small in size but really the part we care about).

What ever happened to 'summary' then 'summary with more depth' then 'summary with even more depth'?

<comment><USER12> This is not what The New Yorker tries to do. It's a different type of journalism. It do not worry about the lenghs of its articles.</comment></comment>


Why People Should Learn Python - <AUTHOR> https://iluxonchik.github.io/why-you-should-learn-python/ <comment><USER2> Is it just my feeling or are there languages which are preferred on HN?

Indicators for languages which are liked by the HN community:

\- Positive stories get many upvotes and are instantly at the top of HN

\- Most comments are positive

\- Frequent coverage on HN about the language

Indicators for languages which are not liked by the HN community:

\- Rants get many upvotes and are instantly at the top of HN

\- Most comments are negative about the language

\- Rare coverage on HN

Applying those indicators, following languages should have these like-levels (on a range from 1 to 5):

      Python  *****
      Go      ****
      Node.js *
      C#/.NET ***
      Java    ***
      RoR     **
      Ruby    ***
      Elixir  ***
      Haskell ****
      Closure ***
      C       ****
      Rust    *****
      C++     ***
      PHP     *
      Swift   ****

This is just my feeling, happy to hear your perception. And this does not reflect my opinion about those languages in any way.

<comment><USER3> At a glance, it looks like all languages are reasonably well loved except PHP and Node.js (shouldn't this just be Javascript? Does having a different standard lib make it a different language?)

I think that's due to people having flash backs to terribly written code in both languages.

For example, PHP apps pre-2005 rarely if ever used frontend controllers, preferring instead to twine in configuration and helper functions into isolated pages that were directly served by Apache. This meant that there was no single obvious place to put things like caching.

Additionally, once upon a time, PHP didn't really have well developed ORMS, so everyone just wrote queries and hopefully parameterised queries---but not always. So if you had a hobby project that took off, then within 3 years it'd be a roiling ball of mud where it's difficult to do basic things like tracing which calls happen per page.

PHP came to prominence early in the web, and it was easy to use so _everyone_ had a hobby project that took off, or had to work within something like that. Hence the hate.

To compare it to Javascript, I think the disdain for both languages is due to their initial nature---they were both written over an short period of time, and evolved in spurts rather than being carefully designed from day one or designed to overcome the shortcomings of another language.

The quality of the above really shows up in the inconsistency of the standard libraries, and the fact that certain semantics were inconsistent or danger zones in practice but couldn't be changed.

\---

I do remember though when everyone hated C, and loved C++, then went back to loving C... so maybe programmers are just fickle and none of what I said makes sense. :)

<comment><USER4> just to chime in on this, modern PHP is nothing like what people remember from the pre ROR days, i still use it to this day and the major frameworks and libs are really well written to the point i feel some are even over engineered and almost looking like Java (just take a look at the latest Guzzle PHP Library)

<comment><USER5> As my PHP fluent colleague said to me - don't bother learning PHP or you will end up having to fix (our) Wordpress sites and thats a nightmare. We have a Python application, and while its not written well, the PHP sites we have seem to be a fair bit worse even the ones not done in Wordpress.

<comment><USER6> Wordpress is horrible. Never going to touch that again (Senior PHP Dev myself) or anything else like it (Magento, Drupal,...).

And yes there is a lot of horrible PHP code out there, writting by bad developers. I guess newer or not as accessible languages don't have as many bad devs as PHP/JS.

But if you have good devs, I prefer a good PHP codebase to any other language that I have come across (I tried a lot of them).

<comment><USER7> Around 2001 I felt jaded with the industry and determined to do whatever was expedient to earn money. That meant using PHP, because that was purportedly the place to be at the time.

I have never felt so uninspired by a language. My overwhelming feeling from the get-go was that it was a collection of libraries, which would be fine -- but why create a new language?

PHP has the rare distinction among languages of not containing any novel features or ideas. Not only that, but it borrows liberally from several other languages, resulting in an overabundance of syntax. Instead of a tool for attacking problems, it's a reheated leftover. There is not suggestion of a better or worse way to accomplish some goal.

Finally, the original premise of PHP, interspersed functional code and HTML, is high on my list of really bad ideas. Code written in this style is doubly- challenged in terms of reuse and maintainability. Again, there is again no semblance of better or worse with this approach. If you had Richard Stallman and Linus Torvalds produce the same website with PHP, there would be very little in common between their approaches.

Given the success of Facebook et al, I presume that the interspersed code/html thing has been superseded, and further that the egregious security issues in PHP have been fixed. But the picture in my mind's eye is something that lumbers along despite its thorough mediocrity. I will allow that making PHP adequately consistent and secure are impressive feats.

Sincere question: how am I wrong here?

</comment></comment></comment></comment></comment></comment><comment> edit </comment><comment> Drop extra 'the'

<comment><USER8> Since when did something need to have novel features or ideas in order to be productive?

I'll counter that one of the main reasons PHP became such a used language (other than ease of deploy) is exactly the reasons you dislike it. There are multiple ways to do things - that coincide with the ways you do things in many other languages (which it steals from).

You dont have to learn anything radically new to use it, and that makes it very easy to do things quickly.

<comment><USER7> "Since when did something need to have novel features or ideas in order to be productive?"

It doesn't. But if your language ideas are _admittedly_ no better than the next guy's, what's the point?

<comment><USER8> It's easy, fast (both Development speed and Page Views), and can deploy anywhere with a few clicks? It has no novel concepts that require learning something new. It has a plethora of existing code, documentation, frameworks, and StackOverflow answers to solve almost any problem (albeit, not always correctly).

Its the lowest barrier to entry web language - and honestly, I'm ok with people writing shit PHP. It gets good Ideas out there quickly - allowing the business to grow without a focus on the Development. This creates opportunities for sharp developers to come in and work on proven ideas.

</comment></comment></comment></comment><comment><USER9> I switched to writing python as my main backend/scripting language with my new job. Previously I was using Ruby for most of this stuff with some Go. I must say that so far Python has been a vastly inferior experience for me. The languages are fairly similar all though I prefer the more talkative/english style of ruby and the use of functions over list and dict comprehension.

The thing that really stands out to me though is the poor state of python tooling and the library ecosystem. Having used Rubygems and bundler pip feels like taking an enormous step back. It's much less expressive. It doesn't handle the difference between production only dependencies, regular dependencies and development only dependencies in a good way. It's difficult to differentiate between locked dependencies vs desired dependencies(the Gemfile/Gemfile.lock distinction). PyPi and especially using private PyPi registers is more complex than it is in Ruby. There seems to be fewer nice libraries and they seem to be spread across the web whereas Ruby centralises around GitHub. I also find that Python libraries have lacking or hard to find documentation in many cases.

I wonder how much of this is due to the community spending a lot of time on the unproductive Python2/Python3 debacle instead of moving the language and the ecosystem forward. It seems like Ruby and Python would be in more similar positions had it not been for the problems around the Python 2/Python 3 upgrade. Ruby had the risk of a similar problems in the 1.8 to 1.9 upgrade, but it seems to have worked out better than the Python 2/Python 3 upgrade.

<comment><USER10> I wish Ruby had the math and scientific library support of Python.

Also, I've been using Jupyter lately to brush up on data structures and algorithms. It's been pretty sweet to write code in a web notebook and render output including digraphs beneath each block of code. Is there anything like that for Ruby?

<comment><USER11> You can use other kernels in jupyter notebooks. I've never tried ruby, but you could try this: [https://github.com/SciRuby/iruby](https://github.com/SciRuby/iruby)

</comment></comment></comment><comment><USER12> Now that I have a chance to ask: I've always found the def __init__ method with 4 (!) underscores one of the ugliest things I've seen in a programming language.

Don't get me wrong: I like Python and know it's truly good, but __why__??

<comment><USER13> Call me ignorant, but I agree. Having something like __init__ and __new__ is very confusing for a beginner, along with stuff like "__init__.py".

It doesn't stop there: Take something like __str__ and __repr__ combined with str(), repr(), vars(), dir(), print(), pprint.pprint()... I have no idea how anyone ever thought it was a good idea to have that many ways to output the content of a variable. Even after all the time I used python I never managed to find a consistent way to output variable values.

I generally like python as a scripting language. But stuff like this made me constantly lookup obscure details in the documentation which made it very hard for me to completely embrace the language. It also prevented me from diving into more arcane stuff like meta-programming...

<comment><USER14> The dunder functions are there to provide an interface to global functions and operators.

If you want to affect how str(x) behaves, you override x's __str__() method. Operators behave this way too: for example, if you want to overload the == operator for a class, you override __eq__() in that class (for any Java developers reading: Python == works like Java equals(); the Python version of Java == is the _is_ operator, which cannot be overloaded). Yes, that means _x == y_ is just syntactic sugar for _x.__eq__(y)_.

The dunders are there so you have the freedom to name your methods what you want without having to worry that you'll accidentally clobber a method that affects a global or an operator, as most people won't just up and decide to both begin and end a method's name with __.

</comment></comment></comment><comment><USER15> I use python on daily basis for 5 years now. It's a really cool language, but:

* Packaging is horrible

* Releasing python code is a non-standarised nightmare. Every solution has it's own flaws

* Big and complex projects in python are really hard to reason about

* Poor support for concurrency (fixed in py3)

Don't get me wrong, python is really cool as a proof-of-concept scripting language. But for mature and complex stuff, I'd recommend something with type checking and better support for release process.

<comment><USER16> I've programmed in a few other languages than Python. I'm curious to know which languages you think have an easier time managing mature and complex stuff?

<comment><USER17> Statically typed languages. Personally, java or C#.

<comment><USER16> I've worked with large C++ projects and large C# projects (both at large software corporations) and the code was not particularly less unwieldy than an equivalent Python projects I've worked on.

I think static typing alone doesn't make managing complexity easier.

I'd like to know what other features people think Python is lacking that make it much worse than C# or Java when handling large projects.

</comment></comment></comment></comment><comment><USER18> Why you shouldn't learn Python:

1\. In-consistant syntax.

2\. The language uses exceptions to control flow of logic.

3\. Divided community, since Python 3+ included breaking changes to the standard.

4\. Un-discoverable APIs. You better hope the documentation is bulletproof else the API could change in any which way during runtime.

5\. Poor error messages. If an import goes wrong you are not told why, and so on.

6\. Ultimately slow performance for anything marginally complex.

7\. Poor tooling.

<comment><USER19> 1\. a matter of tast

2\. care to elaborate ?

3\. it's not divided, it's evolving from 2 to 3. Java's community is under Oracle's grip, don't know if it's any better...

4\. it's not because you can do it that it's worth to do it, so in practice API are just not changing at runtime

5\. I can say the same of my industrial strength Java environment...

6\. Not a problem since what you do doesn't require speed

7\. Ever used PyCharm ? I wouldn't say it's poor tooling. The debugger is fine. What I miss is something like a good profiler to look at running code.

But I'd say :

8\. Optional typing should've been there from day one, it'd made code editor, tooling better

9\. The date/time handling is not good

10\. SOA support is weak (and in the kind of setting I am, we use tons of XSD, WSDL, XML, security, signature you name it)

<comment><USER20> PyCharm comes with cProfile and let's you "run with Profiler". But maybe that's only the paid version.

<comment><USER21> Yup. The integrated profiler is only in the paid version

</comment></comment></comment></comment><comment><USER22> The person who wrote this obviously has "the hacker's mindset". He praises expressivity and malleability of the language.

Well, in this case it might be advisable to take a look at Perl. And never go back. Perl (especially Perl6) is hacker's paradise, where everything's true and nothing is forbidden.

As somebody who shares the same mindset, I truly enjoyed my years with Perl, and I occasionally return to this experience.

<comment><USER23> > is hacker's paradise

And support engineer's hell

<comment><USER24> So is Python. I've worked on code bases in several languages, and the greatest horrors I've seen are in Python and C++, in that order.

Python is a strange case. The CPython code base seems to be quite well organized and well written, and about 10% of packages on GitHub or PyPI are extremely solid.

The problem is the staggering amount of packages (public or in-house) that have 100000+ LOC and use the worst features of OO, functional, and procedural programming at the same time.

The result is not spaghetti code, but ravioli code where there's zero control flow and "everything happens somewhere else". Add to that the "consenting adult" philosophy, and you have the modern equivalent of goto mixed with self- modifying code.

I'm sure Perl has all that, but it's far from the only language.

</comment></comment></comment><comment><USER25> Always fun writing compact Python:

        import sys
        from collections import Counter

        cnt = Counter([n.strip() for n in sys.stdin.readlines()])

        for key, val in cnt.most_common():
            print(val, key)

This isn't just playing code golf, it goes with point (2) - awesome libraries. One of the joys and curses with Python is finding the Pythonic ways to do things that you've hamfistedly written your own way for months.

<comment><USER26> FWIW the Counter line can be simplified slightly:

        cnt = Counter(n.strip() for n in sys.stdin)

* no need for a list comprehension, a generator expression will do

* file objects are iterable, and iterated line-wise

</comment></comment><comment><USER27> A new & free introduction to Python from Jake VanderPlas can be gotten here:

[http://www.oreilly.com/programming/free/a-whirlwind-tour- of-...](http://www.oreilly.com/programming/free/a-whirlwind-tour-of- python.csp)

I was looking for a dense & concise guide to the language, but one of the most popular books "Learning Python 5e" by Lutz was too long @ 1600 pages.

<comment><USER28> His book is awesome! I wish I had it when I started Python. I had gone through the documentation tutorial thrice, the entire documentation once or twice. Started writing code and then understood python.

I found that learning Python is simple, mastering it is difficult, the magic which happens all the times makes it difficult to understand, which is the reason why static languages are awesome as less magic as possible.

</comment></comment><comment><USER29> > Another thing is strict typing and debugging: since Python is an interpreted > language, finding bugs wasn't as easy

Wait, what? How is its interpreter remotely relevant to its semantics?

> if you have a syntax error in C, the program will simply not compile, on the > other hand, in interpreted languages, the problem might go unnoticed for > quite some time

Hang on--Python allows you to run code that has syntax errors? I'm so confused.

<comment><USER30> To my knowledge syntax within a file is completely checked when that file is loaded. I think the OP referred more to errors in method and other reference names.

<comment><USER31> If you write this:

      #!/usr/bin/python
      import time
      time.sleep(5)
      fdafadsfadsf

It'll take 5 seconds to crash with a "NameError: name 'fdafadsfadsf' is not defined". Likewise, if you do this:

      #!/usr/bin/python
      import time
      x = 1
      time.sleep(5)
      print "x is "+x

It'll take 5 seconds to crash with "TypeError: cannot concatenate 'str' and 'int' objects"

In a language like Java, these would be syntax errors, and they'd be picked up at compile time as they're illegal in Java's syntax.

I assume they're legal in Python's syntax, because for all the syntax checker knows, time.sleep() might define fdafadsfadsf and convert x into a string. So they're runtime errors in Python.

Of course, Java still has runtime exceptions - null pointer exceptions, for example.

<comment><USER32> Java and Python are apples and oranges - one is a compiled language, the other is interpreted. Better to look at other interpreted languages like Ruby & Perl...and they have the same issue. It's a side-effect of using an interpreted language, but it also allows such languages to be self-modifying which can be very powerful when you need it.

<comment><USER29> Python and Java are both compiled to bytecode and both interpreted as bytecode. It has absolutely nothing to do with compilers.

</comment></comment></comment></comment></comment><comment><USER33> Addressing some criticism from various comments:

> Python 2 vs 3

It is mostly over. All important libraries are accessible on 3. It is clear that if you are starting from scratch you should go with 3. There are good tools to help with the migration (e.g. six). Some libraries start dropping python 2 support.

> Bad tooling

Having used Emacs with jedi for a while I switched to PyCharm to leverage optional typing. PyCharm beats my tooling experience in all languages I used expect Java:

\- Auto imports work well.

\- Code completion works very well, especially on optionally typed code.

\- Even refactoring works very well, failing short only to my experience with Java, but beating C++ or even Scala.

\- Debugging is actually a highlight - I can drop into full fledged REPL at break point. "Evaluate expression" in C++ or Java didn't come close.

Profiling is indeed not as advanced as what I was used to on the JVM, but IMO it is sufficient for 95% of use cases. For that 5% it is clear from the start that you shouldn't go with python. Language specific performance monitoring in production becomes less of an issue when you start running inside containers and it starts to make more sense to monitor containers rather than individual processes.

> Maintainability at scale

I am using Python 3.5 optional typing and I feel my project is much more maintainable as a result. Among others, refactoring in PyCharm works very well.

> Packaging and deployment

As mentioned in another comment, personally I moved away from relying on language specific tools to do more than just "install version X of library Y" and manage my deployment and dependencies using docker. It have it's own problems, but so far it works very well, and it's quickly improving.

> Performance

In domain of scientific computing by using correct libraries and things like Cython or Numba you can come close to C++ performance levels. In other domains it's becoming more common to be disk/network bound, and then it doesn't matter if you use Assembler or Python. Also JITs are coming to Python: [https://lwn.net/Articles/691070/](https://lwn.net/Articles/691070/).

> Syntax/libraries/language features/etc.

It is the most subjective section, but my experience with python in this regard is highly superior to Scala/Java/C++/JS.

</comment><comment><USER34> In PowerShell,

       PS> Import-Csv .\names.txt -Header name | group name | select name, count

       Name  Count
       ----  -----
       cat       3
       dog       2
       mouse     1
       bird      1

<comment><USER35> Seriously, the author seems to have fairly limited knowledge of powershell.

A 13-line python script is presented, followed by a smug "you can almost hear the UNIX folks complain about how verbose the PowerShell version is".

First off, the provided Unix commands _don't work_ (should have used `sort -rn -k 2`), while the provided powershell, as verbose as it is, does work.

Second, _the entire python script along with the wrapping unix commands_ is encapsulated with a powershell 1-liner:

        cat ./names.txt | group | sort -d Count

</comment></comment><comment><USER36> As the author mentioned Python is good for small stuff. If you are building a huge system where the jobs are spread between a dozen of people things could easily get out of control and you could end up in the dirt.

<comment><USER37> Do you feel this way due to the dynamically typed nature of the language (and the possibility of having ambiguous interfaces)? I've had pretty good luck with rather large scale python projects, though we've been pretty strict about working within a framework.

<comment><USER36> Yeah mainly due to that. If you don't have a VERY strict approach to writing code, one wrong decision could turn into a few days of debugging..

<comment><USER32> Where 'strict' == 'unit tested', then I agree with you. You don't need strong typing to make good code.

<comment><USER37> I would agree that complete and meaningful unit and integration tests are definitely a necessity for a large Python projects. Or really for any software project meant to be maintained.

</comment></comment></comment></comment></comment><comment><USER38> Python is in space [1].

There is such a large breadth of fields that it has been successful in. Web apps, data analysis, machine learning, micro controllers, etc.

It has a long track record for being stable, reliable and secure. The popularity over the years has been steady and with the recent addition of data science field and machine learning, it's seeing another growth spike.

In my opinion, it's a highly versatile and very productive language that seems to be able to handle just about any job or industry you can throw at it. I personally switched to it full time and love it.

[1] [https://www.youtube.com/watch?v=Zm08hXeuv-I](https://www.youtube.com/watch?v=Zm08hXeuv-I)

</comment><comment><USER39> If you want to learn more in-depth Python: [https://pythonspot.com](https://pythonspot.com)

<comment><USER40> I just went with the "Fluent Python" O'Reilly book.

[http://shop.oreilly.com/product/0636920032519.do](http://shop.oreilly.com/product/0636920032519.do)

It's a bit pricey, but I managed to score it on a 50% sale, and I have to say I think it's a very good book for people already fluent in other programming languages.

</comment></comment><comment><USER41> If only the JVM didn't have slow startup speed I would almost exclusively write Groovy scripts.

There is just something so awesome about shoving your dependencies right in the script and not worrying about pip, easyinstall, system libraries etc [1].

Maybe Java 10 will fix the startup speed (or whenever jigsaw is done correctly) but I doubt it.

[1]: [http://docs.groovy- lang.org/latest/html/documentation/grape....](http://docs.groovy- lang.org/latest/html/documentation/grape.html)

<comment><USER42> > There is just something so awesome about shoving your dependencies right in > the script

Groovy needs two lines (@Grab and import) to get a dependency. Golang only requires the import. Perhaps Groovy needs to be rewritten in Go.

<comment><USER41> _> Groovy needs two lines (@Grab and import) to get a dependency. Golang only requires the import. Perhaps Groovy needs to be rewritten in Go._

I'm not dissing Golang but it isn't even remotely the same. A major portion of the OP content was about using Python instead of shell scripts.

Golang is not scripting language. You need to compile for each platform and the code is opaque once it is compiled. Not to mention Golang might not have dependency issues at runtime but the overarching consensus is that certainly has issues with compile time dependencies management (auto checking stuff out from github is not effective).

 _> Perhaps Groovy needs to be rewritten in Go._

I normally don't say this... but that is just dumb fanboy comment.

 _EDIT:_ apparently you are working on a scripting language powered by Go called Gro. I'm not sure if you just mistyped and meant Gro instead of Go.... and reading your blog it seems you must have been severely burned by the Groovy community.

<comment><USER42> > I'm not sure if you just mistyped

I didn't mistype. Go would make a good language for writing dynamic languages such as Apache Groovy in. My own Gro is just one example of such a dynamic language.

<comment><USER43> > Go would make a good language for writing dynamic languages such as Apache > Groovy in

Wasn't the whole point of Groovy to have something Ruby-like that was more intimately tied into the JVM ecosystem? Seems to me that Go would be just about the _worst imaginable_ [0] implementation language for that.

[0] Or at least, in the class of extremely poor choices that includes all languages that are not themselves especially targeted to the JVM platform.

<comment><USER41> I think his point was instead of using the JVM runtime to now use the Go lang runtime. Of course you would need the interpreter for each platform like Python does but Go has a unique runtime. I'm not sure how dynamic it is but you can certainly run a scripting language in it and make it easy to access other Go stuff.

I'm not sure how easy it would be to access ad-hoc libraries without recompile though since Go doesn't really have dynamic loading of libraries (or maybe it does now?).

Little tangental ... After reading his blog I'm a little concerned though that the dude (Gavin) has spent an enormous effort defending his stance on how poorly the Groovy organization is... I'm not saying his wrong but is almost obsessive and a waste of energy.

<comment><USER43> > I think his point was instead of using the JVM runtime to now use the Go > lang runtime.

Right, my point was that a central part of the _point_ of Groovy was being for the JVM, it wasn't a language in search of a runtime.

So, while there might be an argument that a language _like_ Groovy for Go runtime might be useful (I'm somewhat unconvinced of that), it just seems not to be the case that Go is a good choice for a language _such as_ Groovy (that is, it is not the case that Groovy is an example of a language for which it would make sense to use Go as the implementation language.)

</comment></comment></comment></comment></comment></comment></comment><comment><USER44> I first learned Python and then much later C, and for me at least learning Python first was very motivating because you can get things done so easily, due to the built-in batteries and the wealth of external libraries. Seeing that the computer needs less than a second for what would have taken me many hours of tedious work was pretty eye opening.

Learning C later on helped me understand some basic concepts better and being able to speed up Python e.g. using Cython can come in quite handy.

</comment><comment><USER45> Python is so easy to write, use and read, I would not understand how a programmer would not know it already.

It literally has everything you want in a language.

Granted, it's not on-the-metal fast, and it still has some gimmicks, like variables behaving like pointers or references (which can be a little awkward), but still, python is the language of the decade.

<comment><USER46> Ugh. I tell you, I keep trying to pick it up because I hear so much about it. However, when I code in it I feel completely uninspired. I can only get so far into a book or tutorial before my eyes glaze over and I'm thinking about something else.

I think when I code, I want to _own_ the machine, not just skate on the surface. Python doesn't even crank my engine the way BASIC did years ago.

I also think part of my problem is it doesn't solve any needs right now.

<comment><USER45> Well python is not for solving hard problems. If you want to own the machine you need C++ or C.

Python is a language that makes programming easy. That's a lot.

</comment></comment></comment><comment><USER47> One my feeling about python is that is a great language for "reference". Probably everybody and write and read it and its already good enough for general programming. There are edges where other languages are better but still is a great language to establish communication among different types of developers

<comment><USER48> Yes, I've seen Python described as "everybody's second-favourite language". It's not the best at anything, but if your main requirement is a bunch of different people all need to be able to read and edit the same code then it's a good choice.

<comment><USER49> Couldn't we say the same thing about Ruby?

<comment><USER21> Not really. Ruby isn't anywhere near as widespread or well supported in fields like finance, math, various scientific fields and GIS. Python on the other hand has at least has pretty good support in just about any domain you happen to stumble into.

</comment></comment></comment></comment><comment><USER50> The long history of Python support in the scientific community has produced a rich catalog of domain-specific analysis libraries. Warts and all, it's simply not possible, in most languages, to achieve the analytical productivity you get when working in Python.

</comment><comment><USER51> I've had to deal with Python for a long time. I even wrote my senior research project with it in college.

But since then, I've never really had a need to use it, and I don't even feel there is a good reason for it.

If you're writing something more than a quick shell script or a project that will last more than a few days, then there a dozen static languages that have better libraries, more powerful IDEs, etc. And misspelling a variable or missing a comma is not going to cause all sorts of problems.

On the other hand, if you do need a quick one-off script or some duct tape, things like Perl or AWK, sed, and friends seem like a better bet.

Python (and Ruby) seem bloated for quick scripts, and wholly lacking for larger projects.

</comment><comment><USER41> I have contemplated writing a "Why You Should Learn Bash" article and then maybe "Why You Should Learn Regex" followed by "When to not use Regex".

If you know bash you probably have a pretty good idea how to manage Linux and roughly how things work on Linux (you could apply this to other Bash capable OS as well).

If you know Bash fairly well you probably also know how to automate many things and you probably already know regex which is fairly useful in GSD. If you know Bash you obviously know the command line well.

You should learn Bash (or some shell but Bash is so ubiquitous).

</comment><comment><USER52> The "namecount" example is rather silly though. It can be solved by a shell script much shorter than one line (sort|uniq -c).

<comment><USER53> Hey there! I've answered a similar observation in the comment section of the blog :)

> Of course it can be done all with command-line tools, but the point was not > to show you how you can count and sort word occurrences. I just wanted to > show that it's possible to include Python scripts in your "command line > workflow". I wanted to give a simple example that would give you a taste of > the idea :)

</comment></comment><comment><USER54> Why do programmers try to convince others which programming language to use? Why should they care what languages others are using?

<comment> jorge-fundido The more popular your preferred tooling, the more opportunities in the job market.

</comment></comment><comment><USER55> I read lots of people complaining about medium-to-large scale Python projects, due to lack of static typing. Aren't people using reflection and dynamic in C#? As soon as the code touches such run-time features, the testing at run- time is essential.

I wonder how large projects such as OpenStack, Django, Edx, PyPy deal without static types?

</comment><comment><USER56> I hate Python. My linux distro comes with version 2.7 yet I should be using version 3?? Yes it makes writing complex programs quickly with its endless libraries, but I don't consider myself learning anything if all I'm doing is copy pasting the API docs. I've recently switched to Go and the language is small and easy, docs/source code are well commented, and lastly I'm actually learning how computer programs work.

<comment><USER32> Most ppl are trying to solve problems/get work done, not learn how to re- implement functionality already available in libraries. Copy & pasting the API docs lets you focus on solving much harder & more interesting problems.

If you want to learn how computer programs work, you don't have to use existing libraries - you can still implement the functionality yourself.

By your argument (I hate python...it has libraries) any modern & moderately successful language with a large library ecosystem is bad, and you prefer Go precisely because it lacks those libraries?

</comment></comment><comment><USER57> I'm learning python just to be able to use tensorflow. I wish there was an interface in Ruby

</comment><comment><USER58> I was a long time Rubyist but switched over to Python because even though its characteristics and philosophy made me reflexively shirk it at first -- particularly significant whitespace, one-line-lambdas, and the clusterfuck of 2.x vs 3.x -- it seemed to me to be an obviously easier language to _teach_ , nevermind its significantly stronger libraries for numerical work.

I'm glad I made that decision. The language was easy enough to pick up in my spare time. I hardly ever use pandas/numpy unless I'm doing some really serious statistical work; the core language and patterns in Python are now pleasant enough for me to use in day-to-day data wrangling and hacking. I'm now mixing in Ruby for a current project -- mostly because I like Rakefiles, and also, Ruby's static site generators like Middleman -- and it's been surprisingly painful to do things in Ruby. I used to hate how when doing a Google search for something in the Python stdlib would bring up 2.x before 3.x...but it almost seems even more problematic in Ruby, where 1.9.3 and 2.0.0 docs keep coming up, even though official maintenance of them has been dropped.

(2.0's maintenance ended in Feb. 2016 [http://ruby- doc.org/core-2.0.0/Array.html](http://ruby-doc.org/core-2.0.0/Array.html))

If I didn't know better, I wouldn't even know what Ruby's official version was. For me, Google results show no other documentation for Ruby's Array, as the next results are from tutorialspoint and sitepoint.

And that's not even getting into the issue of how problematic the do-things- how-you-like freedom of Ruby is incredibly hard to discern as an outsider. To give one example I recently had: how do you iterate through a human-calendar- based time interval, such as a month (i.e. any interval that doesn't have a constant length in seconds)?

In Python, I remember how frustrating it was to figure out what the hell the difference was between datetime, time, calendar, the gmtime function, and then timedelta, and then the third-party libraries of pytz and python-dateutil. But once I understood them, which was largely a matter of realizing how naive I was about the nature of measuring time, doing something like iterating over calendar intervals is straightforward.

This is my first Google result for "python iterate month intervals" \-- it's from 2008 and still works perfectly: [http://stackoverflow.com/questions/153584/how-to-iterate- ove...](http://stackoverflow.com/questions/153584/how-to-iterate-over-a- timespan-after-days-hours-weeks-and-months-in-python)

Now here's the result for Ruby; all of the answers, as far as I can tell, are not only not-very-Rubyish, but not reliable either: [http://stackoverflow.com/questions/1724639/iterate-every- mon...](http://stackoverflow.com/questions/1724639/iterate-every-month-with- date-objects)

FWIW, I remember solving this problem by using the DateTimein ActiveSupport. Which is a credit to how much fun and joy it is to build things in Rails right out of the box. But that complexity catches up quick, and including ActiveSupport in day-to-day Ruby work can sometimes to unwanted monkeypatching.

<comment><USER59> As a Ruby fan who is baffled by Python's continued popularity, I like this comment - I'm always interested to hear why people have such different experiences than me trying to move from the former to the latter.

That said, unless "recently" was years and years ago, Date#next_month is a thing:

      require 'date'
      date = Date.today
      while (date < Date.today.next_year)
        puts date
        date = date.next_month
      end

One thing that I think throws a lot of people is that the official Ruby docs are actually _better_ than Google for figuring stuff like this out - it only takes a couple minutes to find #next_month by perusing the method listing at [http://ruby-doc.org/stdlib-2.3.1/libdoc/date/rdoc/Date.html](http://ruby- doc.org/stdlib-2.3.1/libdoc/date/rdoc/Date.html), but do a web search and you get a bunch of Railsy crap.

Though I will grant you that even searches on ruby-doc.org keep bringing up results for Ruby 2.0.x and whatnot. That part is pretty bizarre.

</comment></comment><comment><USER60> I thought this was a helpful article!

</comment><comment><USER61> I just wanted to point out a bit of syntax from the article. I'm a huge fan of compact variable declaration if statements. The code from the article:

        if name in names:
            names[name] += 1
        else:
            names[name] = 1

That can actually be written as just one neat, readable line:

        names[name] = names[name] + 1 if name in names else 1

It may not be as readable to some who are used to spelling it out as an if/else block, but I really prefer the one line way. It reads closer to regular English I think.

<comment><USER32> NO. That is not more readable, and is warned against in most style guides and code-quality books.

In the original, at a glance I know exactly what is going on.

In your version, I have to read the whole sentence carefully to notice that it's even a conditional and not a normal assignment, and then I have to mentally unpack it to understand the logic that you're trying to implement. If/else should never be a one-liner.

Good code is boring code :)

<comment><USER62> Yup, but the right way to do it would be a defaultdict, or a Counter anyway.

        from collections import defaultdict

        def count_names():
            names = defaultdict(int)
            for name in sys.stdin.readlines():
                name = name.strip()
                names[name] += 1
                ...

</comment></comment></comment><comment><USER63> My initial progression of languages was: QBasic, Visual Basic, C++, Java (1.5), Python. Python was leaps and bounds more fun and productive than flailing away with any of those.

But today, I'm less excited about it. I've gotten spoiled by C#, that has static typing, and consequently, better tooling, as well as a good measure of the flexibility of Python, through dynamics, lambdas, LINQ.

It seems Python is finally getting over the 2.X -> 3.X schism, but that kind of threw a wrench into things for a number of years, as well.

</comment><comment><USER64> Is it 2006 again?

</comment><comment><USER65> For all its virtues, I feel introductions to Python tiptoe around a very important topic: yes, Python is nice and easy, but _you 'll have to program the Python way too_.

Try running a nested loop on a non-trivial example, and you can end up spending minutes in what would take milliseconds in any other language. If you want to program in Python, you must get used to the functional paradigm. Not "should", "must".

Now, I'm all in for bringing more functional programming into daily life, but I find a bit dishonest that beginners are not told about this. My first serious program took about a week for a task, and shaving that time down involved a lot of pain changing simple nested loops into multiplications of Numpy matrices. There was a lot of caching too. I have no idea how someone without a CS degree would have dealt with this.

<comment><USER66> > Try running a nested loop on a non-trivial example, and you can end up > spending minutes in what would take milliseconds in any other language.

Huh? Can you provide an example? There's nothing about writing nested loops in Python that is qualitatively different from other languages.

> If you want to program in Python, you must get used to the functional > paradigm. Not "should", "must".

This is incorrect, much to my chagrin. Python's trend over the past few years has been not only _not_ to encourage functional programming, but to explicitly _discourage_ functional programming. For example, starting in Python 3, `reduce` has been banished from the language proper to the standard library.

<comment><USER21> _Huh? Can you provide an example?_

Well he did mention numpy and obviously doing something that can be done in numpy with nested loops will be much much slower. However that is as much a case of numpy being really fast as python being slow.

For example, just tested elementwise multiplication of two 10kX10k matrices and with numpy and numpy arrays it took ~350 ms vs ~15 seconds with a nested for loops and python lists. Still minutes vs. milliseconds seems like extreme hyperbole.

edit: Figured I'd test how long the naive nested loop approach would take in a 'fast' language like Julia, and much to my surprise it took over 50 seconds.

edit2: cleaned up all my code to try to make it as similar as possible across both languages:

Nested for loops python 2.7: ~30 seconds

Nested for loops Julia 4.6; ~50 seconds

Nested for loops PyPy 5.3.1 ~0.9 seconds

Make of these numbers what you will

<comment><USER67> Could you post the Julia code? Unfortunately, you do have to make sure that the types are correctly inferred by the compiler in high-performance loops. Or maybe you used a global.

[http://docs.julialang.org/en/release-0.4/manual/performance-...](http://docs.julialang.org/en/release-0.4/manual/performance- tips/)

<comment><USER21> I'll admit I don't really know that much Julia and basically wrote naive MATLAB code (and I wanted to make it as close as possible to my python code):

      s=10000;
      a=ones(s,s);
      b=ones(s,s);
      c=zeros(s,s);

      tic();
      for i in 1:s
        for j in 1:s
          c[i,j]=a[i,j]*b[i,j]
        end
      end
      toc();

Obviously in real code I'd simple write

      c=a.*b

and get basically the same performance as numpy

<comment><USER67> Tarrosion is right, on my machine, putting the code inside a function is 10X faster, and then swapping the order of the loops gives another 10X improvement:

        function f(s)
          a=ones(s,s);
          b=ones(s,s);
          c=zeros(s,s);

          tic();
          for j in 1:s
             for i in 1:s
              c[i,j]=a[i,j]*b[i,j]
             end
          end
          toc();
          return c
        end

Note that you also might have to run the function twice, because the first time JIT-compilation kicks in.

Julia 0.5 also now automatically devectorizes some code (eg. x = a .* b .+ c) so that you don't have to write explicit loops to get performance benefits.

<comment><USER21> Thanks. To both you and Tarrosion.

Guess I need to take some time at some point to sit down and actually learn 'proper' Julia.</comment></comment></comment></comment></comment></comment></comment>


Train Philosophers with Pearl and Kahneman, Not Plato and Kant - <AUTHOR> http://lesswrong.com/lw/frp/train_philosophers_with_pearl_and_kahneman_not/ <comment><USER2> Plato and Kant got a lot right about human nature. For example, Plato sort of nailed the idea of vocational education in the latest techniques (rhetoric for politicians in post tyranny Athens) verus teaching people to think deeply and question their own assumptions. While, Kant described the limits of empirical knowledge and is sort of where the idea that what we know about things in the world is probabilistic (versus certain _a priori_ or Humeian fantasmagora) comes from.

<comment><AUTHOR> I agree. Aristotle also got the rough shape of what it is to give a formal account of logic right, and the rough shape of ethics.</comment></comment>


Java 10 – Specification for Value Types - <AUTHOR> http://cr.openjdk.java.net/~dlsmith/values.html <comment><USER3> The following bytecodes have been introduced: vaload vastore vbox vdefault vload vreturn vstore vunbox vwithnewfield

These will have direct impact on the JVM. Is there a test version of the JVM with these bytecodes already implemented? Not sure if Java follows similar rules as the IETF, show us working code with the specifications.

<comment><USER4> If they're doing breaking changes to the VM, I wonder if we can get rid of type erasure. Generics seem really nice until you actually use them to do anything mildly complicated, then type erasure rears it's ugly head and you now have a type parameter and a Thing.class normal parameter and reflection.

<comment><USER5> Those are not breaking changes. Existing byte code will continue to work.

My understanding of generic type erasure is that it is more of a language issue than a VM issue. Nothing in the VM prevents reified generics, but when generics were introduced in Java 5 they decided to use type erasure in order to avoid having to produce a whole new collections standard library or break compatibility with existing code.

<comment><USER4> I meant breaking in that the Java 10 compiler would be targeting a newer version of the JVM and could dispense with having to be backwards compatible, which was one of the reasons I remember reading for generics.

They could have simply left the old library, deprecated it, and added in their genericed one as a a new package. It had to be modified anyway.

<comment><USER6> Right, it breaks _forward_ compatibility: bytecode for a newer VM won't work on an older VM. IMO backward compatibility is much more important for a programming language, though.

</comment></comment></comment></comment></comment><comment><USER7> Hi all - interested in early access builds and / or helping out? Head to OpenJDK.java.net and join the Valhalla project and the Adoption Group.

Disclaimer - I help run the adoption group and maintain the Valhalla wiki.

</comment><comment><USER8> This title is hugely misleading. The value types specification is for JVM, the platform, not Java 10. You can compile a large number of source languages to run on JVM, such as Java, Scala, Closure, Groovy, Kotlin, Javascript etc. just to name a few.

<comment><USER9> Is it really misleading though?

Does anyone with the slightest familiarity with JVM languages not understand that each major Java language version is also associated with a major JVM version?

<comment><USER8> Given that we are not even close to the official release of Java 9 and whether the proposal to JVM specification will be accepted and implemented for which version, it is largely inaccurate to title the article "Java 10 - Specification for Value Types" for a JVM change proposal. Besides, interchangeably referring to JVM and Java reflects the ignorance about the whole ecosystem of JVM languages.

<comment><USER9> Well the specification explicitly talks about version 54 class files, which it then goes on to state "Version 54.1 class files are intended to be supported only by implementations of Java SE 10.". It's not going to land with Java SE 9.

<comment><USER8> With all the prolonged delays of features in Java, whether it is Java 10, 11, or 12 is still in question. As an instance, Jigsaw was proposed to be a driving feature of Java 7, and then delayed to Java 8 and even Java 9. And recently it was voted "No" by IBM, Red Hat etc.

<comment><USER9> Sure, features slip, but value types are coming in java 10 until they aren't.

</comment></comment></comment></comment></comment></comment><comment><USER10> In its own way this is a sad day. Yes, Java's getting better, but it marks another nail in the coffin for the dream of a smart compiler. In this case, escape analysis never lived up to its promise.

<comment><USER11> Could you elaborate on what a smart compiler is?

<comment><USER12> A [http://wiki.c2.com/?SufficientlySmartCompiler](http://wiki.c2.com/?SufficientlySmartCompiler) in this case would be a compiler capable of automatically converting classes into value types such that the developer did not have to think about this detail of memory management themselves.

I felt the same slight sadness when I saw the complexity of the planning involved in Java value types, and the many-year path to get there. Intuitively, a sufficiently smart compiler _should_ have been able to take care of this, and in some cases it does do. So it's worth reflecting on why adding new opcodes and such is necessary.

HotSpot and some other JVMs can do an optimisation called "scalar replacement", which converts objects into collections of local variables which are then subject to further optimisation. So for example a Triple<A, B, C> type class could be converted into three variables, then the optimiser notices that the second element of the triple was never actually used anywhere, and eliminates it entirely.

Scalar replacement is one of several optimisations that relies on the output of an escape analysis. Escape analysis is a better known term so people often use the name of the analysis to mean the name of the optimisations it unlocks, although that's not quite precise. Sometimes people talk about stack allocation, but that isn't quite right either. Only objects that don't "escape" [the thread] can be scalar replaced.

There are several reasons this isn't enough and why Java needs explicit support for value types.

Firstly, the JVM implements two different EA algorithms. The production algorithm that is used with the out of the box JIT compiler (C2) is somewhat limited. It can identify an object as escaping just because it might escape in some situations, even if it often doesn't. There is a better algorithm called "partial escape analysis" implemented in the experimental Graal JIT compiler, but Graal isn't used by default. In Java 8 it requires a special VM download. It'll be usable in Java 9 via command line switches. PEA can unlock optimisation of objects along specific code paths within a method even if in others it would escape, because it enables the un-optimisation of escaped objects back onto the heap.

Graal unfortunately can't be just dropped in. For one, you don't just replace the entire JIT compiler for a production grade mission critical VM like HotSpot. It may take years to shake out all the odd edge case bugs revealed by usage on the full range of software. For another, Graal is itself written in Java and thus suffers warmup periods where it compiles itself. The ahead-of- time compilation work done in Java 9 is - I believe - partly intended to pave a path to production for Graal.

Implementing PEA in C2 is theoretically possible, but I get the sense that there isn't much major work being done on C2 at the moment beyond some neat auto-vectorisation work being contributed by other parties. All the effort is going into Graal. I've heard that this is partly because it's a very complex bit of C++ and they're afraid of destabilising the compiler if they make major changes to it.

Unfortunately even with PEA, that still isn't enough to replace value types.

(P)EA can only work on code that is in-memory when the compilers optimisation passes are operating, and moreover, only in-memory in the form of compiler intermediate representation. In a per-method compilation environment like most Java JITCs that means it can only work if the compiler inlines enough code into the method it's compiling. Graal inlines more aggressively than C2 does but it still has the fundamental limitation that it can't do inter-procedural escape analysis.

It can be asked, why not? What's so hard about inter-procedural EA?

That's a really good question that I wish John Rose, Brian Goetz and the others had written a solid full design doc or presentation on somewhere. They've said it would be incredibly painful, and obviously view the Valhalla (also incredibly painful) path as superior, so it must be hard. In the absence of such a talk I'll try and summarise what I learned by reading and watching what's out there.

Firstly, auto-valueization - which is what we're talking about here - <USER2> necessitate much, much larger changes to the JVM. Methods and classes are the atoms of a JVM and changing anything about their definition has a ripple effect over millions of lines of C++. HotSpot isn't just _any_ C++ though - it's one of the most massively complex pieces of systems software out there, with big chunks written in assembly (or C++ that generates assembly). The Java architects have sometimes made references to the huge cost of changing the VM. They clearly perceive changing the Java language as expensive too, but compared to the cost of changing HotSpot, it can still be cheaper to complicate the language. Frankly it sounds like Java is groaning under the weight of HotSpot - it's fantastically stable and highly optimised, but that came at large complexity and testing costs that have to be considered for every feature. Part of the justification for doing Graal is that the JITC is the biggest part of the JVM and by rewriting it in Java, they win back some agility.

As an example of why this gets hard fast, consider that the compiled form of a "valueized" version of a parameter is different to the reference form. Same for return value (a value can be packed into multiple registers). So you either have to pick a form and then box/unbox if the compiled version doesn't 'fit' into the call site, or compile multiple forms and then keep track of them inside the VM to ensure they're linked correctly. And a class that embeds an array may wish to be specialised to a value-type array, but then you need to detect that and auto-specialise, detecting any codepaths that might assume reference semantics like nullability or synchronisation, and then you have to be able to undo all that work if you load a new class that violates that constraint.

Secondly, detecting if something can be converted into a value type (probably) requires a whole program analysis. Java is very badly suited to this kind of analysis because it's designed on the assumption of very dynamic, very lazy loading of code. Not just because of applets and other things that download code on the fly, but it's quite common in Java-land for libraries to write and load bytecode at runtime too. Also Java users and developers expect to be able to download some random JAR and run it, or a random pile of JARs and run it, with essentially no startup delay. HotSpot can run hello world in 80 milliseconds on my laptop and the very fast edit-run cycle is one of the things that makes Java development productive relative to C++.

All that said, applets are less important than they once were, and Java 9 does introduce a "jlink" tool that does some kinds of link-time optimisations:

[https://docs.oracle.com/javase/9/tools/jlink.htm#JSWOR- GUID-...](https://docs.oracle.com/javase/9/tools/jlink.htm#JSWOR-GUID- CECAC52B-CFEE-46CB-8166-F17A8E9280E9)

[https://gist.github.com/mikehearn/930e3001e76415ae1e2d85dcdf...](https://gist.github.com/mikehearn/930e3001e76415ae1e2d85dcdfc5d4eb)

<comment><USER13> That's a long post and I admit I didn't read all of it. But, is anything you've said there a consideration for most "business" software? I can understand on a cache-hitting trading app perhaps.

<comment><USER10> It's important if you care about GC pauses (which you will eventually). Take a look at this report from the .NET based stack overflow guys: [http://blog.marcgravell.com/2011/10/assault-by- gc.html?m=1](http://blog.marcgravell.com/2011/10/assault-by-gc.html?m=1)

Note that they solved their problems using exactly the features being added in Java 9.

</comment></comment></comment></comment></comment><comment><USER14> C++ has had these things for years.

Seriously though, the JVM is killing it. Can't wait until I can compile my .jar to verilog and run it on my FPGA, or send the verilog to TSMC and get some ASICs printed out.

<comment><USER15> > _C++ has had these things for years._

C++ is _built_ on these things. The "value type" thing informs everything about the language. I love it, and it's my go-to language when something just has to be fast, but it comes along with some hefty baggage. Java gets a lot of mileage out of its reference-first philosophy that C++ would be wise to try out -- the "everything must be possible with no overhead" philosophy of the language bleeds into the programming community in unhealthy ways, and the amount of language machinery to make the value stuff work "as well as possible" ([https://stackoverflow.com/questions/3601602/what-are- rvalues...](https://stackoverflow.com/questions/3601602/what-are-rvalues- lvalues-xvalues-glvalues-and-prvalues)) is absurd.

> _Seriously though, the JVM is killing it_

As for the JVM and Java, this seems like a guilty admission that the CLR and C# ate their lunch a decade ago (perhaps not wrt performance, but for language features....) I've never worked in the Microsoft stack, but the stories have been the same since the dawn of the age -- "C# is Java with real generics, value types and lambda functions." First they laughed...

<comment><USER16> As great as structs in C# for performance reasons are: They also provide several new ways to shoot yourself in the foot if you don't understand their behavior to 100%.

One example that I run many years ago was having a struct Foo { int x; } which was nested inside a List<Foo> myList. Trying to mutate one list element with syntax like myList[0].x = 27 silently failed, because the indexer of the list returns a copy of Foo and you mutate that inside of writing it back. Nowadays its a compiler error, but back then it wasn't. The workaround is something like "Foo f = myList[0]; f.x = 27; myList[0] = f;". Which is quite weird if you are coming from C++. From a C# point of view it's actually obvious, since you can't pass around references and pointers to structs inside of other things - and if you also add this functionality, you get lots of the complexity from C++ back (references to structs, functions/properties that return references, questions around reference lifetimes, etc.). Without having also reference semantics for structs, structs also can't achieve the kind of performance which "value types" in C++ have, since they sometimes need to be copied fully.

Another nice gotcha is the surprising behavior of readonly structs, which e.g. is documented here: [https://bytes.com/topic/c-sharp/answers/261922-readonly- beha...](https://bytes.com/topic/c-sharp/answers/261922-readonly-behavior- structs)

For those reasons I think it's quite fair that Java and other languages did not immediately jump onto the value types train. I think introducing them carefully, and trading off the performance gains against the amount of new complexity in the language is fine.

<comment><USER17> When Java was introduced we already had plenty of GC enabled languages with value types.

Eiffel, Oberon, Modula-2+, Modula-3, VB, are just a few examples.

Hence why I always saw as lost opportunity not to provide them in first place.

</comment></comment></comment></comment><comment><USER18> What is the use of value types if it cannot be the argument of generics?

<comment><USER19> Many

[http://cr.openjdk.java.net/~jrose/values/values-0.html](http://cr.openjdk.java.net/~jrose/values/values-0.html)

<comment><USER20> This was handy to glance through. Thanks!

I did notice one point that struck me as odd (under "Details, details"):

> Can a value class implement interfaces? Yes. Boxing may occur when we treat > a value as an interface instance.

I don't normally think of values in other languages as having methods directly or an interface like this. I wonder what the primary reasons for this would be. Backwards compatibility might be one reason.

<comment><USER17> A few examples are D, C#, VB, F#, Eiffel, Delphi, C++, Rust.

<comment><USER20> Thanks for the examples. Seems clear that I don't fully understand it then, but it's good to know that it's common so I can look around some more.

</comment></comment></comment></comment></comment><comment><USER21> Does anyone how this would work if there were two value types with the same members/definition but from different libraries - would they be interoperable? For example, a mathematical function library and a chart plotting library might both define complex numbers. It would be good if you didn't have to write (trivial) conversion functions every time you wanted to use them together, like you do now. (I think this is called 'structural typing'?)

<comment><USER22> It appears not and that's a good thing here. Java is based on explicit typing and I do not expect things to work w/ implicit conversions between value types. Granted, that doesn't mean that a JVM implementation may not have a higher performing way to perform a cast between two of the same struct. There is of course boxing/unboxing that comes w/ converting between value and ref classes.

</comment></comment><comment><USER23> Looks cool, but I didn't see Java 10 anywhere. What's the timeframe for this? Is this truly a part of Java 10, considering that Java 9 is still in development?

<comment><USER24> The new release date is September 21: [https://jaxenter.com/java-9-schedule- change-sept-134484.html](https://jaxenter.com/java-9-schedule-change- sept-134484.html)

It's been moved a lot and AGAIN, but I believe "the target is starting to move slower" :)

</comment></comment><comment><USER25> Why not just "value types" and "value class types" instead of long "direct value class types"?

<comment><USER26> Because references are values.

</comment></comment><comment><USER27> Is there a TL:DR version of this?

<comment><USER28> Java 10 may support defining your own primitives.

e.g. You could define a tuple-like type that will get passed and stored by value. Now you can make a billion member Array of 'em and not incur the speed and memory overhead you'd get from using Objects that need to be tracked, dereferenced, garbage-collected, etc...

</comment></comment><comment><USER29> <3 Java and Java EE! Glad to see the innovation wheels turning</comment>