Chinese erotic writer jailed for more than 10 years over gay sex scenes in novel - <AUTHOR> https://www.scmp.com/news/china/politics/article/2173814/outcry-chinese-erotic-writer-jailed-more-10-years-over-gay-sex <comment><USER2> The Reddit discussion said the issue was not about being gay but about being pornography, sold illegally, to minors, violating all sorts of copyright and porn related regulations.

Apologies for not having a link handy from my cell phone, I'd appreciate if someone could look it up and respond with links.

<comment><USER3> Thank you. Correlation != Causation, but I guess logic doesn't apply when it comes to anything China related discussion here. Saying that, this is still an ridiculously outdated law, 10 years sentence seems rediciculous.

</comment></comment><comment><USER4> Its not safe to be queer. There are broad societal protection that one loses out on the second they aren't straight acting and cis-gendered, all it takes is a shift in the political winds for governments to start rounding up any class or minority they don't like.

There are "safer" places in the world to be queer, but the threat of danger and violence never goes away, even in gayborhoods like here in parts of Seattle.

<comment><USER5> > cis-gendered

Not to take anything away from your point, but if you mean white males, then why not say so? And what do I (a white male) have to do with any of this? Should I feel guilty now?

<comment><USER6> "cis-gendered" just means "same gender identity as birth" \- [https://en.wikipedia.org/wiki/Cisgender](https://en.wikipedia.org/wiki/Cisgender)

Nothing to do with white or male.

</comment></comment></comment><comment><USER7> Censorship of erotic material isn't a purely Chinese thing, other countries practice it at smaller scales. In England, various fictional artworks can be prosecuted under the Obscene Publications act, even _possession_ of some cartoon pornography can be prosecuted under CAJA 2009 s.63. To the dismay of LGBTQ+ and sexual freedom campaigners, even possession of simulated rape pornography is illegal.

That's not to say that homosexuality can be compared to fetish porn, but people should be aware that in some countries without constitutional protection for free speech (or those with one but with an "obscenity" exception, looking at you Canada) censorship of harmless porn is par for the course.

(Would someone mind explaining why this is being downvoted? In my experience, few people are familiar with such laws in Western countries, and of course I condemn China's laws more; is this comment off topic? Please tell me if I can improve.)

<comment><USER8> I would contend that there can be harms involved even with porn that you might consider harmless; harms such as addiction, damaged relationships, objectification of partners. Porn can damage minds.

<comment><USER7> Many things (like alcohol) can be addictive and damage relationships. Who objectifies others should be a personal choice, censorship should not be performed for the purpose of protecting people from thinking in a certain way. Plenty of other information can damage minds too, can't it? Who's to decide which porn is more or less damaging, unless you support a blanket ban on all pornography? Come to think of it, even non-pornographic fiction can damage minds; films involving drug use, James Joyce's Ulysses.

<comment><USER9> > Who's to decide which porn is more or less damaging

Or just what is meant by 'damage'? Discourse here is unlikely to be a friendly academic discussion on the meaning of words, but instead another instance of the eternal question "Who? Whom?"</comment></comment></comment></comment>


Florida's citrus industry is being killed by an incurable disease - <AUTHOR> https://www.washingtonpost.com/climate-environment/2019/11/09/end-florida-orange-juice-lethal-disease-is-decimating-its-citrus-industry/ <comment><USER2> Florida Citrus Industry Is Facing An Existential Threat From Bacteria, But A Virus Offers Hope [https://www.forbes.com/sites/stevensavage/2019/04/30/can- the...](https://www.forbes.com/sites/stevensavage/2019/04/30/can-the-florida- citrus-industry-be-saved/#380c47a92ecf)</comment>


Facebook will filter 'fake news' in Germany - <AUTHOR> http://www.businessinsider.com/facebook-tests-fake-news-filtering-in-germany-2017-1 <comment><USER2> Oh man I'm so sick of this 'fake news' hype here in Germany.

It's the internet. Get used to it!

edit: I'm also very curious how Facebook plans to accomplish it. They probably gave the usual response 'yeah we'll fix it', so our politicians and news outlets stop talking about this nonsense.

</comment><comment><USER3> Major news outlets in Germany publish unverifiable news from undisclosed sources as well. I fail to see why Facebook should get a special treatment in this regard.

<comment><USER4> Left wing media and governments go hand in hand.

</comment></comment><comment><USER5> correctiv.org is independent non-profit, based in Berlin, which is interesting setting. This is a rather small company, with editorial office in Essen. It looks like it relies on 16 journalists (some of them high-calibre), and one traineeto fulfil their duties. Of which Facebook is not the only one. Which inevitably poses a question, how good is this backup and is this not another kind of attempt of a Dutch boy stopping flood of the whole Nederland with his finger in the dam?

The dam is of course the German FB, standing the flood of fake news. I failed to recognise how many 'viral' stories on FB are fake, however, so can't really estimate that Correctiv is much of a finger in such a dam.</comment>


Education Killed The Creativity [In Me] - <AUTHOR> http://www.creativitykilledtherecession.com/?p=332

<comment><USER2> I'm currently struggling with a daughter who is academic and artistic but hates school because creative subjects are shoved out of the schedule to meaningless exams, tests and repetitive class work. If she gets creative within a maths lesson she is marked down for not using the prescribed method!

She's only eleven and she already thinks that school is a waste of time despite achieving great results.

There is a lot wrong with the education system in the UK.

<comment><USER3> Do you think it's reasonable or fair to expect the public school system to be the ones to develop an extraordinary child?

How many resources should they apply to the top 2% of children?

<comment><USER4> Yes. If they don't, who should? The parents? Many parents are completely useless (or worse); should we just write off their children?

<comment><USER3> The point is that I'd be content if government served the general case of population well. Super-bright kids are a special case there will be less harm to society should they be under-developed.

I know that's horribly utilitarian of me but I'm just being pragmatic.

</comment></comment></comment></comment><comment><USER5> Schools receive their incentives for (1) high standardized test scores, and (2) promotion/graduation rates. There are no incentives for producing creative, literate, and critically thinking people. Until that changes, schools will continue to break our kid's minds and corrupt their way in the world.

</comment><comment><USER6> "Every school I've visited was participating in the systematic suppression of creative genius" - Gordon MacKenzie

<comment><USER7> How many schools did he visit?

</comment></comment><comment><USER8> "creativity should be as important to education as literacy"

We seem to have found the problem.

</comment><comment><USER3> As far as I'm concerned the schools are just putting all their eggs in the basket of kids who can do well in the system vs the abstractly intelligent ones who do well when they are so inspired/inclined.

For all the gifted and unique snow-flakes out there, the sooner they learn to cope with a system (and a world) which doesn't cater to their extraordinary talents, the better off they'll be.

</comment><comment><USER9> I dropped out of school at 16 (in the UK), and I did a deal with the school whereby I'd take just 5 exams (maths, english, history, double science). I got C's in them, not enough to get into a good college.

I tried a computer course, failed after year 1, and took Media Studies for 2 years and passed.

A few years passed, and then I completed my first ever computer course (eg, having never passed any computer course) which was a 3-year degree.

It is possible to do higher learning further into adulthood, and easier!

[Edit] What has this to do with creativity? Well, not a lot. But I've written stories all my life. I also think excessive use of alcohol kills creativity, and these days, a degree is half about getting drunk and socialising.

My only advice is: Keep writing, even if it means missing out socially. Also try and keep things interesting. I signed up to help a computing legend with his software before my degree, and that was no end of fun. I was volunteering at an ACM conference when I was barely into my 2nd year of University, thanks to this work. He also guest lectured at my University, which was really strange.

In short - find a mentor, even if it doesn't pay.

</comment><comment><USER10> I think this is inevitable when governments have anything to do with schools. Even though we're seeing a lot of these "education today sucks!" articles, I realize very few of the authors (and very few people here at HN) would agree.

</comment><comment><USER11> Great Ted Talk on education & creativity: [http://www.ted.com/index.php/talks/ken_robinson_says_schools...](http://www.ted.com/index.php/talks/ken_robinson_says_schools_kill_creativity.html)</comment>


Snowden leaks: GCHQ 'spied on Facebook and YouTube' - <AUTHOR> http://www.bbc.co.uk/news/technology-25927844

<comment><USER2> Of course they spied on YouTube and Facebook.

What's more important is whether they obeyed the law; and whether we know what the law actually means; and if the peoe doing the oversighting know what the law says.

One example is the debate about granting access to metadata. GCHQ was involved in that debate and lots of reporters wrote articles. But no one noticed that the existing laws exempt GCHQ and that they had been slurping this data all along.

All the data collected by squeaky dolphin appears to be publicly posted information, and used in bulk, so it's less troubling than privately posted information or individual's identified data.

I'm not sure if the "identify and target for propaganda" programme was ever created or was just a proposal.

The Interceptions of Communications Commissioner has reports about mistakes in information interception, and GCHQ does report mistakes to that office. But oversight still needs to be improved.

</comment><comment><AUTHOR> > An earlier GCHQ operation [...] to identify users and target them with > propaganda

I like how the UK gov thinks of "democracy".

> However, the prime minister has previously said that: "I'm satisfied that > the work these agencies do is not only vital but is properly overseen."

Ok, we can go back to sleep then. Thanks a lot, Mr. Cameron.</comment>


Ask HN: How to Design a University Course? - <AUTHOR> I will be teaching an undergraduate course next semester (on computer graphics if it matters). Are there any tips/tools/books/resources on how to design and teach a course? <comment><USER2> General advice: giving a brief historical context and why the stuff you're teaching matters is incredibly motivating for students. The best professors I've have had didn't just teach, they shared their love for the subject at hand.

One econ professor I had was even able to do this for a general econ 101 type class by prefacing the material with something along the lines of "I know this stuff is boring and in many cases wrong, but you have to learn it to critique it - if you're sitting here questioning this stuff take my ___ course" This empathy for us, the students, went a long way.</comment>


Startup buys porn ads to try to reach Sam Altman - <AUTHOR> http://www.wired.com/2015/08/pink-cow/ <comment><USER2> He claims they are looking for people who spend their money efficiently. Signs to me like this ad is getting more band for its buck than other ads. Signs efficient to me.

</comment><comment><USER3> I've read a bunch of articles about a "clever hack" the author used to get into an accelerator where they hyper-target a facebook ad to one or two influential people to convince them to accept their application. I imagine it's being done a lot now.

I wonder what percentage of the ads sama sees while browsing facebook linkedin or twitter are hyper-targeted specifically towards him.

<comment><USER4> I read that article, too. But it was done with LinkedIn targeting and it didn't receive the recipient directly, but he got a lot of messages from friends. So in the end, they got the contact. Don't recall the URL, though. :(

<comment><USER5> [https://news.ycombinator.com/item?id=9957083](https://news.ycombinator.com/item?id=9957083) ?

</comment></comment></comment><comment><USER6> And this "culture" keeps surprising us. The awkward part is that a lot of people find it "GENIUS!11!!111!!"

</comment><comment><USER5> This is what happens when startup culture turns "growth hacking" into a fetish. (pun intended)

</comment><comment><USER7> There are many problems with what they did, but I would not list using money inefficiently as one of them. Unless you're anti-advertising as a general concept. Weird.

<comment><USER8> Do you know what Be a Pink Cow even is? I've read the few linked articles about this, and watched their video, looked around on their Medium page, and they've obviously been mentioned in Wired now, and I have _no idea._

They appear to have spent money to prompt random porn viewers to send Sam Altman to a video begging him to pay more attention to them... on a site with no other information at all about who they are and what they do and why anyone should even care. They have all this traffic from Pornhub and Wired and TheNextWeb which is doing them no good, and the only fact anyone seems to know about them is that they seem like nice people, if inept at self promotion. That does seem inefficient at best.

</comment></comment><comment><USER9> Aside from being something of a personal violation, this behavior is also front-row evidence to a lack of creativity.

Do not pass go, and do not collect $12,000.

<comment><USER10> well we don't know what the startup idea is, perhaps it's related to pxxxhub ads. if not then i totally agree with you.

</comment></comment><comment><USER11> In the context of it being a porn ad it's funny how she dives down from the screen (multiple times).

Was it intentional?

</comment><comment><USER12> So the reader of the tech blog The Next Web go to pornhub without ublock or adblock?

<comment><USER13> I'm quite sure very many people do.</comment></comment>


How to Persuade More Effectively: 6 Principles to Help You Be More Persuasive Immediately - <AUTHOR> http://www.thenegotiationguru.com/how-to-persuade-more-effectively-6-principles-to-help-you-be-more-persuasive-immediately If you are in any business or interested in being more persuasive in your field, here are the 6 principles of persuasion that will help you be more persuasive immediately. It's powerful, effective and easy to read. <comment><USER2> Read "how to win friends and influence people" by Dale Carnegie. As a people guy I would say that this is the best book ever written on the subject. It's quite a short read, so really there is no excuse ;-)

Besides it's great fun to read his real life stories and examples, since the book is from 1936. No Meg Whitman or Steve Jobs, but lots of Dale Carnegie and Thomas Jefferson.

If you ar e too lazy or too broke to get the book the main points are listed here: [http://www.westegg.com/unmaintained/carnegie/win- friends.htm...](http://www.westegg.com/unmaintained/carnegie/win-friends.html)</comment>


The Miracle of the Internet Not Breaking - <AUTHOR> https://www.theatlantic.com/ideas/archive/2020/05/miracle-internet-not-breaking/611212/ <comment><USER2> It's _not_ a miracle.

That's exactly what the internet was _designed_ to do.

The Internet is a DARPA research project that was designed to keep on being useful even if huge chunks of it were destroyed in a nuclear attack. History has been revised to state this is just a 'rumor', but it is a fact that the Government gave funding via ARPA/DARPA to the inventors on this premise.</comment>


Understanding Cognitive Surplus and Television for Smoother Social Apps - <AUTHOR> http://www.zacharyburt.com/2010/07/understanding-cognitive-surplus-and-television-for-smoother-social-apps/ <comment><USER2> In summary:

People do nigh-everything socially, seemingly one-way interactions help define group activities (and are thus not just one-way), and doing stuff for free really _is_ its own reward, in a way, because people like feeling generous and it raises their view of themselves / their standing in society.

It... feels too loosely aimed, however. Like a set of rambling thoughts rather than a discussion with a purpose.

\---

edit: also, personally, I think the line spacing is too large. It feels like I'm reading _lines_ instead of _sentences_ which continue past the line. Probably because the side-bar has normal spacing, and the spacing between the <p> tags is roughly the same as the vertical gap between the content and the sidebar, which makes the content feel divided up more than it is.

<comment><AUTHOR> Hey, thanks for the feedback. I agree.

What do you think of the line spacing now?

<http://www.zacharyburt.com>

<comment><USER2> Much better :) After flipping to a couple other blogs, I realized yours is pretty much at the "standard" right now. Which could very well account for part of the reason why I like it better - you're used to what you're used to.

If you like it wider, a little poking around suggests to me that another couple pixels doesn't seem to make reading any harder, and then it still stands out a bit compared to other blogs. (what was it at before? I didn't check the # until just now)

Something I noticed later: I'd also recommend shrinking the Disqus container a bit, as the text there is rather noticeably larger than your posts' text (especially since their h3 is the first thing you see after your content).

</comment></comment></comment><comment><USER3> Love the observation from David Weinberger via Clay Shirky via this post: "Clarity is violence" -- meaning, clear rules can be coercive and over- constraining in social situations, and especially-dangerously-prematurely so in an evolving online community.

<comment><USER4> That's an amazing observation, and explains (or justifies) why, when I'm organizing social events, I'm reluctant to try to set up rules and such, until there is some notable problem that some clear, enforced rule might solve.

I'm a big fan of setting up situations and seeing where people want to go and helping to give it form. The trick is to choose some initial basic constraints while not being an impediment to some more interesting spontaneous behavior.</comment></comment>


Awesome JS+Canvas particles (WebKit) - <AUTHOR> http://typeslashcode.com/

<comment><AUTHOR> This seems to only work on Webkit powered browsers.. Too slow in FF?

</comment><comment><USER2> The website is still in the oven.</comment>


Why are we so bad at software engineering? - <AUTHOR> https://www.bitlog.com/2020/02/12/why-are-we-so-bad-at-software-engineering/ <comment><USER2> Those aren't the only reasons. Some more:

1\. Software is easy to fix. Compared to other construction disciplines, software is the cheapest to fix.

2\. The consequences of failure are typically minimal. Most software is not mission critical. Failure means minor inconvenience for a lot of people. Moreover, since your competitors are no more reliable than you are, you don't even stand to lose customers by having an occasional failure. (This is the one the article discusses.)

3\. Software has more degrees of freedom. Unlike physical building disciplines where the types of materials are limited and generally well known, and where the number of physical dimensions is at most three, software takes place in an extremely complex and multidimensional operation space. Moreover, we have no evolved intuition about how things behave in that space.

4\. Comparatively minor errors result in comparatively major failures. If you forget one rivet on a building, the engineering tolerances will make it such that the building will not fail as a result. But it is not really possible to make a server resilient to null pointer exceptions. It either works or it doesn't. Software fails partially much less often than physical things.

<comment><USER3> One of my favorite quotes from my old data structures professor in college was this: "software is non-linear."

If you're laying down the bricks for a house, there really aren't any cheap tricks or shortcuts. Building two houses takes twice as many bricks as building a single house, and so on. We could say that brick usage scales linearly with house construction. This is a bit of a bummer for bricklaying enthusiasts, but on the other hand, a single brick out of place is probably not going to bring your roof crashing down.

Software is different. Often, the distinction between processing 100 files and 1,000 files is minimal at best. This ability to scale effortlessly is often the best thing about software, but the same non-linearity is also the price we pay for power. A single subtle programming error can cause millions of dollars' worth of damage and bring massive systems to a grinding halt.

Overall, I think this is what makes software interesting and exciting to work with, but I can understand how it can pose a steep challenge once things like reliability and security become major concerns. The way I see it, much of research and development into better programming languages and tools has been to see how we can systematically mitigate these kinds of propagating errors and make our systems more robust to mistakes.

<comment><USER4> Yes but aren't we past this "software development is a production process" misunderstanding by now? We have solved the production problem: I can churn out thousands of identical binaries per day through my build system. The production side of things are solved. Other disciplines look at us enviously for it.

That's not where the troubles come in. The problems we face are generally those of design.

<comment><USER5> Are you sure? Pretty much every book and methodology over the past two decades has focused on how software engineering needs to "grow up" and be more like other production industries. A cornerstone in the latest fad, "new agile" with books like the Phoenix Project, is how the software delivery process should learn from lean production industries.

<comment><USER6> Yes, we've seen this before. It was called the "software factory" back in the 1980's and 1990's. It was snake oil then and it's snake oil now.

</comment></comment></comment></comment></comment><comment><USER7> Fashion. We're not bad at software engineering. We're bad at correctly labeling what is "software engineering" and what is not.

"Software engineer" today is just a job title. It makes us sound cool, but it has about as much to do with engineering as "Scrum master" has to do with rugby. In lots of places, you don't need to have an engineering degree or be a certified engineer to claim the title.

Teams that actually practice Software _Engineering_ , like the famous Lockheed shuttle group, are as good at it as we are at any other branch of engineering.

It's wildly misleading to use the same word for an app written by a few programmers in a couple of months (like the Iowa Caucus app) and the Space Shuttle control software (maintained by hundreds of people, with an average of 1 bug per release), and everything in between. It'd be like letting anyone call themselves Doctor.

<comment><USER8> > Teams that actually practice Software Engineering, like the famous Lockheed > shuttle group, are as good at it as we are at any other branch of > engineering.

I'm not sure that's true, or perhaps more accurately, I'm not sure that bar is high enough.

No other branch of engineering, at least none that come to mind, have to deal with the number of variables that any medium-sized software application does. Software, being infinitely malleable and running on a seemingly infinite variety of hardware in an infinite number of environments, is vastly harder than any other engineering.

<comment><USER9> >No other branch of engineering, at least none that come to mind, have to deal with the number of variables that any medium-sized software application does. Software, being infinitely malleable and running on a seemingly infinite variety of hardware in an infinite number of environments, is vastly harder than any other engineering.

I'm not going to argue if this is right or wrong as I don't have a Software Engineer's perspective. My background is in chemical and materials engineering in heavy industry (mining and manufacturing sectors).

I've worked in Operations and Process Engineering positions ([https://en.wikipedia.org/wiki/Process_engineering](https://en.wikipedia.org/wiki/Process_engineering)) and in more recent years I've been employed as a "Technology engineer" which is essentially integrating new technology and process improvements and optimizations into existing industrial applications.

The scope and complexity of what Engineers deal with daily might surprise you, software is complex so to is something like a smelter or refinery.

</comment></comment></comment><comment><USER10> You can't discuss software in this context without discussing IBM global services, oracle, computer associates, accenture etc etc.

Shut them all down, watch the average quality of software improve dramatically, immediately.

Next have someone qualified on every single board of directors and have each board create an appropriate sub committee. The same way you have a qualified accountant and audit sub-comittee. Have someone with proper CS & IT credentials and an IT oversight comittee. (Sure mostly the initial massive win is warding of the vampires mentioned in the previous paragraph but there is huge, huge value beyond that.)

Having ignorant people making the resource allocation decision is idiotic in the extreme and leads to overpriced, rubbish quality outcomes. Like deciding to use some garbage vote countinga app that doesn't and can't work - who did that? Can we just say they are utterly ignorant of the field they made important decisions concerning or do we need to get their name and demonstrate it.

Why are we so bad at software engineering? We are not! We just aren't. We do amazing things. We can do it reliably. We can do it economically. Software is f*&king awesome.

Why is there so much corruption in the decision making process leading to garbage quality overpriced, risky and idiotic software? Now that is a better question to ask.

Why is the idea of actually regulating foolhardy risk-taking startups (self- driving, privacy invading, turn-key facist state surveiling etc) so controversial? Because we can't even make good decisions about CRUD development at a policy level in a fortune 500 company - forget a policy decsion at a government level, you know it's going to be awful and redolent with regulatory capture.

We just need to grow up and stop blaming the geeks for the utter manure shoveled at us by ignorant jocks on golf courses determined to exclude anyone with actual understanding, insight and knowledge. And the manure shoveled by the actual geeks with a massive risk appetite and zero care for externalities beyond their startup making cash. Really. That's it. That's all.

Software is FINE. Decision making about software is SO bad, so awful, so hideous we try not to think about it lest it rots our minds with despair.

<comment><USER11> Interesting you think IBM global services, oracle, computer associates, accenture are making low quality software.

Which company is making good Quality software in your view?

<comment><USER10> Well at the risk of being flippant if you don't know what good quality software looks like, you probably work for, with or on behalf of these vampire-squid-consultancies. Or you have no business having an opinion about it or trying to form one. Really. It's as pointless as forming opinions about who is a good contemporary composer if you've been deaf since birth.

Look at all the successful startups disrupting industry by writing software. Note the total lack of vampire-squid consultancy in their codebase.

Look at the public service orders of magnitude cost blowouts and non- functionality that is the norm. Look at the domination of vampire-squid consultancies right there.

It's really not difficult to see unless you're determined to keep your eyes shut. The vampire-squid consultancies should not exist and are a symptom and proof of the prevalence of misallocation of resource in decisions made by the wholly ignorant.

<comment><USER12> 99% with you, but the vampire-squid consultancies came into being on the backs of market participants trading bank notes for software, and continue to exist on a steady supply of bucks for bugs.

So what did they do right, and what are they continuing to do right other than infecting large organizations with their blood sucking tentacles in a manner which closely resembles a Hokusai woodblock-print and removing some of the favored organs of the management end of the org chart that to be honest, they probably weren't using anyway?

<comment><USER13> They are just horrible to deal with.

I get rafted in as a vendor for part of a package that several of them offer to Fortune 500s. They provide negative value - it would be massively easier if I could just hook up directly with the end-customer's IT and do what needed to be done with competent people who know their job and what they want. But I have to play telephone through two or three layers of project managers in Delhi and Bangalore and an ever-shifting array of other people of indeterminate status and role.

</comment></comment></comment></comment></comment><comment><USER14> Contarian opinion:

We are great at software engineering, and our field is the best at engineering generally.

Engineering isn't about how reliable or high quality the system is.

Engineering is about balancing the costs and the benefits.

It can be great engineering to make a running shoe that wears out after 2 races, if it wins marathons.

People who think software engineering is bad, are actually bad engineers, who think engineering is all about quality or reliablity, rather than about optimizing tradeoffs. Software engineering is doing an amazing job at producing products that balance cost and benefit, which is why software is eating the world.

Engineers from traditional disciplines would struggle here, because it's a lot easier to follow super expensive tried and true process, than to situationally make tradeoffs like we do.

<comment><USER15> facebook did ~$70B in revenue with ~$24B in profit, which implies that it spent about ~$40B to keep itself running, for a year.

The Big Dig cost ~20B over the course of a decade. The Apollo Program clocks in at ~25B The Large Hadron Collider at a svelte 10B.

Software isn't cheap it's just that no one pays attention to the costs.

<comment><USER14> Facebook is an outlier monopoly, having successfully privatized a massive commons, so I'm not sure their ability to spend 40B a year proves much of anything. (If those figures are even the right lens.)

As a counterpoint, Wikipedia's operating costs are ~40M a year, 1000 times less, and I know which one I'd rather keep!

<comment><USER16> > having successfully privatized a massive commons

They created the commons.

It didn't exist before and several companies tried to do what they did. They all failed. MySpace had a huge head start. Didn't matter. Everyone shit on the Instagram acquisition for $1B when it was first reported and years afterwards (in fact Zuck made the decision without even consulting the board and right before a risky IPO where the public was sure to take it as a negative signal). He saw the value and invested in it. Today Instagram is a $100B entity that does more revenue than YouTube.

Credit where credit is due.

That's not to say there aren't issues. There obviously are and they're well documented.

</comment></comment></comment></comment><comment><USER17> I don't think that lack of incentive is the primary reason that software is so unreliable. It might be part of the reason, maybe even a significant part, but not more than that.

I suspect that software is just ridiculously hard. The complexity ceiling for programming is insanely high, and there is very little tolerance for oversights. If I'm designing a drill for instance, I don't have to consider every possible use the drill will be put to. I can just overbuild the crap out of it, and cover 95% of my bases.

Can you think of an analogous man made thing that is as inherently complex and fragile as a major distributed system?

And importantly it never really stops. As soon as a reliable solution to a problem has been discovered we immediately use it to build a billion complex new systems on top of it.

And testing is _hard_.

And programmers rarely get to truly specialize. We spend a great deal of time learning on the go because it would be nearly impossible to truly be an expert in a field where the tools and goals change so rapidly.

I'm sure I could come up with more if I kept thinking about it.

<comment><USER18> Author here!

This is a good point. Earlier drafts of my post did include "the unbelievable depth and complexity of the modern technical stack" as a factor. I deferred it until a future post because I think that we accept that we don't fully understand the systems and stack because of our incentives. Imagine if the stakes were much higher: if our software was hooked up to pacemakers that stopped pumping blood when the software was down. This isn't so crazy: people who design airliners go on the test flights. We would band together as an industry to produce guidelines and recommendations around safe ways to run sites. We'd quit rather than work on a doomed project for a paycheck. Any work would require something equivalent to expert certification and design validation at every layer of the stack. We'd do everything in our power to avoid creating these emergent systems. But the situation is reversed - in many circumstances, the cost of failure couldn't be lower. We tolerate the fact that we don't understand the emergent systems that we create because it doesn't matter enough for us to care.

But I don't want to make it sound like I'm dismissing you're point. You're right and it's super important. I think there's a better angle that I can use to explore this. It might be "Why do so many websites get hacked and lose our personal data?", but I feel like that doesn't fully capture it. I have to think about it. But this seems less relevant to the election case, which is what I wanted to address - that these projects fail because "launch and iterate" has become so ingrained in our process that it's difficult to recognize when it doesn't apply.

<comment><USER19> I think your focus on incentives is correct.

If I'm building a plane, I'm going to understand every last little piece going into it.

If I'm building a web app, I'm mostly ok if I do not understand every line of Rails, and the C code in the Ruby interpreter.

There _are_ embedded systems where people have that kind of understanding. And they do _fewer things_ than systems where people happily throw libraries in and whip out new features regularly.

If I built a web app with the same kind of line by line control as that embedded system, it would get outcompeted by someone playing more 'fast and loose'. Maybe mine would have a 99.999% uptime (or maybe not, because AWS or someone might have problems too) and theirs has 99.95, but way more features, so the market will gravitate to them.

</comment></comment></comment><comment><USER20> We're bad for several reasons:

\-- Outside FAANG type companies, most businesses are not willing to actually put in the investment it takes to do _engineer_ software. They want a "solution" as cheap and fast as they can get it. I'm not even passing judgment here, as in some cases that approach is entirely justified.

\-- Software is a world of eternal September, where the industry memory is only about five years long. Lots of us move on to other fields or roles and the memory is forgotten, and here comes another generation to reinvent the wheel all over again.

\-- Education: Computer science is not software engineering. Code schools are teaching programming more than software engineering. Thus, software engineering is something learned through doing rather than something studied as a discipline.

<comment><USER21> I'm curious, what evidence do we have that the FAANG type companies are significantly better at software engineering than anyone else?

<comment><USER20> When we look at the entirety of companies around the world, it's reasonable to think that there are non-FAANG companies that could be just as skilled. That stated, my own casual observations:

1) The FAANGs that I've seen have significantly more robust practices than the non-FAANGs. Everything from office conditions to build / test / release systems. The reason for this is:

2) Incentives. When your business is completely focused around software technology, you are incentivized to build out systems and processes that other places would not, because it is core to your business.

Anecdata from many years in and out of various kinds of companies, as both an employee and/or road-warrior consultant.

<comment><USER22> I still see bugs and rollbacks in FAANG releases. I conclude that the incentive isnt enough. The practices are inadequate.

<comment><USER23> A process with occasional bugs and rollbacks is rational if it produces more net utility than a process which only ships "perfection".

What's your error budget? Not spending your error budget is velocity left on the table.

<comment><USER24> That's... like the whole point of the article. That mentality causes less reliable software.

</comment></comment></comment></comment></comment></comment><comment><USER25> One of the more profound insights from Stephen Wolfram's "A New Kind of Science" is that small rules can lead to extraordinary complexity.

Wolfram famously gives Rule 30 as an example, where you cannot know what each line of squares will look like before you do the computation, but he goes on to cover many other phenomena, like prime numbers and tag systems.[1]

One conclusion is that complexity stemming from simple rules is relatively common in our Universe and that historically, humans have done a good job of solving the narrow pool of problems where you can know the end solution with a simple calculation (say predicting the location of a ball as you toss it up in the air).

The implication is that there's a whole "universe" of problems where we cannot know the end solution until we've done the work necessary to compute it.

And in that sense, it wouldn't be surprising if software engineering was a bit like Rule 30: an endless array of patterns we can't predict until we've actually created the code or computed what it can do.

It's not so much that we're bad at software engineering. It's just that emergent complexity is part of our reality (assuming Stephen Wolfram's insights apply to software engineering. Maybe they don't. But it sure feels like it).

I think the ancients knew this well. The Tower of Babel story is pretty old. Details the problems humans run into when they try to scale and work with many different elements: Something nobody could predict and possibly, catastrophe.

[1][http://mathworld.wolfram.com/TagSystem.html](http://mathworld.wolfram.com/TagSystem.html)

<comment><USER26> Would Conway's Game of Life be a good example of small rules that lead to complexity?

<comment><USER27> Both Game of Life and Rule 30 are cellular automata so yes!

</comment></comment></comment><comment><USER28> Software Engineering is new. The romans were working on solving civil and mechanical engineering problems hundreds of years ago and we've had a lot of time to study their failures.

Unfortunately, our industry is dominated by an obsession over "use the latest, cuz is the greatest" mindset, where if you're not using the hottest new language or sexy framework, you're already behind somehow.

I can imagine maybe it was this way when the romans were trying to figure out how to build roads. "Oh hey, did you hear how Steve built his road? Yeah, they compacted gravel an inch out of time underneath the bed. Something about water drainage. Oh and they're waaaaaay more productive. Yeah, productive. Lets do use that framework for building our roads!"

<comment><USER29> > Software Engineering is new.

I believe one of the examples for extreme reliability used in the XKCD cartoon included in the article is air travel, which is about as old as computers.

I was told once that the reason air travel is so reliable is because they have mechanical backup systems for when the electronics inevitably fail. Is that true?

How reliable are electronics as a whole?

<comment><USER30> No, it's not true. Air travel is as reliable as it is because it's considered to be a _system_ Airplanes, their design, manufacture and maintenance. Airports, communication systems, procedures for entering, transiting and exiting an airspace, fault reporting, accident investigation and much more. _All_ of those things are part of the "system."

What a lot of people fail to do when making comparisons with software is to look at the act of coding in isolation from everything that surrounds it. That leads to a very skewed perspective.

</comment></comment></comment><comment><USER31> Time.

Airplanes reuse and refine, reuse and refine the same system, more so with elevators. Software is often greenfield.

Realistically, software projects get budgeted for half/third of the time they actually need to be anywhere near complete. Then following the "delays" that should have originally been budgeted in, you might be over or under that more realistic estimation, but the software is worse for it. Why? Because the engineers were stressed/crunched/whatever into performing worse under the false pretense that "we're almost there," when that is not the case.

If companies put a greater emphasis on correctness and budgeted time/money/testers/etc for it, things would be better. This low grade output is what the industry wants and that is what it is getting. People in charge of scheduling may say they care about correctness, they never really do, they care about fast and the appearance of some facsimile of function ("it looks like it works").

Imagine if you went to your higher up and said that the 6 month greenfield voting app, would actually take 1.5 years and require more resources just to ensure correctness. You would just get taken off the project. "It's just a voting app, one screen and server." But that misses the point, because it's a voting app, it needs to be hardened, just like an airplane.

So you learn to shut up and either produce worse output, or work overtime to make something halfway decent. Either way it basically goes unnoticed until something blows up.

<comment><USER32> It doesn't help that developers pretend this crap isn't going on either. Verbalize and acknowledge it but everyone's so worried that unrealistic requests and budgets are somehow _actually realistic_ and they're just underperforming compared to their peers. So many actually suffering from imposter syndrome and worried that they're suffering from the Dunning-Krueger effect.

Every group I've worked with seems to start in a defensive state projecting confidence but I casually admit to my struggles with everyone on the team and folks start to open up. By the end, you have a team of developers who can help set more realistic expectations. The less developers communicate amongst themselves honestly, the more they can be exploited by unrealistic expectations for development.

It's a much nicer environment when everyone knows and admits they're slinging garbage code because of resource constraints than everyone pretending they're not.

</comment></comment><comment><USER33> 1\. The economic incentives are aligned to ensure most software isn't very good because it doesn't "need" to be.

2\. The tools that help make software really, really good are really, really baroque and remain that way because the people who invent them are subtly incentivized to be the high-priest of an esoteric technology church rather than making them for the masses. Which ensures that making really, really good software doesn't get cheaper with time and also ensures that you'll only reach for these tools if _you absolutely have to_ because of #1.

3\. Software Programmers/Engineers see their jobs, and appear to derive most of their joy, from doing the actual coding. Not in using rigorous design tools ahead of time which are unmoored from their implementation, nor in doing the testing & verification of the implementation after the fact.

3a. Software Designers/Architects have made a bad name for themselves by spending more time pontificating at whiteboards than writing down verifiable specs & models.

3b. Software Testers/QA folks are treated as lower-class than Programmers/Engineers, and don't get paid nearly well enough, so they're setup to follow rote scripts and procedures rather than empowered to deploy sophisticated testing methods like property-based testing or mutation testing.

4\. All those factors appear to have created a feedback loop which has become self-reinforcing.

</comment><comment><USER34> The Iowa caucus app didn't fail because "we" are bad at software engineering. It failed because someone did a shitty job of understanding the problem space, deciding on an appropriate solution, and finding proper resources to make sure that solution was well-implemented.

<comment><USER35> I think properly implementing a solution appropriate to the problem space is the literal definition of engineering. So "we" aren't bad at software engineering but someone definitely is.

<comment><USER34> I'm saying the fundamental problem was _not_ the software implementation but the non-engineering decision-making surrounding it.

<comment><USER30> What the poster is saying is that the project management aspects _are_ a part of engineering a product. It's an opinion I happen to agree with.

</comment></comment></comment></comment><comment><USER36> Missing here and in the comments so far: because we categorically refuse to learn lessons from the past and to improve on what is already there, instead we keep on re-inventing the wheel in interesting and novel ways which then come with their own sets of problems which usually only show up after a couple of years. By then of course it is too late.

This has been going on as long as I've been active with computers. I think a part of the driving force behind this was the speed with which computers became faster. There simply wasn't time enough to absorb the lessons before the 'old' stuff looked small and obsolete in view of our new powers.

But if you look at old and mature software systems (Common LISP, Erlang, D and others of that vintage) you'll find that the degree of thought that went into them and making them bullet-proof is still very much absent of that which drives a large portion of the software industry today. We're probably destined to go through this sort of cycle at least a couple of more times, if not forever.

</comment><comment><USER37> I appreciate posts like this one for the commentary they generate, but it's still disheartening to see the obvious (to me, anyway) stuff getting missed each time.

The overall post was okay, but the lead-in was too topical to be relevant more than a few months from now, and also wasn't really about software quality at all. What happened in Iowa was a lot of bureaucratic cock-up -- they got the wrong people (because favors or political or personal connections) to do entirely the wrong thing, and didn't bother to consult any of the people who could have helped them make better decisions. It's not substantially different from someone buying the shittiest available used car from a private party and the wheels falling off on the highway, and then trying to use that as an indictment of automotive build quality.

But.

Clearly software suffers from far more issues than other technical disciplines, and that is 100% because there are no common standards and, thus, no certification process or professional licensing.

When software developers and managers compare big software projects to big civil engineering projects, they keep missing the obvious educational and certification process required to become a civil engineer. Galloping Gerties and John Hancock Towers still happen occasionally, but then they become case studies for every prospective new engineer. Contrast this to software, where we all slang out code until we're bored and then self-educate by reading other people's war stories online.

The average man or woman that can build a porch for your house is subject to more licensing requirements than the developer that builds voting software. Let _that_ sink in for a minute.

Programmers are quick to screech at the thought of being subjected to any kind of licensing or certification process, and yet we all have the horror stories of being forced to build and deploy utter garbage by upper management. Professional licensing would really help us here. We'd have a defense that would go, "I can't do that, it would make my license get reviewed."

I don't think this will ever happen though. The majority of programmers don't want it and nobody knows how to do it. They'd rather write and share navel- gazing articles from time to time, wondering about what could possibly be so very different in software from every other engineering discipline. 'Tis a mystery.

<comment><USER38> The problem is thinking that there could be such a thing as a license for software engineers _in general_. That would be crazy, almost like having to be licensed to write English. Software is applied to too many different domains and tasks.

What _could_ work would be licensing for software engineers in certain specific domains. For example, we could require election systems software engineers to be certified for that field. Same with automotive software, power plant software, etc. etc. I agree that we should do this. But it's going to apply to only a small fraction of all software engineers, because licensing Web app developers is not practical.

<comment><USER39> The Soviet Union licensed photocopiers.

<comment><USER40> True, but not a model we should emulate.

<comment><USER39> Certainly not. I meant to argue that oppression is a real risk, not a desirable option.

<comment><USER40> Ah, I see. I agree.

</comment></comment></comment></comment></comment></comment><comment><USER41> Our society has produced software teams that produce highly reliable software (space shuttle, banking technology, and embedded systems) that don't leave room for a great deal of speed it innovation. Let's call them Type 1 teams.

We've also produced "move fast and break stuff" software organizations, who have realized moving fast and failing is a better financial outcome than safely arriving at a suboptimal destination (Facebook, Twitter, tons of startups). Let's call them Type 2 teams.

Neither Type 1 nor Type 2 teams are Bad or Evil. They are simply suited to different applications. What happened here is, an organization that was unable to distinguish between these types of teams hired a Type 2 team for a Type 1 job.

Generalized example, usual caveats apply.

<comment><USER42> The majority of software teams fall outside of this classification.

<comment><USER41> Agreed. It's really more of a gradient. Regardless, the problem is not where a team lies on the gradient, but where a team lies in relation to the fundamental needs of the problem to be solved.

</comment></comment></comment><comment><USER43> The Boeing Company invested nearly $10 billion in developing the new 777

The Voting software cost $60K I believe, that includes distribution.

That about sums it up for me, if you want reliable software it costs a lot of money.

<comment><USER44> Money, time and planning. Agile is, for the most part, the absence of planning. "Figuring it out quickly and changing it if we need to." But, by design, most "physical" engineering tasks require tremendous planning, and include (from what I understand) very little "Just figure it out as we go."

The reason for a lack of planning, is because tremendous planning increases time to ship, which effectively costs money.

<comment><USER45> Agile is not the absence of planning. It's about adapting to change. When it comes to business software this makes sense because the end result is typically unknown. Aircraft avionics have a pretty well-defined end goal. The other big difference is really quality control. Most software doesn't invest heavily in quality because a failure isn't that costly. It's the same calculation that goes into producing absolutely everything. The cost of a quality failure in an airplane is humongous. I use apps that crash at least once a day, but I just restart them and keep going.

<comment><USER44> Yeah I guess I just very much do not agree with your stance on Agile, but that's okay. YMMV.

</comment></comment></comment></comment><comment><USER46> We aren't bad at software engineering. The Author does a good job of analyzing the situation. It highlights mostly what we care about is delivering a stream of value knowing that we can correct things when they go wrong. We care that it is mostly right. We 100% embrace the "soft" of software

Thing we struggle more with is "Must work correctly on first _REAL_ use". So we now have the PITA world of other engineering disciplines where adapting is often not an option once "deployed", though even there those disciplines look for ways to get "soft".

Having spent a lot of my career doing Embedded Systems/Industrial Control Systems the approach to getting things correct is very different from developing web/backend/mobile software. Lots of rules are introduced, design approaches are limited to certain ways of doing things and many techniques are avoided. Functionality is carefully considered on how difficult it is to build correctly. Testing is more convoluted. Robustness and Performance are far more pressing concerns. But again, in this environment it is often desired to be more "soft" and various things are done to try and achieve that. It can be handled much like other engineering and often is managed as part of a multi disciplinary effort of mechanical/electronic/software engineering and treated as a "system"

Getting some mobile app developers to make something that is supposed to be "robust" on first deployment is mainly a problem in not recognizing what skills and experience are needed to be employed to get what you want.

<comment><USER47> > We aren't bad at software engineering.

I have never seen perfect software, have you? They always end up with bugs that make you vulnerable...

<comment><USER46> I'm not sure I've ever seen anything engineered that is perfect, have you? what even is "perfect"? seems like an unrealistic standard and not the goal at all of any engineered system (civil, mechanical, electronic), I don't think that a good definition of "bad" is everything less than "perfect". I have seen software the fulfills the necessary criteria, especially in the world of control of machines, eg, engine control, industrial control systems, etc. Many things get built that work within their "engineering" criteria. I have one particular software application on a microcontroller that performs various functions and control, I've only ever deployed 1 version on thousands and thousands of devices, it has worked for a decade so far. It went through rigorous review and testing before being released.

<comment><USER47> > [...] it has worked for a decade so far

if you don't change the inputs or any other variables, it is different from a program deployed in the wild... but yeah not very many things are perfect (even the sun will die one day).

<comment><USER46> It is deployed in the wild. But what you are describing is essentially engineering, being in control of your variables is key. Engineering is all about understanding what the variables are, what tolerances you need and building to those criteria. Pretty much anything engineered is on wobbly ground when it gets inputs/conditions unanticipated. Part of that is being clever about when things are in uncontrolled environments to find a way to sandbox things into a controlled environment. Much like the software world, a simple thing is put it in a "container"/box/case and limit the "input" to the underlying system.

</comment></comment></comment></comment></comment><comment><USER48> My take on this problem is that other (engineering) fields operate with much, much longer time frames - not to mention that they also have vastly more experience and best practices to rest on.

Software is young, things move very fast, and new practices can emerge quickly (and older get canned).

Imagine if aerospace company was to design, test, and build a new airplane in only 6 months. It would probably end up as a disaster. In the realm of software, it is very likely that you need to come up with something new, very fast - you don't always get the luxury to step back and lay the proper groundwork - you're doing it on the go.

<comment><USER49> Our tooling's pretty shitty, generally, isn't well-standardized or anywhere close to it across the industry, is changing rapidly (largely not to any real purpose, though sure, sometimes improvements slip in) and we're all expected to become semi-competent (nowhere near enough time or brain space for full competence) on an absurdly large set of these tools—not just using them, but using several different ones for similar purposes but with different interfaces and quirks (boy do they have quirks), how to set them up, how to fix them when they break, and so on.

Oh and you can make a name (and pile of money) for yourself if you manage to promote some mediocre beta-quality-at-best tool and trick enough other people into using it, and if people call you out on it _they 're_ the assholes.

Every thought experiment that immediately comes to mind, in which other professions of various sorts had to deal with some similar situation on an ongoing and apparently never-ending basis, read as absurdist comedy. Yet here we are.

[EDIT] ok here's a fun one: imagine if framers had to use a different hammer depending on the brand of wood at the job site. Like, hammer weight is different and you even hold it differently. For some you can use air nailers, others you can't, some of them only work with air nailers that have a second person working the trigger while one holds them, all the different air nailers need different compressors, some of those take different voltage, shit like that. Imagine it's like that for _all their tools_ and that these differences manifest in all sorts of ways based on the _combinations_ of materials & tools.

We wouldn't say "gee why are house-builders so bad at building houses, and LOL they can never even tell us about how long it'll take", we'd say "holy shit how do _any_ houses get built, it's practically a miracle, we should fix it so it's not so hard—for no good reason—to build houses"

</comment></comment><comment><USER50> A former Professor of mine published a paper on a very related topic. He interviewed 54 software developer at the International Conference on Software Engineering. I found it a very nice read.

[https://arxiv.org/pdf/1911.09971.pdf](https://arxiv.org/pdf/1911.09971.pdf)

<comment><USER51> Indeed, a fascinating study, thank you for the link. Breaking down the summary:

Conclusion: More software engineering research should concern itself with:

\- emergence phenomena

\- how engineering trade offs are made

\- the assumptions underlying research works

\- creating certain taxonomies.

Such work would also allow software engineering to become more evidence-based.

</comment></comment><comment><USER52> There are a few issues... First, the people writing the checks aren't concerned about the material quality or the craft discipline output. It's also not so different from manufacturing... you can have a hand crafted oak cabinet with dovetail joins and meticulous material selection, or you can have a flat pack cabinet from Ikea. Both have value, and quality is considered a measure of value that meets or exceeds expectations.

Building an aircraft has many layers of bureaucracy behind it, and the cost that goes with it. In the end, software is practiced as a craft more than an engineering discipline. And that's okay. It's usually when politics and management decisions override the needs of the software/product that things tend to go sideways.

There are also more than a handful of software developers that don't really care about the quality or maintainability of the software. Let alone learning and using the methods for interfacing with said software.

Just the past couple days, I was looking for a React + Material UI template to work from. Many looked great, but actually following the code was a byzantine mess. Mirrored trees of files are my biggest pet peeve in front end projects. Let alone not even using the methods available for use and extension. That's just a recent example.

I've seen others, where a given set of methods is copy/pasted across dozens of case statements... Worse still is when you see a schema design that makes no sense and/or variables that are lost and meaningless, that everything around it loses all context. Such as the passwords having the variable w23 and the value is stored in the "Phone2" table under "extra1" columns.

Managers give lip service to "quality" and "testing" but when push comes to shove you skip testing to ship on time, and have to deal with bugs several years into an initial 9-month project timeline.

</comment><comment><USER53> Slightly off-topic, but here's what I would've done: just build a website.

No real client-side code; lean on the HTML platform for everything it can be used for. HTML is the simplest, most stable, and most battle-tested client- side framework ever created, and it's stateless and runs on all devices without even an installation step. Toss some CSS on there to make it look pretty; CSS is stateless too so it won't introduce any real bugs. It doesn't have to look perfect; you're not marketing it to anyone. You're just collecting input from users who don't have a choice of product.

For the back-end use something dead-simple like PHP or NodeJS. The business logic can't be that complicated; I assume it's really a very simple CRUD app. Just accept form submissions, authenticate, validate, drop it in the database. Done.

Spin up a couple of hand-managed VMs on DigitalOcean or the like for hosting both the databases (probably just one database?) and the web servers. It's not a small user load, but it's much smaller than the typical internet company, and it only needs to run for a few hours. You don't need automated scaling and restarting while you sleep. Sit a task force at their desks during the event to repair any outages as they happen.

Use the right tool for the job. Your stack should be exactly as complicated as the task demands, and no more.

</comment><comment><USER54> [https://www.microsoft.com/en-us/research/publication/the- inf...](https://www.microsoft.com/en-us/research/publication/the-influence-of- organizational-structure-on-software-quality-an-empirical-case-study/)

The biggest predictor of bugs is "Organizational Structure", the main problems sound like they're not in the computer(s)

</comment><comment><USER55> Are we? I mean, we're constantly driving every software organization to take on more and more complexity in their projects all the way to the point that they reach their cognitive limits.

Yes, a lot of the progress we've made is just an illusion driven by Moore's law and lots of bloatware that it enabled, but a lot of it is genuine and needs to be taken into account.

</comment><comment><USER56> I'm confused why this is even a puzzle.

There is a bubble. Companies with half a product, no distinguishing technology, and no path to profitability are getting valuations of billions.

Therefore the big guys - the FANGS - become worth trillions, and can afford to hire the best of the best and pay them millions (over the last 5 years, with stock, yeah, that's what sr engineers make in Seattle/Silicon valley)

So, you've got the best software engineers in the world making adtech.

Which means the only ones the Iowa Democrats can find for a 60k budget aren't inept, but they're certainly not in the top tier. See also: The Obamacare online rollout.

What will fix it?

A market correction. One, as a well paid sr engineer, I am not looking forward to. But it will make a bunch of actually competent software devs available for smaller market projects, and suddenly the quality of government software will increase dramatically as lower-paid-but-outstanding-benefits government jobs seem way more cushier all of a sudden.

<comment><USER57> All this shit applies at the FANGS too so I don't think any market correction will magically solve the problem.

</comment></comment><comment><USER58> Why are we so bad at software engineering? Because most software comes without any liability.

<comment><USER59> I completely agree. At the some point, businesses need to to have reason to say "no, what you want is feasible, and we won't risk the consequences of failure."

For safety critical applications, technical decision makers need to have reason to say "no, we can't do this, and we won't because my career and possibly freedom are at stake if we do."

Right now a huge portion of the risk of software failure to software builders is reputational. This can be sufficient, as pointed out in many cases in the article, but as we see in these examples is often not.

</comment></comment><comment><USER60> I use to explain it this way: nowadays, bits and bytes are everywhere, most everybody you know has a notion of what a `megabyte` is. But this is a shockingly recent idea.

My grandma was 20 when a guy named Claude Shannon in 1949 invented/popularized the concept of a "bit" and described what information is, the way Newton described matter and how to model physics with mathematics.

Really, "information" was a just vague concept until Shannon and this was just 70 years ago!!1

It's only natural that a lack of generations of craftsmanship in this industry makes it pretty low quality/hard to master. On the other hand, imagination is the limiting reagent, since (arguably) the bottleneck is good ideas, specially with all these decades of Moore's law.

</comment><comment><USER61> Because it's not actually taught anywhere. There are no widely adopted standards. Web development is literally a patchwork of random libraries and npm modules stitched together.

Computer science programs don't teach students how to build good software, they teach computer science. CS professors wouldn't even be qualified to teach good software engineering practices if they had to.

Code is also difficult. It's one thing writing your own code and understanding it, but understanding someone else's code is very difficult since you have no idea what was going on in their head when they wrote it. Of course there are practices you can follow to make things simpler, but it's never going to be as easy as say reading plain English.

<comment><USER62> > _Because it 's not actually taught anywhere_

Oh, I don't know. I have a bachelor of Software Engineering that is accredited by the Australian Institute of Engineers.

The vast majority of my degree was on the discipline of SE, so rigorous testing standards, theory of large scale system design, hard real time systems, Improving your personal software process.

For our final year project 16 of us worked for a very large company to design and build them a piece of software. It was awesome.

</comment></comment><comment><USER19> > Many common practices in software engineering come from environments where > failures can be retried and new features are lucrative. And failure truly is > cheap. If any online service provided by the top 10 public companies by > market capitalization were completely offline for two hours, it would be > forgotten within a week. This premise is driven home in mantras like "Move > fast and break things" and "launch and iterate."

Good article. We're not really that _bad_ at it, we just optimize differently than building airplanes or elevators, for most things. No one cares if the site I built for the local bookseller goes down at 3 am for an hour or two. Or even for a few minutes during business hours.

</comment><comment><USER63> Except airplanes and elevators need software and therefore software engineers so this should really be about web and app development.

</comment><comment><USER64> There is one major difference between the disciplines. Engineers are legally responsible for their creation and sign with their names. Software 'engineers' do not do that, if something goes wrong nobody feels responsible and usually nobody has signed some document that testifies legal responsibility.

</comment><comment><USER65> I think there's one true fundamental reason: software is a lot more difficult than we assume. We underestimate the degree to which even the most mundane program is the task of creating an entire universe and ecosystem from scraps. Even the most helpful programming languages and libraries don't get you as much as you think.

Just look at the world software engineers play in. In a world of 7 billion people, nearly all of us use one of a tiny number of compilers on computers running one of just three or four kernels, atop one of two processor instruction sets.

Yet look at the plethora of programming languages. The fundamental building blocks of our industry are apparently so terrible that many of our brethren devote years trying to replace them.

</comment><comment><USER66> This discussion reminds me of a talk given by Glenn Vanderburg ([https://www.youtube.com/watch?v=RhdlBHHimeM](https://www.youtube.com/watch?v=RhdlBHHimeM)), wherein he explores the question of whether "software engineering" is or can really be engineering. Around the 27 minute mark, he quotes Billy Vaughn Koen, a nuclear engineer, as follows: "The Engineering Method is the use of heuristics to cause the best change in a poorly understood or uncertain situation within the available resources."

That's hardly the end of the debate, and I'm still undecided, but it seemed worth lobbing in before hitting the rack.

</comment><comment><USER67> Obligatory post : [http://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036...](http://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036.html)

(Dijkstra on software engineering)

</comment><comment><USER68> The app used for the caucus was built for $60k. That's enough to pay what, 3 developers for 2 months? Not much time for testing. I don't know how complex the app is, but that's doing things on the cheap and they paid the price in the end.

</comment><comment><USER69> That cartoon points out that airplane engineers create good, reliable things, and software engineers don't. But airplanes are reliable in part _because_ of the software that they run. So to me the question boils down to "How come we create good software in one case and terrible software in another?". Well, if you give a programmer incorrect/incomplete requirements, not enough time, don't do enough testing, and generally cut corners, the software is probably not going to work very well. And while it's true that some programmers are better than others, poor quality work should never make it past the testing phase.

</comment><comment><USER70> Software as a wide spread phenomenon has been around since what late 1970s (I am talking about business and individuals using it). Compare it to building industry, accounting, law and any of the hard sciences. They have been around for __thousands __of years. It is kind of crazy that we insist on perfection withe less than 50 years of practice. Heck we can 't even decide on one language yet - and the one that are standard (I am talking about you JavaScript) - people keep re-inventing them (Coffee Script, TypeScript, etc.)!!!

</comment><comment><USER71> One thing I didn't see mentioned about creating software is that the tools and the systems often have serious problems to begin with, even before you start writing a program.

Some well known examples are:

\- major programming languages such as C don't do enough to help with memory safety issues

\- lack of explicit types in python function arguments can lead to surprising bugs which recur again and again

\- operating systems and platforms, such as different web browsers, adhere to widely different conventions about how dependencies on other libraries and code is managed

<comment><USER22> > the tools and the systems often have serious problems

And running a train over a floating bridge has serious problems, but it is done. These software "problems" are trivial compared to algorithmic complexity when analyzing 10billion events a day in countless ways realtime.

</comment></comment><comment><USER72> > Why are we so bad at software engineering?

"Why are we so bad at _apps and websites_ software engineering?" might be a more appropriated title.

My hard drives never stopped spinning. I am sure there is some amount of software engineering on them. Last night I finally finished The Witcher 3. It never hanged up and was a joy to see and play.

Please, when referring to software engineering, stop assuming is _just_ shitty phone apps and rushed webapps. That's just a small tiny fraction.

</comment><comment><USER73> We don't have standards in software engineering. We don't reuse components effectively. It's difficult disassemble software and reengineer components.

</comment><comment><USER74> Several more humorous articles on the subject:

1\. [https://www.stilldrinking.org/programming- sucks](https://www.stilldrinking.org/programming-sucks) 2\. [https://mickens.seas.harvard.edu/wisdom-james- mickens](https://mickens.seas.harvard.edu/wisdom-james-mickens) (I love The Slow Winter, but To Wash it All Away is probably more relevant here)

</comment><comment><USER75> Typically Time and Consequences of Failure:

1) People typically die when things go wrong and are heavily regulated. Most software that works in this space are tested in accordance to those industry standards. Your desktop browser doesn't need to be engineered to those standards.

2) Other engineering disciplines have in certain cases had 100s if not 1000s of years progress. We've had less than 60 years.

</comment><comment><USER76> Airplanes and elevators both use software. It is obvious that software engineering can be as reliable as any other engineering discipline. World is full of imperfect physical pieces of engineering from roads to night cabinets, therefore it really is a silly position to take, that software is somehow worse than anything else.

</comment><comment><USER77> Food for thought: Should we establish a system that gives people license to code? Doctors need to get a license too because their practice involves human lives at stake.

I don't see how software doesn't put human lives at stake. Honestly, the number of lives at stake is increasing with each new self-driving car hitting the road.

<comment><USER78> I'm continually shocked that there's still no "bar exam" for programmers. It wouldn't even need to be something that you need in order to write a program, just some actual meaningful certification that hiring authorities could put some credibility behind.

</comment></comment><comment><USER79> I'd say the complexity has increased, the requirements are not well defined and the schedules are tight.

</comment><comment><USER80> The real world has a base set of stable frameworks on which everyone can rely, such as physics. Software must implement not only a thing but the entire universe in which that thing operates.

Everyone claiming tooling or funding or this or that is missing the forest for the detail of trees in the client spec.

</comment><comment><USER81> We're not. We're great at it.

</comment><comment><USER82> Because we dont know how to manage software projects at all. The technology is not a problem, we dont need new credentials for CS&IT people. Organization and management is and we need better education about that for everybody - especially for CS&IT people.

</comment><comment><USER83> If I rephrase the title to "Why are the results of software engineering so bad?" it makes it easier for me to point out the most obvious problems, in my experience at least.

The alternative to nearly all software solutions is "bunch of people do that thing" or "hire costly expert." Software is usually a combination of transitory, relatively quick to implement, mostly opaque, and very inexpensive in up-front costs in comparison to the alternative, which both reduces the perceived risks as well as makes it unusually attractive as a means to achieve some goal. That combination also lowers the bar for what qualifies as "a goal," providing a much larger universe of possible actions available for those who are establishing the goals.

With the lowered bar of what constitutes a goal, given that the overall costs are perceived to be so very low, the decision maker feels unburdened enough to choose goals without vetting the decision and its consequences against the standards that would be applied to "hire a bunch of people to do that." You've taken what used to be a decision that carried with it all the forethought and examination that one would usually apply to purchasing an automobile or home and replaced it with a decision that has no more consequential cost than to decide whether one wants fries or carrot sticks with a sandwich.

As with the choice of fries, it's not the single instance of such a choice that lays you out on a cardiac surgeon's table but instead the repeated accumulation of what seemed to be inconsequential decisions to get fries. If you asked someone if eating fries with every meal could set someone up for a heart attack they'd most likely say "of course, that's obvious." But our patient did not encounter the situation in which they had to make the decision as to what side they will have with every forthcoming meal, and all the forethought answering such a question would entail. They only made one little decision whose importance was as fleeting as a single meal. Until it wasn't.

This effect on decision-making, with regards to software, almost always spreads to the entire organization (and often beyond, to infect customers and the general public as well). The CEO, the VPs, the Directors, the Managers, not to mention the programmers themselves, all make decisions and set goals that meet the lowered standard (or at least a standard lower than what should or used to be the standard).

When you lower the standards for decisions and goals, you get results commensurate with those standards.

</comment><comment><USER84> 1) We hardly ever try (most projects aren't given adequate time to model and prototype)

2) There's not enough specialisation (most software shops will tackle any project and IT people tend to specialise in technologies, not business areas).

</comment><comment><USER85> Ask any locksmith about any door security.

"I am not putting my breakfast behind a locked door, he will say. I always carry it with my backpack and my backpack is always on my back."

</comment><comment><USER86> Because there are a lot of bad actors with plenty of time on their hands and resources to backup their endeavors.

</comment><comment><USER87> Because this is a baby industry/technology, compared to other engineering practices?

<comment><USER88> But we have established engineering industries best practices to learn from, but it seems like we aren't.

<comment><USER87> Like, building bridges? (even there, we still struggle with, not technological issues, but budgeting/governance/contracting/management).

How many generations of people have passed since computers are around?

Not enough that we can pragmatically reflect upon it.

See, we can say we collectively have some understanding of what the invention of the printing press, and its consequences on societies, cultures, and history. As well as, before it, writing.

I think we still can't say much the same about computers. We barely see (and not everyone yet) how much it impacted and enabled to impact our societies, public & private discourse, political endeavours and regimes.

</comment></comment></comment><comment><USER22> Obligatory (and still relevant): [https://vimeo.com/9270320](https://vimeo.com/9270320)

<comment><USER29> I have his book and refer to it often.

</comment></comment><comment><USER89> Because software engineering is recursive and has an unlimited stack.

</comment><comment><USER90> Now ask the airplane engineers about the F-35.

Voting apps are basically political money laundering schemes. Using them to make comments about software vs other industries suggests either stupidity, laziness, or malice.

</comment><comment><USER91> "Software engineers" are not. Engineers. "Software engineering" is not. Engineering. Delusions of grandeur is the problem.

</comment><comment><USER92> Because people can be irrational and dogmatic, and follow incoherent people for no reason.

</comment><comment><USER93> Not about the article specifically, but as far as the XKCD comic goes, I'm not sure it necessarily tells people what they think it does. The electronic voting system concept is arguably the only concept software engineers would universally say is a bad idea/worry about.

And that's more because of the practicalities involved in that particular idea rather than something about software enginering as a whole. What's necessary for a secure and trustworthy voting system puts it at odds with any attempt to provide such via a machine/app/program/website.

Ask the same people about say, reliable hashing/encryption libraries or what not, and their attitudes would probably mirror those of the aircraft designer or building engineer.

There are probably also examples of things in aircraft design or construction that would have people in those fields freak out too.

<comment><USER94> Why would that be? There are far more dangerous software jobs, and also examples of successful digital systems like in Brazil.

</comment></comment><comment><USER95> Obligatory xkcd:

[https://xkcd.com/2030/](https://xkcd.com/2030/)

<comment><USER95> Wow!

</comment></comment><comment><USER96> This article doesn't make any sense. It already fails at the premise. Airplanes and elevators have nothing in common with code written by normal software companies.

They optimize for a compromise between short-term gain and long-term gain. In essence, a startup can mathematically proof its software and go bankrupt before they even reach a prototype stage. Or they can shell out utter crap quickly, get some early feedback, funding and beta testers and slowly figure out what to improve first for the biggest bang.

Same goes for giants, like Google. They technically have the resources to prove SOME projects and they DO. For instance, Amazon proves their hardware virtualization software and other core pieces, where bugs would undermine the safety of the entire cloud.

I am getting a bit sick of "Why is software engineering so bad compared to XYZ". All the people asking are either not in this field or are junior engineers who just don't yet understand why things are the way they are and why writing bug-free, mathematically proven software is infeasible, unnecessary, impractical, undesired, etc.

Consider another analogy. When building a house, how expensive is it to decide that you actually want to have a ceiling with 5 meters high in the second floor, because the buyer of that floor wants higher ceilings? Simple: Tear down the building and start over...

In software? Usually it comes down to adding an IF-statement or some such. Software follows completely different economic principles and serves completely different needs.

There is at least one dimension though where this problem currently shines through: Self-Driving Cars.

I think auto-pilots in self-driving cars SHOULD follow the same rigor as airplane auto-pilots and they are most certainly not doing that, which will and already has resulted in lost lives.

<comment><USER18> "They optimize for a compromise between short-term gain and long-term gain"

Hey! Author here. This is the exact point that I make in my essay. I don't compare it to any other fields, and certainly don't mention airplanes or elevators. I explain how this compromise is deliberately reached for people who aren't familiar with the dynamics of software projects. I recommend that you read the post! It sounds like we agree on a lot.

</comment></comment><comment><USER97> [The Pure Function Pipeline Data Flow v3.0 with Warehouse/Workshop Model]([https://github.com/linpengcheng/PurefunctionPipelineDataflow](https://github.com/linpengcheng/PurefunctionPipelineDataflow))

The idea of simplicity and unity is an important guiding ideology of scientific research. It can be found from the knowledge of various disciplines:

the more universally applicable a unified theory, the simpler it is, and the more basic it is, the greater it is.

[The Pure Function Pipeline Data Flow v3.0 with Warehouse/Workshop Model] systematically simulates integrated circuit systems and large industrial production lines. It perfectly achieves simplicity and unity. It must be the ultimate programming methodology.

In the computer field, for the first time, it was realized that the unification of hardware engineering and software engineering on the logical model. It has been extended from `Lisp language-level code and data unification` to `system engineering-level software and hardware unification`.

It brings large industrial production theory and methods to software engineering. It incorporates IT industry into modern standardized large industrial production systems, This is an epoch-making innovative theory and method.

1\. The replaceability, insertability, observability, and readability of the pipeline are very strong.

2\. The observability, standard, fairness of the dataflow are very strong. Data standards (data interfaces, data specifications) are better than code interfaces. and non-IT practitioners can understand.

3\. The warehouse/workshop model is widely adaptable, simple, and reliable, and the workshops are isolated from each other.

There are only five basic components:

1\. Pipeline (pure function)

2\. Branch

3\. Reflow (feedback, whirlpool, recursion)

4\. Shunt (concurrent, parallel)

5\. Confluence.

</comment><comment><USER98> Software is built like crap because when it fails it doesn't kill anyone. Except the software that does kill people when it fails, and that's engineered properly -- not perfectly, but in line with other things like airliners and elevators.

</comment><comment><USER99> How do these articles exist without mentioning formal methods? You realize there exists systems and techniques that can prove your program to be 100 percent correct? Whatever the pitfalls articles about code quality and the quality of software in general are pure ignorance if this aspect of software isn't mentioned.

</comment><comment><USER100> We're "bad at software engineering" because we're bad at whole-systems design, and we're bad at whole-systems design because the market does not incentivize it.

</comment><comment><USER101> We just stricter type systems.</comment>


Rocket Reply for safe messaging while driving - <AUTHOR> Hi! My name is Alexey. I am developing an app for car drivers who do texting while driving. Rocket Reply is an Android app which makes text messaging more convenient and less dangerous while driving by eliminating the need to type but use voice to text transformation and quick replies.
I am looking for any feedback from car drivers.
Thank you! http://www.komparato.com/reply/ <comment><USER2> On one level the idea is not that bad.

On another level, however, you are "normalizing deviance" that it is OK to use apps in the car. If somebody crashes their car while using the app they might blame you and sue you.

There is a huge amount of controversy about where the dangers lie in using technology with driving. Personally I am not concerned about the hands as I am the eyes. If you have to look away from the road for two seconds you are covering a frightful amount of ground and have no chance of detecting something dangerous. To some extent you can mitigate this by carefully setting up the field of view such that your device is the center of the view but you still have peripheral version out the front window. Even if your hands and eyes are not affected, there is still the risk you could be mentally or emotionally tilted by some message you exchange with people. In fact, when I took a defensive driving class they said I shouldn't even get in the car if I am emotionally upset about something.

There has been some talk about distracted driving risks with the police. Police are trained to use the techniques I mention above to mitigate risk. They drive a lot, and there are a lot of them, so every so often it gets in the news that they hit a child or a cyclist because they were chatting with other cops via their proprietary chat system.

Funny there has not been a lot of talk about the risks of CB radio and ham radio operators. There aren't that many hams, they don't have the money to be good lawsuit targets, but the ARRL wants us to be careful so that we don't start hearing about the risks on the news.

<comment><AUTHOR> Paul, many thanks for such a detailed feedback! My idea is to use the Rocket Reply app when my phone is hold in a cradle. Problem: people want to communicate via messengers (stay online) even when they are driving a car despite it is not safe (distracts from the road) and prohibited by law in some countries. The problem is popular among car drivers, it's growing, fines are growing (first offence in NY can cost you $200, third offence will be $450) Solution: replace typing with voice recognition and smart reply suggestions to respond with a single tap on the phone screen reducing the distraction time.

</comment></comment><comment><USER3> How about you just don't pay attention to your phone while operating a two ton death machine?

<comment><AUTHOR> Agreed. However we waste time in traffic jams</comment></comment>


Pioneer announces MEMS-based LIDAR for self driving cars [pdf] - <AUTHOR> http://pioneer.jp/en/news/press/2015/pdf/0901-1.pdf

<comment><USER2> If those things are as good as they look from that press release then robotics companies are going to be snapping those sensors up as fast as they can.

EDIT: My coworker points me to an article that clarifies that "low cost" here means $80, not $800 like I'd assumed.

[http://www.itworld.com/article/2980018/pioneer-harnessing- la...](http://www.itworld.com/article/2980018/pioneer-harnessing-laserdisc- tech-for-low-cost-lidar.html)

</comment><comment><USER3> Sounds very interesting. After reading the press release though, I can't find any references to MEMS or why/how this is better than existing LIDAR sensors. Any other resources OP?

<comment><USER2> Generally LIDARs work by rotating a mirror that the laser bounces off of giving you a 2D plane or cone where you can detect things. Super expensive LIDARs like the one on the Google car use a number of cones with different angles to approximate a 3D system and cost tens of thousands of dollars. Other systems like the ones you see in the DARPA challenge might rotate the plane of the LIDAR through a full circle, but that takes a while.

Spinning mirrors are also somewhat delicate which can be a problem for automotive applications.

<comment><AUTHOR> Exactly. That expensive rotating Velodyne thing is a research tool, not a production auto part. Production cars need better, preferably with no big moving parts. The Pioneer MEMS device qualifies (the moving parts are at IC scale). The new Kinect is a no-moving-parts 3D LIDAR. The Advanced Scientific Concepts flash LIDAR is also a good 3D LIDAR, usable outdoors in daylight. It costs about $100K because it's hand-made by physicists in Santa Barbara for DOD and Space-X, but the technology isn't inherently expensive.

The Pioneer press release doesn't say much. The usual specs are X and Y pixels, field of view, and range, none of which they give.

</comment></comment></comment><comment> TD-Linux Neat, it's too bad they seem to be tying it into some awful software and mapping system. Just sell the part that you're good at and let other people do the software!</comment>


Ask Better Questions - <AUTHOR> http://www.chrisbrogan.com/ask-better-questions/

<comment><USER2> Great post! I don't think I spend enough time on questions.</comment>


Ask HN: How do you cope up with the emotional rollercoaster of a start up? - <AUTHOR>

Some days I feel like a king and some days just plain hopeless.<p>I am having a rough ride guys. <comment><USER2> There's no real answer to this question. Here are some things that might help:

1) Separate your work from your life. Even if you work 12-14 hours a day, have something you do that is away from a computer and distinct from work. Work from home? Set up an office area and leave the house at the end of the day. I like going for long walks.

2) Get a hobby. Something that doesn't demand too much time but is engrossing. I like to read fiction. Currently reading Spook City by William Gibson. Psuedo-sequel to Pattern Recognition. It's decent. Reading can be, if you let it, a micro vacation.

3) Exercise. It helps with anxiety, depression and fatigue.

4) Know that this is not the end of the road. You have to try your hardest to make this work, but if it doesn't it's not a big deal. It's not going to ruin you. Or be a mark against you on your resume. It's going to be the best thing to have ever happened to you, success or no: Learning from your mistakes is an option. Success is an option. Failure is not.

/techpep talk

Also, some other perspectives: <http://www.youtube.com/watch?v=2CdJTfGiRCI> <http://www.youtube.com/watch?v=TROhlThs9qY>

</comment><comment><USER3> [http://grasshopper.com/blog/founders/2010/04/06/take-a- ride-...](http://grasshopper.com/blog/founders/2010/04/06/take-a-ride-on-the- entrepreneur-roller-coaster/)

<comment><USER4> Worth watching; nice pick.

Important note: the timing between each milestone will vary among businesses and even among founders in and among those businesses. In my experience, founders with "Let's Change The World" ideas -- as opposed to, say, "Let's Make A Facebook for Kids" ideas -- tend to spend most of their time in between Informed Pessimism and Hopeful Realization.

In other words, the more revolutionary your goal, the more time you'll spend trying to reconcile whether that goal is indeed worth reaching. And that's before you even begin to assess whether your way of reaching that goal is indeed the best way to do so.

</comment></comment><comment><USER5> I remind myself of how far I've come since the days I was a clueless adolescent always thinking about "starting my own business."

</comment><comment><USER6> I wake up every day and say to myself 'I'm the luckiest person in the world to even have a chance to live my dream'.</comment>