New Storage Device is Very Small, as in 12 Atoms - <AUTHOR> http://www.nytimes.com/2012/01/13/science/smaller-magnetic-materials-push-boundaries-of-nanotechnology.html?hp <comment><USER2> 12 atoms is the quantity used to store a single bit (~100 atoms / byte) near absolute zero. As usual that important nugget was buried in the middle of the article; just reading the first 3 paragraphs, one would think they had fit an entire floppy into 12 atoms.

<comment><USER3> I'm not sure how you got that impression. I only skimmed the article, and picked up the one bit per 12 atoms detail, thinking, "Well, it's not quite memory diamond, but it's as close as we've ever come."

</comment></comment><comment><USER4> I think it's worth noting that this was an experiment done as part of a research paper. The HN title basically implies that it's something ready for production. Nonetheless very impressive.

</comment><comment><USER5> Extrapolating this, wouldn't that mean we could possibly get as much as 6.02e23 / (12 + C) bytes per mol of storage material?

Where C is the number of atoms required to communicate with that segment of memory. It might be (6.02e23 / 12) + C, even!

That's a terrifying 5,569,618.84 petabytes for what could amount to a few hundred grams of materials (assuming it's a compound, and not just iron (at approx 56 grams a mol)).

Perhaps even many thousand petabytes in a storage space the size of a paperclip.

The potential is incredible; and it makes me worry: what will developers waste that space on next?

<comment><USER6> The article is about arranging atoms on a surface with a scanning tunnelling microscope. You you don't seem to be counting this required surface it in your "storage material". And that's still ignoring all the machinery needed to actually arrange/detect the atoms.

<comment><USER5> The reading material was considered with the C at the end of the calculation; but even still, if it takes 100 times as much equipment to read and write as it does to store, those are still ridiculous numbers. That was my point.</comment></comment></comment>


Tool to help generate DNS CAA record for SSL certs - <AUTHOR> https://sslmate.com/labs/caa/ <comment><AUTHOR> [https://tools.ietf.org/html/rfc6844](https://tools.ietf.org/html/rfc6844) :

"The Certification Authority Authorization (CAA) DNS Resource Record allows a DNS domain name holder to specify one or more Certification Authorities (CAs) authorized to issue certificates for that domain."

This tool is provided by sslmate, but works regardless of which CA and which DNS provider you are using. It shows you exactly what records to add to your DNS based on simple html form.</comment>


Most coders have sleep problems, need 'hygiene and care' - <AUTHOR> http://www.theregister.co.uk/2010/11/23/coders_insomnia_mental_hygiene/

<comment><USER2> _The test subjects were 91 software engineers [...].

[...] no fewer than 20.9 per cent [...] and a further 35.2 per cent [...]_

Sorry, but I can't take a study serious that has less than 100 participants and presents results with a place after the decimal point.

<comment><USER3> Not to mention some serious sampling bias. The reason for stress could be cultural or geographic. To say it's because they're coders is silly.

<comment><USER4> "The test subjects were 91 software engineers working at a Mysore-based development firm."

<comment><USER5> I initially read "Mysore" as a strange spelling for "Missouri" and then realized Mysore is a city in India.

</comment></comment></comment></comment><comment><USER6> _"Lifestyle management programmes which include sleep hygiene and care should be incorporated as a policy matter in the IT industry."_

Yes, because adults need other adults to tell them when they should go to sleep..

All you really need is management that knows a thing or two about software engineering and a bit of a "don't worry, we'll fix it tomorrow, go home"-attitude. If you're a manager and your team is constantly stressed out, you should be fired immediately.

<comment><USER7> I don't think it's entirely a management issue.

There are plenty of coders working normal hours then coding into the night on personal projects and missing sleep as a result. It's almost become a macho thing amongst developers - when was the last time you pulled an all-nighter, how many hours have you put into project X.

<comment><USER8> When actually it should be the opposite... The less time you need to code a project, the better coder you are.

</comment></comment></comment><comment><USER8> Ha! The key phrase is the following: "The test subjects were 91 software engineers working at a Mysore-based development firm." It must suck to work there...

<comment><USER9> I don't know, I work at my university as a mobile software engineer and I have trouble going to sleep at night as well.

It's hard to think about a million different problems a day and how to solve them then just cut all that off at 10-11 or so each night.

It probably doesn't help that coding is my hobby as well as my profession, so it really doesn't ever get turned off.

<comment><USER10> 2 things that in combination completely eliminate sleep issues for me:

\- f.lux (time of day dependent display colour temperature)

\- physical exercise

If I drop either for more than a few days, I'm back in insomnia land. If I exercised every single day, I'm sure I wouldn't need f.lux. Unfortunately, exercise takes quite a bit of willpower in winter (no nearby gyms).

<comment><USER11> I wonder if there's correlation between below-average physical activity and insomnia? I assume that a lot of coders---by necessity of their task and the tools they use---spend most of a week sitting as opposed to, say, a mail carrier or store clerk. Given that we're a species that evolved to leverage mobility for our survival, it's not a stretch to imagine that our bodies developed a dependency on lactic acid buildup and other "fatigue toxins" as one of the signals that sleepytime is at hand.

Has anyone done this study? It seems like a pretty straightforward study to do if some grad student out there is looking for a paper.

<comment><USER10> It's a recurring topic on HN. There seems to be plenty of anecdotal evidence for it.

</comment></comment></comment></comment></comment><comment><USER12> There is something magic in the mind when you work until late, it seems that the lack of distractions really help, but if your body is not prepared, the next day you loose al the time that you have saved triying just to be wake up.

</comment><comment><USER13> > "Lifestyle management programmes which include sleep hygiene and care should > be incorporated as a policy matter in the IT industry."

I really wonder how would they consider puling this off. :)

Most coders(like me) are by nature caffeine addicted and expose themselves to insomnia just for the fun of getting some really cool job done. It's up to the coders to find out for themselves where are their boundaries(biological), and since most coders are probably intelligent people, I think they won't have much difficulty with this. In other words: we need no nanny.

</comment><comment><USER14> Today I got up at 3am. At least I get a lot done, there aren't too many distractions at that time of day.

<comment><USER15> I get up at 345am, seven days a week and haven't missed a day for months. Productivity gain has been huge for me. No one bothers me for about six hours after I get up, which is a long time....

</comment></comment><comment><USER16> So let's summarize: 50 of the 91 coders who work for a given company have mild to severe insomnia.

That really doesn't warrant the ridiculously vague and sensationalistic title we ended up with.

</comment><comment><USER17> Going to sleep at 3AM and waking up at 9AM for work. Yep. :/

</comment><comment><USER18> just to mention a few: \- natural light light bulbs \- walk outside during daytime, under sunlight as weather permits \- only go to bed when sleepy

Drink water a lot. I sometimes not drink enough and get real bad headaches by the end of the day.

</comment><comment><USER19> So all I need to be a great programmer is just stop sleeping and bathing. Thanks HN!

<comment><USER20> The headline is a little misleading. The word "hygiene" means more than just "cleanliness", although the two are frequently used interchangeably. In this case, it refers specifically to "sleep hygiene", which is a medical term for sleep-related behaviors conducive to health (setting a regular sleep schedule, getting enough sleep, not consuming food/caffeine/alcohol too close to bedtime, etc.).

<http://en.wikipedia.org/wiki/Sleep_hygiene></comment></comment>


Eric Weinstein's Four Quadrant Model - <AUTHOR> https://tipolitics.com/eric-weinsteins-four-quadrant-model-55852b1f791b <comment><USER3> Uhm, seems like this article has borrowed a lot from the original one[1] by The Knife Media which was mentioned by Weintstein on the Rubin Report[2]. I highly recommend that RR episode.

1\. [https://medium.com/@rljunco/eric-weinsteins-four-quadrant- mo...](https://medium.com/@rljunco/eric-weinsteins-four-quadrant-model-the- knife-media-6e642ff3f54b)

2\. [https://www.youtube.com/watch?v=MmXq97do- tQ](https://www.youtube.com/watch?v=MmXq97do-tQ)

<comment><USER4> I wanted to watch, but seriously who TF is going to watch a nearly 3 hour video?

<comment><USER3> People have no problem watching movies that long, that have much fewer insights and are often less entertaining than this. At least to me - it surely is subjective.

You can try skipping to the Q&A ~2h 14m. Then you're left with just 30mins, most Q&A questions were reasonable.

</comment></comment></comment><comment><USER5> I love this model, but I'm confused by this article since it keeps saying "the media's preferred policies" as if there is one singular policy that the entire media is in favour of, which I'm not sure I can buy into.

<comment><USER6> Yes, I think you need to bound off specific subsets of policy initiatives and "the media" for this model to make sense. It seems to me that Fox News and MSNBC do this symmetrically to each other over the same policies - Fox: Liberals promote affirmative action because they hate white people, MSNBC: Conservatives oppose affirmative action because they hate black people.

Informed people should be absolutely allergic to the suggestion that someone promoting or opposing a given policy does do because they have malicious intent (and frankly, be suspicious of suggestions that someone are doing it because they have particularly pure intent).

<comment><USER7> > Informed people should be absolutely allergic to the suggestion that someone > promoting or opposing a given policy does do because they have malicious > intent

This. Once you dig into the research, or just have a lot of conversations with people who have opposing viewpoints, you realize that the vast majority i.e. 99% of people have the same goals and well-meaning intentions. For example, conservatives and democrats both want to maximize human flourishing, they both want people to be happy, they want to prevent poverty and provide people with an environment that maximizes well being for everyone. All sides come from this same starting place morally.

Its on the best pathway to achieve our common goals that people differ in opinions. This is why I always advocate for more scientific method in politics and governmental processes, we need better data and models and we need to test those models so we can better discern which paths are best towards reaching our common goals.

<comment><USER8> > Once you dig into the research, or just have a lot of conversations with > people who have opposing viewpoints, you realize that the vast majority i.e. > 99% of people have the same goals and well-meaning intentions.

Suppose counterfactually that one side or both sides are in fact motivated by malice. Do you think you would be able to detect this by examining their officially stated desires and morality?

Nearly all people want to think of themselves as morally good and are good at telling stories (often to themselves) about why they are good. This doesn't provide strong evidence that they are in fact good.

<comment><USER7> > Do you think you would be able to detect [malicious motivation] by examining > their officially stated desires and morality?

Insofar as you can define a measurement for malicious intent, you could detect this. The set of measurements you define might not tell you everything you want to know about malicious intentions but they would give you a decent approximation.

For example:

Image we have a controlled random study, we have two groups A and B. We show both groups a medication K, and give group A a motivation (perhaps monetary say $10) for K to be approved, we give an equal weighted motivation $10 to group B for not approving. Then we show both groups information showing that K has some detrimental effect. Observe outcomes, and those who approve for the monetary motivation would be displaying "malicious intent" as defined by the measurements in this study.

Generalizing from studies, however, to what particular persons in political parties at a particular point in time are thinking is something for historians or debaters and a set of methods I am unfamiliar with, outside of a general knowledge of the dialectic.

<comment><USER6> The problem is, first you need a reliable measure of good and evil. Sure, most reasonable people can agree on some set of actions that are clearly either, but there's a lot in between. Working out this answer is basically the entire field of philosophy.

Your experiment won't measure evilness, perhaps something along the lines of honesty when faced with conflicting incentives (which of course is a desirable trait in leaders, political and otherwise). But what if one of the subjects in group A needs the money to buy a loaf of bread for their starving family, to illustrate using a favourite ethical dilemma? What do you do if a given policy will benefit yourself, but you know for certain that it's a right and good policy?

<comment><USER7> The only way I see of alleviating some of the problems you mention is by controlling for them and doing further experiments.

My experiment was not intended to measure evilness but rather to give an example of a general approach to the problem, and was for measuring malicious intent. Regardless, given time I'm sure we could come up with much better experiments for studying both.

You're right though at what you are getting at, and I think the underlying idea here is that there is a continuous scale from bad (0) to good (1) for any given action/policy, where given additional context our internal moral barometer can move across this domain as well as an independent societal human flourishing barometer which measures the broader repercussions of the action/policy.

Honestly, I do not know enough about ethics or research in this general area to have a truly insightful conversation on this topic.

I'd just like to add that 1) I think good and evil are so broad in their scope that we'd be virtually forced to break them into separate definitions and measurements and study these constituent parts if we wanted to understand the general concepts and 2) we'd need to make a clear distinction between what humans perceive as good/evil in each context vs. what has good/evil repercussions at different levels (individual/familial/societal) based on measurements of brain states, pain/suffering, and human flourishing.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> It's definitely not just the media, it's throughout the culture at this point. When it comes to agenda on both broad political sides, it has largely become a scenario of: if you're not with us 100%, then you're on the other side and against us / evil.

The political machine is now fully running on that. The extremism of the partisans in the US is at fever pitch.

This is fundamentally how someone like Matt Damon gets lit on fire - <USER2> nearly having his career destroyed - for saying perfectly reasonable and rational things, while simultaneously fully agreeing with the MeToo movement. There is no room for anything except for extremism and absolute agreement. If you so much as tilt your head the wrong way at the wrong time the political correctness machine will destroy you. Damon made the mistake of thinking you can still ask questions and call for reason to be in any of the hot topic discussions in the culture. Reason is out the window, it's all emotionalism all the time, spiked to the nth level.

</comment><comment><USER10> An important related idea is Russell Conjugation [1]. It would be good to have tools that analyze and rate bias in text based upon the use of emotionally loaded phrasing around positions.

[1] [https://en.wikipedia.org/wiki/Emotive_conjugation](https://en.wikipedia.org/wiki/Emotive_conjugation)

</comment><comment><USER11> While I understand how it applies in the shown cases I'm still puzzled about the quadrant naming. Are people on the top right aalways a Dupe? Is there any contrarian that would push for more policy on something?

How does this apply to something like "Should we put dangerous criminals in jail?"

<comment><USER12> The model suggests, if I read it correctly, that media would portray Contrarians (who dislike crime but don't think jail is the solution) as Troglodytes (who oppose jail because they favor crime.)

The "Dupe" label would then apply to those who accept the jail policy because they dislike crime. Perhaps the model tries to disparage people who make this leap, without themselves considering if the policy is a good way to combat crime.

I guess I see your point; there might be people who dislike crime and have done their homework on the policy, and the model unfairly labels them as dupes.

Perhaps the scope of the model is implicitly limited to reason about those who blindly accept the media's points. The article seems to focus elsewhere: on the case where media pushes the top left quadrant toward the diagonal.

Equipped with Hanlon's razor, one may alternatively posit a dimensionality reduction which, albeit lossy, improves the performance of media.

<comment><USER11> > I guess I see your point; there might be people who dislike crime and have > done their homework on the policy, and the model unfairly labels them as > dupes.

That's exactly what I was trying to say. Thank you for putting it down so clearly.

</comment></comment></comment><comment><USER13> Using Sam Harris as an example is way too charitable to the average person.

Most people are not first-principles thinkers about politics. They are self- interested. Where that motivation is considered unsavory, they will deny it, even to the point of deluding themselves. Race is a prime example, because, like all forms of identity, everyone is biased in favor of their own ingroup, but admitting this bias is far more stigmatized than admitting other forms of ingroup bias.

If you take nearly everyone in the United States at their word, then racism has been effectively wiped out. And yet the indirect evidence of it is everywhere. Which means people aren't being honest with others--and possibly even not with themselves--about being biased in favor of people who look, speak, and act like they do.

This gets even worse when we start talking about _politicians_ , who are trained to routinely lie, to the point where the media starts to safely, and correctly, assume that what they say is motivated entirely by self-interest and completely detached from the truth if it deviates in any way from what would advance their self-interest.

Notably, politicians who buck this trend--Justin Amash, Bernie Sanders, Thomas Massie--aren't generally stigmatized by the mainstream media as self- interested troglodytes. Their views are criticized, surely, but few question that they genuinely hold those views from first-principles reasoning, because they've demonstrated that they're principled people.

This suggests that the media is actually pretty good at distinguishing troglodytes from the truly principled, and while that may rub off on people like Ben Affleck the wrong way, directing your hatred at "the media" is misplacing it.

Also, the guy who wrote this article wrote a conspiracy screen about Obama's "plot to overturn the 2016 election" literally two days before. So him calling out others for impugning their political enemies' motives as coming from bad faith is pretty rich.

</comment><comment><USER14> Perhaps it is part of the point, but the stigmatized narrative is no more correct according to the 4Q model than the sanctioned one. That is it appears to map 'troglodytes' as morally virtuous. That is surely the correct narrative moves up and down the y-axis, and why can't that exist?

I struggle to put it into words, but for me the model has a koanic quality and is somewhat less of a clear insight than I initially thought it would be.

<comment><USER4> >> Perhaps it is part of the point, but the stigmatized narrative is no more correct according to the 4Q model than the sanctioned one. That is it appears to map 'troglodytes' as morally virtuous.

Your first statement is often correct, the stigmatized narrative may not be "correct". But no-one is trying to say the 'troglodytes' are morally virtuous either, and suggesting that's what TFA is trying to claim would be it's own attack on the model.

The whole point is that the model contains 4 types of people and the media will only allow for the existence to 2. Anyone that may have something valid to say that doesn't agree with the agenda is dumped into the lower left quadrant even though that may not be who they are.</comment></comment>


Slimbox – Declutters your inbox in seconds - <AUTHOR> https://slimboxapp.com <comment><USER2> A refreshing transparent privacy policy, I like it a lot.

I recommend adding 'Show HN' to the title so the submission goes to [https://news.ycombinator.com/show](https://news.ycombinator.com/show)

<comment><AUTHOR> Thanks so much for checking out for spending the time to check out the policy. We put a lot of thought into making it as human and transparent as possible. And really appreciate the tip.

</comment></comment><comment><USER3> I guess this could be useful for people who do not know how to use email rules / filters.</comment>


History of Python development since 1990 with Gource - <AUTHOR> http://www.youtube.com/watch?v=aPk1BqK8zzI <comment><USER2> I feel like a super-intelligent alien race that has come across Earth in its interplanetary travels and has made instructional documentaries about "human life" for its young in the same way that we make documentaries about insects and cell biology.</comment>


EduTech Idea – Your Opinion - <AUTHOR> This is a business idea. Please read it and let me know what you think.
A chatbot/app that helps you decide the best skills to learn based on market demand, your skill set, interests, and natural talents. All to make you competitive and help you gain a unique talent stack in the market. (A good example of that is Scott Adams - Intersection of comedy/humor, decent drawing, and insight into the business world).
Question #1: Do you think that this is a good idea?
Question #2: Would you purchase this product, if it helps you build a long term edge?
Question: How much would you pay?
Cheers!<AUTHOR> <comment><USER2> This is a good comment and i have made a new [https://aetech.shop](https://aetech.shop) and buying and selling affiliate items.</comment>


Git.Works Give your issues an address - <AUTHOR> https://git.works

<comment><USER2> Logging in means granting access to all kinds of private data, including deploy keys. This doesn't seem right.</comment>


Framework to create Plugins for all browsers by writing a single JS - <AUTHOR> http://www.besttoolbars.net/products/addon_framework/

<comment><AUTHOR> l</comment>


How to Start a Company in US as Non-US Resident/Foreigner - <AUTHOR> I am in the process of starting a company in North Carolina (the chosen one). I am living in the Philippines and a non-american citizen. Can someone help or guide me? I want to know about creating an LLC, having a bank account, and ways to receive payments from US to PH. <comment><USER2> why would you considering how you're treated at the border as foreigner by the current regime?</comment>


Cool User File Systems: GlusterFS - <AUTHOR> http://www.linux-mag.com/id/7833 <comment><USER2> I've been using Gluster in a production environment for over a year now with an absolutely perfect track record. I use it for the "centralized" filesystem for a virtualization cluster (using KVM). Currently, I have three virt hosts setup with dual NICs each to get 2Gbps throughput, sharing a RAID1-style filesystem (with priorities to the local copy for r/w speed).

I was (and still am) fairly skeptical about using Gluster in this role for a production environment having come from a fiber/SAN shop before. However, I gained a lot of faith in Gluster after a non-incident a few months back. One of our core switches fell offline (the one all three virt hosts were plugged into) and remained offline for about an hour I believe. When the switch was brought back online, Gluster managed to put everything back together and continue operation without any human intervention. All 20 or so VMs were still humming along like nothing had ever happened. fsck confirmed this.

Even in the case of a failure (which I've not experienced), your data is still sitting on disk (at least that's how it is with RAID1, I don't know how it's RAID0 setup works) -- you'll always have access to the raw files Gluster uses.

<comment><USER3> Glad to hear its being used successfully. I'm betting things improved greatly in the last couple of years. Thanks for sharing this. :)

<comment><USER2> It definitely has. Two years ago, I even had issues with some features of KVM/QEMU (live migration, for instance), let alone trying to run that on a distributed FS. It's amazing how fast these techs are coming along.

</comment></comment></comment><comment><USER3> No. Just no. Untested, unproven, and unstable. I've made comments about this before, but the short of it is: don't trust your production file system to something only researchers use.

When the developers of the product can't get it to work on your servers, and you've lost a good 3-4 days of uptime, you'll rethink not buying that SAN.</comment>


Show HN: Project management software for movie production - <AUTHOR>

This software is primarily aimed at micro budget and student producers who do much of their collaboration over the Internet rather than face to face in an office on a studio lot. I think the key selling feature is the ability to export a script from Final Draft and then automatically create the scene breakdowns and production board. The dominant player in the space is Movie Magic with their Scheduling and Budgeting software which run about $500 each. $1,000 is the whole budget for a lot of student films. For simpler productions this captures the basic features that they would need in addition to the project management and not being tied to a laptop or desktop.
I'd love to hear any and all feedback either via comments below or email to info at the domain name.
http://www.indieintern.com
Thanks.
p.s. I created a coupon for one month free on the first two levels of plans for anybody that wants to try it out. Use "hackernews" when signing up. <comment><AUTHOR> Link:

<http://www.indieintern.com></comment>


Show HN: Creating a website should be as easy as creating a Word doc - <AUTHOR> http://42pag.es

<comment><USER2> Great job. Nice execution. I would recommend adding some type of simple tutorial walk-through after the new page is first created. It took a little while for it to sink in that this is a blank canvas and I can start typing, etc. Maybe the fact that the canvas is the same color as the background of the site, it is not easy that this is a "word doc" where I can start building. I know you probably want to keep it minimalistic, but it might help to outline the canvas so this fact is more obvious.

</comment><comment><USER3> My first reaction: creating a website already is as easy as creating a Word doc.

How is this different (or how will it be different) from creating a Google Doc and sharing it with the public? Or using Google Sites for that matter?

(Not trying to jam you up -- genuinely interested in this and your answer.)

<comment><AUTHOR> That's why I'm doing this — I think it's crazy that our best solution for creating a simple, one-off site is to use a collaborative document service and share the link. Google Docs is built for documents, and you can tell—hopefully this can evolve into a better solution to the use case where you want to create a really simple web page. Thanks for the feedback!

<comment><USER3> "As easy as a Word doc" is misleading then. Editing CSS and JS isn't as easy as Word, even if it's considered a feature and a convenience.

I do think it's a good, catchy phrase, but it isn't necessarily hitting your target market.

Are you pretty similar to a hosted Jekyll service, then?

</comment></comment></comment><comment><USER4> Amazing take at simplicity. Anyone can edit by default is great, email-only authentication better yet.

I just don't know how to add more pages.</comment>


Hacking E.T. for the Atari 2600 to make it better - <AUTHOR> http://www.neocomputer.org/projects/et/

<comment><USER2> I saw this link on Reddit, pretty cool that he shows how a few small fixes could have averted a huge financial loss.

This was a game I played as a small child, and was able to figure out the way to the ending. Also seem to remember it was my first encounter with a software bug.</comment>


How to Build Good Software - <AUTHOR> https://www.csc.gov.sg/articles/how-to-build-good-software <comment><USER3> This struck a chord with me: "Software Is about Developing Knowledge More than Writing Code"

I've experienced more issues caused by management passing around tasks between teams and never paying attention to knowledge and knowledge transfer.

What's amazing, is that in over 18 years as a software engineer, I've seen this so many times. Teams will function well, then the institution tries to change. Often they will try to open up the "innovation" by throwing money at R&D, basically trying to add bodies in order to grow. Then you have tons of teams, and communication becomes very challenging, so then they grow some kind of "task management" layer. Management that never understands who actually _knows_ something, just tracks how much "theoretical bandwidth" they have and a wishlist of features to create. And then the crapware really starts flowing. And then I get bored and move on to the next place.

<comment><USER4> > "Software Is about Developing Knowledge More than Writing Code"

The company I work for uses Scrum. They consider the User Stories + the code to be everything you need. I struggle with this, but my manager says they don't want to get tied up doing documentation "because it goes out of date". Beside, they are being Agile which "prefers working code over comprehensive documentation".

I am wondering what other companies do to capture this "distilled knowledge". The backend services I rely on are undocumented beside some paltry swagger that leaves much to be desired. The front end has no product-level "spec", if you want to rebuild the thing from scratch. There isn't even a data dictionary, so everyone calls the same thing by different terms (in code, and conversation).

There are just user stories (thousands) and code.

Does anyone have any suggestions on how to fix this?

<comment><USER5> "prefers working code over comprehensive documentation" does not mean "don't do documentation".

Documentation is essential. How things work is an important thing to document. Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date. It still has problems (What do you do when the code and the documentation disagree? Which is correct?), but they're not as severe as the problems that arise when there is no documentation at all.

What is less useful is having comprehensive documentation for those things that are yet to exist. Writing a few hundred pages of specification and handing it over to the dev team is waterfall, and it is _this_ that the Agile manifesto signatories were interested in making clear.

I'd fix it with strategic DDD - I'd develop at least a "ubiquitous language" (or a UL): I'd get others to work with me on having clear terminology and making sure that is used consistently both in user stories and in the code base. That's table stakes.

I'd then event storm the contexts I'm working in and start to develop high level documentation.

Even at this point relationships between systems emerge, and you get to draw circles around things and name them (domains, contexts), and the UL gets a bit better. At this point you can start to think about describing some of your services using the UL and the language of domains and contexts.

By that point, people should start to click that this makes life easier - there is less confusion, and you're now all working together to get a shared understand of the design and the point of DDD is that the design and the code match.

The first part (all 100+ pages of it), of the Millet and Tune book on DDD will pay you dividends here.

If that doesn't work, look around for somewhere else to work that understands that software development is often a team sport and is committed to making that happen.

<comment><USER6>
        Documentation is essential. How things work is an important thing to
        document. Ideally it should be in version control and be generated
        from the code, because then it's less likely to go out of date.

My solution to this is old and fairly unpopular, but I stand by it: _anything in the codebase that 's not obvious to a new maintainer should have a brief, explanatory code comment._

Generally, this falls into two categories.

1\. Hacks/kludges to get around bugs in hardware, external services, or included libraries. These manifest in code as incomprehensible, ugly bits of code that are difficult to distinguish from code that is simply "sloppy" or uninformed. More importantly, they represent _hard-won knowledge._ It often takes many programmer-hours to discover that knowledge, and therefore many dollars. Why throw it away? (Tip: include the version of the dependency in the comment, ie)

        # work around bug in libfoo 2.3, see blahblahblah.com/issues/libfoo/48987 for info
        # should go away once we can upgrade to libfoo 3..
        if error_code == 42 reset_buffer()

...so that future programmers (including you) can more easily judge whether the kludge is still needed in the future.

2\. Business logic. This too is difficult/impossible to discern from looking at code. Often, one's git commit history is sufficient. But there are any number of scenarios where version control history can become divorced from the code, or require a fair bit of git/hg/svn/whatever spelunking to access. And this of course becomes increasingly onerous as a module grows. If there are 200 lines of code in a given module, it is a significant time investment to go git spelunking for the origins of all 200 lines of code. Some concise internal documentation in the form of code comments can save an order of magnitude or two of effort.

        It still has problems (What do you do when the code and the
        documentation disagree? Which is correct?), but they're not as
        severe as the problems that arise when there is no documentation at all.

This is pretty easy to enforce at code review time, prior to merging.

In the first place, only a true maniac would intentionally update

        # no sales tax in Kerplakistan on Mondays
        return nil if country_code==56 and day_of_week==1

...without updating the associated comment. If they do neglect to update it, that's an easy catch at review time.

<comment><USER7> While I still would add a comment about the why, your last bit of code probably should be written without magic constants.

        # Some countries have sales tax rules dependent on the day of the week
        return nil if country_code==KERPLAKISTAN and day_of_week==MONDAY

The exact comment here could probably be more specific (e.g. where do you find these rules), but it also most likely shouldn't repeat the code (and the code should make clear what it represents).

<comment><USER8> But don't do what one memorably awful project I had to maintain did - to use that example they would have done:

country_code==FIFTY_FIVE and day_of_week==ONE

<comment><USER9> But what if the definition of 55 changes? You'll be glad to have your table of constants then.

<comment><USER8> The project also defined HTTP, COLON, SLASH, WWW and DOT so that you would have:

       string url = HTTP + COLON + SLASH + SLASH + WWW + DOT ...

I swear I'm not making this up....

<comment><USER10> Sounds like a PHP codebase I'm currently working in. I shit you not, $LI = '<li>' is in the functions file, along with $LI_END.

<comment><USER8> It was a _very_ enterprisey Java codebase from the bad old days of J2EE - <USER2> had somewhere over 30 layers of abstractions between the code in a JSP and a web service call.

[NB 30 isn't an exaggeration - I think the vast team who wrote it were paid by the abstraction or something].

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Some good tidbits from the government perspective on software development:

" _Beware of bureaucratic goals masquerading as problem statements. "Drivers feel frustrated when dealing with parking coupons" is a problem. "We need to build an app for drivers as part of our Ministry Family Digitisation Plans" is not. "Users are annoyed at how hard it is to find information on government websites" is a problem. "As part of the Digital Government Blueprint, we need to rebuild our websites to conform to the new design service standards" is not. If our end goal is to make citizens' lives better, we need to explicitly acknowledge the things that are making their lives worse._ "

<comment><USER12> This also very much reads like something from Singapore.

<comment><USER13> A quasi-Orwellian dystopia Singapore may be, but their government is effective.

<comment><USER14> Only for certain strained definitions of "effective".

And, if you are on the wrong side, it is very "effective" at ruining your life.

Most of us would take a bit less "effective" in order to avoid that, thanks.

<comment><USER13> I don't understand the point of replying like this. Clearly we agree that the Singaporean government is very good at getting things done, and we agree that the things it wants to get done are horrible. Why are you speaking as if our opinions differ? Why manufacture conflict where none exists? Is calling something a quasi-Orwellian dystopia now too subtle an expression of disapproval?

<comment><USER15> > _Clearly we agree that the Singaporean government is very good at getting > things done, and we agree that the things it wants to get done are > horrible._

Singaporean here. The government's mainly effective for tasks that are on a happy path. If your particular case falls through the cracks, it often takes phone calls, printing, postage, and weeks or months of waiting to get stuff done.

(Personal experience trying to get business stuff done _not_ as a Private Limited company.)

<comment><USER16> > If your particular case falls through the cracks, it often takes phone > calls, printing, postage, and weeks or months of waiting to get stuff done.

That sounds like the happy path for dealing with the Canadian government. Well, except the months part.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> I thought "what a bold title, if someone's figured it out we can just close HN" and upon reading, hey it's not far off.

The following is a wonderful point I have hardly ever heard said directly:

"The main value in software is not the code produced, but the knowledge accumulated by the people who produced it."

<comment><USER18> weird, I've heard it said frequently for decades in various forms,

"value your knowledge workers"

"your employees are your most valuable asset"

Some companies don't treat employees well, and some employees at good companies feel they are not treated well enough

If the above quotes do not strike a chord with you, you might just be a software engineer who thinks you're more important than non-SEs.

<comment><USER17> The difference, for me, is that neither of those quotes explain why you should value knowledge workers or why employees are valuable (maybe hiring is expensive, maybe turnover reduces morale, etc), nor do they suggest the mechanism that creates this value.

I'm sure another author has put the same sentiment out there before, but it's not every day I see such a nice phrasing of it.

<comment><USER18> > neither of those quotes explain why you should value knowledge

I mean, the point of short quotes is to be memorable and get future listeners to hunt for the reason behind them. "The sun will rise tomorrow" may also be meaningless for some people on its own.

Nothing wrong with elaborating on this subject again via a blog post, I was just pointing out to the commenter who's never heard this expressed before that it has a long history, that's all.

</comment></comment></comment></comment><comment><USER19> > Reusing software lets you build good things quickly

It also introduces unknown amounts of debt and increases the likelihood that you'll end up with intractable performance/quality/velocity problems that can only be solved by re-writing large portions of your codebase.

This can be a dangerous cultural value when it's not presented with caution, which it isn't here. I think it's best to present it alongside Joel Spoelsky's classic advice: "If it's a core business function — do it yourself, no matter what".

[https://www.joelonsoftware.com/2001/10/14/in-defense-of- not-...](https://www.joelonsoftware.com/2001/10/14/in-defense-of-not-invented- here-syndrome/)

<comment><USER20> Great article. I liked this quote:

```

The best advice I can offer:

If it's a core business function — do it yourself, no matter what.

Pick your core business competencies and goals, and do those in house. If you're a software company, writing excellent code is how you're going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you're a pharmaceutical company, write software for drug research, but don't write your own accounting package. If you're a web accounting service, write your own accounting package, but don't try to create your own magazine ads. If you have customers, never outsource customer service.

```

This all rings true in my experience. You should write the software that's critical to your core business competency yourself, because the maintenance cost is worth paying if you can achieve better software. But if it's not a core competency and your business isn't directly going to benefit from having best in class vs good enough, then it may be worth outsourcing.

</comment></comment><comment><USER21> I found this to be an incredibly accessible and easy to read guide for software development. It's a very short read - just a few minutes - but it's full of practical examples and written in a way that speaks to non-engineers (like bureaucrats). If you are a non-technical person handling software stuff, this article should definitely be high on the reading list.

The author seems like an unknown in the software development world, but they're one of the managers for Singapore's fairly successful digital government initiative. So it does feel safe to say they have some experience.

<comment><USER22> Li Hongyi is the son of Singapore PM Lee Hsien Loong, as well as a deputy director in GovTech Singapore (the Government Technology Agency). (He's also an MIT CS grad, and a past Googler.)

I suppose he wrote this for other people in the Singapore civil service.

</comment></comment><comment><USER23> Nice post! Agreed on keeping the initial stuff simple as possible.

In python, I typically follow a pattern of keeping stuff in __name__ == '__main__' block and running it directly, then splitting to functions with basics args/kwargs, and finally classes. I divide into functions based on testability btw. Which is another win, since functional tests are great to assert against and cover/fuzz with pytest.mark.parameterize [1]

If the content of this post interested you: _Code Complete: A Practical Handbook of Software Construction_ by Steve McConnell would make good further reading.

Aside: If the domain .gov.sg caught your eye: [https://en.wikipedia.org/wiki/Civil_Service_College_Singapor...](https://en.wikipedia.org/wiki/Civil_Service_College_Singapore)

[1] [https://docs.pytest.org/en/latest/parametrize.html](https://docs.pytest.org/en/latest/parametrize.html)

<comment><USER24> I prefer putting the main code into a "main" function (called from the __name__ == '__main__' block) fairly early, since otherwise the functions you extract might accidentally keep relying on global variables.

<comment><USER23> Good point

I like to do it early also, to make sure that the new script, if imported from by a sibling module, is inert.

An example would be a scripts/ folder and sharing a few functions between scripts w/o duplicating.

In some cases I don't have a choice. Initialization of a flask app/ORM stuff/etc has to be done in the correct order.

I think the general rule of thumb I follow is: avoiding keeping code that'd "run" in the root level. Keep it in blocks (normally to me functions) has the added effect of labeling what is does.

What I don't do: I don't introduce classes until very late. In hindsight, every time I tried to introduce a complicated object model, I feel I tended to overengineer / encounter YAGNI

</comment></comment></comment><comment><USER25> The article appears to be written by Singaporean prime minister Lee Hsien Loong's son, Li Hongyi.

[http://theindependent.sg/li-hongyi-singapore-has-a-lot-of- pr...](http://theindependent.sg/li-hongyi-singapore-has-a-lot-of-problems-but- we-have-political-stability-and-resources/)

<comment><USER26> Now I'm wondering why the children romanized their surname as Li not Lee.

I came across this article: [https://mothership.sg/2015/03/lee-hsien-yang- reveals-the-sto...](https://mothership.sg/2015/03/lee-hsien-yang-reveals-the- story-behind-the-names-in-the-lee-family/)

> I have taught my children never to mention or flaunt their relationship to > their grandfather, that they needed to make their own way in the world only > on their own merits and industry.

<comment><USER25> Singapore's older generations speak/spoke Chinese "dialects" like Hakka (Lee Kuan Yew's heritage), but there has been a massive government-led push towards standardizing on Mandarin as the one true Chinese. Hence Lee Hsien Loong's children all have their names officially romanized in Mandarin pinyin (Li), not Hakka (Lee). The underlying character, 李, is still the same.

</comment></comment></comment><comment><USER27> Regarding "Seek Out Problems and Iterate", it's a bit of an understatement how important this is. I've invested a lot of time helping my coworkers understand the distinction between tasks and problems. The end goal being only tracking problems in the ticketing system. It's not easy to do this and it takes constant effort, but it pays off very quickly. I've yet to see a real "problem" ticket stay unresolved for a long time, whereas "task" tickets tend to stay around until they're either irrelevant or they get closed after getting kicked between a few people.

A good example of this is:

\- Add worker thread for X to offload Y

When the actual problem is more along the lines of:

\- Latency spikes on Tuesdays at 3pm in main thread

Which may be caused by a cronjob kicking off and hogging disk IO for a few minutes.

A good rule of thumb I've found is that task tickets tend to have exactly one way of solving them, whereas problem tickets can be solved in many ways.

<comment><USER28> Can you explain the 3pm on Tuesdays issue? My sister works for LLS and she said their servers get very slow at a precise time every Tuesday. Not saying it's the same bug, but what was the solution in your specific case?

<comment><USER29> The next sentence suggested that the cause of the problem in this probably hypothetical situation might be "a cronjob kicking off and hogging disk IO for a few minutes".

So in that case, I guess either run the job with a lower priority and see if that helps, or execute the job more often so it doesn't have to catch-up all at once one time per week, or rewrite it so that it performs I/O with smaller chunks of data at a time and sleeps for a little while in-between reading or writing chunks of data. Basically, do something so that you no longer have this one huge job consuming all of the IO bandwidth for several minutes every week.

</comment></comment></comment><comment><USER30> Building good software requires mainly achieving two things:

1\. Making sure what you build is what was really requested (correct), and

2\. Making sure what you've built doesn't have a higher running "cost" than the thing it replaced (either manual process or old automated solutions).

Everything else, IME, is ancillary. Performance, choice of platform, frameworks, methodology to build, maintainability etc are sub-objectives and should never be prioritized over the first two objectives. I have worked on many projects where the team focussed mostly on the "how to build" parts and have inevitably dropped the ball on the "what" to build of the projects. Result: failure.

Sauce: personal experience with several years of different projects (n = 1; episodes = 20+ projects that have gone live and have remained live versus 100+ projects lying by the wayside).

Writing software is _not_ easy.

<comment><USER31> I agree with you but what I've noticed is for all the large projects I've worked on it was impossible to get an official answer as to whether or not the whole endeavor had a positive ROI. In fact, with a little back of the napkin math and some knowledge of the project's resource allocation it was obvious in most cases there would not ever be a positive ROI.

</comment></comment><comment><USER32> > 3\. Hire the best engineers you can.

This is where most companies fail. Yes, they do want the best developers, but for the budget of an average junior/medior dev.

For some reason most companies/managers I worked for do not understand the financial impact of a not so good developer. Or the other way around; they fail to value the best developers and are unable recognize them.

I've worked for plenty companies where they let mediocre dev's build a big app from scratch (including the architecture), in an Agile self managed team.. These are the codebases that always need to be rewritten entirely because they have become an unmanageble buggy mess of bad ideas and wrong solutions.

<comment><USER33> >"3\. Hire the best engineers you can."

If every single company wants that, where is he space to grow and learn from mistakes?

Maybe I'm wrong but I think those "mediocre dev's" learned a lot building a big app from scratch, solving bugs and refactoring.

<comment><USER34> They can learn them on their own time outside of work.

<comment><USER35> This is just an awful, awful mindset.

If you want great devs, you're going to have to invest in junior devs, and you're going to have to expect them to be learning at work. This is also why the best use of your senior devs is as mentors to your less experienced ones.

<comment><USER36> I agree that it is an awful mindset, but it's also true that they should not be given a task that they can barely do. Instead the formerly mentioned great devs should mentor over them, so they can become similarly good.

</comment></comment></comment></comment></comment><comment><USER37> One of the principles the article highlights is that additional features make a software complex and therefore more likely to fail. This is true, but I'd argue it's not for the reason the article claims.

The claim is:

> Stakeholders who want to increase the priority for a feature have to also > consider what features they are willing to deprioritise. Teams can start on > the most critical objectives, working their way down the list as time and > resources allow.

In other words, the argument is "competing priorities in a large-scale project make it more likely to fail, _because stakeholders can 't figure out which ones to do first_." Actually, in this very paragraph, the author glosses over the real issue: _" Teams can start on the most critical objectives, working their way down the list"_ \- treating development as an assembly line input- to-output process.

I argue that it's not time constraints that complex programs bad, but instead the mere act of thinking that throwing more developers at the work will make it any better. Treating the application as a "todo list" rather than a clockwork of engineering makes a huge difference in the quality of the work. When developers are given a list of customer-facing features to achieve, more often than not the code winds up a giant ball of if-statements and special cases.

So yes, I do agree that complex software is worse and more prone to failure than simple software - but not for the reason that there's "too much to do" or that prioritizing is hard. Complex software sucks because it's requirement- driven, instead of crafted by loving hands. No one takes the time to understand the rest of the team's architecture or frameworks when just throwing in another special case takes a tenth of the time.

<comment><USER38> I've also seen the failures in requirement driven software. When engineers receive unfiltered customer requests as requirements or tasks they tend to focus simply on getting that functionality into the software. Most times not understanding the job the customer is trying to get done.

There are different personalities of engineers, those who thrive on explicit requirements and can accomplish difficult engineering tasks when they are given clear requirements. But those engineers should only be given those requirements once the job that the customer is trying to get done is clearly understood. Some engineers have the ability to find creative solutions, that customers or product managers can't see, when they are provided with problems and jobs rather than requirements and tasks.

Managers would be wise to distinguish between the type of engineers they are managing and play to their strengths. Whatever type you have, understanding the job the end user is trying to get done must occur, preferably by an engineer that's capable of articulating that, if needed, to team members as technical requirements.

<comment> pure-awesome Paraphrasing, you said

> There are engineers who can accomplish difficult engineering tasks when they > are given clear requirements and engineers have the ability to find creative > solutions when they are provided with problems and jobs rather than > requirements and tasks.

I feel like I could perform adequately in either environment. The problem is I've previously found myself in environments where I'm expected to come up with creative solutions to a problem, but I have no access to the customer or even a simulated environment where I could try to do something similar to what a customer would do.

In this kind of case, it's impossible to really know how to articulate your requirements, because all you can use is a fantasy model of hypotheticals. But requests for more precise requirements are potentially brushed off as wanting to be spoon-fed what you need to do and having inability or unwillingness to think creatively.

</comment></comment></comment><comment><USER39> I've personally been thinking about this for some time and wondering if in the real world this looks like building as much as possible at the database level and treating your DB as a state machine for your app, aiming to disallow whole classes of errors and communicating the design of the business logic at the SQL functions/triggers/data layer, separate from the API, Services, Programming Language, and Frontend layer(s).

This means that instead of lots of issues with business logic being separate from the data the business logic and data sit together and prevent your system from getting into bad states.

Thinking about this, maybe I just stole this thought from Derek Sivers: [https://sivers.org/pg](https://sivers.org/pg)

<comment><USER40> Yes, the data model is probably the most important aspect of your application, it defines the relations and constraints. With a good data model, you don't need to write a lot of code to deal with it. Having lots of code that deals with weird situations in the database means your data model needs some serious consideration.

A database in my opinion is not a good place to write business logic with functions and triggers, since there is lack of tooling that would make development and debugging easy. Let the database do what it does well, which is storing and querying data.

</comment></comment><comment><USER41> Nice Post. But everyone needs to understand something. Even if you follow these principles to the letter T, you can still produce very bad software. In fact you can also find many cases where people did the exact opposite of what this guy said and still produced great software. I'm sure many people can name examples of software that just came together out of blind luck.

Why?

Because there is no formal definition for what is bad or good software. Nobody knows exactly why software gets bad or why software gets good or what it even exactly is... It's like predicting the weather. The interacting variables form a movement so complex that it is somewhat impossible to predict with 100% accuracy.

What you're reading from this guy is the classic anecdotal post of design opinions that you literally can get from thousands of other websites. I'm seriously tired of reading this stuff year over year rehashing the same BS over and over again, yet still seeing most software inevitably become bloated and harder to work with over time.

What I want to see is a formal theory of software design and by formal I mean mathematically formal. A axiomatic theory that tells me definitively the consequences of a certain design. An algorithm that when applied to a formal model produces a better model.

We have ways to formally prove a program 100% correct negating the need for unit tests, but do we have a formal theory on how to modularize code and design things so that they are future proof and remain flexible and understandable to future programmers? No we don't. Can we develop such a theory? I think it's possible.

<comment><USER42> So you're not talking about "formal methods"? [https://en.wikipedia.org/wiki/Formal_methods](https://en.wikipedia.org/wiki/Formal_methods)

The Applied Category Theory folks have some very interesting stuff, like Categorical Query Language.

[https://www.appliedcategorytheory.org/](https://www.appliedcategorytheory.org/)

[https://www.categoricaldata.net/](https://www.categoricaldata.net/)

But it sounds to me what you mean is more like if "Pattern Language" was symbolic and rigorous, eh?

<comment><USER41> Yes, this is exactly what I mean. Though I feel patterns can be formalized within thr framework of category theory.

<comment><USER42> Have you read "Introduction to Cybernetics" by Ashby?

(PDF available here: [http://pespmc1.vub.ac.be/ASHBBOOK.html](http://pespmc1.vub.ac.be/ASHBBOOK.html) )

Cybernetics might be the "missing link" for what you're talking about.

<comment><USER41> I didn't dive to deep into this so I could be wrong but this looks like control theory with elements of category theory.

I'm looking more for a theory of modules and relationships. Something that can formalize the ways we organize code.

<comment><USER42> From my POV control theory is rediscovering cybernetics, but yeah.

It sounds like CT is what you're after (to the extent that we have it at all yet...)

</comment></comment></comment></comment></comment></comment><comment><USER43> All very good points. Don't write code, solve the problem. For that, first understand the problem. Take time to reduce complexity, else you won't be able to evolve. Gather knowledge along th he way.

This all takes a bird-eye view and a long perspective, very unlike quarter- results-driven development.

</comment><comment><USER26> This is great. So many quotable quotes. If only we could make it required reading for our clients!

This one struck me, because as soon as I read it I knew it was true yet had never considered it:

> Most people only give feedback once. If you start by launching to a large > audience, everyone will give you the same obvious feedback and you'll have > nowhere to go from there.

I've been on both sides of that fence and it rings true.

<comment><USER44> Anonymous feedback (like really anonymous) is the answer. people can't give real feedback and be nice at the same time.

<comment><USER26> I think feedback fatigue is a real thing though. The comment about only leaving feedback once hit home to me. It's rare I bother reviewing something twice even if asked, and especially if my original round of feedback didn't seem to change anything (which I understand is totally reasonable in many cases, but still a little disappointing).

</comment></comment></comment><comment><USER45> Jesus Christ that's a well-written article. There's no fat to it. All signal, no noise.

</comment><comment><USER46> Write code. Not too much. Mostly test-covered.

<comment><USER47> I like this Pollan reference

</comment></comment><comment><USER48> >>> The hard limit to system complexity is not the quantity of engineering effort, but its quality.

This article is full of good ideas, an antidote to creeping corporate take over of software projects - make this required reading for software projects.

</comment><comment><USER49> The initial proposition of the article, that software is bad because it follows the lifecycle "gather requirements - write software - deliver it" is simply wrong. There are huge projects in specialized domains that are delivered on time and on budget and use this approach.

The problem is lack of knowledge. The successful projects mentioned above did not have a lack of knowledge, and so they were finished successfully.

When there is a lack of knowledge, then it makes sense to use the iterative approach...as knowledge is slowly gathered, the software gets improved. As with all things in life!

<comment><USER50> Yes, the lack of knowledge is definitively one of the issue.

But starting a "gather requirements - write software - deliver it" lifecycle because you are confident that you have all the knowledge is one as well.

</comment></comment><comment><USER51> I like the article, it gets to the point. I would, however, change this: " 3\. Hire the best engineers you can." to: "3\. Hire and work hard to keep the best managers and engineers you can." As they mention, accumulating knowledge is important. Keeping that knowledge around is therefore also important (and sometimes difficult). The best managers will know what technical debt is, how to handle pressure from higher-ups, and how to keep a team happy, healthy and productive.

</comment><comment><USER48> Years ago I wrote [http://oss4gov.org/manifesto](http://oss4gov.org/manifesto) saying that governments needed to not only embrace OSS but that it is the only moral option to take.

Now we have government digital systems leading the charge across most western countries, and we have excellent polemics like this. I am just so happy to see this level of insightful ness at top levels of government.

I am so glad they listened to me :-)

</comment><comment><USER52> > _Overall, good engineers are so much more effective not because they produce > a lot more code, but because the decisions they make save you from work you > did not know could be avoided._

This is spot on, and very much my experience (of the good engineers I've come across).

Kind of : management had planned extensive and painful testing of a component that turned out to be discarded entirely (not because of functionality reduction but because it was actually unecessary).

</comment><comment><USER53> Keep it simple software should be open source. Government software often has similar demands as other countries. Share and reuse.

Reusing good modules and software will make the software work.

Kiss engineering still works keep it simple stupid. Make it as simple as possible. Simple software and systems are easy to maintain and understand.

Use modules as these can be swapped out.

Use proven boring technology such as SQL and JSON. Boring tech has been tried by others and generally works well.

<comment><USER54> >Government software often has similar demands as other countries.

What makes you think so?

</comment></comment><comment><USER55> >The better your engineers, the bigger your system can get before it collapses under its own weight. This is why the most successful tech companies insist on the best talent despite their massive size.

Translation: the successful tech companies have so much poorly documented legacy enterprise spaghetti code and tooling that they need the best talent they can get just to make sense of it and maintain it

<comment><USER56> Alternate translation: Bad devs are worse than no devs and all your competent devs will spend most of their time dealing with the former's crappy code until they quit. (my code is of course perfect and free of all technical debt)

</comment></comment><comment><USER57> The article lists the characteristics of a good engineer:

      * has a better grasp of existing software they can reuse
      * (has) a better grasp of engineering tools, automating away most of the routine aspects of their own job
      * design systems that are more robust and easier to understand by others
      * the decisions they make save you from work you did not know could be avoided

I obviously concord with the analysis (not sure about the 10X myth). It also states that:

      * Google, Facebook, Amazon, Netflix, and Microsoft all run a dizzying number of the largest technology systems in the world, yet, they famously have some of the most selective interview processes

This sounds a bit like a paradox to me. Given the current state of "selective interview processes" (algo riddles, whiteboard coding, etc.), none of the above traits can be easily evaluated in a candidate during an interview. On the other hand, these companies _do hire_ stellar engineers: the technological supremacy of FAANG is irrefutable.

<comment><USER58> Former Googler here.

Google views picking new engineers like picking quality construction metals. In the end, the machine melts you down and hammers you into a pristine cog.

</comment></comment><comment><USER59> Excellent article, well grounded in the reality of software development. New developers and managers would benefit from understanding the practical points made here as early as possible.

I do think perhaps there is too much emphasis on reuse and particularly cloud services. Ironically, this is partly for the reasons given elsewhere in the article. If you rely on outsourcing important things, you also naturally outsource the deep understanding of those important things, which can leave you vulnerable to problems you didn't anticipate. Also, any integration is a source of technical debt, so dependencies on external resources can be more fragile than they appear, and if something you rely on changes or even disappears then that is a new kind of potentially very serious problem that you didn't have to deal with before. Obviously I'm not advocating building every last thing in-house in every case, but deciding when to build in-house and when to bring something in can be more difficult than the article here might suggest.

</comment><comment><USER60> > Software has characteristics that make it hard to build with traditional > management techniques

Perhaps some software development techniques would work though...

> The main value in software is not the code produced, but the knowledge > accumulated by the people who produced it.

Those people go on to work on other things or for other organization. So, while that statement might have some truth to it, it's still the case that the code has to be useful, robust, and able to impart knowledge to those who read it (and the documentation).

> Start as Simple as Possible

That's a solid suggestion to many (most?) software projects; but - if your goal is to write something comprehensive and flexible, you may need to replace it with:

"Start by simplifying your implementation objectives as much as possible"

and it's even sometimes the case that you want to sort of do the opposite, i.e.

"Start as complex as possible, leading you to immediately avoid the complex specifics in favor of a powerful generalization, which is simpler".

<comment><USER61> > > Software has characteristics that make it hard to build with traditional > management techniques

> Perhaps some software development techniques would work though...

As you go up the management chain, you usually run into some layer where people are traditional managers, who want to run a software project like a traditional project. And behold, you're at this problem. Saying "software development techniques would work" is useless unless you can get those managers to change. And when you get them to change, the problem moves up one layer.

</comment></comment><comment><USER62> One additional principle is this:

When faced with a standard solution, use a standard component if you can. If you can't use a standard component, build a standard component. Keep your components simple, well-understood, and easy to maintain.

</comment><comment><USER63> Hiring the best engineers, technically-wise, is a good thing but it's not enough. In my experience it's better to hire somebody with a previous experience in the domain. Somebody that already built something similar or related. Those engineers will ask the right questions, make the customer think about the system in the right way, not lose time on worthless details. Even if the implementation is not shiny it will be working. It beats shiny but misguided. And if you can find great engineers with great skills, that's even better.

</comment><comment><USER58> After a brief visit and further readings, I've learned to admire Singapore's 1st world transformation. I'm a absolutely in awe that a government body can produce such a high quality article.

</comment><comment><USER64> > The root cause of bad software has less to do with specific engineering > choices, and more to do with how development projects are managed.

...While I do agree that "project-management" is important, I think the tools we are using today are really underpowered to deal with complexity/human-error - Which is the bigger problem IMO.

</comment><comment><USER65> > The main value in software is not the code produced, but the knowledge > accumulated by the people who produced it

The problem is most CEOs see the binary as the asset, not the knowledge gained. I've tried to explain this concept to multiple startup CEOs, who hire outside development firms, for which it rarely works out for them.

</comment><comment><USER66> > Software has characteristics that make it hard to build with traditional > management techniques; effective development requires a different, more > exploratory and iterative approach.

Or the management techniques considered "traditional" are overlooking a century of iterative development outside of software. See Deming.

</comment><comment><USER67> The author is very cavalier about open source licenses - they seem to be implying you can just use open source code whenever you want, even for closed- source, proprietary applications. Whether or not that is true depends on the licenses involved.

</comment><comment><USER68> #1 web rule: Don't require JS to read an article.

This site is an empty page without JS.

<comment><USER69> Strangely, all the HTML elements are there but the opacity of <body> stays at 0 without JavaScript.

</comment></comment><comment><USER70> The lead image features two laptops and a desktop. And three hardcopies of code. All with a light-on-dark color scheme. I'd wager they had a bit of fun taking this programmery photo.

</comment><comment><USER71> "Software Is about Developing Knowledge More than Writing Code"

This is also the real problem with vendor lock-in.

You are more often locked in by the knowledge of your employees than by your tech stack.

</comment><comment><USER72> What excellent article. It summarizes things that sometimes need decades to learn by ourselves. Reading books and good articles like this give us a shortcut for such wisdom.

</comment><comment><USER73> `There is no such thing as platonically good engineering` +1

</comment><comment><USER74> > 3\. Hire the best engineers you can.

What is the definition of "best engineers"? Those with extensive experience? those who follow design patterns and coding standards religiously? those who solve algorithms on a whiteboard? I would like to see if there is a definition for this.

I would say build the right culture (collaborative, always learning from mistakes and revise decisions and no blame or pointing fingers).

You can get a bunch of great _coders_ /engineers _who follow code standards, break down codes to zillions of functions/methods ... etc_ but will fail to work together and conflicts will raise quickly.

</comment><comment><USER75> This reads like conventional wisdom. No one is going to argue against "Hire the Best Engineers You Can" and "Start as Simple as Possible".

<comment><USER76> If only.

Industry this days is more about headcount than quality itself. Why hire two good engineers when you can have three mediocre ones for the same price?

On simplicity, common wisdom these days dictate that we should use bloated kitchen-sink backend MVC frameworks that generate dozens of directories after `init`, because supposedly nobody knows how to use routers. Frontend compiler pipelines are orders of magnitude more complex than the reactive frameworks themselves, because IE11. And even deployment now requires a different team or expensive paid services from the get go. We're definitely not seeking simplicity.

The second point is also something that most developers and managers would balk at: "To build good software, you need to first build bad software, then actively seek out problems to improve on your solution". Very similar to the Fred Brooks "throw one away" advice that no one ever followed.

</comment></comment><comment><USER77> I'd start from here [http://0.30000000000000004.com/](http://0.30000000000000004.com/)

</comment><comment><USER40> This is beautiful, I feel like I should memorize the entire thing.

</comment><comment><USER78> Impressive that this comes from a Civil service college.

</comment><comment><USER79> Now, how do I convince my management this is a problem?

</comment><comment><USER80> I think only hire the best is wrong

</comment><comment><USER81> Except for the 10x myth a fairly good article.

<comment><USER40> I disagree with that, if you describe it as stated in the article: "Overall, good engineers are so much more effective not because they produce a lot more code, but because the decisions they make save you from work you did not know could be avoided."

I've seen plenty of poor decisions that cause 10x the work, and end up with something 10x less maintainable.

<comment><USER81> I'm not disagreeing with the idea that there is variability in developer productivity. However, quantifying the most productive engineers by throwing around a specific random factor such as "10x" is rather idiotic.

You have entire blog posts by Steve McConnell of Code Complete fame devoted to defending the 10x claim by citing 20 to 50 year old research that shows 5x to 20x differences across certain dimensions and then him falling back to the 10x thing. Not one single sentence where he is being self aware enough to spell out the most likely reason for "10x" being so prominent: 10 is the base of the decimal system and as such psychologically attractive to use.

> Both Steve Jobs and Mark Zuckerberg have said that the best engineers are at > least 10 times more productive than an average engineer.

I know I'm venturing into ad hominem territory with this, but first of all: Steve Jobs wasn't a programmer. Mark Zuckerberg, well does he even qualify as a programmer nowadays? How well can he quantify programmer productivity? His decision to use PHP led Facebook to create HHVM and Hack. Is this the 10x developer way?

Anyways, the question to me is: Is it possible for average software engineers to write good software?

<comment><USER82> Perhaps "10X engineer" is just an easier thing to say than "5 to 20X engineer as described by this paper." Perfect numerical accuracy is not needed to make the point that there is a lot more variance in the productivity of engineers than there is with most jobs.

If someone suggests you focus on the 20% of customers who make 80% of your revenue, and you run the numbers and find a 75-25 distribution, should you call the person making the suggestion an idiot?

</comment></comment></comment></comment><comment><USER83> > Surprisingly, the root cause of bad software has less to do with specific > engineering choices, and more to do with how development projects are > managed. The worst software projects often proceed in a very particular way:

> The project owners start out wanting to build a specific solution and never > explicitly identify the problem they are trying to solve. ...

At this point, it looks like the article will reveal specific techniques for problem identification. Instead, it wraps this nugget in a lasagna of other stuff (hiring good developers, software reuse, the value of iteration), without explicitly keeping the main idea in the spotlight at all times.

Take the first sentences in the section "Reusing Software Lets You Build Good Things Quickly":

> Software is easy to copy. At a mechanical level, lines of code can literally > be copied and pasted onto another computer. ...

By the time the author has finished talking about open source and cloud computing, it's easy to have forgotten the promise the article seemed to make: teaching you how to identify the problem to be solved.

The section returns to this idea in the last paragraph, but by then it's too little too late:

> You cannot make technological progress if all your time is spent on > rebuilding existing technology. Software engineering is about building > automated systems, and one of the first things that gets automated away is > routine software engineering work. The point is to understand what the right > systems to reuse are, how to customise them to fit your unique requirements, > and fixing novel problems discovered along the way.

I would re-write this section by starting with a sentence that clearly states the goal - something like:

"Paradoxically, identifying a software problem will require your team to write software. But the software you write early will be quite different than the software you put into production. Your first software iteration will be a guess, more or less, designed to elicit feedback from your target audience and will deliberately built in great haste. Later iterations will solve the real problem you uncover and will emphasize quality. Still, you cannot make technical progress, particularly at the crucial fact-gathering stage, if all your time is spent on rebuilding existing technology. Fortunately, there are two powerful sources of prefabricated software you can draw from: open source and cloud computing."

The remainder of the section would then give specific examples, and skip the weirdly simpleminded introductory talk.

More problematically, though, the article lacks an overview of the process the author will be teaching. Its lack makes the remaining discussion even harder to follow. I'll admit to guessing the author's intent for the section above.

Unfortunately, the entire article is structured so as to prevent the main message ("find the problem first") from getting through. As a result, the reader is left without any specific action to take today. S/he might feel good after having read the article, but won't be able to turn the author's clear experience with the topic into something that prevents more bad software from entering the world.

</comment><comment><USER84> nice one

</comment><comment><USER85> How many more decades are we going to have to spend learning this lesson before we learn it?

<comment><USER86> There's a saying in most other fields of engineering (civil, chemical, mechanical, etc.): "Regulations are written in blood." A whole lot of bridges collapsed and a whole lot of people died before strong requirements were put in place.

It seems we are on the path to repeat history with software engineering, what with how software and the internet is being developed with such little regard for public safety and long term consequences.

Unfortunately, it appears that the "free love" phase of software engineering is coming to an end, as society now relies more and more on software and major tech players for life and safety. It's starting to get real for software engineering.

Luckily, other engineering fields have been here before, so this sort of transition shouldn't be anything new.

Relevant Tom Scott video: [https://www.youtube.com/watch?v=LZM9YdO_QKk](https://www.youtube.com/watch?v=LZM9YdO_QKk)

<comment><USER87> > It seems we are on the path to repeat history with software engineering, > what with how software and the internet is being developed with such little > regard for public safety and long term consequences.

> Unfortunately, it appears that the "free love" phase of software engineering > is coming to an end, as society now relies more and more on software and > major tech players for life and safety. It's starting to get real for > software engineering.

Software will always be a spread of reliability requirements, from pacemakers on one side to excel reports on the other. Part of being a responsible user is choosing software with the right balance of economics and reliability for the job.

</comment></comment></comment><comment><USER88> "2\. Seek out problems and iterate;"

This is bad advice. It's like saying "go into a bar and start picking up fights".

If some part of the software has problems, runs slow or has bugs but _nobody_ is complaining, then there's no problem. Why waste time improving it?

Almost 100% of the time when you solve a problem you just create new problems of different kind in turn.

Be lazy. The less code you write the better off you are.

<comment><USER59> _If some part of the software has problems, runs slow or has bugs but nobody is complaining, then there 's no problem._

This depends very much on context. To pick an extreme example, if you're writing the control software for a nuclear weapon and you know you have a bug that might cause it to activate unintentionally if you eat a banana while it's raining outside, I think we can reasonably agree that this is still a problem even if so far you have always chosen an apple for lunch on wet days.

<comment><USER88> In your example youd hope some stakeholder to complain and raise an issue.</comment></comment></comment>