The Insecurity of OpenBSD (2010) - <AUTHOR> https://allthatiswrong.wordpress.com/2010/01/20/the-insecurity-of-openbsd/ <comment><USER2> There are two different schools of thought to a secure operating system. One is to add features / layers to enhance security, such as with GRSecurity. The other is to ensure correctness, so that there are no bugs to exploit. OpenBSD takes the latter approach, as they're of the opinion that additional features and layers result in more complexity with increased probability of bugs and/or misuse. Also, it's a misconception that OpenBSD's primary focus is security; the primary focus is in fact on correctness, with security being an important but auxiliary benefit of that.

The article posted makes the argument that the first approach is the only true approach to a secure operating system, and that OpenBSD is thus insecure by definition. There's not enough data / samples of secure operating systems to make an intellectually honest assertion that one approach is definitively superior, but OpenBSD likely has the best security track record for an operating system in real-world scenarios.

point of reference: years of following the OpenBSD mailing lists

<comment><USER3> _There 's not enough data / samples of secure operating systems to make an intellectually honest assertion that one approach is definitively superior, but OpenBSD likely has the best security track record for an operating system in real-world scenarios._

While auditing is definitely a good idea, the latter approach only works when you only run services and applications that are part of OpenBSD. Since that is not the case for most realistic usage scenarios, OpenBSD gives you very little, while the former approach (think e.g. SELinux) can help you with isolating applications that you trust, but not as much as, say OpenBSD.

<comment><USER4> The comment you are replying to does not mention auditing.

<comment><USER5> It's implied. The comment mentions code correctness and, per the article, this comes from "the extensive code auditing performed on the base system".

</comment></comment></comment></comment><comment><USER6> Aren't all of these solved by pledge ?

Also his counter-argument on secure-levels is not convincing.

 _" Securelevels are an interesting concepts and they do help with security somewhat. Securelevels can only be increased not decreased on a running system. The higher levels prevent writing to /dev/mem and /dev/kmem, removing file immutable flags, loading kernel modules and changing pf rules. These all help to restrict what an attacker can do, but do absolutely nothing to prevent reading or changing database records, obtaining user info, running malicious programs etc."_

Securelevels are what they are, and should be used accordingly. This is just bad judgement of the technology. Please don't assume something does more than what it actually does, this is not a good practice, please read the documentation and/or talk with the community, securelevels are well documented. It is _very bad_ to expect securelevels to protect your database from being changed or your code from being abused and use that as an argument against the OS.

(edited, small typos: added an "it" and changed SO to OS)

<comment><USER7> pledge and, say, SELinux solve two different problems:

Pledge stops trusted software from being exploited. I trust Apache (I run it as root) but the authors can't guarantee that it can't get exploited, so it's protected by pledge. If I didn't trust Apache, I wouldn't trust that it pledged anything.

SELinux secures un-trusted code. So if I download a program off the internet and run it as nobody, it shouldn't exploit something to get root. This is useful when downloading random closed-source programs (such as on Android).

Unfortunately, Linux relies on SELinux to solve problem #1 (how do I know which files does Apache really need, and which won't help security exploits), which is why it gets such a terrible reputation.

<comment><USER8> I've had to make AppArmor profiles in the past and they can get a bit painful. I've never had to configure SELinux so I don't know how much better/worse it is.

That being said, I think Docker and other container systems are going to remove the need for these security profile systems.

Yes, if there is exploitable code in the container, it can read/write to other things in the container. But so long as you keep the docker daemon up to date on its security patches, application exploits shouldn't be able to leak beyond the container, with the exception of volumes and linked containers/ports.

<comment><USER9> The typical configuration I see in /etc/selinux/config is:

       SELINUX=disabled

<comment><USER10> Its definitely one of the things that shows up when you call about a lot of vendor's software support. I find it rather annoying that a company tells you to buy a specific Linux distribution then pulls exactly what you say.

Thinking about it (maybe because I am watching a update go as I type), what the heck does this say about how we program? I guess in some ways its why I like the idea of pledge. It makes me think better of the programmer because they have put some thought into their program. I'm not sure what I should think when I see SELINUX=disabled as a possible solution.

<comment><USER7> >Thinking about it (maybe because I am watching a update go as I type), what the heck does this say about how we program?

SELinux is not complex because we program in complex ways, but because we don't know the target program.

For example, (again, nothing against Apache but...) if I want to secure Apache, there's no way for me (as a sysadmin) to tell exactly which files, exactly which syscalls, and exactly which libs does it need to function, and there's no way for me to stay on top of it.

And the same applies to any other complicated software. How to I lock down X? Firefox?

Really, the beauty of a "pledge" like system is that the programmer/PM of the code (which he should understand) should know how to lock it down

<comment><USER11> You can do that in a SELinux-like system. There's been tools and policy languages for it. Even better, if the language is high-level & declarative, then it can be used to generate security policies on many different OS's and mechanisms instead of just SELinux or pledge.

<comment><USER10> Which projects are doing this with SELinux?

<comment><USER11> I haven't followed SELinux in a while. I recall Tresys made tools with dialog boxes to make it about as easy as Windows firewalls. A quick Google leads me to Lobster being an example of what I was thinking of:

[https://selinuxproject.org/files/2008_selinux_developer_summ...](https://selinuxproject.org/files/2008_selinux_developer_summit/2008_summit_white.pdf)

<comment><USER10> Given how fast pledge was able to be put into production, I think it was the right move. I get SELinux can and has (2008) had tools for this, but it really doesn't seem to have caught on.

<comment><USER11> SELinux was a demonstrator of Type Enforcement by Mitre. It got put into production because why not. There's simpler schemes out there for MAC even on Linux. I'd have recommended OpenBSD clean-slate something like them.

I do like pledge, though. I promoted API reduction a long time. Even deleting the code in kernel for appliances a la Poly2 project. Only so much can be gained with it, though.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I would simply say OpenBSD runs.

It runs with it's hardening mechanisms always on. Ports are considered in whether a new mechanism or aggressiveness of an existing mechanism is suitable. The idea is to have a platform where good quality software runs and bad quality software crashes. See the evolution of the default malloc.conf options or more recently the explicit enforcement of W^X.

It's true OpenBSD was not founded with the intention of being "the secure OS" but the ethos of designing secure software is very much something that is core to the developers today.

Others have already pointed out cons as well as pros of MAC style systems which this post laments the lack of in OpenBSD. I'll simply say I prefer the approach of pledge. Yes you need to build it into the application but I think that kind of upfront explicitness is better than a policy system and I hope other platforms adopt it.

</comment><comment><USER13> _OpenBSD by itself contains a feature set similar in comparison to the GRSecurity patch for Linux without the ACL or RBAC implementation. GRSecurity and the Openwall project actually pioneered many of the protections that occurred later in OpenBSD such as Executable Space Protection, chroot restrictions, PID randomization and attempts to prevent race conditions_

On the one hand: I think OpenBSD _did_ pioneer some of these things (PID randomization --- randomization of all monotonically increasing IDs, really --- was something TdR was doing in the 1990s). Privsep daemons might be another example.

On the other hand, I'm not sure I buy that OpenBSD has a feature set comparable to GRsecurity without the ACL/RBAC stuff.

<comment><USER14> Just to try and explain where I feel the OpenBSD project is coming from... their philosophy is a little different.

At the very beginning OpenBSD was not about security per se, but about simplicity of implementation and stark, clean code. It turns out this results in a rather secure system (go figure).

It was only after the fork from NetBSD that OpenBSD gained a real security orientation... I think there was some client who wanted a secure OS, and they helped fund the audit of OpenBSD's codebase. Yet even today OpenBSD's philosophy is better explained as a focus on stark simplicity.

Coming from that perspective, many of their decisions make sense. ACLs are a terribly complex mechanism that would require a lot of new code added to the kernel. Any new code is a deadweight that needs to be tested, audited, maintained, etc. across a large number of platforms, not just x86 and ARM.

OpenBSD (historically) has no hypervisor for the same reason. It's an incredibly complicated feature that would be difficult to implement securely and succinctly. As Theo put it (in characteristically brusque style):

>x86 virtualization is about basically placing another nearly full kernel, full of new bugs, on top of a nasty x86 architecture which barely has correct page protection. Then running your operating system on the other side of this brand new pile of shit.

>You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.

That said, OpenBSD has pioneered a lot of security features. They created sudo in 1997, recently _replaced_ sudo with a simpler program `doas`. OpenBSD was the first mainstream OS to have ASLR turned on by default. And their new privilege-drop feature `pledge` looks great.

In the end, they're a smaller project with less manpower so it's impressive they achieve what they do. Obviously auditing the entire ports tree is going to be out of scope... auditing Firefox alone would probably be a multi-million dollar project.

<comment><USER13> Yeah, I was there at the beginning of the OpenBSD audit (there, as in, in Canada, at times in Theo's basement), and for a few years was the person writing the advisories (I'm particularly proud of rfork and the I/O signal advisory).

I don't dispute that OpenBSD is good at the small-bore Unix stuff. They are! I think OpenBSD is the undisputed master of small-bore Unix security. If you want something in userland priv-separated or some ID randomized, call the OpenBSD team.

The problem though is exactly what this article says it is: the code OpenBSD has custody over is a small fraction of the code users need to run, and OpenBSD's code just isn't up to the challenge of securing other people's code. Modern Linux security has taken the other road: part of the kernel's job is --- at least in GRsecurity-world --- to assume that you're running insecure C code, and still not lose your kernel to an attacker's persistence tools.

<comment><USER14> Did not mean to preach to the choir, hah!

I would argue that pledge goes a long way towards addressing the problems of insecure C code, although it's definitely not as fine-grained as ACLs.

</comment></comment></comment></comment><comment><USER15> Ports might be unaudited, but OpenBSD base has lots of useful stuff that many people don't know about. Like OpenBSD's own web server, LDAP server, IPSEC/VPN implementation, time server ...

All priv-sep'd, simplified, pledged

Also, these days the preferred MTA is OpenSMTP, and the preferred name server is NSD

</comment><comment><USER16> Ah, the age old promise of MACs: perfectly secure, as long as you don't think very hard about the fact that somebody has to administer the access control rules.

<comment><USER3> MACs come in many different flavors. E.g. AppArmor is very easy to administrate an provides security over running 'random' applications without MAC.

<comment><USER16> There's two separate issues here, both of which apply:

1\. A knob that can't be configured can't be misconfigured (IIRC Debian had a hole in its default selinux config for years that created a vulnerability selinux-less machines didn't have)

2\. (This was more what I meant) the promise that makes people like MACs is that you no longer need a single all-powerful root user to do everything (this bleeds into ACLs and capabilities, too), but keeps running into two icebergs:

2a. Somebody still has to administer those controls, and so can effectively become that all-powerful root, and

2b. The only stuff I actually care about is the actual data that the application being sandboxed can access to begin with.

This gives us the particularly pointless exercise in which somebody who steals my laptop can impersonate me on gmail and social media, empty my bank account, and delete all my github repos, but can't change the laptop's time zone. The system files that I can download for free at any time are protected, but the manuscript for my novel is not.

<comment><USER17> > capabilities

Well. There just aren't many useful caps to begin with. For example, consider backups. You'd want the backup system to have full read access to everything. Well. You can't do that with capabilities. You only get DAC_OVERRIDE, which is one of the many root-equivalent caps in Linux. Probably half of them are.

<comment><USER18> Even better, in my opinion, is that many of the caps that you would use to sandbox an app (like PID and FS namespaces) require you to have the sysadmin capability set on the process.

Great. You need to give a process more or less root so that it can deny itself privileges.

<comment><USER17> The idea is probably that the service manager ( _systemd_ ) manages the NSs for the applications contained therein. _systemd_ of course doesn't have to check it's own privileges.

<comment><USER16> GNU PIES, however, does check its own privileges. I have no idea why it didn't get a harder look during the init wars. I get that Shepherd "isn't there yet", but IMO Pies is.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> > There is no good reason to avoid implementing extended access controls when > the greater security and control they provide is irrefutable.

Disagreed. MAC is a crutch and a hack, not a solution. We should prefer not to rely on half-assed, overcomplicated, and formally unverifiable palliative measures.

If you want security, you fundamentally must use secure software. No amount of sandboxing or access control wrapped around a vulnerable garbage heap is going to stop the garbage heap from getting hacked in the first place.

<comment><USER11> There were numerous examples of MAC, esp SELinux, stopping vulnerabilities. There's also plenty of examples of bugs and vulnerabilities in software with OpenBSD even fixing bugs regularly. The claim that one should reduce bugs and have damage limitation to reduce overall risk is rock solid with proof in the field.

The refrain of OpenBSD supporters is, "It's not necessary and won't help..."

The reality in field deployments, "It was and did."

QED.

<comment><USER19> > There's also plenty of examples of bugs and vulnerabilities in software with > OpenBSD even fixing bugs regularly.

As the OpenBSD devs will tell you, there have only been two known vulnerabilities in default OpenBSD installs in the history of OpenBSD. This strongly suggests that having decent software quality is an effective way to prevent vulnerabilities from existing in the first place.

> The refrain of OpenBSD supporters is, "It's not necessary and won't help..."

No, the refrain is "it's better to spend effort curing the disease than treating the symptoms". If we put half the effort we put into palliative defense measures into formal verification or doing things right the first time, I suspect it would be drastically more effective.

<comment><USER11> "As the OpenBSD devs will tell you, there have only been two known vulnerabilities in default OpenBSD installs in the history of OpenBSD. This strongly suggests that having decent software quality is an effective way to prevent vulnerabilities from existing in the first place."

I addressed that myth in my main comment. See here:

[http://pastebin.com/6cW3FyJE](http://pastebin.com/6cW3FyJE)

They just assess and count differently than most folks. ;)

" If we put half the effort we put into palliative defense measures into formal verification or doing things right the first time, I suspect it would be drastically more effective."

I agree being on side of high-assurance security. OpenBSD doesn't so I don't know your statement supports what they do. They systematically avoid formal verification like what you described, safer languages, coding styles for static analysis, etc. It's people from my side of things promoting or building on those. They just hunt bugs in C code by eye and hope their probabilistic mitigations always work. Totally different.

</comment></comment></comment></comment><comment><USER20> What mission critical systems is OpenBSD running _anywhere_? I'm talking about the OS itself, not one of the pieces of software developed as part of the project and ported to other platforms(eg openssh). Not trolling, but genuinely curious where OpenBSD is running outside of our basements.

<comment><USER11> Genua on milcron's list is probably most significant. They are a defense contractor that sells firewalls, VPN's, data diodes, etc for business and government (including military). You can bet significant applications are protected with them.

This is the kind of role most of its users put it in. Others just use it like any other server OS with better reputation for reliability and security. Less patching... a time-consuming activity... at the least. Last one was similar reason for adoption of Argus Pitbull and SELinux w/ Tresys tools in their niches. Less worries when malware is going viral on the Internet and in media. ;)

</comment></comment><comment><USER21> Out of curiosity, anyone ever worked with an hardened OpenBSD or Linux + GRSec server farm? How do you handle updates? It must a sysadmin deployment nightmare.

<comment><USER14> I don't operate a server farm but OpenBSD updates are pretty easy. Just do `pkg_add -Uu`.

If your dependencies are minimal, keeping OpenBSD up-to-date is actually pretty relaxing. Don't need to install security patches all the time as with Linux.

<comment><USER22> pkg_add -Uu won't address a single update under [https://www.openbsd.org/errata.html](https://www.openbsd.org/errata.html) , i.e. the very most important ones.

Also as far as I know pkg_add still doesn't know about restarting services, so vulnerable processes will be left running.

</comment></comment></comment><comment><USER23> This guys makes some good points, but the OpenBSD guys have been able to improve the security of OBSD with code quality.

It's also not completely accurate to say that OpenBSD hasn't contributed anything to security. They have. Maybe it was more true in 2010, but it's certainly not the case now.

I have found lots of Unix people (devs and admins) don't really seem to care much for MAC. I don't really understand it, but its true.

<comment><USER9> Admins in particular like to (a) understand how things are configured and (b) avoid changing things (creates work and potential update conflicts).

MAC configuration is complicated, tedious, error prone, and seems to bite you in the ass with every update. All things that make a bad day for an admin.

<comment><USER23> I'm an admin. Have been for many, many, moons. I've found that when things seen to behave that way it's because I have failed to sufficiently adjust my understanding of them. Often, we can be guilty of expecting that everything should behave like the least common denominator set of features between all host systems. Often, there is great value in finding "the way" that a particular system is trying to exemplify instead of forcing it to match my own.

To be fair, that least common denominator set of features can be very powerful and capable. It's not a slam, I'm just saying that finding out the in depth method of operation for a given system can often be rewarding.

</comment></comment></comment><comment><USER22> Much to like, but one point is somewhat outdated and another is missing.

1\. The lack of an AppArmor-type MAC implementation is somewhat outdated since Theo rolled his own with "pledge". I'm not a huge fan of how a cabal of Theo plus one or two guys basically hacks out something of new cloth on a whim to solve a problem that's been done many times before, on the arrogant supposition they're doing it better than anyone else has ever done, and then promptly stuck it into production. This has happened many times before. Certainly some results have been good, but case in point, "doas". You upgrade one point release and suddenly sudo is not there any more, you have a new tool lacking basic features with completely different semantics. Yes, you can install sudo as a package if you want. Yes, maybe doas is smaller and easier to audit. But was sudo really a problem? Sure the fanboys will love it, but most normal UNIX people are probably not going to appreciate something like sudo just going away. It lacks little "features" like credential caching, which I am sure the fanboys will tell you is bad to begin with, but which most of the rest of us will find a pain in the ass. This sort of thing happens with OpenBSD semiregularly.

Of course, many of these homegrown solutions are produced after years of Theo & cabal insisting that there was no need for it and it was wrongheaded. There's "pledge," but then there's little things like full-disk encryption, which is basically a requirement for use on mobile, but which OpenBSD never had any use for, until it did, and came out with its own homegrown thing (which still doesn't work that great, especially when upgrading).

*And since so many others have brought up pledge, it's not really a solution on the same scale since you have to build the pledges into the application, there's not an easy system for imposing pledges on an application externally. This makes maintenance and adoption much harder, basically nonexistent for most of the package tree.

2\. The big reason OpenBSD is insecure, is its lack of any meaningful update mechanism to their supposedly rock-solid secure base system. Literally the official way to do security updates is to monitor OpenBSD's website, download and apply patches by hand to a source install, rebuild, and run a series of listed commands by hand. If you want to automate this further, you are on your own. It's been this way forever. It's craziness, and it a big reason that OpenBSD is basically not an option for production in many settings.

Upgrading to new releases is a similar deal. The homegrown sysmerge hack has made this slightly less awful, but manual hackery is still required, unreliable, can wipe out customization, and doing a clean reinstall is still urged as the best path in many cases.

<comment><USER24> > It lacks little "features" like credential caching

This is wrong, see the 'persist' keyword. It was also implemented as a kernel assisted feature, rather than filesystem tickets.

[http://www.tedunangst.com/flak/post/doas- mastery](http://www.tedunangst.com/flak/post/doas-mastery)

[http://marc.info/?l=openbsd- cvs&m=147283992915418&w=2](http://marc.info/?l=openbsd- cvs&m=147283992915418&w=2)

<comment><USER22> For some reason my post was deleted, but to repeat, you are pointing to a recent commit made to CVS.

</comment></comment></comment><comment><USER11> Interesting for this post to turn up as I was drafting and getting review for my own rebuttal to OpenBSD propaganda on Schneier's forum. I couldn't find it as I needed it. (shrugs) Quick attempt at redoing it here.

Started with this on Lobste.rs forum that's popular for OpenBSD developers:

[https://home.nuug.no/~peter/blug2016/img1.html](https://home.nuug.no/~peter/blug2016/img1.html)

Similar to some of the BS allthatiswrong counters. I'm just focusing on a few as it will already be a long writeup with many examples from history on where secure coding and UNIX really came from. In this Pastebin:

[http://pastebin.com/6cW3FyJE](http://pastebin.com/6cW3FyJE)</comment>


LastPass – users can host their data in Europe - <AUTHOR> https://lastpass.com/upgrade.php?ver=3.1.0&lastver=3.0.22&type=cr&upgrade=3.0.22&lang=en-GB

<comment><USER2> Presumably this is marketed to get around data sovereignty / Patriot Act issues. Sadly, the fact that LastPass is a US company, trumps these protection measures.

</comment><comment><USER3> Sure, but ONLY for premium users. Important distinction as everyone I know who uses this application, uses the free version.</comment>


Toward Go 2 - <AUTHOR> https://blog.golang.org/toward-go2 <comment><USER2>
        > I can't answer a design question like whether to support
        > generic methods, which is to say methods that are
        > parameterized separately from the receiver.

I work on the Dart language. Dart was initially designed with generic classes but not generic methods. Even at the time, some people on the team felt Dart should have had both.

We proceeded that way for several years. It was annoying, but tolerable because of Dart's optional type system -- you can sneak around the type checker really easily anyway, so in most cases you can just use "dynamic" instead of a generic method and get your code to run. Of course, it won't be type safe, but it will at least mostly do what you want.

When we later moved to a sound static type system, generic methods were a key part of that. Even though end users don't define their own generic methods very often, they use them _all_ the time. Critical common core library methods like Iterable.map() are generic methods and need to be in order to be safely, precisely typed.

This is partially because functional-styled code is fairly idiomatic on Dart. You see lots of higher-order methods for things like manipulating sequences. Go has lambdas, but stylistically tends to be more imperative, so I'm not sure if they'll feel the same pressure.

I do think if you add generic types without generic methods, you will run into their lack. Methods are how you abstract over and reuse behavior. If you have generic methods without generic classes, you lose the ability to abstract over operations that happen to use generic classes.

A simple example is a constructor function. If you define a generic class that needs some kind of initialization (discouraged in Go, but it still happens), you really need that constructor to be generic too.

<comment><USER3> I edited a book on Dart a few years ago. It's a shame the language hasn't gotten much traction outside of Google, as I enjoyed learning and using it.

<comment><USER4> I think it'll see more traction with Flutter and the like over the next year

</comment></comment></comment><comment><USER5> I should send this to rsc, but it's fairly easy to find examples where the lack of generics caused an opportunity cost.

(1) I started porting our high-performance, concurrent cuckoo hashing code to Go about 4 years ago. I quit. You can probably guess why from the comments at the top of the file about boxing things with interface{}. It just got slow and gross, to the point where libcuckoo-go was slower and more bloated than the integrated map type, just because of all the boxing: [https://github.com/efficient/go- cuckoo/blob/master/cuckoo.go](https://github.com/efficient/go- cuckoo/blob/master/cuckoo.go)

(my research group created libcuckoo.)

Go 1.9 offers a native concurrent map type, four years after we looked at getting libcuckoo on go -- because fundamental containers like this really benefit from being type-safe and fast.

(2) I chose to very tightly restrict the initial set of operations we initially accepted into the TensorFlow Go API because there was no non-gross way that I could see to manipulate Tensor types without adding the syntactic equivalent of the bigint library, where everything was Tensor.This(a, b), and Tensor.That(z, q). [https://github.com/tensorflow/tensorflow/pull/1237](https://github.com/tensorflow/tensorflow/pull/1237) and [https://github.com/tensorflow/tensorflow/pull/1771](https://github.com/tensorflow/tensorflow/pull/1771)

I love go, but the lack of generics simply causes me to look elsewhere for certain large classes of development and research. We need them.

<comment><USER6> please do send that to rsc

<comment><USER7> Care to explain?

<comment><USER8> The blog post was written by Russ Cox, aka rsc on HN. The person you replied to suggested sending the feedback to rsc, like the blog post requested.

</comment></comment></comment></comment><comment><USER9> The paragraph I was looking for is this:

> For example, I've been examining generics recently, but I don't have in my > mind a clear picture of the detailed, concrete problems that Go users need > generics to solve. As a result, I can't answer a design question like > whether to support generic methods, which is to say methods that are > parameterized separately from the receiver. If we had a large set of real- > world use cases, we could begin to answer a question like this by examining > the significant ones.

This is a much more nuanced position than the Go team has expressed in the past, which amounted to "fuck generics," but it puts the onus on the community to come up with a set of scenarios where generics could solve significant issues. I wonder if Go's historical antipathy towards this feature has driven away most of the people who would want it, or if there is still enough latent desire for generics that serious Go users will be able to produce the necessary mountain of real-world use cases to get something going here.

<comment><USER10> Several members of the Go team have invested significant effort in studying generics and designing proposals, since before Go 1.0. For example, Ian Lance Taylor published several of his previous efforts, which had shortcomings he was dissatisfied with.

I believe your impression of the Go team's position has been corrupted (likely unintentionally) by intermediaries.

<comment><USER11> Except they seem to focus only on Java, .NET and C++, forgetting that generics were initially implemented in CLU back in 1975, and there were several programming languages since those days that had some form of generics being designed into them.

<comment><USER12> Any thoughts on Eiffel's implementation of generics?

[https://en.wikipedia.org/wiki/Eiffel_(programming_language)#...](https://en.wikipedia.org/wiki/Eiffel_\(programming_language\)#Genericity)

<comment><USER11> I liked using it at the university back in the day, and didn't had to repeat code.

Eiffel is a very powerfull language with good tooling, just failed adoption, because licenses were too expensive and software industry still has issues valuing quality.

<comment><USER12> Interesting to hear your experience. I had read a good amount about Eiffel and also large parts of Bertrand Meyer's book Object-Oriented Software Construction (quite a thick book too, and which uses the syntax and semantics of Eiffel, or something close to it, IIRC [1]), some years ago. Had found the language very interesting and also was impressed by it (and by the book). Read some case studies / success stories about it, which were good too. He and his team seem to have taken a lot of pains and thought a lot about the design as well as implementation of the language, from both an academic and industrial use perspective - so it seemed to me.

There is also an interesting anecdote about use of Eiffel in the book, here:

[https://en.wikipedia.org/wiki/Object- Oriented_Software_Const...](https://en.wikipedia.org/wiki/Object- Oriented_Software_Construction#Notation)

That section shows the the 1st edition of the book did use Eiffel (as I said above).

<comment><USER11> With Eiffel tools you got the IDE with interactive development, including a VM for rapid prototyping. Then you would use the AOT compiler (via compilation to C) to produce shippable binaries.

So combining the easiness of interactive development with performance when it was time to ship the product.

It was also available before Java was a thing.

This is what I always kind of missed with Java and .NET, the previous generation of languages (Eiffel, Modula-3, Oberon(-2)) all had a mix of interactive development and AOT compilation for shipping release builds.

<comment><USER13> You seem to imply that Eiffel's solution was superior to Java's but nothing could be further from the truth.

The Eiffel compiler required four distinct sequential processes (note: not phases. I really mean that four programs needed to be run in sequence, and each of these four programs implemented who knows how many passes). Eiffel generated C++, huge executables, was incredibly slow (even with contract stuff turned off).

It was very hard to debug, with gdb routinely giving up on the complex .o generated. It was a very verbose language that implemented all known features under the sun, a bit like Ada and Scala.

Eiffel didn't take off for some very solid reasons.

<comment><USER14> > The Eiffel compiler required four distinct sequential processes (note: not > phases. I really mean that four programs needed to be run in sequence, and > each of these four programs implemented who knows how many passes). Eiffel > generated C++, huge executables, was incredibly slow (even with contract > stuff turned off).

This was not an inherent problem of the language, but one of the implementation.

I know this because I wrote a compiler for an Eiffel dialect for my Ph.D. thesis and there have been other Eiffel compilers without these shortcomings.

Also, EiffelStudio – which seems to be what you are talking about – generated C code, not C++ and had the option to generate an intermediate representation instead that could be mixed with compiled code.

> It was a very verbose language that implemented all known features under the > sun, a bit like Ada and Scala.

It was actually pretty minimalist, not at all like Ada or Scala.

Code was verbose because (1) programmers were encouraged to annotate methods with contracts and (2) exactly because it was minimalist and didn't have a lot of alternative ways of expressing the same functionality.

Edit: There are two hard parts about writing an Eiffel compiler.

1\. Getting the semantics of feature renaming during inheritance right. If you aren't careful, it's easy to introduce errors during this step.

2\. Doing separate compilation for mutually dependent classes. This is where much of the complexity of the EiffelStudio compiler comes from. Java solved the problem by simply not allowing separate compilation in those cases. If class A has a member of type B and B has a member of type A, they need to be compiled together.

Everything else that's involved in writing an Eiffel compiler does not require more than than what you learn in your typical compiler course. (Though, obviously, experience will get you better results.)

<comment><USER12> >Also, EiffelStudio – which seems to be what you are talking about – generated C code, not C++ and had the option to generate an intermediate representation instead that could be mixed with compiled code.

I think that must be what I was referring to by my mention about melting or freezing in a nearby comment. Didn't remember the technical details. Interesting feature. Wonder if other languages have something like that.

<comment><USER14> That's EiffelStudio's Melting Ice stuff [1]. One of Bertrand Meyer's trademarks is his penchant for coming up with funny names for such features.

(Edit: Reading over it again, I realize that this may be read as criticism. To clarify, while I disagree with Meyer on some things, this isn't about disagreement. He is a teacher at heart – in fact, a very good teacher – and you can see some of his habits for keeping students engaged sometimes carry over into his writing.)

You don't see this approach really anymore, because these days it's so cheap to just generated unoptimized native code if compilation speed is an issue.

Obviously, if you have a JIT-compiler, then something similar happens internally in that some methods will be compiled, while others will be left in IR form (and sometimes, methods will even revert to using their IR). This was technology from back when CPU speed was still expressed in MHz.

OCaml had and still has both a bytecode and a native code format (for similar reason), but they cannot be mixed. OCaml's bytecode still sees use for (1) the bootstrap process, which allows you to build OCaml from scratch, (2) to run OCaml code on oddball architectures that the native compiler doesn't support, (3) to run code in the timetraveling debugger, and (4) when you need compact rather than fast executables.

OCaml's bytecode interpreter makes use of the fact that OCaml is statically typed and therefore doesn't incur nearly as much of a performance penalty as bytecode interpreters for dynamically typed languages.

[1] [https://www.eiffel.org/doc/eiffelstudio/Melting%20Ice%20Tech...](https://www.eiffel.org/doc/eiffelstudio/Melting%20Ice%20Technology)

<comment><USER12> Interesting, thanks for all the info.

>Obviously, if you have a JIT-compiler, then something similar happens internally in that some methods will be compiled, while others will be left in IR form (and sometimes, methods will even revert to using their IR).

I was going to say (in my parent comment to which you replied) that maybe that Eiffel's Melting Ice tech is something like Java JIT tech, but wasn't sure if they were the same thing.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER15> Java`s generics have had issues due to use site variance, plus the language isn't expressive enough, leading its users into a corner where they start wishing for reified generics (although arguably it's a case of missing the forest from the trees).

But even so, even with all the shortcomings, once Java 5 was released people migrated to usage of generics, even if generics in Java are totally optional by design.

My guess to why that happens is that the extra type safety and expressivity is definitely worth it in a language and without generics that type system ends up staying in your way. I personally can tolerate many things, but not a language without generics.

You might as well use a dynamic language. Not Python of course, but something like Erlang would definitely fit the bill for Google's notion of "systems programming".

The Go designers are right to not want to introduce generics though, because if you don't plan for generics from the get go, you inevitably end up with a broken implementation due to backwards compatibility concerns, just like Java before it.

But just like Java before it, Go will have half-assed generics. It's inevitable.

Personally I'm sad because Google had an opportunity to introduce a better language, given their marketing muscle. New mainstream languages are in fact a rare event. They had an opportunity here to really improve the status quo. And we got Go, yay!

<comment><USER16> After some initial enthusiasm due to its gentle learning curve (actually, the Golang learning curve is nearly non-existent for an experienced programmer), I got sick of Go programming pretty quickly. Writing anything non-trivial will have you fighting against the limitations of the language (e.g., lack of generics which is particularly aggravating when trying to write a library).

I've mostly moved on to Clojure for programming backend services. I still use Go for the occasional small utility program due to its speed and ease of deployment, though.

<comment><USER17> I've programmed scores of libraries in Go and found it pleasant, in fact it's more pleasant writing a library in Go than in any other language I've used.

I've never once even considered the fact that I might need generics because I've never run into an unsolvable problem or an extremely inelegant engineering solution. I see a lot of people claiming that the lack of generics is a big problem but no one is actually showing a concrete case where generics are necessary.

C doesn't have generics but we never have this discussion when we talk about C.

<comment><USER18> If you have a []T and you want to apply a function to make it into []K, you need to explicitly make a loop in every case. You might not think that's that bad, but note that T and K could also be interfaces such that T is a superset of K. In that case Go, T can be used as K but []T cannot be used as []K (meaning you have to do an explicit loop for something that should be an implicit type conversion).

That is a trivial example where generics help. And yes, the endless boxing and runtime type assertion does make your code slower eventually (see the comment from the libcuckoo authors).

This is one of the reasons why the container and sort stdlb libraries are such a joke. They're not generic so they have to special case every different type. With generics, the entire sort package would just be a function that takes []Ordinal and sorts it. Container would similarly only require one implementation of each data structure (because they would be generic).

I find it hard to believe that you've programmed "scores of Go" and you've never hit any of these issues. I also have programmed plenty of Go, and these problems are just frustrating.

<comment> herewego- Been using it for years now in a setting with lots of data and never had any problems.

<comment><USER18> If that's true then you might not benefit from generics directly. But you will benefit from having an ecosystem of libraries that take advantage of generics. And even if you don't use other people's libraries, not everyone in the world happens to be as lucky as you and they do need generics to make their lives easier.

But again, I still severely doubt you've never run into a problem with Go's lack of generics. I ran into it after about 3 months of working with Go. Maybe you didn't find the work-arounds problematic, but that doesn't mean that the original limitation (no generics) doesn't cause problems. You just choose to accept the work-arounds as the "right way" of doing things.

If you haven't already, I would recommend looking at Rust or another modern language that has generics to see what sort of benefits you can get out of it that you can't directly get with Go. Personally the fact that if T implements an interface I won't let you use []T as []I is pretty silly IMO. Rust lets you do that with the From trait, and you can even do it even more generically with the From trait using a generic T.

</comment></comment></comment></comment></comment></comment><comment><USER19> I get that everyone would love to have a functional language that's eager by default with optional lazy constructs, great polymorphism, statically typed with inference, generics, great concurrency story, an efficient GC, that compiles quickly to self contained binaries with simple and effective tooling which takes only seconds to setup while giving you perfomance that equals java and can rival C, with a low memory footprint.

But, I don't know of one, and maybe that's because the Go team is right, some tradeoffs need to be made, and they did, and so Go is what it is. You can't add all the other great features you want and eat the Go cake too.

Disclaimer: I'm no language design expert. Just thinking this from the fact that I've yet to hear of such a language.

<comment><USER20> Nearly every item on your list is available with OCaml.

<comment><USER21> Apart from "great concurrency story", because of the well known problems with the GC. This might be a huge shortcoming for a number of people.

<comment><USER22> Multicore OCaml is in the works and is looking amazing, but is indeed a ways away. They're taking their time and doing it properly so things are fast and safe.

<comment><USER23> The thing is, until it's shipped, it's still vaporware for most people, especially if they want to use it in a commercial setting.

KDE 4 was released in 2008 and it was supposed to be ported to Windows. We're in 2017 and KDE 5 is still far from being considered stable on Windows. And I'm sure you can also think of many examples of programs promised and never delivered or underdelivered.

<comment><USER24> > was supposed to be ported to Windows

It _was_ ported to Windows. I cannot check the remarks about stability because I use Linux, but it sounds plausible. There's only a handful (maybe 2 or 3) developers working on Windows support in their spare-time AFAIK. I would guess that KDE had anticipated more Windows developers joining the project as it progressed towards maturity.

That's always the problem with open-source projects: It's very hard to do planning and forecasting with a bunch of volunteers. Even if they commit to a roadmap, there will always be someone who has to step down because of private issues (new job, new child, etc.). Go is in a much better position since Google has headcount assigned to it (again, AFAIK).

</comment></comment></comment></comment></comment></comment><comment><USER25>
        > To minimize disruption, each change will require
        > careful thought, planning, and tooling, which in
        > turn limits the number of changes we can make.
        > Maybe we can do two or three, certainly not more than five.

        > ... I'm focusing today on possible major changes,
        > such as additional support for error handling, or
        > introducing immutable or read-only values, or adding
        > some form of generics, or other important topics
        > not yet suggested. We can do only a few of those
        > major changes. We will have to choose carefully.

This makes very little sense to me. If you _finally_ have the opportunity to break backwards-compatibility, just do it. Especially if, as he mentions earlier, they want to build tools to ease the transition from 1 to 2.

        > Once all the backwards-compatible work is done,
        > say in Go 1.20, then we can make the backwards-
        > incompatible changes in Go 2.0. If there turn out
        > to be no backwards-incompatible changes, maybe we
        > just declare that Go 1.20 is Go 2.0. Either way,
        > at that point we will transition from working on
        > the Go 1.X release sequence to working on the
        > Go 2.X sequence, perhaps with an extended support
        > window for the final Go 1.X release.

If there aren't any backwards-incompatible changes, why call it Go 2? Why confuse anyone?

\---

Additionally, I'm of the opinion that more projects should adopt faster release cycles. The Linux kernel has a new release roughly every ~7-8 weeks. GitLab releases monthly. This allows a tight, quick iterate-and-feedback loop.

Set a timetable, and cut a release with whatever is ready at the time. If there are concerns of stability, you could do separate LTS releases. Two releases per year is far too short, I feel. Besides, isn't the whole idea of Go to _go fast_?

<comment> ainar-g (Copying my reply from Reddit.)

>If you finally have the opportunity to break backwards-compatibility, just do it.

I think Russ explained pretty clearly why this is a bad idea. Remember Python 3? Angular 2? We don't want that to happen with Go 2.0.

>Additionally, I'm of the opinion that more projects should adopt faster release cycles.

I am of the opposite opinion. In fact, I consider quick releases to be harmful. Releases should be planned and executed very carefully. There are production codebases with millions of lines of Go code. Updating them every month means that no progress will be made at all. The current pace is very pleasant, as most of the codebases I work with can benefit from a leap to newer version every six months.

<comment><USER26> > I think Russ explained pretty clearly why this is a bad idea. Remember > Python 3? Angular 2? We don't want that to happen with Go 2.0.

The problem with Python 2/3 is that Python 3 didn't add enough new features to make people want to move to 3.

The problem with Angular 2 is that it just didn't know what it wanted to be.

If Go2 doesn't break enough yet still break it will be no different from Python 2/3 fiasco.

Go has serious design issues, the go team ignoring them hindered Go adoption pace at first place.

<comment><USER27> Breaking things aren't want makes people want to move; it's a cost, not a benefit. Now, you need to offer benefits to get people to pay the cost for those benefits. The Python 2/3 problem was because there was too much breakage for the perceived benefit (especially early on) for many users, not because there was too little breakage.

<comment><USER28> Fwiw, I think breaking implies improving. Arguing that breaking doesn't inherently mean improving is.. well, duh. So in the case of many peoples comments here, "not breaking enough" means not improving enough. I know this is an obvious statement, but I feel like you're arguing a moot argument.. i.e., being a bit pedantic.

As an aside, since you're making the distinction, can you have meaningful benefit without breakage? Eg, you're specifically separating the two - so can you have significant improvements without breakage?

It would seem that pretty much any language change, from keyword changes to massive new features, breaks compatibility.

<comment><USER27> > As an aside, since you're making the distinction, can you have meaningful > benefit without breakage?

Sure, in two ways:

(1) Performance improvements with no semantic changes.

(2) New opt-in features that don't break existing code (such as where code using the new feature would have just been syntactically invalid in the old version, so the new feature won't conflict with any existing code.)

There be no reason for SemVer minor versions if you couldn't make meaningful improvements while maintaining full backward-compatibility.

<comment><USER28> Seems I can't edit my post, but:

I think I'm simply wrong here. I was envisioning "breaking" as being incompatible with Go1. If Go2 was a superset of Go1, it would be allow Go1 code to run flawlessly in Go2 and still allow any new keywords/features.

My assumptions were incorrect, and writing my reply to you sussed it in my head. Thank you for your reply, sorry for wasting your time :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER29> Here be Opinions:

I hate generics. also, I hate exceptions.

Too many people are wanting "magic" in their software. All some people want is to write the "Happy Path" through their code to get some Glory.

If it's your pet project to control your toilet with tweets then that's fine. But if it's for a program that will run 24/7 without human intervention then the code had better be plain, filled with the Unhappy Paths and boring.

Better one hour writing "if err" than two hours looking at logs at ohshit.30am.

<comment><USER30> there's nothing magic about generics. every time you make a channel or a map in go, you're using a generic function even if go people don't want you to call it that.

there's nothing magic about exceptions, too, it's that it's harder than necessary to use them correctly and that's why it's not as big of a deal to not have them in go - as evidenced by this thread.

<comment><USER31> I can sympathize with the dislike of exceptions. I very much prefer Rust's error handling model (return Result<T, Err>)

The fact that Go has had this much success without generics is mind-boggling to me.

<comment><USER11> Because Google.

Go is quite similar to Limbo, by some of the same authors, how much success did it have outside AT&T?

<comment><USER32> Limbo was not freely available.

<comment><USER11> Vita Nuova has it on an open source license since March 2000, which means it has been freely available for the last 17 years!

</comment></comment></comment></comment></comment></comment><comment><USER33> > For example, I've been examining generics recently, but I don't have in my > mind a clear picture of the detailed, concrete problems that Go users need > generics to solve.

This is sampling bias at work. The people who need generics have long since given up on Go and no longer even bother participating in Go-related discussions, because they've believe it will never happen. Meanwhile, if you're still using Go, you must have use cases where the lack of generics is not a problem and the existing language features are good enough. Sampling Go users to try and find compelling use cases for adding generics is not going to yield any useful data almost by definition.

<comment><USER34> We have a similar viewpoint w.r.t Higher Kinded Types in F#. Much of the motivation for HKTs comes from people wishing to use them how they've used them in other languages, but we've yet to see something like this in our language suggestions or design repositories:

"Here is a concrete example of a task I must accomplish with <design involving HKTs>, and the lack of HKTs prohibits me from doing this in such a way that I must completely re-think my approach.

(Concrete example explained)

I am of the opinion that lacking HKTs prevents me from having any kind of elegant and extensible design which will hold strong for years as the code around it changes."

Not that HKTs and TypeClasses aren't interesting, but without that kind of motivation, it's quite difficult to justify the _incredibly_ large cost of implementing them well. And that cost isn't just in the code and testing of the compiler. This cost also surfaces in tooling, documentation, and mindshare. It could also have an unknown effect on the ethos of F#, which for better or for worse, does bring with it a bit of a somewhat intangible quality that many people like.

Personally, I think Golang should implement generics. But I'm sympathetic with the views expressed by Russ Cox here. And I don't think it's just sampling bias.

<comment><USER35> This is apples and hand grenades, though. You're looking for a use case for HKTs. _I just want to make a tree without copy-pasting._

<comment><USER34> Fair point.

</comment></comment></comment></comment><comment> loup-vaillant > _For example, I 've been examining generics recently, but I don't have in my > mind a clear picture of the detailed, concrete problems that Go users need > generics to solve. […] If we had a large set of real-world use cases, we > could begin to answer a question like this by examining the significant > ones._

Not implementing generics, _then_ suggesting that it would be nice to have examples of generics being used in the wild… You had it coming, obviously.

Now what's the next step, refusing to implement generics because nobody uses it?

> _Every major potential change to Go should be motivated by one or more > experience reports documenting how people use Go today and why that 's not > working well enough._

My goodness, it looks like that _is_ the next step. Go users have put up with the absence of generics, so they're not likely to complain too loudly at this point (besides, I hear the empty interface escape hatch, while not very safe, does work). More exacting developers have probably dismissed Go from the outset, so the won't be able to provide those experience reports.

<comment> why-el > Not implementing generics, then suggesting that it would be nice to have > examples of generics being used in the wild… You had it coming, obviously.

I think you misunderstood. Clearly he meant to ask for examples from the real world that _lack_ generics, but shows how adding them would improve the system.

<comment> loup-vaillant > _Clearly he meant to ask for examples from the real world that lack > generics, but shows how adding them would improve the system._

I don't think many such examples will emerge.

First, you have the empty interface escape hatch. It's cumbersome, but it works. The _real_ reason why the repeated use of this escape hatch (instead of proper generics), is lack of compile time type safety, which is replaced by dynamic checks. This introduces elements of dynamic typing that generics could have avoided. This has a cost, which unfortunately is hard to asses.

Second, Go users tolerate the absence of generics for some reason. Maybe they don't really need it, but then they don't have a compelling use case. Maybe they _do_ need them, but didn't realise the limitations of the language would hurt them down the line, but are they competent enough to articulate why generics would be better? They made quite the blunder when they chose Go, after all.

That said, he also wrote this:

> _For example, I 've been examining generics recently, but I don't have in my > mind a clear picture of the detailed, concrete problems that Go users need > generics to solve._

Of course he doesn't: Go doesn't have generics. Go users found other ways, thus proving they didn't really need generics. And generics users don't use Go…

<comment> why-el Ok, thanks for the clarification. I wonder if there are situations in big teams in which one team that uses generics can take a look at a large Go codebase within the same company and see what they think? They must have received such feedback at Google, surely?

<comment> loup-vaillant Most probably. But they might ignore it anyway.

There's at least one high profile example: the standard library itself. It must have been obvious by the time they designed the standard library, _before_ the language was out. They _had_ to have feedback then, just look at the trail of rage against the absence of generics.

They ignored it then —I have no idea why. I'm not sure they'll listen now.

<comment><USER11> This was when I kind of lost interest, as it became clear generics would never happen.

In the early days I was enthusiastic enough that I tried to provide support for the _os.user_ implementation on Windows.

Nowadays I just advocate Go for C use cases, where the use of a tracing GC is not an hindrance as a way for us to enjoy safer computing.

Oh, and the code of the new _syncmap.Map_ is full of _unsafe.Pointer_ as workaround for lack of generics, while achieving good performance.

[https://github.com/golang/sync/blob/master/syncmap/map.go](https://github.com/golang/sync/blob/master/syncmap/map.go)

</comment></comment></comment></comment></comment></comment><comment><USER36> Go doesn't have const structs, maps or other objects:

[https://stackoverflow.com/questions/43368604/constant- struct...](https://stackoverflow.com/questions/43368604/constant-struct-in-go)

[https://stackoverflow.com/questions/18342195/how-to- declare-...](https://stackoverflow.com/questions/18342195/how-to-declare- constant-map-in-golang)

This is a remarkable oversight which makes it impossible to write purely- functional code with Go. We also see this same problem in most other imperative languages, with organizations going to great lengths to emulate const data:

[https://facebook.github.io/immutable- js/](https://facebook.github.io/immutable-js/)

Const-ness in the spirit of languages like Clojure would seem to be a relatively straightforward feature to add, so I don't really understand the philosophy of leaving it out. Hopefully someone here knows and can enlighten us!

<comment><USER37> I believe part of the reason was also some experience with C++, in which you sometimes have to "unconst" some fields of your const classes (the _mutable_ keyword). This is a really ugly and nonintuitive design, so I assume they'd rather take extra care to make sure they don't have to repeat it. Even if it means no const at all.

<comment><USER38> I don't think this kind of thing is all that non-intuitive if you reframe const as shared vs. unique references. Rust is a good example of this, although with Go you would want to sidestep all the Cell stuff since it's unnecessary.

</comment></comment></comment><comment><USER39> There are many comments griping about generics. There are many comments griping about the Go team daring to even _ask_ what problems the lack of generics cause.

But take a look at this article about the design goals of Go: [https://talks.golang.org/2012/splash.article](https://talks.golang.org/2012/splash.article) Look especially at section 4, "Pain Points". _That_ is what Go is trying to solve. So what the Go team is asking for, I suspect, is concrete ways that the lack of generics hinders Go from solving _those_ problems.

You say those aren't _your_ problems? That's fine. You're free to use Go for your problems, but you aren't their target audience. Feel free to use another language that is more to your liking.

Note well: I'm not on the Go team, and I don't speak for them. This is my impression of what's going on - that there's a disconnect in what they're asking for and what the comments here are supplying.

(And by the way, for those here who say - or imply - that the Go team is ignorant of other languages and techniques, note in section 7 the casual way they say "oh, yeah, this technique has been used since the 1970s, Modula 2 and Ada used it, so don't think we're so brilliant to have come up with this one". These people know their stuff, they know their history, they know more languages than you think they do. They probably know more languages than you do - even pjmlp. Stop assuming they're ignorant of how generics are done in other languages. Seriously. Just stop it.)

</comment><comment><USER40> As much as I want them to fix the big things like lack of generics, I hope they fix some of the little things that the compiler doesn't catch but could/should. One that comes to mind is how easy it is to accidentally write:

      for foo := range(bar)

Instead of:

      for _, foo := range(bar)

When you just want to iterate over the contents of a slice and don't care about the indices. Failing to unpack both the index and the value should be a compile error.

<comment><USER40> Here's a toy version of a real bug that I wasted a bit of time debugging which was due to this behavior: [https://play.golang.org/p/6pBUPBTTvj](https://play.golang.org/p/6pBUPBTTvj)

<comment><USER37> Would be cool if you could consider doing a writeup about it, and linking it on the mentioned wiki page.

</comment></comment></comment><comment><USER41> The beauty of Go is that you get developer productivity pretty close to Ruby/Python levels with performance that is similar to Java/C++

Improvements to package management is probably the highest item on my wishlist for Go 2.

<comment><USER42> There is so much boilerplate you have to write that productivity drops considerably, both because you have to re-re-re-re-implement things (or use code generation) and because you need to scan lot's of redundant code before making sense of it.

The alternative is often to just use interface{}, which hurts performance and type safety.

<comment><USER41> This has so far been a theoretical concern for me only. If you want to write reusable data structures you obviously need generics. But none of the code I've written involves reusable data structures, so I can't say I really miss generics.

</comment></comment></comment><comment><USER43> Generics have never stopped me from building in Go... But without them I often do my prototyping in python, javascript, or php.

Working with batch processing I'm often changing my maps to lists or hashes multiple times during discovery. Go makes me rewrite all my code each time I change the variable type.

<comment><USER28> It's weird, I see these complaints so often and I just.. don't.. get it. I'm not sure what I do differently, but I'm just so used to Go's method of non- generic that I don't run into frustration at all.

The only time I even _notice_ it is if I have to write a method like `AcceptInt(), AcceptString(), AcceptBool()` and etc.

I enjoy generics in Rust, I'm just not sure what I'm doing differently in Go that causes me to not miss them.

<comment><USER43> It's possible your data (and types) are short lived.

When I'm doing text processing for example, I pass the same strings/dicts/hashes through dozens of functions for cleaning, sorting, organising, benchmarking, comparing, etc..

It's not just in->save->out CRUD work.

<comment><USER44> I'm actually coding a LITTLE project with 7 tables and thinking if Golang was the right choice.. but I picked Golang because there is some async realtime component. My pace of dev is very slow.

<comment><USER28> What aspect is causing it to be slow for you? Note that there are definitely some areas of Go I find terrible, and SQL is one of them. Check out the SQLx library, it's far less painful than the stdlib SQL is.

<comment><USER44> Error handling, avoiding nulls, writing queries...

</comment></comment></comment></comment></comment></comment><comment><USER45> How about fixing all the GOPATH crap?

<comment><USER41> We've worked around it like this and it works pretty well: [https://github.com/getstream/vg](https://github.com/getstream/vg)

<comment><USER28> I'm so undecided on using GOPATH like that. Though, I may just do it.

Personally I hate Go's imports, and I think the GOPATH is a terrible, terrible idea. Yet, I quite like the vendor directory, and I expect the new `dep` tool to be the final touch.

Now my only problem is that I often forget which projects of mine are in my GOPATH, so I'm afraid to delete them -_-

<comment><USER46> vendor has big problems too. There are some proposals on how to start fixing them, after the dep tool lands (originally slated for 1.7 or 1.8 but still in Alpha?)

To support the uncommon edge case, src/foo/vendor/dep is completely different to src/bar/vendor/dep.

Your src/foo code can't pass src/foo/vendor/dep types to src/bar code expecting src/bar/vendor/dep types. Even if the deps are identical checkouts. Code can become uncompilable by the act of vendoring a dependency.

You have two completely different sets of globals. Code can stop working by the act of vendoring a dependency.

The initialization runs twice. So well known packages like glog will panic if two packages has it vendored because they both try to update settings in the standard library.

GOPATH would be preferable, but none of the tools for managing packages at that level support pinning. So you dare to be developing more than one package at a time, you end up with a single monolithic repository representing your GOPATH and pin your dependencies in src/, or end up with symlink hacks and other unmaintainable tricks.

<comment><USER28> Oh I definitely agree there, I didn't mean to say it was flawless. When I wrote that, I was more thinking of the UX of vendoring.

Eg, `import "foo/lib"` pulls seamlessly from vendor, which is a really nice UX if people want to vendor.

With that said, I still think a proper package manager Ala Rust Cargo.toml will be better. Here's hoping Go's `dep` solves this crap basket :)

</comment></comment></comment></comment></comment><comment><USER47> As a ruby and go dev, I'm a bit sad to see backward-compatibility going. Thinking I could write code with minimum dependencies and that would just work as is years later was really refreshing compared to the high level of maintenance needed in a ruby app.

But well, I trust the core team to make the best choices.

</comment><comment><USER48> I like Go concept: very simple and minimalistic language, yet usable enough for many projects, even at cost of some repetition. Generics are not a concern for me. But error handling is the thing I don't like at all. I think that exceptions are best construct for error handling: they are not invasive and if you didn't handle error, it won't die silently, you have to be explicit about that. In my programs there's very little error handling, usually some generic handling at layer boundaries (unhandled exception leads to transaction rollback; unhandled exception returns as HTTP 500, etc) and very few cases when I want to handle it differently. And this produces correct and reliable program with very little effort. Now with Go I must handle every error. If I'm lazy, I'm handling it with `if err != nil { return err; }`, but this style doesn't preserve stack trace and it might be hard to understand what's going on. If I want to wrap original error, standard library don't even have this pattern, I have to roll my own wrapper or use 3-rd library for such a core concept.

What I'd like is some kind of automatic error propagation, so any unhandled error will return from function wrapped with some special class with enough information to find out what happened.

</comment><comment><USER49> > For example, I've been examining generics recently, but I don't have in my > mind a clear picture of the detailed, concrete problems that Go users need > generics to solve.

Collections?

</comment><comment><USER50> But I always heard never to use Go 2! :P

</comment><comment><USER51> "We estimate that there are at least half a million Go developers worldwide, which means there are millions of Go source files and at least a billion of lines of Go code"

<comment><USER52> Perhaps fewer than a billion lines of Go code if there were generics.

</comment></comment><comment><USER53> I must say that whenever there is a discussion about the merits of the Go programming language, it really feels hostile in the discussion thread. It seems that people are seriously angry that others even consider using the language. It is sort of painful reading through the responses which implicitly declare that anybody who enjoys programming with Go is clueless.

It also really makes me wonder if I am living in some sort of alternate reality. I am a professional programmer working at a large company and I am pretty sure that 95% of my colleagues (myself included, as difficult as it is for me to admit) have no idea what a reified generic is. I have run into some problems where being able to define custom generic containers would be nice, but I don't feel like that has seriously hindered my ability to deliver safe, functional, and maintainable software.

What I appreciate most about Go is that I am sure that I can look at 99% of the Go code written in the world and I can understand it immediately. When maintaining large code bases with many developers of differing skill levels, this advantage can't be understated. That is the reason there are so many successful new programs popping up in Go with large open-source communities. It is because Go is accessible and friendly to people of varying skill levels, unlike most of the opinions expressed in this thread.

<comment><USER54> I think the negative attitude comes more from frustration than from anything else.

> I am a professional programmer working at a large company and I am pretty > sure that 95% of my colleagues (myself included, as difficult as it is for > me to admit) have no idea what a reified generic is.

I didn't know what it was called either until I saw them mentioned here, but now that I know about it, I get why it would be useful. Before that, I kind of assumed all languages with generics would also allow you to access the type information at runtime.

> I have run into some problems where being able to define custom generic > containers would be nice, but I don't feel like that has seriously hindered > my ability to deliver safe, functional, and maintainable software.

I don't mean any disrespect, but this is a very good example of the Blub Paradox: [http://wiki.c2.com/?BlubParadox](http://wiki.c2.com/?BlubParadox)

<comment><USER53> > I don't mean any disrespect, but this is a very good example of the Blub > Paradox: [http://wiki.c2.com/?BlubParadox](http://wiki.c2.com/?BlubParadox)

Interesting read, and I admit there is some of that. But my point isn't to say that those features aren't useful or powerful, but rather that with the constraint of working with a large group of programmers of varying skills, simplicity has more value than power (as long as we can deliver software that meets our requirements). It is similar to point 4 in the "Problems with the Blub Paradox" section.

<comment><USER55> One of the problems is who decides what is too simple. E.g. why are for loops, function calls and lamdbas considered simple enough to be in go, but generics aren't? When I TA'ed CS introductory courses, student usually had a lot less trouble with understanding generics than lambdas.

Not including feature X in programming language Y will also ensure that no-one who primarily uses Y will ever come to understand or appreciate X.

The blub paradox basically represents the opinion that it is always better for languages to include more powerful features.

I think that goes too far, working primarily in Scala I am seeing firsthand how much diminishing returns extra complexity in the language can have, but I'm certainly of the opinion that Go leans too heavily toward handicapping the language in pursuit of simplicity.

<comment><USER56> The complexity from generics isn't from the conceptual standpoint, it's from the resulting code standpoint... for the same reason that the ternary ?: operator is "too complex": it's really easy to say that `foo := cond ? 1 : 0` is better than the if/else alternative, but `foo := (a > (b.Active() ? Compare(b, a) : useDefault() ? DEFAULT : panic()) ? "worked" : "failed"` is a mess waiting to happen.

Same with generics. It's easy to point to simple examples. It hard to ensure that terrible metaprogramming monstrosities don't arise.

It's possible to write bad code with go as it is, but it's actually difficult to make such bad code inscrutable.

<comment><USER55> Maybe, but the solution to this is not to say: Only the compiler implementers are smart enough people to not create messy code with generics, only a few built-in data types are allowed to be generic and the rest will have to copy- paste implementations or use lose type information through `interface {}`

</comment></comment></comment></comment></comment></comment><comment><USER57> Sorry, but if "a major cloud platform suffers a production outage at midnight" is the bar for effecting change in Go, then I want no part of it.

</comment><comment><USER58> "Go 2 considered harmful" \- Edsger Dijkstra, 1968

<comment><USER59> Underrated comment. :)

Haha, thanks.

If there's a Go2 there will be a _lot_ of these headlines for blog posts/rants.

It doesn't matter if it is true or not, but that phrase is wonderful.

</comment></comment><comment><USER60> This is a frustratingly common way to design mainstream PLs: "show me the use case". I've seen it firsthand for a number of big PL projects. People are trapped in their little bubble. My approach is to be wildly polyglot, actively searching for good ideas in other languages. Also, I try to write complex code outside the intended domain space and understand why it's harder than it should be.

For example, in Go it's difficult to implement state machines in a clean way. Another is handling events and timeouts is easier with Reactive programming. Distributed programming is easier with Erlang or Akka. Don't wait for problem reports in the Go community. Look at the problems in other PLs and proactively improve Go.

</comment><comment><USER61> I would love to see uniform-function-call-syntax.

Turning: func (f Foo) name() string

Into: func name(f Foo) string

Callable like this: f.name() or name(f)

Extending foreign structs from another package should be possible too, just without access to private fields.

Other than that, if-as-expression would be nice to have, too.

<comment><USER62> One of my favorite features of D.

</comment></comment><comment><USER63> The leap second problem reminds me of this post[0].

[0] [https://news.ycombinator.com/item?id=14121780](https://news.ycombinator.com/item?id=14121780)

</comment><comment><USER64> Disclaimer: I mean this with love

This post really frustrates me, because the lengthy discussion about identifying problems and implementing solutions is pure BS. Go read the years worth of tickets asking for monotonic time, and see how notable names in the core team responded. Pick any particular issue people commonly have with golang, and you'll likely find a ticket with the same pattern: overt dismissal, with a heavy moralizing tone that you should feel bad for even asking about the issue. It's infuriating that the same people making those comments are now taking credit for the solution, when they had to be dragged into even admitting the issue was legitimate.

<comment><USER65> > asking for monotonic time

His anecdote about Google/Amazon using leap smears to solve the problem is telling. I suspect that they were unable to see outside their own bubble to think about how others may be impacted.

> We did what we always do when there's a problem without a clear solution: we > waited

The original problem described the issue and the potential consequences very well and the problem didn't change between the initial report and when cloudflare hit it. It was only until a Serious Industrial User (to borrow a term from Xavier @ OCaml) got bit in a very public way that they actually began thinking about what a clear solution would look like.

<comment><USER13> > We did what we always do when there's a problem without a clear solution: we > waited

And this is exactly why the Go designers don't understand language design, and how this ignorance shines through every single place in their language.

Language design is about compromises. There is never a perfect solution, only one that satisfies certain parameters while compromising others. You, the designer, are here to make the tough choice and steer the language in a direction that satisfies your users.

Besides, characterizing their position as "We waited" is very disingenuous. First of all, this is more stalling than waiting, and second, waiting implies they at least acknowledge the problems, which the Go team famously never does. Read how various issues are answered and then summarily closed with smug and condescending tones.

<comment><USER66> You are being deliberately negative here. Choosing to forego generics in favor of simplicity (and its impact along several axes) is a postcard example of a compromise. It is a tough choice that many people will be unhappy with, but there are also many Go programmers that are extremely satisfied with that direction.

As for acknowledging, well, they have always been very clear about their position. It makes no sense to spend a decade answering the same question over and over with a long and elaborate response which the person asking has already seen and dismissed. I can understand them becoming condescending after a decade of facing people who act with a deliberately obtuse and holier-than- thou attitude.

It's not like they have been lazy - every release of Go has had a large amount of improvements that matter. Working on generics would have meant sacrificing a (probably large) amount of them.

(for the record, I dearly miss generics too!)

</comment></comment></comment></comment><comment><USER67> Regarding the lacking of generics problem, is there a way to get around it, there are always plenty of tools doing that, if the IDE can patch the syntax and support certain kind of prigma to generate the template code, then the problem is almost solved, not sure if it'll cover all cases like Java does though.

</comment><comment><USER68> This was announced at GopherCon today. FYI, if folks are interested in following along other conference proceedings, there is no livestream, but there is an official liveblog: [https://sourcegraph.com/gophercon](https://sourcegraph.com/gophercon)

<comment><USER69> It's on twitch I believe

</comment></comment><comment><USER70> "Play it cool" [https://youtu.be/BHIo6qwJarI](https://youtu.be/BHIo6qwJarI) Go! by Public Service Broadcasting.

(Sorry just discovered this song a few days ago)

</comment><comment><USER71> about generics : i've never had a deep look at it but i've always wondered if most of the problem couldn't be solved by having base types ( int, string, date, float, ..) implements fundamental interfaces (sortable, hashable, etc). i suppose that if the solution were that simple people would've already thought about it.

in particular, i think it could help with the method dispatch, but probably not with the memory allocation ( although go already uses interfaces pretty extensively).

</comment><comment><USER72> Most of the discussion here seems to be around generics, and it sounds like they still don't see the benefit of generics.

I like Go, but the maintainers have a maddeningly stubborn attitude towards generics and package managers and won't ease up even with many voices asking for these features.

</comment><comment><USER73> > _We did what we always do when there 's a problem without a clear solution: > we waited. Waiting gives us more time to add experience and understanding of > the problem and also more time to find a good solution. In this case, > waiting added to our understanding of the significance of the problem, in > the form of a thankfully minor outage at Cloudflare. Their Go code timed DNS > requests during the end-of-2016 leap second as taking around negative 990 > milliseconds, which caused simultaneous panics across their servers, > breaking 0.2% of DNS queries at peak._

This is absurd. Waiting to fix a known language design issue until a production outage of a major customer is a _failure_ of process, not an _achievement_. The fact that the post presents this as a positive aspect of Go's development process is beyond comprehension to me.

<comment><USER74> They did the same thing with adding a "round" function. As a result, there are myriad implementations of round() in the wild in Go, and almost every single one is broken in several ways.

</comment></comment><comment><USER75> I didn't expect to get namechecked in that.

Shows the value of constantly being transparent and supporting open source projects.

<comment><USER76> It absolutely does, and thank you. (I thought you might be here so I could thank you in person.)

<comment><USER75> You're most welcome. We love Go and use it extensively.

</comment></comment></comment><comment><USER77> Forget generics. What I missed most are algebraic data structures.

</comment><comment><USER78> I was newly married and my husband and I intended on getting a house but our credit scores were low and I also had a few bank debts. I spoke to a friend about it who happens to be a tech guy and he introduced me to a hacker (blackbutcher). I contacted blackbutcher and he helped my husband and I boost our credit scores and he also helped me clear my bank debts. This hacker is a genius and comes highly recommended by a lot of people. He's affordable and genuine unlike a lot of fakes I saw on the internet. Contact him on his mail firstname.lastname@example.com

</comment><comment><USER79> My main wish:

Please don't _refuse_ to compile just because there are unused imports.

Please do warn, and loudly say it's NON-CONFORMANT or whatever will embarass me enough from sharing my piece of Go code with someone else.. but.. can I please just run my code, in private, when experimenting?

<comment><USER80> I like that Go don't allows unused import and variables.

If you're working with inexperienced programmers (and some experienced but bad programmers), you should be certain that they WILL let that code garbage there if the compiler allow it.

<comment><USER81> That's the kind of thing that should be detected by a linter and stop code submission though, not necessarily stop it from even compiling. What if you just want to comment something out quickly? I don't program in Go, but it seems like it would be annoying to have to repeatedly add and remove the main dependencies while I was iterating on something.

<comment><USER82> This problem is fixed by using the "goimports" tool that will automatically add and remove imports as they're used on save. Pretty much every editor supports it. With goimports and gofmt that formats code, I no longer have to think about formatting and imports at all until I git diff, the whole problem vanishes.

</comment></comment></comment></comment><comment><USER83> Will there be _gotos_ in Go 2? Asking for a friend.

<comment><USER76> Have you actually used Go 1's goto? There's not a lot it can actually do. It's there for generated code state machines, and that's almost all it's good for. But yes, goto will stay in the absence of a very compelling benefit to take it out, a benefit that would far outweigh the cost of breaking all the programs that use it.

</comment></comment><comment><USER84> I suspect the authors of Golang got drunk, and challenged each other to see how many times they could get people to type 'if err != nil' in the next decade.

<comment><USER69> Actually what really happened was instead of trying to catch em 'all with pokemon try/catching, they decided to treat errors like it was part of the actual code. Errors are values. You can actually code with them. Amazing, right?

</comment></comment><comment><USER85> I know there is a lot of discussion about generics, but I am not sure if that is missing the point. I mean 'generics' sounds like a complex concept from the java development and I am uncertain if that's really what we need in go.

From my experience I think we should talk about container formats because they make 80% of what we would like to have generics for. Actually, go feels as if it has only two container data structures: Slices and Maps. And both feel as if they are pretty hard coded into the language.

Yes, I am sure there are more container formats and it is possible to build your own, but I think it is not easy enough.

<comment><USER86> > I mean 'generics' sounds like a complex concept from the java development > and I am uncertain if that's really what we need in go.

That's inane and insane. Java generics do almost nothing, they don't even exist at runtime in a language with (much like Go) heavy runtime semantics. Here's what Java's generics do: get typechecked, and insert implicit casts.

> Yes, I am sure there are more container formats

First, you forgot Channels and arrays (which are distinct from slices) as well as "range" to iterate over them generically, second there are more containers than that _in Go 's own standard library_: [https://golang.org/pkg/container/](https://golang.org/pkg/container/), [https://github.com/golang/go/issues/18177](https://github.com/golang/go/issues/18177)

> it is possible to build your own, but I think it is not easy enough

What does that mean? It is not easy enough what? To write containers? To have generics?

<comment><USER85> So if generics in java don't do anything beyond 'get typechecked, and insert implicit casts.' the empty interface is enough for all use-cases in go?

<comment><USER87> Yes, in a sense that structured loops don't do anything beyond inserting implicit if/goto, and so the latter is enough for all use-cases.</comment></comment></comment></comment>


Nodemailer v1.0 - <AUTHOR> http://www.andrisreinman.com/nodemailer-v1-0/

<comment><USER2> It's interesting that you would mention that Nodemailer is not appropriate for mass mail due to the callback structure - does it make sense to write off mass mail because of this? I'm not convinced it does. What if you could specify a write stream that the result of each send could be streamed to? Or maybe do callbacks with batch results? Throwing away mass mailing in the biggest mail library available for Node seems sort of silly to me.

Of course, a big thanks is in order - Nodemailer has been a great system to work with and I'm glad it's still getting the love it deserves. Keep up the great work Andris.

<comment><AUTHOR> What I actually mean is that you probably should not try to build your own SendGrid on top of Nodemailer. You might try and it might even work but you probably should not.

If you want to send 10 000 e-mails at once, then it works just fine. If you want to send 10 000 000 messages from a for loop then it probably breaks.

<comment><USER2> Sounds reasonable! :)

</comment></comment></comment><comment><USER3> Great work! Also great to see more of the node community adopting semantic versioning

</comment><comment><USER4> I've only had the chance to use this once on a client project, but it solved my problem in ~5 minutes (email attachments). Great package :)

</comment><comment><USER5> I've been using it in a small side project of mine. Love it, thanks!

</comment><comment><USER6> Great, thanks! Nodemailer has been very useful.

</comment><comment><USER7> Nice indeed. Good work Andris!

</comment><comment><USER8> Very nice. Thanks ya'll!</comment>


Academics Isn't Setup for Open Source - <AUTHOR> http://www.ethanmick.com/academics-isnt-setup-for-open-source/ <comment><USER2> As a point of nomenclature, that should be "Education", not "Academics".

For example, the Institute for Advanced Study is an academic institution even though it has no students.

While the circumstances described are not relevant for, say, graduate student research (or for that matter undergrads doing independent study).

<comment><AUTHOR> Yeah, I agree with that. I changed it to College, since that's more what I was talking about. Thanks!</comment></comment>


Ask HN: Consulting the right way? - <AUTHOR>

I partnered up with a friend to consult in web/mobile/creative. We have a bunch of clients and it's very unorganized and makes me nervous. I do this on the side along my current full-time job which I'm looking to quit once this takes off.
1) How do I prepare a proper contract for a simple freelance job ?
2) How do I handle payment and invoicing? I don't want to do it for just cash. We agreed to invoice it under my name for now, but how do I deal with the tax implications of this extra money (I'm in Ontario, Canada).
3) I remember reading about this 17 year old kids consultancy on HN once but I can't for the life of me remember the link.
Are there any good blogs HN recommends to follow in relation to consulting/freelancing? I only know of the close.io blogs which are great and patio11's work.
Thanks. <comment><USER2> 1\. There are a number of predefined contracts which can be used on the internet. Obviously the best thing to do is get a lawyer to look over what ever you choose.</comment>